2024-02-02T13:12:00.136 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-02T13:12:00.136 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --name vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405 --verbose --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/orig.config.yaml 2024-02-02T13:12:00.184 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-02T13:12:00.274 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405 branch: wip-vshankar-testing1-reef-2024-02-02-1152 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: vshankar@redhat.com first_in_suite: false job_id: '7543405' kernel: client: branch: testing flavor: default kdb: true sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing1-reef-2024-02-02-1152 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 0fb327883c124fd83155c94db0804f823b805a12 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0fb327883c124fd83155c94db0804f823b805a12 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-reef-2024-02-02-1152 sha1: 0fb327883c124fd83155c94db0804f823b805a12 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1628 sha1: 0fb327883c124fd83155c94db0804f823b805a12 sleep_before_teardown: 0 subset: 50/1024 suite: fs suite_branch: wip-vshankar-testing1-reef-2024-02-02-1152 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 0fb327883c124fd83155c94db0804f823b805a12 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi091.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCdICmb/r8PUIMuJy7WjtsllZdEjiLuEeZ/8qghtbmxYAKoEFdGpcR5nZbsIBxVY/ApKVDN249OmUhF5FXx8LA= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxTOeL51Zfohp5F+kujWCvAxf0T/oaoI5vrfYOa0yAXSgNzJzhyZ+0KzGVKxl60ZjkC0P6sfN6ruIBB0pVukGw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-02_11:17:23 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-02T13:12:00.274 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa; will attempt to use it 2024-02-02T13:12:00.275 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks 2024-02-02T13:12:00.275 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-02T13:12:00.277 INFO:teuthology.task.internal:Checking packages... 2024-02-02T13:12:00.305 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0fb327883c124fd83155c94db0804f823b805a12' 2024-02-02T13:12:00.305 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-02T13:12:00.305 INFO:teuthology.packaging:ref: None 2024-02-02T13:12:00.305 INFO:teuthology.packaging:tag: None 2024-02-02T13:12:00.305 INFO:teuthology.packaging:branch: wip-vshankar-testing1-reef-2024-02-02-1152 2024-02-02T13:12:00.305 INFO:teuthology.packaging:sha1: 0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T13:12:00.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing1-reef-2024-02-02-1152 2024-02-02T13:12:00.529 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-325.g0fb32788 2024-02-02T13:12:00.531 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-02T13:12:00.538 INFO:teuthology.task.internal:no buildpackages task found 2024-02-02T13:12:00.538 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-02T13:12:00.545 INFO:teuthology.task.internal:Saving configuration 2024-02-02T13:12:00.563 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-02T13:12:00.569 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T13:12:00.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 13:06:27.517825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCdICmb/r8PUIMuJy7WjtsllZdEjiLuEeZ/8qghtbmxYAKoEFdGpcR5nZbsIBxVY/ApKVDN249OmUhF5FXx8LA='} 2024-02-02T13:12:00.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 13:06:27.516014', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxTOeL51Zfohp5F+kujWCvAxf0T/oaoI5vrfYOa0yAXSgNzJzhyZ+0KzGVKxl60ZjkC0P6sfN6ruIBB0pVukGw='} 2024-02-02T13:12:00.621 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-02T13:12:00.628 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-02-02T13:12:00.628 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-02-02T13:12:00.628 INFO:teuthology.run_tasks:Running task console_log... 2024-02-02T13:12:00.696 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb1624b90d0>, signals=[15]) 2024-02-02T13:12:00.696 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-02T13:12:00.702 INFO:teuthology.task.internal:Opening connections... 2024-02-02T13:12:00.702 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:12:00.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:12:00.797 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:12:00.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:12:00.884 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-02T13:12:00.890 DEBUG:teuthology.orchestra.run.smithi091:> uname -m 2024-02-02T13:12:00.907 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2024-02-02T13:12:00.908 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Stream" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="9" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="9" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el9" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:LOGO="fedora-logo-icon" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://centos.org/" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-02T13:12:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-02T13:12:00.965 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-02T13:12:00.965 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2024-02-02T13:12:00.984 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-02-02T13:12:01.002 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-02-02T13:12:01.002 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-02-02T13:12:01.059 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-02T13:12:01.060 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-02-02T13:12:01.060 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-02T13:12:01.060 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-02T13:12:01.060 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-02T13:12:01.060 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-02-02T13:12:01.076 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-02T13:12:01.085 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-02T13:12:01.091 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-02T13:12:01.091 DEBUG:teuthology.orchestra.run.smithi091:> test '!' -e /home/ubuntu/cephtest 2024-02-02T13:12:01.094 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-02-02T13:12:01.115 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-02T13:12:01.121 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-02T13:12:01.121 DEBUG:teuthology.orchestra.run.smithi091:> test -z $(ls -A /var/lib/ceph) 2024-02-02T13:12:01.150 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-02-02T13:12:01.424 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-02T13:12:01.477 INFO:teuthology.run_tasks:Running task kernel... 2024-02-02T13:12:01.494 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'} 2024-02-02T13:12:01.494 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-02-02T13:12:01.494 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-02-02T13:12:01.495 DEBUG:teuthology.orchestra.run.smithi091:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-02T13:12:01.495 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-02T13:12:01.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:12:01.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-02T13:12:01.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:12:01.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-02T13:12:01.775 DEBUG:teuthology.task.kernel:sha1 for client.1 is cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:12:01.775 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "cd42140e628558a450ba1d39a9d0835a2fa62cda"... 2024-02-02T13:12:01.776 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-02-02T13:12:01.794 DEBUG:teuthology.task.kernel:sha1 for client.0 is cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:12:01.794 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "cd42140e628558a450ba1d39a9d0835a2fa62cda"... 2024-02-02T13:12:01.794 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2024-02-02T13:12:01.799 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-361.el9.x86_64 2024-02-02T13:12:01.800 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:12:01.800 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-02T13:12:01.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum upgrade -y linux-firmware 2024-02-02T13:12:01.814 INFO:teuthology.orchestra.run.smithi091.stdout:5.14.0-361.el9.x86_64 2024-02-02T13:12:01.815 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:12:01.815 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-02T13:12:01.815 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum upgrade -y linux-firmware 2024-02-02T13:12:04.099 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - AppStream 10 MB/s | 6.6 MB 00:00 2024-02-02T13:12:04.327 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 8.4 MB/s | 6.6 MB 00:00 2024-02-02T13:12:05.725 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-02-02T13:12:05.936 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-02-02T13:12:06.729 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-02-02T13:12:07.240 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-02-02T13:12:09.191 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-02T13:12:09.807 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 19 MB 00:02 2024-02-02T13:12:13.813 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.9 MB 00:00 2024-02-02T13:12:13.867 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 5.9 MB 00:01 2024-02-02T13:12:15.242 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2024-02-02T13:12:15.556 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-02-02T13:12:16.495 INFO:teuthology.orchestra.run.smithi091.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2024-02-02T13:12:21.476 INFO:teuthology.orchestra.run.smithi091.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-02T13:12:23.785 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrading: 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrade 1 Package 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:12:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 375 M 2024-02-02T13:12:23.797 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2024-02-02T13:12:33.251 INFO:teuthology.orchestra.run.smithi091.stdout:linux-firmware-20240109-142.el9.noarch.rpm 40 MB/s | 375 MB 00:09 2024-02-02T13:12:33.251 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2024-02-02T13:12:33.252 INFO:teuthology.orchestra.run.smithi091.stdout:Total 40 MB/s | 375 MB 00:09 2024-02-02T13:12:33.252 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2024-02-02T13:12:33.351 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2024-02-02T13:12:33.351 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2024-02-02T13:12:35.785 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2024-02-02T13:12:35.786 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2024-02-02T13:12:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 907 kB/s | 20 MB 00:22 2024-02-02T13:12:41.922 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2024-02-02T13:12:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-02-02T13:12:42.224 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:12:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-02-02T13:12:43.567 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:12:43.567 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-02-02T13:12:43.949 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:12:43.949 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:12:43.950 INFO:teuthology.orchestra.run.smithi091.stdout:Upgraded: 2024-02-02T13:12:43.950 INFO:teuthology.orchestra.run.smithi091.stdout: linux-firmware-20240109-142.el9.noarch 2024-02-02T13:12:43.950 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:12:43.950 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2024-02-02T13:12:44.228 INFO:teuthology.task.kernel:Downloading kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.0... 2024-02-02T13:12:44.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:12:44.462 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ 2024-02-02T13:12:44.463 DEBUG:teuthology.orchestra.run.smithi091:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_gcd42140e6285-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-02T13:12:44.465 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi091.front.sepia.ceph.com to complete... 2024-02-02T13:12:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-02-02T13:12:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-02-02T13:12:45.878 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-02-02T13:12:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 1 Package 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 375 M 2024-02-02T13:12:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-02-02T13:12:53.585 INFO:teuthology.orchestra.run.smithi113.stdout:linux-firmware-20240109-142.el9.noarch.rpm 49 MB/s | 375 MB 00:07 2024-02-02T13:12:53.587 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-02-02T13:12:53.587 INFO:teuthology.orchestra.run.smithi113.stdout:Total 49 MB/s | 375 MB 00:07 2024-02-02T13:12:53.587 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-02-02T13:12:53.682 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-02-02T13:12:53.682 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-02-02T13:12:55.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02 13:12:55 URL:https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/kernel-6.7.0_gcd42140e6285-1.x86_64.rpm [330265785/330265785] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-02T13:12:55.818 INFO:teuthology.orchestra.run.smithi091.stderr:FINISHED --2024-02-02 13:12:55-- 2024-02-02T13:12:55.818 INFO:teuthology.orchestra.run.smithi091.stderr:Total wall clock time: 11s 2024-02-02T13:12:55.818 INFO:teuthology.orchestra.run.smithi091.stderr:Downloaded: 1 files, 315M in 11s (28.5 MB/s) 2024-02-02T13:12:55.821 INFO:teuthology.task.kernel:Installing kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.0... 2024-02-02T13:12:55.821 DEBUG:teuthology.orchestra.run.smithi091:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-02T13:12:56.166 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-02-02T13:12:56.166 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-02-02T13:12:57.724 INFO:teuthology.orchestra.run.smithi091.stdout:Verifying... ######################################## 2024-02-02T13:12:57.850 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing... ######################################## 2024-02-02T13:12:57.934 INFO:teuthology.orchestra.run.smithi091.stdout:Updating / installing... 2024-02-02T13:13:02.567 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-02-02T13:13:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-02-02T13:13:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:13:04.062 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:13:04.073 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-02-02T13:13:04.449 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-02T13:13:04.450 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:13:04.450 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-02-02T13:13:04.450 INFO:teuthology.orchestra.run.smithi113.stdout: linux-firmware-20240109-142.el9.noarch 2024-02-02T13:13:04.450 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:13:04.450 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-02-02T13:13:04.732 INFO:teuthology.task.kernel:Downloading kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.1... 2024-02-02T13:13:04.733 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T13:13:04.960 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ 2024-02-02T13:13:04.960 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_gcd42140e6285-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-02T13:13:04.963 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi113.front.sepia.ceph.com to complete... 2024-02-02T13:13:05.307 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-6.7.0_gcd42140e6285-1 ######################################## 2024-02-02T13:13:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02 13:13:16 URL:https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/kernel-6.7.0_gcd42140e6285-1.x86_64.rpm [330265785/330265785] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-02T13:13:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:FINISHED --2024-02-02 13:13:16-- 2024-02-02T13:13:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:Total wall clock time: 11s 2024-02-02T13:13:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:Downloaded: 1 files, 315M in 11s (28.9 MB/s) 2024-02-02T13:13:16.183 INFO:teuthology.task.kernel:Installing kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.1... 2024-02-02T13:13:16.184 DEBUG:teuthology.orchestra.run.smithi113:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-02T13:13:18.081 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying... ######################################## 2024-02-02T13:13:18.179 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing... ######################################## 2024-02-02T13:13:18.258 INFO:teuthology.orchestra.run.smithi113.stdout:Updating / installing... 2024-02-02T13:13:20.940 INFO:teuthology.orchestra.run.smithi091.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-02T13:13:25.625 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-6.7.0_gcd42140e6285-1 ######################################## 2024-02-02T13:13:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-02T13:13:43.326 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi091.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-02T13:13:43.327 DEBUG:teuthology.orchestra.run.smithi091:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/System.map-6.7.0-gcd42140e6285 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/config-6.7.0-gcd42140e6285 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/vmlinuz-6.7.0-gcd42140e6285 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch 2024-02-02T13:13:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-02T13:13:43.407 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-02T13:13:43.408 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-02T13:13:43.408 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-02T13:13:43.408 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-02T13:13:43.408 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-02T13:13:43.408 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-02T13:13:43.411 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-02T13:13:43.411 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-02T13:13:43.411 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-02T13:13:43.411 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ansi_cprng.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/anubis.ko 2024-02-02T13:13:43.412 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/arc4.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_memcpy.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_pq.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_tx.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_xor.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/raid6test.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authenc.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authencesn.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blake2b_generic.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_common.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_generic.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/camellia_generic.ko 2024-02-02T13:13:43.413 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast5_generic.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast6_generic.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast_common.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ccm.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cmac.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crc32_generic.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_engine.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_user.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cts.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/des_generic.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/dh_generic.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/echainiv.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/essiv.ko 2024-02-02T13:13:43.414 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/fcrypt.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/khazad.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4hc.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/md4.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/michael_mic.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcbc.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcrypt.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/rmd160.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/seed.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/serpent_generic.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tcrypt.ko 2024-02-02T13:13:43.415 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tea.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_common.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_generic.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/vmac.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/wp512.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xcbc.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xor.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xxhash_generic.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_pad.ko 2024-02-02T13:13:43.416 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/custom_method.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/ec_sys.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/platform_profile.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbs.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbshc.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/video.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/acard-ahci.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ahci_platform.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ata_generic.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/libahci_platform.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_acpi.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ali.ko 2024-02-02T13:13:43.417 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_amd.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_artop.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atiixp.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atp867x.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd640.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd64x.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cypress.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_efar.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt366.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt37x.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-02T13:13:43.418 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it8213.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it821x.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_jmicron.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_marvell.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_mpiix.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_netcell.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ninja32.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87410.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87415.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_oldpiix.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_opti.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_optidma.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pcmcia.ko 2024-02-02T13:13:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_piccolo.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_rdc.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sch.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_serverworks.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sil680.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sis.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sl82c105.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_triflex.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_via.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pdc_adma.ko 2024-02-02T13:13:43.420 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_inic162x.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_mv.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_nv.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_promise.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_qstor.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil24.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sis.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_svw.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sx4.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_uli.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_via.ko 2024-02-02T13:13:43.421 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_vsc.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay/ks0108.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma/bcma.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe/aoe.ko 2024-02-02T13:13:43.422 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/brd.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd/drbd.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/floppy.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/loop.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/nbd.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk/null_blk.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/pktcdvd.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/rbd.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/virtio_blk.ko 2024-02-02T13:13:43.423 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkfront.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram/zram.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hangcheck-timer.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/via-rng.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-02T13:13:43.424 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/lp.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave/mwave.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ppdev.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tlclk.ko 2024-02-02T13:13:43.425 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/uv_mmtimer.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/virtio_console.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-02T13:13:43.426 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-aes.ko 2024-02-02T13:13:43.427 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-sha.ko 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca/dca.ko 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat 2024-02-02T13:13:43.428 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/amd64_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/e752x_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/edac_mce_amd.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3000_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3200_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5100_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5400_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7300_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7core_edac.ko 2024-02-02T13:13:43.429 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i82975x_edac.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/ie31200_edac.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/sb_edac.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/x38_edac.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/edd.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-ich.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-appleir.ko 2024-02-02T13:13:43.430 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-aureal.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-axff.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-dr.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elecom.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elo.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-emsff.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gaff.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gt683r.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gyration.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-02T13:13:43.431 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtekff.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-hyperv.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-icade.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-keytouch.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-kye.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lcpower.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-led.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lenovo.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-02T13:13:43.432 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-multitouch.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-ortek.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-penmount.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-petalynx.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-picolcd.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-pl.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-plantronics.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-primax.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-rmi.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-common.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-02T13:13:43.433 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-saitek.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-samsung.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-02T13:13:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sjoy.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sony.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-speedlink.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-steelseries.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sunplus.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tivo.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tmff.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-topseed.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-twinhan.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-uclogic.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-waltop.ko 2024-02-02T13:13:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-wiimote.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-xinmo.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zpff.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zydacron.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/uhid.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/wacom.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_balloon.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_utils.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_vmbus.ko 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon 2024-02-02T13:13:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru3.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7414.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7418.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adc128d818.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1025.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1026.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1029.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1031.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm9240.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ads7828.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7410.ko 2024-02-02T13:13:43.437 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7411.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7462.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7470.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7475.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7x10.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/amc6821.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/applesmc.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asb100.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asc7621.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/atxp1.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/coretemp.ko 2024-02-02T13:13:43.438 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dme1737.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds1621.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds620.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc1403.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc6w201.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71805f.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71882fg.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f75375s.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fam15h_power.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fschmd.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g760a.ko 2024-02-02T13:13:43.439 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g762.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl518sm.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl520sm.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5500_temp.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5k_amb.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmaem.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmpex.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina209.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina2xx.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/it87.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k10temp.ko 2024-02-02T13:13:43.440 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k8temp.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lineage-pem.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm63.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm73.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm75.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm77.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm78.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm80.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm83.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm85.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm87.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm90.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm92.ko 2024-02-02T13:13:43.441 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm93.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95234.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95241.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95245.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc2945.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4151.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4215.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4222.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4245.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4260.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4261.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max16065.ko 2024-02-02T13:13:43.442 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1619.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1668.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max197.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6639.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6650.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6697.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/mcp3021.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6683.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775-core.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct7802.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87360.ko 2024-02-02T13:13:43.443 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87427.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pcf8591.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-02T13:13:43.444 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/powr1220.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5627.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5636.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht15.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht21.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/shtc1.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sis5595.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47b397.ko 2024-02-02T13:13:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m1.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m192.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/thmc50.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp102.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp103.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp401.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp421.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via-cputemp.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via686a.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt1211.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt8231.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627ehf.ko 2024-02-02T13:13:43.446 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627hf.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83781d.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83791d.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83792d.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83793.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83795.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l785ts.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l786ng.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses 2024-02-02T13:13:43.447 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-02T13:13:43.448 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-dev.ko 2024-02-02T13:13:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-mux.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-smbus.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-stub.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_core.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-02T13:13:43.450 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib 2024-02-02T13:13:43.451 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-02T13:13:43.452 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/gameport.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/joydev.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/appletouch.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/bcm5974.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/cyapatp.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/sermouse.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-02T13:13:43.453 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/altera_ps2.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/arc_ps2.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/serio_raw.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/sparse-keymap.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-blinkm.ko 2024-02-02T13:13:43.454 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lm3530.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lp3944.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-ss4200.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-02T13:13:43.455 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache/bcache.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-bio-prison.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache-smq.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-crypt.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-delay.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-flakey.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-log-userspace.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-multipath.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-queue-length.ko 2024-02-02T13:13:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-raid.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-round-robin.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-service-time.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-switch.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-thin-pool.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-verity.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/faulty.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/linear.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/multipath.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid0.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid1.ko 2024-02-02T13:13:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid10.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid456.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptbase.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptctl.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptfc.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptlan.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptsas.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptscsih.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptspi.ko 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd 2024-02-02T13:13:43.458 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_ich.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_sch.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/mfd-core.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/sm501.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/viperboard.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/vx855.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/wl1273-core.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds9802als.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds990x.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/bh1770glc.ko 2024-02-02T13:13:43.459 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710/cb710.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/at24.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/max6875.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/enclosure.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/hpilo.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29003.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29020.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-02T13:13:43.460 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-me.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-txe.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_7xx1.ko 2024-02-02T13:13:43.461 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_core.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tsl2550.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_balloon.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding/bonding.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-02T13:13:43.462 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dummy.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/eql.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-02T13:13:43.463 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd 2024-02-02T13:13:43.464 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-02T13:13:43.465 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-02T13:13:43.466 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco 2024-02-02T13:13:43.467 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-02T13:13:43.468 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dnet.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ethoc.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/fealnx.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-02T13:13:43.469 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-02T13:13:43.470 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/jme.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-02T13:13:43.471 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-02T13:13:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-02T13:13:43.473 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-02T13:13:43.474 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-02T13:13:43.475 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-02T13:13:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/geneve.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ifb.ko 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan 2024-02-02T13:13:43.477 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvlan.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvtap.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mii.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/net_failover.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/netconsole.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/nlmon.ko 2024-02-02T13:13:43.478 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/amd.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm87xx.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/broadcom.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/cicada.ko 2024-02-02T13:13:43.479 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/davicom.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/dp83640.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/et1011c.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/icplus.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/lxt.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/marvell.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/micrel.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/national.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/phylink.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/qsemi.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/realtek.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/smsc.ko 2024-02-02T13:13:43.480 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/ste10Xp.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/vitesse.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_async.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppoe.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppox.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pptp.ko 2024-02-02T13:13:43.481 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slhc.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slip.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/sungem_phy.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tap.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_random.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tun.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/veth.ko 2024-02-02T13:13:43.482 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/virtio_net.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan/vxlan.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netfront.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb/ntb.ko 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host 2024-02-02T13:13:43.483 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme-core.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_cs.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_pc.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_serial.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie 2024-02-02T13:13:43.484 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/xen-pcifront.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/i82092.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/pd6729.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-02T13:13:43.485 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acerhdf.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/hdaps.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-02T13:13:43.486 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel_ips.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi.ko 2024-02-02T13:13:43.487 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps_parport.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/pps_core.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp_kvm.ko 2024-02-02T13:13:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-em3027.ko 2024-02-02T13:13:43.489 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-max6900.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-02T13:13:43.490 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-x1205.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-9xxx.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-sas.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-xxxx.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/BusLogic.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/a100u2w.ko 2024-02-02T13:13:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/advansys.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/am53c974.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/atp870u.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-02T13:13:43.492 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa/bfa.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ch.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-02T13:13:43.493 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dc395x.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dmx3191d.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esp_scsi.ko 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe 2024-02-02T13:13:43.494 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic/fnic.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hpsa.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hptiop.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hv_storvsc.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/initio.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ipr.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ips.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci/isci.ko 2024-02-02T13:13:43.495 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc/libfc.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas/libsas.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid.ko 2024-02-02T13:13:43.496 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvumi.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pmcraid.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla1280.ko 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx 2024-02-02T13:13:43.497 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/raid_class.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_debug.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ses.ko 2024-02-02T13:13:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/st.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/stex.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/virtio_scsi.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/wd719x.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb/ssb.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-02T13:13:43.499 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_file.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_iblock.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_mod.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_pscsi.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_user.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal 2024-02-02T13:13:43.500 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-02T13:13:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_gsm.ko 2024-02-02T13:13:43.502 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_hdlc.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/nozomi.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/arc_uart.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-02T13:13:43.503 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_aec.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_cif.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_pci_generic.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_sercos3.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-acm.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usblp.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usbtmc.ko 2024-02-02T13:13:43.504 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/mdc800.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/microtek.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/adutux.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/appledisplay.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi26.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi62.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ezusb.ko 2024-02-02T13:13:43.505 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/idmouse.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/iowarrior.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ldusb.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/legousbtower.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usb3503.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usblcd.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/uss720.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/yurex.ko 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy 2024-02-02T13:13:43.506 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy/phy-generic.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/aircable.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ark3116.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ch341.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cp210x.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cyberjack.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/empeg.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-02T13:13:43.507 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_ti.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipaq.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipw.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ir-usb.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mct_u232.ko 2024-02-02T13:13:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7720.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7840.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/navman.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/omninet.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/opticon.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/option.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/oti6858.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/pl2303.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcaux.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcserial.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/quatech2.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/safe_serial.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/sierra.ko 2024-02-02T13:13:43.509 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ssu100.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/symbolserial.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_debug.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/visor.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/whiteheat.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/uas.ko 2024-02-02T13:13:43.510 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-karma.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-02T13:13:43.511 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/usb-storage.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-02T13:13:43.512 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_net.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_scsi.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/apple_bl.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/lcd.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio.ko 2024-02-02T13:13:43.513 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_balloon.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_mmio.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_ring.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2482.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2490.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-02T13:13:43.514 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/wire.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-02T13:13:43.515 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/hpwdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/i6300esb.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ib700wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ibmasr.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it87_wdt.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/machzwd.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/nv_tco.ko 2024-02-02T13:13:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/softdog.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/via_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/wdt_pci.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/xen_wdt.ko 2024-02-02T13:13:43.517 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-evtchn.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntalloc.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntdev.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-privcmd.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p/9p.ko 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs 2024-02-02T13:13:43.518 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs/affs.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs/befs.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/binfmt_misc.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs/btrfs.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles/cachefiles.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph/ceph.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda/coda.ko 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs 2024-02-02T13:13:43.519 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs/cramfs.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm/dlm.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs/f2fs.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/fat.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/msdos.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/vfat.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache/fscache.ko 2024-02-02T13:13:43.520 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/cuse.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/fuse.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2/gfs2.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs/hfs.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus/hfsplus.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs/isofs.ko 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs 2024-02-02T13:13:43.521 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs/jfs.ko 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd/lockd.ko 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix/minix.ko 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs/netfs.ko 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout 2024-02-02T13:13:43.522 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfs.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv3.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv4.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/grace.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/nfs_acl.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd/nfsd.ko 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2 2024-02-02T13:13:43.523 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2/nilfs2.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-celtic.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-centeuro.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-croatian.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-cyrillic.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-gaelic.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-greek.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-iceland.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-inuit.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-roman.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-romanian.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-turkish.ko 2024-02-02T13:13:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1250.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1251.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1255.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp737.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp775.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp850.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp852.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp855.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp857.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp860.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp861.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp862.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp863.ko 2024-02-02T13:13:43.525 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp864.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp865.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp866.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp869.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp874.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp932.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp936.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp949.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp950.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_euc-jp.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-1.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-13.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-14.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-15.ko 2024-02-02T13:13:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-2.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-3.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-4.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-5.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-6.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-7.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-9.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-r.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-ru.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-u.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_utf8.ko 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster 2024-02-02T13:13:43.527 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs/overlay.ko 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore 2024-02-02T13:13:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore/ramoops.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs/reiserfs.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs/romfs.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client/cifs.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_arc4.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_md4.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs/squashfs.ko 2024-02-02T13:13:43.529 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv/sysv.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf/udf.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs/ufs.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs/xfs.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu/rcutorture.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/torture.ko 2024-02-02T13:13:43.530 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/asn1_encoder.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc-itu-t.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc8.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libarc4.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libdes.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/libcrc32c.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lru_cache.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4_compress.ko 2024-02-02T13:13:43.531 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4hc_compress.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math/cordic.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6/raid6_pq.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_bm.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_fsm.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_kmp.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd/zstd_compress.ko 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm 2024-02-02T13:13:43.532 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm/hwpoison-inject.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/6lowpan.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_dest.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_fragment.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_hop.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_mobility.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_routing.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_udp.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/garp.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/mrp.ko 2024-02-02T13:13:43.533 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/p8022.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/psnap.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/stp.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q/8021q.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_fd.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_rdma.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_virtio.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv/batman-adv.ko 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge 2024-02-02T13:13:43.534 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/br_netfilter.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/bridge.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-02T13:13:43.535 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtables.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-02T13:13:43.536 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph/libceph.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/failover.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/pktgen.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_diag.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv4.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv6.ko 2024-02-02T13:13:43.537 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver/dns_resolver.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/dsa_core.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_brcm.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_dsa.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_none.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_trailer.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-02T13:13:43.538 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ah4.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/esp4.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/fou.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/gre.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/inet_diag.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_gre.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_tunnel.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_vti.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipcomp.ko 2024-02-02T13:13:43.539 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipip.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-02T13:13:43.540 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_bic.ko 2024-02-02T13:13:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_dctcp.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_diag.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_highspeed.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_htcp.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_hybla.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_illinois.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_lp.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_scalable.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_vegas.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_veno.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_westwood.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_yeah.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tunnel4.ko 2024-02-02T13:13:43.542 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_diag.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_tunnel.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ah6.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/esp6.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/fou6.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_tunnel.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_vti.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ipcomp6.ko 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter 2024-02-02T13:13:43.543 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-02T13:13:43.544 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-02T13:13:43.545 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/sit.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/tunnel6.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key/af_key.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_core.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_eth.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip6.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_netlink.ko 2024-02-02T13:13:43.546 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ppp.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc/llc.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154/mac802154.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls/mpls_gso.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-02T13:13:43.547 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-02T13:13:43.548 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-02T13:13:43.549 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conncount.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-02T13:13:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_log_syslog.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_irc.ko 2024-02-02T13:13:43.551 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_sip.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_tables.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_log.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_chain_nat.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_compat.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_ct.ko 2024-02-02T13:13:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fib.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_hash.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_limit.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_log.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_masq.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_nat.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_numgen.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_queue.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_quota.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_redir.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject.ko 2024-02-02T13:13:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject_inet.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_socket.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_tunnel.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_AUDIT.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CT.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_DSCP.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HL.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HMARK.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-02T13:13:43.554 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LED.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LOG.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NETMAP.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFLOG.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_RATEEST.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_SECMARK.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TEE.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TPROXY.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TRACE.ko 2024-02-02T13:13:43.555 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_addrtype.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_bpf.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cgroup.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cluster.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_comment.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connbytes.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlabel.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlimit.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connmark.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_conntrack.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cpu.ko 2024-02-02T13:13:43.556 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dccp.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_devgroup.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dscp.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ecn.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_esp.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hashlimit.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_helper.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hl.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipcomp.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_iprange.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipvs.ko 2024-02-02T13:13:43.557 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_l2tp.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_length.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_limit.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mac.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mark.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_multiport.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nat.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nfacct.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_osf.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_owner.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_physdev.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_pkttype.ko 2024-02-02T13:13:43.558 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_policy.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_quota.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_rateest.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_realm.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_recent.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_sctp.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_set.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_socket.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_state.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_statistic.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_string.ko 2024-02-02T13:13:43.559 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_tcpmss.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_time.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_u32.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink/netlink_diag.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh/nsh.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/openvswitch.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-geneve.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-gre.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-vxlan.ko 2024-02-02T13:13:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet/af_packet_diag.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_rdma.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_tcp.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_csum.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_gact.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_ipt.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_mirred.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_nat.ko 2024-02-02T13:13:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_pedit.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_police.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_simple.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_skbedit.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_vlan.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_basic.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_bpf.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_flow.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_fw.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_route.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_u32.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_cmp.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_ipset.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_meta.ko 2024-02-02T13:13:43.562 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_nbyte.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_text.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_u32.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_choke.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_codel.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_drr.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_fq.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_gred.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hfsc.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hhf.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_htb.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_ingress.ko 2024-02-02T13:13:43.563 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio_lib.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_multiq.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_netem.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_pie.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_plug.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_prio.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_qfq.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_red.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfb.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfq.ko 2024-02-02T13:13:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_tbf.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_teql.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp_diag.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/sunrpc.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-02T13:13:43.565 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/diag.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/tipc.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix/unix_diag.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm 2024-02-02T13:13:43.566 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys/trusted.ko 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib/irqbypass.ko 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias.bin 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin 2024-02-02T13:13:43.567 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.alias.bin 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.bin 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.modinfo 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep.bin 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.devname 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.order 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.softdep 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols 2024-02-02T13:13:43.568 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols.bin 2024-02-02T13:13:43.569 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-gcd42140e6285 2024-02-02T13:13:43.569 DEBUG:teuthology.orchestra.run.smithi091:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:postinstall scriptlet (using /bin/sh): 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-gcd42140e6285 -a -r /boot/System.map-6.7.0-gcd42140e6285 ]; then 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:cp /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:cp /boot/System.map-6.7.0-gcd42140e6285 /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:rm -f /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/System.map-6.7.0-gcd42140e6285 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:/sbin/installkernel 6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:rm -f /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2024-02-02T13:13:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:new-kernel-pkg --remove 6.7.0-gcd42140e6285 --rminitrd --initrdfile=/boot/initramfs-6.7.0-gcd42140e6285.img 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-install remove 6.7.0-gcd42140e6285 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:/sbin/update-bootloader --remove 6.7.0-gcd42140e6285 2024-02-02T13:13:43.592 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2024-02-02T13:13:43.593 DEBUG:teuthology.orchestra.run.smithi091:> sudo rpm -qi grub2-tools 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Name : grub2-tools 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Epoch : 1 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Version : 2.06 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Release : 68.el9 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: x86_64 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Group : Unspecified 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:Size : 8264254 2024-02-02T13:13:43.668 INFO:teuthology.orchestra.run.smithi091.stdout:License : GPLv3+ 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Packager : builder@centos.org 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Vendor : CentOS 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Summary : Support tools for GRUB. 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:Description : 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout:hardware devices. 2024-02-02T13:13:43.669 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:13:43.670 INFO:teuthology.orchestra.run.smithi091.stdout:This subpackage provides tools for support of all platforms. 2024-02-02T13:13:43.670 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-02T13:13:43.670 INFO:teuthology.task.kernel:Updating grub on smithi091 to boot 6.7.0-gcd42140e6285 2024-02-02T13:13:43.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-02T13:13:44.352 INFO:teuthology.orchestra.run.smithi091.stderr:Generating grub configuration file ... 2024-02-02T13:13:45.707 INFO:teuthology.orchestra.run.smithi091.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-02T13:13:45.739 INFO:teuthology.orchestra.run.smithi091.stderr:done 2024-02-02T13:13:45.741 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2024-02-02T13:13:45.756 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.dWhi5Sb1Ng 2024-02-02T13:13:45.756 DEBUG:teuthology.orchestra.run.smithi091:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dWhi5Sb1Ng 2024-02-02T13:13:45.825 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.dWhi5Sb1Ng 2024-02-02T13:13:45.967 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.dWhi5Sb1Ng is 6KB 2024-02-02T13:13:46.015 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.dWhi5Sb1Ng 2024-02-02T13:13:46.029 DEBUG:teuthology.orchestra.run.smithi091:> sudo /bin/ls /boot/loader/entries || true 2024-02-02T13:13:46.093 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-02T13:13:46.093 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-02T13:13:46.093 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-02T13:13:46.093 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285.conf 2024-02-02T13:13:46.095 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285 2024-02-02T13:13:46.261 DEBUG:teuthology.orchestra.run.smithi091:> sudo shutdown -r now 2024-02-02T13:14:04.012 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi113.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-02T13:14:04.012 DEBUG:teuthology.orchestra.run.smithi113:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/System.map-6.7.0-gcd42140e6285 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/config-6.7.0-gcd42140e6285 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/vmlinuz-6.7.0-gcd42140e6285 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-02T13:14:04.083 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-02T13:14:04.084 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm.ko 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto 2024-02-02T13:14:04.085 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ansi_cprng.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/anubis.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/arc4.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_memcpy.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_pq.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_tx.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_xor.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/raid6test.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authenc.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authencesn.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blake2b_generic.ko 2024-02-02T13:14:04.086 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_common.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/camellia_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast5_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast6_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast_common.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ccm.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cmac.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crc32_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_engine.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_user.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cts.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/des_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/dh_generic.ko 2024-02-02T13:14:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/echainiv.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/essiv.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/fcrypt.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/khazad.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4hc.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/md4.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/michael_mic.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcbc.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcrypt.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/rmd160.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/seed.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/serpent_generic.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tcrypt.ko 2024-02-02T13:14:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tea.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_common.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_generic.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/vmac.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/wp512.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xcbc.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xor.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xxhash_generic.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_pad.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/custom_method.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/ec_sys.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/platform_profile.ko 2024-02-02T13:14:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbs.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbshc.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/video.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/acard-ahci.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ahci_platform.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ata_generic.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/libahci_platform.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_acpi.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ali.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_amd.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_artop.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atiixp.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atp867x.ko 2024-02-02T13:14:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd640.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd64x.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cypress.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_efar.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt366.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt37x.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it8213.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it821x.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_jmicron.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_marvell.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_mpiix.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_netcell.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ninja32.ko 2024-02-02T13:14:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87410.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87415.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_oldpiix.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_opti.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_optidma.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pcmcia.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_piccolo.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_rdc.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sch.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_serverworks.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sil680.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sis.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sl82c105.ko 2024-02-02T13:14:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_triflex.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_via.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pdc_adma.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_inic162x.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_mv.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_nv.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_promise.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_qstor.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil24.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sis.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_svw.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sx4.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_uli.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_via.ko 2024-02-02T13:14:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_vsc.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay/ks0108.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma/bcma.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe/aoe.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/brd.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd/drbd.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/floppy.ko 2024-02-02T13:14:04.094 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/loop.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/nbd.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk/null_blk.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/pktcdvd.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/rbd.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/virtio_blk.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkfront.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram/zram.ko 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char 2024-02-02T13:14:04.095 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hangcheck-timer.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/via-rng.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/lp.ko 2024-02-02T13:14:04.096 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave/mwave.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ppdev.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tlclk.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/uv_mmtimer.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/virtio_console.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-02T13:14:04.097 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-aes.ko 2024-02-02T13:14:04.098 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-sha.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca/dca.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/amd64_edac.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/e752x_edac.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/edac_mce_amd.ko 2024-02-02T13:14:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3000_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3200_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5100_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5400_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7300_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7core_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i82975x_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/ie31200_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/sb_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/x38_edac.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/edd.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio 2024-02-02T13:14:04.100 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-ich.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-appleir.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-aureal.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-axff.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-dr.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elecom.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elo.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-emsff.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gaff.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gt683r.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gyration.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-02T13:14:04.101 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtekff.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-hyperv.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-icade.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-keytouch.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-kye.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lcpower.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-led.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lenovo.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-multitouch.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-ortek.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-penmount.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-petalynx.ko 2024-02-02T13:14:04.102 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-picolcd.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-pl.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-plantronics.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-primax.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-rmi.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-common.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-02T13:14:04.103 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-saitek.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-samsung.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sjoy.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sony.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-speedlink.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-steelseries.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sunplus.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tivo.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tmff.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-topseed.ko 2024-02-02T13:14:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-twinhan.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-uclogic.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-waltop.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-wiimote.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-xinmo.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zpff.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zydacron.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/uhid.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/wacom.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_balloon.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_utils.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_vmbus.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru3.ko 2024-02-02T13:14:04.105 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7414.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7418.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adc128d818.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1025.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1026.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1029.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1031.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm9240.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ads7828.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7410.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7411.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7462.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7470.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7475.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7x10.ko 2024-02-02T13:14:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/amc6821.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/applesmc.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asb100.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asc7621.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/atxp1.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/coretemp.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dme1737.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds1621.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds620.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc1403.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc6w201.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71805f.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71882fg.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f75375s.ko 2024-02-02T13:14:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fam15h_power.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fschmd.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g760a.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g762.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl518sm.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl520sm.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5500_temp.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5k_amb.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmaem.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmpex.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina209.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina2xx.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/it87.ko 2024-02-02T13:14:04.108 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k10temp.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k8temp.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lineage-pem.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm63.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm73.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm75.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm77.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm78.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm80.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm83.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm85.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm87.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm90.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm92.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm93.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95234.ko 2024-02-02T13:14:04.109 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95241.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95245.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc2945.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4151.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4215.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4222.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4245.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4260.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4261.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max16065.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1619.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1668.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max197.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6639.ko 2024-02-02T13:14:04.110 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6650.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6697.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/mcp3021.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6683.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775-core.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct7802.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87360.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87427.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pcf8591.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-02T13:14:04.111 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/powr1220.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5627.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5636.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht15.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht21.ko 2024-02-02T13:14:04.112 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/shtc1.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sis5595.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47b397.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m1.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m192.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/thmc50.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp102.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp103.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp401.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp421.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via-cputemp.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via686a.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt1211.ko 2024-02-02T13:14:04.113 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt8231.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627ehf.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627hf.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83781d.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83791d.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83792d.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83793.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83795.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l785ts.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l786ng.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-02T13:14:04.114 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-02T13:14:04.115 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-dev.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-mux.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-smbus.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-stub.ko 2024-02-02T13:14:04.116 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_core.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4 2024-02-02T13:14:04.117 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt 2024-02-02T13:14:04.118 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/gameport.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/joydev.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/appletouch.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/bcm5974.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/cyapatp.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-02T13:14:04.119 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/sermouse.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/altera_ps2.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/arc_ps2.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/serio_raw.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/sparse-keymap.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-blinkm.ko 2024-02-02T13:14:04.120 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lm3530.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lp3944.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-ss4200.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md 2024-02-02T13:14:04.121 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache/bcache.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-bio-prison.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache-smq.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-crypt.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-delay.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-flakey.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-log-userspace.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-multipath.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-queue-length.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-raid.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-round-robin.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-service-time.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-switch.ko 2024-02-02T13:14:04.122 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-thin-pool.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-verity.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/faulty.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/linear.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/multipath.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid0.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid1.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid10.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid456.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptbase.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptctl.ko 2024-02-02T13:14:04.123 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptfc.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptlan.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptsas.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptscsih.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptspi.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_ich.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_sch.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/mfd-core.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/sm501.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/viperboard.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/vx855.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/wl1273-core.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-02T13:14:04.124 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds9802als.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds990x.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/bh1770glc.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710/cb710.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/at24.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/max6875.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/enclosure.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/hpilo.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29003.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29020.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-02T13:14:04.125 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-me.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-txe.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_7xx1.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_core.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tsl2550.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_balloon.ko 2024-02-02T13:14:04.126 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding/bonding.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dummy.ko 2024-02-02T13:14:04.127 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/eql.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere 2024-02-02T13:14:04.128 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-02T13:14:04.129 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna 2024-02-02T13:14:04.130 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-02T13:14:04.131 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dnet.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-02T13:14:04.132 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ethoc.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/fealnx.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb 2024-02-02T13:14:04.133 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/jme.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-02T13:14:04.134 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia 2024-02-02T13:14:04.135 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-02T13:14:04.136 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-02T13:14:04.137 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via 2024-02-02T13:14:04.138 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/geneve.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ifb.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvlan.ko 2024-02-02T13:14:04.139 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvtap.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mii.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/net_failover.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/netconsole.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/nlmon.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-02T13:14:04.140 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/amd.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm87xx.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/broadcom.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/cicada.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/davicom.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/dp83640.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/et1011c.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/icplus.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/lxt.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/marvell.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/micrel.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/national.ko 2024-02-02T13:14:04.141 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/phylink.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/qsemi.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/realtek.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/smsc.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/ste10Xp.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/vitesse.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_async.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppoe.ko 2024-02-02T13:14:04.142 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppox.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pptp.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slhc.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slip.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/sungem_phy.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tap.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_random.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tun.ko 2024-02-02T13:14:04.143 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/veth.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/virtio_net.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan/vxlan.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netfront.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb/ntb.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme-core.ko 2024-02-02T13:14:04.144 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_cs.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_pc.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_serial.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/xen-pcifront.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/i82092.ko 2024-02-02T13:14:04.145 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/pd6729.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acerhdf.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-02T13:14:04.146 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/hdaps.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel_ips.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-02T13:14:04.147 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps_parport.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/pps_core.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp.ko 2024-02-02T13:14:04.148 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp_kvm.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-em3027.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-02T13:14:04.149 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-max6900.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-02T13:14:04.150 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-x1205.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-9xxx.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-sas.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-xxxx.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/BusLogic.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/a100u2w.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/advansys.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/am53c974.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-02T13:14:04.151 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/atp870u.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa/bfa.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ch.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-02T13:14:04.152 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dc395x.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dmx3191d.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esp_scsi.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-02T13:14:04.153 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic/fnic.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hpsa.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hptiop.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hv_storvsc.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/initio.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ipr.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ips.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci/isci.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc/libfc.ko 2024-02-02T13:14:04.154 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas/libsas.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-02T13:14:04.155 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvumi.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pmcraid.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla1280.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/raid_class.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_debug.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-02T13:14:04.156 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ses.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/st.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/stex.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/virtio_scsi.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/wd719x.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb/ssb.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-02T13:14:04.157 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_file.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_iblock.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_mod.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_pscsi.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_user.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-02T13:14:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty 2024-02-02T13:14:04.159 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_gsm.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_hdlc.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/nozomi.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/arc_uart.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-02T13:14:04.160 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_aec.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_cif.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_pci_generic.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_sercos3.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-acm.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usblp.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usbtmc.ko 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host 2024-02-02T13:14:04.161 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/mdc800.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/microtek.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/adutux.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/appledisplay.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi26.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi62.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ezusb.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/idmouse.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/iowarrior.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ldusb.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/legousbtower.ko 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga 2024-02-02T13:14:04.162 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usb3503.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usblcd.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/uss720.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/yurex.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy/phy-generic.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/aircable.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ark3116.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ch341.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cp210x.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cyberjack.ko 2024-02-02T13:14:04.163 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/empeg.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_ti.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipaq.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipw.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ir-usb.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-02T13:14:04.164 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mct_u232.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7720.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7840.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/navman.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/omninet.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/opticon.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/option.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/oti6858.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/pl2303.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcaux.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcserial.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/quatech2.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/safe_serial.ko 2024-02-02T13:14:04.165 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/sierra.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ssu100.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/symbolserial.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_debug.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/visor.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/whiteheat.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/uas.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-02T13:14:04.166 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-karma.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/usb-storage.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-02T13:14:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_net.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_scsi.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/apple_bl.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/lcd.ko 2024-02-02T13:14:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_balloon.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_mmio.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_ring.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2482.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2490.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-02T13:14:04.169 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/wire.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/hpwdt.ko 2024-02-02T13:14:04.170 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/i6300esb.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ib700wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ibmasr.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it87_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/machzwd.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/nv_tco.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/softdog.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-02T13:14:04.171 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/via_wdt.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/wdt_pci.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/xen_wdt.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-evtchn.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntalloc.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntdev.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-privcmd.ko 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs 2024-02-02T13:14:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p/9p.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs/affs.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs/befs.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/binfmt_misc.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs/btrfs.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles/cachefiles.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph/ceph.ko 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda 2024-02-02T13:14:04.173 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda/coda.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs/cramfs.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm/dlm.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs/f2fs.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/fat.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/msdos.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/vfat.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache/fscache.ko 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse 2024-02-02T13:14:04.174 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/cuse.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/fuse.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2/gfs2.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs/hfs.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus/hfsplus.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs/isofs.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs/jfs.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd/lockd.ko 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix 2024-02-02T13:14:04.175 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix/minix.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs/netfs.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfs.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv3.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv4.ko 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common 2024-02-02T13:14:04.176 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/grace.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/nfs_acl.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd/nfsd.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2/nilfs2.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-celtic.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-centeuro.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-croatian.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-cyrillic.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-gaelic.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-greek.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-iceland.ko 2024-02-02T13:14:04.177 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-inuit.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-roman.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-romanian.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-turkish.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1250.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1251.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1255.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp737.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp775.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp850.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp852.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp855.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp857.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp860.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp861.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp862.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp863.ko 2024-02-02T13:14:04.178 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp864.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp865.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp866.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp869.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp874.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp932.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp936.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp949.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp950.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_euc-jp.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-1.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-13.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-14.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-15.ko 2024-02-02T13:14:04.179 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-2.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-3.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-4.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-5.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-6.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-7.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-9.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-r.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-ru.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-u.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_utf8.ko 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2 2024-02-02T13:14:04.180 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs/overlay.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore/ramoops.ko 2024-02-02T13:14:04.181 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs/reiserfs.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs/romfs.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client/cifs.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_arc4.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_md4.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs/squashfs.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv/sysv.ko 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf 2024-02-02T13:14:04.182 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf/udf.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs/ufs.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs/xfs.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu/rcutorture.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/torture.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/asn1_encoder.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc-itu-t.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc8.ko 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto 2024-02-02T13:14:04.183 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libarc4.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libdes.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/libcrc32c.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lru_cache.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4_compress.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4hc_compress.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math/cordic.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6/raid6_pq.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_bm.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_fsm.ko 2024-02-02T13:14:04.184 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_kmp.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd/zstd_compress.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm/hwpoison-inject.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/6lowpan.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_dest.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_fragment.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_hop.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_mobility.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_routing.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_udp.ko 2024-02-02T13:14:04.185 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/garp.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/mrp.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/p8022.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/psnap.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/stp.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q/8021q.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_fd.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_rdma.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_virtio.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv/batman-adv.ko 2024-02-02T13:14:04.186 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/br_netfilter.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/bridge.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-02T13:14:04.187 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtables.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-02T13:14:04.188 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph/libceph.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/failover.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/pktgen.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_diag.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv4.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv6.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver/dns_resolver.ko 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa 2024-02-02T13:14:04.189 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/dsa_core.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_brcm.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_dsa.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_none.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_trailer.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ah4.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/esp4.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/fou.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/gre.ko 2024-02-02T13:14:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/inet_diag.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_gre.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_tunnel.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_vti.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipcomp.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipip.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-02T13:14:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_bic.ko 2024-02-02T13:14:04.192 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_dctcp.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_diag.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_highspeed.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_htcp.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_hybla.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_illinois.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_lp.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_scalable.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_vegas.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_veno.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_westwood.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_yeah.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tunnel4.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_diag.ko 2024-02-02T13:14:04.193 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_tunnel.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ah6.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/esp6.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/fou6.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_tunnel.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_vti.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ipcomp6.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-02T13:14:04.194 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-02T13:14:04.195 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/sit.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/tunnel6.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key/af_key.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_core.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-02T13:14:04.196 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_eth.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip6.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_netlink.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ppp.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc/llc.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154/mac802154.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls/mpls_gso.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-02T13:14:04.197 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-02T13:14:04.198 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conncount.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack.ko 2024-02-02T13:14:04.199 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table.ko 2024-02-02T13:14:04.200 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_log_syslog.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_irc.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_sip.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_tables.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_log.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-02T13:14:04.201 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_chain_nat.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_compat.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_ct.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fib.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_hash.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_limit.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_log.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_masq.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_nat.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_numgen.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_queue.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_quota.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_redir.ko 2024-02-02T13:14:04.202 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject_inet.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_socket.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_tunnel.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_AUDIT.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CT.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_DSCP.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HL.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HMARK.ko 2024-02-02T13:14:04.203 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LED.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LOG.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NETMAP.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFLOG.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_RATEEST.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_SECMARK.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TEE.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TPROXY.ko 2024-02-02T13:14:04.204 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TRACE.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_addrtype.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_bpf.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cgroup.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cluster.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_comment.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connbytes.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlabel.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlimit.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connmark.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_conntrack.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cpu.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dccp.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_devgroup.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dscp.ko 2024-02-02T13:14:04.205 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ecn.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_esp.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hashlimit.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_helper.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hl.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipcomp.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_iprange.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipvs.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_l2tp.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_length.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_limit.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mac.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mark.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_multiport.ko 2024-02-02T13:14:04.206 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nat.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nfacct.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_osf.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_owner.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_physdev.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_pkttype.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_policy.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_quota.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_rateest.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_realm.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_recent.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_sctp.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_set.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_socket.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_state.ko 2024-02-02T13:14:04.207 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_statistic.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_string.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_tcpmss.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_time.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_u32.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink/netlink_diag.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh/nsh.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/openvswitch.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-geneve.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-gre.ko 2024-02-02T13:14:04.208 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-vxlan.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet/af_packet_diag.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_rdma.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_tcp.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_csum.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_gact.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_ipt.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_mirred.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_nat.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_pedit.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_police.ko 2024-02-02T13:14:04.209 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_simple.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_skbedit.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_vlan.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_basic.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_bpf.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_flow.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_fw.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_route.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_u32.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_cmp.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_ipset.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_meta.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_nbyte.ko 2024-02-02T13:14:04.210 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_text.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_u32.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_choke.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_codel.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_drr.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_fq.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_gred.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hfsc.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hhf.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_htb.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_ingress.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio_lib.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_multiq.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_netem.ko 2024-02-02T13:14:04.211 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_pie.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_plug.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_prio.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_qfq.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_red.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfb.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfq.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_tbf.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_teql.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp_diag.ko 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss 2024-02-02T13:14:04.212 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/sunrpc.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/diag.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/tipc.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix/unix_diag.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-02T13:14:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys/trusted.ko 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib 2024-02-02T13:14:04.214 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib/irqbypass.ko 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias.bin 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.alias.bin 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.bin 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.modinfo 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep.bin 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.devname 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.order 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.softdep 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols 2024-02-02T13:14:04.215 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols.bin 2024-02-02T13:14:04.216 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-gcd42140e6285 2024-02-02T13:14:04.216 DEBUG:teuthology.orchestra.run.smithi113:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:postinstall scriptlet (using /bin/sh): 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-gcd42140e6285 -a -r /boot/System.map-6.7.0-gcd42140e6285 ]; then 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:cp /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:cp /boot/System.map-6.7.0-gcd42140e6285 /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:rm -f /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/System.map-6.7.0-gcd42140e6285 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/installkernel 6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:rm -f /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2024-02-02T13:14:04.238 INFO:teuthology.orchestra.run.smithi113.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:new-kernel-pkg --remove 6.7.0-gcd42140e6285 --rminitrd --initrdfile=/boot/initramfs-6.7.0-gcd42140e6285.img 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-install remove 6.7.0-gcd42140e6285 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/update-bootloader --remove 6.7.0-gcd42140e6285 2024-02-02T13:14:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2024-02-02T13:14:04.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo rpm -qi grub2-tools 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Name : grub2-tools 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Epoch : 1 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Version : 2.06 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Release : 68.el9 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: x86_64 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Group : Unspecified 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Size : 8264254 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:License : GPLv3+ 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-02T13:14:04.314 INFO:teuthology.orchestra.run.smithi113.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Packager : builder@centos.org 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Vendor : CentOS 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Summary : Support tools for GRUB. 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:Description : 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:hardware devices. 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:14:04.315 INFO:teuthology.orchestra.run.smithi113.stdout:This subpackage provides tools for support of all platforms. 2024-02-02T13:14:04.316 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-02T13:14:04.316 INFO:teuthology.task.kernel:Updating grub on smithi113 to boot 6.7.0-gcd42140e6285 2024-02-02T13:14:04.317 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-02T13:14:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2024-02-02T13:14:06.198 INFO:teuthology.orchestra.run.smithi113.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-02T13:14:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:done 2024-02-02T13:14:06.240 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-02-02T13:14:06.255 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.4aLUYdBPHE 2024-02-02T13:14:06.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4aLUYdBPHE 2024-02-02T13:14:06.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.4aLUYdBPHE 2024-02-02T13:14:06.472 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.4aLUYdBPHE is 6KB 2024-02-02T13:14:06.521 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.4aLUYdBPHE 2024-02-02T13:14:06.535 DEBUG:teuthology.orchestra.run.smithi113:> sudo /bin/ls /boot/loader/entries || true 2024-02-02T13:14:06.600 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-02T13:14:06.600 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-02T13:14:06.600 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-02T13:14:06.600 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285.conf 2024-02-02T13:14:06.601 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285 2024-02-02T13:14:06.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo shutdown -r now 2024-02-02T13:14:16.264 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-02T13:14:16.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-02T13:14:16.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:14:36.769 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-02-02T13:14:36.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:14:36.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:15:11.292 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-02-02T13:15:16.272 DEBUG:teuthology.orchestra.remote:timed out 2024-02-02T13:15:20.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:15:20.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:15:23.356 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-02-02T13:15:25.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-02T13:15:25.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:15:26.676 DEBUG:teuthology.orchestra.run.smithi091:> true 2024-02-02T13:15:27.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-02T13:15:27.420 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.7.0-gcd42140e6285"... 2024-02-02T13:15:27.420 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2024-02-02T13:15:27.472 INFO:teuthology.orchestra.run.smithi091.stdout:6.7.0-gcd42140e6285 2024-02-02T13:15:27.472 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-gcd42140e6285 vs 6.7.0-gcd42140e6285 2024-02-02T13:15:27.472 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-02T13:15:27.472 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-02T13:15:28.475 DEBUG:teuthology.parallel:result is None 2024-02-02T13:15:35.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:15:35.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:16:07.189 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-02-02T13:16:07.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:16:07.879 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.7.0-gcd42140e6285"... 2024-02-02T13:16:07.879 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-02-02T13:16:07.914 INFO:teuthology.orchestra.run.smithi113.stdout:6.7.0-gcd42140e6285 2024-02-02T13:16:07.915 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-gcd42140e6285 vs 6.7.0-gcd42140e6285 2024-02-02T13:16:07.915 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-02T13:16:07.915 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-02T13:16:08.915 DEBUG:teuthology.parallel:result is None 2024-02-02T13:16:08.916 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-02T13:16:08.923 INFO:teuthology.task.internal:Creating test directory... 2024-02-02T13:16:08.923 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-02T13:16:08.933 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-02T13:16:08.984 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-02T13:16:08.991 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-02T13:16:08.998 INFO:teuthology.task.internal:Creating archive directory... 2024-02-02T13:16:08.998 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-02T13:16:09.003 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-02T13:16:09.078 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-02T13:16:09.084 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-02T13:16:09.085 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-02T13:16:09.092 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-02T13:16:09.218 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T13:16:09.269 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T13:16:09.275 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T13:16:09.328 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T13:16:09.333 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-02T13:16:09.341 INFO:teuthology.task.internal:Configuring sudo... 2024-02-02T13:16:09.341 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-02T13:16:09.347 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-02T13:16:09.431 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-02T13:16:09.440 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-02T13:16:09.440 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-02T13:16:09.447 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-02T13:16:09.493 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T13:16:09.561 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T13:16:09.665 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T13:16:09.746 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T13:16:09.840 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:16:09.840 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-02T13:16:09.932 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T13:16:10.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T13:16:10.090 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T13:16:10.161 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T13:16:10.265 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:16:10.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-02T13:16:10.349 DEBUG:teuthology.orchestra.run.smithi091:> sudo service rsyslog restart 2024-02-02T13:16:10.356 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-02-02T13:16:10.481 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T13:16:10.491 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T13:16:10.928 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-02T13:16:10.936 INFO:teuthology.task.internal:Starting timer... 2024-02-02T13:16:10.936 INFO:teuthology.run_tasks:Running task pcp... 2024-02-02T13:16:10.946 INFO:teuthology.run_tasks:Running task selinux... 2024-02-02T13:16:10.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo service auditd rotate 2024-02-02T13:16:11.082 INFO:teuthology.orchestra.run.smithi091.stdout:Rotating logs: 2024-02-02T13:16:11.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-02-02T13:16:11.192 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-02-02T13:16:11.195 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-02T13:16:11.196 DEBUG:teuthology.orchestra.run.smithi091:> /usr/sbin/getenforce 2024-02-02T13:16:11.258 INFO:teuthology.orchestra.run.smithi091.stdout:Permissive 2024-02-02T13:16:11.258 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-02-02T13:16:11.309 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-02-02T13:16:11.309 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi091.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-02-02T13:16:11.309 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T13:16:11.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:16:11.396 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T13:16:11.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:16:11.479 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-02T13:16:11.479 DEBUG:teuthology.orchestra.run.smithi091:> sudo /usr/sbin/setenforce permissive 2024-02-02T13:16:11.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-02-02T13:16:11.757 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-02T13:16:11.767 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-02T13:16:11.774 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-02T13:16:11.924 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-02T13:16:11.940 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-02T13:16:11.941 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi091.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-02T13:22:04.269 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi091.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-02-02T13:22:04.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-02T13:22:04.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:22:04.414 DEBUG:teuthology.orchestra.run.smithi091:> true 2024-02-02T13:22:04.563 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2024-02-02T13:22:04.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:22:04.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T13:22:04.704 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-02-02T13:22:04.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-02T13:22:04.856 INFO:teuthology.run_tasks:Running task clock... 2024-02-02T13:22:04.866 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-02T13:22:04.866 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-02T13:22:04.866 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T13:22:04.874 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-02T13:22:04.874 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T13:22:04.969 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-02T13:22:04.970 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-02T13:22:05.021 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-02T13:22:05.022 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-02T13:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: ntpd: command not found 2024-02-02T13:22:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-02-02T13:22:05.156 INFO:teuthology.orchestra.run.smithi091.stdout:506 Cannot talk to daemon 2024-02-02T13:22:05.160 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-02-02T13:22:05.204 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-02T13:22:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-02T13:22:05.258 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-02T13:22:05.264 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-02T13:22:05.377 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2024-02-02T13:22:05.383 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-02-02T13:22:05.384 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T13:22:05.385 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2024-02-02T13:22:05.385 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.385 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.385 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.385 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:22:05.393 INFO:teuthology.run_tasks:Running task install... 2024-02-02T13:22:05.402 DEBUG:teuthology.task.install:project ceph 2024-02-02T13:22:05.402 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}} 2024-02-02T13:22:05.402 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'} 2024-02-02T13:22:05.402 INFO:teuthology.task.install:Using flavor: default 2024-02-02T13:22:05.407 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-02T13:22:05.408 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-02-02T13:22:05.408 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'tag': None, 'wait_for_package': False} 2024-02-02T13:22:05.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T13:22:05.410 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'tag': None, 'wait_for_package': False} 2024-02-02T13:22:05.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T13:22:05.558 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/ 2024-02-02T13:22:05.559 INFO:teuthology.task.install.rpm:Package version is 18.2.1-325.g0fb32788 2024-02-02T13:22:05.560 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/ 2024-02-02T13:22:05.560 INFO:teuthology.task.install.rpm:Package version is 18.2.1-325.g0fb32788 2024-02-02T13:22:05.671 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-02T13:22:05.671 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:22:05.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-02T13:22:05.727 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-02T13:22:05.728 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:22:05.728 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-02T13:22:05.771 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-02T13:22:05.772 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0fb327883c124fd83155c94db0804f823b805a12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-02T13:22:05.827 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-02T13:22:05.827 DEBUG:teuthology.orchestra.run.smithi091:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0fb327883c124fd83155c94db0804f823b805a12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-02T13:22:05.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-02T13:22:05.962 DEBUG:teuthology.orchestra.run.smithi091:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-02T13:22:06.043 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-02T13:22:06.083 DEBUG:teuthology.orchestra.run.smithi091:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-02T13:22:06.134 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-02-02T13:22:06.140 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-02-02T13:22:06.169 INFO:teuthology.orchestra.run.smithi091.stdout:check_obsoletes = 1 2024-02-02T13:22:06.172 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum clean all 2024-02-02T13:22:06.488 INFO:teuthology.orchestra.run.smithi113.stdout:74 files removed 2024-02-02T13:22:06.520 INFO:teuthology.orchestra.run.smithi091.stdout:74 files removed 2024-02-02T13:22:06.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-02T13:22:06.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-02T13:22:07.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 200 kB/s | 77 kB 00:00 2024-02-02T13:22:07.556 INFO:teuthology.orchestra.run.smithi091.stdout:ceph packages for x86_64 157 kB/s | 77 kB 00:00 2024-02-02T13:22:08.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 19 kB/s | 9.5 kB 00:00 2024-02-02T13:22:08.123 INFO:teuthology.orchestra.run.smithi091.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2024-02-02T13:22:08.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-02-02T13:22:08.782 INFO:teuthology.orchestra.run.smithi091.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-02-02T13:22:09.411 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - AppStream 12 MB/s | 6.6 MB 00:00 2024-02-02T13:22:09.836 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 5.9 MB/s | 6.6 MB 00:01 2024-02-02T13:22:11.119 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2024-02-02T13:22:11.478 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-02-02T13:22:12.370 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-02-02T13:22:12.712 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-02-02T13:22:16.991 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-02-02T13:22:17.215 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-02-02T13:22:21.459 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.9 MB 00:00 2024-02-02T13:22:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 5.9 MB 00:00 2024-02-02T13:22:23.193 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-02T13:22:23.592 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-02-02T13:22:25.295 INFO:teuthology.orchestra.run.smithi091.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-02T13:22:25.924 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 20 MB 00:02 2024-02-02T13:22:30.986 INFO:teuthology.orchestra.run.smithi091.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-02T13:22:31.685 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-02T13:22:33.703 INFO:teuthology.orchestra.run.smithi091.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-02T13:22:33.704 INFO:teuthology.orchestra.run.smithi091.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-02T13:22:33.725 INFO:teuthology.orchestra.run.smithi091.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-02T13:22:33.726 INFO:teuthology.orchestra.run.smithi091.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-02T13:22:33.728 INFO:teuthology.orchestra.run.smithi091.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-02T13:22:33.729 INFO:teuthology.orchestra.run.smithi091.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-02T13:22:33.732 INFO:teuthology.orchestra.run.smithi091.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-02T13:22:33.736 INFO:teuthology.orchestra.run.smithi091.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-02T13:22:33.737 INFO:teuthology.orchestra.run.smithi091.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-02T13:22:33.819 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2024-02-02T13:22:33.842 INFO:teuthology.orchestra.run.smithi091.stdout:=========================================================================================== 2024-02-02T13:22:33.842 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout:=========================================================================================== 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph x86_64 2:18.2.1-325.g0fb32788.el9 ceph 6.5 k 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base x86_64 2:18.2.1-325.g0fb32788.el9 ceph 5.2 M 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 840 k 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-325.g0fb32788.el9 ceph 142 k 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr x86_64 2:18.2.1-325.g0fb32788.el9 ceph 1.4 M 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-cephadm noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 132 k 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-dashboard noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 1.8 M 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 7.4 M 2024-02-02T13:22:33.843 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-rook noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 50 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw x86_64 2:18.2.1-325.g0fb32788.el9 ceph 7.7 M 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test x86_64 2:18.2.1-325.g0fb32788.el9 ceph 40 M 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-volume noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 262 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: cephadm noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 221 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-mirror x86_64 2:18.2.1-325.g0fb32788.el9 ceph 219 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-top noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 18 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel x86_64 2:18.2.1-325.g0fb32788.el9 ceph 31 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 659 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel x86_64 2:18.2.1-325.g0fb32788.el9 ceph 127 k 2024-02-02T13:22:33.844 INFO:teuthology.orchestra.run.smithi091.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cephfs x86_64 2:18.2.1-325.g0fb32788.el9 ceph 162 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rados x86_64 2:18.2.1-325.g0fb32788.el9 ceph 321 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rbd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 299 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rgw x86_64 2:18.2.1-325.g0fb32788.el9 ceph 99 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 86 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.0 M 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 171 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-02T13:22:33.845 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrading: 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager x86_64 1:1.45.90-1.el9 CentOS-BaseOS 2.3 M 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-libnm x86_64 1:1.45.90-1.el9 CentOS-BaseOS 1.8 M 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-team x86_64 1:1.45.90-1.el9 CentOS-BaseOS 42 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-tui x86_64 1:1.45.90-1.el9 CentOS-BaseOS 247 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-02T13:22:33.846 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: librados2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.3 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.0 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout:Installing dependencies: 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common x86_64 2:18.2.1-325.g0fb32788.el9 ceph 18 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-grafana-dashboards noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 23 k 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds x86_64 2:18.2.1-325.g0fb32788.el9 ceph 2.1 M 2024-02-02T13:22:33.847 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-modules-core noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 242 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon x86_64 2:18.2.1-325.g0fb32788.el9 ceph 4.5 M 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 18 M 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-prometheus-alerts noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 15 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux x86_64 2:18.2.1-325.g0fb32788.el9 ceph 25 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-02T13:22:33.848 INFO:teuthology.orchestra.run.smithi091.stdout: libcephsqlite x86_64 2:18.2.1-325.g0fb32788.el9 ceph 165 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 475 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 4.5 M 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-02T13:22:33.849 INFO:teuthology.orchestra.run.smithi091.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 45 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-common x86_64 2:18.2.1-325.g0fb32788.el9 ceph 124 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-02T13:22:33.850 INFO:teuthology.orchestra.run.smithi091.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-02T13:22:33.851 INFO:teuthology.orchestra.run.smithi091.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-02T13:22:33.852 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-02T13:22:33.853 INFO:teuthology.orchestra.run.smithi091.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout:Installing weak dependencies: 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout:=========================================================================================== 2024-02-02T13:22:33.854 INFO:teuthology.orchestra.run.smithi091.stdout:Install 133 Packages 2024-02-02T13:22:33.855 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrade 18 Packages 2024-02-02T13:22:33.855 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:22:33.855 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 201 M 2024-02-02T13:22:33.855 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2024-02-02T13:22:34.415 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-02T13:22:34.417 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-02T13:22:34.438 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-02T13:22:34.440 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-02T13:22:34.441 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-02T13:22:34.443 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-02T13:22:34.445 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-02T13:22:34.449 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-02T13:22:34.450 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-02T13:22:34.529 INFO:teuthology.orchestra.run.smithi091.stdout:(1/151): ceph-18.2.1-325.g0fb32788.el9.x86_64.r 15 kB/s | 6.5 kB 00:00 2024-02-02T13:22:34.533 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================== 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================== 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:18.2.1-325.g0fb32788.el9 ceph 6.5 k 2024-02-02T13:22:34.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:18.2.1-325.g0fb32788.el9 ceph 5.2 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 840 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-325.g0fb32788.el9 ceph 142 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:18.2.1-325.g0fb32788.el9 ceph 1.4 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 132 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 1.8 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 7.4 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 50 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:18.2.1-325.g0fb32788.el9 ceph 7.7 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:18.2.1-325.g0fb32788.el9 ceph 40 M 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 262 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 221 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:18.2.1-325.g0fb32788.el9 ceph 219 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 18 k 2024-02-02T13:22:34.552 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:18.2.1-325.g0fb32788.el9 ceph 31 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 659 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:18.2.1-325.g0fb32788.el9 ceph 127 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-02T13:22:34.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:18.2.1-325.g0fb32788.el9 ceph 162 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:18.2.1-325.g0fb32788.el9 ceph 321 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 299 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:18.2.1-325.g0fb32788.el9 ceph 99 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 86 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.0 M 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 171 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager x86_64 1:1.45.90-1.el9 CentOS-BaseOS 2.3 M 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm x86_64 1:1.45.90-1.el9 CentOS-BaseOS 1.8 M 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team x86_64 1:1.45.90-1.el9 CentOS-BaseOS 42 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui x86_64 1:1.45.90-1.el9 CentOS-BaseOS 247 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-02T13:22:34.554 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.3 M 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 3.0 M 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-02T13:22:34.555 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:18.2.1-325.g0fb32788.el9 ceph 18 M 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 23 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:18.2.1-325.g0fb32788.el9 ceph 2.1 M 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 242 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:18.2.1-325.g0fb32788.el9 ceph 4.5 M 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:18.2.1-325.g0fb32788.el9 ceph 18 M 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:18.2.1-325.g0fb32788.el9 ceph-noarch 15 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:18.2.1-325.g0fb32788.el9 ceph 25 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-02T13:22:34.556 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:18.2.1-325.g0fb32788.el9 ceph 165 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 475 k 2024-02-02T13:22:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:18.2.1-325.g0fb32788.el9 ceph 4.5 M 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-02T13:22:34.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:18.2.1-325.g0fb32788.el9 ceph 45 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:18.2.1-325.g0fb32788.el9 ceph 124 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-02T13:22:34.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-02T13:22:34.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-02T13:22:34.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-02T13:22:34.562 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================== 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout:Install 133 Packages 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 18 Packages 2024-02-02T13:22:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:22:34.564 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 201 M 2024-02-02T13:22:34.564 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-02-02T13:22:34.762 INFO:teuthology.orchestra.run.smithi091.stdout:(2/151): ceph-base-18.2.1-325.g0fb32788.el9.x86 7.7 MB/s | 5.2 MB 00:00 2024-02-02T13:22:34.863 INFO:teuthology.orchestra.run.smithi091.stdout:(3/151): ceph-immutable-object-cache-18.2.1-325 1.4 MB/s | 142 kB 00:00 2024-02-02T13:22:34.955 INFO:teuthology.orchestra.run.smithi091.stdout:(4/151): ceph-fuse-18.2.1-325.g0fb32788.el9.x86 1.9 MB/s | 840 kB 00:00 2024-02-02T13:22:35.096 INFO:teuthology.orchestra.run.smithi091.stdout:(5/151): ceph-mds-18.2.1-325.g0fb32788.el9.x86_ 9.1 MB/s | 2.1 MB 00:00 2024-02-02T13:22:35.388 INFO:teuthology.orchestra.run.smithi091.stdout:(6/151): ceph-mon-18.2.1-325.g0fb32788.el9.x86_ 15 MB/s | 4.5 MB 00:00 2024-02-02T13:22:35.463 INFO:teuthology.orchestra.run.smithi091.stdout:(7/151): ceph-mgr-18.2.1-325.g0fb32788.el9.x86_ 2.8 MB/s | 1.4 MB 00:00 2024-02-02T13:22:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:(1/151): ceph-18.2.1-325.g0fb32788.el9.x86_64.r 12 kB/s | 6.5 kB 00:00 2024-02-02T13:22:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:(2/151): ceph-fuse-18.2.1-325.g0fb32788.el9.x86 855 kB/s | 840 kB 00:00 2024-02-02T13:22:36.730 INFO:teuthology.orchestra.run.smithi091.stdout:(8/151): ceph-osd-18.2.1-325.g0fb32788.el9.x86_ 13 MB/s | 18 MB 00:01 2024-02-02T13:22:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:(3/151): ceph-immutable-object-cache-18.2.1-325 1.1 MB/s | 142 kB 00:00 2024-02-02T13:22:36.797 INFO:teuthology.orchestra.run.smithi091.stdout:(9/151): ceph-selinux-18.2.1-325.g0fb32788.el9. 385 kB/s | 25 kB 00:00 2024-02-02T13:22:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:(4/151): ceph-base-18.2.1-325.g0fb32788.el9.x86 2.8 MB/s | 5.2 MB 00:01 2024-02-02T13:22:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:(5/151): ceph-mgr-18.2.1-325.g0fb32788.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-02-02T13:22:37.922 INFO:teuthology.orchestra.run.smithi091.stdout:(10/151): ceph-common-18.2.1-325.g0fb32788.el9. 4.8 MB/s | 18 MB 00:03 2024-02-02T13:22:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:(6/151): ceph-mon-18.2.1-325.g0fb32788.el9.x86_ 6.7 MB/s | 4.5 MB 00:00 2024-02-02T13:22:38.064 INFO:teuthology.orchestra.run.smithi091.stdout:(11/151): ceph-radosgw-18.2.1-325.g0fb32788.el9 3.0 MB/s | 7.7 MB 00:02 2024-02-02T13:22:38.089 INFO:teuthology.orchestra.run.smithi091.stdout:(12/151): cephfs-mirror-18.2.1-325.g0fb32788.el 1.3 MB/s | 219 kB 00:00 2024-02-02T13:22:38.123 INFO:teuthology.orchestra.run.smithi091.stdout:(13/151): libcephfs-devel-18.2.1-325.g0fb32788. 540 kB/s | 31 kB 00:00 2024-02-02T13:22:38.324 INFO:teuthology.orchestra.run.smithi091.stdout:(14/151): libcephsqlite-18.2.1-325.g0fb32788.el 829 kB/s | 165 kB 00:00 2024-02-02T13:22:38.424 INFO:teuthology.orchestra.run.smithi091.stdout:(15/151): libcephfs2-18.2.1-325.g0fb32788.el9.x 1.9 MB/s | 659 kB 00:00 2024-02-02T13:22:38.457 INFO:teuthology.orchestra.run.smithi091.stdout:(16/151): librados-devel-18.2.1-325.g0fb32788.e 956 kB/s | 127 kB 00:00 2024-02-02T13:22:38.816 INFO:teuthology.orchestra.run.smithi091.stdout:(17/151): libradosstriper1-18.2.1-325.g0fb32788 1.2 MB/s | 475 kB 00:00 2024-02-02T13:22:38.924 INFO:teuthology.orchestra.run.smithi091.stdout:(18/151): python3-ceph-argparse-18.2.1-325.g0fb 419 kB/s | 45 kB 00:00 2024-02-02T13:22:39.108 INFO:teuthology.orchestra.run.smithi091.stdout:(19/151): python3-ceph-common-18.2.1-325.g0fb32 682 kB/s | 124 kB 00:00 2024-02-02T13:22:39.225 INFO:teuthology.orchestra.run.smithi091.stdout:(20/151): python3-cephfs-18.2.1-325.g0fb32788.e 1.4 MB/s | 162 kB 00:00 2024-02-02T13:22:39.375 INFO:teuthology.orchestra.run.smithi091.stdout:(21/151): python3-rados-18.2.1-325.g0fb32788.el 2.1 MB/s | 321 kB 00:00 2024-02-02T13:22:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:(7/151): ceph-mds-18.2.1-325.g0fb32788.el9.x86_ 805 kB/s | 2.1 MB 00:02 2024-02-02T13:22:39.509 INFO:teuthology.orchestra.run.smithi091.stdout:(22/151): python3-rbd-18.2.1-325.g0fb32788.el9. 2.2 MB/s | 299 kB 00:00 2024-02-02T13:22:39.609 INFO:teuthology.orchestra.run.smithi091.stdout:(23/151): python3-rgw-18.2.1-325.g0fb32788.el9. 1.0 MB/s | 99 kB 00:00 2024-02-02T13:22:39.676 INFO:teuthology.orchestra.run.smithi091.stdout:(24/151): rbd-fuse-18.2.1-325.g0fb32788.el9.x86 1.3 MB/s | 86 kB 00:00 2024-02-02T13:22:39.738 INFO:teuthology.orchestra.run.smithi113.stdout:(8/151): ceph-osd-18.2.1-325.g0fb32788.el9.x86_ 10 MB/s | 18 MB 00:01 2024-02-02T13:22:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:(9/151): ceph-selinux-18.2.1-325.g0fb32788.el9. 440 kB/s | 25 kB 00:00 2024-02-02T13:22:40.309 INFO:teuthology.orchestra.run.smithi091.stdout:(25/151): ceph-test-18.2.1-325.g0fb32788.el9.x8 11 MB/s | 40 MB 00:03 2024-02-02T13:22:40.410 INFO:teuthology.orchestra.run.smithi091.stdout:(26/151): librgw2-18.2.1-325.g0fb32788.el9.x86_ 2.3 MB/s | 4.5 MB 00:01 2024-02-02T13:22:40.443 INFO:teuthology.orchestra.run.smithi091.stdout:(27/151): rbd-nbd-18.2.1-325.g0fb32788.el9.x86_ 1.3 MB/s | 171 kB 00:00 2024-02-02T13:22:40.485 INFO:teuthology.orchestra.run.smithi091.stdout:(28/151): ceph-grafana-dashboards-18.2.1-325.g0 312 kB/s | 23 kB 00:00 2024-02-02T13:22:40.519 INFO:teuthology.orchestra.run.smithi091.stdout:(29/151): ceph-mgr-cephadm-18.2.1-325.g0fb32788 1.7 MB/s | 132 kB 00:00 2024-02-02T13:22:40.602 INFO:teuthology.orchestra.run.smithi091.stdout:(30/151): rbd-mirror-18.2.1-325.g0fb32788.el9.x 3.2 MB/s | 3.0 MB 00:00 2024-02-02T13:22:40.769 INFO:teuthology.orchestra.run.smithi091.stdout:(31/151): ceph-mgr-modules-core-18.2.1-325.g0fb 1.4 MB/s | 242 kB 00:00 2024-02-02T13:22:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:(32/151): ceph-mgr-rook-18.2.1-325.g0fb32788.el 556 kB/s | 50 kB 00:00 2024-02-02T13:22:40.961 INFO:teuthology.orchestra.run.smithi091.stdout:(33/151): ceph-prometheus-alerts-18.2.1-325.g0f 148 kB/s | 15 kB 00:00 2024-02-02T13:22:41.136 INFO:teuthology.orchestra.run.smithi091.stdout:(34/151): ceph-volume-18.2.1-325.g0fb32788.el9. 1.5 MB/s | 262 kB 00:00 2024-02-02T13:22:41.245 INFO:teuthology.orchestra.run.smithi091.stdout:(35/151): cephadm-18.2.1-325.g0fb32788.el9.noar 2.0 MB/s | 221 kB 00:00 2024-02-02T13:22:41.295 INFO:teuthology.orchestra.run.smithi091.stdout:(36/151): ceph-mgr-dashboard-18.2.1-325.g0fb327 2.2 MB/s | 1.8 MB 00:00 2024-02-02T13:22:41.504 INFO:teuthology.orchestra.run.smithi091.stdout:(37/151): ceph-mgr-diskprediction-local-18.2.1- 7.5 MB/s | 7.4 MB 00:00 2024-02-02T13:22:41.537 INFO:teuthology.orchestra.run.smithi091.stdout:(38/151): cephfs-top-18.2.1-325.g0fb32788.el9.n 62 kB/s | 18 kB 00:00 2024-02-02T13:22:41.721 INFO:teuthology.orchestra.run.smithi091.stdout:(39/151): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-02-02T13:22:41.755 INFO:teuthology.orchestra.run.smithi091.stdout:(40/151): boost-program-options-1.75.0-8.el9.x8 489 kB/s | 106 kB 00:00 2024-02-02T13:22:41.904 INFO:teuthology.orchestra.run.smithi091.stdout:(41/151): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 920 kB 00:00 2024-02-02T13:22:41.956 INFO:teuthology.orchestra.run.smithi091.stdout:(42/151): elfutils-libelf-devel-0.190-2.el9.x86 88 kB/s | 20 kB 00:00 2024-02-02T13:22:41.989 INFO:teuthology.orchestra.run.smithi091.stdout:(43/151): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 307 kB 00:00 2024-02-02T13:22:42.023 INFO:teuthology.orchestra.run.smithi091.stdout:(44/151): flexiblas-3.0.4-8.el9.x86_64.rpm 500 kB/s | 32 kB 00:00 2024-02-02T13:22:42.181 INFO:teuthology.orchestra.run.smithi091.stdout:(45/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-02-02T13:22:42.206 INFO:teuthology.orchestra.run.smithi091.stdout:(46/151): flexiblas-openblas-openmp-3.0.4-8.el9 79 kB/s | 17 kB 00:00 2024-02-02T13:22:42.231 INFO:teuthology.orchestra.run.smithi091.stdout:(47/151): libaio-devel-0.3.111-13.el9.x86_64.rp 50 kB/s | 10 kB 00:00 2024-02-02T13:22:42.265 INFO:teuthology.orchestra.run.smithi091.stdout:(48/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.4 MB/s | 44 kB 00:00 2024-02-02T13:22:42.290 INFO:teuthology.orchestra.run.smithi091.stdout:(49/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-02T13:22:42.346 INFO:teuthology.orchestra.run.smithi091.stdout:(50/151): librdkafka-1.6.1-102.el9.x86_64.rpm 5.7 MB/s | 661 kB 00:00 2024-02-02T13:22:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:(10/151): ceph-test-18.2.1-325.g0fb32788.el9.x8 15 MB/s | 40 MB 00:02 2024-02-02T13:22:42.391 INFO:teuthology.orchestra.run.smithi091.stdout:(51/151): libtool-2.4.6-45.el9.x86_64.rpm 4.7 MB/s | 578 kB 00:00 2024-02-02T13:22:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:(11/151): cephfs-mirror-18.2.1-325.g0fb32788.el 4.4 MB/s | 219 kB 00:00 2024-02-02T13:22:42.408 INFO:teuthology.orchestra.run.smithi091.stdout:(52/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 441 kB/s | 24 kB 00:00 2024-02-02T13:22:42.424 INFO:teuthology.orchestra.run.smithi091.stdout:(53/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 643 kB/s | 46 kB 00:00 2024-02-02T13:22:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:(12/151): libcephfs-devel-18.2.1-325.g0fb32788. 632 kB/s | 31 kB 00:00 2024-02-02T13:22:42.458 INFO:teuthology.orchestra.run.smithi091.stdout:(54/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-02-02T13:22:42.483 INFO:teuthology.orchestra.run.smithi091.stdout:(55/151): openblas-0.3.21-2.el9.x86_64.rpm 618 kB/s | 34 kB 00:00 2024-02-02T13:22:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:(13/151): libcephfs2-18.2.1-325.g0fb32788.el9.x 7.1 MB/s | 659 kB 00:00 2024-02-02T13:22:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:(14/151): libcephsqlite-18.2.1-325.g0fb32788.el 3.3 MB/s | 165 kB 00:00 2024-02-02T13:22:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:(15/151): librados-devel-18.2.1-325.g0fb32788.e 2.5 MB/s | 127 kB 00:00 2024-02-02T13:22:42.684 INFO:teuthology.orchestra.run.smithi091.stdout:(56/151): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-02-02T13:22:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:(16/151): libradosstriper1-18.2.1-325.g0fb32788 6.2 MB/s | 475 kB 00:00 2024-02-02T13:22:42.801 INFO:teuthology.orchestra.run.smithi091.stdout:(57/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2024-02-02T13:22:42.859 INFO:teuthology.orchestra.run.smithi091.stdout:(58/151): postgresql-13.11-1.el9.x86_64.rpm 4.1 MB/s | 1.5 MB 00:00 2024-02-02T13:22:42.893 INFO:teuthology.orchestra.run.smithi091.stdout:(59/151): postgresql-contrib-13.11-1.el9.x86_64 4.1 MB/s | 811 kB 00:00 2024-02-02T13:22:42.918 INFO:teuthology.orchestra.run.smithi091.stdout:(60/151): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-02-02T13:22:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:(17/151): librgw2-18.2.1-325.g0fb32788.el9.x86_ 15 MB/s | 4.5 MB 00:00 2024-02-02T13:22:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:(18/151): python3-ceph-argparse-18.2.1-325.g0fb 911 kB/s | 45 kB 00:00 2024-02-02T13:22:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:(19/151): python3-ceph-common-18.2.1-325.g0fb32 2.5 MB/s | 124 kB 00:00 2024-02-02T13:22:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:(20/151): python3-cephfs-18.2.1-325.g0fb32788.e 3.2 MB/s | 162 kB 00:00 2024-02-02T13:22:43.168 INFO:teuthology.orchestra.run.smithi091.stdout:(61/151): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-02-02T13:22:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:(21/151): python3-rados-18.2.1-325.g0fb32788.el 5.4 MB/s | 321 kB 00:00 2024-02-02T13:22:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:(22/151): python3-rbd-18.2.1-325.g0fb32788.el9. 5.1 MB/s | 299 kB 00:00 2024-02-02T13:22:43.285 INFO:teuthology.orchestra.run.smithi091.stdout:(62/151): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-02-02T13:22:43.310 INFO:teuthology.orchestra.run.smithi091.stdout:(63/151): python3-jinja2-2.11.3-4.el9.noarch.rp 585 kB/s | 228 kB 00:00 2024-02-02T13:22:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:(23/151): python3-rgw-18.2.1-325.g0fb32788.el9. 2.0 MB/s | 99 kB 00:00 2024-02-02T13:22:43.335 INFO:teuthology.orchestra.run.smithi091.stdout:(64/151): python3-jsonpatch-1.21-16.el9.noarch. 158 kB/s | 24 kB 00:00 2024-02-02T13:22:43.361 INFO:teuthology.orchestra.run.smithi091.stdout:(65/151): python3-jsonpointer-2.0-4.el9.noarch. 370 kB/s | 18 kB 00:00 2024-02-02T13:22:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:(24/151): rbd-fuse-18.2.1-325.g0fb32788.el9.x86 1.7 MB/s | 86 kB 00:00 2024-02-02T13:22:43.386 INFO:teuthology.orchestra.run.smithi091.stdout:(66/151): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-02-02T13:22:43.411 INFO:teuthology.orchestra.run.smithi091.stdout:(67/151): python3-markupsafe-1.1.1-12.el9.x86_6 424 kB/s | 32 kB 00:00 2024-02-02T13:22:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:(68/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.4 MB/s | 217 kB 00:00 2024-02-02T13:22:43.587 INFO:teuthology.orchestra.run.smithi091.stdout:(69/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-02-02T13:22:43.612 INFO:teuthology.orchestra.run.smithi091.stdout:(70/151): python3-oauthlib-3.1.1-5.el9.noarch.r 885 kB/s | 174 kB 00:00 2024-02-02T13:22:43.637 INFO:teuthology.orchestra.run.smithi091.stdout:(71/151): python3-prettytable-0.7.2-27.el9.noar 223 kB/s | 41 kB 00:00 2024-02-02T13:22:43.679 INFO:teuthology.orchestra.run.smithi091.stdout:(72/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-02-02T13:22:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:(25/151): ceph-common-18.2.1-325.g0fb32788.el9. 2.1 MB/s | 18 MB 00:08 2024-02-02T13:22:43.712 INFO:teuthology.orchestra.run.smithi091.stdout:(73/151): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-02-02T13:22:43.738 INFO:teuthology.orchestra.run.smithi091.stdout:(74/151): python3-pytz-2021.1-5.el9.noarch.rpm 480 kB/s | 47 kB 00:00 2024-02-02T13:22:43.763 INFO:teuthology.orchestra.run.smithi091.stdout:(75/151): python3-requests-oauthlib-1.3.0-12.el 526 kB/s | 43 kB 00:00 2024-02-02T13:22:43.797 INFO:teuthology.orchestra.run.smithi091.stdout:(76/151): python3-toml-0.10.2-6.el9.noarch.rpm 676 kB/s | 39 kB 00:00 2024-02-02T13:22:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:(26/151): rbd-mirror-18.2.1-325.g0fb32788.el9.x 7.2 MB/s | 3.0 MB 00:00 2024-02-02T13:22:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:(27/151): rbd-nbd-18.2.1-325.g0fb32788.el9.x86_ 1.3 MB/s | 171 kB 00:00 2024-02-02T13:22:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:(28/151): ceph-grafana-dashboards-18.2.1-325.g0 351 kB/s | 23 kB 00:00 2024-02-02T13:22:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:(29/151): ceph-mgr-cephadm-18.2.1-325.g0fb32788 1.6 MB/s | 132 kB 00:00 2024-02-02T13:22:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:(77/151): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-02-02T13:22:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:(78/151): uuid-1.6.2-55.el9.x86_64.rpm 278 kB/s | 56 kB 00:00 2024-02-02T13:22:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:(30/151): ceph-mgr-dashboard-18.2.1-325.g0fb327 11 MB/s | 1.8 MB 00:00 2024-02-02T13:22:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:(31/151): ceph-mgr-modules-core-18.2.1-325.g0fb 4.1 MB/s | 242 kB 00:00 2024-02-02T13:22:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:(32/151): ceph-mgr-rook-18.2.1-325.g0fb32788.el 1.0 MB/s | 50 kB 00:00 2024-02-02T13:22:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:(33/151): ceph-prometheus-alerts-18.2.1-325.g0f 359 kB/s | 15 kB 00:00 2024-02-02T13:22:44.223 INFO:teuthology.orchestra.run.smithi091.stdout:(79/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-02-02T13:22:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:(34/151): ceph-volume-18.2.1-325.g0fb32788.el9. 4.5 MB/s | 262 kB 00:00 2024-02-02T13:22:44.247 INFO:teuthology.orchestra.run.smithi091.stdout:(80/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 337 kB/s | 87 kB 00:00 2024-02-02T13:22:44.273 INFO:teuthology.orchestra.run.smithi091.stdout:(81/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 251 kB/s | 63 kB 00:00 2024-02-02T13:22:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:(35/151): cephadm-18.2.1-325.g0fb32788.el9.noar 3.8 MB/s | 221 kB 00:00 2024-02-02T13:22:44.328 INFO:teuthology.orchestra.run.smithi091.stdout:(82/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 559 kB/s | 44 kB 00:00 2024-02-02T13:22:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:(36/151): cephfs-top-18.2.1-325.g0fb32788.el9.n 444 kB/s | 18 kB 00:00 2024-02-02T13:22:44.357 INFO:teuthology.orchestra.run.smithi091.stdout:(83/151): fuse-2.9.9-15.el9.x86_64.rpm 736 kB/s | 79 kB 00:00 2024-02-02T13:22:44.390 INFO:teuthology.orchestra.run.smithi091.stdout:(84/151): jq-1.6-16.el9.x86_64.rpm 1.6 MB/s | 186 kB 00:00 2024-02-02T13:22:44.424 INFO:teuthology.orchestra.run.smithi091.stdout:(85/151): libquadmath-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 194 kB 00:00 2024-02-02T13:22:44.466 INFO:teuthology.orchestra.run.smithi091.stdout:(86/151): libgfortran-11.4.1-3.el9.x86_64.rpm 7.4 MB/s | 802 kB 00:00 2024-02-02T13:22:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:(87/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 217 kB 00:00 2024-02-02T13:22:44.524 INFO:teuthology.orchestra.run.smithi091.stdout:(88/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 241 kB 00:00 2024-02-02T13:22:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:(37/151): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-02-02T13:22:44.550 INFO:teuthology.orchestra.run.smithi091.stdout:(89/151): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-02-02T13:22:44.600 INFO:teuthology.orchestra.run.smithi091.stdout:(90/151): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-02-02T13:22:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:(38/151): bison-3.7.4-5.el9.x86_64.rpm 9.7 MB/s | 920 kB 00:00 2024-02-02T13:22:44.625 INFO:teuthology.orchestra.run.smithi091.stdout:(91/151): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-02-02T13:22:44.650 INFO:teuthology.orchestra.run.smithi091.stdout:(92/151): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 113 kB 00:00 2024-02-02T13:22:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:(39/151): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 106 kB 00:00 2024-02-02T13:22:44.684 INFO:teuthology.orchestra.run.smithi091.stdout:(93/151): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-02-02T13:22:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:(40/151): elfutils-libelf-devel-0.190-2.el9.x86 628 kB/s | 20 kB 00:00 2024-02-02T13:22:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:(41/151): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2024-02-02T13:22:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:(42/151): flexiblas-3.0.4-8.el9.x86_64.rpm 958 kB/s | 32 kB 00:00 2024-02-02T13:22:44.851 INFO:teuthology.orchestra.run.smithi091.stdout:(94/151): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-02-02T13:22:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:(43/151): ceph-radosgw-18.2.1-325.g0fb32788.el9 1.4 MB/s | 7.7 MB 00:05 2024-02-02T13:22:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:(44/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-02-02T13:22:45.057 INFO:teuthology.orchestra.run.smithi113.stdout:(45/151): flexiblas-openblas-openmp-3.0.4-8.el9 143 kB/s | 17 kB 00:00 2024-02-02T13:22:45.091 INFO:teuthology.orchestra.run.smithi113.stdout:(46/151): libaio-devel-0.3.111-13.el9.x86_64.rp 312 kB/s | 10 kB 00:00 2024-02-02T13:22:45.116 INFO:teuthology.orchestra.run.smithi113.stdout:(47/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-02T13:22:45.141 INFO:teuthology.orchestra.run.smithi113.stdout:(48/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 917 kB/s | 44 kB 00:00 2024-02-02T13:22:45.143 INFO:teuthology.orchestra.run.smithi091.stdout:(95/151): fmt-8.1.1-5.el9.x86_64.rpm 226 kB/s | 111 kB 00:00 2024-02-02T13:22:45.191 INFO:teuthology.orchestra.run.smithi113.stdout:(49/151): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-02-02T13:22:45.225 INFO:teuthology.orchestra.run.smithi113.stdout:(50/151): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-02-02T13:22:45.251 INFO:teuthology.orchestra.run.smithi113.stdout:(51/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 454 kB/s | 24 kB 00:00 2024-02-02T13:22:45.268 INFO:teuthology.orchestra.run.smithi091.stdout:(96/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 530 kB/s | 309 kB 00:00 2024-02-02T13:22:45.276 INFO:teuthology.orchestra.run.smithi113.stdout:(52/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2024-02-02T13:22:45.293 INFO:teuthology.orchestra.run.smithi091.stdout:(97/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 174 kB/s | 26 kB 00:00 2024-02-02T13:22:45.310 INFO:teuthology.orchestra.run.smithi113.stdout:(53/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-02-02T13:22:45.335 INFO:teuthology.orchestra.run.smithi113.stdout:(54/151): openblas-0.3.21-2.el9.x86_64.rpm 589 kB/s | 34 kB 00:00 2024-02-02T13:22:45.452 INFO:teuthology.orchestra.run.smithi091.stdout:(98/151): libunwind-1.6.2-1.el9.x86_64.rpm 429 kB/s | 67 kB 00:00 2024-02-02T13:22:45.477 INFO:teuthology.orchestra.run.smithi091.stdout:(99/151): liboath-2.6.7-2.el9.x86_64.rpm 266 kB/s | 49 kB 00:00 2024-02-02T13:22:45.486 INFO:teuthology.orchestra.run.smithi113.stdout:(55/151): ceph-mgr-diskprediction-local-18.2.1- 4.7 MB/s | 7.4 MB 00:01 2024-02-02T13:22:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:(56/151): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-02-02T13:22:45.719 INFO:teuthology.orchestra.run.smithi113.stdout:(57/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.9 MB/s | 3.0 MB 00:00 2024-02-02T13:22:45.761 INFO:teuthology.orchestra.run.smithi091.stdout:(100/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-02-02T13:22:45.769 INFO:teuthology.orchestra.run.smithi113.stdout:(58/151): postgresql-13.11-1.el9.x86_64.rpm 5.6 MB/s | 1.5 MB 00:00 2024-02-02T13:22:45.803 INFO:teuthology.orchestra.run.smithi113.stdout:(59/151): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 811 kB 00:00 2024-02-02T13:22:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:(60/151): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-02-02T13:22:45.853 INFO:teuthology.orchestra.run.smithi091.stdout:(101/151): python3-asyncssh-2.13.2-3.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-02-02T13:22:45.906 INFO:teuthology.orchestra.run.smithi091.stdout:(102/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 301 kB/s | 43 kB 00:00 2024-02-02T13:22:46.020 INFO:teuthology.orchestra.run.smithi091.stdout:(103/151): libarrow-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-02-02T13:22:46.045 INFO:teuthology.orchestra.run.smithi091.stdout:(104/151): python3-certifi-2021.10.8-2.el9.noar 108 kB/s | 15 kB 00:00 2024-02-02T13:22:46.070 INFO:teuthology.orchestra.run.smithi091.stdout:(105/151): python3-cachetools-4.2.4-1.el9.noarc 148 kB/s | 32 kB 00:00 2024-02-02T13:22:46.129 INFO:teuthology.orchestra.run.smithi091.stdout:(106/151): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-02-02T13:22:46.137 INFO:teuthology.orchestra.run.smithi113.stdout:(61/151): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-02-02T13:22:46.162 INFO:teuthology.orchestra.run.smithi091.stdout:(107/151): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-02-02T13:22:46.179 INFO:teuthology.orchestra.run.smithi113.stdout:(62/151): python3-jinja2-2.11.3-4.el9.noarch.rp 654 kB/s | 228 kB 00:00 2024-02-02T13:22:46.213 INFO:teuthology.orchestra.run.smithi091.stdout:(108/151): python3-google-auth-2.26.2-1.el9.noa 1.4 MB/s | 202 kB 00:00 2024-02-02T13:22:46.230 INFO:teuthology.orchestra.run.smithi091.stdout:(109/151): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-02-02T13:22:46.255 INFO:teuthology.orchestra.run.smithi091.stdout:(110/151): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-02-02T13:22:46.304 INFO:teuthology.orchestra.run.smithi113.stdout:(63/151): postgresql-server-13.11-1.el9.x86_64. 11 MB/s | 5.7 MB 00:00 2024-02-02T13:22:46.322 INFO:teuthology.orchestra.run.smithi091.stdout:(111/151): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-02-02T13:22:46.329 INFO:teuthology.orchestra.run.smithi113.stdout:(64/151): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 24 kB 00:00 2024-02-02T13:22:46.339 INFO:teuthology.orchestra.run.smithi091.stdout:(112/151): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-02-02T13:22:46.354 INFO:teuthology.orchestra.run.smithi113.stdout:(65/151): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2024-02-02T13:22:46.356 INFO:teuthology.orchestra.run.smithi091.stdout:(113/151): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-02-02T13:22:46.371 INFO:teuthology.orchestra.run.smithi113.stdout:(66/151): python3-markupsafe-1.1.1-12.el9.x86_6 780 kB/s | 32 kB 00:00 2024-02-02T13:22:46.397 INFO:teuthology.orchestra.run.smithi113.stdout:(67/151): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-02-02T13:22:46.423 INFO:teuthology.orchestra.run.smithi091.stdout:(114/151): python3-jwt+crypto-2.4.0-1.el9.noarc 91 kB/s | 9.0 kB 00:00 2024-02-02T13:22:46.448 INFO:teuthology.orchestra.run.smithi091.stdout:(115/151): python3-jwt-2.4.0-1.el9.noarch.rpm 378 kB/s | 41 kB 00:00 2024-02-02T13:22:46.549 INFO:teuthology.orchestra.run.smithi091.stdout:(116/151): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-02-02T13:22:46.582 INFO:teuthology.orchestra.run.smithi091.stdout:(117/151): python3-more-itertools-8.12.0-2.el9. 592 kB/s | 79 kB 00:00 2024-02-02T13:22:46.647 INFO:teuthology.orchestra.run.smithi113.stdout:(68/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-02-02T13:22:46.657 INFO:teuthology.orchestra.run.smithi091.stdout:(118/151): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-02-02T13:22:46.672 INFO:teuthology.orchestra.run.smithi113.stdout:(69/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 730 kB/s | 217 kB 00:00 2024-02-02T13:22:46.706 INFO:teuthology.orchestra.run.smithi113.stdout:(70/151): python3-oauthlib-3.1.1-5.el9.noarch.r 585 kB/s | 174 kB 00:00 2024-02-02T13:22:46.716 INFO:teuthology.orchestra.run.smithi091.stdout:(119/151): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-02-02T13:22:46.756 INFO:teuthology.orchestra.run.smithi113.stdout:(71/151): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 41 kB 00:00 2024-02-02T13:22:46.758 INFO:teuthology.orchestra.run.smithi091.stdout:(120/151): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-02-02T13:22:46.783 INFO:teuthology.orchestra.run.smithi091.stdout:(121/151): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-02-02T13:22:46.814 INFO:teuthology.orchestra.run.smithi113.stdout:(72/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-02-02T13:22:46.839 INFO:teuthology.orchestra.run.smithi113.stdout:(73/151): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 211 kB 00:00 2024-02-02T13:22:46.841 INFO:teuthology.orchestra.run.smithi091.stdout:(122/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 720 kB/s | 90 kB 00:00 2024-02-02T13:22:46.865 INFO:teuthology.orchestra.run.smithi113.stdout:(74/151): python3-pytz-2021.1-5.el9.noarch.rpm 446 kB/s | 47 kB 00:00 2024-02-02T13:22:46.867 INFO:teuthology.orchestra.run.smithi091.stdout:(123/151): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-02-02T13:22:46.899 INFO:teuthology.orchestra.run.smithi113.stdout:(75/151): python3-requests-oauthlib-1.3.0-12.el 735 kB/s | 43 kB 00:00 2024-02-02T13:22:46.909 INFO:teuthology.orchestra.run.smithi091.stdout:(124/151): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-02-02T13:22:46.993 INFO:teuthology.orchestra.run.smithi091.stdout:(125/151): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2024-02-02T13:22:47.018 INFO:teuthology.orchestra.run.smithi091.stdout:(126/151): python3-tempora-5.0.0-2.el9.noarch.r 238 kB/s | 36 kB 00:00 2024-02-02T13:22:47.043 INFO:teuthology.orchestra.run.smithi091.stdout:(127/151): python3-typing-extensions-4.4.0-2.el 384 kB/s | 51 kB 00:00 2024-02-02T13:22:47.152 INFO:teuthology.orchestra.run.smithi091.stdout:(128/151): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-02-02T13:22:47.166 INFO:teuthology.orchestra.run.smithi113.stdout:(76/151): python3-toml-0.10.2-6.el9.noarch.rpm 129 kB/s | 39 kB 00:00 2024-02-02T13:22:47.177 INFO:teuthology.orchestra.run.smithi091.stdout:(129/151): python3-websocket-client-1.2.3-2.el9 567 kB/s | 90 kB 00:00 2024-02-02T13:22:47.199 INFO:teuthology.orchestra.run.smithi113.stdout:(77/151): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-02-02T13:22:47.219 INFO:teuthology.orchestra.run.smithi091.stdout:(130/151): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-02-02T13:22:47.261 INFO:teuthology.orchestra.run.smithi091.stdout:(131/151): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-02-02T13:22:47.303 INFO:teuthology.orchestra.run.smithi091.stdout:(132/151): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-02-02T13:22:47.395 INFO:teuthology.orchestra.run.smithi091.stdout:(133/151): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-02-02T13:22:47.433 INFO:teuthology.orchestra.run.smithi113.stdout:(78/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-02-02T13:22:47.436 INFO:teuthology.orchestra.run.smithi091.stdout:(134/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-02-02T13:22:47.458 INFO:teuthology.orchestra.run.smithi113.stdout:(79/151): uuid-1.6.2-55.el9.x86_64.rpm 195 kB/s | 56 kB 00:00 2024-02-02T13:22:47.484 INFO:teuthology.orchestra.run.smithi113.stdout:(80/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 322 kB/s | 87 kB 00:00 2024-02-02T13:22:47.509 INFO:teuthology.orchestra.run.smithi113.stdout:(81/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-02-02T13:22:47.534 INFO:teuthology.orchestra.run.smithi113.stdout:(82/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 601 kB/s | 44 kB 00:00 2024-02-02T13:22:47.545 INFO:teuthology.orchestra.run.smithi091.stdout:(135/151): NetworkManager-1.45.90-1.el9.x86_64. 21 MB/s | 2.3 MB 00:00 2024-02-02T13:22:47.559 INFO:teuthology.orchestra.run.smithi113.stdout:(83/151): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-02-02T13:22:47.593 INFO:teuthology.orchestra.run.smithi113.stdout:(84/151): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2024-02-02T13:22:47.626 INFO:teuthology.orchestra.run.smithi113.stdout:(85/151): libgfortran-11.4.1-3.el9.x86_64.rpm 8.7 MB/s | 802 kB 00:00 2024-02-02T13:22:47.629 INFO:teuthology.orchestra.run.smithi091.stdout:(136/151): NetworkManager-libnm-1.45.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-02-02T13:22:47.652 INFO:teuthology.orchestra.run.smithi113.stdout:(86/151): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 194 kB 00:00 2024-02-02T13:22:47.662 INFO:teuthology.orchestra.run.smithi091.stdout:(137/151): NetworkManager-team-1.45.90-1.el9.x8 1.3 MB/s | 42 kB 00:00 2024-02-02T13:22:47.685 INFO:teuthology.orchestra.run.smithi113.stdout:(87/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-02-02T13:22:47.710 INFO:teuthology.orchestra.run.smithi113.stdout:(88/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-02-02T13:22:47.737 INFO:teuthology.orchestra.run.smithi091.stdout:(138/151): librbd1-18.2.1-325.g0fb32788.el9.x86 6.9 MB/s | 3.0 MB 00:00 2024-02-02T13:22:47.761 INFO:teuthology.orchestra.run.smithi113.stdout:(89/151): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-02-02T13:22:47.780 INFO:teuthology.orchestra.run.smithi091.stdout:(139/151): NetworkManager-tui-1.45.90-1.el9.x86 2.1 MB/s | 247 kB 00:00 2024-02-02T13:22:47.786 INFO:teuthology.orchestra.run.smithi113.stdout:(90/151): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-02-02T13:22:47.805 INFO:teuthology.orchestra.run.smithi091.stdout:(140/151): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 69 kB 00:00 2024-02-02T13:22:47.811 INFO:teuthology.orchestra.run.smithi113.stdout:(91/151): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-02-02T13:22:47.830 INFO:teuthology.orchestra.run.smithi091.stdout:(141/151): elfutils-debuginfod-client-0.190-2.e 723 kB/s | 35 kB 00:00 2024-02-02T13:22:47.837 INFO:teuthology.orchestra.run.smithi113.stdout:(92/151): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 113 kB 00:00 2024-02-02T13:22:47.889 INFO:teuthology.orchestra.run.smithi091.stdout:(142/151): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 191 kB 00:00 2024-02-02T13:22:47.890 INFO:teuthology.orchestra.run.smithi113.stdout:(93/151): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-02-02T13:22:47.922 INFO:teuthology.orchestra.run.smithi091.stdout:(143/151): elfutils-libs-0.190-2.el9.x86_64.rpm 2.8 MB/s | 254 kB 00:00 2024-02-02T13:22:47.973 INFO:teuthology.orchestra.run.smithi091.stdout:(144/151): iproute-6.7.0-2.el9.x86_64.rpm 9.8 MB/s | 802 kB 00:00 2024-02-02T13:22:48.006 INFO:teuthology.orchestra.run.smithi091.stdout:(145/151): iproute-tc-6.7.0-2.el9.x86_64.rpm 5.5 MB/s | 433 kB 00:00 2024-02-02T13:22:48.032 INFO:teuthology.orchestra.run.smithi091.stdout:(146/151): libacl-2.3.1-4.el9.x86_64.rpm 408 kB/s | 22 kB 00:00 2024-02-02T13:22:48.082 INFO:teuthology.orchestra.run.smithi091.stdout:(147/151): openssl-3.0.7-25.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-02-02T13:22:48.104 INFO:teuthology.orchestra.run.smithi113.stdout:(94/151): python3-devel-3.9.17-2.el9.x86_64.rpm 846 kB/s | 245 kB 00:00 2024-02-02T13:22:48.165 INFO:teuthology.orchestra.run.smithi091.stdout:(148/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-02-02T13:22:48.199 INFO:teuthology.orchestra.run.smithi091.stdout:(149/151): xfsdump-3.1.12-4.el9.x86_64.rpm 3.0 MB/s | 321 kB 00:00 2024-02-02T13:22:48.273 INFO:teuthology.orchestra.run.smithi091.stdout:(150/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 10 MB/s | 1.1 MB 00:00 2024-02-02T13:22:48.304 INFO:teuthology.orchestra.run.smithi113.stdout:(95/151): fmt-8.1.1-5.el9.x86_64.rpm 238 kB/s | 111 kB 00:00 2024-02-02T13:22:48.366 INFO:teuthology.orchestra.run.smithi091.stdout:(151/151): librados2-18.2.1-325.g0fb32788.el9.x 3.0 MB/s | 3.3 MB 00:01 2024-02-02T13:22:48.374 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2024-02-02T13:22:48.374 INFO:teuthology.orchestra.run.smithi091.stdout:Total 14 MB/s | 201 MB 00:14 2024-02-02T13:22:48.396 INFO:teuthology.orchestra.run.smithi113.stdout:(96/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 285 kB/s | 26 kB 00:00 2024-02-02T13:22:48.424 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2024-02-02T13:22:48.446 INFO:teuthology.orchestra.run.smithi113.stdout:(97/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 556 kB/s | 309 kB 00:00 2024-02-02T13:22:48.488 INFO:teuthology.orchestra.run.smithi113.stdout:(98/151): liboath-2.6.7-2.el9.x86_64.rpm 532 kB/s | 49 kB 00:00 2024-02-02T13:22:48.530 INFO:teuthology.orchestra.run.smithi113.stdout:(99/151): libunwind-1.6.2-1.el9.x86_64.rpm 812 kB/s | 67 kB 00:00 2024-02-02T13:22:48.681 INFO:teuthology.orchestra.run.smithi113.stdout:(100/151): python3-asyncssh-2.13.2-3.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-02-02T13:22:48.764 INFO:teuthology.orchestra.run.smithi113.stdout:(101/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-02-02T13:22:48.769 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2024-02-02T13:22:48.769 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2024-02-02T13:22:48.790 INFO:teuthology.orchestra.run.smithi113.stdout:(102/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2024-02-02T13:22:48.890 INFO:teuthology.orchestra.run.smithi113.stdout:(103/151): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-02-02T13:22:48.915 INFO:teuthology.orchestra.run.smithi113.stdout:(104/151): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-02-02T13:22:48.932 INFO:teuthology.orchestra.run.smithi113.stdout:(105/151): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-02-02T13:22:48.991 INFO:teuthology.orchestra.run.smithi113.stdout:(106/151): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-02-02T13:22:49.025 INFO:teuthology.orchestra.run.smithi113.stdout:(107/151): python3-google-auth-2.26.2-1.el9.noa 2.2 MB/s | 202 kB 00:00 2024-02-02T13:22:49.058 INFO:teuthology.orchestra.run.smithi113.stdout:(108/151): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-02-02T13:22:49.083 INFO:teuthology.orchestra.run.smithi113.stdout:(109/151): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-02-02T13:22:49.109 INFO:teuthology.orchestra.run.smithi113.stdout:(110/151): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-02-02T13:22:49.134 INFO:teuthology.orchestra.run.smithi113.stdout:(111/151): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2024-02-02T13:22:49.159 INFO:teuthology.orchestra.run.smithi113.stdout:(112/151): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-02-02T13:22:49.185 INFO:teuthology.orchestra.run.smithi113.stdout:(113/151): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-02-02T13:22:49.210 INFO:teuthology.orchestra.run.smithi113.stdout:(114/151): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2024-02-02T13:22:49.235 INFO:teuthology.orchestra.run.smithi113.stdout:(115/151): python3-jwt-2.4.0-1.el9.noarch.rpm 546 kB/s | 41 kB 00:00 2024-02-02T13:22:49.286 INFO:teuthology.orchestra.run.smithi113.stdout:(116/151): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-02-02T13:22:49.311 INFO:teuthology.orchestra.run.smithi113.stdout:(117/151): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-02-02T13:22:49.353 INFO:teuthology.orchestra.run.smithi113.stdout:(118/151): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-02-02T13:22:49.378 INFO:teuthology.orchestra.run.smithi113.stdout:(119/151): python3-natsort-7.1.1-5.el9.noarch.r 632 kB/s | 58 kB 00:00 2024-02-02T13:22:49.412 INFO:teuthology.orchestra.run.smithi113.stdout:(120/151): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-02-02T13:22:49.437 INFO:teuthology.orchestra.run.smithi113.stdout:(121/151): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-02-02T13:22:49.454 INFO:teuthology.orchestra.run.smithi113.stdout:(122/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-02-02T13:22:49.488 INFO:teuthology.orchestra.run.smithi113.stdout:(123/151): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-02-02T13:22:49.513 INFO:teuthology.orchestra.run.smithi113.stdout:(124/151): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-02-02T13:22:49.538 INFO:teuthology.orchestra.run.smithi113.stdout:(125/151): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-02-02T13:22:49.563 INFO:teuthology.orchestra.run.smithi113.stdout:(126/151): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-02-02T13:22:49.589 INFO:teuthology.orchestra.run.smithi113.stdout:(127/151): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-02-02T13:22:49.622 INFO:teuthology.orchestra.run.smithi113.stdout:(128/151): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-02-02T13:22:49.648 INFO:teuthology.orchestra.run.smithi113.stdout:(129/151): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-02-02T13:22:49.681 INFO:teuthology.orchestra.run.smithi113.stdout:(130/151): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-02-02T13:22:49.707 INFO:teuthology.orchestra.run.smithi113.stdout:(131/151): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-02-02T13:22:49.749 INFO:teuthology.orchestra.run.smithi113.stdout:(132/151): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-02T13:22:49.832 INFO:teuthology.orchestra.run.smithi113.stdout:(133/151): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-02T13:22:49.874 INFO:teuthology.orchestra.run.smithi113.stdout:(134/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-02-02T13:22:49.991 INFO:teuthology.orchestra.run.smithi113.stdout:(135/151): NetworkManager-1.45.90-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-02-02T13:22:50.108 INFO:teuthology.orchestra.run.smithi113.stdout:(136/151): librbd1-18.2.1-325.g0fb32788.el9.x86 8.4 MB/s | 3.0 MB 00:00 2024-02-02T13:22:50.167 INFO:teuthology.orchestra.run.smithi113.stdout:(137/151): NetworkManager-libnm-1.45.90-1.el9.x 10 MB/s | 1.8 MB 00:00 2024-02-02T13:22:50.192 INFO:teuthology.orchestra.run.smithi113.stdout:(138/151): NetworkManager-team-1.45.90-1.el9.x8 556 kB/s | 42 kB 00:00 2024-02-02T13:22:50.226 INFO:teuthology.orchestra.run.smithi113.stdout:(139/151): NetworkManager-tui-1.45.90-1.el9.x86 4.2 MB/s | 247 kB 00:00 2024-02-02T13:22:50.251 INFO:teuthology.orchestra.run.smithi113.stdout:(140/151): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-02-02T13:22:50.285 INFO:teuthology.orchestra.run.smithi113.stdout:(141/151): elfutils-debuginfod-client-0.190-2.e 618 kB/s | 35 kB 00:00 2024-02-02T13:22:50.318 INFO:teuthology.orchestra.run.smithi113.stdout:(142/151): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 191 kB 00:00 2024-02-02T13:22:50.344 INFO:teuthology.orchestra.run.smithi113.stdout:(143/151): elfutils-libs-0.190-2.el9.x86_64.rpm 4.4 MB/s | 254 kB 00:00 2024-02-02T13:22:50.386 INFO:teuthology.orchestra.run.smithi113.stdout:(144/151): iproute-6.7.0-2.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-02-02T13:22:50.419 INFO:teuthology.orchestra.run.smithi113.stdout:(145/151): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.3 MB/s | 433 kB 00:00 2024-02-02T13:22:50.445 INFO:teuthology.orchestra.run.smithi113.stdout:(146/151): libacl-2.3.1-4.el9.x86_64.rpm 403 kB/s | 22 kB 00:00 2024-02-02T13:22:50.503 INFO:teuthology.orchestra.run.smithi113.stdout:(147/151): openssl-3.0.7-25.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-02-02T13:22:50.587 INFO:teuthology.orchestra.run.smithi113.stdout:(148/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 16 MB/s | 2.1 MB 00:00 2024-02-02T13:22:50.620 INFO:teuthology.orchestra.run.smithi113.stdout:(149/151): xfsdump-3.1.12-4.el9.x86_64.rpm 3.0 MB/s | 321 kB 00:00 2024-02-02T13:22:50.679 INFO:teuthology.orchestra.run.smithi113.stdout:(150/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-02T13:22:50.870 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2024-02-02T13:22:50.871 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2024-02-02T13:22:51.171 INFO:teuthology.orchestra.run.smithi113.stdout:(151/151): librados2-18.2.1-325.g0fb32788.el9.x 2.2 MB/s | 3.3 MB 00:01 2024-02-02T13:22:51.185 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-02-02T13:22:51.185 INFO:teuthology.orchestra.run.smithi113.stdout:Total 12 MB/s | 201 MB 00:16 2024-02-02T13:22:51.236 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-02-02T13:22:51.580 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-02-02T13:22:51.581 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-02-02T13:22:53.706 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-02-02T13:22:53.706 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-02-02T13:22:57.549 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2024-02-02T13:22:57.692 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/169 2024-02-02T13:22:57.766 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/169 2024-02-02T13:22:57.850 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 3/169 2024-02-02T13:22:57.935 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/169 2024-02-02T13:22:57.999 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/169 2024-02-02T13:22:58.042 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/169 2024-02-02T13:22:58.147 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/169 2024-02-02T13:22:58.569 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/169 2024-02-02T13:22:58.630 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : librados2-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:22:58.782 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librados2-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:22:58.815 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:22:58.923 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:22:58.961 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 11/169 2024-02-02T13:22:59.122 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/169 2024-02-02T13:22:59.379 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/169 2024-02-02T13:22:59.437 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/169 2024-02-02T13:22:59.462 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:22:59.603 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:22:59.638 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 16/169 2024-02-02T13:22:59.721 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 16/169 2024-02-02T13:23:00.051 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-02-02T13:23:00.074 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/169 2024-02-02T13:23:00.224 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/169 2024-02-02T13:23:00.290 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-libnm-1:1.45.90-1.el9.x86_64 18/169 2024-02-02T13:23:00.381 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.90-1.el9.x86_64 18/169 2024-02-02T13:23:00.468 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/169 2024-02-02T13:23:00.575 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 3/169 2024-02-02T13:23:00.588 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:00.638 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:00.662 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/169 2024-02-02T13:23:00.722 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/169 2024-02-02T13:23:00.756 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/169 2024-02-02T13:23:00.815 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/169 2024-02-02T13:23:01.204 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/169 2024-02-02T13:23:01.244 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:23:01.280 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:01.359 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/169 2024-02-02T13:23:01.395 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:23:01.420 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:23:01.463 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/169 2024-02-02T13:23:01.535 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/169 2024-02-02T13:23:01.574 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/169 2024-02-02T13:23:01.584 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:23:01.626 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 11/169 2024-02-02T13:23:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/169 2024-02-02T13:23:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/169 2024-02-02T13:23:01.761 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/169 2024-02-02T13:23:01.794 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/169 2024-02-02T13:23:01.855 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/169 2024-02-02T13:23:01.925 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/169 2024-02-02T13:23:02.053 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/169 2024-02-02T13:23:02.074 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/169 2024-02-02T13:23:02.087 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x 30/169 2024-02-02T13:23:02.127 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/169 2024-02-02T13:23:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9 31/169 2024-02-02T13:23:02.152 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:23:02.254 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 32/169 2024-02-02T13:23:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:23:02.328 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 16/169 2024-02-02T13:23:02.434 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/169 2024-02-02T13:23:02.445 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 16/169 2024-02-02T13:23:02.504 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/169 2024-02-02T13:23:02.763 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/169 2024-02-02T13:23:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-libnm-1:1.45.90-1.el9.x86_64 18/169 2024-02-02T13:23:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.90-1.el9.x86_64 18/169 2024-02-02T13:23:03.146 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/169 2024-02-02T13:23:03.230 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:03.243 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/169 2024-02-02T13:23:03.287 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:03.292 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/169 2024-02-02T13:23:03.605 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 38/169 2024-02-02T13:23:03.651 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/169 2024-02-02T13:23:03.728 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.90-1.el9.x86_64 19/169 2024-02-02T13:23:03.777 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/169 2024-02-02T13:23:03.848 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 40/169 2024-02-02T13:23:03.886 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/169 2024-02-02T13:23:03.918 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 41/169 2024-02-02T13:23:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/169 2024-02-02T13:23:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/169 2024-02-02T13:23:04.052 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/169 2024-02-02T13:23:04.106 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/169 2024-02-02T13:23:04.111 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 42/169 2024-02-02T13:23:04.169 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 43/169 2024-02-02T13:23:04.196 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/169 2024-02-02T13:23:04.240 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/169 2024-02-02T13:23:04.281 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/169 2024-02-02T13:23:04.312 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : re2-1:20211101-3.el9.x86_64 44/169 2024-02-02T13:23:04.364 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 45/169 2024-02-02T13:23:04.375 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/169 2024-02-02T13:23:04.419 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x 30/169 2024-02-02T13:23:04.433 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/169 2024-02-02T13:23:04.468 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9 31/169 2024-02-02T13:23:04.524 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 47/169 2024-02-02T13:23:04.563 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 48/169 2024-02-02T13:23:04.569 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 32/169 2024-02-02T13:23:04.603 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 49/169 2024-02-02T13:23:04.668 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 50/169 2024-02-02T13:23:04.741 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/169 2024-02-02T13:23:04.752 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 51/169 2024-02-02T13:23:04.803 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/169 2024-02-02T13:23:04.813 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/169 2024-02-02T13:23:05.008 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/169 2024-02-02T13:23:05.063 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/169 2024-02-02T13:23:05.109 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/169 2024-02-02T13:23:05.249 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/169 2024-02-02T13:23:05.286 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 57/169 2024-02-02T13:23:05.332 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/169 2024-02-02T13:23:05.369 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/169 2024-02-02T13:23:05.408 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/169 2024-02-02T13:23:05.463 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/169 2024-02-02T13:23:05.559 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/169 2024-02-02T13:23:05.573 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : fmt-8.1.1-5.el9.x86_64 61/169 2024-02-02T13:23:05.625 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/169 2024-02-02T13:23:05.658 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/169 2024-02-02T13:23:05.698 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 38/169 2024-02-02T13:23:05.705 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/169 2024-02-02T13:23:05.750 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/169 2024-02-02T13:23:05.784 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/169 2024-02-02T13:23:05.922 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 40/169 2024-02-02T13:23:05.997 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 41/169 2024-02-02T13:23:06.032 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/169 2024-02-02T13:23:06.086 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/169 2024-02-02T13:23:06.181 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 42/169 2024-02-02T13:23:06.233 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 43/169 2024-02-02T13:23:06.267 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/169 2024-02-02T13:23:06.339 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/169 2024-02-02T13:23:06.364 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/169 2024-02-02T13:23:06.376 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 44/169 2024-02-02T13:23:06.425 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 45/169 2024-02-02T13:23:06.432 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/169 2024-02-02T13:23:06.485 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/169 2024-02-02T13:23:06.489 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/169 2024-02-02T13:23:06.532 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 72/169 2024-02-02T13:23:06.584 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 47/169 2024-02-02T13:23:06.585 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : jq-1.6-16.el9.x86_64 73/169 2024-02-02T13:23:06.625 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 48/169 2024-02-02T13:23:06.631 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : fuse-2.9.9-15.el9.x86_64 74/169 2024-02-02T13:23:06.663 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 49/169 2024-02-02T13:23:06.708 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 75/169 2024-02-02T13:23:06.731 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 50/169 2024-02-02T13:23:06.771 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 76/169 2024-02-02T13:23:06.787 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 51/169 2024-02-02T13:23:06.839 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : uuid-1.6.2-55.el9.x86_64 77/169 2024-02-02T13:23:06.847 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/169 2024-02-02T13:23:06.888 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 78/169 2024-02-02T13:23:07.019 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 79/169 2024-02-02T13:23:07.037 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/169 2024-02-02T13:23:07.089 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/169 2024-02-02T13:23:07.107 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 80/169 2024-02-02T13:23:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/169 2024-02-02T13:23:07.153 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 81/169 2024-02-02T13:23:07.222 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 82/169 2024-02-02T13:23:07.260 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/169 2024-02-02T13:23:07.263 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 83/169 2024-02-02T13:23:07.306 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 57/169 2024-02-02T13:23:07.356 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/169 2024-02-02T13:23:07.389 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/169 2024-02-02T13:23:07.408 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 84/169 2024-02-02T13:23:07.427 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/169 2024-02-02T13:23:07.452 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 85/169 2024-02-02T13:23:07.621 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fmt-8.1.1-5.el9.x86_64 61/169 2024-02-02T13:23:07.626 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 86/169 2024-02-02T13:23:07.679 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/169 2024-02-02T13:23:07.690 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/169 2024-02-02T13:23:07.758 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/169 2024-02-02T13:23:07.839 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/169 2024-02-02T13:23:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/169 2024-02-02T13:23:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/169 2024-02-02T13:23:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/169 2024-02-02T13:23:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/169 2024-02-02T13:23:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/169 2024-02-02T13:23:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/169 2024-02-02T13:23:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/169 2024-02-02T13:23:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 72/169 2024-02-02T13:23:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-16.el9.x86_64 73/169 2024-02-02T13:23:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 74/169 2024-02-02T13:23:08.664 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/169 2024-02-02T13:23:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 75/169 2024-02-02T13:23:08.723 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/169 2024-02-02T13:23:08.757 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/169 2024-02-02T13:23:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 76/169 2024-02-02T13:23:08.789 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/169 2024-02-02T13:23:08.831 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 77/169 2024-02-02T13:23:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 78/169 2024-02-02T13:23:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 79/169 2024-02-02T13:23:09.030 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openblas-0.3.21-2.el9.x86_64 92/169 2024-02-02T13:23:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 80/169 2024-02-02T13:23:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 93/169 2024-02-02T13:23:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 81/169 2024-02-02T13:23:09.159 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 82/169 2024-02-02T13:23:09.188 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/169 2024-02-02T13:23:09.198 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 83/169 2024-02-02T13:23:09.342 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 84/169 2024-02-02T13:23:09.386 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 85/169 2024-02-02T13:23:09.523 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 86/169 2024-02-02T13:23:09.592 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/169 2024-02-02T13:23:09.983 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/169 2024-02-02T13:23:10.121 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/169 2024-02-02T13:23:10.585 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/169 2024-02-02T13:23:10.633 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/169 2024-02-02T13:23:10.667 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/169 2024-02-02T13:23:10.700 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/169 2024-02-02T13:23:10.964 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 92/169 2024-02-02T13:23:10.996 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 93/169 2024-02-02T13:23:11.093 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/169 2024-02-02T13:23:11.630 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/169 2024-02-02T13:23:11.716 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/169 2024-02-02T13:23:11.754 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/169 2024-02-02T13:23:11.787 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/169 2024-02-02T13:23:11.892 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/169 2024-02-02T13:23:12.025 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/169 2024-02-02T13:23:12.129 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/169 2024-02-02T13:23:12.155 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 102/169 2024-02-02T13:23:12.307 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 102/169 2024-02-02T13:23:12.336 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 103/169 2024-02-02T13:23:12.390 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/169 2024-02-02T13:23:12.831 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 105/169 2024-02-02T13:23:12.857 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 106/169 2024-02-02T13:23:12.948 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 106/169 2024-02-02T13:23:12.976 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 107/169 2024-02-02T13:23:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/169 2024-02-02T13:23:13.628 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/169 2024-02-02T13:23:13.664 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/169 2024-02-02T13:23:13.698 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/169 2024-02-02T13:23:14.037 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/169 2024-02-02T13:23:14.075 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 102/169 2024-02-02T13:23:14.194 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 102/169 2024-02-02T13:23:14.222 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 103/169 2024-02-02T13:23:14.277 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/169 2024-02-02T13:23:14.727 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 105/169 2024-02-02T13:23:14.753 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 106/169 2024-02-02T13:23:14.885 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 106/169 2024-02-02T13:23:14.913 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 107/169 2024-02-02T13:23:15.365 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:15.431 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:16.051 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:16.100 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 109/169 2024-02-02T13:23:16.307 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 109/169 2024-02-02T13:23:16.308 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-02T13:23:16.308 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-02T13:23:16.308 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:16.351 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 110/169 2024-02-02T13:23:17.331 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:17.368 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:17.990 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 108/169 2024-02-02T13:23:18.030 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 109/169 2024-02-02T13:23:18.174 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 109/169 2024-02-02T13:23:18.174 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-02T13:23:18.174 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-02T13:23:18.175 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:18.214 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 110/169 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 110/169 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /sys 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /proc 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /mnt 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /var/tmp 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /home 2024-02-02T13:23:28.893 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /root 2024-02-02T13:23:28.894 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /tmp 2024-02-02T13:23:28.894 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:29.112 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 111/169 2024-02-02T13:23:29.161 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 111/169 2024-02-02T13:23:29.161 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:29.161 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-02T13:23:29.161 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 112/169 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 110/169 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-02-02T13:23:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 112/169 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-02T13:23:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:31.181 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 111/169 2024-02-02T13:23:31.199 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 113/169 2024-02-02T13:23:31.227 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 111/169 2024-02-02T13:23:31.227 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:31.227 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-02T13:23:31.228 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 113/169 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-02T13:23:31.267 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:31.758 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 114/169 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 114/169 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-02T13:23:31.825 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:31.952 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : automake-1.16.2-8.el9.noarch 115/169 2024-02-02T13:23:32.565 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: cephadm-2:18.2.1-325.g0fb32788.el9.noarch 116/169 2024-02-02T13:23:32.603 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephadm-2:18.2.1-325.g0fb32788.el9.noarch 116/169 2024-02-02T13:23:32.657 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 117/169 2024-02-02T13:23:32.761 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 118/169 2024-02-02T13:23:32.790 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 119/169 2024-02-02T13:23:32.900 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 112/169 2024-02-02T13:23:32.974 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 112/169 2024-02-02T13:23:32.974 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:32.974 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-02T13:23:32.975 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-02T13:23:32.975 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-02T13:23:32.975 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:33.107 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 119/169 2024-02-02T13:23:33.149 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 120/169 2024-02-02T13:23:33.200 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 113/169 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 113/169 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-02T13:23:33.268 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 114/169 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 114/169 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-02T13:23:33.865 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:33.986 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 115/169 2024-02-02T13:23:34.233 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 120/169 2024-02-02T13:23:34.266 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 121/169 2024-02-02T13:23:34.406 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 121/169 2024-02-02T13:23:34.530 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:18.2.1-325.g0fb32788.el9.noarch 116/169 2024-02-02T13:23:34.563 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:18.2.1-325.g0fb32788.el9.noarch 116/169 2024-02-02T13:23:34.593 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9 122/169 2024-02-02T13:23:34.610 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 117/169 2024-02-02T13:23:34.627 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 123/169 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 123/169 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-02T13:23:34.699 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:34.724 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 118/169 2024-02-02T13:23:34.757 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 119/169 2024-02-02T13:23:34.761 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 124/169 2024-02-02T13:23:34.799 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 124/169 2024-02-02T13:23:34.905 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-2:18.2.1-325.g0fb32788.el9.x86_64 125/169 2024-02-02T13:23:35.081 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 119/169 2024-02-02T13:23:35.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 120/169 2024-02-02T13:23:35.680 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/169 2024-02-02T13:23:35.712 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 127/169 2024-02-02T13:23:35.778 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 127/169 2024-02-02T13:23:35.778 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:35.779 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-02T13:23:35.779 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-02T13:23:35.779 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-02T13:23:35.779 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:35.821 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 128/169 2024-02-02T13:23:35.890 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-325.g0fb327 128/169 2024-02-02T13:23:35.890 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:35.890 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-02T13:23:35.890 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:35.956 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 129/169 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 129/169 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-02T13:23:36.044 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:36.222 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 120/169 2024-02-02T13:23:36.250 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 121/169 2024-02-02T13:23:36.388 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 121/169 2024-02-02T13:23:36.414 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 130/169 2024-02-02T13:23:36.493 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 130/169 2024-02-02T13:23:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-02T13:23:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-02T13:23:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-02T13:23:36.494 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:36.568 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9 122/169 2024-02-02T13:23:36.602 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 123/169 2024-02-02T13:23:36.667 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 123/169 2024-02-02T13:23:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-02T13:23:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-02T13:23:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-02T13:23:36.668 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:36.720 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 124/169 2024-02-02T13:23:36.758 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 124/169 2024-02-02T13:23:36.854 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:18.2.1-325.g0fb32788.el9.x86_64 125/169 2024-02-02T13:23:37.629 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/169 2024-02-02T13:23:37.672 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 127/169 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 127/169 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-02T13:23:37.740 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:37.797 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 128/169 2024-02-02T13:23:37.887 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-325.g0fb327 128/169 2024-02-02T13:23:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-02T13:23:37.887 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:37.965 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 129/169 2024-02-02T13:23:38.069 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 129/169 2024-02-02T13:23:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-02T13:23:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-02T13:23:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-02T13:23:38.070 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:38.424 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 130/169 2024-02-02T13:23:38.511 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 130/169 2024-02-02T13:23:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-02T13:23:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-02T13:23:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-02T13:23:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-02T13:23:38.512 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:40.478 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 131/169 2024-02-02T13:23:40.534 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 132/169 2024-02-02T13:23:40.588 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 133/169 2024-02-02T13:23:40.626 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 134/169 2024-02-02T13:23:40.886 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 135/169 2024-02-02T13:23:41.008 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/169 2024-02-02T13:23:41.100 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 137/169 2024-02-02T13:23:41.182 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 138/169 2024-02-02T13:23:41.227 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 139/169 2024-02-02T13:23:41.256 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_6 140/169 2024-02-02T13:23:41.305 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-team-1:1.45.90-1.el9.x86_64 141/169 2024-02-02T13:23:41.418 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-tui-1:1.45.90-1.el9.x86_64 142/169 2024-02-02T13:23:41.480 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/169 2024-02-02T13:23:41.551 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/169 2024-02-02T13:23:42.440 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:42.486 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 131/169 2024-02-02T13:23:42.490 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:42.533 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 132/169 2024-02-02T13:23:42.561 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:42.588 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 133/169 2024-02-02T13:23:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/169 2024-02-02T13:23:42.636 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 134/169 2024-02-02T13:23:42.924 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 135/169 2024-02-02T13:23:43.043 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/169 2024-02-02T13:23:43.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 137/169 2024-02-02T13:23:43.175 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 138/169 2024-02-02T13:23:43.211 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 139/169 2024-02-02T13:23:43.249 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_6 140/169 2024-02-02T13:23:43.296 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-team-1:1.45.90-1.el9.x86_64 141/169 2024-02-02T13:23:43.434 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-tui-1:1.45.90-1.el9.x86_64 142/169 2024-02-02T13:23:43.511 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/169 2024-02-02T13:23:43.553 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/169 2024-02-02T13:23:43.953 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 147/169 2024-02-02T13:23:44.038 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/169 2024-02-02T13:23:44.090 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/169 2024-02-02T13:23:44.214 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/169 2024-02-02T13:23:44.276 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/169 2024-02-02T13:23:44.316 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/169 2024-02-02T13:23:44.350 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/169 2024-02-02T13:23:44.392 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/169 2024-02-02T13:23:44.392 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/169 2024-02-02T13:23:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:44.459 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:44.524 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/169 2024-02-02T13:23:44.545 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/169 2024-02-02T13:23:44.574 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/169 2024-02-02T13:23:44.619 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/169 2024-02-02T13:23:44.622 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/169 2024-02-02T13:23:44.692 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/169 2024-02-02T13:23:44.730 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 159/169 2024-02-02T13:23:44.773 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 160/169 2024-02-02T13:23:44.814 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 161/169 2024-02-02T13:23:44.815 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/169 2024-02-02T13:23:44.943 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/169 2024-02-02T13:23:44.973 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2024-02-02T13:23:45.009 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:45.009 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:45.322 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:45.323 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2024-02-02T13:23:45.492 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2024-02-02T13:23:45.543 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 166/169 2024-02-02T13:23:45.576 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/169 2024-02-02T13:23:45.615 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 168/169 2024-02-02T13:23:45.746 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:45.931 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 147/169 2024-02-02T13:23:46.009 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/169 2024-02-02T13:23:46.009 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:46.009 INFO:teuthology.orchestra.run.smithi091.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-02T13:23:46.009 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:46.056 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/169 2024-02-02T13:23:46.187 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/169 2024-02-02T13:23:46.228 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/169 2024-02-02T13:23:46.263 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/169 2024-02-02T13:23:46.295 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/169 2024-02-02T13:23:46.336 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/169 2024-02-02T13:23:46.336 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/169 2024-02-02T13:23:46.462 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/169 2024-02-02T13:23:46.522 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/169 2024-02-02T13:23:46.569 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/169 2024-02-02T13:23:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/169 2024-02-02T13:23:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 159/169 2024-02-02T13:23:46.702 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 160/169 2024-02-02T13:23:46.744 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 161/169 2024-02-02T13:23:46.745 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/169 2024-02-02T13:23:46.856 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/169 2024-02-02T13:23:46.887 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2024-02-02T13:23:46.927 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:46.927 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:47.236 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2024-02-02T13:23:47.236 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2024-02-02T13:23:47.379 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2024-02-02T13:23:47.415 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 166/169 2024-02-02T13:23:47.448 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/169 2024-02-02T13:23:47.487 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 168/169 2024-02-02T13:23:47.646 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:47.907 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:47.908 INFO:teuthology.orchestra.run.smithi113.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-02T13:23:47.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-2:18.2.1-325.g0fb32788.el9.x86_64 1/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 2/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 3/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 4/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 5/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 6/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 7/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 8/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 11/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 12/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 13/169 2024-02-02T13:23:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_6 14/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 16/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 17/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 18/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 19/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9 20/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x 21/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 22/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 23/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 24/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 25/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 26/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 27/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 28/169 2024-02-02T13:23:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 29/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 30/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 31/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 32/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9 33/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 34/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 35/169 2024-02-02T13:23:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 36/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephadm-2:18.2.1-325.g0fb32788.el9.noarch 37/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 38/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/169 2024-02-02T13:23:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 57/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 62/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 63/169 2024-02-02T13:23:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 64/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 65/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 66/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 67/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 68/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 69/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 70/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 71/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 72/169 2024-02-02T13:23:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 73/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 80/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 82/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 83/169 2024-02-02T13:23:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : jq-1.6-16.el9.x86_64 84/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 85/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 86/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 87/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 92/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 93/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 97/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 98/169 2024-02-02T13:23:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 101/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 108/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/169 2024-02-02T13:23:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/169 2024-02-02T13:23:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados2-2:18.2.1-325.g0fb32788.el9.x86_64 134/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 136/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-1:1.45.90-1.el9.x86_64 140/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/169 2024-02-02T13:23:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-libnm-1:1.45.90-1.el9.x86_64 142/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-team-1:1.45.90-1.el9.x86_64 144/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-tui-1:1.45.90-1.el9.x86_64 146/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 150/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 151/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 152/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 153/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 154/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 155/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 156/169 2024-02-02T13:23:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 157/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 158/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 159/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/169 2024-02-02T13:23:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:18.2.1-325.g0fb32788.el9.x86_64 1/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 2/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 3/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 4/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 5/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 6/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 7/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 8/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 9/169 2024-02-02T13:23:52.518 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 10/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 11/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 12/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 13/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_6 14/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 15/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 16/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 17/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_ 18/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 19/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9 20/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x 21/169 2024-02-02T13:23:52.519 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 22/169 2024-02-02T13:23:52.520 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 23/169 2024-02-02T13:23:52.520 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 24/169 2024-02-02T13:23:52.520 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 25/169 2024-02-02T13:23:52.520 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 26/169 2024-02-02T13:23:52.520 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 27/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 28/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 29/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noar 30/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.no 31/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 32/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9 33/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 34/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 35/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 36/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:18.2.1-325.g0fb32788.el9.noarch 37/169 2024-02-02T13:23:52.532 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 38/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/169 2024-02-02T13:23:52.533 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 57/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 62/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 63/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 64/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 65/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 66/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 67/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 68/169 2024-02-02T13:23:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 69/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 70/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 71/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 72/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 73/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 80/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 82/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 83/169 2024-02-02T13:23:52.535 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-16.el9.x86_64 84/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 85/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 86/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 87/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 92/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 93/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 97/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 98/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/169 2024-02-02T13:23:52.536 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 101/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 108/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/169 2024-02-02T13:23:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/169 2024-02-02T13:23:52.538 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:18.2.1-325.g0fb32788.el9.x86_64 134/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 136/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.45.90-1.el9.x86_64 140/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.45.90-1.el9.x86_64 142/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.45.90-1.el9.x86_64 144/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.45.90-1.el9.x86_64 146/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/169 2024-02-02T13:23:52.539 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 150/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 151/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 152/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 153/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 154/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 155/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 156/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 157/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 158/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 159/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/169 2024-02-02T13:23:52.540 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/169 2024-02-02T13:23:52.541 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/169 2024-02-02T13:23:52.541 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/169 2024-02-02T13:23:52.541 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/169 2024-02-02T13:23:52.945 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:52.945 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout:Upgraded: 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-libnm-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-team-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-tui-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: acl-2.3.1-4.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: librados2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.946 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: automake-1.16.2-8.el9.noarch 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: bison-3.7.4-5.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-immutable-object-cache-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.947 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: cephadm-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:52.948 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: flex-2.6.4-9.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: jq-1.6-16.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-02T13:23:52.949 INFO:teuthology.orchestra.run.smithi091.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-02T13:23:52.950 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-13.11-1.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-02T13:23:52.951 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-02T13:23:52.952 INFO:teuthology.orchestra.run.smithi091.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ply-3.11-14.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-02T13:23:52.953 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-02T13:23:52.954 INFO:teuthology.orchestra.run.smithi091.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: re2-1:20211101-3.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:23:52.955 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2024-02-02T13:23:53.531 DEBUG:teuthology.parallel:result is None 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/169 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:54.691 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui-1:1.45.90-1.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: acl-2.3.1-4.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:54.692 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:18.2.1-325.g0fb32788.el9.noarch 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-02T13:23:54.694 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-16.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-02T13:23:54.695 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-02T13:23:54.696 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.11-1.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-02T13:23:54.697 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-02T13:23:54.698 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-02T13:23:54.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-02T13:23:54.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:18.2.1-325.g0fb32788.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-02T13:23:54.701 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:23:54.702 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-02-02T13:23:55.363 DEBUG:teuthology.parallel:result is None 2024-02-02T13:23:55.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T13:23:55.622 DEBUG:teuthology.orchestra.run.smithi091:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-02T13:23:55.680 INFO:teuthology.orchestra.run.smithi091.stdout:18.2.1-325.g0fb32788.el9 2024-02-02T13:23:55.681 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-325.g0fb32788.el9 2024-02-02T13:23:55.681 INFO:teuthology.task.install:The correct ceph version 18.2.1-325.g0fb32788 is installed. 2024-02-02T13:23:55.683 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T13:23:55.817 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-02T13:23:55.872 INFO:teuthology.orchestra.run.smithi113.stdout:18.2.1-325.g0fb32788.el9 2024-02-02T13:23:55.872 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-325.g0fb32788.el9 2024-02-02T13:23:55.873 INFO:teuthology.task.install:The correct ceph version 18.2.1-325.g0fb32788 is installed. 2024-02-02T13:23:55.874 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-02T13:23:55.875 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:55.875 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-02T13:23:55.958 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:55.958 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-02T13:23:56.035 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-02T13:23:56.036 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:56.036 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/daemon-helper 2024-02-02T13:23:56.116 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-02T13:23:56.233 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:56.233 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-02-02T13:23:56.312 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-02T13:23:56.423 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-02T13:23:56.423 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:56.423 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-02T13:23:56.497 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-02T13:23:56.607 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:56.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-02T13:23:56.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-02T13:23:56.810 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-02T13:23:56.810 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:56.810 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/stdin-killer 2024-02-02T13:23:56.887 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-02T13:23:57.000 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:57.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-02-02T13:23:57.078 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-02T13:23:57.197 INFO:teuthology.run_tasks:Running task ceph... 2024-02-02T13:23:57.261 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-02-02T13:23:57.261 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 777 /var/log/ceph 2024-02-02T13:23:57.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-02-02T13:23:57.344 INFO:tasks.ceph:Disabling ceph logrotate... 2024-02-02T13:23:57.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-02T13:23:57.348 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-02T13:23:57.419 INFO:tasks.ceph:Creating extra log directories... 2024-02-02T13:23:57.420 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-02T13:23:57.423 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-02T13:23:57.500 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-02-02T13:23:57.500 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-02-02T13:23:57.500 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-02-02T13:23:57.500 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:57.500 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-02T13:23:57.539 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-02T13:23:57.752 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-02T13:23:57.819 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:57.819 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-02T13:23:57.863 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-02T13:23:58.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-02T13:23:58.153 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-02-02T13:23:58.153 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-02-02T13:23:58.153 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405', 'branch': 'wip-vshankar-testing1-reef-2024-02-02-1152', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7543405', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-reef-2024-02-02-1152'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-reef-2024-02-02-1152', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1628, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'sleep_before_teardown': 0, 'subset': '50/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-reef-2024-02-02-1152', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi091.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCdICmb/r8PUIMuJy7WjtsllZdEjiLuEeZ/8qghtbmxYAKoEFdGpcR5nZbsIBxVY/ApKVDN249OmUhF5FXx8LA=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxTOeL51Zfohp5F+kujWCvAxf0T/oaoI5vrfYOa0yAXSgNzJzhyZ+0KzGVKxl60ZjkC0P6sfN6ruIBB0pVukGw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'd9fdb2209e15b39d9f061fd85399f352ce0f0894', 'timestamp': '2024-02-02_11:17:23', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-02-02T13:23:58.154 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-02T13:23:58.159 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-02T13:23:58.197 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/run/ceph 2024-02-02T13:23:58.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-02-02T13:23:58.313 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:23:58.313 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2024-02-02T13:23:58.352 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-02T13:23:58.353 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_1 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 640 Links: 1 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-02 13:23:46.427259493 +0000 2024-02-02T13:23:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-02 13:21:26.645513411 +0000 2024-02-02T13:23:58.434 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-02 13:21:26.645513411 +0000 2024-02-02T13:23:58.434 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-02 13:21:26.645513411 +0000 2024-02-02T13:23:58.434 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-02T13:23:58.541 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-02T13:23:58.541 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-02T13:23:58.541 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000327175 s, 1.6 MB/s 2024-02-02T13:23:58.543 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-02T13:23:58.582 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_2 2024-02-02T13:23:58.665 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-02T13:23:58.665 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 657 Links: 1 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-02 13:23:46.428259468 +0000 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-02 13:21:27.227498887 +0000 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-02 13:21:27.227498887 +0000 2024-02-02T13:23:58.666 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-02 13:21:27.227498887 +0000 2024-02-02T13:23:58.666 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-02T13:23:58.796 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-02T13:23:58.796 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-02T13:23:58.796 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000302318 s, 1.7 MB/s 2024-02-02T13:23:58.800 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-02T13:23:58.855 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_3 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-02 13:23:46.429259443 +0000 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-02 13:21:27.786484936 +0000 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-02 13:21:27.786484936 +0000 2024-02-02T13:23:58.902 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-02 13:21:27.786484936 +0000 2024-02-02T13:23:58.903 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-02T13:23:58.979 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-02T13:23:58.979 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-02T13:23:58.979 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000345005 s, 1.5 MB/s 2024-02-02T13:23:58.983 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-02T13:23:59.032 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_4 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 694 Links: 1 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2024-02-02 13:23:46.429259443 +0000 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2024-02-02 13:21:28.390469862 +0000 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2024-02-02 13:21:28.390469862 +0000 2024-02-02T13:23:59.079 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2024-02-02 13:21:28.390469862 +0000 2024-02-02T13:23:59.079 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-02T13:23:59.164 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2024-02-02T13:23:59.164 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2024-02-02T13:23:59.164 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.00034938 s, 1.5 MB/s 2024-02-02T13:23:59.169 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-02T13:23:59.223 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-02T13:23:59.223 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:23:59.223 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-02-02T13:23:59.269 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-02T13:23:59.270 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-02-02T13:23:59.348 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-02 13:23:48.354047954 +0000 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-02 13:21:33.565444624 +0000 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-02 13:21:33.565444624 +0000 2024-02-02T13:23:59.349 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-02 13:21:33.565444624 +0000 2024-02-02T13:23:59.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-02T13:23:59.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-02T13:23:59.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-02T13:23:59.460 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000386385 s, 1.3 MB/s 2024-02-02T13:23:59.463 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-02T13:23:59.504 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-02-02T13:23:59.588 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-02T13:23:59.588 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:59.588 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-02-02T13:23:59.588 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:59.589 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:59.589 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-02 13:23:48.355047933 +0000 2024-02-02T13:23:59.589 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-02 13:21:34.138432868 +0000 2024-02-02T13:23:59.589 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-02 13:21:34.138432868 +0000 2024-02-02T13:23:59.589 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-02 13:21:34.138432868 +0000 2024-02-02T13:23:59.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-02T13:23:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-02T13:23:59.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-02T13:23:59.739 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00029865 s, 1.7 MB/s 2024-02-02T13:23:59.740 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-02T13:23:59.784 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-02-02T13:23:59.866 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-02 13:23:48.355047933 +0000 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-02 13:21:34.711421112 +0000 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-02 13:21:34.711421112 +0000 2024-02-02T13:23:59.867 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-02 13:21:34.711421112 +0000 2024-02-02T13:23:59.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-02T13:23:59.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-02T13:23:59.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-02T13:23:59.980 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000330983 s, 1.5 MB/s 2024-02-02T13:23:59.982 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-02T13:24:00.023 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-02-02T13:24:00.104 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-02 13:23:48.356047913 +0000 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-02 13:21:35.289409254 +0000 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-02 13:21:35.289409254 +0000 2024-02-02T13:24:00.105 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-02 13:21:35.289409254 +0000 2024-02-02T13:24:00.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-02T13:24:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-02T13:24:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-02T13:24:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000338796 s, 1.5 MB/s 2024-02-02T13:24:00.222 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-02T13:24:00.262 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:00.262 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-02T13:24:00.262 INFO:tasks.ceph:Generating config... 2024-02-02T13:24:00.263 INFO:tasks.ceph:[client] client mount timeout = 600 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client] debug client = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client] debug ms = 1 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client] pid file = 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.0] debug client = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.1] debug client = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-02-02T13:24:00.264 INFO:tasks.ceph:[mds] debug mds = 20 2024-02-02T13:24:00.264 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] debug ms = 1 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] mds debug frag = True 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mgr] debug ms = 1 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mon] debug mon = 20 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mon] debug ms = 1 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mon] debug paxos = 20 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-02-02T13:24:00.265 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-02-02T13:24:00.266 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-02-02T13:24:00.266 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] debug ms = 1 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] debug osd = 20 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-02-02T13:24:00.266 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-02-02T13:24:00.267 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-02-02T13:24:00.267 INFO:tasks.ceph:Setting up mon.a... 2024-02-02T13:24:00.267 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-02-02T13:24:00.420 INFO:teuthology.orchestra.run.smithi091.stdout:creating /etc/ceph/ceph.keyring 2024-02-02T13:24:00.427 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-02-02T13:24:00.548 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-02T13:24:00.626 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.91'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2024-02-02T13:24:00.626 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.91,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-02-02T13:24:00.627 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:00.627 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-02-02T13:24:00.670 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.91 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:monmaptool: generated fsid cc006970-f900-4781-8947-81ed22cd959b 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:setting min_mon_release = pacific 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 0 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:fsid cc006970-f900-4781-8947-81ed22cd959b 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:last_changed 2024-02-02T13:24:00.805799+0000 2024-02-02T13:24:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:created 2024-02-02T13:24:00.805799+0000 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:min_mon_release 16 (pacific) 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:election_strategy: 1 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:0: [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] mon.a 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2024-02-02T13:24:00.807 INFO:teuthology.orchestra.run.smithi091.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-02-02T13:24:00.809 DEBUG:teuthology.orchestra.run.smithi091:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-02-02T13:24:00.852 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cc006970-f900-4781-8947-81ed22cd959b... 2024-02-02T13:24:00.853 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-02T13:24:00.897 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-02T13:24:01.040 INFO:teuthology.orchestra.run.smithi091.stdout:[global] 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: chdir = "" 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: auth supported = cephx 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: filestore xattr use omap = true 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: mon clock drift allowed = 1.000 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: osd crush chooseleaf type = 0 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: auth debug = true 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on old message = true 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on bug = true 2024-02-02T13:24:01.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon pg warn max object skew = 0 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: # disable pg_autoscaler by default for new pools 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default size = 2 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow primary affinity = true 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow pg remap = true 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on legacy crush tunables = false 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on crush straw calc version zero = false 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on no sortbitwise = false 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on osd down out interval zero = false 2024-02-02T13:24:01.042 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on too few osds = false 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon_allow_pool_size_one = true 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: osd default data pool replay window = 5 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon allow pool delete = true 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon cluster log file level = debug 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: debug asserts on shutdown = true 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon health detail to clog = false 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: mon host = "172.21.15.91,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on skipped message = False 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: fsid = cc006970-f900-4781-8947-81ed22cd959b 2024-02-02T13:24:01.043 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout:[osd] 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd journal size = 100 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub load threshold = 5.0 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub max interval = 600 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd mclock profile = high_recovery_ops 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd recover clone overlap = true 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd recovery max chunk = 1048576 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug shutdown = true 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug op order = true 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug verify stray on activate = true 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.044 INFO:teuthology.orchestra.run.smithi091.stdout: osd open classes on start = true 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug pg log writeout = true 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: osd deep scrub update digest min age = 30 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: osd map max advance = 10 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: journal zero on create = true 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: filestore ondisk finisher threads = 3 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: filestore apply finisher threads = 3 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: bdev debug aio = true 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug misdirected ops = true 2024-02-02T13:24:01.045 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore block size = 96636764160 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore fsck on mount = True 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluefs = 20 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluestore = 20 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: debug osd = 20 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: debug rocksdb = 10 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: memstore device bytes = 200000000 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd backfillfull_ratio = 0.85 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd full ratio = 0.9 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd nearfull ratio = 0.8 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: osd failsafe full ratio = 1.0 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: osd mon report interval = 5 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: osd objectstore = memstore 2024-02-02T13:24:01.046 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr] 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug mgr = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: mgr/telemetry/nag = false 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout:[mon] 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug paxos = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2024-02-02T13:24:01.047 INFO:teuthology.orchestra.run.smithi091.stdout: mon data avail warn = 5 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon mgr mkfs grace = 240 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd reporter subtree level = osd 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd prime pg temp = true 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: auth mon ticket ttl = 660 # 11m 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: auth service ticket ttl = 240 # 4m 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: # don't complain about insecure global_id in the test suite 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.048 INFO:teuthology.orchestra.run.smithi091.stdout: # 1m isn't quite enough 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon_down_mkfs_grace = 2m 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_filestore_osds = false 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon op complaint time = 120 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd backfillfull ratio = 0.6 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd full ratio = 0.7 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd nearfull ratio = 0.6 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout:[client] 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: rgw cache enabled = true 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable ops log = true 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable usage log = true 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: client mount timeout = 600 2024-02-02T13:24:01.049 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: pid file = "" 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 900 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 900 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.a] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.b] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.c] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[client.0] 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: debug objectcacher = 20 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout: debug objecter = 20 2024-02-02T13:24:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:[client.1] 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug objectcacher = 20 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug objecter = 20 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout:[mds] 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug mds = 20 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug mds balancer = 20 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug frag = True 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug scatterstat = True 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: mds op complaint time = 180 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: mds verify scatter = True 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 900 2024-02-02T13:24:01.051 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 900 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.068 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.069 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-02T13:24:01.070 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.91,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = cc006970-f900-4781-8947-81ed22cd959b 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.071 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-02-02T13:24:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-02-02T13:24:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: memstore device bytes = 200000000 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 1.0 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: osd mon report interval = 5 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = memstore 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-02-02T13:24:01.074 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-02-02T13:24:01.075 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-02-02T13:24:01.076 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull ratio = 0.6 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.7 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.6 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-02-02T13:24:01.077 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout: debug objectcacher = 20 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout: debug objecter = 20 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-02-02T13:24:01.078 INFO:teuthology.orchestra.run.smithi113.stdout: debug objectcacher = 20 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: debug objecter = 20 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-02-02T13:24:01.079 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-02-02T13:24:01.113 INFO:tasks.ceph:Creating admin key on mon.a... 2024-02-02T13:24:01.113 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-02-02T13:24:01.242 INFO:tasks.ceph:Copying monmap to all nodes... 2024-02-02T13:24:01.242 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:01.242 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-02-02T13:24:01.284 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:01.284 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-02-02T13:24:01.374 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:24:01.374 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:01.375 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-02T13:24:01.375 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-02T13:24:01.552 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:01.552 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:01.643 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:24:01.643 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:01.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-02T13:24:01.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-02T13:24:01.776 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:01.776 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:01.871 INFO:tasks.ceph:Setting up mon nodes... 2024-02-02T13:24:01.871 INFO:tasks.ceph:Setting up mgr nodes... 2024-02-02T13:24:01.871 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-02-02T13:24:02.018 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-02-02T13:24:02.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-02-02T13:24:02.240 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-02-02T13:24:02.252 INFO:tasks.ceph:Setting up mds nodes... 2024-02-02T13:24:02.252 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-02-02T13:24:02.435 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-02-02T13:24:02.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-02-02T13:24:02.538 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-02-02T13:24:02.718 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-02-02T13:24:02.727 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-02-02T13:24:02.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-02-02T13:24:02.999 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-02-02T13:24:03.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-02-02T13:24:03.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-02-02T13:24:03.262 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-02-02T13:24:03.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-02-02T13:24:03.348 INFO:tasks.ceph_client:Setting up client nodes... 2024-02-02T13:24:03.349 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-02T13:24:03.478 INFO:teuthology.orchestra.run.smithi091.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-02-02T13:24:03.533 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-02T13:24:03.660 INFO:teuthology.orchestra.run.smithi091.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-02-02T13:24:03.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-02T13:24:03.836 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-02-02T13:24:03.879 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-02-02T13:24:03.984 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-02-02T13:24:04.031 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-02-02T13:24:04.031 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-02T13:24:04.031 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-02-02T13:24:04.133 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:04.133 INFO:tasks.ceph:role: osd.0 2024-02-02T13:24:04.133 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:24:04.133 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-02T13:24:04.231 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:04.231 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:04.231 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:04.231 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:04.232 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:04.751 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2024-02-02T13:24:04.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2024-02-02T13:24:04.758 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-02-02T13:24:04.937 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-02-02T13:24:05.040 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-02-02T13:24:05.133 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:05.134 INFO:tasks.ceph:role: osd.1 2024-02-02T13:24:05.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:24:05.134 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:05.230 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:05.694 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2024-02-02T13:24:05.700 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2024-02-02T13:24:05.700 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-02-02T13:24:05.827 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-02-02T13:24:05.907 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-02-02T13:24:05.982 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:05.982 INFO:tasks.ceph:role: osd.2 2024-02-02T13:24:05.982 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:24:05.982 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-02T13:24:06.079 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:06.079 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:06.079 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:06.079 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:06.079 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:06.080 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:06.080 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:06.080 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:06.080 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:06.080 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:06.610 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2024-02-02T13:24:06.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2024-02-02T13:24:06.617 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-02-02T13:24:06.786 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-02-02T13:24:06.884 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-02-02T13:24:06.976 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:06.977 INFO:tasks.ceph:role: osd.3 2024-02-02T13:24:06.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:24:06.977 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-02T13:24:07.072 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:07.530 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2024-02-02T13:24:07.536 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2024-02-02T13:24:07.537 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-02-02T13:24:07.651 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-02-02T13:24:07.730 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:07.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:07.864+0000 7f0e1bb9b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-02-02T13:24:07.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:07.864+0000 7f0e1bb9b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-02-02T13:24:07.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:07.867+0000 7f0e1bb9b500 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-02-02T13:24:07.892 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-02T13:24:07.980 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:08.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.116+0000 7f862cd39500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-02-02T13:24:08.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.117+0000 7f862cd39500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-02-02T13:24:08.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.119+0000 7f862cd39500 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-02-02T13:24:08.138 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-02T13:24:08.221 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:08.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.357+0000 7f4602c12500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-02-02T13:24:08.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.358+0000 7f4602c12500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-02-02T13:24:08.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.359+0000 7f4602c12500 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-02-02T13:24:08.378 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-02T13:24:08.454 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.590+0000 7fd1b75e2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-02-02T13:24:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.591+0000 7fd1b75e2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-02-02T13:24:08.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:08.593+0000 7fd1b75e2500 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-02-02T13:24:08.611 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-02T13:24:08.696 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-02-02T13:24:08.797 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:08.797 INFO:tasks.ceph:role: osd.4 2024-02-02T13:24:08.797 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:24:08.797 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:08.892 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:09.432 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-02-02T13:24:09.439 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-02-02T13:24:09.439 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-02-02T13:24:09.598 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-02-02T13:24:09.705 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-02-02T13:24:09.793 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:09.793 INFO:tasks.ceph:role: osd.5 2024-02-02T13:24:09.793 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:24:09.793 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:09.889 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:10.419 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-02-02T13:24:10.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-02-02T13:24:10.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-02-02T13:24:10.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-02-02T13:24:10.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-02-02T13:24:10.757 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:10.757 INFO:tasks.ceph:role: osd.6 2024-02-02T13:24:10.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:24:10.758 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-02T13:24:10.854 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:10.854 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:10.854 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:10.854 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:10.855 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:11.384 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-02-02T13:24:11.390 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-02-02T13:24:11.391 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-02-02T13:24:11.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-02-02T13:24:11.647 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-02-02T13:24:11.736 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-02T13:24:11.736 INFO:tasks.ceph:role: osd.7 2024-02-02T13:24:11.736 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:24:11.736 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-02T13:24:11.834 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-02T13:24:11.834 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-02T13:24:11.834 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-02T13:24:11.834 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-02T13:24:11.834 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-02T13:24:11.835 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-02-02T13:24:11.835 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-02T13:24:11.835 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-02T13:24:11.835 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-02T13:24:11.835 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-02T13:24:12.373 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-02-02T13:24:12.379 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-02-02T13:24:12.379 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-02-02T13:24:12.542 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-02-02T13:24:12.630 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:12.777+0000 7fb9a19e2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-02-02T13:24:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:12.780+0000 7fb9a19e2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-02-02T13:24:12.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:12.787+0000 7fb9a19e2500 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-02-02T13:24:12.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-02T13:24:12.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:13.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.032+0000 7fe15000c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-02-02T13:24:13.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.033+0000 7fe15000c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-02-02T13:24:13.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.040+0000 7fe15000c500 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-02-02T13:24:13.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-02T13:24:13.157 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.287+0000 7f846fe2a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-02-02T13:24:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.288+0000 7f846fe2a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-02-02T13:24:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.290+0000 7f846fe2a500 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-02-02T13:24:13.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-02-02T13:24:13.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:13.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.540+0000 7fb6dc81f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-02-02T13:24:13.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.541+0000 7fb6dc81f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-02-02T13:24:13.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:13.543+0000 7fb6dc81f500 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-02-02T13:24:13.562 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-02-02T13:24:13.641 INFO:tasks.ceph:Reading keys from all nodes... 2024-02-02T13:24:13.641 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:13.641 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-02-02T13:24:13.739 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:13.740 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-02-02T13:24:13.836 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:13.837 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-02-02T13:24:13.928 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:13.928 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-02-02T13:24:14.015 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:14.016 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-02-02T13:24:14.105 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:14.105 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-02-02T13:24:14.195 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:14.195 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-02-02T13:24:14.287 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-02-02T13:24:14.386 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.386 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-02-02T13:24:14.480 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.480 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-02-02T13:24:14.573 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-02-02T13:24:14.710 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.710 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-02-02T13:24:14.799 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.799 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-02-02T13:24:14.871 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:14.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-02-02T13:24:15.007 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:15.007 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-02-02T13:24:15.064 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:15.064 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-02-02T13:24:15.112 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:15.113 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-02-02T13:24:15.166 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:15.166 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-02-02T13:24:15.211 INFO:tasks.ceph:Adding keys to all mons... 2024-02-02T13:24:15.211 DEBUG:teuthology.orchestra.run.smithi091:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-02T13:24:15.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.y] 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA== 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ== 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q== 2024-02-02T13:24:15.306 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.0] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.1] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.2] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.3] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.x] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag== 2024-02-02T13:24:15.307 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.4] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.5] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.6] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.7] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[client.0] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[client.2] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[client.1] 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA== 2024-02-02T13:24:15.308 INFO:teuthology.orchestra.run.smithi091.stdout:[client.3] 2024-02-02T13:24:15.309 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ== 2024-02-02T13:24:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q== 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-02-02T13:24:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw== 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA== 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA== 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA== 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2024-02-02T13:24:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA== 2024-02-02T13:24:15.349 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-02T13:24:15.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-02T13:24:15.496 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:15.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:15.643 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:15.648 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:15.791 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:15.796 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:15.938 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:15.946 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.084 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.089 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.224 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.364 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-02T13:24:16.371 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-02T13:24:16.510 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:16.515 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:16.659 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:16.667 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-02T13:24:16.807 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:16.964 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:17.101 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:17.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:17.241 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:17.248 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-02T13:24:17.386 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.534 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.541 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.679 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.824 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-02T13:24:17.976 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-02-02T13:24:17.976 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-02-02T13:24:18.071 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-02T13:24:18.227 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-02-02T13:24:18.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-02-02T13:24:18.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-02T13:24:18.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-02-02T13:24:18.650 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-02-02T13:24:18.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-02T13:24:18.897 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-02-02T13:24:18.979 DEBUG:teuthology.orchestra.run.smithi091:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:18.986 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-02T13:24:19.041 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-02-02T13:24:19.041 INFO:tasks.ceph.mon.a:Restarting daemon 2024-02-02T13:24:19.041 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-02-02T13:24:19.048 INFO:tasks.ceph.mon.a:Started 2024-02-02T13:24:19.048 INFO:tasks.ceph.mon.b:Restarting daemon 2024-02-02T13:24:19.048 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-02-02T13:24:19.053 INFO:tasks.ceph.mon.b:Started 2024-02-02T13:24:19.053 INFO:tasks.ceph.mon.c:Restarting daemon 2024-02-02T13:24:19.053 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-02-02T13:24:19.058 INFO:tasks.ceph.mon.c:Started 2024-02-02T13:24:19.058 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-02-02T13:24:19.058 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-02-02T13:24:19.058 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-02-02T13:24:19.063 INFO:tasks.ceph.mgr.y:Started 2024-02-02T13:24:19.063 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-02-02T13:24:19.063 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-02-02T13:24:19.067 INFO:tasks.ceph.mgr.x:Started 2024-02-02T13:24:19.067 INFO:tasks.ceph:Setting crush tunables to default 2024-02-02T13:24:19.067 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd crush tunables default 2024-02-02T13:24:19.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.242+0000 7f840ac00640 1 Processor -- start 2024-02-02T13:24:19.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.242+0000 7f840ac00640 1 -- start start 2024-02-02T13:24:19.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f840ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f840ac00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 0x7f84040d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f840ac00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84040d3710 con 0x7f84040d32e0 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f840ac00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040b4930 con 0x7f84040d2890 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f840ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040b4c10 con 0x7f8404160610 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.243+0000 7f8409800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:44496/0 (socket says 172.21.15.91:44496) 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 learned_addr learned my addr 172.21.15.91:0/38357427 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 msgr2=0x7f84040d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 msgr2=0x7f84040d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8408e00640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 0x7f84040d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8408e00640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 msgr2=0x7f84040d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-02T13:24:19.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8408e00640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 msgr2=0x7f84040d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-02T13:24:19.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8408e00640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 0x7f84040d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-02T13:24:19.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.244+0000 7f8408e00640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 0x7f84040d2da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T13:24:19.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084010265 0 0) 0x7f84040d3710 con 0x7f84040d32e0 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e4003520 con 0x7f84040d32e0 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560575348 0 0) 0x7f83e4003520 con 0x7f84040d32e0 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 msgr2=0x7f84040d2350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.113:3300/0 conn(0x7f8404160610 0x7f84040d2350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 msgr2=0x7f84040d2da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 --2- 172.21.15.91:0/38357427 >> v2:172.21.15.91:3300/0 conn(0x7f84040d2890 0x7f84040d2da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040b4ef0 con 0x7f84040d32e0 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f83fc0019f0 con 0x7f84040d32e0 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 >> v1:172.21.15.91:6789/0 conn(0x7f84040d32e0 legacy=0x7f84040d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f83e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83e40047c0 0x7f83e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e4005180 0x7f83e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84040b4930 con 0x7f83e4005180 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84040b4c10 con 0x7f83e4003970 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.263+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e400c080 con 0x7f83e40047c0 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f83e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f840a200640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e4005180 0x7f83e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8408e00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83e40047c0 0x7f83e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83e40047c0 msgr2=0x7f83e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83e40047c0 0x7f83e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e4005180 msgr2=0x7f83e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e4005180 0x7f83e400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8409800640 1 -- 172.21.15.91:0/38357427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4003520 con 0x7f83e4003970 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f840a200640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e4005180 0x7f83e400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f840a200640 1 -- 172.21.15.91:0/38357427 reap_dead start 2024-02-02T13:24:19.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.264+0000 7f8408e00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83e40047c0 0x7f83e4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:19.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.265+0000 7f8409800640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f83e4003d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009890 tx=0x7f83e40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127b026dc9cd5268 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:19.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.266+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4030660 con 0x7f83e4003970 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.266+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f40307c0 con 0x7f83e4003970 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.266+0000 7f8403e00640 1 -- 172.21.15.91:0/38357427 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4030b70 con 0x7f83e4003970 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.266+0000 7f840ac00640 1 -- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 msgr2=0x7f83e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.266+0000 7f840ac00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f83e4003d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009890 tx=0x7f83e40057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.267+0000 7f840ac00640 1 -- 172.21.15.91:0/38357427 shutdown_connections 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.267+0000 7f840ac00640 1 --2- 172.21.15.91:0/38357427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f83e4003d90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.267+0000 7f840ac00640 1 -- 172.21.15.91:0/38357427 >> 172.21.15.91:0/38357427 conn(0x7f8404070690 msgr2=0x7f8404070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.267+0000 7f840ac00640 1 -- 172.21.15.91:0/38357427 shutdown_connections 2024-02-02T13:24:19.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.267+0000 7f840ac00640 1 -- 172.21.15.91:0/38357427 wait complete. 2024-02-02T13:24:19.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.268+0000 7f840ac00640 1 Processor -- start 2024-02-02T13:24:19.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.269+0000 7f840ac00640 1 -- start start 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.269+0000 7f840ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f8404152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.269+0000 7f840ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f840415aa50 0x7f840415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.269+0000 7f840ac00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 0x7f8404158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840ac00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84040cdea0 con 0x7f840415b3d0 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8404158550 con 0x7f83e4003970 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415b850 con 0x7f840415aa50 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f8409800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f8404152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 0x7f8404158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f8408e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f840415aa50 0x7f840415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f8409800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f8404152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:44516/0 (socket says 172.21.15.91:44516) 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f8409800640 1 -- 172.21.15.91:0/2644201230 learned_addr learned my addr 172.21.15.91:0/2644201230 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 -- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f840415aa50 msgr2=0x7f840415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:19.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f840415aa50 0x7f840415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 -- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 msgr2=0x7f8404152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:19.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f8404152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.270+0000 7f840a200640 1 -- 172.21.15.91:0/2644201230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040cdbc0 con 0x7f840415b3d0 2024-02-02T13:24:19.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f840a200640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 0x7f8404158ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f84040b59c0 tx=0x7f83fc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc003b80 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc003ce0 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84041587f0 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00fac0 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8404153310 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f83fc0050d0 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.271+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f83fc005380 con 0x7f840415b3d0 2024-02-02T13:24:19.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.272+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84041583b0 con 0x7f840415b3d0 2024-02-02T13:24:19.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.275+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f84041583b0 con 0x7f840415b3d0 2024-02-02T13:24:19.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.346+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8404158d10 con 0x7f840415b3d0 2024-02-02T13:24:19.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.350+0000 7f8401600640 1 -- 172.21.15.91:0/2644201230 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8404158d10 con 0x7f840415b3d0 2024-02-02T13:24:19.352 INFO:teuthology.orchestra.run.smithi091.stderr:adjusted tunables profile to default 2024-02-02T13:24:19.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.351+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 msgr2=0x7f8404158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.351+0000 7f840ac00640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 0x7f8404158ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f84040b59c0 tx=0x7f83fc00cf00 comp rx=0 tx=0).stop 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 shutdown_connections 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f840415aa50 0x7f840415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f840415b3d0 0x7f8404158ff0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 --2- 172.21.15.91:0/2644201230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4003970 0x7f8404152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 >> 172.21.15.91:0/2644201230 conn(0x7f8404070690 msgr2=0x7f84040b7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 shutdown_connections 2024-02-02T13:24:19.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:19.352+0000 7f840ac00640 1 -- 172.21.15.91:0/2644201230 wait complete. 2024-02-02T13:24:19.369 INFO:tasks.ceph:check_enable_crimson: False 2024-02-02T13:24:19.369 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-02-02T13:24:19.370 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:19.370 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-02-02T13:24:19.449 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:19.450 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-02-02T13:24:19.572 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:19.572 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-02-02T13:24:19.693 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:24:19.693 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-02-02T13:24:19.820 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:19.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-02-02T13:24:19.907 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:19.908 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-02-02T13:24:19.970 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:19.968+0000 7ff2faa43040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T13:24:19.986 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:19.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-02-02T13:24:20.006 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.005+0000 7f4b338d6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T13:24:20.029 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.027+0000 7ff2faa43040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T13:24:20.071 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.070+0000 7f4b338d6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T13:24:20.087 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:24:20.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-02-02T13:24:20.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-02-02T13:24:20.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.443+0000 7f4e9b000640 1 Processor -- start 2024-02-02T13:24:20.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.443+0000 7f4e9b000640 1 -- start start 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e9b000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e940d0070 0x7f4e940b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e9b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e9b000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940cd930 con 0x7f4e940b6260 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e9b000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e940b8f90 con 0x7f4e940d0070 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e9b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940d1680 con 0x7f4e940b2260 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52798/0 (socket says 172.21.15.113:52798) 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e940d0070 0x7f4e940b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.445+0000 7f4e8b200640 1 -- 172.21.15.113:0/2595088955 learned_addr learned my addr 172.21.15.113:0/2595088955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.446+0000 7f4e8b200640 1 -- 172.21.15.113:0/2595088955 >> v2:172.21.15.113:3300/0 conn(0x7f4e940d0070 msgr2=0x7f4e940b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.446+0000 7f4e8b200640 1 --2- 172.21.15.113:0/2595088955 >> v2:172.21.15.113:3300/0 conn(0x7f4e940d0070 0x7f4e940b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.446+0000 7f4e8b200640 1 -- 172.21.15.113:0/2595088955 >> v1:172.21.15.91:6789/0 conn(0x7f4e940b6260 legacy=0x7f4e940d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.446+0000 7f4e8b200640 1 -- 172.21.15.113:0/2595088955 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940d1960 con 0x7f4e940b2260 2024-02-02T13:24:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e8b200640 1 --2- 172.21.15.113:0/2595088955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009870 tx=0x7f4e7c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9037c8e2bbee5872 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e93400640 1 -- 172.21.15.113:0/2595088955 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c031640 con 0x7f4e940b2260 2024-02-02T13:24:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e93400640 1 -- 172.21.15.113:0/2595088955 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0317a0 con 0x7f4e940b2260 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e93400640 1 -- 172.21.15.113:0/2595088955 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c031b30 con 0x7f4e940b2260 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e9b000640 1 -- 172.21.15.113:0/2595088955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 msgr2=0x7f4e940d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.447+0000 7f4e9b000640 1 --2- 172.21.15.113:0/2595088955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009870 tx=0x7f4e7c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 -- 172.21.15.113:0/2595088955 shutdown_connections 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 --2- 172.21.15.113:0/2595088955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940d0450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 --2- 172.21.15.113:0/2595088955 >> v2:172.21.15.113:3300/0 conn(0x7f4e940d0070 0x7f4e940b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 -- 172.21.15.113:0/2595088955 >> 172.21.15.113:0/2595088955 conn(0x7f4e94070660 msgr2=0x7f4e94070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 -- 172.21.15.113:0/2595088955 shutdown_connections 2024-02-02T13:24:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.448+0000 7f4e9b000640 1 -- 172.21.15.113:0/2595088955 wait complete. 2024-02-02T13:24:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.449+0000 7f4e9b000640 1 Processor -- start 2024-02-02T13:24:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.449+0000 7f4e9b000640 1 -- start start 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 0x7f4e940a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e940d0070 0x7f4e9415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940aae40 con 0x7f4e940d0070 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940aa2c0 con 0x7f4e940b6260 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e9b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940aa5a0 con 0x7f4e940b2260 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.450+0000 7f4e93e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 0x7f4e940a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52806/0 (socket says 172.21.15.113:52806) 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 0x7f4e940a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53468/0 (socket says 172.21.15.113:53468) 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 -- 172.21.15.113:0/3604142926 learned_addr learned my addr 172.21.15.113:0/3604142926 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 -- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 msgr2=0x7f4e940a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 0x7f4e940a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 -- 172.21.15.113:0/3604142926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e940d0070 msgr2=0x7f4e9415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e940d0070 0x7f4e9415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 -- 172.21.15.113:0/3604142926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0094d0 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.451+0000 7f4e93e00640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400efc0 tx=0x7f4e8400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84014480 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8400f040 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84014800 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e940aa880 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e940a9fb0 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e84014c60 con 0x7f4e940b2260 2024-02-02T13:24:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.452+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4e8401dd40 con 0x7f4e940b2260 2024-02-02T13:24:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.453+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e94073830 con 0x7f4e940b2260 2024-02-02T13:24:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.456+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f4e84018020 con 0x7f4e940b2260 2024-02-02T13:24:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.533+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f4e94066cf0 con 0x7f4e940b2260 2024-02-02T13:24:20.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.540+0000 7f4e90c00640 1 -- 172.21.15.113:0/3604142926 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e8401d4d0 con 0x7f4e940b2260 2024-02-02T13:24:20.541 INFO:teuthology.orchestra.run.smithi113.stdout:0 2024-02-02T13:24:20.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 msgr2=0x7f4e940b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400efc0 tx=0x7f4e8400c490 comp rx=0 tx=0).stop 2024-02-02T13:24:20.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 shutdown_connections 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e940d0070 0x7f4e9415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e940b6260 0x7f4e940a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 --2- 172.21.15.113:0/3604142926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e940b2260 0x7f4e940b17a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.541+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 >> 172.21.15.113:0/3604142926 conn(0x7f4e94070660 msgr2=0x7f4e940d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.542+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 shutdown_connections 2024-02-02T13:24:20.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.542+0000 7f4e9b000640 1 -- 172.21.15.113:0/3604142926 wait complete. 2024-02-02T13:24:20.560 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-02-02T13:24:20.619 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.618+0000 7ff2faa43040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T13:24:20.679 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.678+0000 7ff2faa43040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T13:24:20.723 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.722+0000 7f4b338d6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T13:24:20.730 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.728+0000 7ff2faa43040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T13:24:20.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.751+0000 7f777a800640 1 Processor -- start 2024-02-02T13:24:20.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.751+0000 7f777a800640 1 -- start start 2024-02-02T13:24:20.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f777a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77740d0070 0x7f77740b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f777a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77740b2260 0x7f77740d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f777a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740cd930 con 0x7f77740b6260 2024-02-02T13:24:20.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f777a800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740b8f90 con 0x7f77740b2260 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f777a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740d1680 con 0x7f77740d0070 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77740d0070 0x7f77740b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77740d0070 0x7f77740b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53480/0 (socket says 172.21.15.113:53480) 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 -- 172.21.15.113:0/3955900827 learned_addr learned my addr 172.21.15.113:0/3955900827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7772a00640 1 --2- 172.21.15.113:0/3955900827 >> v2:172.21.15.91:3300/0 conn(0x7f77740b2260 0x7f77740d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 -- 172.21.15.113:0/3955900827 >> v2:172.21.15.91:3300/0 conn(0x7f77740b2260 msgr2=0x7f77740d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 --2- 172.21.15.113:0/3955900827 >> v2:172.21.15.91:3300/0 conn(0x7f77740b2260 0x7f77740d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 -- 172.21.15.113:0/3955900827 >> v1:172.21.15.113:6789/0 conn(0x7f77740b6260 legacy=0x7f77740d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.753+0000 7f7773400640 1 -- 172.21.15.113:0/3955900827 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740d1960 con 0x7f77740d0070 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.754+0000 7f7773400640 1 --2- 172.21.15.113:0/3955900827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740d0070 0x7f77740b8a50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77740b5700 tx=0x7f776402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e174d38ef40acc16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.755+0000 7f7772000640 1 -- 172.21.15.113:0/3955900827 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764030860 con 0x7f77740d0070 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.755+0000 7f7772000640 1 -- 172.21.15.113:0/3955900827 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640309c0 con 0x7f77740d0070 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.755+0000 7f7772000640 1 -- 172.21.15.113:0/3955900827 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764030d50 con 0x7f77740d0070 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.755+0000 7f777a800640 1 -- 172.21.15.113:0/3955900827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740d0070 msgr2=0x7f77740b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.755+0000 7f777a800640 1 --2- 172.21.15.113:0/3955900827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740d0070 0x7f77740b8a50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77740b5700 tx=0x7f776402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 -- 172.21.15.113:0/3955900827 shutdown_connections 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 --2- 172.21.15.113:0/3955900827 >> v2:172.21.15.91:3300/0 conn(0x7f77740b2260 0x7f77740d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 --2- 172.21.15.113:0/3955900827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740d0070 0x7f77740b8a50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 -- 172.21.15.113:0/3955900827 >> 172.21.15.113:0/3955900827 conn(0x7f7774070660 msgr2=0x7f7774070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:20.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 -- 172.21.15.113:0/3955900827 shutdown_connections 2024-02-02T13:24:20.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.756+0000 7f777a800640 1 -- 172.21.15.113:0/3955900827 wait complete. 2024-02-02T13:24:20.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.757+0000 7f777a800640 1 Processor -- start 2024-02-02T13:24:20.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.757+0000 7f777a800640 1 -- start start 2024-02-02T13:24:20.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 0x7f7774159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 0x7f77740ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cd510 con 0x7f77740b6260 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cbf30 con 0x7f77740b2260 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f777a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cc210 con 0x7f77740d0070 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.758+0000 7f7773e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 0x7f7774159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52810/0 (socket says 172.21.15.113:52810) 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 -- 172.21.15.113:0/3629499558 learned_addr learned my addr 172.21.15.113:0/3629499558 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7772a00640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 0x7f77740ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:20.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 -- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 msgr2=0x7f7774159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 0x7f7774159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 -- 172.21.15.113:0/3629499558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 msgr2=0x7f77740ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 0x7f77740ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 -- 172.21.15.113:0/3629499558 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77640094d0 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7772a00640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 0x7f77740ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773400640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 0x7f7774159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.759+0000 7f7773e00640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c730 tx=0x7f776c00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c003b70 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c003cf0 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77740cc4f0 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c003ea0 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77740ccab0 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f776c00a640 con 0x7f77740d0070 2024-02-02T13:24:20.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.760+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f776c012650 con 0x7f77740d0070 2024-02-02T13:24:20.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.761+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774066cf0 con 0x7f77740d0070 2024-02-02T13:24:20.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.763+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f77740cd930 con 0x7f77740d0070 2024-02-02T13:24:20.792 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.791+0000 7ff2faa43040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T13:24:20.793 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.792+0000 7f4b338d6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T13:24:20.841 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.839+0000 7ff2faa43040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T13:24:20.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.842+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7f777415c9f0 con 0x7f77740d0070 2024-02-02T13:24:20.849 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.849+0000 7f4b338d6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T13:24:20.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.849+0000 7f776b400640 1 -- 172.21.15.113:0/3629499558 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f776c00aad0 con 0x7f77740d0070 2024-02-02T13:24:20.850 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 msgr2=0x7f77740cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c730 tx=0x7f776c00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 shutdown_connections 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77740d0070 0x7f77740cb500 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77740b6260 0x7f77740ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 --2- 172.21.15.113:0/3629499558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77740b2260 0x7f7774159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 >> 172.21.15.113:0/3629499558 conn(0x7f7774070660 msgr2=0x7f77740cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 shutdown_connections 2024-02-02T13:24:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:20.850+0000 7f777a800640 1 -- 172.21.15.113:0/3629499558 wait complete. 2024-02-02T13:24:20.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-02-02T13:24:20.919 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.918+0000 7f4b338d6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T13:24:20.953 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:20.952+0000 7ff2faa43040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T13:24:20.974 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:20.973+0000 7f4b338d6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T13:24:21.032 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.030+0000 7ff2faa43040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T13:24:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.051+0000 7f5e4ae00640 1 Processor -- start 2024-02-02T13:24:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.051+0000 7f5e4ae00640 1 -- start start 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e4ae00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e440d0070 0x7f5e440b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e4ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 0x7f5e440d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e4ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440cd930 con 0x7f5e440b6260 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e4ae00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e440b8f90 con 0x7f5e440d0070 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e4ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440d1680 con 0x7f5e440b2260 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e3b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 0x7f5e440d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e48e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5e440b6260 0x7f5e440d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44176/0 (socket says 172.21.15.113:44176) 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e43e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e440d0070 0x7f5e440b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e48e00640 1 -- 172.21.15.113:0/3592327471 learned_addr learned my addr 172.21.15.113:0/3592327471 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-02-02T13:24:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e43e00640 1 -- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 msgr2=0x7f5e440d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:21.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e43e00640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 0x7f5e440d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.053+0000 7f5e43e00640 1 -- 172.21.15.113:0/3592327471 >> v1:172.21.15.113:6789/0 conn(0x7f5e440b6260 legacy=0x7f5e440d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.054+0000 7f5e43e00640 1 -- 172.21.15.113:0/3592327471 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440d1960 con 0x7f5e440d0070 2024-02-02T13:24:21.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.054+0000 7f5e3b200640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 0x7f5e440d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e43e00640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e440b8a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e3402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a45a32ffaa4818a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e43400640 1 -- 172.21.15.113:0/3592327471 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3402cc50 con 0x7f5e440d0070 2024-02-02T13:24:21.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e43400640 1 -- 172.21.15.113:0/3592327471 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3402cdb0 con 0x7f5e440d0070 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e43400640 1 -- 172.21.15.113:0/3592327471 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34034660 con 0x7f5e440d0070 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e4ae00640 1 -- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 msgr2=0x7f5e440b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.055+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e440b8a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e3402bea0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 -- 172.21.15.113:0/3592327471 shutdown_connections 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b2260 0x7f5e440d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/3592327471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e440b8a50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 -- 172.21.15.113:0/3592327471 >> 172.21.15.113:0/3592327471 conn(0x7f5e44070660 msgr2=0x7f5e44070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 -- 172.21.15.113:0/3592327471 shutdown_connections 2024-02-02T13:24:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.056+0000 7f5e4ae00640 1 -- 172.21.15.113:0/3592327471 wait complete. 2024-02-02T13:24:21.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.057+0000 7f5e4ae00640 1 Processor -- start 2024-02-02T13:24:21.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.057+0000 7f5e4ae00640 1 -- start start 2024-02-02T13:24:21.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 0x7f5e441f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e441f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441f6890 con 0x7f5e440b2260 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441f6b10 con 0x7f5e440d0070 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e4ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441f6df0 con 0x7f5e440b6260 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.058+0000 7f5e3b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e48e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e441f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52822/0 (socket says 172.21.15.113:52822) 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 -- 172.21.15.113:0/4006746272 learned_addr learned my addr 172.21.15.113:0/4006746272 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e43e00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 0x7f5e441f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 -- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 msgr2=0x7f5e441f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e441f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 -- 172.21.15.113:0/4006746272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 msgr2=0x7f5e441f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 0x7f5e441f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 -- 172.21.15.113:0/4006746272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009400 con 0x7f5e440b6260 2024-02-02T13:24:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e43e00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 0x7f5e441f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.059+0000 7f5e3b200640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ebe0 tx=0x7f5e2c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c009760 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00cc30 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00eee0 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e34009a70 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e441f73b0 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5e2c013770 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.060+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5e2c01c450 con 0x7f5e440b6260 2024-02-02T13:24:21.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.061+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e44066cf0 con 0x7f5e440b6260 2024-02-02T13:24:21.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.063+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f5e2c021060 con 0x7f5e440b6260 2024-02-02T13:24:21.085 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.083+0000 7ff2faa43040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T13:24:21.097 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.097+0000 7f4b338d6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T13:24:21.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.137+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f5e44074120 con 0x7f5e440b6260 2024-02-02T13:24:21.145 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.143+0000 7ff2faa43040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T13:24:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.147+0000 7f5e40c00640 1 -- 172.21.15.113:0/4006746272 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e2c02a020 con 0x7f5e440b6260 2024-02-02T13:24:21.148 INFO:teuthology.orchestra.run.smithi113.stdout:2 2024-02-02T13:24:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.148+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 msgr2=0x7f5e440ab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ebe0 tx=0x7f5e2c00c520 comp rx=0 tx=0).stop 2024-02-02T13:24:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 shutdown_connections 2024-02-02T13:24:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e440b6260 0x7f5e440ab5e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e440d0070 0x7f5e441f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 --2- 172.21.15.113:0/4006746272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e440b2260 0x7f5e441f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 >> 172.21.15.113:0/4006746272 conn(0x7f5e44070660 msgr2=0x7f5e440b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 shutdown_connections 2024-02-02T13:24:21.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.149+0000 7f5e4ae00640 1 -- 172.21.15.113:0/4006746272 wait complete. 2024-02-02T13:24:21.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-02-02T13:24:21.183 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.183+0000 7f4b338d6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T13:24:21.199 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.197+0000 7ff2faa43040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T13:24:21.240 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.239+0000 7f4b338d6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T13:24:21.304 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.303+0000 7f4b338d6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T13:24:21.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.344+0000 7f9f3e200640 1 Processor -- start 2024-02-02T13:24:21.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.344+0000 7f9f3e200640 1 -- start start 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3e200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f380d2a80 0x7f9f380b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3e200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f380d17b0 con 0x7f9f3815d110 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3e200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380d25f0 con 0x7f9f380d2a80 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380b6fd0 con 0x7f9f380cff00 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f2fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f380d2a80 0x7f9f380b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52828/0 (socket says 172.21.15.113:52828) 2024-02-02T13:24:21.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3d800640 1 -- 172.21.15.113:0/1384096697 learned_addr learned my addr 172.21.15.113:0/1384096697 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f2fe00640 1 -- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 msgr2=0x7f9f380d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f2fe00640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f2fe00640 1 -- 172.21.15.113:0/1384096697 >> v1:172.21.15.91:6789/0 conn(0x7f9f3815d110 legacy=0x7f9f380cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f2fe00640 1 -- 172.21.15.113:0/1384096697 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f380d19e0 con 0x7f9f380d2a80 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.346+0000 7f9f3d800640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f2fe00640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f380d2a80 0x7f9f380b89d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f200097a0 tx=0x7f9f2002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fafbe16e712e8906 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f37e00640 1 -- 172.21.15.113:0/1384096697 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2002d040 con 0x7f9f380d2a80 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f37e00640 1 -- 172.21.15.113:0/1384096697 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20031810 con 0x7f9f380d2a80 2024-02-02T13:24:21.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f37e00640 1 -- 172.21.15.113:0/1384096697 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20031ba0 con 0x7f9f380d2a80 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f3e200640 1 -- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f380d2a80 msgr2=0x7f9f380b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.348+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f380d2a80 0x7f9f380b89d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f200097a0 tx=0x7f9f2002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 -- 172.21.15.113:0/1384096697 shutdown_connections 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380cff00 0x7f9f380d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1384096697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f380d2a80 0x7f9f380b89d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 -- 172.21.15.113:0/1384096697 >> 172.21.15.113:0/1384096697 conn(0x7f9f38070690 msgr2=0x7f9f38070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 -- 172.21.15.113:0/1384096697 shutdown_connections 2024-02-02T13:24:21.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.349+0000 7f9f3e200640 1 -- 172.21.15.113:0/1384096697 wait complete. 2024-02-02T13:24:21.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.350+0000 7f9f3e200640 1 Processor -- start 2024-02-02T13:24:21.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.350+0000 7f9f3e200640 1 -- start start 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f380cff00 0x7f9f3815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 0x7f9f381f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380b53b0 con 0x7f9f380cff00 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380b47a0 con 0x7f9f3815d110 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.351+0000 7f9f3e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380b4a80 con 0x7f9f380d2a80 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52832/0 (socket says 172.21.15.113:52832) 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f2fe00640 1 -- 172.21.15.113:0/1949356097 learned_addr learned my addr 172.21.15.113:0/1949356097 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 0x7f9f381f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3ce00640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f380cff00 0x7f9f3815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 -- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 msgr2=0x7f9f381f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 -- 172.21.15.113:0/1949356097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f380cff00 msgr2=0x7f9f3815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f380cff00 0x7f9f3815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 -- 172.21.15.113:0/1949356097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f30009b30 con 0x7f9f3815d110 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f2fe00640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.352+0000 7f9f3d800640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 0x7f9f381f6230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ef90 tx=0x7f9f3000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30018070 con 0x7f9f3815d110 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3000f040 con 0x7f9f3815d110 2024-02-02T13:24:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000cd40 con 0x7f9f3815d110 2024-02-02T13:24:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20009400 con 0x7f9f3815d110 2024-02-02T13:24:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f380b5040 con 0x7f9f3815d110 2024-02-02T13:24:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9f30003c90 con 0x7f9f3815d110 2024-02-02T13:24:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.353+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9f30014480 con 0x7f9f3815d110 2024-02-02T13:24:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.354+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3804dff0 con 0x7f9f3815d110 2024-02-02T13:24:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.357+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f9f30004840 con 0x7f9f3815d110 2024-02-02T13:24:21.361 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.361+0000 7f4b338d6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T13:24:21.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.430+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f9f38067610 con 0x7f9f3815d110 2024-02-02T13:24:21.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.440+0000 7f9f35600640 1 -- 172.21.15.113:0/1949356097 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f3002b0b0 con 0x7f9f3815d110 2024-02-02T13:24:21.441 INFO:teuthology.orchestra.run.smithi113.stdout:3 2024-02-02T13:24:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 msgr2=0x7f9f381f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 0x7f9f381f6230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ef90 tx=0x7f9f3000c490 comp rx=0 tx=0).stop 2024-02-02T13:24:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 shutdown_connections 2024-02-02T13:24:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f380d2a80 0x7f9f381f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3815d110 0x7f9f381f6230 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 --2- 172.21.15.113:0/1949356097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f380cff00 0x7f9f3815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 >> 172.21.15.113:0/1949356097 conn(0x7f9f38070690 msgr2=0x7f9f3815a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 shutdown_connections 2024-02-02T13:24:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.441+0000 7f9f3e200640 1 -- 172.21.15.113:0/1949356097 wait complete. 2024-02-02T13:24:21.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-02-02T13:24:21.645 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.644+0000 7ff2faa43040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T13:24:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.683+0000 7f9c9a800640 1 Processor -- start 2024-02-02T13:24:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.683+0000 7f9c9a800640 1 -- start start 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.684+0000 7f9c9a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9415d110 0x7f9c940b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.684+0000 7f9c9a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 0x7f9c940d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.684+0000 7f9c9a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c940d17b0 con 0x7f9c940cf7f0 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c9a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c940d25f0 con 0x7f9c9415d110 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c9a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940b6fd0 con 0x7f9c940d2a80 2024-02-02T13:24:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9415d110 0x7f9c940b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c98a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 0x7f9c940d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9415d110 0x7f9c940b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53552/0 (socket says 172.21.15.113:53552) 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 -- 172.21.15.113:0/2024477831 learned_addr learned my addr 172.21.15.113:0/2024477831 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 -- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 msgr2=0x7f9c940d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 0x7f9c940d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 -- 172.21.15.113:0/2024477831 >> v1:172.21.15.113:6789/0 conn(0x7f9c940cf7f0 legacy=0x7f9c940b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c99400640 1 -- 172.21.15.113:0/2024477831 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940d19e0 con 0x7f9c9415d110 2024-02-02T13:24:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.685+0000 7f9c98a00640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 0x7f9c940d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.686+0000 7f9c99400640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9415d110 0x7f9c940b89d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009870 tx=0x7f9c8402ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fa3a365222212e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c93e00640 1 -- 172.21.15.113:0/2024477831 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8402f830 con 0x7f9c9415d110 2024-02-02T13:24:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c93e00640 1 -- 172.21.15.113:0/2024477831 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8402f990 con 0x7f9c9415d110 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c93e00640 1 -- 172.21.15.113:0/2024477831 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8402fd20 con 0x7f9c9415d110 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 -- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9415d110 msgr2=0x7f9c940b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9415d110 0x7f9c940b89d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009870 tx=0x7f9c8402ba10 comp rx=0 tx=0).stop 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 -- 172.21.15.113:0/2024477831 shutdown_connections 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940d2a80 0x7f9c940d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2024477831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9415d110 0x7f9c940b89d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.687+0000 7f9c9a800640 1 -- 172.21.15.113:0/2024477831 >> 172.21.15.113:0/2024477831 conn(0x7f9c94070690 msgr2=0x7f9c94070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.688+0000 7f9c9a800640 1 -- 172.21.15.113:0/2024477831 shutdown_connections 2024-02-02T13:24:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.688+0000 7f9c9a800640 1 -- 172.21.15.113:0/2024477831 wait complete. 2024-02-02T13:24:21.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.689+0000 7f9c9a800640 1 Processor -- start 2024-02-02T13:24:21.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.689+0000 7f9c9a800640 1 -- start start 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 0x7f9c940b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9415dcf0 con 0x7f9c9415d110 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c941f35a0 con 0x7f9c940d2a80 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c9a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c941f2ac0 con 0x7f9c940cf7f0 2024-02-02T13:24:21.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c99400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c99400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52856/0 (socket says 172.21.15.113:52856) 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53558/0 (socket says 172.21.15.113:53558) 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.690+0000 7f9c99400640 1 -- 172.21.15.113:0/2716175956 learned_addr learned my addr 172.21.15.113:0/2716175956 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 -- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 msgr2=0x7f9c941f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c99e00640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 0x7f9c940b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 -- 172.21.15.113:0/2716175956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 msgr2=0x7f9c940b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 0x7f9c940b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 -- 172.21.15.113:0/2716175956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c009400 con 0x7f9c940d2a80 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c99400640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c99e00640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 0x7f9c940b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c98a00640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e8d0 tx=0x7f9c7c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00c940 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.691+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00caa0 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.692+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c840094d0 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.692+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00ce50 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.692+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c941f2fe0 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.692+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c7c005c90 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.692+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9c7c006530 con 0x7f9c940d2a80 2024-02-02T13:24:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.693+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9415dac0 con 0x7f9c940d2a80 2024-02-02T13:24:21.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.695+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f9c9415dac0 con 0x7f9c940d2a80 2024-02-02T13:24:21.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.772+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f9c9415de60 con 0x7f9c940d2a80 2024-02-02T13:24:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.782+0000 7f9c91600640 1 -- 172.21.15.113:0/2716175956 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c9415de60 con 0x7f9c940d2a80 2024-02-02T13:24:21.783 INFO:teuthology.orchestra.run.smithi113.stdout:4 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 msgr2=0x7f9c941f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e8d0 tx=0x7f9c7c00ed90 comp rx=0 tx=0).stop 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 shutdown_connections 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9415d110 0x7f9c940b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940d2a80 0x7f9c941f2580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 --2- 172.21.15.113:0/2716175956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940cf7f0 0x7f9c941f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.783+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 >> 172.21.15.113:0/2716175956 conn(0x7f9c94070690 msgr2=0x7f9c940b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.784+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 shutdown_connections 2024-02-02T13:24:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.784+0000 7f9c9a800640 1 -- 172.21.15.113:0/2716175956 wait complete. 2024-02-02T13:24:21.802 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-02-02T13:24:21.804 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.803+0000 7ff2faa43040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T13:24:21.828 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.827+0000 7f4b338d6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T13:24:21.926 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.924+0000 7ff2faa43040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T13:24:21.978 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:21.976+0000 7ff2faa43040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T13:24:21.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.985+0000 7f9f66a00640 1 Processor -- start 2024-02-02T13:24:21.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.985+0000 7f9f66a00640 1 -- start start 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 0x7f9f600cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9f600d05e0 0x7f9f6006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600ce450 con 0x7f9f600cf0f0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600cf5e0 con 0x7f9f600d05e0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cf880 con 0x7f9f6015cfb0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f600cf0f0 0x7f9f60160c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44204/0 (socket says 172.21.15.113:44204) 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f65600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 0x7f9f600cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.987+0000 7f9f66000640 1 -- 172.21.15.113:0/2984238263 learned_addr learned my addr 172.21.15.113:0/2984238263 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f64c00640 1 --2- 172.21.15.113:0/2984238263 >> v2:172.21.15.91:3300/0 conn(0x7f9f600d05e0 0x7f9f6006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f5fe00640 1 -- 172.21.15.113:0/2984238263 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752842428 0 0) 0x7f9f600ce450 con 0x7f9f600cf0f0 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f65600640 1 -- 172.21.15.113:0/2984238263 >> v2:172.21.15.91:3300/0 conn(0x7f9f600d05e0 msgr2=0x7f9f6006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f65600640 1 --2- 172.21.15.113:0/2984238263 >> v2:172.21.15.91:3300/0 conn(0x7f9f600d05e0 0x7f9f6006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f65600640 1 -- 172.21.15.113:0/2984238263 >> v1:172.21.15.113:6789/0 conn(0x7f9f600cf0f0 legacy=0x7f9f60160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.988+0000 7f9f65600640 1 -- 172.21.15.113:0/2984238263 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600cfb20 con 0x7f9f6015cfb0 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.989+0000 7f9f65600640 1 --2- 172.21.15.113:0/2984238263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 0x7f9f600cdf10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f500097a0 tx=0x7f9f5002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dd99810d9e1897 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.989+0000 7f9f5fe00640 1 -- 172.21.15.113:0/2984238263 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5002d040 con 0x7f9f6015cfb0 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.989+0000 7f9f5fe00640 1 -- 172.21.15.113:0/2984238263 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50031810 con 0x7f9f6015cfb0 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.989+0000 7f9f5fe00640 1 -- 172.21.15.113:0/2984238263 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50031ba0 con 0x7f9f6015cfb0 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 -- 172.21.15.113:0/2984238263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 msgr2=0x7f9f600cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 --2- 172.21.15.113:0/2984238263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 0x7f9f600cdf10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f500097a0 tx=0x7f9f5002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 -- 172.21.15.113:0/2984238263 shutdown_connections 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 --2- 172.21.15.113:0/2984238263 >> v2:172.21.15.91:3300/0 conn(0x7f9f600d05e0 0x7f9f6006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 --2- 172.21.15.113:0/2984238263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f6015cfb0 0x7f9f600cdf10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 -- 172.21.15.113:0/2984238263 >> 172.21.15.113:0/2984238263 conn(0x7f9f600d0c80 msgr2=0x7f9f600d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:21.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 -- 172.21.15.113:0/2984238263 shutdown_connections 2024-02-02T13:24:21.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.990+0000 7f9f66a00640 1 -- 172.21.15.113:0/2984238263 wait complete. 2024-02-02T13:24:21.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.991+0000 7f9f66a00640 1 Processor -- start 2024-02-02T13:24:21.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 -- start start 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f6015cfb0 0x7f9f600af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601f5eb0 con 0x7f9f6015cfb0 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601f6150 con 0x7f9f600cf0f0 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.992+0000 7f9f66a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601f6430 con 0x7f9f600d05e0 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f65600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f65600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53570/0 (socket says 172.21.15.113:53570) 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f65600640 1 -- 172.21.15.113:0/3623297499 learned_addr learned my addr 172.21.15.113:0/3623297499 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52884/0 (socket says 172.21.15.113:52884) 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f66000640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f6015cfb0 0x7f9f600af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 -- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 msgr2=0x7f9f600b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 -- 172.21.15.113:0/3623297499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f6015cfb0 msgr2=0x7f9f600af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f6015cfb0 0x7f9f600af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f64c00640 1 -- 172.21.15.113:0/3623297499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009400 con 0x7f9f600d05e0 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.993+0000 7f9f65600640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f64c00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d520 tx=0x7f9f4800d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480125f0 con 0x7f9f600d05e0 2024-02-02T13:24:21.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48012750 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f601f6710 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48012ae0 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600b56f0 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9f4801e460 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.994+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9f48014a50 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.995+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6004dff0 con 0x7f9f600d05e0 2024-02-02T13:24:21.997 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:21.996+0000 7f4b338d6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T13:24:21.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:21.997+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f9f48014cf0 con 0x7f9f600d05e0 2024-02-02T13:24:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.075+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f9f60066600 con 0x7f9f600d05e0 2024-02-02T13:24:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.084+0000 7f9f5d600640 1 -- 172.21.15.113:0/3623297499 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f480145a0 con 0x7f9f600d05e0 2024-02-02T13:24:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:5 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 msgr2=0x7f9f600b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d520 tx=0x7f9f4800d9e0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 shutdown_connections 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f600d05e0 0x7f9f600b1020 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f6015cfb0 0x7f9f600af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 --2- 172.21.15.113:0/3623297499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f600cf0f0 0x7f9f600b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 >> 172.21.15.113:0/3623297499 conn(0x7f9f600d0c80 msgr2=0x7f9f6006ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 shutdown_connections 2024-02-02T13:24:22.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.086+0000 7f9f66a00640 1 -- 172.21.15.113:0/3623297499 wait complete. 2024-02-02T13:24:22.088 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:22.085+0000 7ff2faa43040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T13:24:22.103 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-02-02T13:24:22.134 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.133+0000 7f4b338d6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T13:24:22.190 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.189+0000 7f4b338d6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T13:24:22.196 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:22.194+0000 7ff2faa43040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T13:24:22.305 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.304+0000 7f4b338d6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T13:24:22.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.324+0000 7fb116400640 1 Processor -- start 2024-02-02T13:24:22.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.324+0000 7fb116400640 1 -- start start 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb116400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 0x7fb1100b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb116400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1100cff00 0x7fb1100d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb116400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1100d17b0 con 0x7fb11015d110 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb116400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1100d25f0 con 0x7fb1100cff00 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb116400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100b6fd0 con 0x7fb1100d2a80 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb115a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1100cff00 0x7fb1100d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb10fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 0x7fb1100b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb115a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1100cff00 0x7fb1100d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53576/0 (socket says 172.21.15.113:53576) 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb10fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 0x7fb1100b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52888/0 (socket says 172.21.15.113:52888) 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.326+0000 7fb115a00640 1 -- 172.21.15.113:0/3122925399 learned_addr learned my addr 172.21.15.113:0/3122925399 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.327+0000 7fb115a00640 1 -- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 msgr2=0x7fb1100b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.327+0000 7fb115a00640 1 --2- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 0x7fb1100b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.327+0000 7fb115a00640 1 -- 172.21.15.113:0/3122925399 >> v1:172.21.15.91:6789/0 conn(0x7fb11015d110 legacy=0x7fb1100cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.327+0000 7fb115a00640 1 -- 172.21.15.113:0/3122925399 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100d19e0 con 0x7fb1100cff00 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.328+0000 7fb115a00640 1 --2- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100cff00 0x7fb1100d1270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb108009e50 tx=0x7fb10802bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7991549b824fcc64 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.328+0000 7fb10f400640 1 -- 172.21.15.113:0/3122925399 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10802cb90 con 0x7fb1100cff00 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.328+0000 7fb10f400640 1 -- 172.21.15.113:0/3122925399 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10802ccf0 con 0x7fb1100cff00 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.328+0000 7fb10f400640 1 -- 172.21.15.113:0/3122925399 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108034610 con 0x7fb1100cff00 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 -- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100cff00 msgr2=0x7fb1100d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 --2- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100cff00 0x7fb1100d1270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb108009e50 tx=0x7fb10802bff0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 -- 172.21.15.113:0/3122925399 shutdown_connections 2024-02-02T13:24:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 --2- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100cff00 0x7fb1100d1270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 --2- 172.21.15.113:0/3122925399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1100d2a80 0x7fb1100b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 -- 172.21.15.113:0/3122925399 >> 172.21.15.113:0/3122925399 conn(0x7fb110070690 msgr2=0x7fb110070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:22.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 -- 172.21.15.113:0/3122925399 shutdown_connections 2024-02-02T13:24:22.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.329+0000 7fb116400640 1 -- 172.21.15.113:0/3122925399 wait complete. 2024-02-02T13:24:22.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.331+0000 7fb116400640 1 Processor -- start 2024-02-02T13:24:22.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.331+0000 7fb116400640 1 -- start start 2024-02-02T13:24:22.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1100cff00 0x7fb1101f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 0x7fb110154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101f36e0 con 0x7fb1100cff00 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110151bc0 con 0x7fb1100d2a80 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb116400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11015b9f0 con 0x7fb11015d110 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb115a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb10fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 0x7fb110154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb115a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52894/0 (socket says 172.21.15.113:52894) 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb115a00640 1 -- 172.21.15.113:0/456904088 learned_addr learned my addr 172.21.15.113:0/456904088 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.332+0000 7fb115000640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1100cff00 0x7fb1101f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 -- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 msgr2=0x7fb110154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 -- 172.21.15.113:0/456904088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1100cff00 msgr2=0x7fb1101f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1100cff00 0x7fb1101f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 -- 172.21.15.113:0/456904088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108009b30 con 0x7fb1100d2a80 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb115a00640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10fe00640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 0x7fb110154670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e820 tx=0x7fb0f800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80095b0 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f800c940 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800ccf0 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11015bcd0 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.333+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11015c290 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.334+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb0f8005c90 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.334+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb0f8006840 con 0x7fb1100d2a80 2024-02-02T13:24:22.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.335+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11004dff0 con 0x7fb1100d2a80 2024-02-02T13:24:22.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.337+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fb0f8021060 con 0x7fb1100d2a80 2024-02-02T13:24:22.353 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:22.351+0000 7ff2faa43040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T13:24:22.401 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:22.400+0000 7ff2faa43040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T13:24:22.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.414+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fb1100b4c30 con 0x7fb1100d2a80 2024-02-02T13:24:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.422+0000 7fb10cc00640 1 -- 172.21.15.113:0/456904088 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb0f8029020 con 0x7fb1100d2a80 2024-02-02T13:24:22.423 INFO:teuthology.orchestra.run.smithi113.stdout:6 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.423+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 msgr2=0x7fb110154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.423+0000 7fb116400640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 0x7fb110154670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e820 tx=0x7fb0f800ece0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.423+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 shutdown_connections 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb11015d110 0x7fb110154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1100d2a80 0x7fb110154670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 --2- 172.21.15.113:0/456904088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1100cff00 0x7fb1101f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 >> 172.21.15.113:0/456904088 conn(0x7fb110070690 msgr2=0x7fb110070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 shutdown_connections 2024-02-02T13:24:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.424+0000 7fb116400640 1 -- 172.21.15.113:0/456904088 wait complete. 2024-02-02T13:24:22.428 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.427+0000 7f4b338d6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T13:24:22.441 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-02-02T13:24:22.556 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:22.554+0000 7ff2faa43040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T13:24:22.595 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.594+0000 7f4b338d6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T13:24:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.624+0000 7f1ec8a00640 1 Processor -- start 2024-02-02T13:24:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.625+0000 7f1ec8a00640 1 -- start start 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec00b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec8a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec0161090 con 0x7f1ec00b2470 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec0161330 con 0x7f1ec00d3850 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0161cd0 con 0x7f1ec015d330 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec00b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1ec00d3850 0x7f1ec00d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44208/0 (socket says 172.21.15.113:44208) 2024-02-02T13:24:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.626+0000 7f1ec6a00640 1 -- 172.21.15.113:0/234249782 learned_addr learned my addr 172.21.15.113:0/234249782 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-02-02T13:24:22.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.627+0000 7f1ec6000640 1 -- 172.21.15.113:0/234249782 >> v1:172.21.15.113:6789/0 conn(0x7f1ec00d3850 legacy=0x7f1ec00d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.627+0000 7f1ec6000640 1 -- 172.21.15.113:0/234249782 >> v1:172.21.15.91:6789/0 conn(0x7f1ec00b2470 legacy=0x7f1ec00d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.627+0000 7f1ec6000640 1 -- 172.21.15.113:0/234249782 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0161610 con 0x7f1ec015d330 2024-02-02T13:24:22.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.627+0000 7f1ec4c00640 1 -- 172.21.15.113:0/234249782 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915984568 0 0) 0x7f1ec0161330 con 0x7f1ec00d3850 2024-02-02T13:24:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.628+0000 7f1ec6000640 1 --2- 172.21.15.113:0/234249782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec00b8b40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097c0 tx=0x7f1eb802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8238f46bc98bfab1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.628+0000 7f1ec4c00640 1 -- 172.21.15.113:0/234249782 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8031760 con 0x7f1ec015d330 2024-02-02T13:24:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.628+0000 7f1ec4c00640 1 -- 172.21.15.113:0/234249782 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb80318c0 con 0x7f1ec015d330 2024-02-02T13:24:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.628+0000 7f1ec4c00640 1 -- 172.21.15.113:0/234249782 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8031c50 con 0x7f1ec015d330 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 -- 172.21.15.113:0/234249782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 msgr2=0x7f1ec00b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/234249782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec00b8b40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097c0 tx=0x7f1eb802b800 comp rx=0 tx=0).stop 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 -- 172.21.15.113:0/234249782 shutdown_connections 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/234249782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec00b8b40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 -- 172.21.15.113:0/234249782 >> 172.21.15.113:0/234249782 conn(0x7f1ec0070660 msgr2=0x7f1ec0070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 -- 172.21.15.113:0/234249782 shutdown_connections 2024-02-02T13:24:22.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.629+0000 7f1ec8a00640 1 -- 172.21.15.113:0/234249782 wait complete. 2024-02-02T13:24:22.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.631+0000 7f1ec8a00640 1 Processor -- start 2024-02-02T13:24:22.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.631+0000 7f1ec8a00640 1 -- start start 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ec00b2470 0x7f1ec0157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 0x7f1ec00b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b5820 con 0x7f1ec00b2470 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00c9300 con 0x7f1ec00d3850 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01f6ab0 con 0x7f1ec015d330 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 0x7f1ec00b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:52924/0 (socket says 172.21.15.113:52924) 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec6a00640 1 -- 172.21.15.113:0/2532650103 learned_addr learned my addr 172.21.15.113:0/2532650103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-02T13:24:22.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec6000640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ec00b2470 0x7f1ec0157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 -- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 msgr2=0x7f1ec01f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 -- 172.21.15.113:0/2532650103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ec00b2470 msgr2=0x7f1ec0157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ec00b2470 0x7f1ec0157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec5600640 1 -- 172.21.15.113:0/2532650103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80093a0 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.632+0000 7f1ec6a00640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ec5600640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 0x7f1ec00b2af0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c390 tx=0x7f1ea800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80052d0 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8005430 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec00b5540 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8003830 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec00c9880 con 0x7f1ec00d3850 2024-02-02T13:24:22.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ea8003c90 con 0x7f1ec00d3850 2024-02-02T13:24:22.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.633+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1ea8011e00 con 0x7f1ec00d3850 2024-02-02T13:24:22.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.634+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0067400 con 0x7f1ec00d3850 2024-02-02T13:24:22.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.636+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f1ea8017070 con 0x7f1ec00d3850 2024-02-02T13:24:22.648 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.647+0000 7f4b338d6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T13:24:22.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.713+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f1ec00d3600 con 0x7f1ec00d3850 2024-02-02T13:24:22.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.723+0000 7f1ebe000640 1 -- 172.21.15.113:0/2532650103 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ea8011590 con 0x7f1ec00d3850 2024-02-02T13:24:22.724 INFO:teuthology.orchestra.run.smithi113.stdout:7 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 msgr2=0x7f1ec00b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 0x7f1ec00b2af0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c390 tx=0x7f1ea800c850 comp rx=0 tx=0).stop 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 shutdown_connections 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ec015d330 0x7f1ec01f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ec00d3850 0x7f1ec00b2af0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 --2- 172.21.15.113:0/2532650103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ec00b2470 0x7f1ec0157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 >> 172.21.15.113:0/2532650103 conn(0x7f1ec0070660 msgr2=0x7f1ec00d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 shutdown_connections 2024-02-02T13:24:22.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-02T13:24:22.725+0000 7f1ec8a00640 1 -- 172.21.15.113:0/2532650103 wait complete. 2024-02-02T13:24:22.741 INFO:tasks.ceph.osd.0:Restarting daemon 2024-02-02T13:24:22.741 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-02-02T13:24:22.748 INFO:tasks.ceph.osd.0:Started 2024-02-02T13:24:22.748 INFO:tasks.ceph.osd.1:Restarting daemon 2024-02-02T13:24:22.748 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-02-02T13:24:22.755 INFO:tasks.ceph.osd.1:Started 2024-02-02T13:24:22.755 INFO:tasks.ceph.osd.2:Restarting daemon 2024-02-02T13:24:22.755 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-02-02T13:24:22.763 INFO:tasks.ceph.osd.2:Started 2024-02-02T13:24:22.763 INFO:tasks.ceph.osd.3:Restarting daemon 2024-02-02T13:24:22.763 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-02-02T13:24:22.769 INFO:tasks.ceph.osd.3:Started 2024-02-02T13:24:22.769 INFO:tasks.ceph.osd.4:Restarting daemon 2024-02-02T13:24:22.770 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-02-02T13:24:22.776 INFO:tasks.ceph.osd.4:Started 2024-02-02T13:24:22.777 INFO:tasks.ceph.osd.5:Restarting daemon 2024-02-02T13:24:22.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-02-02T13:24:22.783 INFO:tasks.ceph.osd.5:Started 2024-02-02T13:24:22.783 INFO:tasks.ceph.osd.6:Restarting daemon 2024-02-02T13:24:22.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-02-02T13:24:22.788 INFO:tasks.ceph.osd.6:Started 2024-02-02T13:24:22.788 INFO:tasks.ceph.osd.7:Restarting daemon 2024-02-02T13:24:22.789 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-02-02T13:24:22.794 INFO:tasks.ceph.osd.7:Started 2024-02-02T13:24:22.794 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:22.820 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:22.819+0000 7f4b338d6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T13:24:22.942 INFO:tasks.ceph.osd.3.smithi091.stderr:2024-02-02T13:24:22.940+0000 7f6cfe2fa500 -1 Falling back to public interface 2024-02-02T13:24:22.943 INFO:tasks.ceph.osd.1.smithi091.stderr:2024-02-02T13:24:22.942+0000 7fe6b5dab500 -1 Falling back to public interface 2024-02-02T13:24:22.944 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-02-02T13:24:22.944+0000 7fa98828c500 -1 Falling back to public interface 2024-02-02T13:24:22.946 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-02-02T13:24:22.945+0000 7fef6ea8c500 -1 Falling back to public interface 2024-02-02T13:24:22.952 INFO:tasks.ceph.osd.2.smithi091.stderr:2024-02-02T13:24:22.951+0000 7f4bf18ad500 -1 Falling back to public interface 2024-02-02T13:24:22.958 INFO:tasks.ceph.osd.0.smithi091.stderr:2024-02-02T13:24:22.957+0000 7f4817afa500 -1 Falling back to public interface 2024-02-02T13:24:22.960 INFO:tasks.ceph.osd.2.smithi091.stderr:2024-02-02T13:24:22.958+0000 7f4bf0400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6820/0: (98) Address already in use 2024-02-02T13:24:22.963 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-02-02T13:24:22.962+0000 7fa98828c500 -1 osd.5 0 log_to_monitors true 2024-02-02T13:24:22.965 INFO:tasks.ceph.osd.1.smithi091.stderr:2024-02-02T13:24:22.964+0000 7fe6b5dab500 -1 osd.1 0 log_to_monitors true 2024-02-02T13:24:22.965 INFO:tasks.ceph.osd.3.smithi091.stderr:2024-02-02T13:24:22.964+0000 7f6cfe2fa500 -1 osd.3 0 log_to_monitors true 2024-02-02T13:24:22.966 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-02-02T13:24:22.965+0000 7f3fea012500 -1 Falling back to public interface 2024-02-02T13:24:22.967 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-02-02T13:24:22.967+0000 7fef6ea8c500 -1 osd.7 0 log_to_monitors true 2024-02-02T13:24:22.976 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-02-02T13:24:22.975+0000 7f7a19620500 -1 Falling back to public interface 2024-02-02T13:24:22.982 INFO:tasks.ceph.osd.2.smithi091.stderr:2024-02-02T13:24:22.981+0000 7f4bf18ad500 -1 osd.2 0 log_to_monitors true 2024-02-02T13:24:22.985 INFO:tasks.ceph.osd.0.smithi091.stderr:2024-02-02T13:24:22.984+0000 7f4817afa500 -1 osd.0 0 log_to_monitors true 2024-02-02T13:24:22.992 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-02-02T13:24:22.992+0000 7f3fea012500 -1 osd.6 0 log_to_monitors true 2024-02-02T13:24:23.005 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-02-02T13:24:23.004+0000 7f7a19620500 -1 osd.4 0 log_to_monitors true 2024-02-02T13:24:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.042+0000 7f7bd1800640 1 Processor -- start 2024-02-02T13:24:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.042+0000 7f7bd1800640 1 -- start start 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bcc0d2a80 0x7f7bcc0d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 0x7f7bcc0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0d01c0 con 0x7f7bcc0cf7f0 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0d17b0 con 0x7f7bcc0d2a80 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0d25f0 con 0x7f7bcc15d110 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bd0e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 0x7f7bcc0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bcc0d2a80 0x7f7bcc0d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:50512/0 (socket says 172.21.15.91:50512) 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.043+0000 7f7bcbe00640 1 -- 172.21.15.91:0/2406730458 learned_addr learned my addr 172.21.15.91:0/2406730458 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bcbe00640 1 -- 172.21.15.91:0/2406730458 >> v2:172.21.15.113:3300/0 conn(0x7f7bcc0d2a80 msgr2=0x7f7bcc0d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bcbe00640 1 --2- 172.21.15.91:0/2406730458 >> v2:172.21.15.113:3300/0 conn(0x7f7bcc0d2a80 0x7f7bcc0d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bcbe00640 1 -- 172.21.15.91:0/2406730458 >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 msgr2=0x7f7bcc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bcbe00640 1 --2- 172.21.15.91:0/2406730458 >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 0x7f7bcc0cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bcbe00640 1 -- 172.21.15.91:0/2406730458 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0d19e0 con 0x7f7bcc15d110 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.044+0000 7f7bd0e00640 1 --2- 172.21.15.91:0/2406730458 >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 0x7f7bcc0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.045+0000 7f7bcbe00640 1 --2- 172.21.15.91:0/2406730458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06b9a0 tx=0x7f7bbc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef95357d9b8476aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:23.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.045+0000 7f7bcaa00640 1 -- 172.21.15.91:0/2406730458 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc030840 con 0x7f7bcc15d110 2024-02-02T13:24:23.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.045+0000 7f7bcaa00640 1 -- 172.21.15.91:0/2406730458 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0309a0 con 0x7f7bcc15d110 2024-02-02T13:24:23.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.045+0000 7f7bcaa00640 1 -- 172.21.15.91:0/2406730458 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc030d30 con 0x7f7bcc15d110 2024-02-02T13:24:23.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 -- 172.21.15.91:0/2406730458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 msgr2=0x7f7bcc0b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 --2- 172.21.15.91:0/2406730458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc06b9a0 tx=0x7f7bbc02b980 comp rx=0 tx=0).stop 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 -- 172.21.15.91:0/2406730458 shutdown_connections 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 --2- 172.21.15.91:0/2406730458 >> v2:172.21.15.91:3300/0 conn(0x7f7bcc0cf7f0 0x7f7bcc0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 --2- 172.21.15.91:0/2406730458 >> v2:172.21.15.113:3300/0 conn(0x7f7bcc0d2a80 0x7f7bcc0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 --2- 172.21.15.91:0/2406730458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc15d110 0x7f7bcc0b89d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 -- 172.21.15.91:0/2406730458 >> 172.21.15.91:0/2406730458 conn(0x7f7bcc070690 msgr2=0x7f7bcc070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 -- 172.21.15.91:0/2406730458 shutdown_connections 2024-02-02T13:24:23.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.046+0000 7f7bd1800640 1 -- 172.21.15.91:0/2406730458 wait complete. 2024-02-02T13:24:23.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.047+0000 7f7bd1800640 1 Processor -- start 2024-02-02T13:24:23.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.048+0000 7f7bd1800640 1 -- start start 2024-02-02T13:24:23.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.048+0000 7f7bd1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 0x7f7bcc14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.048+0000 7f7bd1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bcc0d2a80 0x7f7bcc0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.048+0000 7f7bd1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b3780 con 0x7f7bcc15d110 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1e70f0 con 0x7f7bcc0d2a80 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1e5d60 con 0x7f7bcc0cf7f0 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 0x7f7bcc14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50818/0 (socket says 172.21.15.91:50818) 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 -- 172.21.15.91:0/1808248548 learned_addr learned my addr 172.21.15.91:0/1808248548 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bcb400640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bcc0d2a80 0x7f7bcc0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 -- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 msgr2=0x7f7bcc14f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 0x7f7bcc14f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 -- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bcc0d2a80 msgr2=0x7f7bcc0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bcc0d2a80 0x7f7bcc0b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bd0e00640 1 -- 172.21.15.91:0/1808248548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4009a70 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.049+0000 7f7bcbe00640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 0x7f7bcc14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bd0e00640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400eed0 tx=0x7f7bc400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4010040 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc400bd20 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc0094d0 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4003bb0 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.050+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1e6320 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.051+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7bc401d710 con 0x7f7bcc15d110 2024-02-02T13:24:23.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.051+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7bc401c3b0 con 0x7f7bcc15d110 2024-02-02T13:24:23.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.051+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc0c9ed0 con 0x7f7bcc15d110 2024-02-02T13:24:23.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.054+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f7bcc0c9ed0 con 0x7f7bcc15d110 2024-02-02T13:24:23.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.116+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bcc04dff0 con 0x7f7bcc15d110 2024-02-02T13:24:23.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.117+0000 7f7bc3e00640 1 -- 172.21.15.91:0/1808248548 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7bc4018030 con 0x7f7bcc15d110 2024-02-02T13:24:23.119 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:23.119 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":10,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:22.715632+0000","last_up_change":"0.000000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 msgr2=0x7f7bcc1e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400eed0 tx=0x7f7bc400b480 comp rx=0 tx=0).stop 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 shutdown_connections 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bcc15d110 0x7f7bcc1e7550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bcc0d2a80 0x7f7bcc0b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 --2- 172.21.15.91:0/1808248548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bcc0cf7f0 0x7f7bcc14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:23.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 >> 172.21.15.91:0/1808248548 conn(0x7f7bcc070690 msgr2=0x7f7bcc14fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:23.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 shutdown_connections 2024-02-02T13:24:23.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:23.118+0000 7f7bd1800640 1 -- 172.21.15.91:0/1808248548 wait complete. 2024-02-02T13:24:23.137 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-02-02T13:24:23.137 INFO:tasks.ceph:Waiting for OSDs to come up 2024-02-02T13:24:23.699 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:23.698+0000 7ff2faa43040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T13:24:23.941 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:23.944 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:23.944+0000 7f4b338d6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T13:24:24.029 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:24.027+0000 7ff2faa43040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T13:24:24.090 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:24.089+0000 7ff2faa43040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T13:24:24.105 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.104+0000 7f1c02800640 1 Processor -- start 2024-02-02T13:24:24.106 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.104+0000 7f1c02800640 1 -- start start 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bfc15d210 0x7f1bfc0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1bfc0b8e70 0x7f1bfc0d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0d3a30 0x7f1bfc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc0d3ea0 con 0x7f1bfc15d210 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc161a30 con 0x7f1bfc0b8e70 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.105+0000 7f1c02800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc160e60 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bfc15d210 0x7f1bfc0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bfc15d210 0x7f1bfc0b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50824/0 (socket says 172.21.15.91:50824) 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 -- 172.21.15.91:0/879948705 learned_addr learned my addr 172.21.15.91:0/879948705 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfaa00640 1 --2- 172.21.15.91:0/879948705 >> v2:172.21.15.113:3300/0 conn(0x7f1bfc0b8e70 0x7f1bfc0d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfbe00640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0d3a30 0x7f1bfc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 -- 172.21.15.91:0/879948705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0d3a30 msgr2=0x7f1bfc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:24.108 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0d3a30 0x7f1bfc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 -- 172.21.15.91:0/879948705 >> v2:172.21.15.113:3300/0 conn(0x7f1bfc0b8e70 msgr2=0x7f1bfc0d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 --2- 172.21.15.91:0/879948705 >> v2:172.21.15.113:3300/0 conn(0x7f1bfc0b8e70 0x7f1bfc0d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.106+0000 7f1bfb400640 1 -- 172.21.15.91:0/879948705 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc161100 con 0x7f1bfc15d210 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.107+0000 7f1bfb400640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc15d210 0x7f1bfc0b8930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc08c4f0 tx=0x7f1bec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee4c2e59ddc7a3d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.108+0000 7f1bfa000640 1 -- 172.21.15.91:0/879948705 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec031870 con 0x7f1bfc15d210 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.108+0000 7f1bfa000640 1 -- 172.21.15.91:0/879948705 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec0319d0 con 0x7f1bfc15d210 2024-02-02T13:24:24.109 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.108+0000 7f1bfa000640 1 -- 172.21.15.91:0/879948705 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec031d60 con 0x7f1bfc15d210 2024-02-02T13:24:24.110 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.108+0000 7f1c02800640 1 -- 172.21.15.91:0/879948705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc15d210 msgr2=0x7f1bfc0b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:24.110 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.108+0000 7f1c02800640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc15d210 0x7f1bfc0b8930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc08c4f0 tx=0x7f1bec02b910 comp rx=0 tx=0).stop 2024-02-02T13:24:24.110 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 -- 172.21.15.91:0/879948705 shutdown_connections 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0d3a30 0x7f1bfc0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 --2- 172.21.15.91:0/879948705 >> v2:172.21.15.113:3300/0 conn(0x7f1bfc0b8e70 0x7f1bfc0d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 --2- 172.21.15.91:0/879948705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc15d210 0x7f1bfc0b8930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 -- 172.21.15.91:0/879948705 >> 172.21.15.91:0/879948705 conn(0x7f1bfc070470 msgr2=0x7f1bfc070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 -- 172.21.15.91:0/879948705 shutdown_connections 2024-02-02T13:24:24.111 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.109+0000 7f1c02800640 1 -- 172.21.15.91:0/879948705 wait complete. 2024-02-02T13:24:24.112 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.110+0000 7f1c02800640 1 Processor -- start 2024-02-02T13:24:24.112 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.111+0000 7f1c02800640 1 -- start start 2024-02-02T13:24:24.114 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.111+0000 7f1c02800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc0b8e70 0x7f1bfc0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.114 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.111+0000 7f1c02800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.111+0000 7f1c02800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 0x7f1bfc157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1c02800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc15ee40 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1c02800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc158100 con 0x7f1bfc0b8e70 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1c02800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0c81b0 con 0x7f1bfc15d210 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50836/0 (socket says 172.21.15.91:50836) 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 -- 172.21.15.91:0/907413661 learned_addr learned my addr 172.21.15.91:0/907413661 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfbe00640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 0x7f1bfc157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 -- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 msgr2=0x7f1bfc157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 0x7f1bfc157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 -- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc0b8e70 msgr2=0x7f1bfc0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc0b8e70 0x7f1bfc0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 -- 172.21.15.91:0/907413661 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec0094d0 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.115 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfbe00640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 0x7f1bfc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.112+0000 7f1bfaa00640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1be400d7d0 tx=0x7f1be400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be40096a0 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be4009800 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc0c8490 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4009b90 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc0c8a50 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1be401e460 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.113+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7f1be4014750 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.116 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.114+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc073c50 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.118 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.116+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f1be40149c0 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.183 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.181+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bfc04dff0 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.183+0000 7f1bf3400640 1 -- 172.21.15.91:0/907413661 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1be4014b60 con 0x7f1bfc0d3a30 2024-02-02T13:24:24.185 INFO:teuthology.misc.health.smithi091.stdout: 2024-02-02T13:24:24.185 INFO:teuthology.misc.health.smithi091.stdout:{"epoch":11,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:23.719129+0000","last_up_change":"0.000000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:24.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.184+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 msgr2=0x7f1bfc160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:24.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.184+0000 7f1c02800640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1be400d7d0 tx=0x7f1be400dc90 comp rx=0 tx=0).stop 2024-02-02T13:24:24.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.184+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 shutdown_connections 2024-02-02T13:24:24.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.184+0000 7f1c02800640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc15d210 0x7f1bfc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.184+0000 7f1c02800640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bfc0d3a30 0x7f1bfc160210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.185+0000 7f1c02800640 1 --2- 172.21.15.91:0/907413661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc0b8e70 0x7f1bfc0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:24.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.185+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 >> 172.21.15.91:0/907413661 conn(0x7f1bfc070470 msgr2=0x7f1bfc0cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:24.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.185+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 shutdown_connections 2024-02-02T13:24:24.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:24.185+0000 7f1c02800640 1 -- 172.21.15.91:0/907413661 wait complete. 2024-02-02T13:24:24.198 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-02-02T13:24:24.208 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:24.207+0000 7ff2faa43040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T13:24:24.250 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:24.249+0000 7f4b338d6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T13:24:24.311 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:24.311+0000 7f4b338d6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T13:24:24.422 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-02-02T13:24:24.421+0000 7f4b338d6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T13:24:24.763 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-02-02T13:24:24.762+0000 7fa983200640 -1 osd.5 0 waiting for initial osdmap 2024-02-02T13:24:24.765 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-02-02T13:24:24.764+0000 7f7a14000640 -1 osd.4 0 waiting for initial osdmap 2024-02-02T13:24:24.765 INFO:tasks.ceph.osd.1.smithi091.stderr:2024-02-02T13:24:24.763+0000 7fe6b0c00640 -1 osd.1 0 waiting for initial osdmap 2024-02-02T13:24:24.765 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-02-02T13:24:24.765+0000 7f3fe4e00640 -1 osd.6 0 waiting for initial osdmap 2024-02-02T13:24:24.767 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-02-02T13:24:24.766+0000 7fa97b000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.767 INFO:tasks.ceph.osd.0.smithi091.stderr:2024-02-02T13:24:24.765+0000 7f4812a00640 -1 osd.0 0 waiting for initial osdmap 2024-02-02T13:24:24.767 INFO:tasks.ceph.osd.1.smithi091.stderr:2024-02-02T13:24:24.766+0000 7fe6a8a00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.768 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-02-02T13:24:24.767+0000 7f7a0be00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.768 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-02-02T13:24:24.767+0000 7f3fdcc00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.770 INFO:tasks.ceph.osd.0.smithi091.stderr:2024-02-02T13:24:24.769+0000 7f480a800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.774 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-02-02T13:24:24.773+0000 7fef69a00640 -1 osd.7 0 waiting for initial osdmap 2024-02-02T13:24:24.776 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-02-02T13:24:24.775+0000 7fef61800640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.793 INFO:tasks.ceph.osd.3.smithi091.stderr:2024-02-02T13:24:24.791+0000 7f6cf9200640 -1 osd.3 0 waiting for initial osdmap 2024-02-02T13:24:24.795 INFO:tasks.ceph.osd.2.smithi091.stderr:2024-02-02T13:24:24.794+0000 7f4bec800640 -1 osd.2 0 waiting for initial osdmap 2024-02-02T13:24:24.795 INFO:tasks.ceph.osd.3.smithi091.stderr:2024-02-02T13:24:24.794+0000 7f6cf1000640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:24.798 INFO:tasks.ceph.osd.2.smithi091.stderr:2024-02-02T13:24:24.797+0000 7f4be4600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T13:24:25.273 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:25.271+0000 7ff29ba00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-02T13:24:25.275 INFO:tasks.ceph.mgr.y.smithi091.stderr:2024-02-02T13:24:25.274+0000 7ff29ba00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-02T13:24:28.155 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:24:28.164 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:24:31.005 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:31.179 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.178+0000 7fbe91e00640 1 Processor -- start 2024-02-02T13:24:31.179 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.178+0000 7fbe91e00640 1 -- start start 2024-02-02T13:24:31.180 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe91e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe8c15b0a0 0x7fbe8c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.181 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe91e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe8c0b0f80 0x7fbe8c06b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.181 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe91e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c06f840 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.181 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe91e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c06f630 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.181 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe91e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c06fb20 con 0x7fbe8c15b0a0 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.179+0000 7fbe8b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe8c0b0f80 0x7fbe8c06b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe8c0b0f80 0x7fbe8c06b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54456/0 (socket says 172.21.15.91:54456) 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 -- 172.21.15.91:0/20246170 learned_addr learned my addr 172.21.15.91:0/20246170 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8aa00640 1 --2- 172.21.15.91:0/20246170 >> v2:172.21.15.113:3300/0 conn(0x7fbe8c15b0a0 0x7fbe8c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 -- 172.21.15.91:0/20246170 >> v2:172.21.15.113:3300/0 conn(0x7fbe8c15b0a0 msgr2=0x7fbe8c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 --2- 172.21.15.91:0/20246170 >> v2:172.21.15.113:3300/0 conn(0x7fbe8c15b0a0 0x7fbe8c15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 -- 172.21.15.91:0/20246170 >> v1:172.21.15.91:6789/0 conn(0x7fbe8c0b1be0 legacy=0x7fbe8c0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8b400640 1 -- 172.21.15.91:0/20246170 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0714e0 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.182 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.180+0000 7fbe8a000640 1 -- 172.21.15.91:0/20246170 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419074299 0 0) 0x7fbe8c06f840 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.183 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.181+0000 7fbe8b400640 1 --2- 172.21.15.91:0/20246170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c06b830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0b1930 tx=0x7fbe8402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ba378e3e42b30d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.183 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.182+0000 7fbe8a000640 1 -- 172.21.15.91:0/20246170 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8402cb20 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.183 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.182+0000 7fbe8a000640 1 -- 172.21.15.91:0/20246170 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8402cc80 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.183 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.182+0000 7fbe8a000640 1 -- 172.21.15.91:0/20246170 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe840343f0 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.182+0000 7fbe91e00640 1 -- 172.21.15.91:0/20246170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b0f80 msgr2=0x7fbe8c06b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.182+0000 7fbe91e00640 1 --2- 172.21.15.91:0/20246170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c06b830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0b1930 tx=0x7fbe8402bf80 comp rx=0 tx=0).stop 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 -- 172.21.15.91:0/20246170 shutdown_connections 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 --2- 172.21.15.91:0/20246170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c06b830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 --2- 172.21.15.91:0/20246170 >> v2:172.21.15.113:3300/0 conn(0x7fbe8c15b0a0 0x7fbe8c15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 -- 172.21.15.91:0/20246170 >> 172.21.15.91:0/20246170 conn(0x7fbe8c0ced70 msgr2=0x7fbe8c0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:31.184 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 -- 172.21.15.91:0/20246170 shutdown_connections 2024-02-02T13:24:31.185 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.183+0000 7fbe91e00640 1 -- 172.21.15.91:0/20246170 wait complete. 2024-02-02T13:24:31.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.185+0000 7fbe91e00640 1 Processor -- start 2024-02-02T13:24:31.186 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.185+0000 7fbe91e00640 1 -- start start 2024-02-02T13:24:31.187 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe8c15b0a0 0x7fbe8c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c6ea0 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c6290 con 0x7fbe8c0b0f80 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe91e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c6570 con 0x7fbe8c15b0a0 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe81e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe81e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54462/0 (socket says 172.21.15.91:54462) 2024-02-02T13:24:31.188 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.186+0000 7fbe81e00640 1 -- 172.21.15.91:0/2111397826 learned_addr learned my addr 172.21.15.91:0/2111397826 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:31.189 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe8aa00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c0cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.189 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe8b400640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe8c15b0a0 0x7fbe8c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.189 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 -- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe8c15b0a0 msgr2=0x7fbe8c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.189 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe8c15b0a0 0x7fbe8c1588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 -- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 msgr2=0x7fbe8c0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c0cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 -- 172.21.15.91:0/2111397826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84009b30 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe8aa00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe81e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c4b0 tx=0x7fbe7400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74005480 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe740035f0 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740039e0 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c0c6850 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.187+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c0c5020 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.190 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.189+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c073c50 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.191 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.190+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbe74003b40 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.192 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.190+0000 7fbe83e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 0x7fbe6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.192 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.191+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe74087c90 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.192 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.191+0000 7fbe8aa00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 0x7fbe6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.192 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.191+0000 7fbe8aa00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 0x7fbe6406c8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.197 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.196+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbe74053530 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.388 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.386+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe8c04dff0 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.390 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.388+0000 7fbe83e00640 1 -- 172.21.15.91:0/2111397826 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fbe74017020 con 0x7fbe8c0b1be0 2024-02-02T13:24:31.390 INFO:teuthology.misc.health.smithi091.stdout: 2024-02-02T13:24:31.390 INFO:teuthology.misc.health.smithi091.stdout:{"epoch":16,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:28.801621+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:31.392 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 msgr2=0x7fbe6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 0x7fbe6406c8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 msgr2=0x7fbe8c0a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c4b0 tx=0x7fbe7400c970 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 shutdown_connections 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe6406a430 0x7fbe6406c8f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe8c15b0a0 0x7fbe8c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe8c0b1be0 0x7fbe8c0a84c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 --2- 172.21.15.91:0/2111397826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8c0b0f80 0x7fbe8c0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 >> 172.21.15.91:0/2111397826 conn(0x7fbe8c0ced70 msgr2=0x7fbe8c0afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 shutdown_connections 2024-02-02T13:24:31.393 INFO:teuthology.misc.health.smithi091.stderr:2024-02-02T13:24:31.391+0000 7fbe91e00640 1 -- 172.21.15.91:0/2111397826 wait complete. 2024-02-02T13:24:31.407 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-02-02T13:24:31.407 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-02-02T13:24:31.407 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:24:31.408 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:24:31.412 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:24:31.412 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:24:31.412 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:24:31.419 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:24:31.419 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:24:31.419 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:24:31.427 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:24:31.427 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:24:31.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:24:31.434 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:24:31.434 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-02-02T13:24:31.434 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:31.570 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:24:31.571 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:24:31.592 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:24:31.592 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:24:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.621+0000 7f63d3800640 1 Processor -- start 2024-02-02T13:24:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.621+0000 7f63d3800640 1 -- start start 2024-02-02T13:24:31.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.622+0000 7f63d3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 0x7f63cc0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.622+0000 7f63d3800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 0x7f63cc0af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.622+0000 7f63d3800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc0af6b0 con 0x7f63cc15d2c0 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.622+0000 7f63d3800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0cf4f0 con 0x7f63cc070e50 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.622+0000 7f63d3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0cf790 con 0x7f63cc066020 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63d1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 0x7f63cc0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 0x7f63cc0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63d1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 0x7f63cc0af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:59754/0 (socket says 172.21.15.91:59754) 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63d1800640 1 -- 172.21.15.91:0/2592709390 learned_addr learned my addr 172.21.15.91:0/2592709390 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63cbe00640 1 -- 172.21.15.91:0/2592709390 >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 msgr2=0x7f63cc0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63cbe00640 1 --2- 172.21.15.91:0/2592709390 >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 0x7f63cc0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63cbe00640 1 -- 172.21.15.91:0/2592709390 >> v1:172.21.15.113:6789/0 conn(0x7f63cc15d2c0 legacy=0x7f63cc065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.623+0000 7f63cbe00640 1 -- 172.21.15.91:0/2592709390 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc06ac20 con 0x7f63cc066020 2024-02-02T13:24:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.624+0000 7f63cbe00640 1 --2- 172.21.15.91:0/2592709390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 0x7f63cc0cdd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63b40097a0 tx=0x7f63b402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7988a9c79063ef13 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.624+0000 7f63cb400640 1 -- 172.21.15.91:0/2592709390 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4031780 con 0x7f63cc066020 2024-02-02T13:24:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.624+0000 7f63cb400640 1 -- 172.21.15.91:0/2592709390 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b40318e0 con 0x7f63cc066020 2024-02-02T13:24:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.624+0000 7f63cb400640 1 -- 172.21.15.91:0/2592709390 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4031c70 con 0x7f63cc066020 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 -- 172.21.15.91:0/2592709390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 msgr2=0x7f63cc0cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 --2- 172.21.15.91:0/2592709390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 0x7f63cc0cdd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63b40097a0 tx=0x7f63b402b820 comp rx=0 tx=0).stop 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 -- 172.21.15.91:0/2592709390 shutdown_connections 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 --2- 172.21.15.91:0/2592709390 >> v2:172.21.15.113:3300/0 conn(0x7f63cc070e50 0x7f63cc0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 --2- 172.21.15.91:0/2592709390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc066020 0x7f63cc0cdd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 -- 172.21.15.91:0/2592709390 >> 172.21.15.91:0/2592709390 conn(0x7f63cc0cfdc0 msgr2=0x7f63cc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 -- 172.21.15.91:0/2592709390 shutdown_connections 2024-02-02T13:24:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.625+0000 7f63d3800640 1 -- 172.21.15.91:0/2592709390 wait complete. 2024-02-02T13:24:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.627+0000 7f63d3800640 1 Processor -- start 2024-02-02T13:24:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.627+0000 7f63d3800640 1 -- start start 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63cc066020 0x7f63cc14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc15d2c0 0x7f63cc15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1493d0 con 0x7f63cc070e50 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1496b0 con 0x7f63cc066020 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc149990 con 0x7f63cc15d2c0 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54502/0 (socket says 172.21.15.91:54502) 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 -- 172.21.15.91:0/3274029366 learned_addr learned my addr 172.21.15.91:0/3274029366 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d1800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc15d2c0 0x7f63cc15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63d0e00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63cc066020 0x7f63cc14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 -- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc15d2c0 msgr2=0x7f63cc15d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc15d2c0 0x7f63cc15d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 -- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63cc066020 msgr2=0x7f63cc14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63cc066020 0x7f63cc14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.628+0000 7f63cbe00640 1 -- 172.21.15.91:0/3274029366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009400 con 0x7f63cc070e50 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63cbe00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63b4034040 tx=0x7f63b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4032820 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b40329a0 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4032d50 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc149c70 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.629+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc1f2ae0 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.631+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc14a190 con 0x7f63cc070e50 2024-02-02T13:24:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.631+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f63b404e440 con 0x7f63cc070e50 2024-02-02T13:24:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.632+0000 7f63c8c00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 0x7f63a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.632+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63b40aa500 con 0x7f63cc070e50 2024-02-02T13:24:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.632+0000 7f63d0e00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 0x7f63a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.633+0000 7f63d0e00640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 0x7f63a406c8f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000f60 tx=0x7f63bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:31.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.637+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f63cc14a190 con 0x7f63cc070e50 2024-02-02T13:24:31.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.829+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63cc14a190 con 0x7f63cc070e50 2024-02-02T13:24:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.831+0000 7f63c8c00640 1 -- 172.21.15.91:0/3274029366 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f63cc14a190 con 0x7f63cc070e50 2024-02-02T13:24:31.833 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:31.833 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:28.801621+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:31.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 msgr2=0x7f63a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 0x7f63a406c8f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000f60 tx=0x7f63bc009040 comp rx=0 tx=0).stop 2024-02-02T13:24:31.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 msgr2=0x7f63cc1f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63b4034040 tx=0x7f63b40046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 shutdown_connections 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f63a406a430 0x7f63a406c8f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63cc15d2c0 0x7f63cc15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63cc070e50 0x7f63cc1f25a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 --2- 172.21.15.91:0/3274029366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63cc066020 0x7f63cc14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 >> 172.21.15.91:0/3274029366 conn(0x7f63cc0cfdc0 msgr2=0x7f63cc0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 shutdown_connections 2024-02-02T13:24:31.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:31.834+0000 7f63d3800640 1 -- 172.21.15.91:0/3274029366 wait complete. 2024-02-02T13:24:31.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:24:31.853 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:24:32.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.048+0000 7f1957e00640 1 Processor -- start 2024-02-02T13:24:32.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.049+0000 7f1957e00640 1 -- start start 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f195815d3a0 0x7f195806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19580c9450 0x7f19580c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19580ce3b0 con 0x7f19580add80 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19580adbd0 con 0x7f19580c9450 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19580c59d0 con 0x7f195815d3a0 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19580c9450 0x7f19580c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19580c9450 0x7f19580c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54504/0 (socket says 172.21.15.91:54504) 2024-02-02T13:24:32.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1957400640 1 -- 172.21.15.91:0/489130752 learned_addr learned my addr 172.21.15.91:0/489130752 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.050+0000 7f1956a00640 1 --2- 172.21.15.91:0/489130752 >> v2:172.21.15.113:3300/0 conn(0x7f195815d3a0 0x7f195806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.051+0000 7f1957400640 1 -- 172.21.15.91:0/489130752 >> v2:172.21.15.113:3300/0 conn(0x7f195815d3a0 msgr2=0x7f195806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.051+0000 7f1957400640 1 --2- 172.21.15.91:0/489130752 >> v2:172.21.15.113:3300/0 conn(0x7f195815d3a0 0x7f195806b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.051+0000 7f1957400640 1 -- 172.21.15.91:0/489130752 >> v1:172.21.15.113:6789/0 conn(0x7f19580add80 legacy=0x7f19580706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.051+0000 7f1957400640 1 -- 172.21.15.91:0/489130752 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580ce8e0 con 0x7f19580c9450 2024-02-02T13:24:32.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.052+0000 7f1957400640 1 --2- 172.21.15.91:0/489130752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f19580c98e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f195806b360 tx=0x7f195002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b822278eb468b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.052+0000 7f1956000640 1 -- 172.21.15.91:0/489130752 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195002c960 con 0x7f19580c9450 2024-02-02T13:24:32.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.052+0000 7f1956000640 1 -- 172.21.15.91:0/489130752 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195002cac0 con 0x7f19580c9450 2024-02-02T13:24:32.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.052+0000 7f1956000640 1 -- 172.21.15.91:0/489130752 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195002ce50 con 0x7f19580c9450 2024-02-02T13:24:32.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 -- 172.21.15.91:0/489130752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 msgr2=0x7f19580c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 --2- 172.21.15.91:0/489130752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f19580c98e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f195806b360 tx=0x7f195002bdc0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 -- 172.21.15.91:0/489130752 shutdown_connections 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 --2- 172.21.15.91:0/489130752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f19580c98e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 --2- 172.21.15.91:0/489130752 >> v2:172.21.15.113:3300/0 conn(0x7f195815d3a0 0x7f195806b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 -- 172.21.15.91:0/489130752 >> 172.21.15.91:0/489130752 conn(0x7f19580cea90 msgr2=0x7f19580ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 -- 172.21.15.91:0/489130752 shutdown_connections 2024-02-02T13:24:32.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.053+0000 7f1957e00640 1 -- 172.21.15.91:0/489130752 wait complete. 2024-02-02T13:24:32.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.055+0000 7f1957e00640 1 Processor -- start 2024-02-02T13:24:32.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.055+0000 7f1957e00640 1 -- start start 2024-02-02T13:24:32.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19580add80 0x7f19580aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195815d3a0 0x7f19580b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580af1b0 con 0x7f19580c9450 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580b0a70 con 0x7f19580add80 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19580af9d0 con 0x7f195815d3a0 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54508/0 (socket says 172.21.15.91:54508) 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 -- 172.21.15.91:0/1894047048 learned_addr learned my addr 172.21.15.91:0/1894047048 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1956a00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19580add80 0x7f19580aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f1957400640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195815d3a0 0x7f19580b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 -- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195815d3a0 msgr2=0x7f19580b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195815d3a0 0x7f19580b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 -- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19580add80 msgr2=0x7f19580aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.056+0000 7f194de00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19580add80 0x7f19580aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f194de00640 1 -- 172.21.15.91:0/1894047048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1950009b30 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f194de00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f194000c3f0 tx=0x7f194000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940016070 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940007020 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400073b0 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19580afcb0 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.057+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19580b0270 con 0x7f19580c9450 2024-02-02T13:24:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.058+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195804dff0 con 0x7f19580c9450 2024-02-02T13:24:32.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.059+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1940007510 con 0x7f19580c9450 2024-02-02T13:24:32.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.060+0000 7f194f400640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 0x7f193006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.060+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1940087ed0 con 0x7f19580c9450 2024-02-02T13:24:32.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.060+0000 7f1956a00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 0x7f193006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.061+0000 7f1956a00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 0x7f193006c8f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1948006fd0 tx=0x7f1948008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.065+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1940053770 con 0x7f19580c9450 2024-02-02T13:24:32.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.242+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19580b4a10 con 0x7f19580c9450 2024-02-02T13:24:32.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.242+0000 7f194f400640 1 -- 172.21.15.91:0/1894047048 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1940056d90 con 0x7f19580c9450 2024-02-02T13:24:32.244 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 msgr2=0x7f193006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 0x7f193006c8f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1948006fd0 tx=0x7f1948008040 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 msgr2=0x7f1958151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f194000c3f0 tx=0x7f194000c8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 shutdown_connections 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f193006a430 0x7f193006c8f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195815d3a0 0x7f19580b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19580c9450 0x7f1958151ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 --2- 172.21.15.91:0/1894047048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19580add80 0x7f19580aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 >> 172.21.15.91:0/1894047048 conn(0x7f19580cea90 msgr2=0x7f195815c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:32.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 shutdown_connections 2024-02-02T13:24:32.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.246+0000 7f1957e00640 1 -- 172.21.15.91:0/1894047048 wait complete. 2024-02-02T13:24:32.264 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:24:32.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.455+0000 7fa22a200640 1 Processor -- start 2024-02-02T13:24:32.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.455+0000 7fa22a200640 1 -- start start 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa22a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 0x7fa2240b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa22a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2240b0c40 0x7fa2240b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa22a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22415fa00 con 0x7fa224151f90 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa22a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2240b10d0 con 0x7fa2240b0c40 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa22a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa224070f80 con 0x7fa22415f5a0 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa223e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2240b0c40 0x7fa2240b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.457+0000 7fa21a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 0x7fa2240b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa21a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 0x7fa2240b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:34628/0 (socket says 172.21.15.91:34628) 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa224151f90 0x7fa22406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:60390/0 (socket says 172.21.15.91:60390) 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa21a800640 1 -- 172.21.15.91:0/714860920 learned_addr learned my addr 172.21.15.91:0/714860920 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2240b0c40 0x7fa2240b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:59786/0 (socket says 172.21.15.91:59786) 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223e00640 1 -- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 msgr2=0x7fa2240b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223e00640 1 --2- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 0x7fa2240b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223e00640 1 -- 172.21.15.91:0/714860920 >> v1:172.21.15.113:6789/0 conn(0x7fa224151f90 legacy=0x7fa22406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.458+0000 7fa223e00640 1 -- 172.21.15.91:0/714860920 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224152370 con 0x7fa2240b0c40 2024-02-02T13:24:32.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.460+0000 7fa223e00640 1 --2- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2240b0c40 0x7fa2240b3af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009d90 tx=0x7fa21c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb802cbb4a15049 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.460+0000 7fa222a00640 1 -- 172.21.15.91:0/714860920 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02cc50 con 0x7fa2240b0c40 2024-02-02T13:24:32.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.460+0000 7fa222a00640 1 -- 172.21.15.91:0/714860920 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c02cdb0 con 0x7fa2240b0c40 2024-02-02T13:24:32.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.460+0000 7fa222a00640 1 -- 172.21.15.91:0/714860920 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c034660 con 0x7fa2240b0c40 2024-02-02T13:24:32.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 -- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2240b0c40 msgr2=0x7fa2240b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 --2- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2240b0c40 0x7fa2240b3af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009d90 tx=0x7fa21c02bea0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 -- 172.21.15.91:0/714860920 shutdown_connections 2024-02-02T13:24:32.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 --2- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2240b0c40 0x7fa2240b3af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 --2- 172.21.15.91:0/714860920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa22415f5a0 0x7fa2240b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 -- 172.21.15.91:0/714860920 >> 172.21.15.91:0/714860920 conn(0x7fa2240cec80 msgr2=0x7fa2240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:32.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 -- 172.21.15.91:0/714860920 shutdown_connections 2024-02-02T13:24:32.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.461+0000 7fa22a200640 1 -- 172.21.15.91:0/714860920 wait complete. 2024-02-02T13:24:32.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.462+0000 7fa22a200640 1 Processor -- start 2024-02-02T13:24:32.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.462+0000 7fa22a200640 1 -- start start 2024-02-02T13:24:32.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa224151f90 0x7fa2241f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa22415f5a0 0x7fa22415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22415e4c0 con 0x7fa2240b0c40 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240b49b0 con 0x7fa22415f5a0 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.463+0000 7fa22a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2241f15a0 con 0x7fa224151f90 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54514/0 (socket says 172.21.15.91:54514) 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 -- 172.21.15.91:0/4253714082 learned_addr learned my addr 172.21.15.91:0/4253714082 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223e00640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa22415f5a0 0x7fa22415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa21a800640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa224151f90 0x7fa2241f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 -- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa224151f90 msgr2=0x7fa2241f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa224151f90 0x7fa2241f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 -- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa22415f5a0 msgr2=0x7fa22415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa22415f5a0 0x7fa22415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 -- 172.21.15.91:0/4253714082 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c009a70 con 0x7fa2240b0c40 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa223400640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c3e0 tx=0x7fa21400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.464+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140074e0 con 0x7fa2240b0c40 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.465+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214007640 con 0x7fa2240b0c40 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.465+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140049e0 con 0x7fa2240b0c40 2024-02-02T13:24:32.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.465+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22415d940 con 0x7fa2240b0c40 2024-02-02T13:24:32.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.465+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22415df00 con 0x7fa2240b0c40 2024-02-02T13:24:32.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.466+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa224066cf0 con 0x7fa2240b0c40 2024-02-02T13:24:32.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.466+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa214004b40 con 0x7fa2240b0c40 2024-02-02T13:24:32.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.467+0000 7fa21be00640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 0x7fa1fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.467+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa214087ed0 con 0x7fa2240b0c40 2024-02-02T13:24:32.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.467+0000 7fa21a800640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 0x7fa1fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.468+0000 7fa21a800640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 0x7fa1fc06c920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c89f0 tx=0x7fa20c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.472+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa214053770 con 0x7fa2240b0c40 2024-02-02T13:24:32.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.699+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa22415fa00 con 0x7fa2240b0c40 2024-02-02T13:24:32.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.700+0000 7fa21be00640 1 -- 172.21.15.91:0/4253714082 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fa22415fa00 con 0x7fa2240b0c40 2024-02-02T13:24:32.702 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:32.702 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-02-02T13:24:32.703 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 3 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.704+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 msgr2=0x7fa1fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.704+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 0x7fa1fc06c920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c89f0 tx=0x7fa20c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.704+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 msgr2=0x7fa2241f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.704+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c3e0 tx=0x7fa21400c8a0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 shutdown_connections 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1fc06a460 0x7fa1fc06c920 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa224151f90 0x7fa2241f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa22415f5a0 0x7fa22415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 --2- 172.21.15.91:0/4253714082 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2240b0c40 0x7fa2241f00a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 >> 172.21.15.91:0/4253714082 conn(0x7fa2240cec80 msgr2=0x7fa2240ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:32.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 shutdown_connections 2024-02-02T13:24:32.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.705+0000 7fa22a200640 1 -- 172.21.15.91:0/4253714082 wait complete. 2024-02-02T13:24:32.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:32.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.915+0000 7f235fe00640 1 Processor -- start 2024-02-02T13:24:32.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.915+0000 7f235fe00640 1 -- start start 2024-02-02T13:24:32.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f236015cee0 0x7f23600b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f23600d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23600b7d40 con 0x7f23600b6990 2024-02-02T13:24:32.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23600b81d0 con 0x7f236015cee0 2024-02-02T13:24:32.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b7130 con 0x7f23600ce1d0 2024-02-02T13:24:32.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f2357e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f23600b6990 0x7f23600cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52502/0 (socket says 172.21.15.91:52502) 2024-02-02T13:24:32.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f2357e00640 1 -- 172.21.15.91:0/3109979887 learned_addr learned my addr 172.21.15.91:0/3109979887 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235f400640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f23600d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235ea00640 1 --2- 172.21.15.91:0/3109979887 >> v2:172.21.15.113:3300/0 conn(0x7f236015cee0 0x7f23600b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920472791 0 0) 0x7f23600b7d40 con 0x7f23600b6990 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.917+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2340003520 con 0x7f23600b6990 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148580116 0 0) 0x7f2340003520 con 0x7f23600b6990 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 msgr2=0x7f23600d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f23600d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 >> v2:172.21.15.113:3300/0 conn(0x7f236015cee0 msgr2=0x7f23600b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 --2- 172.21.15.91:0/3109979887 >> v2:172.21.15.113:3300/0 conn(0x7f236015cee0 0x7f23600b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600b7410 con 0x7f23600b6990 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2348001670 con 0x7f23600b6990 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 >> v1:172.21.15.91:6789/0 conn(0x7f23600b6990 legacy=0x7f23600cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f2340003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2340004d00 0x7f2340005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b81d0 con 0x7f2340003ef0 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b7130 con 0x7f2340004d00 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234000b980 con 0x7f23600ce1d0 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.918+0000 7f2357e00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f235ea00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f2340003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f235f400640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2340004d00 0x7f2340005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f2357e00640 1 -- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 msgr2=0x7f2340003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f2357e00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23600ce1d0 0x7f2340003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f2357e00640 1 -- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2340004d00 msgr2=0x7f2340005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:32.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f2357e00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2340004d00 0x7f2340005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f2357e00640 1 -- 172.21.15.91:0/3109979887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340003520 con 0x7f2340003ef0 2024-02-02T13:24:32.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.919+0000 7f235f400640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2340004d00 0x7f2340005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:32.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.920+0000 7f2357e00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23400047c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f234800c240 tx=0x7f23400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a47333807f6acb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.920+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234802cae0 con 0x7f2340003ef0 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.920+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234802cc40 con 0x7f2340003ef0 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.920+0000 7f235e000640 1 -- 172.21.15.91:0/3109979887 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005af0 con 0x7f2340003ef0 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.920+0000 7f235fe00640 1 -- 172.21.15.91:0/3109979887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 msgr2=0x7f23400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235fe00640 1 --2- 172.21.15.91:0/3109979887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23400047c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f234800c240 tx=0x7f23400057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235f400640 1 -- 172.21.15.91:0/3109979887 reap_dead start 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235fe00640 1 -- 172.21.15.91:0/3109979887 shutdown_connections 2024-02-02T13:24:32.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235fe00640 1 -- 172.21.15.91:0/3109979887 >> 172.21.15.91:0/3109979887 conn(0x7f2360070690 msgr2=0x7f2360070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:32.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235fe00640 1 -- 172.21.15.91:0/3109979887 shutdown_connections 2024-02-02T13:24:32.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.921+0000 7f235fe00640 1 -- 172.21.15.91:0/3109979887 wait complete. 2024-02-02T13:24:32.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.922+0000 7f235fe00640 1 Processor -- start 2024-02-02T13:24:32.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.923+0000 7f235fe00640 1 -- start start 2024-02-02T13:24:32.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.923+0000 7f235fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.923+0000 7f235fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2360148900 0x7f2360148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23601f15d0 0x7f23601f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236015d5e0 con 0x7f2340003ef0 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236015f7a0 con 0x7f2360148900 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236015fa80 con 0x7f23601f15d0 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54536/0 (socket says 172.21.15.91:54536) 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 -- 172.21.15.91:0/3155607314 learned_addr learned my addr 172.21.15.91:0/3155607314 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 -- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23601f15d0 msgr2=0x7f23601f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23601f15d0 0x7f23601f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 -- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2360148900 msgr2=0x7f2360148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2360148900 0x7f2360148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 -- 172.21.15.91:0/3155607314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236015fd60 con 0x7f2340003ef0 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f235ea00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2350011600 tx=0x7f2350011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.924+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350015870 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.925+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23500159d0 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.925+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350015d40 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.925+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2360160040 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.925+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23601f2170 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.926+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236004dff0 con 0x7f2340003ef0 2024-02-02T13:24:32.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.926+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f235000f490 con 0x7f2340003ef0 2024-02-02T13:24:32.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.927+0000 7f2356a00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 0x7f233806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:32.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.927+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2350088960 con 0x7f2340003ef0 2024-02-02T13:24:32.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.928+0000 7f2357e00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 0x7f233806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:32.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.928+0000 7f2357e00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 0x7f233806c920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f234802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:32.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:32.932+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2350054330 con 0x7f2340003ef0 2024-02-02T13:24:33.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.122+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23600b6120 con 0x7f2340003ef0 2024-02-02T13:24:33.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.123+0000 7f2356a00640 1 -- 172.21.15.91:0/3155607314 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f2350022090 con 0x7f2340003ef0 2024-02-02T13:24:33.125 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:33.125 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:28.801621+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 msgr2=0x7f233806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 0x7f233806c920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f234802e000 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 msgr2=0x7f23600b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2350011600 tx=0x7f2350011ac0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 shutdown_connections 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f233806a460 0x7f233806c920 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23601f15d0 0x7f23601f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2360148900 0x7f2360148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 --2- 172.21.15.91:0/3155607314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2340003ef0 0x7f23600b5270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 >> 172.21.15.91:0/3155607314 conn(0x7f2360070690 msgr2=0x7f23600c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 shutdown_connections 2024-02-02T13:24:33.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.126+0000 7f235fe00640 1 -- 172.21.15.91:0/3155607314 wait complete. 2024-02-02T13:24:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:24:33.144 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:24:33.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.349+0000 7f3228000640 1 Processor -- start 2024-02-02T13:24:33.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.349+0000 7f3228000640 1 -- start start 2024-02-02T13:24:33.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.349+0000 7f3228000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f322015a010 0x7f32200c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3228000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200cc7c0 con 0x7f32200ce130 2024-02-02T13:24:33.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3228000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200ce930 con 0x7f32200b0700 2024-02-02T13:24:33.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3228000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200cd640 con 0x7f322015a010 2024-02-02T13:24:33.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3224c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f32200ce130 0x7f32200ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52508/0 (socket says 172.21.15.91:52508) 2024-02-02T13:24:33.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3224c00640 1 -- 172.21.15.91:0/2493049897 learned_addr learned my addr 172.21.15.91:0/2493049897 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:33.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f3225600640 1 --2- 172.21.15.91:0/2493049897 >> v2:172.21.15.113:3300/0 conn(0x7f322015a010 0x7f32200c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748662036 0 0) 0x7f32200cc7c0 con 0x7f32200ce130 2024-02-02T13:24:33.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.350+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3200003520 con 0x7f32200ce130 2024-02-02T13:24:33.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303587175 0 0) 0x7f3200003520 con 0x7f32200ce130 2024-02-02T13:24:33.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 >> v2:172.21.15.113:3300/0 conn(0x7f322015a010 msgr2=0x7f32200c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 --2- 172.21.15.91:0/2493049897 >> v2:172.21.15.113:3300/0 conn(0x7f322015a010 0x7f32200c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 >> v1:172.21.15.113:6789/0 conn(0x7f32200b0700 legacy=0x7f322006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322006c890 con 0x7f32200ce130 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3208001670 con 0x7f32200ce130 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 >> v1:172.21.15.91:6789/0 conn(0x7f32200ce130 legacy=0x7f32200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 0x7f3200003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000047c0 0x7f3200004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200005180 0x7f320000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200cd640 con 0x7f3200003970 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320000c020 con 0x7f32000047c0 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320000c1e0 con 0x7f3200005180 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f3225600640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 0x7f3200003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f3225600640 1 -- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200005180 msgr2=0x7f320000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f3225600640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200005180 0x7f320000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f3225600640 1 -- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000047c0 msgr2=0x7f3200004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.351+0000 7f3225600640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000047c0 0x7f3200004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.352+0000 7f3225600640 1 -- 172.21.15.91:0/2493049897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200003520 con 0x7f3200003970 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.352+0000 7f3226000640 1 -- 172.21.15.91:0/2493049897 reap_dead start 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.352+0000 7f3225600640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 0x7f3200003d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3210009ac0 tx=0x7f32000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ee352e2a13327f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321002c6d0 con 0x7f3200003970 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321002c830 con 0x7f3200003970 2024-02-02T13:24:33.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f321fe00640 1 -- 172.21.15.91:0/2493049897 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321002cbc0 con 0x7f3200003970 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 -- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 msgr2=0x7f3200003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 0x7f3200003d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3210009ac0 tx=0x7f32000057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 -- 172.21.15.91:0/2493049897 shutdown_connections 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 --2- 172.21.15.91:0/2493049897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3200003970 0x7f3200003d90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 -- 172.21.15.91:0/2493049897 >> 172.21.15.91:0/2493049897 conn(0x7f32200cec80 msgr2=0x7f32200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:33.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.353+0000 7f3228000640 1 -- 172.21.15.91:0/2493049897 shutdown_connections 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.354+0000 7f3228000640 1 -- 172.21.15.91:0/2493049897 wait complete. 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.355+0000 7f3228000640 1 Processor -- start 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.355+0000 7f3228000640 1 -- start start 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 0x7f322014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322014e210 0x7f32201f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200ad6a0 con 0x7f32201f6410 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200ad980 con 0x7f322014e210 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3228000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32200adc60 con 0x7f3200003970 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3226000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3226000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54550/0 (socket says 172.21.15.91:54550) 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.356+0000 7f3226000640 1 -- 172.21.15.91:0/2442435095 learned_addr learned my addr 172.21.15.91:0/2442435095 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:33.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3225600640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 0x7f322014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3224c00640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322014e210 0x7f32201f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 -- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 msgr2=0x7f322014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 0x7f322014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 -- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322014e210 msgr2=0x7f32201f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322014e210 0x7f32201f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 -- 172.21.15.91:0/2442435095 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200adf40 con 0x7f32201f6410 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3225600640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 0x7f322014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3226000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32200cdc50 tx=0x7f321800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003970 con 0x7f32201f6410 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218003af0 con 0x7f32201f6410 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003ea0 con 0x7f32201f6410 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.357+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32200ae220 con 0x7f32201f6410 2024-02-02T13:24:33.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.358+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322014efc0 con 0x7f32201f6410 2024-02-02T13:24:33.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.359+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220066cf0 con 0x7f32201f6410 2024-02-02T13:24:33.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.359+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3218004000 con 0x7f32201f6410 2024-02-02T13:24:33.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.360+0000 7f321d600640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 0x7f31f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.360+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f32180878e0 con 0x7f32201f6410 2024-02-02T13:24:33.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.361+0000 7f3225600640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 0x7f31f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.361+0000 7f3225600640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 0x7f31f806c8f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3210036000 tx=0x7f3210030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.364+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3218053180 con 0x7f32201f6410 2024-02-02T13:24:33.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.538+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32200b4c00 con 0x7f32201f6410 2024-02-02T13:24:33.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.538+0000 7f321d600640 1 -- 172.21.15.91:0/2442435095 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32180567a0 con 0x7f32201f6410 2024-02-02T13:24:33.540 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 msgr2=0x7f31f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 0x7f31f806c8f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3210036000 tx=0x7f3210030000 comp rx=0 tx=0).stop 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 msgr2=0x7f32201f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32200cdc50 tx=0x7f321800c8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 shutdown_connections 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f31f806a430 0x7f31f806c8f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32201f6410 0x7f32201f68a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322014e210 0x7f32201f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 --2- 172.21.15.91:0/2442435095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3200003970 0x7f322014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 >> 172.21.15.91:0/2442435095 conn(0x7f32200cec80 msgr2=0x7f32200af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:33.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 shutdown_connections 2024-02-02T13:24:33.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.541+0000 7f3228000640 1 -- 172.21.15.91:0/2442435095 wait complete. 2024-02-02T13:24:33.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:24:33.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.748+0000 7f102ee00640 1 Processor -- start 2024-02-02T13:24:33.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.749+0000 7f102ee00640 1 -- start start 2024-02-02T13:24:33.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.750+0000 7f102ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f102806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.750+0000 7f102ee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10280cb580 0x7f10280cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.750+0000 7f102ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102806d910 con 0x7f102815ad10 2024-02-02T13:24:33.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.750+0000 7f102ee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102806d730 con 0x7f10280cb580 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.750+0000 7f102ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280cbf50 con 0x7f102815b0f0 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10280cb580 0x7f10280cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10280cb580 0x7f10280cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:59868/0 (socket says 172.21.15.91:59868) 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 -- 172.21.15.91:0/2220897157 learned_addr learned my addr 172.21.15.91:0/2220897157 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f101fe00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f102806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 -- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 msgr2=0x7f102806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f102806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 -- 172.21.15.91:0/2220897157 >> v1:172.21.15.113:6789/0 conn(0x7f102815ad10 legacy=0x7f102806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.751+0000 7f102ce00640 1 -- 172.21.15.91:0/2220897157 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280cc850 con 0x7f10280cb580 2024-02-02T13:24:33.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.753+0000 7f102ce00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10280cb580 0x7f10280cba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1020009e50 tx=0x7f102002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f028931c31486c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.753+0000 7f1027400640 1 -- 172.21.15.91:0/2220897157 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102002cb00 con 0x7f10280cb580 2024-02-02T13:24:33.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.753+0000 7f1027400640 1 -- 172.21.15.91:0/2220897157 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102002cc60 con 0x7f10280cb580 2024-02-02T13:24:33.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.753+0000 7f1027400640 1 -- 172.21.15.91:0/2220897157 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200343f0 con 0x7f10280cb580 2024-02-02T13:24:33.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 -- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10280cb580 msgr2=0x7f10280cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10280cb580 0x7f10280cba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1020009e50 tx=0x7f102002bf60 comp rx=0 tx=0).stop 2024-02-02T13:24:33.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 -- 172.21.15.91:0/2220897157 shutdown_connections 2024-02-02T13:24:33.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10280cb580 0x7f10280cba10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 --2- 172.21.15.91:0/2220897157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f102806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 -- 172.21.15.91:0/2220897157 >> 172.21.15.91:0/2220897157 conn(0x7f10280b0700 msgr2=0x7f10280b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:33.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 -- 172.21.15.91:0/2220897157 shutdown_connections 2024-02-02T13:24:33.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.754+0000 7f102ee00640 1 -- 172.21.15.91:0/2220897157 wait complete. 2024-02-02T13:24:33.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.756+0000 7f102ee00640 1 Processor -- start 2024-02-02T13:24:33.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.756+0000 7f102ee00640 1 -- start start 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 0x7f10281500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f10281f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b03b0 con 0x7f10280cb580 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10281505f0 con 0x7f102815ad10 2024-02-02T13:24:33.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f102ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280ac8d0 con 0x7f102815b0f0 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.757+0000 7f1027e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54560/0 (socket says 172.21.15.91:54560) 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 -- 172.21.15.91:0/3361584475 learned_addr learned my addr 172.21.15.91:0/3361584475 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f101fe00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 0x7f10281500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 -- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 msgr2=0x7f10281f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f102ce00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f10281f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f10281f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 -- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 msgr2=0x7f10281500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 0x7f10281500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 -- 172.21.15.91:0/3361584475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020009b30 con 0x7f10280cb580 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f101fe00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 0x7f10281500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1027e00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f101800c450 tx=0x7f101800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180052d0 con 0x7f10280cb580 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018005430 con 0x7f10280cb580 2024-02-02T13:24:33.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018012870 con 0x7f10280cb580 2024-02-02T13:24:33.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.758+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280ca0c0 con 0x7f10280cb580 2024-02-02T13:24:33.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.759+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10280ca680 con 0x7f10280cb580 2024-02-02T13:24:33.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.760+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028066cf0 con 0x7f10280cb580 2024-02-02T13:24:33.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.760+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f10180129d0 con 0x7f10280cb580 2024-02-02T13:24:33.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.761+0000 7f1024c00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 0x7f100006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:33.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.761+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f10180880c0 con 0x7f10280cb580 2024-02-02T13:24:33.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.761+0000 7f101fe00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 0x7f100006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:33.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.762+0000 7f101fe00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 0x7f100006c8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10280b1190 tx=0x7f1010008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:33.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.765+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1018053a00 con 0x7f10280cb580 2024-02-02T13:24:33.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.954+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f10280ccde0 con 0x7f10280cb580 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.955+0000 7f1024c00640 1 -- 172.21.15.91:0/3361584475 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1018057020 con 0x7f10280cb580 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:24:33.957 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:24:33.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 msgr2=0x7f100006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 0x7f100006c8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10280b1190 tx=0x7f1010008040 comp rx=0 tx=0).stop 2024-02-02T13:24:33.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 msgr2=0x7f10280afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f101800c450 tx=0x7f101800c910 comp rx=0 tx=0).stop 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 shutdown_connections 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100006a430 0x7f100006c8f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f102815b0f0 0x7f10281f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102815ad10 0x7f10281500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 --2- 172.21.15.91:0/3361584475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10280cb580 0x7f10280afe70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 >> 172.21.15.91:0/3361584475 conn(0x7f10280b0700 msgr2=0x7f102815d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 shutdown_connections 2024-02-02T13:24:33.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:33.958+0000 7f102ee00640 1 -- 172.21.15.91:0/3361584475 wait complete. 2024-02-02T13:24:33.977 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:24:33.977 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:24:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.178+0000 7f494f600640 1 Processor -- start 2024-02-02T13:24:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.178+0000 7f494f600640 1 -- start start 2024-02-02T13:24:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494f600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494806d7e0 0x7f49480cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494f600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f494806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494f600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480ca070 con 0x7f49480c9b00 2024-02-02T13:24:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494f600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480cba50 con 0x7f494806d7e0 2024-02-02T13:24:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494f600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480cbd30 con 0x7f494815e070 2024-02-02T13:24:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494806d7e0 0x7f49480cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f4947e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f49480c9b00 0x7f494806dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52514/0 (socket says 172.21.15.91:52514) 2024-02-02T13:24:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494806d7e0 0x7f49480cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54576/0 (socket says 172.21.15.91:54576) 2024-02-02T13:24:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f4947e00640 1 -- 172.21.15.91:0/1679363342 learned_addr learned my addr 172.21.15.91:0/1679363342 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494d600640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f494806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 -- 172.21.15.91:0/1679363342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 msgr2=0x7f494806fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f494806fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 -- 172.21.15.91:0/1679363342 >> v1:172.21.15.91:6789/0 conn(0x7f49480c9b00 legacy=0x7f494806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494cc00640 1 -- 172.21.15.91:0/1679363342 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480cc010 con 0x7f494806d7e0 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f4947400640 1 -- 172.21.15.91:0/1679363342 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922878948 0 0) 0x7f49480ca070 con 0x7f49480c9b00 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.180+0000 7f494d600640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f494806fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.181+0000 7f494cc00640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f49480cc530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49400097a0 tx=0x7f494002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd73760f78cd7236 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f4947400640 1 -- 172.21.15.91:0/1679363342 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940031780 con 0x7f494806d7e0 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f4947400640 1 -- 172.21.15.91:0/1679363342 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400318e0 con 0x7f494806d7e0 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f4947400640 1 -- 172.21.15.91:0/1679363342 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940031c70 con 0x7f494806d7e0 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 -- 172.21.15.91:0/1679363342 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 msgr2=0x7f49480cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f49480cc530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49400097a0 tx=0x7f494002b820 comp rx=0 tx=0).stop 2024-02-02T13:24:34.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 -- 172.21.15.91:0/1679363342 shutdown_connections 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f494806fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 --2- 172.21.15.91:0/1679363342 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f49480cc530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 -- 172.21.15.91:0/1679363342 >> 172.21.15.91:0/1679363342 conn(0x7f49480b0fc0 msgr2=0x7f49480b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.182+0000 7f494f600640 1 -- 172.21.15.91:0/1679363342 shutdown_connections 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.183+0000 7f494f600640 1 -- 172.21.15.91:0/1679363342 wait complete. 2024-02-02T13:24:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.184+0000 7f494f600640 1 Processor -- start 2024-02-02T13:24:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.184+0000 7f494f600640 1 -- start start 2024-02-02T13:24:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49480c9b00 0x7f4948155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f49481e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481e34f0 con 0x7f494806d7e0 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494815cd40 con 0x7f49480c9b00 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494f600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480c9110 con 0x7f494815e070 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.185+0000 7f494cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54586/0 (socket says 172.21.15.91:54586) 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 -- 172.21.15.91:0/3346605357 learned_addr learned my addr 172.21.15.91:0/3346605357 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494d600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f49481e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f4947e00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49480c9b00 0x7f4948155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 -- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 msgr2=0x7f49481e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f49481e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 -- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49480c9b00 msgr2=0x7f4948155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49480c9b00 0x7f4948155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 -- 172.21.15.91:0/3346605357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009400 con 0x7f494806d7e0 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f494cc00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4940034040 tx=0x7f49400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.186+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940032960 con 0x7f494806d7e0 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.187+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940032ac0 con 0x7f494806d7e0 2024-02-02T13:24:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.187+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940032e70 con 0x7f494806d7e0 2024-02-02T13:24:34.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.187+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49481e3700 con 0x7f494806d7e0 2024-02-02T13:24:34.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.187+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49481e3cc0 con 0x7f494806d7e0 2024-02-02T13:24:34.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.188+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49480cf2b0 con 0x7f494806d7e0 2024-02-02T13:24:34.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.188+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f494003b3f0 con 0x7f494806d7e0 2024-02-02T13:24:34.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.189+0000 7f4944c00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 0x7f492006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:34.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.189+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49400aa9b0 con 0x7f494806d7e0 2024-02-02T13:24:34.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.190+0000 7f4947e00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 0x7f492006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:34.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.190+0000 7f4947e00640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 0x7f492006c8f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49481f6630 tx=0x7f4930009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:34.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.195+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4940076380 con 0x7f494806d7e0 2024-02-02T13:24:34.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:34.387+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49480aeca0 con 0x7f494806d7e0 2024-02-02T13:24:35.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.247+0000 7f4944c00640 1 -- 172.21.15.91:0/3346605357 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49400799a0 con 0x7f494806d7e0 2024-02-02T13:24:35.248 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.252+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 msgr2=0x7f492006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.252+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 0x7f492006c8f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49481f6630 tx=0x7f4930009040 comp rx=0 tx=0).stop 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.252+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 msgr2=0x7f494815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.252+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4940034040 tx=0x7f49400046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 shutdown_connections 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f492006a430 0x7f492006c8f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494815e070 0x7f49481e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49480c9b00 0x7f4948155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 --2- 172.21.15.91:0/3346605357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494806d7e0 0x7f494815c800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 >> 172.21.15.91:0/3346605357 conn(0x7f49480b0fc0 msgr2=0x7f4948156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:35.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.253+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 shutdown_connections 2024-02-02T13:24:35.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.254+0000 7f494f600640 1 -- 172.21.15.91:0/3346605357 wait complete. 2024-02-02T13:24:35.275 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:24:35.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.478+0000 7f18d0800640 1 Processor -- start 2024-02-02T13:24:35.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.478+0000 7f18d0800640 1 -- start start 2024-02-02T13:24:35.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18d0800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18c80b2260 0x7f18c80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18d0800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18d0800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80cd930 con 0x7f18c80d0070 2024-02-02T13:24:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18d0800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80b8f90 con 0x7f18c80b2260 2024-02-02T13:24:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18d0800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80d1680 con 0x7f18c80b6260 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.480+0000 7f18ce800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18cd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18c80b2260 0x7f18c80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:34712/0 (socket says 172.21.15.91:34712) 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 -- 172.21.15.91:0/575052022 learned_addr learned my addr 172.21.15.91:0/575052022 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 -- 172.21.15.91:0/575052022 >> v2:172.21.15.113:3300/0 conn(0x7f18c80b2260 msgr2=0x7f18c80b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 --2- 172.21.15.91:0/575052022 >> v2:172.21.15.113:3300/0 conn(0x7f18c80b2260 0x7f18c80b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 -- 172.21.15.91:0/575052022 >> v1:172.21.15.113:6789/0 conn(0x7f18c80d0070 legacy=0x7f18c80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.481+0000 7f18ce800640 1 -- 172.21.15.91:0/575052022 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80d1960 con 0x7f18c80b6260 2024-02-02T13:24:35.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.483+0000 7f18ce800640 1 --2- 172.21.15.91:0/575052022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009870 tx=0x7f18b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d38ce3123f06571 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:35.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.483+0000 7f18cca00640 1 -- 172.21.15.91:0/575052022 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8030840 con 0x7f18c80b6260 2024-02-02T13:24:35.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.483+0000 7f18cca00640 1 -- 172.21.15.91:0/575052022 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b80309a0 con 0x7f18c80b6260 2024-02-02T13:24:35.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.483+0000 7f18cca00640 1 -- 172.21.15.91:0/575052022 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8030d30 con 0x7f18c80b6260 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 -- 172.21.15.91:0/575052022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 msgr2=0x7f18c80b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 --2- 172.21.15.91:0/575052022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009870 tx=0x7f18b802b980 comp rx=0 tx=0).stop 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 -- 172.21.15.91:0/575052022 shutdown_connections 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 --2- 172.21.15.91:0/575052022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c80b66f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 --2- 172.21.15.91:0/575052022 >> v2:172.21.15.113:3300/0 conn(0x7f18c80b2260 0x7f18c80b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 -- 172.21.15.91:0/575052022 >> 172.21.15.91:0/575052022 conn(0x7f18c8070660 msgr2=0x7f18c8070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 -- 172.21.15.91:0/575052022 shutdown_connections 2024-02-02T13:24:35.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.484+0000 7f18d0800640 1 -- 172.21.15.91:0/575052022 wait complete. 2024-02-02T13:24:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.486+0000 7f18d0800640 1 Processor -- start 2024-02-02T13:24:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.487+0000 7f18d0800640 1 -- start start 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c80b2260 0x7f18c81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ca200 con 0x7f18c80d0070 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80caaa0 con 0x7f18c80b2260 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18d0800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8159f00 con 0x7f18c80b6260 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18ce800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18ce800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54602/0 (socket says 172.21.15.91:54602) 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18ce800640 1 -- 172.21.15.91:0/1779369322 learned_addr learned my addr 172.21.15.91:0/1779369322 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18cd400640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.488+0000 7f18cde00640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c80b2260 0x7f18c81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 -- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 msgr2=0x7f18c815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c815f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 -- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c80b2260 msgr2=0x7f18c81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:35.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c80b2260 0x7f18c81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 -- 172.21.15.91:0/1779369322 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80094d0 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18ce800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18b802b9c0 tx=0x7f18b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.489+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8039c90 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.490+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8039e10 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.490+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80446c0 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.490+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c815f120 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.490+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c80ca730 con 0x7f18c80d0070 2024-02-02T13:24:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.491+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c8066cf0 con 0x7f18c80d0070 2024-02-02T13:24:35.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.492+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f18b804d3b0 con 0x7f18c80d0070 2024-02-02T13:24:35.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.493+0000 7f18c6000640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 0x7f18a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:35.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.493+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f18b803e070 con 0x7f18c80d0070 2024-02-02T13:24:35.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.493+0000 7f18cde00640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 0x7f18a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:35.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.494+0000 7f18cde00640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 0x7f18a006c8f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f18c00012e0 tx=0x7f18c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:35.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.500+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f18b8076b30 con 0x7f18c80d0070 2024-02-02T13:24:35.693 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:35.692+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f18c80ca8c0 con 0x7f18c80d0070 2024-02-02T13:24:36.271 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:24:36.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.267+0000 7f18c6000640 1 -- 172.21.15.91:0/1779369322 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f18b807a150 con 0x7f18c80d0070 2024-02-02T13:24:36.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 msgr2=0x7f18a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 0x7f18a006c8f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f18c00012e0 tx=0x7f18c0009040 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 msgr2=0x7f18c815fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18b802b9c0 tx=0x7f18b8004730 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 shutdown_connections 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18a006a430 0x7f18a006c8f0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f18c00012e0 tx=0x7f18c0009040 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18c80b6260 0x7f18c815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c80d0070 0x7f18c815fbc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 --2- 172.21.15.91:0/1779369322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c80b2260 0x7f18c81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.277+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 >> 172.21.15.91:0/1779369322 conn(0x7f18c8070660 msgr2=0x7f18c80b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:36.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.279+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 shutdown_connections 2024-02-02T13:24:36.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.279+0000 7f18d0800640 1 -- 172.21.15.91:0/1779369322 wait complete. 2024-02-02T13:24:36.303 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:24:36.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.490+0000 7fe42a200640 1 Processor -- start 2024-02-02T13:24:36.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.490+0000 7fe42a200640 1 -- start start 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe42415a010 0x7fe4240c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 0x7fe4240ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4240cd470 0x7fe4240b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4240cc7c0 con 0x7fe4240cd470 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4240ce930 con 0x7fe4240ce130 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.491+0000 7fe42a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42406c890 con 0x7fe42415a010 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4240cd470 0x7fe4240b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4240cd470 0x7fe4240b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54612/0 (socket says 172.21.15.91:54612) 2024-02-02T13:24:36.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 -- 172.21.15.91:0/3125420050 learned_addr learned my addr 172.21.15.91:0/3125420050 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 -- 172.21.15.91:0/3125420050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe42415a010 msgr2=0x7fe4240c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423400640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe42415a010 0x7fe4240c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe422a00640 1 --2- 172.21.15.91:0/3125420050 >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 0x7fe4240ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe42415a010 0x7fe4240c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 -- 172.21.15.91:0/3125420050 >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 msgr2=0x7fe4240ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 --2- 172.21.15.91:0/3125420050 >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 0x7fe4240ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe423e00640 1 -- 172.21.15.91:0/3125420050 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42406cb70 con 0x7fe4240cd470 2024-02-02T13:24:36.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.492+0000 7fe422a00640 1 --2- 172.21.15.91:0/3125420050 >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 0x7fe4240ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.493+0000 7fe423e00640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240cd470 0x7fe4240b0700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe42406d7d0 tx=0x7fe41c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d022fa44103089a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.493+0000 7fe422000640 1 -- 172.21.15.91:0/3125420050 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c031870 con 0x7fe4240cd470 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.493+0000 7fe422000640 1 -- 172.21.15.91:0/3125420050 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c0319d0 con 0x7fe4240cd470 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.493+0000 7fe422000640 1 -- 172.21.15.91:0/3125420050 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c031d60 con 0x7fe4240cd470 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 -- 172.21.15.91:0/3125420050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240cd470 msgr2=0x7fe4240b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240cd470 0x7fe4240b0700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe42406d7d0 tx=0x7fe41c02b910 comp rx=0 tx=0).stop 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 -- 172.21.15.91:0/3125420050 shutdown_connections 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240cd470 0x7fe4240b0700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 --2- 172.21.15.91:0/3125420050 >> v2:172.21.15.113:3300/0 conn(0x7fe4240ce130 0x7fe4240ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 --2- 172.21.15.91:0/3125420050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe42415a010 0x7fe4240c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 -- 172.21.15.91:0/3125420050 >> 172.21.15.91:0/3125420050 conn(0x7fe4240cec80 msgr2=0x7fe4240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 -- 172.21.15.91:0/3125420050 shutdown_connections 2024-02-02T13:24:36.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.494+0000 7fe42a200640 1 -- 172.21.15.91:0/3125420050 wait complete. 2024-02-02T13:24:36.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.496+0000 7fe42a200640 1 Processor -- start 2024-02-02T13:24:36.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.496+0000 7fe42a200640 1 -- start start 2024-02-02T13:24:36.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42415a010 0x7fe42415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4240c7a40 0x7fe42415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424150a00 con 0x7fe4240ce130 2024-02-02T13:24:36.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240c7fa0 con 0x7fe42415a010 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe42a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240c8280 con 0x7fe4240c7a40 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe423400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe423400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54622/0 (socket says 172.21.15.91:54622) 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.497+0000 7fe423400640 1 -- 172.21.15.91:0/1258059508 learned_addr learned my addr 172.21.15.91:0/1258059508 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe422a00640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42415a010 0x7fe42415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423e00640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4240c7a40 0x7fe42415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 -- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4240c7a40 msgr2=0x7fe42415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4240c7a40 0x7fe42415ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 -- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42415a010 msgr2=0x7fe42415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42415a010 0x7fe42415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 -- 172.21.15.91:0/1258059508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c0094d0 con 0x7fe4240ce130 2024-02-02T13:24:36.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe423400640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ccd0 tx=0x7fe41400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414003990 con 0x7fe4240ce130 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414003af0 con 0x7fe4240ce130 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414004020 con 0x7fe4240ce130 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.498+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4240c8560 con 0x7fe4240ce130 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.499+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe424150b70 con 0x7fe4240ce130 2024-02-02T13:24:36.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.500+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4240c80e0 con 0x7fe4240ce130 2024-02-02T13:24:36.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.501+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe414004180 con 0x7fe4240ce130 2024-02-02T13:24:36.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.501+0000 7fe41b400640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 0x7fe3fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.502+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe414088c90 con 0x7fe4240ce130 2024-02-02T13:24:36.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.502+0000 7fe422a00640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 0x7fe3fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.502+0000 7fe422a00640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 0x7fe3fc06c8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00a990 tx=0x7fe40c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.507+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe4240c80e0 con 0x7fe4240ce130 2024-02-02T13:24:36.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.732+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe4240c80e0 con 0x7fe4240ce130 2024-02-02T13:24:36.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.744+0000 7fe41b400640 1 -- 172.21.15.91:0/1258059508 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe4240c80e0 con 0x7fe4240ce130 2024-02-02T13:24:36.745 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 2 and data pool 3 2024-02-02T13:24:36.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 msgr2=0x7fe3fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 0x7fe3fc06c8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00a990 tx=0x7fe40c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:36.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 msgr2=0x7fe4240c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ccd0 tx=0x7fe41400a5c0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 shutdown_connections 2024-02-02T13:24:36.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3fc06a430 0x7fe3fc06c8f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4240c7a40 0x7fe42415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42415a010 0x7fe42415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 --2- 172.21.15.91:0/1258059508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4240ce130 0x7fe4240c7500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.748+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 >> 172.21.15.91:0/1258059508 conn(0x7fe4240cec80 msgr2=0x7fe42415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:36.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.750+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 shutdown_connections 2024-02-02T13:24:36.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.750+0000 7fe42a200640 1 -- 172.21.15.91:0/1258059508 wait complete. 2024-02-02T13:24:36.778 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:36.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.956+0000 7fc983600640 1 Processor -- start 2024-02-02T13:24:36.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.956+0000 7fc983600640 1 -- start start 2024-02-02T13:24:36.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.957+0000 7fc983600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 0x7fc97c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.957+0000 7fc983600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.957+0000 7fc983600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c15fa00 con 0x7fc97c151f90 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.957+0000 7fc983600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0b10d0 con 0x7fc97c0b0c40 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.957+0000 7fc983600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c070f80 con 0x7fc97c15f5a0 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.958+0000 7fc981600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.958+0000 7fc97be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 0x7fc97c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.958+0000 7fc981600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:59926/0 (socket says 172.21.15.91:59926) 2024-02-02T13:24:36.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.958+0000 7fc981600640 1 -- 172.21.15.91:0/558040696 learned_addr learned my addr 172.21.15.91:0/558040696 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97b400640 1 -- 172.21.15.91:0/558040696 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073026278 0 0) 0x7fc96c001490 con 0x7fc97c151f90 2024-02-02T13:24:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97b400640 1 -- 172.21.15.91:0/558040696 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c003520 con 0x7fc97c151f90 2024-02-02T13:24:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97be00640 1 -- 172.21.15.91:0/558040696 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 msgr2=0x7fc97c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97be00640 1 --2- 172.21.15.91:0/558040696 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97be00640 1 -- 172.21.15.91:0/558040696 >> v1:172.21.15.113:6789/0 conn(0x7fc97c151f90 legacy=0x7fc97c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc97be00640 1 -- 172.21.15.91:0/558040696 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c152370 con 0x7fc97c15f5a0 2024-02-02T13:24:36.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.959+0000 7fc981600640 1 --2- 172.21.15.91:0/558040696 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:36.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.960+0000 7fc97be00640 1 --2- 172.21.15.91:0/558040696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 0x7fc97c0b0700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097a0 tx=0x7fc96402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805d70c96bbbc450 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.960+0000 7fc97b400640 1 -- 172.21.15.91:0/558040696 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964031830 con 0x7fc97c15f5a0 2024-02-02T13:24:36.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.960+0000 7fc97b400640 1 -- 172.21.15.91:0/558040696 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964031990 con 0x7fc97c15f5a0 2024-02-02T13:24:36.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.960+0000 7fc97b400640 1 -- 172.21.15.91:0/558040696 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964031d00 con 0x7fc97c15f5a0 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 -- 172.21.15.91:0/558040696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 msgr2=0x7fc97c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 --2- 172.21.15.91:0/558040696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 0x7fc97c0b0700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097a0 tx=0x7fc96402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 -- 172.21.15.91:0/558040696 shutdown_connections 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 --2- 172.21.15.91:0/558040696 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b0c40 0x7fc97c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 --2- 172.21.15.91:0/558040696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c15f5a0 0x7fc97c0b0700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 -- 172.21.15.91:0/558040696 >> 172.21.15.91:0/558040696 conn(0x7fc97c0cec80 msgr2=0x7fc97c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 -- 172.21.15.91:0/558040696 shutdown_connections 2024-02-02T13:24:36.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.961+0000 7fc983600640 1 -- 172.21.15.91:0/558040696 wait complete. 2024-02-02T13:24:36.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.963+0000 7fc983600640 1 Processor -- start 2024-02-02T13:24:36.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.964+0000 7fc983600640 1 -- start start 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c0b0c40 0x7fc97c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c151f90 0x7fc97c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c6f70 con 0x7fc97c15f5a0 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c7210 con 0x7fc97c151f90 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc983600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c74f0 con 0x7fc97c0b0c40 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc981600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc981600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54632/0 (socket says 172.21.15.91:54632) 2024-02-02T13:24:36.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc981600640 1 -- 172.21.15.91:0/1101106851 learned_addr learned my addr 172.21.15.91:0/1101106851 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc97be00640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c151f90 0x7fc97c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.965+0000 7fc980c00640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c0b0c40 0x7fc97c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 -- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c0b0c40 msgr2=0x7fc97c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c0b0c40 0x7fc97c0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 -- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c151f90 msgr2=0x7fc97c0c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c151f90 0x7fc97c0c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 -- 172.21.15.91:0/1101106851 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740094d0 con 0x7fc97c15f5a0 2024-02-02T13:24:36.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc981600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e870 tx=0x7fc97400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974004020 con 0x7fc97c15f5a0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc974004180 con 0x7fc97c15f5a0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400c940 con 0x7fc97c15f5a0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964009400 con 0x7fc97c15f5a0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.966+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0c7ab0 con 0x7fc97c15f5a0 2024-02-02T13:24:36.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.967+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c06a010 con 0x7fc97c15f5a0 2024-02-02T13:24:36.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.968+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc97400caa0 con 0x7fc97c15f5a0 2024-02-02T13:24:36.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.969+0000 7fc978c00640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:36.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.969+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc974088780 con 0x7fc97c15f5a0 2024-02-02T13:24:36.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.970+0000 7fc980c00640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:36.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.970+0000 7fc980c00640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0015f0 tx=0x7fc96c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:36.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:36.974+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc974053d70 con 0x7fc97c15f5a0 2024-02-02T13:24:37.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.165+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc97c066cf0 con 0x7fc97c15f5a0 2024-02-02T13:24:37.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.168+0000 7fc978c00640 1 -- 172.21.15.91:0/1101106851 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16050 (secure 0 0 0) 0x7fc974057390 con 0x7fc97c15f5a0 2024-02-02T13:24:37.171 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:37.171 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":19,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:36.734305+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:37.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 msgr2=0x7fc95406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0015f0 tx=0x7fc96c009040 comp rx=0 tx=0).stop 2024-02-02T13:24:37.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 msgr2=0x7fc97c0c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e870 tx=0x7fc97400ed30 comp rx=0 tx=0).stop 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 shutdown_connections 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c15f5a0 0x7fc97c0c32c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c151f90 0x7fc97c0c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 --2- 172.21.15.91:0/1101106851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c0b0c40 0x7fc97c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.171+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 >> 172.21.15.91:0/1101106851 conn(0x7fc97c0cec80 msgr2=0x7fc97c159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.172+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 shutdown_connections 2024-02-02T13:24:37.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.172+0000 7fc983600640 1 -- 172.21.15.91:0/1101106851 wait complete. 2024-02-02T13:24:37.191 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:37.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.397+0000 7f3250800640 1 Processor -- start 2024-02-02T13:24:37.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.397+0000 7f3250800640 1 -- start start 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f3250800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32480ce130 0x7f32480c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f3250800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480cc7c0 con 0x7f324815a010 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f3250800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480ce930 con 0x7f32480b0700 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f3250800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32480cd640 con 0x7f32480ce130 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f324de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f324815a010 0x7f32480ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52520/0 (socket says 172.21.15.91:52520) 2024-02-02T13:24:37.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.398+0000 7f324de00640 1 -- 172.21.15.91:0/3124661699 learned_addr learned my addr 172.21.15.91:0/3124661699 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893242083 0 0) 0x7f32480cc7c0 con 0x7f324815a010 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324d400640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32480ce130 0x7f32480c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3228003520 con 0x7f324815a010 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795911399 0 0) 0x7f3228003520 con 0x7f324815a010 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32480ce130 msgr2=0x7f32480c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32480ce130 0x7f32480c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 >> v1:172.21.15.113:6789/0 conn(0x7f32480b0700 legacy=0x7f324806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324806c890 con 0x7f324815a010 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3238001670 con 0x7f324815a010 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 >> v1:172.21.15.91:6789/0 conn(0x7f324815a010 legacy=0x7f32480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32280039a0 0x7f3228003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32280047c0 0x7f3228004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3228005180 0x7f322800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480cd640 con 0x7f3228005180 2024-02-02T13:24:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322800c080 con 0x7f32280039a0 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322800c240 con 0x7f32280047c0 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.399+0000 7f324d400640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32480ce130 0x7f32480c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3228005180 0x7f322800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324de00640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32280039a0 0x7f3228003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324d400640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32280047c0 0x7f3228004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 -- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32280047c0 msgr2=0x7f3228004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32280047c0 0x7f3228004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 -- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32280039a0 msgr2=0x7f3228003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32280039a0 0x7f3228003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.400+0000 7f324e800640 1 -- 172.21.15.91:0/3124661699 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228003520 con 0x7f3228005180 2024-02-02T13:24:37.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.401+0000 7f324e800640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3228005180 0x7f322800b980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3240001e50 tx=0x7f32280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf0520cad75d413 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240003b70 con 0x7f3228005180 2024-02-02T13:24:37.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240003cd0 con 0x7f3228005180 2024-02-02T13:24:37.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f324ca00640 1 -- 172.21.15.91:0/3124661699 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240005310 con 0x7f3228005180 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f3250800640 1 -- 172.21.15.91:0/3124661699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3228005180 msgr2=0x7f322800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f3250800640 1 --2- 172.21.15.91:0/3124661699 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3228005180 0x7f322800b980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3240001e50 tx=0x7f32280057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f324e800640 1 -- 172.21.15.91:0/3124661699 reap_dead start 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f3250800640 1 -- 172.21.15.91:0/3124661699 shutdown_connections 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.402+0000 7f3250800640 1 -- 172.21.15.91:0/3124661699 >> 172.21.15.91:0/3124661699 conn(0x7f32480cec80 msgr2=0x7f32480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.403+0000 7f3250800640 1 -- 172.21.15.91:0/3124661699 shutdown_connections 2024-02-02T13:24:37.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.403+0000 7f3250800640 1 -- 172.21.15.91:0/3124661699 wait complete. 2024-02-02T13:24:37.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.404+0000 7f3250800640 1 Processor -- start 2024-02-02T13:24:37.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.404+0000 7f3250800640 1 -- start start 2024-02-02T13:24:37.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3228005180 0x7f3248155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324815c590 0x7f324815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480c80b0 con 0x7f32480c76f0 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480ac970 con 0x7f3228005180 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.405+0000 7f3250800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32480ad9d0 con 0x7f324815c590 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54654/0 (socket says 172.21.15.91:54654) 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 -- 172.21.15.91:0/622218365 learned_addr learned my addr 172.21.15.91:0/622218365 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324de00640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3228005180 0x7f3248155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324e800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324815c590 0x7f324815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 -- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324815c590 msgr2=0x7f324815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324815c590 0x7f324815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 -- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3228005180 msgr2=0x7f3248155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3228005180 0x7f3248155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 -- 172.21.15.91:0/622218365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480acc10 con 0x7f32480c76f0 2024-02-02T13:24:37.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f324d400640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3230010810 tx=0x7f3230010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000c930 con 0x7f32480c76f0 2024-02-02T13:24:37.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000ca90 con 0x7f32480c76f0 2024-02-02T13:24:37.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000ce40 con 0x7f32480c76f0 2024-02-02T13:24:37.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.406+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32480acef0 con 0x7f32480c76f0 2024-02-02T13:24:37.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.407+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32480ad4b0 con 0x7f32480c76f0 2024-02-02T13:24:37.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.408+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248066cf0 con 0x7f32480c76f0 2024-02-02T13:24:37.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.409+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3230015470 con 0x7f32480c76f0 2024-02-02T13:24:37.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.409+0000 7f3246000640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 0x7f322006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.409+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32300890f0 con 0x7f32480c76f0 2024-02-02T13:24:37.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.410+0000 7f324de00640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 0x7f322006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.410+0000 7f324de00640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 0x7f322006c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3238001eb0 tx=0x7f3238000f50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.414+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3230054700 con 0x7f32480c76f0 2024-02-02T13:24:37.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.603+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32480b4c00 con 0x7f32480c76f0 2024-02-02T13:24:37.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.605+0000 7f3246000640 1 -- 172.21.15.91:0/622218365 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f3230057d20 con 0x7f32480c76f0 2024-02-02T13:24:37.607 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:37.607 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":19,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:36.734305+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.607+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 msgr2=0x7f322006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.607+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 0x7f322006c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3238001eb0 tx=0x7f3238000f50 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.607+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 msgr2=0x7f32480c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.607+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3230010810 tx=0x7f3230010cd0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 shutdown_connections 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f322006a430 0x7f322006c8f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324815c590 0x7f324815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32480c76f0 0x7f32480c7b70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 --2- 172.21.15.91:0/622218365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3228005180 0x7f3248155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 >> 172.21.15.91:0/622218365 conn(0x7f32480cec80 msgr2=0x7f324806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 shutdown_connections 2024-02-02T13:24:37.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.608+0000 7f3250800640 1 -- 172.21.15.91:0/622218365 wait complete. 2024-02-02T13:24:37.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:24:37.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.835+0000 7f6b8c200640 1 Processor -- start 2024-02-02T13:24:37.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.835+0000 7f6b8c200640 1 -- start start 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8c200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6b840c9f90 0x7f6b840b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 0x7f6b840ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8c200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b840b1020 con 0x7f6b8415a1e0 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8c200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840cca70 con 0x7f6b840c9f90 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840cc770 con 0x7f6b840b0bd0 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b89800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6b8415a1e0 0x7f6b84070ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52534/0 (socket says 172.21.15.91:52534) 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b89800640 1 -- 172.21.15.91:0/990607377 learned_addr learned my addr 172.21.15.91:0/990607377 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b88e00640 1 --2- 172.21.15.91:0/990607377 >> v2:172.21.15.91:3300/0 conn(0x7f6b840c9f90 0x7f6b840b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b8a200640 1 --2- 172.21.15.91:0/990607377 >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 0x7f6b840ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.837+0000 7f6b88e00640 1 -- 172.21.15.91:0/990607377 >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 msgr2=0x7f6b840ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.838+0000 7f6b88e00640 1 --2- 172.21.15.91:0/990607377 >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 0x7f6b840ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.838+0000 7f6b88e00640 1 -- 172.21.15.91:0/990607377 >> v1:172.21.15.91:6789/0 conn(0x7f6b8415a1e0 legacy=0x7f6b84070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.838+0000 7f6b83e00640 1 -- 172.21.15.91:0/990607377 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892811896 0 0) 0x7f6b74001110 con 0x7f6b8415a1e0 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.838+0000 7f6b88e00640 1 -- 172.21.15.91:0/990607377 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840ca570 con 0x7f6b840c9f90 2024-02-02T13:24:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.838+0000 7f6b8a200640 1 --2- 172.21.15.91:0/990607377 >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 0x7f6b840ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b88e00640 1 --2- 172.21.15.91:0/990607377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b840c9f90 0x7f6b840b0690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9618c9d71275c2a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b83e00640 1 -- 172.21.15.91:0/990607377 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c031830 con 0x7f6b840c9f90 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b83e00640 1 -- 172.21.15.91:0/990607377 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c031990 con 0x7f6b840c9f90 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b83e00640 1 -- 172.21.15.91:0/990607377 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c031d20 con 0x7f6b840c9f90 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b8c200640 1 -- 172.21.15.91:0/990607377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b840c9f90 msgr2=0x7f6b840b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.839+0000 7f6b8c200640 1 --2- 172.21.15.91:0/990607377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b840c9f90 0x7f6b840b0690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 -- 172.21.15.91:0/990607377 shutdown_connections 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 --2- 172.21.15.91:0/990607377 >> v2:172.21.15.113:3300/0 conn(0x7f6b840b0bd0 0x7f6b840ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 --2- 172.21.15.91:0/990607377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b840c9f90 0x7f6b840b0690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 -- 172.21.15.91:0/990607377 >> 172.21.15.91:0/990607377 conn(0x7f6b840ceb90 msgr2=0x7f6b840cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 -- 172.21.15.91:0/990607377 shutdown_connections 2024-02-02T13:24:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.840+0000 7f6b8c200640 1 -- 172.21.15.91:0/990607377 wait complete. 2024-02-02T13:24:37.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.841+0000 7f6b8c200640 1 Processor -- start 2024-02-02T13:24:37.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.841+0000 7f6b8c200640 1 -- start start 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 0x7f6b840ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b840c9f90 0x7f6b8414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8414ea40 con 0x7f6b8415a1e0 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840acdd0 con 0x7f6b840b0bd0 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.842+0000 7f6b8c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840ad0b0 con 0x7f6b840c9f90 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54676/0 (socket says 172.21.15.91:54676) 2024-02-02T13:24:37.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 -- 172.21.15.91:0/1622748389 learned_addr learned my addr 172.21.15.91:0/1622748389 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b88e00640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b840c9f90 0x7f6b8414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 -- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b840c9f90 msgr2=0x7f6b8414e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b840c9f90 0x7f6b8414e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b89800640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 0x7f6b840ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 -- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 msgr2=0x7f6b840ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 0x7f6b840ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 -- 172.21.15.91:0/1622748389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009400 con 0x7f6b8415a1e0 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b89800640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 0x7f6b840ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.843+0000 7f6b8a200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00cef0 tx=0x7f6b7c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.844+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00d840 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.844+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c00d9c0 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.844+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00dd70 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.844+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b840ad390 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.844+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b840ad950 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.845+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b840adb40 con 0x7f6b8415a1e0 2024-02-02T13:24:37.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.846+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6b7c01e3b0 con 0x7f6b8415a1e0 2024-02-02T13:24:37.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.847+0000 7f6b81600640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 0x7f6b5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:37.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.847+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b7c089480 con 0x7f6b8415a1e0 2024-02-02T13:24:37.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.847+0000 7f6b89800640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 0x7f6b5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.848+0000 7f6b89800640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 0x7f6b5c06c8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6b74000f60 tx=0x7f6b74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:37.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:37.851+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6b7c054a90 con 0x7f6b8415a1e0 2024-02-02T13:24:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.077+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b84066cf0 con 0x7f6b8415a1e0 2024-02-02T13:24:38.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.755+0000 7f6b81600640 1 -- 172.21.15.91:0/1622748389 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b7c0580b0 con 0x7f6b8415a1e0 2024-02-02T13:24:38.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 msgr2=0x7f6b5c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:38.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 0x7f6b5c06c8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6b74000f60 tx=0x7f6b74009040 comp rx=0 tx=0).stop 2024-02-02T13:24:38.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 msgr2=0x7f6b840c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:38.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00cef0 tx=0x7f6b7c00a710 comp rx=0 tx=0).stop 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 shutdown_connections 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5c06a430 0x7f6b5c06c8f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b840c9f90 0x7f6b8414e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8415a1e0 0x7f6b840c65b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 --2- 172.21.15.91:0/1622748389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b840b0bd0 0x7f6b840ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 >> 172.21.15.91:0/1622748389 conn(0x7f6b840ceb90 msgr2=0x7f6b840cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 shutdown_connections 2024-02-02T13:24:38.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.760+0000 7f6b8c200640 1 -- 172.21.15.91:0/1622748389 wait complete. 2024-02-02T13:24:38.783 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2024-02-02T13:24:38.783 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:24:38.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.993+0000 7f92fe200640 1 Processor -- start 2024-02-02T13:24:38.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.993+0000 7f92fe200640 1 -- start start 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fe200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92f8159c50 0x7f92f80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fe200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80ca740 con 0x7f92f806cce0 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fe200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f815a030 con 0x7f92f806fc00 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fe200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f815d710 con 0x7f92f8159c50 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92f8159c50 0x7f92f80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92f8159c50 0x7f92f80cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54680/0 (socket says 172.21.15.91:54680) 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 -- 172.21.15.91:0/600105934 learned_addr learned my addr 172.21.15.91:0/600105934 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fd800640 1 --1- 172.21.15.91:0/600105934 >> v1:172.21.15.91:6789/0 conn(0x7f92f806cce0 0x7f92f815db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52542/0 (socket says 172.21.15.91:52542) 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 -- 172.21.15.91:0/600105934 >> v1:172.21.15.113:6789/0 conn(0x7f92f806fc00 legacy=0x7f92f806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:38.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 -- 172.21.15.91:0/600105934 >> v1:172.21.15.91:6789/0 conn(0x7f92f806cce0 legacy=0x7f92f815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92fce00640 1 -- 172.21.15.91:0/600105934 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80c9bc0 con 0x7f92f8159c50 2024-02-02T13:24:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.995+0000 7f92f7400640 1 -- 172.21.15.91:0/600105934 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943385828 0 0) 0x7f92f0001490 con 0x7f92f806cce0 2024-02-02T13:24:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.997+0000 7f92fce00640 1 --2- 172.21.15.91:0/600105934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f8159c50 0x7f92f80cbe00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009870 tx=0x7f92e802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922a312b575dda8c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.997+0000 7f92f7400640 1 -- 172.21.15.91:0/600105934 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8030850 con 0x7f92f8159c50 2024-02-02T13:24:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.997+0000 7f92f7400640 1 -- 172.21.15.91:0/600105934 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e80309b0 con 0x7f92f8159c50 2024-02-02T13:24:38.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.997+0000 7f92f7400640 1 -- 172.21.15.91:0/600105934 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8030d40 con 0x7f92f8159c50 2024-02-02T13:24:38.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 -- 172.21.15.91:0/600105934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f8159c50 msgr2=0x7f92f80cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:38.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 --2- 172.21.15.91:0/600105934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f8159c50 0x7f92f80cbe00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009870 tx=0x7f92e802b990 comp rx=0 tx=0).stop 2024-02-02T13:24:38.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 -- 172.21.15.91:0/600105934 shutdown_connections 2024-02-02T13:24:39.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 --2- 172.21.15.91:0/600105934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f8159c50 0x7f92f80cbe00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:39.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 -- 172.21.15.91:0/600105934 >> 172.21.15.91:0/600105934 conn(0x7f92f80b0560 msgr2=0x7f92f80b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:39.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 -- 172.21.15.91:0/600105934 shutdown_connections 2024-02-02T13:24:39.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:38.998+0000 7f92fe200640 1 -- 172.21.15.91:0/600105934 wait complete. 2024-02-02T13:24:39.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.000+0000 7f92fe200640 1 Processor -- start 2024-02-02T13:24:39.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.000+0000 7f92fe200640 1 -- start start 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f806cce0 0x7f92f80c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 0x7f92f80b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80c5580 con 0x7f92f806fc00 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80c4a10 con 0x7f92f8159c50 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fe200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80c4ce0 con 0x7f92f806cce0 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54686/0 (socket says 172.21.15.91:54686) 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 -- 172.21.15.91:0/118472651 learned_addr learned my addr 172.21.15.91:0/118472651 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fd800640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 0x7f92f80b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fce00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f806cce0 0x7f92f80c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:39.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 -- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f806cce0 msgr2=0x7f92f80c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f806cce0 0x7f92f80c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 -- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 msgr2=0x7f92f80b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 0x7f92f80b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92f7e00640 1 -- 172.21.15.91:0/118472651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80094d0 con 0x7f92f806fc00 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.001+0000 7f92fd800640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 0x7f92f80b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92f7e00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92e000c430 tx=0x7f92e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0011790 con 0x7f92f806fc00 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e00118f0 con 0x7f92f806fc00 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80c4fc0 con 0x7f92f806fc00 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0011ce0 con 0x7f92f806fc00 2024-02-02T13:24:39.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.002+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f80c5c30 con 0x7f92f806fc00 2024-02-02T13:24:39.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.003+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f80cc2c0 con 0x7f92f806fc00 2024-02-02T13:24:39.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.004+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92e001c460 con 0x7f92f806fc00 2024-02-02T13:24:39.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.005+0000 7f92f4c00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 0x7f92d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:39.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.005+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92e0088880 con 0x7f92f806fc00 2024-02-02T13:24:39.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.005+0000 7f92fce00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 0x7f92d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:39.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.006+0000 7f92fce00640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 0x7f92d006c8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92e8036000 tx=0x7f92e803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.009+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f92e0053e60 con 0x7f92f806fc00 2024-02-02T13:24:39.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:39.240+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f92f804dff0 con 0x7f92f806fc00 2024-02-02T13:24:40.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.241+0000 7f92f4c00640 1 -- 172.21.15.91:0/118472651 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92e0057480 con 0x7f92f806fc00 2024-02-02T13:24:40.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 msgr2=0x7f92d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 0x7f92d006c8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92e8036000 tx=0x7f92e803b040 comp rx=0 tx=0).stop 2024-02-02T13:24:40.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 msgr2=0x7f92f80c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92e000c430 tx=0x7f92e000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 shutdown_connections 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92d006a430 0x7f92d006c8f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f8159c50 0x7f92f80b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f806fc00 0x7f92f80c44d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 --2- 172.21.15.91:0/118472651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f806cce0 0x7f92f80c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.244+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 >> 172.21.15.91:0/118472651 conn(0x7f92f80b0560 msgr2=0x7f92f80ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.245+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 shutdown_connections 2024-02-02T13:24:40.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.245+0000 7f92fe200640 1 -- 172.21.15.91:0/118472651 wait complete. 2024-02-02T13:24:40.266 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:24:40.266 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:24:40.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.470+0000 7fade4a00640 1 Processor -- start 2024-02-02T13:24:40.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.470+0000 7fade4a00640 1 -- start start 2024-02-02T13:24:40.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.471+0000 7fade4a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fade4a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade00d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fade4a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade00b7d40 con 0x7fade015cee0 2024-02-02T13:24:40.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fade4a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00b81d0 con 0x7fade00ce1d0 2024-02-02T13:24:40.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fade4a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00b7130 con 0x7fade00b6990 2024-02-02T13:24:40.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fadd7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7faddf200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade00d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fadd7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57040/0 (socket says 172.21.15.91:57040) 2024-02-02T13:24:40.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.472+0000 7fadd7e00640 1 -- 172.21.15.91:0/1405433410 learned_addr learned my addr 172.21.15.91:0/1405433410 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:40.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.473+0000 7faddf200640 1 -- 172.21.15.91:0/1405433410 >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 msgr2=0x7fade00b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:40.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.473+0000 7faddf200640 1 --2- 172.21.15.91:0/1405433410 >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.473+0000 7faddf200640 1 -- 172.21.15.91:0/1405433410 >> v1:172.21.15.113:6789/0 conn(0x7fade015cee0 legacy=0x7fade00cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.473+0000 7faddf200640 1 -- 172.21.15.91:0/1405433410 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00b7410 con 0x7fade00b6990 2024-02-02T13:24:40.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.473+0000 7fadd7e00640 1 --2- 172.21.15.91:0/1405433410 >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:40.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.474+0000 7faddf200640 1 --2- 172.21.15.91:0/1405433410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade00d1130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009d90 tx=0x7fadd802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e81fecff703906 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.475+0000 7faddde00640 1 -- 172.21.15.91:0/1405433410 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd802cc50 con 0x7fade00b6990 2024-02-02T13:24:40.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.475+0000 7faddde00640 1 -- 172.21.15.91:0/1405433410 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd802cdb0 con 0x7fade00b6990 2024-02-02T13:24:40.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.475+0000 7faddde00640 1 -- 172.21.15.91:0/1405433410 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8034660 con 0x7fade00b6990 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.475+0000 7fade4a00640 1 -- 172.21.15.91:0/1405433410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 msgr2=0x7fade00d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.475+0000 7fade4a00640 1 --2- 172.21.15.91:0/1405433410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade00d1130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009d90 tx=0x7fadd802bea0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 -- 172.21.15.91:0/1405433410 shutdown_connections 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 --2- 172.21.15.91:0/1405433410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade00d1130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 --2- 172.21.15.91:0/1405433410 >> v2:172.21.15.113:3300/0 conn(0x7fade00ce1d0 0x7fade00b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 -- 172.21.15.91:0/1405433410 >> 172.21.15.91:0/1405433410 conn(0x7fade0070690 msgr2=0x7fade0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:40.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 -- 172.21.15.91:0/1405433410 shutdown_connections 2024-02-02T13:24:40.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.476+0000 7fade4a00640 1 -- 172.21.15.91:0/1405433410 wait complete. 2024-02-02T13:24:40.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.477+0000 7fade4a00640 1 Processor -- start 2024-02-02T13:24:40.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.477+0000 7fade4a00640 1 -- start start 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade0152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015cee0 0x7fade00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade015d2c0 con 0x7fade00ce1d0 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade015db60 con 0x7fade015cee0 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.478+0000 7fade4a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00b4eb0 con 0x7fade00b6990 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53072/0 (socket says 172.21.15.91:53072) 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 -- 172.21.15.91:0/3184919916 learned_addr learned my addr 172.21.15.91:0/3184919916 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadde800640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade0152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7faddf200640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015cee0 0x7fade00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 -- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 msgr2=0x7fade0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade0152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 -- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015cee0 msgr2=0x7fade00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015cee0 0x7fade00b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 -- 172.21.15.91:0/3184919916 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80094d0 con 0x7fade00ce1d0 2024-02-02T13:24:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.479+0000 7fadd7e00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e9d0 tx=0x7fadc800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.480+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800ca60 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.480+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800cbc0 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.480+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8013470 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.480+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8009a70 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.480+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade015d7f0 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.481+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade00b79b0 con 0x7fade00ce1d0 2024-02-02T13:24:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.482+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fadc80135d0 con 0x7fade00ce1d0 2024-02-02T13:24:40.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.482+0000 7fadd6a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.483+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadc8088c80 con 0x7fade00ce1d0 2024-02-02T13:24:40.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.483+0000 7fadde800640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.483+0000 7fadde800640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000f60 tx=0x7fadd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.487+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fade00b79b0 con 0x7fade00ce1d0 2024-02-02T13:24:40.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.708+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fade00b79b0 con 0x7fade00ce1d0 2024-02-02T13:24:40.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.710+0000 7fadd6a00640 1 -- 172.21.15.91:0/3184919916 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fade00b79b0 con 0x7fade00ce1d0 2024-02-02T13:24:40.711 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:40.712 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:24:40.712 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 msgr2=0x7fadb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000f60 tx=0x7fadd0009040 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 msgr2=0x7fade01f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e9d0 tx=0x7fadc800ee90 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 shutdown_connections 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015cee0 0x7fade00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00ce1d0 0x7fade01f1d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 --2- 172.21.15.91:0/3184919916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade00b6990 0x7fade0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 >> 172.21.15.91:0/3184919916 conn(0x7fade0070690 msgr2=0x7fade00c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 shutdown_connections 2024-02-02T13:24:40.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.712+0000 7fade4a00640 1 -- 172.21.15.91:0/3184919916 wait complete. 2024-02-02T13:24:40.731 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:40.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.933+0000 7f39b4c00640 1 Processor -- start 2024-02-02T13:24:40.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.933+0000 7f39b4c00640 1 -- start start 2024-02-02T13:24:40.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.934+0000 7f39b4c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39b015f5a0 0x7f39b00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.934+0000 7f39b4c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39b00b0c40 0x7f39b00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.934+0000 7f39b4c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b00b1070 con 0x7f39b0151f90 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.934+0000 7f39b4c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b015fa30 con 0x7f39b015f5a0 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.934+0000 7f39b4c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b0070f80 con 0x7f39b00b0c40 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39b015f5a0 0x7f39b00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39b015f5a0 0x7f39b00b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53084/0 (socket says 172.21.15.91:53084) 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 -- 172.21.15.91:0/1432655285 learned_addr learned my addr 172.21.15.91:0/1432655285 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39ae800640 1 --2- 172.21.15.91:0/1432655285 >> v2:172.21.15.113:3300/0 conn(0x7f39b00b0c40 0x7f39b00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 -- 172.21.15.91:0/1432655285 >> v2:172.21.15.113:3300/0 conn(0x7f39b00b0c40 msgr2=0x7f39b00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 --2- 172.21.15.91:0/1432655285 >> v2:172.21.15.113:3300/0 conn(0x7f39b00b0c40 0x7f39b00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 -- 172.21.15.91:0/1432655285 >> v1:172.21.15.113:6789/0 conn(0x7f39b0151f90 legacy=0x7f39b006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:40.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.935+0000 7f39a7e00640 1 -- 172.21.15.91:0/1432655285 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b0152370 con 0x7f39b015f5a0 2024-02-02T13:24:40.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.936+0000 7f39a7e00640 1 --2- 172.21.15.91:0/1432655285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b00b0700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39b00ca520 tx=0x7f399802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1fcf9ba872abd9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39ad400640 1 -- 172.21.15.91:0/1432655285 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998031870 con 0x7f39b015f5a0 2024-02-02T13:24:40.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39ad400640 1 -- 172.21.15.91:0/1432655285 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39980319d0 con 0x7f39b015f5a0 2024-02-02T13:24:40.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39ad400640 1 -- 172.21.15.91:0/1432655285 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998031d60 con 0x7f39b015f5a0 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 -- 172.21.15.91:0/1432655285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 msgr2=0x7f39b00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1432655285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b00b0700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39b00ca520 tx=0x7f399802b910 comp rx=0 tx=0).stop 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 -- 172.21.15.91:0/1432655285 shutdown_connections 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1432655285 >> v2:172.21.15.113:3300/0 conn(0x7f39b00b0c40 0x7f39b00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1432655285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b00b0700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 -- 172.21.15.91:0/1432655285 >> 172.21.15.91:0/1432655285 conn(0x7f39b00cec80 msgr2=0x7f39b00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 -- 172.21.15.91:0/1432655285 shutdown_connections 2024-02-02T13:24:40.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.937+0000 7f39b4c00640 1 -- 172.21.15.91:0/1432655285 wait complete. 2024-02-02T13:24:40.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.939+0000 7f39b4c00640 1 Processor -- start 2024-02-02T13:24:40.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.939+0000 7f39b4c00640 1 -- start start 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b00b0c40 0x7f39b014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 0x7f39b00c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b014d7b0 con 0x7f39b015f5a0 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00c79f0 con 0x7f39b00b0c40 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39b4c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b015bf20 con 0x7f39b0151f90 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39ae800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39ae800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53096/0 (socket says 172.21.15.91:53096) 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39ae800640 1 -- 172.21.15.91:0/1762282475 learned_addr learned my addr 172.21.15.91:0/1762282475 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39ade00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b00b0c40 0x7f39b014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.940+0000 7f39a7e00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 0x7f39b00c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 -- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 msgr2=0x7f39b00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 0x7f39b00c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 -- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b00b0c40 msgr2=0x7f39b014d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b00b0c40 0x7f39b014d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 -- 172.21.15.91:0/1762282475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980094d0 con 0x7f39b015f5a0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39a7e00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 0x7f39b00c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39ae800640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007cf0 tx=0x7f39a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8014420 con 0x7f39b015f5a0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a80145a0 con 0x7f39b015f5a0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b015c170 con 0x7f39b015f5a0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8014950 con 0x7f39b015f5a0 2024-02-02T13:24:40.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.941+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b015c730 con 0x7f39b015f5a0 2024-02-02T13:24:40.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.942+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b0066cf0 con 0x7f39b015f5a0 2024-02-02T13:24:40.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.943+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f39a8014ab0 con 0x7f39b015f5a0 2024-02-02T13:24:40.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.945+0000 7f39a6000640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 0x7f398806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:40.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.945+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39a80891b0 con 0x7f39b015f5a0 2024-02-02T13:24:40.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.945+0000 7f39ade00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 0x7f398806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:40.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.945+0000 7f39ade00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 0x7f398806c8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39a000a730 tx=0x7f39a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:40.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:40.949+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f39a80547a0 con 0x7f39b015f5a0 2024-02-02T13:24:41.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.140+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b006c7b0 con 0x7f39b015f5a0 2024-02-02T13:24:41.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.142+0000 7f39a6000640 1 -- 172.21.15.91:0/1762282475 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f39a8057dc0 con 0x7f39b015f5a0 2024-02-02T13:24:41.144 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:41.144 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:41.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.144+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 msgr2=0x7f398806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 0x7f398806c8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39a000a730 tx=0x7f39a0008040 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 msgr2=0x7f39b015a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007cf0 tx=0x7f39a800e4b0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 shutdown_connections 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f398806a430 0x7f398806c8f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b0151f90 0x7f39b00c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39b015f5a0 0x7f39b015a690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 --2- 172.21.15.91:0/1762282475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b00b0c40 0x7f39b014d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 >> 172.21.15.91:0/1762282475 conn(0x7f39b00cec80 msgr2=0x7f39b00ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 shutdown_connections 2024-02-02T13:24:41.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.145+0000 7f39b4c00640 1 -- 172.21.15.91:0/1762282475 wait complete. 2024-02-02T13:24:41.163 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:24:41.164 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:41.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.365+0000 7f9529c00640 1 Processor -- start 2024-02-02T13:24:41.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.365+0000 7f9529c00640 1 -- start start 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.366+0000 7f9529c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95241597a0 0x7f952406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.366+0000 7f9529c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 0x7f95240cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.366+0000 7f9529c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952406b5f0 con 0x7f95240cd690 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.366+0000 7f9529c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952406c450 con 0x7f95241597a0 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.366+0000 7f9529c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9524159b80 con 0x7f95240c9950 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95241597a0 0x7f952406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95241597a0 0x7f952406c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53104/0 (socket says 172.21.15.91:53104) 2024-02-02T13:24:41.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 -- 172.21.15.91:0/3256349372 learned_addr learned my addr 172.21.15.91:0/3256349372 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:41.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9523400640 1 --2- 172.21.15.91:0/3256349372 >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 0x7f95240cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 -- 172.21.15.91:0/3256349372 >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 msgr2=0x7f95240cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:41.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 --2- 172.21.15.91:0/3256349372 >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 0x7f95240cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 -- 172.21.15.91:0/3256349372 >> v1:172.21.15.113:6789/0 conn(0x7f95240cd690 legacy=0x7f95240c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:41.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9522a00640 1 -- 172.21.15.91:0/3256349372 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952406b840 con 0x7f95241597a0 2024-02-02T13:24:41.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.367+0000 7f9523400640 1 --2- 172.21.15.91:0/3256349372 >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 0x7f95240cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:41.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.368+0000 7f9522a00640 1 --2- 172.21.15.91:0/3256349372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95241597a0 0x7f952406c8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95240b11b0 tx=0x7f951402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d0d105b15494ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.368+0000 7f9521600640 1 -- 172.21.15.91:0/3256349372 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514031870 con 0x7f95241597a0 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9521600640 1 -- 172.21.15.91:0/3256349372 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140319d0 con 0x7f95241597a0 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9521600640 1 -- 172.21.15.91:0/3256349372 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514031d60 con 0x7f95241597a0 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 -- 172.21.15.91:0/3256349372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95241597a0 msgr2=0x7f952406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 --2- 172.21.15.91:0/3256349372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95241597a0 0x7f952406c8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95240b11b0 tx=0x7f951402b910 comp rx=0 tx=0).stop 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 -- 172.21.15.91:0/3256349372 shutdown_connections 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 --2- 172.21.15.91:0/3256349372 >> v2:172.21.15.113:3300/0 conn(0x7f95240c9950 0x7f95240cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 --2- 172.21.15.91:0/3256349372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95241597a0 0x7f952406c8e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 -- 172.21.15.91:0/3256349372 >> 172.21.15.91:0/3256349372 conn(0x7f95240cec80 msgr2=0x7f95240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 -- 172.21.15.91:0/3256349372 shutdown_connections 2024-02-02T13:24:41.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.369+0000 7f9529c00640 1 -- 172.21.15.91:0/3256349372 wait complete. 2024-02-02T13:24:41.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.371+0000 7f9529c00640 1 Processor -- start 2024-02-02T13:24:41.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.371+0000 7f9529c00640 1 -- start start 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 0x7f95240c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95241597a0 0x7f95240c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240c6630 con 0x7f95240c9950 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240c6910 con 0x7f95240cd690 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9529c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95240c6bf0 con 0x7f95241597a0 2024-02-02T13:24:41.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9522a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9522a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53108/0 (socket says 172.21.15.91:53108) 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.372+0000 7f9522a00640 1 -- 172.21.15.91:0/2979059810 learned_addr learned my addr 172.21.15.91:0/2979059810 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9523400640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95241597a0 0x7f95240c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522000640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 0x7f95240c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 -- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95241597a0 msgr2=0x7f95240c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95241597a0 0x7f95240c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 -- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 msgr2=0x7f95240c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 0x7f95240c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 -- 172.21.15.91:0/2979059810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140094d0 con 0x7f95240c9950 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522000640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 0x7f95240c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9522a00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9514000c00 tx=0x7f95140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514031840 con 0x7f95240c9950 2024-02-02T13:24:41.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140443f0 con 0x7f95240c9950 2024-02-02T13:24:41.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140447a0 con 0x7f95240c9950 2024-02-02T13:24:41.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.373+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95240c6ed0 con 0x7f95240c9950 2024-02-02T13:24:41.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.374+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95240c6200 con 0x7f95240c9950 2024-02-02T13:24:41.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.375+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95240c73f0 con 0x7f95240c9950 2024-02-02T13:24:41.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.375+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f951404d450 con 0x7f95240c9950 2024-02-02T13:24:41.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.376+0000 7f951aa00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 0x7f94fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.376+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f951402d040 con 0x7f95240c9950 2024-02-02T13:24:41.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.377+0000 7f9522000640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 0x7f94fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.377+0000 7f9522000640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 0x7f94fc06c8f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f950c007fd0 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.381+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f95240c73f0 con 0x7f95240c9950 2024-02-02T13:24:41.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.567+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95240c6a80 con 0x7f94fc06a430 2024-02-02T13:24:41.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.579+0000 7f951aa00640 1 -- 172.21.15.91:0/2979059810 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252966 (secure 0 0 0) 0x7f95240c6a80 con 0x7f94fc06a430 2024-02-02T13:24:41.587 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:41.589 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.584+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 msgr2=0x7f94fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.584+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 0x7f94fc06c8f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f950c007fd0 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.584+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 msgr2=0x7f952414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.584+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9514000c00 tx=0x7f95140046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 shutdown_connections 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94fc06a430 0x7f94fc06c8f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95241597a0 0x7f95240c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95240cd690 0x7f95240c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 --2- 172.21.15.91:0/2979059810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95240c9950 0x7f952414d0e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 >> 172.21.15.91:0/2979059810 conn(0x7f95240cec80 msgr2=0x7f952415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 shutdown_connections 2024-02-02T13:24:41.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.585+0000 7f9529c00640 1 -- 172.21.15.91:0/2979059810 wait complete. 2024-02-02T13:24:41.604 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-02-02T13:24:40.236049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560949,"statfs":{"total":1600000000,"available":1598412236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992281"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267085+0000","last_change":"2024-02-02T13:24:36.267085+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267085+0000","last_undegraded":"2024-02-02T13:24:36.267085+0000","last_fullsized":"2024-02-02T13:24:36.267085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272655+0000","last_change":"2024-02-02T13:24:36.272655+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272655+0000","last_undegraded":"2024-02-02T13:24:36.272655+0000","last_fullsized":"2024-02-02T13:24:36.272655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.275029+0000","last_change":"2024-02-02T13:24:36.275029+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.275029+0000","last_undegraded":"2024-02-02T13:24:36.275029+0000","last_fullsized":"2024-02-02T13:24:36.275029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:59.975618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272328+0000","last_change":"2024-02-02T13:24:36.272328+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272328+0000","last_undegraded":"2024-02-02T13:24:36.272328+0000","last_fullsized":"2024-02-02T13:24:36.272328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:35.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.271716+0000","last_change":"2024-02-02T13:24:36.271716+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.271716+0000","last_undegraded":"2024-02-02T13:24:36.271716+0000","last_fullsized":"2024-02-02T13:24:36.271716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267852+0000","last_change":"2024-02-02T13:24:36.267852+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267852+0000","last_undegraded":"2024-02-02T13:24:36.267852+0000","last_fullsized":"2024-02-02T13:24:36.267852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264041+0000","last_change":"2024-02-02T13:24:36.264041+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264041+0000","last_undegraded":"2024-02-02T13:24:36.264041+0000","last_fullsized":"2024-02-02T13:24:36.264041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272976+0000","last_change":"2024-02-02T13:24:36.272976+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272976+0000","last_undegraded":"2024-02-02T13:24:36.272976+0000","last_fullsized":"2024-02-02T13:24:36.272976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:44.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267011+0000","last_change":"2024-02-02T13:24:36.267011+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267011+0000","last_undegraded":"2024-02-02T13:24:36.267011+0000","last_fullsized":"2024-02-02T13:24:36.267011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.266074+0000","last_change":"2024-02-02T13:24:36.266074+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.266074+0000","last_undegraded":"2024-02-02T13:24:36.266074+0000","last_fullsized":"2024-02-02T13:24:36.266074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:59.975618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.271537+0000","last_change":"2024-02-02T13:24:36.271537+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.271537+0000","last_undegraded":"2024-02-02T13:24:36.271537+0000","last_fullsized":"2024-02-02T13:24:36.271537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.265876+0000","last_change":"2024-02-02T13:24:36.265876+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.265876+0000","last_undegraded":"2024-02-02T13:24:36.265876+0000","last_fullsized":"2024-02-02T13:24:36.265876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.263960+0000","last_change":"2024-02-02T13:24:36.263960+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.263960+0000","last_undegraded":"2024-02-02T13:24:36.263960+0000","last_fullsized":"2024-02-02T13:24:36.263960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.275151+0000","last_change":"2024-02-02T13:24:36.275151+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.275151+0000","last_undegraded":"2024-02-02T13:24:36.275151+0000","last_fullsized":"2024-02-02T13:24:36.275151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.253334+0000","last_change":"2024-02-02T13:24:35.253334+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.253334+0000","last_undegraded":"2024-02-02T13:24:35.253334+0000","last_fullsized":"2024-02-02T13:24:35.253334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.272775+0000","last_change":"2024-02-02T13:24:36.272775+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272775+0000","last_undegraded":"2024-02-02T13:24:36.272775+0000","last_fullsized":"2024-02-02T13:24:36.272775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.254080+0000","last_change":"2024-02-02T13:24:35.254080+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.254080+0000","last_undegraded":"2024-02-02T13:24:35.254080+0000","last_fullsized":"2024-02-02T13:24:35.254080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.262675+0000","last_change":"2024-02-02T13:24:36.262675+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.262675+0000","last_undegraded":"2024-02-02T13:24:36.262675+0000","last_fullsized":"2024-02-02T13:24:36.262675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.270628+0000","last_change":"2024-02-02T13:24:36.270628+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.270628+0000","last_undegraded":"2024-02-02T13:24:36.270628+0000","last_fullsized":"2024-02-02T13:24:36.270628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.265987+0000","last_change":"2024-02-02T13:24:36.265987+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.265987+0000","last_undegraded":"2024-02-02T13:24:36.265987+0000","last_fullsized":"2024-02-02T13:24:36.265987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260911+0000","last_change":"2024-02-02T13:24:36.260911+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260911+0000","last_undegraded":"2024-02-02T13:24:36.260911+0000","last_fullsized":"2024-02-02T13:24:36.260911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259911+0000","last_change":"2024-02-02T13:24:36.259911+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259911+0000","last_undegraded":"2024-02-02T13:24:36.259911+0000","last_fullsized":"2024-02-02T13:24:36.259911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":6,"reported_epoch":18,"state":"active","last_fresh":"2024-02-02T13:24:36.280572+0000","last_change":"2024-02-02T13:24:36.280495+0000","last_active":"2024-02-02T13:24:36.280572+0000","last_peered":"2024-02-02T13:24:36.280572+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.280572+0000","last_undegraded":"2024-02-02T13:24:36.280572+0000","last_fullsized":"2024-02-02T13:24:36.280572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249392+0000","last_change":"2024-02-02T13:24:35.249392+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249392+0000","last_undegraded":"2024-02-02T13:24:35.249392+0000","last_fullsized":"2024-02-02T13:24:35.249392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.262840+0000","last_change":"2024-02-02T13:24:36.262840+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.262840+0000","last_undegraded":"2024-02-02T13:24:36.262840+0000","last_fullsized":"2024-02-02T13:24:36.262840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.273180+0000","last_change":"2024-02-02T13:24:36.273180+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.273180+0000","last_undegraded":"2024-02-02T13:24:36.273180+0000","last_fullsized":"2024-02-02T13:24:36.273180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.276656+0000","last_change":"2024-02-02T13:24:36.276656+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.276656+0000","last_undegraded":"2024-02-02T13:24:36.276656+0000","last_fullsized":"2024-02-02T13:24:36.276656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261237+0000","last_change":"2024-02-02T13:24:36.261237+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261237+0000","last_undegraded":"2024-02-02T13:24:36.261237+0000","last_fullsized":"2024-02-02T13:24:36.261237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.253101+0000","last_change":"2024-02-02T13:24:35.253101+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.253101+0000","last_undegraded":"2024-02-02T13:24:35.253101+0000","last_fullsized":"2024-02-02T13:24:35.253101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261120+0000","last_change":"2024-02-02T13:24:36.261120+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261120+0000","last_undegraded":"2024-02-02T13:24:36.261120+0000","last_fullsized":"2024-02-02T13:24:36.261120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259621+0000","last_change":"2024-02-02T13:24:36.259621+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259621+0000","last_undegraded":"2024-02-02T13:24:36.259621+0000","last_fullsized":"2024-02-02T13:24:36.259621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264121+0000","last_change":"2024-02-02T13:24:36.264121+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264121+0000","last_undegraded":"2024-02-02T13:24:36.264121+0000","last_fullsized":"2024-02-02T13:24:36.264121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:35.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261119+0000","last_change":"2024-02-02T13:24:36.261119+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261119+0000","last_undegraded":"2024-02-02T13:24:36.261119+0000","last_fullsized":"2024-02-02T13:24:36.261119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-02-02T13:24:31.265521+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:31.265521+0000","last_peered":"2024-02-02T13:24:31.265521+0000","last_clean":"2024-02-02T13:24:31.265521+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:31.265521+0000","last_undegraded":"2024-02-02T13:24:31.265521+0000","last_fullsized":"2024-02-02T13:24:31.265521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.266261+0000","last_change":"2024-02-02T13:24:36.266261+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.266261+0000","last_undegraded":"2024-02-02T13:24:36.266261+0000","last_fullsized":"2024-02-02T13:24:36.266261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247045+0000","last_change":"2024-02-02T13:24:35.247045+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247045+0000","last_undegraded":"2024-02-02T13:24:35.247045+0000","last_fullsized":"2024-02-02T13:24:35.247045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261995+0000","last_change":"2024-02-02T13:24:36.261995+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261995+0000","last_undegraded":"2024-02-02T13:24:36.261995+0000","last_fullsized":"2024-02-02T13:24:36.261995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.276771+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.276771+0000","last_peered":"2024-02-02T13:24:36.276771+0000","last_clean":"2024-02-02T13:24:36.276771+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.276771+0000","last_undegraded":"2024-02-02T13:24:36.276771+0000","last_fullsized":"2024-02-02T13:24:36.276771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261275+0000","last_change":"2024-02-02T13:24:36.261275+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261275+0000","last_undegraded":"2024-02-02T13:24:36.261275+0000","last_fullsized":"2024-02-02T13:24:36.261275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.263352+0000","last_change":"2024-02-02T13:24:36.263352+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.263352+0000","last_undegraded":"2024-02-02T13:24:36.263352+0000","last_fullsized":"2024-02-02T13:24:36.263352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.248596+0000","last_change":"2024-02-02T13:24:35.248596+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.248596+0000","last_undegraded":"2024-02-02T13:24:35.248596+0000","last_fullsized":"2024-02-02T13:24:35.248596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260904+0000","last_change":"2024-02-02T13:24:36.260904+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260904+0000","last_undegraded":"2024-02-02T13:24:36.260904+0000","last_fullsized":"2024-02-02T13:24:36.260904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.269391+0000","last_change":"2024-02-02T13:24:36.269391+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.269391+0000","last_undegraded":"2024-02-02T13:24:36.269391+0000","last_fullsized":"2024-02-02T13:24:36.269391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247176+0000","last_change":"2024-02-02T13:24:35.247176+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247176+0000","last_undegraded":"2024-02-02T13:24:35.247176+0000","last_fullsized":"2024-02-02T13:24:35.247176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260560+0000","last_change":"2024-02-02T13:24:36.260560+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260560+0000","last_undegraded":"2024-02-02T13:24:36.260560+0000","last_fullsized":"2024-02-02T13:24:36.260560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259642+0000","last_change":"2024-02-02T13:24:36.259642+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259642+0000","last_undegraded":"2024-02-02T13:24:36.259642+0000","last_fullsized":"2024-02-02T13:24:36.259642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247410+0000","last_change":"2024-02-02T13:24:35.247410+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247410+0000","last_undegraded":"2024-02-02T13:24:35.247410+0000","last_fullsized":"2024-02-02T13:24:35.247410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.268167+0000","last_change":"2024-02-02T13:24:36.268167+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.268167+0000","last_undegraded":"2024-02-02T13:24:36.268167+0000","last_fullsized":"2024-02-02T13:24:36.268167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264984+0000","last_change":"2024-02-02T13:24:36.264984+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264984+0000","last_undegraded":"2024-02-02T13:24:36.264984+0000","last_fullsized":"2024-02-02T13:24:36.264984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:44.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249577+0000","last_change":"2024-02-02T13:24:35.249577+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249577+0000","last_undegraded":"2024-02-02T13:24:35.249577+0000","last_fullsized":"2024-02-02T13:24:35.249577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.270088+0000","last_change":"2024-02-02T13:24:36.270088+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.270088+0000","last_undegraded":"2024-02-02T13:24:36.270088+0000","last_fullsized":"2024-02-02T13:24:36.270088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247176+0000","last_change":"2024-02-02T13:24:35.247176+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247176+0000","last_undegraded":"2024-02-02T13:24:35.247176+0000","last_fullsized":"2024-02-02T13:24:35.247176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.254704+0000","last_change":"2024-02-02T13:24:35.254704+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.254704+0000","last_undegraded":"2024-02-02T13:24:35.254704+0000","last_fullsized":"2024-02-02T13:24:35.254704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249587+0000","last_change":"2024-02-02T13:24:35.249587+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249587+0000","last_undegraded":"2024-02-02T13:24:35.249587+0000","last_fullsized":"2024-02-02T13:24:35.249587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359263,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:41.609 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:41.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.855+0000 7fb6da200640 1 Processor -- start 2024-02-02T13:24:41.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.855+0000 7fb6da200640 1 -- start start 2024-02-02T13:24:41.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.856+0000 7fb6da200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6d40addd0 0x7fb6d40cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.856+0000 7fb6da200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 0x7fb6d40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6da200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d40ae1b0 con 0x7fb6d40cd0e0 2024-02-02T13:24:41.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6da200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d406b450 con 0x7fb6d40addd0 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6da200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d4161b30 con 0x7fb6d40c1c70 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6d40addd0 0x7fb6d40cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6d40addd0 0x7fb6d40cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53114/0 (socket says 172.21.15.91:53114) 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 -- 172.21.15.91:0/3298328925 learned_addr learned my addr 172.21.15.91:0/3298328925 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d9800640 1 --1- 172.21.15.91:0/3298328925 >> v1:172.21.15.91:6789/0 conn(0x7fb6d40cd0e0 0x7fb6d41651b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34038/0 (socket says 172.21.15.91:34038) 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6cbc00640 1 --2- 172.21.15.91:0/3298328925 >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 0x7fb6d40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 -- 172.21.15.91:0/3298328925 >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 msgr2=0x7fb6d40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 --2- 172.21.15.91:0/3298328925 >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 0x7fb6d40ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 -- 172.21.15.91:0/3298328925 >> v1:172.21.15.91:6789/0 conn(0x7fb6d40cd0e0 legacy=0x7fb6d41651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6d8e00640 1 -- 172.21.15.91:0/3298328925 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40c9430 con 0x7fb6d40addd0 2024-02-02T13:24:41.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.857+0000 7fb6cbc00640 1 --2- 172.21.15.91:0/3298328925 >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 0x7fb6d40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:41.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.858+0000 7fb6d8e00640 1 --2- 172.21.15.91:0/3298328925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40addd0 0x7fb6d40cc660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406ad90 tx=0x7fb6c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6ab68eb04767ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6d3e00640 1 -- 172.21.15.91:0/3298328925 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4030860 con 0x7fb6d40addd0 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6d3e00640 1 -- 172.21.15.91:0/3298328925 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c40309c0 con 0x7fb6d40addd0 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6d3e00640 1 -- 172.21.15.91:0/3298328925 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4030d50 con 0x7fb6d40addd0 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 -- 172.21.15.91:0/3298328925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40addd0 msgr2=0x7fb6d40cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 --2- 172.21.15.91:0/3298328925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40addd0 0x7fb6d40cc660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406ad90 tx=0x7fb6c402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 -- 172.21.15.91:0/3298328925 shutdown_connections 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 --2- 172.21.15.91:0/3298328925 >> v2:172.21.15.113:3300/0 conn(0x7fb6d40c1c70 0x7fb6d40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 --2- 172.21.15.91:0/3298328925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40addd0 0x7fb6d40cc660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 -- 172.21.15.91:0/3298328925 >> 172.21.15.91:0/3298328925 conn(0x7fb6d40cea90 msgr2=0x7fb6d40ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 -- 172.21.15.91:0/3298328925 shutdown_connections 2024-02-02T13:24:41.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.859+0000 7fb6da200640 1 -- 172.21.15.91:0/3298328925 wait complete. 2024-02-02T13:24:41.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.861+0000 7fb6da200640 1 Processor -- start 2024-02-02T13:24:41.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.861+0000 7fb6da200640 1 -- start start 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6d40addd0 0x7fb6d40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6d40cd0e0 0x7fb6d415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d41f6b50 con 0x7fb6d40c1c70 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40c6bb0 con 0x7fb6d40addd0 2024-02-02T13:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.862+0000 7fb6da200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40b2980 con 0x7fb6d40cd0e0 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53116/0 (socket says 172.21.15.91:53116) 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 -- 172.21.15.91:0/1221084845 learned_addr learned my addr 172.21.15.91:0/1221084845 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6d9800640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6d40cd0e0 0x7fb6d415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6d8e00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6d40addd0 0x7fb6d40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 -- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6d40cd0e0 msgr2=0x7fb6d415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6d40cd0e0 0x7fb6d415e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 -- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6d40addd0 msgr2=0x7fb6d40b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6d40addd0 0x7fb6d40b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 -- 172.21.15.91:0/1221084845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40094d0 con 0x7fb6d40c1c70 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6cbc00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c7f0 tx=0x7fb6bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc007a30 con 0x7fb6d40c1c70 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.863+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc007b90 con 0x7fb6d40c1c70 2024-02-02T13:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.864+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc00ce40 con 0x7fb6d40c1c70 2024-02-02T13:24:41.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.864+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d40b2c60 con 0x7fb6d40c1c70 2024-02-02T13:24:41.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.864+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d40c6e00 con 0x7fb6d40c1c70 2024-02-02T13:24:41.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.865+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d40c7020 con 0x7fb6d40c1c70 2024-02-02T13:24:41.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.866+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6bc0113f0 con 0x7fb6d40c1c70 2024-02-02T13:24:41.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.867+0000 7fb6d1600640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 0x7fb6ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:41.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.867+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc088ae0 con 0x7fb6d40c1c70 2024-02-02T13:24:41.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.867+0000 7fb6d8e00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 0x7fb6ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:41.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.867+0000 7fb6d8e00640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 0x7fb6ac06c8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4036000 tx=0x7fb6c403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:41.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:41.871+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb6bc0540d0 con 0x7fb6d40c1c70 2024-02-02T13:24:42.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.056+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6d40cb0c0 con 0x7fb6ac06a430 2024-02-02T13:24:42.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.077+0000 7fb6d1600640 1 -- 172.21.15.91:0/1221084845 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252966 (secure 0 0 0) 0x7fb6c4030c00 con 0x7fb6ac06a430 2024-02-02T13:24:42.082 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:42.083 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 msgr2=0x7fb6ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 0x7fb6ac06c8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4036000 tx=0x7fb6c403b040 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 msgr2=0x7fb6d40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c7f0 tx=0x7fb6bc00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 shutdown_connections 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6ac06a430 0x7fb6ac06c8f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6d40cd0e0 0x7fb6d415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6d40c1c70 0x7fb6d40b2290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 --2- 172.21.15.91:0/1221084845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6d40addd0 0x7fb6d40b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 >> 172.21.15.91:0/1221084845 conn(0x7fb6d40cea90 msgr2=0x7fb6d4160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 shutdown_connections 2024-02-02T13:24:42.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.082+0000 7fb6da200640 1 -- 172.21.15.91:0/1221084845 wait complete. 2024-02-02T13:24:42.103 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-02-02T13:24:40.236049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560949,"statfs":{"total":1600000000,"available":1598412236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992281"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267085+0000","last_change":"2024-02-02T13:24:36.267085+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267085+0000","last_undegraded":"2024-02-02T13:24:36.267085+0000","last_fullsized":"2024-02-02T13:24:36.267085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272655+0000","last_change":"2024-02-02T13:24:36.272655+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272655+0000","last_undegraded":"2024-02-02T13:24:36.272655+0000","last_fullsized":"2024-02-02T13:24:36.272655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.275029+0000","last_change":"2024-02-02T13:24:36.275029+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.275029+0000","last_undegraded":"2024-02-02T13:24:36.275029+0000","last_fullsized":"2024-02-02T13:24:36.275029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:59.975618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272328+0000","last_change":"2024-02-02T13:24:36.272328+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272328+0000","last_undegraded":"2024-02-02T13:24:36.272328+0000","last_fullsized":"2024-02-02T13:24:36.272328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:35.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.271716+0000","last_change":"2024-02-02T13:24:36.271716+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.271716+0000","last_undegraded":"2024-02-02T13:24:36.271716+0000","last_fullsized":"2024-02-02T13:24:36.271716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267852+0000","last_change":"2024-02-02T13:24:36.267852+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267852+0000","last_undegraded":"2024-02-02T13:24:36.267852+0000","last_fullsized":"2024-02-02T13:24:36.267852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264041+0000","last_change":"2024-02-02T13:24:36.264041+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264041+0000","last_undegraded":"2024-02-02T13:24:36.264041+0000","last_fullsized":"2024-02-02T13:24:36.264041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.272976+0000","last_change":"2024-02-02T13:24:36.272976+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272976+0000","last_undegraded":"2024-02-02T13:24:36.272976+0000","last_fullsized":"2024-02-02T13:24:36.272976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:44.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.267011+0000","last_change":"2024-02-02T13:24:36.267011+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.267011+0000","last_undegraded":"2024-02-02T13:24:36.267011+0000","last_fullsized":"2024-02-02T13:24:36.267011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.266074+0000","last_change":"2024-02-02T13:24:36.266074+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.266074+0000","last_undegraded":"2024-02-02T13:24:36.266074+0000","last_fullsized":"2024-02-02T13:24:36.266074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:59.975618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.271537+0000","last_change":"2024-02-02T13:24:36.271537+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.271537+0000","last_undegraded":"2024-02-02T13:24:36.271537+0000","last_fullsized":"2024-02-02T13:24:36.271537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.265876+0000","last_change":"2024-02-02T13:24:36.265876+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.265876+0000","last_undegraded":"2024-02-02T13:24:36.265876+0000","last_fullsized":"2024-02-02T13:24:36.265876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.263960+0000","last_change":"2024-02-02T13:24:36.263960+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.263960+0000","last_undegraded":"2024-02-02T13:24:36.263960+0000","last_fullsized":"2024-02-02T13:24:36.263960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.275151+0000","last_change":"2024-02-02T13:24:36.275151+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.275151+0000","last_undegraded":"2024-02-02T13:24:36.275151+0000","last_fullsized":"2024-02-02T13:24:36.275151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.253334+0000","last_change":"2024-02-02T13:24:35.253334+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.253334+0000","last_undegraded":"2024-02-02T13:24:35.253334+0000","last_fullsized":"2024-02-02T13:24:35.253334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.272775+0000","last_change":"2024-02-02T13:24:36.272775+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.272775+0000","last_undegraded":"2024-02-02T13:24:36.272775+0000","last_fullsized":"2024-02-02T13:24:36.272775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.254080+0000","last_change":"2024-02-02T13:24:35.254080+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.254080+0000","last_undegraded":"2024-02-02T13:24:35.254080+0000","last_fullsized":"2024-02-02T13:24:35.254080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.262675+0000","last_change":"2024-02-02T13:24:36.262675+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.262675+0000","last_undegraded":"2024-02-02T13:24:36.262675+0000","last_fullsized":"2024-02-02T13:24:36.262675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.270628+0000","last_change":"2024-02-02T13:24:36.270628+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.270628+0000","last_undegraded":"2024-02-02T13:24:36.270628+0000","last_fullsized":"2024-02-02T13:24:36.270628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.265987+0000","last_change":"2024-02-02T13:24:36.265987+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.265987+0000","last_undegraded":"2024-02-02T13:24:36.265987+0000","last_fullsized":"2024-02-02T13:24:36.265987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260911+0000","last_change":"2024-02-02T13:24:36.260911+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260911+0000","last_undegraded":"2024-02-02T13:24:36.260911+0000","last_fullsized":"2024-02-02T13:24:36.260911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259911+0000","last_change":"2024-02-02T13:24:36.259911+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259911+0000","last_undegraded":"2024-02-02T13:24:36.259911+0000","last_fullsized":"2024-02-02T13:24:36.259911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":6,"reported_epoch":18,"state":"active","last_fresh":"2024-02-02T13:24:36.280572+0000","last_change":"2024-02-02T13:24:36.280495+0000","last_active":"2024-02-02T13:24:36.280572+0000","last_peered":"2024-02-02T13:24:36.280572+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.280572+0000","last_undegraded":"2024-02-02T13:24:36.280572+0000","last_fullsized":"2024-02-02T13:24:36.280572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249392+0000","last_change":"2024-02-02T13:24:35.249392+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249392+0000","last_undegraded":"2024-02-02T13:24:35.249392+0000","last_fullsized":"2024-02-02T13:24:35.249392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.262840+0000","last_change":"2024-02-02T13:24:36.262840+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.262840+0000","last_undegraded":"2024-02-02T13:24:36.262840+0000","last_fullsized":"2024-02-02T13:24:36.262840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.273180+0000","last_change":"2024-02-02T13:24:36.273180+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.273180+0000","last_undegraded":"2024-02-02T13:24:36.273180+0000","last_fullsized":"2024-02-02T13:24:36.273180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.276656+0000","last_change":"2024-02-02T13:24:36.276656+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.276656+0000","last_undegraded":"2024-02-02T13:24:36.276656+0000","last_fullsized":"2024-02-02T13:24:36.276656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261237+0000","last_change":"2024-02-02T13:24:36.261237+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261237+0000","last_undegraded":"2024-02-02T13:24:36.261237+0000","last_fullsized":"2024-02-02T13:24:36.261237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.253101+0000","last_change":"2024-02-02T13:24:35.253101+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.253101+0000","last_undegraded":"2024-02-02T13:24:35.253101+0000","last_fullsized":"2024-02-02T13:24:35.253101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261120+0000","last_change":"2024-02-02T13:24:36.261120+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261120+0000","last_undegraded":"2024-02-02T13:24:36.261120+0000","last_fullsized":"2024-02-02T13:24:36.261120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259621+0000","last_change":"2024-02-02T13:24:36.259621+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259621+0000","last_undegraded":"2024-02-02T13:24:36.259621+0000","last_fullsized":"2024-02-02T13:24:36.259621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264121+0000","last_change":"2024-02-02T13:24:36.264121+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264121+0000","last_undegraded":"2024-02-02T13:24:36.264121+0000","last_fullsized":"2024-02-02T13:24:36.264121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:35.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261119+0000","last_change":"2024-02-02T13:24:36.261119+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261119+0000","last_undegraded":"2024-02-02T13:24:36.261119+0000","last_fullsized":"2024-02-02T13:24:36.261119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-02-02T13:24:31.265521+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:31.265521+0000","last_peered":"2024-02-02T13:24:31.265521+0000","last_clean":"2024-02-02T13:24:31.265521+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:31.265521+0000","last_undegraded":"2024-02-02T13:24:31.265521+0000","last_fullsized":"2024-02-02T13:24:31.265521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.266261+0000","last_change":"2024-02-02T13:24:36.266261+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.266261+0000","last_undegraded":"2024-02-02T13:24:36.266261+0000","last_fullsized":"2024-02-02T13:24:36.266261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247045+0000","last_change":"2024-02-02T13:24:35.247045+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247045+0000","last_undegraded":"2024-02-02T13:24:35.247045+0000","last_fullsized":"2024-02-02T13:24:35.247045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261995+0000","last_change":"2024-02-02T13:24:36.261995+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261995+0000","last_undegraded":"2024-02-02T13:24:36.261995+0000","last_fullsized":"2024-02-02T13:24:36.261995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.276771+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.276771+0000","last_peered":"2024-02-02T13:24:36.276771+0000","last_clean":"2024-02-02T13:24:36.276771+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.276771+0000","last_undegraded":"2024-02-02T13:24:36.276771+0000","last_fullsized":"2024-02-02T13:24:36.276771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.261275+0000","last_change":"2024-02-02T13:24:36.261275+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.261275+0000","last_undegraded":"2024-02-02T13:24:36.261275+0000","last_fullsized":"2024-02-02T13:24:36.261275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.263352+0000","last_change":"2024-02-02T13:24:36.263352+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.263352+0000","last_undegraded":"2024-02-02T13:24:36.263352+0000","last_fullsized":"2024-02-02T13:24:36.263352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.248596+0000","last_change":"2024-02-02T13:24:35.248596+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.248596+0000","last_undegraded":"2024-02-02T13:24:35.248596+0000","last_fullsized":"2024-02-02T13:24:35.248596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260904+0000","last_change":"2024-02-02T13:24:36.260904+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260904+0000","last_undegraded":"2024-02-02T13:24:36.260904+0000","last_fullsized":"2024-02-02T13:24:36.260904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.269391+0000","last_change":"2024-02-02T13:24:36.269391+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.269391+0000","last_undegraded":"2024-02-02T13:24:36.269391+0000","last_fullsized":"2024-02-02T13:24:36.269391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247176+0000","last_change":"2024-02-02T13:24:35.247176+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247176+0000","last_undegraded":"2024-02-02T13:24:35.247176+0000","last_fullsized":"2024-02-02T13:24:35.247176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.260560+0000","last_change":"2024-02-02T13:24:36.260560+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.260560+0000","last_undegraded":"2024-02-02T13:24:36.260560+0000","last_fullsized":"2024-02-02T13:24:36.260560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":3,"reported_epoch":18,"state":"creating+activating","last_fresh":"2024-02-02T13:24:36.259642+0000","last_change":"2024-02-02T13:24:36.259642+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.259642+0000","last_undegraded":"2024-02-02T13:24:36.259642+0000","last_fullsized":"2024-02-02T13:24:36.259642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247410+0000","last_change":"2024-02-02T13:24:35.247410+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247410+0000","last_undegraded":"2024-02-02T13:24:35.247410+0000","last_fullsized":"2024-02-02T13:24:35.247410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.268167+0000","last_change":"2024-02-02T13:24:36.268167+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.268167+0000","last_undegraded":"2024-02-02T13:24:36.268167+0000","last_fullsized":"2024-02-02T13:24:36.268167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.264984+0000","last_change":"2024-02-02T13:24:36.264984+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.264984+0000","last_undegraded":"2024-02-02T13:24:36.264984+0000","last_fullsized":"2024-02-02T13:24:36.264984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:44.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249577+0000","last_change":"2024-02-02T13:24:35.249577+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249577+0000","last_undegraded":"2024-02-02T13:24:35.249577+0000","last_fullsized":"2024-02-02T13:24:35.249577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-02T13:24:36.270088+0000","last_change":"2024-02-02T13:24:36.270088+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.270088+0000","last_undegraded":"2024-02-02T13:24:36.270088+0000","last_fullsized":"2024-02-02T13:24:36.270088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.247176+0000","last_change":"2024-02-02T13:24:35.247176+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.247176+0000","last_undegraded":"2024-02-02T13:24:35.247176+0000","last_fullsized":"2024-02-02T13:24:35.247176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.254704+0000","last_change":"2024-02-02T13:24:35.254704+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.254704+0000","last_undegraded":"2024-02-02T13:24:35.254704+0000","last_fullsized":"2024-02-02T13:24:35.254704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:24:36.243291+0000","last_change":"2024-02-02T13:24:36.243291+0000","last_active":"2024-02-02T13:24:36.243291+0000","last_peered":"2024-02-02T13:24:36.243291+0000","last_clean":"2024-02-02T13:24:36.243291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:36.243291+0000","last_undegraded":"2024-02-02T13:24:36.243291+0000","last_fullsized":"2024-02-02T13:24:36.243291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-02-02T13:24:35.249587+0000","last_change":"2024-02-02T13:24:35.249587+0000","last_active":"2024-02-02T13:24:35.233228+0000","last_peered":"2024-02-02T13:24:35.233228+0000","last_clean":"2024-02-02T13:24:35.233228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:24:35.249587+0000","last_undegraded":"2024-02-02T13:24:35.249587+0000","last_fullsized":"2024-02-02T13:24:35.249587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359263,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:42.107 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:42.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.353+0000 7fa62aa00640 1 Processor -- start 2024-02-02T13:24:42.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.353+0000 7fa62aa00640 1 -- start start 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.354+0000 7fa62aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6240b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.354+0000 7fa62aa00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa624070f80 con 0x7fa62415f5a0 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.354+0000 7fa62aa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62406c600 con 0x7fa624151f90 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.354+0000 7fa62aa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62415fa30 con 0x7fa6240b0e10 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa623400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa62415f5a0 0x7fa6240b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34050/0 (socket says 172.21.15.91:34050) 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa623400640 1 -- 172.21.15.91:0/3288496191 learned_addr learned my addr 172.21.15.91:0/3288496191 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:42.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa628a00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6240b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267698076 0 0) 0x7fa624070f80 con 0x7fa62415f5a0 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa604003520 con 0x7fa62415f5a0 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 984766930 0 0) 0x7fa604003520 con 0x7fa62415f5a0 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 msgr2=0x7fa6240b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6240b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 >> v1:172.21.15.113:6789/0 conn(0x7fa624151f90 legacy=0x7fa62406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa624152370 con 0x7fa62415f5a0 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa60c001670 con 0x7fa62415f5a0 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 >> v1:172.21.15.91:6789/0 conn(0x7fa62415f5a0 legacy=0x7fa6240b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6040039a0 0x7fa604003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.355+0000 7fa622a00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa622a00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa604004d00 0x7fa604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62415fa30 con 0x7fa6040039a0 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60400ba90 con 0x7fa604004d00 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60400bc50 con 0x7fa6240b0e10 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6040039a0 0x7fa604003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 -- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 msgr2=0x7fa6040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623400640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6240b0e10 0x7fa6040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 -- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa604004d00 msgr2=0x7fa604005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:42.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa604004d00 0x7fa604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.356+0000 7fa623e00640 1 -- 172.21.15.91:0/3288496191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604003520 con 0x7fa6040039a0 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.357+0000 7fa623e00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6040039a0 0x7fa604003dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa614001720 tx=0x7fa6040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b086791656d84644 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.357+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140052d0 con 0x7fa6040039a0 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.357+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61402e050 con 0x7fa6040039a0 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.357+0000 7fa622a00640 1 -- 172.21.15.91:0/3288496191 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140035f0 con 0x7fa6040039a0 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 -- 172.21.15.91:0/3288496191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6040039a0 msgr2=0x7fa604003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 --2- 172.21.15.91:0/3288496191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6040039a0 0x7fa604003dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa614001720 tx=0x7fa6040057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa628a00640 1 -- 172.21.15.91:0/3288496191 reap_dead start 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 -- 172.21.15.91:0/3288496191 shutdown_connections 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 -- 172.21.15.91:0/3288496191 >> 172.21.15.91:0/3288496191 conn(0x7fa6240cec80 msgr2=0x7fa6240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 -- 172.21.15.91:0/3288496191 shutdown_connections 2024-02-02T13:24:42.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.358+0000 7fa62aa00640 1 -- 172.21.15.91:0/3288496191 wait complete. 2024-02-02T13:24:42.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.360+0000 7fa62aa00640 1 Processor -- start 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.360+0000 7fa62aa00640 1 -- start start 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6040039a0 0x7fa62414fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6240cc7c0 0x7fa624158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240cd970 con 0x7fa6240cde00 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62414fff0 con 0x7fa6240cc7c0 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa62aa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240ccd60 con 0x7fa6040039a0 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa628a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa628a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53132/0 (socket says 172.21.15.91:53132) 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.361+0000 7fa628a00640 1 -- 172.21.15.91:0/689792092 learned_addr learned my addr 172.21.15.91:0/689792092 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa623400640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6240cc7c0 0x7fa624158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa623e00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6040039a0 0x7fa62414fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 -- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6040039a0 msgr2=0x7fa62414fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6040039a0 0x7fa62414fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 -- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6240cc7c0 msgr2=0x7fa624158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6240cc7c0 0x7fa624158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 -- 172.21.15.91:0/689792092 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240cd040 con 0x7fa6240cde00 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa628a00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6240cec30 tx=0x7fa61c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c010040 con 0x7fa6240cde00 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c00bb70 con 0x7fa6240cde00 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6240cd320 con 0x7fa6240cde00 2024-02-02T13:24:42.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c014400 con 0x7fa6240cde00 2024-02-02T13:24:42.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.362+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa624158990 con 0x7fa6240cde00 2024-02-02T13:24:42.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.364+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6240cdae0 con 0x7fa6240cde00 2024-02-02T13:24:42.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.364+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa61c014560 con 0x7fa6240cde00 2024-02-02T13:24:42.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.365+0000 7fa61be00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 0x7fa5fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.365+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa61c088b60 con 0x7fa6240cde00 2024-02-02T13:24:42.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.366+0000 7fa623e00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 0x7fa5fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.366+0000 7fa623e00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 0x7fa5fc06c8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa614033000 tx=0x7fa614009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.370+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa61c054150 con 0x7fa6240cde00 2024-02-02T13:24:42.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.558+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6240cb2f0 con 0x7fa5fc06a430 2024-02-02T13:24:42.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.580+0000 7fa61be00640 1 -- 172.21.15.91:0/689792092 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260054 (secure 0 0 0) 0x7fa614005390 con 0x7fa5fc06a430 2024-02-02T13:24:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:42.587 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 msgr2=0x7fa5fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 0x7fa5fc06c8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa614033000 tx=0x7fa614009030 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 msgr2=0x7fa624150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6240cec30 tx=0x7fa61c00ef30 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 shutdown_connections 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5fc06a430 0x7fa5fc06c8f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6240cde00 0x7fa624150240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6240cc7c0 0x7fa624158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 --2- 172.21.15.91:0/689792092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6040039a0 0x7fa62414fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 >> 172.21.15.91:0/689792092 conn(0x7fa6240cec80 msgr2=0x7fa6240c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 shutdown_connections 2024-02-02T13:24:42.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.586+0000 7fa62aa00640 1 -- 172.21.15.91:0/689792092 wait complete. 2024-02-02T13:24:42.600 INFO:tasks.ceph.mds.d.smithi113.stderr:2024-02-02T13:24:42.598+0000 7f0378c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:24:42.607 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-02-02T13:24:42.236833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560812,"statfs":{"total":1600000000,"available":1598271612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002030"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978392+0000","last_change":"2024-02-02T13:24:36.767566+0000","last_active":"2024-02-02T13:24:40.978392+0000","last_peered":"2024-02-02T13:24:40.978392+0000","last_clean":"2024-02-02T13:24:40.978392+0000","last_became_active":"2024-02-02T13:24:36.767342+0000","last_became_peered":"2024-02-02T13:24:36.767342+0000","last_unstale":"2024-02-02T13:24:40.978392+0000","last_undegraded":"2024-02-02T13:24:40.978392+0000","last_fullsized":"2024-02-02T13:24:40.978392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876609+0000","last_change":"2024-02-02T13:24:36.753974+0000","last_active":"2024-02-02T13:24:39.876609+0000","last_peered":"2024-02-02T13:24:39.876609+0000","last_clean":"2024-02-02T13:24:39.876609+0000","last_became_active":"2024-02-02T13:24:36.753603+0000","last_became_peered":"2024-02-02T13:24:36.753603+0000","last_unstale":"2024-02-02T13:24:39.876609+0000","last_undegraded":"2024-02-02T13:24:39.876609+0000","last_fullsized":"2024-02-02T13:24:39.876609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092881+0000","last_change":"2024-02-02T13:24:36.761233+0000","last_active":"2024-02-02T13:24:41.092881+0000","last_peered":"2024-02-02T13:24:41.092881+0000","last_clean":"2024-02-02T13:24:41.092881+0000","last_became_active":"2024-02-02T13:24:36.761020+0000","last_became_peered":"2024-02-02T13:24:36.761020+0000","last_unstale":"2024-02-02T13:24:41.092881+0000","last_undegraded":"2024-02-02T13:24:41.092881+0000","last_fullsized":"2024-02-02T13:24:41.092881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773368+0000","last_change":"2024-02-02T13:24:36.772920+0000","last_active":"2024-02-02T13:24:36.773368+0000","last_peered":"2024-02-02T13:24:36.773368+0000","last_clean":"2024-02-02T13:24:36.773368+0000","last_became_active":"2024-02-02T13:24:36.765145+0000","last_became_peered":"2024-02-02T13:24:36.765145+0000","last_unstale":"2024-02-02T13:24:36.773368+0000","last_undegraded":"2024-02-02T13:24:36.773368+0000","last_fullsized":"2024-02-02T13:24:36.773368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978323+0000","last_change":"2024-02-02T13:24:36.759645+0000","last_active":"2024-02-02T13:24:40.978323+0000","last_peered":"2024-02-02T13:24:40.978323+0000","last_clean":"2024-02-02T13:24:40.978323+0000","last_became_active":"2024-02-02T13:24:36.759399+0000","last_became_peered":"2024-02-02T13:24:36.759399+0000","last_unstale":"2024-02-02T13:24:40.978323+0000","last_undegraded":"2024-02-02T13:24:40.978323+0000","last_fullsized":"2024-02-02T13:24:40.978323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:52.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777056+0000","last_change":"2024-02-02T13:24:36.774617+0000","last_active":"2024-02-02T13:24:36.777056+0000","last_peered":"2024-02-02T13:24:36.777056+0000","last_clean":"2024-02-02T13:24:36.777056+0000","last_became_active":"2024-02-02T13:24:36.772658+0000","last_became_peered":"2024-02-02T13:24:36.772658+0000","last_unstale":"2024-02-02T13:24:36.777056+0000","last_undegraded":"2024-02-02T13:24:36.777056+0000","last_fullsized":"2024-02-02T13:24:36.777056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978756+0000","last_change":"2024-02-02T13:24:36.755776+0000","last_active":"2024-02-02T13:24:40.978756+0000","last_peered":"2024-02-02T13:24:40.978756+0000","last_clean":"2024-02-02T13:24:40.978756+0000","last_became_active":"2024-02-02T13:24:36.755523+0000","last_became_peered":"2024-02-02T13:24:36.755523+0000","last_unstale":"2024-02-02T13:24:40.978756+0000","last_undegraded":"2024-02-02T13:24:40.978756+0000","last_fullsized":"2024-02-02T13:24:40.978756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211535+0000","last_change":"2024-02-02T13:24:36.759777+0000","last_active":"2024-02-02T13:24:41.211535+0000","last_peered":"2024-02-02T13:24:41.211535+0000","last_clean":"2024-02-02T13:24:41.211535+0000","last_became_active":"2024-02-02T13:24:36.759170+0000","last_became_peered":"2024-02-02T13:24:36.759170+0000","last_unstale":"2024-02-02T13:24:41.211535+0000","last_undegraded":"2024-02-02T13:24:41.211535+0000","last_fullsized":"2024-02-02T13:24:41.211535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978056+0000","last_change":"2024-02-02T13:24:36.757480+0000","last_active":"2024-02-02T13:24:40.978056+0000","last_peered":"2024-02-02T13:24:40.978056+0000","last_clean":"2024-02-02T13:24:40.978056+0000","last_became_active":"2024-02-02T13:24:36.757254+0000","last_became_peered":"2024-02-02T13:24:36.757254+0000","last_unstale":"2024-02-02T13:24:40.978056+0000","last_undegraded":"2024-02-02T13:24:40.978056+0000","last_fullsized":"2024-02-02T13:24:40.978056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:41.311042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775838+0000","last_change":"2024-02-02T13:24:36.775765+0000","last_active":"2024-02-02T13:24:36.775838+0000","last_peered":"2024-02-02T13:24:36.775838+0000","last_clean":"2024-02-02T13:24:36.775838+0000","last_became_active":"2024-02-02T13:24:36.775523+0000","last_became_peered":"2024-02-02T13:24:36.775523+0000","last_unstale":"2024-02-02T13:24:36.775838+0000","last_undegraded":"2024-02-02T13:24:36.775838+0000","last_fullsized":"2024-02-02T13:24:36.775838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778319+0000","last_change":"2024-02-02T13:24:36.778240+0000","last_active":"2024-02-02T13:24:36.778319+0000","last_peered":"2024-02-02T13:24:36.778319+0000","last_clean":"2024-02-02T13:24:36.778319+0000","last_became_active":"2024-02-02T13:24:36.778019+0000","last_became_peered":"2024-02-02T13:24:36.778019+0000","last_unstale":"2024-02-02T13:24:36.778319+0000","last_undegraded":"2024-02-02T13:24:36.778319+0000","last_fullsized":"2024-02-02T13:24:36.778319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753463+0000","last_change":"2024-02-02T13:24:36.766203+0000","last_active":"2024-02-02T13:24:37.753463+0000","last_peered":"2024-02-02T13:24:37.753463+0000","last_clean":"2024-02-02T13:24:37.753463+0000","last_became_active":"2024-02-02T13:24:36.765708+0000","last_became_peered":"2024-02-02T13:24:36.765708+0000","last_unstale":"2024-02-02T13:24:37.753463+0000","last_undegraded":"2024-02-02T13:24:37.753463+0000","last_fullsized":"2024-02-02T13:24:37.753463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211785+0000","last_change":"2024-02-02T13:24:36.768024+0000","last_active":"2024-02-02T13:24:41.211785+0000","last_peered":"2024-02-02T13:24:41.211785+0000","last_clean":"2024-02-02T13:24:41.211785+0000","last_became_active":"2024-02-02T13:24:36.767819+0000","last_became_peered":"2024-02-02T13:24:36.767819+0000","last_unstale":"2024-02-02T13:24:41.211785+0000","last_undegraded":"2024-02-02T13:24:41.211785+0000","last_fullsized":"2024-02-02T13:24:41.211785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093008+0000","last_change":"2024-02-02T13:24:36.755455+0000","last_active":"2024-02-02T13:24:41.093008+0000","last_peered":"2024-02-02T13:24:41.093008+0000","last_clean":"2024-02-02T13:24:41.093008+0000","last_became_active":"2024-02-02T13:24:36.755245+0000","last_became_peered":"2024-02-02T13:24:36.755245+0000","last_unstale":"2024-02-02T13:24:41.093008+0000","last_undegraded":"2024-02-02T13:24:41.093008+0000","last_fullsized":"2024-02-02T13:24:41.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876981+0000","last_change":"2024-02-02T13:24:36.757288+0000","last_active":"2024-02-02T13:24:39.876981+0000","last_peered":"2024-02-02T13:24:39.876981+0000","last_clean":"2024-02-02T13:24:39.876981+0000","last_became_active":"2024-02-02T13:24:36.756687+0000","last_became_peered":"2024-02-02T13:24:36.756687+0000","last_unstale":"2024-02-02T13:24:39.876981+0000","last_undegraded":"2024-02-02T13:24:39.876981+0000","last_fullsized":"2024-02-02T13:24:39.876981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777000+0000","last_change":"2024-02-02T13:24:36.774416+0000","last_active":"2024-02-02T13:24:36.777000+0000","last_peered":"2024-02-02T13:24:36.777000+0000","last_clean":"2024-02-02T13:24:36.777000+0000","last_became_active":"2024-02-02T13:24:36.771102+0000","last_became_peered":"2024-02-02T13:24:36.771102+0000","last_unstale":"2024-02-02T13:24:36.777000+0000","last_undegraded":"2024-02-02T13:24:36.777000+0000","last_fullsized":"2024-02-02T13:24:36.777000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761526+0000","last_change":"2024-02-02T13:24:36.761450+0000","last_active":"2024-02-02T13:24:36.761526+0000","last_peered":"2024-02-02T13:24:36.761526+0000","last_clean":"2024-02-02T13:24:36.761526+0000","last_became_active":"2024-02-02T13:24:36.760875+0000","last_became_peered":"2024-02-02T13:24:36.760875+0000","last_unstale":"2024-02-02T13:24:36.761526+0000","last_undegraded":"2024-02-02T13:24:36.761526+0000","last_fullsized":"2024-02-02T13:24:36.761526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771249+0000","last_change":"2024-02-02T13:24:36.768404+0000","last_active":"2024-02-02T13:24:36.771249+0000","last_peered":"2024-02-02T13:24:36.771249+0000","last_clean":"2024-02-02T13:24:36.771249+0000","last_became_active":"2024-02-02T13:24:36.768048+0000","last_became_peered":"2024-02-02T13:24:36.768048+0000","last_unstale":"2024-02-02T13:24:36.771249+0000","last_undegraded":"2024-02-02T13:24:36.771249+0000","last_fullsized":"2024-02-02T13:24:36.771249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978685+0000","last_change":"2024-02-02T13:24:36.762446+0000","last_active":"2024-02-02T13:24:40.978685+0000","last_peered":"2024-02-02T13:24:40.978685+0000","last_clean":"2024-02-02T13:24:40.978685+0000","last_became_active":"2024-02-02T13:24:36.762021+0000","last_became_peered":"2024-02-02T13:24:36.762021+0000","last_unstale":"2024-02-02T13:24:40.978685+0000","last_undegraded":"2024-02-02T13:24:40.978685+0000","last_fullsized":"2024-02-02T13:24:40.978685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768525+0000","last_change":"2024-02-02T13:24:36.768424+0000","last_active":"2024-02-02T13:24:36.768525+0000","last_peered":"2024-02-02T13:24:36.768525+0000","last_clean":"2024-02-02T13:24:36.768525+0000","last_became_active":"2024-02-02T13:24:36.768203+0000","last_became_peered":"2024-02-02T13:24:36.768203+0000","last_unstale":"2024-02-02T13:24:36.768525+0000","last_undegraded":"2024-02-02T13:24:36.768525+0000","last_fullsized":"2024-02-02T13:24:36.768525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093333+0000","last_change":"2024-02-02T13:24:36.762245+0000","last_active":"2024-02-02T13:24:41.093333+0000","last_peered":"2024-02-02T13:24:41.093333+0000","last_clean":"2024-02-02T13:24:41.093333+0000","last_became_active":"2024-02-02T13:24:36.762030+0000","last_became_peered":"2024-02-02T13:24:36.762030+0000","last_unstale":"2024-02-02T13:24:41.093333+0000","last_undegraded":"2024-02-02T13:24:41.093333+0000","last_fullsized":"2024-02-02T13:24:41.093333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771962+0000","last_change":"2024-02-02T13:24:36.771888+0000","last_active":"2024-02-02T13:24:36.771962+0000","last_peered":"2024-02-02T13:24:36.771962+0000","last_clean":"2024-02-02T13:24:36.771962+0000","last_became_active":"2024-02-02T13:24:36.770589+0000","last_became_peered":"2024-02-02T13:24:36.770589+0000","last_unstale":"2024-02-02T13:24:36.771962+0000","last_undegraded":"2024-02-02T13:24:36.771962+0000","last_fullsized":"2024-02-02T13:24:36.771962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212147+0000","last_change":"2024-02-02T13:24:36.750480+0000","last_active":"2024-02-02T13:24:41.212147+0000","last_peered":"2024-02-02T13:24:41.212147+0000","last_clean":"2024-02-02T13:24:41.212147+0000","last_became_active":"2024-02-02T13:24:36.750125+0000","last_became_peered":"2024-02-02T13:24:36.750125+0000","last_unstale":"2024-02-02T13:24:41.212147+0000","last_undegraded":"2024-02-02T13:24:41.212147+0000","last_fullsized":"2024-02-02T13:24:41.212147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773388+0000","last_change":"2024-02-02T13:24:36.773077+0000","last_active":"2024-02-02T13:24:36.773388+0000","last_peered":"2024-02-02T13:24:36.773388+0000","last_clean":"2024-02-02T13:24:36.773388+0000","last_became_active":"2024-02-02T13:24:36.765475+0000","last_became_peered":"2024-02-02T13:24:36.765475+0000","last_unstale":"2024-02-02T13:24:36.773388+0000","last_undegraded":"2024-02-02T13:24:36.773388+0000","last_fullsized":"2024-02-02T13:24:36.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211624+0000","last_change":"2024-02-02T13:24:36.750736+0000","last_active":"2024-02-02T13:24:41.211624+0000","last_peered":"2024-02-02T13:24:41.211624+0000","last_clean":"2024-02-02T13:24:41.211624+0000","last_became_active":"2024-02-02T13:24:36.750520+0000","last_became_peered":"2024-02-02T13:24:36.750520+0000","last_unstale":"2024-02-02T13:24:41.211624+0000","last_undegraded":"2024-02-02T13:24:41.211624+0000","last_fullsized":"2024-02-02T13:24:41.211624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764120+0000","last_change":"2024-02-02T13:24:36.763039+0000","last_active":"2024-02-02T13:24:36.764120+0000","last_peered":"2024-02-02T13:24:36.764120+0000","last_clean":"2024-02-02T13:24:36.764120+0000","last_became_active":"2024-02-02T13:24:36.762632+0000","last_became_peered":"2024-02-02T13:24:36.762632+0000","last_unstale":"2024-02-02T13:24:36.764120+0000","last_undegraded":"2024-02-02T13:24:36.764120+0000","last_fullsized":"2024-02-02T13:24:36.764120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877008+0000","last_change":"2024-02-02T13:24:36.750634+0000","last_active":"2024-02-02T13:24:39.877008+0000","last_peered":"2024-02-02T13:24:39.877008+0000","last_clean":"2024-02-02T13:24:39.877008+0000","last_became_active":"2024-02-02T13:24:36.750023+0000","last_became_peered":"2024-02-02T13:24:36.750023+0000","last_unstale":"2024-02-02T13:24:39.877008+0000","last_undegraded":"2024-02-02T13:24:39.877008+0000","last_fullsized":"2024-02-02T13:24:39.877008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211911+0000","last_change":"2024-02-02T13:24:36.750339+0000","last_active":"2024-02-02T13:24:41.211911+0000","last_peered":"2024-02-02T13:24:41.211911+0000","last_clean":"2024-02-02T13:24:41.211911+0000","last_became_active":"2024-02-02T13:24:36.749819+0000","last_became_peered":"2024-02-02T13:24:36.749819+0000","last_unstale":"2024-02-02T13:24:41.211911+0000","last_undegraded":"2024-02-02T13:24:41.211911+0000","last_fullsized":"2024-02-02T13:24:41.211911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093260+0000","last_change":"2024-02-02T13:24:36.765687+0000","last_active":"2024-02-02T13:24:41.093260+0000","last_peered":"2024-02-02T13:24:41.093260+0000","last_clean":"2024-02-02T13:24:41.093260+0000","last_became_active":"2024-02-02T13:24:36.765417+0000","last_became_peered":"2024-02-02T13:24:36.765417+0000","last_unstale":"2024-02-02T13:24:41.093260+0000","last_undegraded":"2024-02-02T13:24:41.093260+0000","last_fullsized":"2024-02-02T13:24:41.093260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762953+0000","last_change":"2024-02-02T13:24:36.762822+0000","last_active":"2024-02-02T13:24:36.762953+0000","last_peered":"2024-02-02T13:24:36.762953+0000","last_clean":"2024-02-02T13:24:36.762953+0000","last_became_active":"2024-02-02T13:24:36.761559+0000","last_became_peered":"2024-02-02T13:24:36.761559+0000","last_unstale":"2024-02-02T13:24:36.762953+0000","last_undegraded":"2024-02-02T13:24:36.762953+0000","last_fullsized":"2024-02-02T13:24:36.762953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764632+0000","last_change":"2024-02-02T13:24:36.764552+0000","last_active":"2024-02-02T13:24:36.764632+0000","last_peered":"2024-02-02T13:24:36.764632+0000","last_clean":"2024-02-02T13:24:36.764632+0000","last_became_active":"2024-02-02T13:24:36.764330+0000","last_became_peered":"2024-02-02T13:24:36.764330+0000","last_unstale":"2024-02-02T13:24:36.764632+0000","last_undegraded":"2024-02-02T13:24:36.764632+0000","last_fullsized":"2024-02-02T13:24:36.764632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.765462+0000","last_change":"2024-02-02T13:24:36.765179+0000","last_active":"2024-02-02T13:24:36.765462+0000","last_peered":"2024-02-02T13:24:36.765462+0000","last_clean":"2024-02-02T13:24:36.765462+0000","last_became_active":"2024-02-02T13:24:36.764850+0000","last_became_peered":"2024-02-02T13:24:36.764850+0000","last_unstale":"2024-02-02T13:24:36.765462+0000","last_undegraded":"2024-02-02T13:24:36.765462+0000","last_fullsized":"2024-02-02T13:24:36.765462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753019+0000","last_change":"2024-02-02T13:24:36.755360+0000","last_active":"2024-02-02T13:24:37.753019+0000","last_peered":"2024-02-02T13:24:37.753019+0000","last_clean":"2024-02-02T13:24:37.753019+0000","last_became_active":"2024-02-02T13:24:36.754987+0000","last_became_peered":"2024-02-02T13:24:36.754987+0000","last_unstale":"2024-02-02T13:24:37.753019+0000","last_undegraded":"2024-02-02T13:24:37.753019+0000","last_fullsized":"2024-02-02T13:24:37.753019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772676+0000","last_change":"2024-02-02T13:24:36.288622+0000","last_active":"2024-02-02T13:24:36.772676+0000","last_peered":"2024-02-02T13:24:36.772676+0000","last_clean":"2024-02-02T13:24:36.772676+0000","last_became_active":"2024-02-02T13:24:36.288400+0000","last_became_peered":"2024-02-02T13:24:36.288400+0000","last_unstale":"2024-02-02T13:24:36.772676+0000","last_undegraded":"2024-02-02T13:24:36.772676+0000","last_fullsized":"2024-02-02T13:24:36.772676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092688+0000","last_change":"2024-02-02T13:24:36.765345+0000","last_active":"2024-02-02T13:24:41.092688+0000","last_peered":"2024-02-02T13:24:41.092688+0000","last_clean":"2024-02-02T13:24:41.092688+0000","last_became_active":"2024-02-02T13:24:36.765122+0000","last_became_peered":"2024-02-02T13:24:36.765122+0000","last_unstale":"2024-02-02T13:24:41.092688+0000","last_undegraded":"2024-02-02T13:24:41.092688+0000","last_fullsized":"2024-02-02T13:24:41.092688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774103+0000","last_change":"2024-02-02T13:24:36.284315+0000","last_active":"2024-02-02T13:24:36.774103+0000","last_peered":"2024-02-02T13:24:36.774103+0000","last_clean":"2024-02-02T13:24:36.774103+0000","last_became_active":"2024-02-02T13:24:36.283193+0000","last_became_peered":"2024-02-02T13:24:36.283193+0000","last_unstale":"2024-02-02T13:24:36.774103+0000","last_undegraded":"2024-02-02T13:24:36.774103+0000","last_fullsized":"2024-02-02T13:24:36.774103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.741346+0000","last_change":"2024-02-02T13:24:36.280943+0000","last_active":"2024-02-02T13:24:36.741346+0000","last_peered":"2024-02-02T13:24:36.741346+0000","last_clean":"2024-02-02T13:24:36.741346+0000","last_became_active":"2024-02-02T13:24:36.280565+0000","last_became_peered":"2024-02-02T13:24:36.280565+0000","last_unstale":"2024-02-02T13:24:36.741346+0000","last_undegraded":"2024-02-02T13:24:36.741346+0000","last_fullsized":"2024-02-02T13:24:36.741346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211701+0000","last_change":"2024-02-02T13:24:36.761849+0000","last_active":"2024-02-02T13:24:41.211701+0000","last_peered":"2024-02-02T13:24:41.211701+0000","last_clean":"2024-02-02T13:24:41.211701+0000","last_became_active":"2024-02-02T13:24:36.761460+0000","last_became_peered":"2024-02-02T13:24:36.761460+0000","last_unstale":"2024-02-02T13:24:41.211701+0000","last_undegraded":"2024-02-02T13:24:41.211701+0000","last_fullsized":"2024-02-02T13:24:41.211701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.753498+0000","last_change":"2024-02-02T13:24:36.286417+0000","last_active":"2024-02-02T13:24:36.753498+0000","last_peered":"2024-02-02T13:24:36.753498+0000","last_clean":"2024-02-02T13:24:36.753498+0000","last_became_active":"2024-02-02T13:24:36.286107+0000","last_became_peered":"2024-02-02T13:24:36.286107+0000","last_unstale":"2024-02-02T13:24:36.753498+0000","last_undegraded":"2024-02-02T13:24:36.753498+0000","last_fullsized":"2024-02-02T13:24:36.753498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876738+0000","last_change":"2024-02-02T13:24:36.750157+0000","last_active":"2024-02-02T13:24:39.876738+0000","last_peered":"2024-02-02T13:24:39.876738+0000","last_clean":"2024-02-02T13:24:39.876738+0000","last_became_active":"2024-02-02T13:24:36.749764+0000","last_became_peered":"2024-02-02T13:24:36.749764+0000","last_unstale":"2024-02-02T13:24:39.876738+0000","last_undegraded":"2024-02-02T13:24:39.876738+0000","last_fullsized":"2024-02-02T13:24:39.876738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762177+0000","last_change":"2024-02-02T13:24:36.290816+0000","last_active":"2024-02-02T13:24:36.762177+0000","last_peered":"2024-02-02T13:24:36.762177+0000","last_clean":"2024-02-02T13:24:36.762177+0000","last_became_active":"2024-02-02T13:24:36.290592+0000","last_became_peered":"2024-02-02T13:24:36.290592+0000","last_unstale":"2024-02-02T13:24:36.762177+0000","last_undegraded":"2024-02-02T13:24:36.762177+0000","last_fullsized":"2024-02-02T13:24:36.762177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761656+0000","last_change":"2024-02-02T13:24:36.761589+0000","last_active":"2024-02-02T13:24:36.761656+0000","last_peered":"2024-02-02T13:24:36.761656+0000","last_clean":"2024-02-02T13:24:36.761656+0000","last_became_active":"2024-02-02T13:24:36.761215+0000","last_became_peered":"2024-02-02T13:24:36.761215+0000","last_unstale":"2024-02-02T13:24:36.761656+0000","last_undegraded":"2024-02-02T13:24:36.761656+0000","last_fullsized":"2024-02-02T13:24:36.761656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.758412+0000","last_change":"2024-02-02T13:24:36.281944+0000","last_active":"2024-02-02T13:24:36.758412+0000","last_peered":"2024-02-02T13:24:36.758412+0000","last_clean":"2024-02-02T13:24:36.758412+0000","last_became_active":"2024-02-02T13:24:36.281546+0000","last_became_peered":"2024-02-02T13:24:36.281546+0000","last_unstale":"2024-02-02T13:24:36.758412+0000","last_undegraded":"2024-02-02T13:24:36.758412+0000","last_fullsized":"2024-02-02T13:24:36.758412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093550+0000","last_change":"2024-02-02T13:24:36.775310+0000","last_active":"2024-02-02T13:24:41.093550+0000","last_peered":"2024-02-02T13:24:41.093550+0000","last_clean":"2024-02-02T13:24:41.093550+0000","last_became_active":"2024-02-02T13:24:36.775021+0000","last_became_peered":"2024-02-02T13:24:36.775021+0000","last_unstale":"2024-02-02T13:24:41.093550+0000","last_undegraded":"2024-02-02T13:24:41.093550+0000","last_fullsized":"2024-02-02T13:24:41.093550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777366+0000","last_change":"2024-02-02T13:24:36.267935+0000","last_active":"2024-02-02T13:24:36.777366+0000","last_peered":"2024-02-02T13:24:36.777366+0000","last_clean":"2024-02-02T13:24:36.777366+0000","last_became_active":"2024-02-02T13:24:36.267120+0000","last_became_peered":"2024-02-02T13:24:36.267120+0000","last_unstale":"2024-02-02T13:24:36.777366+0000","last_undegraded":"2024-02-02T13:24:36.777366+0000","last_fullsized":"2024-02-02T13:24:36.777366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753018+0000","last_change":"2024-02-02T13:24:36.758353+0000","last_active":"2024-02-02T13:24:37.753018+0000","last_peered":"2024-02-02T13:24:37.753018+0000","last_clean":"2024-02-02T13:24:37.753018+0000","last_became_active":"2024-02-02T13:24:36.758074+0000","last_became_peered":"2024-02-02T13:24:36.758074+0000","last_unstale":"2024-02-02T13:24:37.753018+0000","last_undegraded":"2024-02-02T13:24:37.753018+0000","last_fullsized":"2024-02-02T13:24:37.753018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.776688+0000","last_change":"2024-02-02T13:24:36.283779+0000","last_active":"2024-02-02T13:24:36.776688+0000","last_peered":"2024-02-02T13:24:36.776688+0000","last_clean":"2024-02-02T13:24:36.776688+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.776688+0000","last_undegraded":"2024-02-02T13:24:36.776688+0000","last_fullsized":"2024-02-02T13:24:36.776688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978240+0000","last_change":"2024-02-02T13:24:36.751477+0000","last_active":"2024-02-02T13:24:40.978240+0000","last_peered":"2024-02-02T13:24:40.978240+0000","last_clean":"2024-02-02T13:24:40.978240+0000","last_became_active":"2024-02-02T13:24:36.751226+0000","last_became_peered":"2024-02-02T13:24:36.751226+0000","last_unstale":"2024-02-02T13:24:40.978240+0000","last_undegraded":"2024-02-02T13:24:40.978240+0000","last_fullsized":"2024-02-02T13:24:40.978240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773129+0000","last_change":"2024-02-02T13:24:36.286732+0000","last_active":"2024-02-02T13:24:36.773129+0000","last_peered":"2024-02-02T13:24:36.773129+0000","last_clean":"2024-02-02T13:24:36.773129+0000","last_became_active":"2024-02-02T13:24:36.286351+0000","last_became_peered":"2024-02-02T13:24:36.286351+0000","last_unstale":"2024-02-02T13:24:36.773129+0000","last_undegraded":"2024-02-02T13:24:36.773129+0000","last_fullsized":"2024-02-02T13:24:36.773129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093755+0000","last_change":"2024-02-02T13:24:36.756739+0000","last_active":"2024-02-02T13:24:41.093755+0000","last_peered":"2024-02-02T13:24:41.093755+0000","last_clean":"2024-02-02T13:24:41.093755+0000","last_became_active":"2024-02-02T13:24:36.756537+0000","last_became_peered":"2024-02-02T13:24:36.756537+0000","last_unstale":"2024-02-02T13:24:41.093755+0000","last_undegraded":"2024-02-02T13:24:41.093755+0000","last_fullsized":"2024-02-02T13:24:41.093755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755625+0000","last_change":"2024-02-02T13:24:36.286698+0000","last_active":"2024-02-02T13:24:36.755625+0000","last_peered":"2024-02-02T13:24:36.755625+0000","last_clean":"2024-02-02T13:24:36.755625+0000","last_became_active":"2024-02-02T13:24:36.282637+0000","last_became_peered":"2024-02-02T13:24:36.282637+0000","last_unstale":"2024-02-02T13:24:36.755625+0000","last_undegraded":"2024-02-02T13:24:36.755625+0000","last_fullsized":"2024-02-02T13:24:36.755625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978581+0000","last_change":"2024-02-02T13:24:36.766477+0000","last_active":"2024-02-02T13:24:40.978581+0000","last_peered":"2024-02-02T13:24:40.978581+0000","last_clean":"2024-02-02T13:24:40.978581+0000","last_became_active":"2024-02-02T13:24:36.765889+0000","last_became_peered":"2024-02-02T13:24:36.765889+0000","last_unstale":"2024-02-02T13:24:40.978581+0000","last_undegraded":"2024-02-02T13:24:40.978581+0000","last_fullsized":"2024-02-02T13:24:40.978581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764428+0000","last_change":"2024-02-02T13:24:36.286937+0000","last_active":"2024-02-02T13:24:36.764428+0000","last_peered":"2024-02-02T13:24:36.764428+0000","last_clean":"2024-02-02T13:24:36.764428+0000","last_became_active":"2024-02-02T13:24:36.286651+0000","last_became_peered":"2024-02-02T13:24:36.286651+0000","last_unstale":"2024-02-02T13:24:36.764428+0000","last_undegraded":"2024-02-02T13:24:36.764428+0000","last_fullsized":"2024-02-02T13:24:36.764428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.769009+0000","last_change":"2024-02-02T13:24:36.768901+0000","last_active":"2024-02-02T13:24:36.769009+0000","last_peered":"2024-02-02T13:24:36.769009+0000","last_clean":"2024-02-02T13:24:36.769009+0000","last_became_active":"2024-02-02T13:24:36.768565+0000","last_became_peered":"2024-02-02T13:24:36.768565+0000","last_unstale":"2024-02-02T13:24:36.769009+0000","last_undegraded":"2024-02-02T13:24:36.769009+0000","last_fullsized":"2024-02-02T13:24:36.769009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775361+0000","last_change":"2024-02-02T13:24:36.271772+0000","last_active":"2024-02-02T13:24:36.775361+0000","last_peered":"2024-02-02T13:24:36.775361+0000","last_clean":"2024-02-02T13:24:36.775361+0000","last_became_active":"2024-02-02T13:24:36.271163+0000","last_became_peered":"2024-02-02T13:24:36.271163+0000","last_unstale":"2024-02-02T13:24:36.775361+0000","last_undegraded":"2024-02-02T13:24:36.775361+0000","last_fullsized":"2024-02-02T13:24:36.775361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.979190+0000","last_change":"2024-02-02T13:24:36.756837+0000","last_active":"2024-02-02T13:24:40.979190+0000","last_peered":"2024-02-02T13:24:40.979190+0000","last_clean":"2024-02-02T13:24:40.979190+0000","last_became_active":"2024-02-02T13:24:36.756508+0000","last_became_peered":"2024-02-02T13:24:36.756508+0000","last_unstale":"2024-02-02T13:24:40.979190+0000","last_undegraded":"2024-02-02T13:24:40.979190+0000","last_fullsized":"2024-02-02T13:24:40.979190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771720+0000","last_change":"2024-02-02T13:24:36.287213+0000","last_active":"2024-02-02T13:24:36.771720+0000","last_peered":"2024-02-02T13:24:36.771720+0000","last_clean":"2024-02-02T13:24:36.771720+0000","last_became_active":"2024-02-02T13:24:36.286967+0000","last_became_peered":"2024-02-02T13:24:36.286967+0000","last_unstale":"2024-02-02T13:24:36.771720+0000","last_undegraded":"2024-02-02T13:24:36.771720+0000","last_fullsized":"2024-02-02T13:24:36.771720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978811+0000","last_change":"2024-02-02T13:24:36.766631+0000","last_active":"2024-02-02T13:24:40.978811+0000","last_peered":"2024-02-02T13:24:40.978811+0000","last_clean":"2024-02-02T13:24:40.978811+0000","last_became_active":"2024-02-02T13:24:36.766216+0000","last_became_peered":"2024-02-02T13:24:36.766216+0000","last_unstale":"2024-02-02T13:24:40.978811+0000","last_undegraded":"2024-02-02T13:24:40.978811+0000","last_fullsized":"2024-02-02T13:24:40.978811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772131+0000","last_change":"2024-02-02T13:24:36.275397+0000","last_active":"2024-02-02T13:24:36.772131+0000","last_peered":"2024-02-02T13:24:36.772131+0000","last_clean":"2024-02-02T13:24:36.772131+0000","last_became_active":"2024-02-02T13:24:36.274927+0000","last_became_peered":"2024-02-02T13:24:36.274927+0000","last_unstale":"2024-02-02T13:24:36.772131+0000","last_undegraded":"2024-02-02T13:24:36.772131+0000","last_fullsized":"2024-02-02T13:24:36.772131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978419+0000","last_change":"2024-02-02T13:24:36.757035+0000","last_active":"2024-02-02T13:24:40.978419+0000","last_peered":"2024-02-02T13:24:40.978419+0000","last_clean":"2024-02-02T13:24:40.978419+0000","last_became_active":"2024-02-02T13:24:36.756748+0000","last_became_peered":"2024-02-02T13:24:36.756748+0000","last_unstale":"2024-02-02T13:24:40.978419+0000","last_undegraded":"2024-02-02T13:24:40.978419+0000","last_fullsized":"2024-02-02T13:24:40.978419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:24:57.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768414+0000","last_change":"2024-02-02T13:24:36.283044+0000","last_active":"2024-02-02T13:24:36.768414+0000","last_peered":"2024-02-02T13:24:36.768414+0000","last_clean":"2024-02-02T13:24:36.768414+0000","last_became_active":"2024-02-02T13:24:36.282798+0000","last_became_peered":"2024-02-02T13:24:36.282798+0000","last_unstale":"2024-02-02T13:24:36.768414+0000","last_undegraded":"2024-02-02T13:24:36.768414+0000","last_fullsized":"2024-02-02T13:24:36.768414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770333+0000","last_change":"2024-02-02T13:24:36.764197+0000","last_active":"2024-02-02T13:24:36.770333+0000","last_peered":"2024-02-02T13:24:36.770333+0000","last_clean":"2024-02-02T13:24:36.770333+0000","last_became_active":"2024-02-02T13:24:36.763970+0000","last_became_peered":"2024-02-02T13:24:36.763970+0000","last_unstale":"2024-02-02T13:24:36.770333+0000","last_undegraded":"2024-02-02T13:24:36.770333+0000","last_fullsized":"2024-02-02T13:24:36.770333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.757774+0000","last_change":"2024-02-02T13:24:36.279681+0000","last_active":"2024-02-02T13:24:36.757774+0000","last_peered":"2024-02-02T13:24:36.757774+0000","last_clean":"2024-02-02T13:24:36.757774+0000","last_became_active":"2024-02-02T13:24:36.279342+0000","last_became_peered":"2024-02-02T13:24:36.279342+0000","last_unstale":"2024-02-02T13:24:36.757774+0000","last_undegraded":"2024-02-02T13:24:36.757774+0000","last_fullsized":"2024-02-02T13:24:36.757774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978828+0000","last_change":"2024-02-02T13:24:36.758113+0000","last_active":"2024-02-02T13:24:40.978828+0000","last_peered":"2024-02-02T13:24:40.978828+0000","last_clean":"2024-02-02T13:24:40.978828+0000","last_became_active":"2024-02-02T13:24:36.757859+0000","last_became_peered":"2024-02-02T13:24:36.757859+0000","last_unstale":"2024-02-02T13:24:40.978828+0000","last_undegraded":"2024-02-02T13:24:40.978828+0000","last_fullsized":"2024-02-02T13:24:40.978828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748840+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:36.748840+0000","last_peered":"2024-02-02T13:24:36.748840+0000","last_clean":"2024-02-02T13:24:36.748840+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:36.748840+0000","last_undegraded":"2024-02-02T13:24:36.748840+0000","last_fullsized":"2024-02-02T13:24:36.748840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755725+0000","last_change":"2024-02-02T13:24:36.285102+0000","last_active":"2024-02-02T13:24:36.755725+0000","last_peered":"2024-02-02T13:24:36.755725+0000","last_clean":"2024-02-02T13:24:36.755725+0000","last_became_active":"2024-02-02T13:24:36.280044+0000","last_became_peered":"2024-02-02T13:24:36.280044+0000","last_unstale":"2024-02-02T13:24:36.755725+0000","last_undegraded":"2024-02-02T13:24:36.755725+0000","last_fullsized":"2024-02-02T13:24:36.755725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211798+0000","last_change":"2024-02-02T13:24:36.762214+0000","last_active":"2024-02-02T13:24:41.211798+0000","last_peered":"2024-02-02T13:24:41.211798+0000","last_clean":"2024-02-02T13:24:41.211798+0000","last_became_active":"2024-02-02T13:24:36.761690+0000","last_became_peered":"2024-02-02T13:24:36.761690+0000","last_unstale":"2024-02-02T13:24:41.211798+0000","last_undegraded":"2024-02-02T13:24:41.211798+0000","last_fullsized":"2024-02-02T13:24:41.211798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748517+0000","last_change":"2024-02-02T13:24:36.291720+0000","last_active":"2024-02-02T13:24:36.748517+0000","last_peered":"2024-02-02T13:24:36.748517+0000","last_clean":"2024-02-02T13:24:36.748517+0000","last_became_active":"2024-02-02T13:24:36.290703+0000","last_became_peered":"2024-02-02T13:24:36.290703+0000","last_unstale":"2024-02-02T13:24:36.748517+0000","last_undegraded":"2024-02-02T13:24:36.748517+0000","last_fullsized":"2024-02-02T13:24:36.748517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093549+0000","last_change":"2024-02-02T13:24:36.766220+0000","last_active":"2024-02-02T13:24:41.093549+0000","last_peered":"2024-02-02T13:24:41.093549+0000","last_clean":"2024-02-02T13:24:41.093549+0000","last_became_active":"2024-02-02T13:24:36.765734+0000","last_became_peered":"2024-02-02T13:24:36.765734+0000","last_unstale":"2024-02-02T13:24:41.093549+0000","last_undegraded":"2024-02-02T13:24:41.093549+0000","last_fullsized":"2024-02-02T13:24:41.093549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.756734+0000","last_change":"2024-02-02T13:24:36.276562+0000","last_active":"2024-02-02T13:24:36.756734+0000","last_peered":"2024-02-02T13:24:36.756734+0000","last_clean":"2024-02-02T13:24:36.756734+0000","last_became_active":"2024-02-02T13:24:36.276189+0000","last_became_peered":"2024-02-02T13:24:36.276189+0000","last_unstale":"2024-02-02T13:24:36.756734+0000","last_undegraded":"2024-02-02T13:24:36.756734+0000","last_fullsized":"2024-02-02T13:24:36.756734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877004+0000","last_change":"2024-02-02T13:24:36.761943+0000","last_active":"2024-02-02T13:24:39.877004+0000","last_peered":"2024-02-02T13:24:39.877004+0000","last_clean":"2024-02-02T13:24:39.877004+0000","last_became_active":"2024-02-02T13:24:36.761718+0000","last_became_peered":"2024-02-02T13:24:36.761718+0000","last_unstale":"2024-02-02T13:24:39.877004+0000","last_undegraded":"2024-02-02T13:24:39.877004+0000","last_fullsized":"2024-02-02T13:24:39.877004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.760484+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.760484+0000","last_peered":"2024-02-02T13:24:36.760484+0000","last_clean":"2024-02-02T13:24:36.760484+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.760484+0000","last_undegraded":"2024-02-02T13:24:36.760484+0000","last_fullsized":"2024-02-02T13:24:36.760484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753283+0000","last_change":"2024-02-02T13:24:36.763372+0000","last_active":"2024-02-02T13:24:37.753283+0000","last_peered":"2024-02-02T13:24:37.753283+0000","last_clean":"2024-02-02T13:24:37.753283+0000","last_became_active":"2024-02-02T13:24:36.763132+0000","last_became_peered":"2024-02-02T13:24:36.763132+0000","last_unstale":"2024-02-02T13:24:37.753283+0000","last_undegraded":"2024-02-02T13:24:37.753283+0000","last_fullsized":"2024-02-02T13:24:37.753283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:03.055262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747812+0000","last_change":"2024-02-02T13:24:36.287910+0000","last_active":"2024-02-02T13:24:36.747812+0000","last_peered":"2024-02-02T13:24:36.747812+0000","last_clean":"2024-02-02T13:24:36.747812+0000","last_became_active":"2024-02-02T13:24:36.287683+0000","last_became_peered":"2024-02-02T13:24:36.287683+0000","last_unstale":"2024-02-02T13:24:36.747812+0000","last_undegraded":"2024-02-02T13:24:36.747812+0000","last_fullsized":"2024-02-02T13:24:36.747812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773408+0000","last_change":"2024-02-02T13:24:36.773275+0000","last_active":"2024-02-02T13:24:36.773408+0000","last_peered":"2024-02-02T13:24:36.773408+0000","last_clean":"2024-02-02T13:24:36.773408+0000","last_became_active":"2024-02-02T13:24:36.772646+0000","last_became_peered":"2024-02-02T13:24:36.772646+0000","last_unstale":"2024-02-02T13:24:36.773408+0000","last_undegraded":"2024-02-02T13:24:36.773408+0000","last_fullsized":"2024-02-02T13:24:36.773408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748406+0000","last_change":"2024-02-02T13:24:36.291402+0000","last_active":"2024-02-02T13:24:36.748406+0000","last_peered":"2024-02-02T13:24:36.748406+0000","last_clean":"2024-02-02T13:24:36.748406+0000","last_became_active":"2024-02-02T13:24:36.290274+0000","last_became_peered":"2024-02-02T13:24:36.290274+0000","last_unstale":"2024-02-02T13:24:36.748406+0000","last_undegraded":"2024-02-02T13:24:36.748406+0000","last_fullsized":"2024-02-02T13:24:36.748406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876617+0000","last_change":"2024-02-02T13:24:36.760028+0000","last_active":"2024-02-02T13:24:39.876617+0000","last_peered":"2024-02-02T13:24:39.876617+0000","last_clean":"2024-02-02T13:24:39.876617+0000","last_became_active":"2024-02-02T13:24:36.759641+0000","last_became_peered":"2024-02-02T13:24:36.759641+0000","last_unstale":"2024-02-02T13:24:39.876617+0000","last_undegraded":"2024-02-02T13:24:39.876617+0000","last_fullsized":"2024-02-02T13:24:39.876617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.746080+0000","last_change":"2024-02-02T13:24:36.281619+0000","last_active":"2024-02-02T13:24:36.746080+0000","last_peered":"2024-02-02T13:24:36.746080+0000","last_clean":"2024-02-02T13:24:36.746080+0000","last_became_active":"2024-02-02T13:24:36.281168+0000","last_became_peered":"2024-02-02T13:24:36.281168+0000","last_unstale":"2024-02-02T13:24:36.746080+0000","last_undegraded":"2024-02-02T13:24:36.746080+0000","last_fullsized":"2024-02-02T13:24:36.746080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211709+0000","last_change":"2024-02-02T13:24:36.750882+0000","last_active":"2024-02-02T13:24:41.211709+0000","last_peered":"2024-02-02T13:24:41.211709+0000","last_clean":"2024-02-02T13:24:41.211709+0000","last_became_active":"2024-02-02T13:24:36.750448+0000","last_became_peered":"2024-02-02T13:24:36.750448+0000","last_unstale":"2024-02-02T13:24:41.211709+0000","last_undegraded":"2024-02-02T13:24:41.211709+0000","last_fullsized":"2024-02-02T13:24:41.211709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774907+0000","last_change":"2024-02-02T13:24:36.289169+0000","last_active":"2024-02-02T13:24:36.774907+0000","last_peered":"2024-02-02T13:24:36.774907+0000","last_clean":"2024-02-02T13:24:36.774907+0000","last_became_active":"2024-02-02T13:24:36.288939+0000","last_became_peered":"2024-02-02T13:24:36.288939+0000","last_unstale":"2024-02-02T13:24:36.774907+0000","last_undegraded":"2024-02-02T13:24:36.774907+0000","last_fullsized":"2024-02-02T13:24:36.774907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093133+0000","last_change":"2024-02-02T13:24:36.756653+0000","last_active":"2024-02-02T13:24:41.093133+0000","last_peered":"2024-02-02T13:24:41.093133+0000","last_clean":"2024-02-02T13:24:41.093133+0000","last_became_active":"2024-02-02T13:24:36.756419+0000","last_became_peered":"2024-02-02T13:24:36.756419+0000","last_unstale":"2024-02-02T13:24:41.093133+0000","last_undegraded":"2024-02-02T13:24:41.093133+0000","last_fullsized":"2024-02-02T13:24:41.093133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.751195+0000","last_change":"2024-02-02T13:24:36.290423+0000","last_active":"2024-02-02T13:24:36.751195+0000","last_peered":"2024-02-02T13:24:36.751195+0000","last_clean":"2024-02-02T13:24:36.751195+0000","last_became_active":"2024-02-02T13:24:36.289863+0000","last_became_peered":"2024-02-02T13:24:36.289863+0000","last_unstale":"2024-02-02T13:24:36.751195+0000","last_undegraded":"2024-02-02T13:24:36.751195+0000","last_fullsized":"2024-02-02T13:24:36.751195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.754234+0000","last_change":"2024-02-02T13:24:36.759429+0000","last_active":"2024-02-02T13:24:37.754234+0000","last_peered":"2024-02-02T13:24:37.754234+0000","last_clean":"2024-02-02T13:24:37.754234+0000","last_became_active":"2024-02-02T13:24:36.759196+0000","last_became_peered":"2024-02-02T13:24:36.759196+0000","last_unstale":"2024-02-02T13:24:37.754234+0000","last_undegraded":"2024-02-02T13:24:37.754234+0000","last_fullsized":"2024-02-02T13:24:37.754234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747537+0000","last_change":"2024-02-02T13:24:36.288653+0000","last_active":"2024-02-02T13:24:36.747537+0000","last_peered":"2024-02-02T13:24:36.747537+0000","last_clean":"2024-02-02T13:24:36.747537+0000","last_became_active":"2024-02-02T13:24:36.288439+0000","last_became_peered":"2024-02-02T13:24:36.288439+0000","last_unstale":"2024-02-02T13:24:36.747537+0000","last_undegraded":"2024-02-02T13:24:36.747537+0000","last_fullsized":"2024-02-02T13:24:36.747537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211609+0000","last_change":"2024-02-02T13:24:36.751484+0000","last_active":"2024-02-02T13:24:41.211609+0000","last_peered":"2024-02-02T13:24:41.211609+0000","last_clean":"2024-02-02T13:24:41.211609+0000","last_became_active":"2024-02-02T13:24:36.751091+0000","last_became_peered":"2024-02-02T13:24:36.751091+0000","last_unstale":"2024-02-02T13:24:41.211609+0000","last_undegraded":"2024-02-02T13:24:41.211609+0000","last_fullsized":"2024-02-02T13:24:41.211609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747419+0000","last_change":"2024-02-02T13:24:36.280128+0000","last_active":"2024-02-02T13:24:36.747419+0000","last_peered":"2024-02-02T13:24:36.747419+0000","last_clean":"2024-02-02T13:24:36.747419+0000","last_became_active":"2024-02-02T13:24:36.279652+0000","last_became_peered":"2024-02-02T13:24:36.279652+0000","last_unstale":"2024-02-02T13:24:36.747419+0000","last_undegraded":"2024-02-02T13:24:36.747419+0000","last_fullsized":"2024-02-02T13:24:36.747419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212026+0000","last_change":"2024-02-02T13:24:36.763668+0000","last_active":"2024-02-02T13:24:41.212026+0000","last_peered":"2024-02-02T13:24:41.212026+0000","last_clean":"2024-02-02T13:24:41.212026+0000","last_became_active":"2024-02-02T13:24:36.762913+0000","last_became_peered":"2024-02-02T13:24:36.762913+0000","last_unstale":"2024-02-02T13:24:41.212026+0000","last_undegraded":"2024-02-02T13:24:41.212026+0000","last_fullsized":"2024-02-02T13:24:41.212026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774478+0000","last_change":"2024-02-02T13:24:36.289669+0000","last_active":"2024-02-02T13:24:36.774478+0000","last_peered":"2024-02-02T13:24:36.774478+0000","last_clean":"2024-02-02T13:24:36.774478+0000","last_became_active":"2024-02-02T13:24:36.289368+0000","last_became_peered":"2024-02-02T13:24:36.289368+0000","last_unstale":"2024-02-02T13:24:36.774478+0000","last_undegraded":"2024-02-02T13:24:36.774478+0000","last_fullsized":"2024-02-02T13:24:36.774478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770307+0000","last_change":"2024-02-02T13:24:36.769265+0000","last_active":"2024-02-02T13:24:36.770307+0000","last_peered":"2024-02-02T13:24:36.770307+0000","last_clean":"2024-02-02T13:24:36.770307+0000","last_became_active":"2024-02-02T13:24:36.769036+0000","last_became_peered":"2024-02-02T13:24:36.769036+0000","last_unstale":"2024-02-02T13:24:36.770307+0000","last_undegraded":"2024-02-02T13:24:36.770307+0000","last_fullsized":"2024-02-02T13:24:36.770307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770353+0000","last_change":"2024-02-02T13:24:36.765383+0000","last_active":"2024-02-02T13:24:36.770353+0000","last_peered":"2024-02-02T13:24:36.770353+0000","last_clean":"2024-02-02T13:24:36.770353+0000","last_became_active":"2024-02-02T13:24:36.765141+0000","last_became_peered":"2024-02-02T13:24:36.765141+0000","last_unstale":"2024-02-02T13:24:36.770353+0000","last_undegraded":"2024-02-02T13:24:36.770353+0000","last_fullsized":"2024-02-02T13:24:36.770353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778010+0000","last_change":"2024-02-02T13:24:36.290626+0000","last_active":"2024-02-02T13:24:36.778010+0000","last_peered":"2024-02-02T13:24:36.778010+0000","last_clean":"2024-02-02T13:24:36.778010+0000","last_became_active":"2024-02-02T13:24:36.290183+0000","last_became_peered":"2024-02-02T13:24:36.290183+0000","last_unstale":"2024-02-02T13:24:36.778010+0000","last_undegraded":"2024-02-02T13:24:36.778010+0000","last_fullsized":"2024-02-02T13:24:36.778010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777961+0000","last_change":"2024-02-02T13:24:36.777865+0000","last_active":"2024-02-02T13:24:36.777961+0000","last_peered":"2024-02-02T13:24:36.777961+0000","last_clean":"2024-02-02T13:24:36.777961+0000","last_became_active":"2024-02-02T13:24:36.777129+0000","last_became_peered":"2024-02-02T13:24:36.777129+0000","last_unstale":"2024-02-02T13:24:36.777961+0000","last_undegraded":"2024-02-02T13:24:36.777961+0000","last_fullsized":"2024-02-02T13:24:36.777961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772162+0000","last_change":"2024-02-02T13:24:36.290121+0000","last_active":"2024-02-02T13:24:36.772162+0000","last_peered":"2024-02-02T13:24:36.772162+0000","last_clean":"2024-02-02T13:24:36.772162+0000","last_became_active":"2024-02-02T13:24:36.289708+0000","last_became_peered":"2024-02-02T13:24:36.289708+0000","last_unstale":"2024-02-02T13:24:36.772162+0000","last_undegraded":"2024-02-02T13:24:36.772162+0000","last_fullsized":"2024-02-02T13:24:36.772162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093334+0000","last_change":"2024-02-02T13:24:36.759606+0000","last_active":"2024-02-02T13:24:41.093334+0000","last_peered":"2024-02-02T13:24:41.093334+0000","last_clean":"2024-02-02T13:24:41.093334+0000","last_became_active":"2024-02-02T13:24:36.759358+0000","last_became_peered":"2024-02-02T13:24:36.759358+0000","last_unstale":"2024-02-02T13:24:41.093334+0000","last_undegraded":"2024-02-02T13:24:41.093334+0000","last_fullsized":"2024-02-02T13:24:41.093334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771605+0000","last_change":"2024-02-02T13:24:36.286535+0000","last_active":"2024-02-02T13:24:36.771605+0000","last_peered":"2024-02-02T13:24:36.771605+0000","last_clean":"2024-02-02T13:24:36.771605+0000","last_became_active":"2024-02-02T13:24:36.286302+0000","last_became_peered":"2024-02-02T13:24:36.286302+0000","last_unstale":"2024-02-02T13:24:36.771605+0000","last_undegraded":"2024-02-02T13:24:36.771605+0000","last_fullsized":"2024-02-02T13:24:36.771605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212029+0000","last_change":"2024-02-02T13:24:36.769601+0000","last_active":"2024-02-02T13:24:41.212029+0000","last_peered":"2024-02-02T13:24:41.212029+0000","last_clean":"2024-02-02T13:24:41.212029+0000","last_became_active":"2024-02-02T13:24:36.768381+0000","last_became_peered":"2024-02-02T13:24:36.768381+0000","last_unstale":"2024-02-02T13:24:41.212029+0000","last_undegraded":"2024-02-02T13:24:41.212029+0000","last_fullsized":"2024-02-02T13:24:41.212029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774155+0000","last_change":"2024-02-02T13:24:36.289368+0000","last_active":"2024-02-02T13:24:36.774155+0000","last_peered":"2024-02-02T13:24:36.774155+0000","last_clean":"2024-02-02T13:24:36.774155+0000","last_became_active":"2024-02-02T13:24:36.289118+0000","last_became_peered":"2024-02-02T13:24:36.289118+0000","last_unstale":"2024-02-02T13:24:36.774155+0000","last_undegraded":"2024-02-02T13:24:36.774155+0000","last_fullsized":"2024-02-02T13:24:36.774155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":640,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194672,"statfs":{"total":200000000,"available":199344612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":63,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195249,"statfs":{"total":200000000,"available":199935017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:42.616 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-02-02T13:24:42.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.858+0000 7f5c64600640 1 Processor -- start 2024-02-02T13:24:42.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.859+0000 7f5c64600640 1 -- start start 2024-02-02T13:24:42.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c5c15ad10 0x7f5c5c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c5c15b0f0 0x7f5c5c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cbf50 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cd610 con 0x7f5c5c15ad10 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.860+0000 7f5c64600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c06d910 con 0x7f5c5c0cb580 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c5c15b0f0 0x7f5c5c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c5c15b0f0 0x7f5c5c06d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53136/0 (socket says 172.21.15.91:53136) 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 -- 172.21.15.91:0/4023776065 learned_addr learned my addr 172.21.15.91:0/4023776065 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61c00640 1 --2- 172.21.15.91:0/4023776065 >> v2:172.21.15.113:3300/0 conn(0x7f5c5c15ad10 0x7f5c5c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c62600640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 -- 172.21.15.91:0/4023776065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 msgr2=0x7f5c5c0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 -- 172.21.15.91:0/4023776065 >> v2:172.21.15.113:3300/0 conn(0x7f5c5c15ad10 msgr2=0x7f5c5c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:42.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 --2- 172.21.15.91:0/4023776065 >> v2:172.21.15.113:3300/0 conn(0x7f5c5c15ad10 0x7f5c5c06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.861+0000 7f5c61200640 1 -- 172.21.15.91:0/4023776065 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0cc850 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.862+0000 7f5c61200640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c06d020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c4402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efff48e6021c8ccc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.862+0000 7f5c5be00640 1 -- 172.21.15.91:0/4023776065 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440317c0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.862+0000 7f5c5be00640 1 -- 172.21.15.91:0/4023776065 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44031920 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.862+0000 7f5c5be00640 1 -- 172.21.15.91:0/4023776065 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44031cb0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 -- 172.21.15.91:0/4023776065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 msgr2=0x7f5c5c06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c06d020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c4402b860 comp rx=0 tx=0).stop 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 -- 172.21.15.91:0/4023776065 shutdown_connections 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 --2- 172.21.15.91:0/4023776065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c06d020 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 --2- 172.21.15.91:0/4023776065 >> v2:172.21.15.113:3300/0 conn(0x7f5c5c15ad10 0x7f5c5c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 -- 172.21.15.91:0/4023776065 >> 172.21.15.91:0/4023776065 conn(0x7f5c5c0b0700 msgr2=0x7f5c5c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:42.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 -- 172.21.15.91:0/4023776065 shutdown_connections 2024-02-02T13:24:42.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.863+0000 7f5c64600640 1 -- 172.21.15.91:0/4023776065 wait complete. 2024-02-02T13:24:42.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.865+0000 7f5c64600640 1 Processor -- start 2024-02-02T13:24:42.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.865+0000 7f5c64600640 1 -- start start 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c1f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c5c15ad10 0x7f5c5c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c1f4800 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0af6d0 con 0x7f5c5c15ad10 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c64600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c15a020 con 0x7f5c5c0cb580 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c62600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c62600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53150/0 (socket says 172.21.15.91:53150) 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.866+0000 7f5c62600640 1 -- 172.21.15.91:0/4177505972 learned_addr learned my addr 172.21.15.91:0/4177505972 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c61c00640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c1f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c61200640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c5c15ad10 0x7f5c5c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 -- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 msgr2=0x7f5c5c1f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c1f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 -- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c5c15ad10 msgr2=0x7f5c5c159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c5c15ad10 0x7f5c5c159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 -- 172.21.15.91:0/4177505972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440094d0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c61c00640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c1f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c62600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400cd70 tx=0x7f5c5400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54003b90 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54003cf0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54004240 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c1f49f0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.867+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c1f4fb0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.869+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c066cf0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.869+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5c540043a0 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.870+0000 7f5c59600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 0x7f5c3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:42.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.870+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c54012070 con 0x7f5c5c15b0f0 2024-02-02T13:24:42.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.870+0000 7f5c61c00640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 0x7f5c3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:42.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.871+0000 7f5c61c00640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 0x7f5c3406c8f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006fd0 tx=0x7f5c4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:42.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:42.874+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5c54054910 con 0x7f5c5c15b0f0 2024-02-02T13:24:43.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.133+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5c5c15e600 con 0x7f5c5c15b0f0 2024-02-02T13:24:43.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.134+0000 7f5c59600640 1 -- 172.21.15.91:0/4177505972 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1004 (secure 0 0 0) 0x7f5c54057f30 con 0x7f5c5c15b0f0 2024-02-02T13:24:43.136 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:43.136 INFO:teuthology.orchestra.run.smithi091.stdout:{"fsid":"cc006970-f900-4781-8947-81ed22cd959b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1706880265,"num_in_osds":8,"osd_in_since":1706880262,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":1728388,"bytes_avail":1598271612,"bytes_total":1600000000,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4255}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-02-02T13:24:19.253210+0000","services":{}},"progress_events":{}} 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 msgr2=0x7f5c3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 0x7f5c3406c8f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006fd0 tx=0x7f5c4c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 msgr2=0x7f5c5c0c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400cd70 tx=0x7f5c5400a660 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 shutdown_connections 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c3406a430 0x7f5c3406c8f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c5c15b0f0 0x7f5c5c0c97c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c5c15ad10 0x7f5c5c159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 --2- 172.21.15.91:0/4177505972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c5c0cb580 0x7f5c5c1f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 >> 172.21.15.91:0/4177505972 conn(0x7f5c5c0b0700 msgr2=0x7f5c5c0c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:43.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 shutdown_connections 2024-02-02T13:24:43.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.137+0000 7f5c64600640 1 -- 172.21.15.91:0/4177505972 wait complete. 2024-02-02T13:24:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-02-02T13:24:43.158 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:43.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.368+0000 7f9c64e00640 1 Processor -- start 2024-02-02T13:24:43.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.369+0000 7f9c64e00640 1 -- start start 2024-02-02T13:24:43.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c64e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c600b8a50 0x7f9c600d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:43.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c64e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600cd930 con 0x7f9c600d0070 2024-02-02T13:24:43.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c64e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600d1680 con 0x7f9c600b2260 2024-02-02T13:24:43.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c64e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c600d1960 con 0x7f9c600b8a50 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c5e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c600b8a50 0x7f9c600d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c5e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c600b8a50 0x7f9c600d2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53160/0 (socket says 172.21.15.91:53160) 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.370+0000 7f9c5e800640 1 -- 172.21.15.91:0/3242533781 learned_addr learned my addr 172.21.15.91:0/3242533781 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.371+0000 7f9c5e800640 1 -- 172.21.15.91:0/3242533781 >> v1:172.21.15.113:6789/0 conn(0x7f9c600b2260 legacy=0x7f9c600b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.371+0000 7f9c5e800640 1 -- 172.21.15.91:0/3242533781 >> v1:172.21.15.91:6789/0 conn(0x7f9c600d0070 legacy=0x7f9c600d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.371+0000 7f9c5e800640 1 -- 172.21.15.91:0/3242533781 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600d1c40 con 0x7f9c600b8a50 2024-02-02T13:24:43.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.372+0000 7f9c5e800640 1 --2- 172.21.15.91:0/3242533781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600b8a50 0x7f9c600d2440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009e50 tx=0x7f9c5802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabf1b8a065904c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:43.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.372+0000 7f9c5ca00640 1 -- 172.21.15.91:0/3242533781 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5802cb10 con 0x7f9c600b8a50 2024-02-02T13:24:43.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.372+0000 7f9c5ca00640 1 -- 172.21.15.91:0/3242533781 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5802cc70 con 0x7f9c600b8a50 2024-02-02T13:24:43.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.372+0000 7f9c5ca00640 1 -- 172.21.15.91:0/3242533781 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580343f0 con 0x7f9c600b8a50 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 -- 172.21.15.91:0/3242533781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600b8a50 msgr2=0x7f9c600d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3242533781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600b8a50 0x7f9c600d2440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009e50 tx=0x7f9c5802bf70 comp rx=0 tx=0).stop 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 -- 172.21.15.91:0/3242533781 shutdown_connections 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3242533781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600b8a50 0x7f9c600d2440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 -- 172.21.15.91:0/3242533781 >> 172.21.15.91:0/3242533781 conn(0x7f9c60070660 msgr2=0x7f9c60070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 -- 172.21.15.91:0/3242533781 shutdown_connections 2024-02-02T13:24:43.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.373+0000 7f9c64e00640 1 -- 172.21.15.91:0/3242533781 wait complete. 2024-02-02T13:24:43.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.375+0000 7f9c64e00640 1 Processor -- start 2024-02-02T13:24:43.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.375+0000 7f9c64e00640 1 -- start start 2024-02-02T13:24:43.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c600b2260 0x7f9c601f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:43.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600b8a50 0x7f9c6015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:43.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:43.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600a9e50 con 0x7f9c600d0070 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c601f2bd0 con 0x7f9c600b8a50 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c64e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6015ede0 con 0x7f9c600b2260 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c5e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c5e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53166/0 (socket says 172.21.15.91:53166) 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.376+0000 7f9c5e800640 1 -- 172.21.15.91:0/3709771911 learned_addr learned my addr 172.21.15.91:0/3709771911 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5d400640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600b8a50 0x7f9c6015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5de00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c600b2260 0x7f9c601f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 -- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c600b2260 msgr2=0x7f9c601f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c600b2260 0x7f9c601f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 -- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600b8a50 msgr2=0x7f9c6015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600b8a50 0x7f9c6015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 -- 172.21.15.91:0/3709771911 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58009b30 con 0x7f9c600d0070 2024-02-02T13:24:43.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c5e800640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c58036040 tx=0x7f9c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:43.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58009510 con 0x7f9c600d0070 2024-02-02T13:24:43.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5803e450 con 0x7f9c600d0070 2024-02-02T13:24:43.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6015f0c0 con 0x7f9c600d0070 2024-02-02T13:24:43.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.377+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5803e800 con 0x7f9c600d0070 2024-02-02T13:24:43.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.378+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c600a9a70 con 0x7f9c600d0070 2024-02-02T13:24:43.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.379+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c60066cf0 con 0x7f9c600d0070 2024-02-02T13:24:43.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.380+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c5803e960 con 0x7f9c600d0070 2024-02-02T13:24:43.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.380+0000 7f9c56000640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 0x7f9c3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:43.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.381+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c580ab680 con 0x7f9c600d0070 2024-02-02T13:24:43.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.381+0000 7f9c5de00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 0x7f9c3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:43.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.381+0000 7f9c5de00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 0x7f9c3806c8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f60 tx=0x7f9c50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:43.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.385+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c58076d00 con 0x7f9c600d0070 2024-02-02T13:24:43.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.569+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c600b8ea0 con 0x7f9c3806a430 2024-02-02T13:24:43.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.577+0000 7f9c56000640 1 -- 172.21.15.91:0/3709771911 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260054 (secure 0 0 0) 0x7f9c500050d0 con 0x7f9c3806a430 2024-02-02T13:24:43.583 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 msgr2=0x7f9c3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 0x7f9c3806c8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f60 tx=0x7f9c50009040 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 msgr2=0x7f9c601f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c58036040 tx=0x7f9c58004a40 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 shutdown_connections 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3806a430 0x7f9c3806c8f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c600d0070 0x7f9c601f53a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600b8a50 0x7f9c6015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 --2- 172.21.15.91:0/3709771911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c600b2260 0x7f9c601f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.583+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 >> 172.21.15.91:0/3709771911 conn(0x7f9c60070660 msgr2=0x7f9c60074010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.584+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 shutdown_connections 2024-02-02T13:24:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:43.584+0000 7f9c64e00640 1 -- 172.21.15.91:0/3709771911 wait complete. 2024-02-02T13:24:43.603 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-02-02T13:24:42.236833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560812,"statfs":{"total":1600000000,"available":1598271612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002030"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978392+0000","last_change":"2024-02-02T13:24:36.767566+0000","last_active":"2024-02-02T13:24:40.978392+0000","last_peered":"2024-02-02T13:24:40.978392+0000","last_clean":"2024-02-02T13:24:40.978392+0000","last_became_active":"2024-02-02T13:24:36.767342+0000","last_became_peered":"2024-02-02T13:24:36.767342+0000","last_unstale":"2024-02-02T13:24:40.978392+0000","last_undegraded":"2024-02-02T13:24:40.978392+0000","last_fullsized":"2024-02-02T13:24:40.978392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876609+0000","last_change":"2024-02-02T13:24:36.753974+0000","last_active":"2024-02-02T13:24:39.876609+0000","last_peered":"2024-02-02T13:24:39.876609+0000","last_clean":"2024-02-02T13:24:39.876609+0000","last_became_active":"2024-02-02T13:24:36.753603+0000","last_became_peered":"2024-02-02T13:24:36.753603+0000","last_unstale":"2024-02-02T13:24:39.876609+0000","last_undegraded":"2024-02-02T13:24:39.876609+0000","last_fullsized":"2024-02-02T13:24:39.876609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092881+0000","last_change":"2024-02-02T13:24:36.761233+0000","last_active":"2024-02-02T13:24:41.092881+0000","last_peered":"2024-02-02T13:24:41.092881+0000","last_clean":"2024-02-02T13:24:41.092881+0000","last_became_active":"2024-02-02T13:24:36.761020+0000","last_became_peered":"2024-02-02T13:24:36.761020+0000","last_unstale":"2024-02-02T13:24:41.092881+0000","last_undegraded":"2024-02-02T13:24:41.092881+0000","last_fullsized":"2024-02-02T13:24:41.092881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773368+0000","last_change":"2024-02-02T13:24:36.772920+0000","last_active":"2024-02-02T13:24:36.773368+0000","last_peered":"2024-02-02T13:24:36.773368+0000","last_clean":"2024-02-02T13:24:36.773368+0000","last_became_active":"2024-02-02T13:24:36.765145+0000","last_became_peered":"2024-02-02T13:24:36.765145+0000","last_unstale":"2024-02-02T13:24:36.773368+0000","last_undegraded":"2024-02-02T13:24:36.773368+0000","last_fullsized":"2024-02-02T13:24:36.773368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978323+0000","last_change":"2024-02-02T13:24:36.759645+0000","last_active":"2024-02-02T13:24:40.978323+0000","last_peered":"2024-02-02T13:24:40.978323+0000","last_clean":"2024-02-02T13:24:40.978323+0000","last_became_active":"2024-02-02T13:24:36.759399+0000","last_became_peered":"2024-02-02T13:24:36.759399+0000","last_unstale":"2024-02-02T13:24:40.978323+0000","last_undegraded":"2024-02-02T13:24:40.978323+0000","last_fullsized":"2024-02-02T13:24:40.978323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:52.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777056+0000","last_change":"2024-02-02T13:24:36.774617+0000","last_active":"2024-02-02T13:24:36.777056+0000","last_peered":"2024-02-02T13:24:36.777056+0000","last_clean":"2024-02-02T13:24:36.777056+0000","last_became_active":"2024-02-02T13:24:36.772658+0000","last_became_peered":"2024-02-02T13:24:36.772658+0000","last_unstale":"2024-02-02T13:24:36.777056+0000","last_undegraded":"2024-02-02T13:24:36.777056+0000","last_fullsized":"2024-02-02T13:24:36.777056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978756+0000","last_change":"2024-02-02T13:24:36.755776+0000","last_active":"2024-02-02T13:24:40.978756+0000","last_peered":"2024-02-02T13:24:40.978756+0000","last_clean":"2024-02-02T13:24:40.978756+0000","last_became_active":"2024-02-02T13:24:36.755523+0000","last_became_peered":"2024-02-02T13:24:36.755523+0000","last_unstale":"2024-02-02T13:24:40.978756+0000","last_undegraded":"2024-02-02T13:24:40.978756+0000","last_fullsized":"2024-02-02T13:24:40.978756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211535+0000","last_change":"2024-02-02T13:24:36.759777+0000","last_active":"2024-02-02T13:24:41.211535+0000","last_peered":"2024-02-02T13:24:41.211535+0000","last_clean":"2024-02-02T13:24:41.211535+0000","last_became_active":"2024-02-02T13:24:36.759170+0000","last_became_peered":"2024-02-02T13:24:36.759170+0000","last_unstale":"2024-02-02T13:24:41.211535+0000","last_undegraded":"2024-02-02T13:24:41.211535+0000","last_fullsized":"2024-02-02T13:24:41.211535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978056+0000","last_change":"2024-02-02T13:24:36.757480+0000","last_active":"2024-02-02T13:24:40.978056+0000","last_peered":"2024-02-02T13:24:40.978056+0000","last_clean":"2024-02-02T13:24:40.978056+0000","last_became_active":"2024-02-02T13:24:36.757254+0000","last_became_peered":"2024-02-02T13:24:36.757254+0000","last_unstale":"2024-02-02T13:24:40.978056+0000","last_undegraded":"2024-02-02T13:24:40.978056+0000","last_fullsized":"2024-02-02T13:24:40.978056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:41.311042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775838+0000","last_change":"2024-02-02T13:24:36.775765+0000","last_active":"2024-02-02T13:24:36.775838+0000","last_peered":"2024-02-02T13:24:36.775838+0000","last_clean":"2024-02-02T13:24:36.775838+0000","last_became_active":"2024-02-02T13:24:36.775523+0000","last_became_peered":"2024-02-02T13:24:36.775523+0000","last_unstale":"2024-02-02T13:24:36.775838+0000","last_undegraded":"2024-02-02T13:24:36.775838+0000","last_fullsized":"2024-02-02T13:24:36.775838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778319+0000","last_change":"2024-02-02T13:24:36.778240+0000","last_active":"2024-02-02T13:24:36.778319+0000","last_peered":"2024-02-02T13:24:36.778319+0000","last_clean":"2024-02-02T13:24:36.778319+0000","last_became_active":"2024-02-02T13:24:36.778019+0000","last_became_peered":"2024-02-02T13:24:36.778019+0000","last_unstale":"2024-02-02T13:24:36.778319+0000","last_undegraded":"2024-02-02T13:24:36.778319+0000","last_fullsized":"2024-02-02T13:24:36.778319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753463+0000","last_change":"2024-02-02T13:24:36.766203+0000","last_active":"2024-02-02T13:24:37.753463+0000","last_peered":"2024-02-02T13:24:37.753463+0000","last_clean":"2024-02-02T13:24:37.753463+0000","last_became_active":"2024-02-02T13:24:36.765708+0000","last_became_peered":"2024-02-02T13:24:36.765708+0000","last_unstale":"2024-02-02T13:24:37.753463+0000","last_undegraded":"2024-02-02T13:24:37.753463+0000","last_fullsized":"2024-02-02T13:24:37.753463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211785+0000","last_change":"2024-02-02T13:24:36.768024+0000","last_active":"2024-02-02T13:24:41.211785+0000","last_peered":"2024-02-02T13:24:41.211785+0000","last_clean":"2024-02-02T13:24:41.211785+0000","last_became_active":"2024-02-02T13:24:36.767819+0000","last_became_peered":"2024-02-02T13:24:36.767819+0000","last_unstale":"2024-02-02T13:24:41.211785+0000","last_undegraded":"2024-02-02T13:24:41.211785+0000","last_fullsized":"2024-02-02T13:24:41.211785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093008+0000","last_change":"2024-02-02T13:24:36.755455+0000","last_active":"2024-02-02T13:24:41.093008+0000","last_peered":"2024-02-02T13:24:41.093008+0000","last_clean":"2024-02-02T13:24:41.093008+0000","last_became_active":"2024-02-02T13:24:36.755245+0000","last_became_peered":"2024-02-02T13:24:36.755245+0000","last_unstale":"2024-02-02T13:24:41.093008+0000","last_undegraded":"2024-02-02T13:24:41.093008+0000","last_fullsized":"2024-02-02T13:24:41.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876981+0000","last_change":"2024-02-02T13:24:36.757288+0000","last_active":"2024-02-02T13:24:39.876981+0000","last_peered":"2024-02-02T13:24:39.876981+0000","last_clean":"2024-02-02T13:24:39.876981+0000","last_became_active":"2024-02-02T13:24:36.756687+0000","last_became_peered":"2024-02-02T13:24:36.756687+0000","last_unstale":"2024-02-02T13:24:39.876981+0000","last_undegraded":"2024-02-02T13:24:39.876981+0000","last_fullsized":"2024-02-02T13:24:39.876981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777000+0000","last_change":"2024-02-02T13:24:36.774416+0000","last_active":"2024-02-02T13:24:36.777000+0000","last_peered":"2024-02-02T13:24:36.777000+0000","last_clean":"2024-02-02T13:24:36.777000+0000","last_became_active":"2024-02-02T13:24:36.771102+0000","last_became_peered":"2024-02-02T13:24:36.771102+0000","last_unstale":"2024-02-02T13:24:36.777000+0000","last_undegraded":"2024-02-02T13:24:36.777000+0000","last_fullsized":"2024-02-02T13:24:36.777000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761526+0000","last_change":"2024-02-02T13:24:36.761450+0000","last_active":"2024-02-02T13:24:36.761526+0000","last_peered":"2024-02-02T13:24:36.761526+0000","last_clean":"2024-02-02T13:24:36.761526+0000","last_became_active":"2024-02-02T13:24:36.760875+0000","last_became_peered":"2024-02-02T13:24:36.760875+0000","last_unstale":"2024-02-02T13:24:36.761526+0000","last_undegraded":"2024-02-02T13:24:36.761526+0000","last_fullsized":"2024-02-02T13:24:36.761526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771249+0000","last_change":"2024-02-02T13:24:36.768404+0000","last_active":"2024-02-02T13:24:36.771249+0000","last_peered":"2024-02-02T13:24:36.771249+0000","last_clean":"2024-02-02T13:24:36.771249+0000","last_became_active":"2024-02-02T13:24:36.768048+0000","last_became_peered":"2024-02-02T13:24:36.768048+0000","last_unstale":"2024-02-02T13:24:36.771249+0000","last_undegraded":"2024-02-02T13:24:36.771249+0000","last_fullsized":"2024-02-02T13:24:36.771249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978685+0000","last_change":"2024-02-02T13:24:36.762446+0000","last_active":"2024-02-02T13:24:40.978685+0000","last_peered":"2024-02-02T13:24:40.978685+0000","last_clean":"2024-02-02T13:24:40.978685+0000","last_became_active":"2024-02-02T13:24:36.762021+0000","last_became_peered":"2024-02-02T13:24:36.762021+0000","last_unstale":"2024-02-02T13:24:40.978685+0000","last_undegraded":"2024-02-02T13:24:40.978685+0000","last_fullsized":"2024-02-02T13:24:40.978685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768525+0000","last_change":"2024-02-02T13:24:36.768424+0000","last_active":"2024-02-02T13:24:36.768525+0000","last_peered":"2024-02-02T13:24:36.768525+0000","last_clean":"2024-02-02T13:24:36.768525+0000","last_became_active":"2024-02-02T13:24:36.768203+0000","last_became_peered":"2024-02-02T13:24:36.768203+0000","last_unstale":"2024-02-02T13:24:36.768525+0000","last_undegraded":"2024-02-02T13:24:36.768525+0000","last_fullsized":"2024-02-02T13:24:36.768525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093333+0000","last_change":"2024-02-02T13:24:36.762245+0000","last_active":"2024-02-02T13:24:41.093333+0000","last_peered":"2024-02-02T13:24:41.093333+0000","last_clean":"2024-02-02T13:24:41.093333+0000","last_became_active":"2024-02-02T13:24:36.762030+0000","last_became_peered":"2024-02-02T13:24:36.762030+0000","last_unstale":"2024-02-02T13:24:41.093333+0000","last_undegraded":"2024-02-02T13:24:41.093333+0000","last_fullsized":"2024-02-02T13:24:41.093333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771962+0000","last_change":"2024-02-02T13:24:36.771888+0000","last_active":"2024-02-02T13:24:36.771962+0000","last_peered":"2024-02-02T13:24:36.771962+0000","last_clean":"2024-02-02T13:24:36.771962+0000","last_became_active":"2024-02-02T13:24:36.770589+0000","last_became_peered":"2024-02-02T13:24:36.770589+0000","last_unstale":"2024-02-02T13:24:36.771962+0000","last_undegraded":"2024-02-02T13:24:36.771962+0000","last_fullsized":"2024-02-02T13:24:36.771962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212147+0000","last_change":"2024-02-02T13:24:36.750480+0000","last_active":"2024-02-02T13:24:41.212147+0000","last_peered":"2024-02-02T13:24:41.212147+0000","last_clean":"2024-02-02T13:24:41.212147+0000","last_became_active":"2024-02-02T13:24:36.750125+0000","last_became_peered":"2024-02-02T13:24:36.750125+0000","last_unstale":"2024-02-02T13:24:41.212147+0000","last_undegraded":"2024-02-02T13:24:41.212147+0000","last_fullsized":"2024-02-02T13:24:41.212147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773388+0000","last_change":"2024-02-02T13:24:36.773077+0000","last_active":"2024-02-02T13:24:36.773388+0000","last_peered":"2024-02-02T13:24:36.773388+0000","last_clean":"2024-02-02T13:24:36.773388+0000","last_became_active":"2024-02-02T13:24:36.765475+0000","last_became_peered":"2024-02-02T13:24:36.765475+0000","last_unstale":"2024-02-02T13:24:36.773388+0000","last_undegraded":"2024-02-02T13:24:36.773388+0000","last_fullsized":"2024-02-02T13:24:36.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211624+0000","last_change":"2024-02-02T13:24:36.750736+0000","last_active":"2024-02-02T13:24:41.211624+0000","last_peered":"2024-02-02T13:24:41.211624+0000","last_clean":"2024-02-02T13:24:41.211624+0000","last_became_active":"2024-02-02T13:24:36.750520+0000","last_became_peered":"2024-02-02T13:24:36.750520+0000","last_unstale":"2024-02-02T13:24:41.211624+0000","last_undegraded":"2024-02-02T13:24:41.211624+0000","last_fullsized":"2024-02-02T13:24:41.211624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764120+0000","last_change":"2024-02-02T13:24:36.763039+0000","last_active":"2024-02-02T13:24:36.764120+0000","last_peered":"2024-02-02T13:24:36.764120+0000","last_clean":"2024-02-02T13:24:36.764120+0000","last_became_active":"2024-02-02T13:24:36.762632+0000","last_became_peered":"2024-02-02T13:24:36.762632+0000","last_unstale":"2024-02-02T13:24:36.764120+0000","last_undegraded":"2024-02-02T13:24:36.764120+0000","last_fullsized":"2024-02-02T13:24:36.764120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877008+0000","last_change":"2024-02-02T13:24:36.750634+0000","last_active":"2024-02-02T13:24:39.877008+0000","last_peered":"2024-02-02T13:24:39.877008+0000","last_clean":"2024-02-02T13:24:39.877008+0000","last_became_active":"2024-02-02T13:24:36.750023+0000","last_became_peered":"2024-02-02T13:24:36.750023+0000","last_unstale":"2024-02-02T13:24:39.877008+0000","last_undegraded":"2024-02-02T13:24:39.877008+0000","last_fullsized":"2024-02-02T13:24:39.877008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211911+0000","last_change":"2024-02-02T13:24:36.750339+0000","last_active":"2024-02-02T13:24:41.211911+0000","last_peered":"2024-02-02T13:24:41.211911+0000","last_clean":"2024-02-02T13:24:41.211911+0000","last_became_active":"2024-02-02T13:24:36.749819+0000","last_became_peered":"2024-02-02T13:24:36.749819+0000","last_unstale":"2024-02-02T13:24:41.211911+0000","last_undegraded":"2024-02-02T13:24:41.211911+0000","last_fullsized":"2024-02-02T13:24:41.211911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093260+0000","last_change":"2024-02-02T13:24:36.765687+0000","last_active":"2024-02-02T13:24:41.093260+0000","last_peered":"2024-02-02T13:24:41.093260+0000","last_clean":"2024-02-02T13:24:41.093260+0000","last_became_active":"2024-02-02T13:24:36.765417+0000","last_became_peered":"2024-02-02T13:24:36.765417+0000","last_unstale":"2024-02-02T13:24:41.093260+0000","last_undegraded":"2024-02-02T13:24:41.093260+0000","last_fullsized":"2024-02-02T13:24:41.093260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762953+0000","last_change":"2024-02-02T13:24:36.762822+0000","last_active":"2024-02-02T13:24:36.762953+0000","last_peered":"2024-02-02T13:24:36.762953+0000","last_clean":"2024-02-02T13:24:36.762953+0000","last_became_active":"2024-02-02T13:24:36.761559+0000","last_became_peered":"2024-02-02T13:24:36.761559+0000","last_unstale":"2024-02-02T13:24:36.762953+0000","last_undegraded":"2024-02-02T13:24:36.762953+0000","last_fullsized":"2024-02-02T13:24:36.762953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764632+0000","last_change":"2024-02-02T13:24:36.764552+0000","last_active":"2024-02-02T13:24:36.764632+0000","last_peered":"2024-02-02T13:24:36.764632+0000","last_clean":"2024-02-02T13:24:36.764632+0000","last_became_active":"2024-02-02T13:24:36.764330+0000","last_became_peered":"2024-02-02T13:24:36.764330+0000","last_unstale":"2024-02-02T13:24:36.764632+0000","last_undegraded":"2024-02-02T13:24:36.764632+0000","last_fullsized":"2024-02-02T13:24:36.764632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.765462+0000","last_change":"2024-02-02T13:24:36.765179+0000","last_active":"2024-02-02T13:24:36.765462+0000","last_peered":"2024-02-02T13:24:36.765462+0000","last_clean":"2024-02-02T13:24:36.765462+0000","last_became_active":"2024-02-02T13:24:36.764850+0000","last_became_peered":"2024-02-02T13:24:36.764850+0000","last_unstale":"2024-02-02T13:24:36.765462+0000","last_undegraded":"2024-02-02T13:24:36.765462+0000","last_fullsized":"2024-02-02T13:24:36.765462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753019+0000","last_change":"2024-02-02T13:24:36.755360+0000","last_active":"2024-02-02T13:24:37.753019+0000","last_peered":"2024-02-02T13:24:37.753019+0000","last_clean":"2024-02-02T13:24:37.753019+0000","last_became_active":"2024-02-02T13:24:36.754987+0000","last_became_peered":"2024-02-02T13:24:36.754987+0000","last_unstale":"2024-02-02T13:24:37.753019+0000","last_undegraded":"2024-02-02T13:24:37.753019+0000","last_fullsized":"2024-02-02T13:24:37.753019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772676+0000","last_change":"2024-02-02T13:24:36.288622+0000","last_active":"2024-02-02T13:24:36.772676+0000","last_peered":"2024-02-02T13:24:36.772676+0000","last_clean":"2024-02-02T13:24:36.772676+0000","last_became_active":"2024-02-02T13:24:36.288400+0000","last_became_peered":"2024-02-02T13:24:36.288400+0000","last_unstale":"2024-02-02T13:24:36.772676+0000","last_undegraded":"2024-02-02T13:24:36.772676+0000","last_fullsized":"2024-02-02T13:24:36.772676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092688+0000","last_change":"2024-02-02T13:24:36.765345+0000","last_active":"2024-02-02T13:24:41.092688+0000","last_peered":"2024-02-02T13:24:41.092688+0000","last_clean":"2024-02-02T13:24:41.092688+0000","last_became_active":"2024-02-02T13:24:36.765122+0000","last_became_peered":"2024-02-02T13:24:36.765122+0000","last_unstale":"2024-02-02T13:24:41.092688+0000","last_undegraded":"2024-02-02T13:24:41.092688+0000","last_fullsized":"2024-02-02T13:24:41.092688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774103+0000","last_change":"2024-02-02T13:24:36.284315+0000","last_active":"2024-02-02T13:24:36.774103+0000","last_peered":"2024-02-02T13:24:36.774103+0000","last_clean":"2024-02-02T13:24:36.774103+0000","last_became_active":"2024-02-02T13:24:36.283193+0000","last_became_peered":"2024-02-02T13:24:36.283193+0000","last_unstale":"2024-02-02T13:24:36.774103+0000","last_undegraded":"2024-02-02T13:24:36.774103+0000","last_fullsized":"2024-02-02T13:24:36.774103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.741346+0000","last_change":"2024-02-02T13:24:36.280943+0000","last_active":"2024-02-02T13:24:36.741346+0000","last_peered":"2024-02-02T13:24:36.741346+0000","last_clean":"2024-02-02T13:24:36.741346+0000","last_became_active":"2024-02-02T13:24:36.280565+0000","last_became_peered":"2024-02-02T13:24:36.280565+0000","last_unstale":"2024-02-02T13:24:36.741346+0000","last_undegraded":"2024-02-02T13:24:36.741346+0000","last_fullsized":"2024-02-02T13:24:36.741346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211701+0000","last_change":"2024-02-02T13:24:36.761849+0000","last_active":"2024-02-02T13:24:41.211701+0000","last_peered":"2024-02-02T13:24:41.211701+0000","last_clean":"2024-02-02T13:24:41.211701+0000","last_became_active":"2024-02-02T13:24:36.761460+0000","last_became_peered":"2024-02-02T13:24:36.761460+0000","last_unstale":"2024-02-02T13:24:41.211701+0000","last_undegraded":"2024-02-02T13:24:41.211701+0000","last_fullsized":"2024-02-02T13:24:41.211701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.753498+0000","last_change":"2024-02-02T13:24:36.286417+0000","last_active":"2024-02-02T13:24:36.753498+0000","last_peered":"2024-02-02T13:24:36.753498+0000","last_clean":"2024-02-02T13:24:36.753498+0000","last_became_active":"2024-02-02T13:24:36.286107+0000","last_became_peered":"2024-02-02T13:24:36.286107+0000","last_unstale":"2024-02-02T13:24:36.753498+0000","last_undegraded":"2024-02-02T13:24:36.753498+0000","last_fullsized":"2024-02-02T13:24:36.753498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876738+0000","last_change":"2024-02-02T13:24:36.750157+0000","last_active":"2024-02-02T13:24:39.876738+0000","last_peered":"2024-02-02T13:24:39.876738+0000","last_clean":"2024-02-02T13:24:39.876738+0000","last_became_active":"2024-02-02T13:24:36.749764+0000","last_became_peered":"2024-02-02T13:24:36.749764+0000","last_unstale":"2024-02-02T13:24:39.876738+0000","last_undegraded":"2024-02-02T13:24:39.876738+0000","last_fullsized":"2024-02-02T13:24:39.876738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762177+0000","last_change":"2024-02-02T13:24:36.290816+0000","last_active":"2024-02-02T13:24:36.762177+0000","last_peered":"2024-02-02T13:24:36.762177+0000","last_clean":"2024-02-02T13:24:36.762177+0000","last_became_active":"2024-02-02T13:24:36.290592+0000","last_became_peered":"2024-02-02T13:24:36.290592+0000","last_unstale":"2024-02-02T13:24:36.762177+0000","last_undegraded":"2024-02-02T13:24:36.762177+0000","last_fullsized":"2024-02-02T13:24:36.762177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761656+0000","last_change":"2024-02-02T13:24:36.761589+0000","last_active":"2024-02-02T13:24:36.761656+0000","last_peered":"2024-02-02T13:24:36.761656+0000","last_clean":"2024-02-02T13:24:36.761656+0000","last_became_active":"2024-02-02T13:24:36.761215+0000","last_became_peered":"2024-02-02T13:24:36.761215+0000","last_unstale":"2024-02-02T13:24:36.761656+0000","last_undegraded":"2024-02-02T13:24:36.761656+0000","last_fullsized":"2024-02-02T13:24:36.761656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.758412+0000","last_change":"2024-02-02T13:24:36.281944+0000","last_active":"2024-02-02T13:24:36.758412+0000","last_peered":"2024-02-02T13:24:36.758412+0000","last_clean":"2024-02-02T13:24:36.758412+0000","last_became_active":"2024-02-02T13:24:36.281546+0000","last_became_peered":"2024-02-02T13:24:36.281546+0000","last_unstale":"2024-02-02T13:24:36.758412+0000","last_undegraded":"2024-02-02T13:24:36.758412+0000","last_fullsized":"2024-02-02T13:24:36.758412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093550+0000","last_change":"2024-02-02T13:24:36.775310+0000","last_active":"2024-02-02T13:24:41.093550+0000","last_peered":"2024-02-02T13:24:41.093550+0000","last_clean":"2024-02-02T13:24:41.093550+0000","last_became_active":"2024-02-02T13:24:36.775021+0000","last_became_peered":"2024-02-02T13:24:36.775021+0000","last_unstale":"2024-02-02T13:24:41.093550+0000","last_undegraded":"2024-02-02T13:24:41.093550+0000","last_fullsized":"2024-02-02T13:24:41.093550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777366+0000","last_change":"2024-02-02T13:24:36.267935+0000","last_active":"2024-02-02T13:24:36.777366+0000","last_peered":"2024-02-02T13:24:36.777366+0000","last_clean":"2024-02-02T13:24:36.777366+0000","last_became_active":"2024-02-02T13:24:36.267120+0000","last_became_peered":"2024-02-02T13:24:36.267120+0000","last_unstale":"2024-02-02T13:24:36.777366+0000","last_undegraded":"2024-02-02T13:24:36.777366+0000","last_fullsized":"2024-02-02T13:24:36.777366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753018+0000","last_change":"2024-02-02T13:24:36.758353+0000","last_active":"2024-02-02T13:24:37.753018+0000","last_peered":"2024-02-02T13:24:37.753018+0000","last_clean":"2024-02-02T13:24:37.753018+0000","last_became_active":"2024-02-02T13:24:36.758074+0000","last_became_peered":"2024-02-02T13:24:36.758074+0000","last_unstale":"2024-02-02T13:24:37.753018+0000","last_undegraded":"2024-02-02T13:24:37.753018+0000","last_fullsized":"2024-02-02T13:24:37.753018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.776688+0000","last_change":"2024-02-02T13:24:36.283779+0000","last_active":"2024-02-02T13:24:36.776688+0000","last_peered":"2024-02-02T13:24:36.776688+0000","last_clean":"2024-02-02T13:24:36.776688+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.776688+0000","last_undegraded":"2024-02-02T13:24:36.776688+0000","last_fullsized":"2024-02-02T13:24:36.776688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978240+0000","last_change":"2024-02-02T13:24:36.751477+0000","last_active":"2024-02-02T13:24:40.978240+0000","last_peered":"2024-02-02T13:24:40.978240+0000","last_clean":"2024-02-02T13:24:40.978240+0000","last_became_active":"2024-02-02T13:24:36.751226+0000","last_became_peered":"2024-02-02T13:24:36.751226+0000","last_unstale":"2024-02-02T13:24:40.978240+0000","last_undegraded":"2024-02-02T13:24:40.978240+0000","last_fullsized":"2024-02-02T13:24:40.978240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773129+0000","last_change":"2024-02-02T13:24:36.286732+0000","last_active":"2024-02-02T13:24:36.773129+0000","last_peered":"2024-02-02T13:24:36.773129+0000","last_clean":"2024-02-02T13:24:36.773129+0000","last_became_active":"2024-02-02T13:24:36.286351+0000","last_became_peered":"2024-02-02T13:24:36.286351+0000","last_unstale":"2024-02-02T13:24:36.773129+0000","last_undegraded":"2024-02-02T13:24:36.773129+0000","last_fullsized":"2024-02-02T13:24:36.773129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093755+0000","last_change":"2024-02-02T13:24:36.756739+0000","last_active":"2024-02-02T13:24:41.093755+0000","last_peered":"2024-02-02T13:24:41.093755+0000","last_clean":"2024-02-02T13:24:41.093755+0000","last_became_active":"2024-02-02T13:24:36.756537+0000","last_became_peered":"2024-02-02T13:24:36.756537+0000","last_unstale":"2024-02-02T13:24:41.093755+0000","last_undegraded":"2024-02-02T13:24:41.093755+0000","last_fullsized":"2024-02-02T13:24:41.093755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755625+0000","last_change":"2024-02-02T13:24:36.286698+0000","last_active":"2024-02-02T13:24:36.755625+0000","last_peered":"2024-02-02T13:24:36.755625+0000","last_clean":"2024-02-02T13:24:36.755625+0000","last_became_active":"2024-02-02T13:24:36.282637+0000","last_became_peered":"2024-02-02T13:24:36.282637+0000","last_unstale":"2024-02-02T13:24:36.755625+0000","last_undegraded":"2024-02-02T13:24:36.755625+0000","last_fullsized":"2024-02-02T13:24:36.755625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978581+0000","last_change":"2024-02-02T13:24:36.766477+0000","last_active":"2024-02-02T13:24:40.978581+0000","last_peered":"2024-02-02T13:24:40.978581+0000","last_clean":"2024-02-02T13:24:40.978581+0000","last_became_active":"2024-02-02T13:24:36.765889+0000","last_became_peered":"2024-02-02T13:24:36.765889+0000","last_unstale":"2024-02-02T13:24:40.978581+0000","last_undegraded":"2024-02-02T13:24:40.978581+0000","last_fullsized":"2024-02-02T13:24:40.978581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764428+0000","last_change":"2024-02-02T13:24:36.286937+0000","last_active":"2024-02-02T13:24:36.764428+0000","last_peered":"2024-02-02T13:24:36.764428+0000","last_clean":"2024-02-02T13:24:36.764428+0000","last_became_active":"2024-02-02T13:24:36.286651+0000","last_became_peered":"2024-02-02T13:24:36.286651+0000","last_unstale":"2024-02-02T13:24:36.764428+0000","last_undegraded":"2024-02-02T13:24:36.764428+0000","last_fullsized":"2024-02-02T13:24:36.764428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.769009+0000","last_change":"2024-02-02T13:24:36.768901+0000","last_active":"2024-02-02T13:24:36.769009+0000","last_peered":"2024-02-02T13:24:36.769009+0000","last_clean":"2024-02-02T13:24:36.769009+0000","last_became_active":"2024-02-02T13:24:36.768565+0000","last_became_peered":"2024-02-02T13:24:36.768565+0000","last_unstale":"2024-02-02T13:24:36.769009+0000","last_undegraded":"2024-02-02T13:24:36.769009+0000","last_fullsized":"2024-02-02T13:24:36.769009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775361+0000","last_change":"2024-02-02T13:24:36.271772+0000","last_active":"2024-02-02T13:24:36.775361+0000","last_peered":"2024-02-02T13:24:36.775361+0000","last_clean":"2024-02-02T13:24:36.775361+0000","last_became_active":"2024-02-02T13:24:36.271163+0000","last_became_peered":"2024-02-02T13:24:36.271163+0000","last_unstale":"2024-02-02T13:24:36.775361+0000","last_undegraded":"2024-02-02T13:24:36.775361+0000","last_fullsized":"2024-02-02T13:24:36.775361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.979190+0000","last_change":"2024-02-02T13:24:36.756837+0000","last_active":"2024-02-02T13:24:40.979190+0000","last_peered":"2024-02-02T13:24:40.979190+0000","last_clean":"2024-02-02T13:24:40.979190+0000","last_became_active":"2024-02-02T13:24:36.756508+0000","last_became_peered":"2024-02-02T13:24:36.756508+0000","last_unstale":"2024-02-02T13:24:40.979190+0000","last_undegraded":"2024-02-02T13:24:40.979190+0000","last_fullsized":"2024-02-02T13:24:40.979190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771720+0000","last_change":"2024-02-02T13:24:36.287213+0000","last_active":"2024-02-02T13:24:36.771720+0000","last_peered":"2024-02-02T13:24:36.771720+0000","last_clean":"2024-02-02T13:24:36.771720+0000","last_became_active":"2024-02-02T13:24:36.286967+0000","last_became_peered":"2024-02-02T13:24:36.286967+0000","last_unstale":"2024-02-02T13:24:36.771720+0000","last_undegraded":"2024-02-02T13:24:36.771720+0000","last_fullsized":"2024-02-02T13:24:36.771720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978811+0000","last_change":"2024-02-02T13:24:36.766631+0000","last_active":"2024-02-02T13:24:40.978811+0000","last_peered":"2024-02-02T13:24:40.978811+0000","last_clean":"2024-02-02T13:24:40.978811+0000","last_became_active":"2024-02-02T13:24:36.766216+0000","last_became_peered":"2024-02-02T13:24:36.766216+0000","last_unstale":"2024-02-02T13:24:40.978811+0000","last_undegraded":"2024-02-02T13:24:40.978811+0000","last_fullsized":"2024-02-02T13:24:40.978811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772131+0000","last_change":"2024-02-02T13:24:36.275397+0000","last_active":"2024-02-02T13:24:36.772131+0000","last_peered":"2024-02-02T13:24:36.772131+0000","last_clean":"2024-02-02T13:24:36.772131+0000","last_became_active":"2024-02-02T13:24:36.274927+0000","last_became_peered":"2024-02-02T13:24:36.274927+0000","last_unstale":"2024-02-02T13:24:36.772131+0000","last_undegraded":"2024-02-02T13:24:36.772131+0000","last_fullsized":"2024-02-02T13:24:36.772131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978419+0000","last_change":"2024-02-02T13:24:36.757035+0000","last_active":"2024-02-02T13:24:40.978419+0000","last_peered":"2024-02-02T13:24:40.978419+0000","last_clean":"2024-02-02T13:24:40.978419+0000","last_became_active":"2024-02-02T13:24:36.756748+0000","last_became_peered":"2024-02-02T13:24:36.756748+0000","last_unstale":"2024-02-02T13:24:40.978419+0000","last_undegraded":"2024-02-02T13:24:40.978419+0000","last_fullsized":"2024-02-02T13:24:40.978419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:24:57.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768414+0000","last_change":"2024-02-02T13:24:36.283044+0000","last_active":"2024-02-02T13:24:36.768414+0000","last_peered":"2024-02-02T13:24:36.768414+0000","last_clean":"2024-02-02T13:24:36.768414+0000","last_became_active":"2024-02-02T13:24:36.282798+0000","last_became_peered":"2024-02-02T13:24:36.282798+0000","last_unstale":"2024-02-02T13:24:36.768414+0000","last_undegraded":"2024-02-02T13:24:36.768414+0000","last_fullsized":"2024-02-02T13:24:36.768414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770333+0000","last_change":"2024-02-02T13:24:36.764197+0000","last_active":"2024-02-02T13:24:36.770333+0000","last_peered":"2024-02-02T13:24:36.770333+0000","last_clean":"2024-02-02T13:24:36.770333+0000","last_became_active":"2024-02-02T13:24:36.763970+0000","last_became_peered":"2024-02-02T13:24:36.763970+0000","last_unstale":"2024-02-02T13:24:36.770333+0000","last_undegraded":"2024-02-02T13:24:36.770333+0000","last_fullsized":"2024-02-02T13:24:36.770333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.757774+0000","last_change":"2024-02-02T13:24:36.279681+0000","last_active":"2024-02-02T13:24:36.757774+0000","last_peered":"2024-02-02T13:24:36.757774+0000","last_clean":"2024-02-02T13:24:36.757774+0000","last_became_active":"2024-02-02T13:24:36.279342+0000","last_became_peered":"2024-02-02T13:24:36.279342+0000","last_unstale":"2024-02-02T13:24:36.757774+0000","last_undegraded":"2024-02-02T13:24:36.757774+0000","last_fullsized":"2024-02-02T13:24:36.757774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978828+0000","last_change":"2024-02-02T13:24:36.758113+0000","last_active":"2024-02-02T13:24:40.978828+0000","last_peered":"2024-02-02T13:24:40.978828+0000","last_clean":"2024-02-02T13:24:40.978828+0000","last_became_active":"2024-02-02T13:24:36.757859+0000","last_became_peered":"2024-02-02T13:24:36.757859+0000","last_unstale":"2024-02-02T13:24:40.978828+0000","last_undegraded":"2024-02-02T13:24:40.978828+0000","last_fullsized":"2024-02-02T13:24:40.978828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748840+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:36.748840+0000","last_peered":"2024-02-02T13:24:36.748840+0000","last_clean":"2024-02-02T13:24:36.748840+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:36.748840+0000","last_undegraded":"2024-02-02T13:24:36.748840+0000","last_fullsized":"2024-02-02T13:24:36.748840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755725+0000","last_change":"2024-02-02T13:24:36.285102+0000","last_active":"2024-02-02T13:24:36.755725+0000","last_peered":"2024-02-02T13:24:36.755725+0000","last_clean":"2024-02-02T13:24:36.755725+0000","last_became_active":"2024-02-02T13:24:36.280044+0000","last_became_peered":"2024-02-02T13:24:36.280044+0000","last_unstale":"2024-02-02T13:24:36.755725+0000","last_undegraded":"2024-02-02T13:24:36.755725+0000","last_fullsized":"2024-02-02T13:24:36.755725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211798+0000","last_change":"2024-02-02T13:24:36.762214+0000","last_active":"2024-02-02T13:24:41.211798+0000","last_peered":"2024-02-02T13:24:41.211798+0000","last_clean":"2024-02-02T13:24:41.211798+0000","last_became_active":"2024-02-02T13:24:36.761690+0000","last_became_peered":"2024-02-02T13:24:36.761690+0000","last_unstale":"2024-02-02T13:24:41.211798+0000","last_undegraded":"2024-02-02T13:24:41.211798+0000","last_fullsized":"2024-02-02T13:24:41.211798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748517+0000","last_change":"2024-02-02T13:24:36.291720+0000","last_active":"2024-02-02T13:24:36.748517+0000","last_peered":"2024-02-02T13:24:36.748517+0000","last_clean":"2024-02-02T13:24:36.748517+0000","last_became_active":"2024-02-02T13:24:36.290703+0000","last_became_peered":"2024-02-02T13:24:36.290703+0000","last_unstale":"2024-02-02T13:24:36.748517+0000","last_undegraded":"2024-02-02T13:24:36.748517+0000","last_fullsized":"2024-02-02T13:24:36.748517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093549+0000","last_change":"2024-02-02T13:24:36.766220+0000","last_active":"2024-02-02T13:24:41.093549+0000","last_peered":"2024-02-02T13:24:41.093549+0000","last_clean":"2024-02-02T13:24:41.093549+0000","last_became_active":"2024-02-02T13:24:36.765734+0000","last_became_peered":"2024-02-02T13:24:36.765734+0000","last_unstale":"2024-02-02T13:24:41.093549+0000","last_undegraded":"2024-02-02T13:24:41.093549+0000","last_fullsized":"2024-02-02T13:24:41.093549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.756734+0000","last_change":"2024-02-02T13:24:36.276562+0000","last_active":"2024-02-02T13:24:36.756734+0000","last_peered":"2024-02-02T13:24:36.756734+0000","last_clean":"2024-02-02T13:24:36.756734+0000","last_became_active":"2024-02-02T13:24:36.276189+0000","last_became_peered":"2024-02-02T13:24:36.276189+0000","last_unstale":"2024-02-02T13:24:36.756734+0000","last_undegraded":"2024-02-02T13:24:36.756734+0000","last_fullsized":"2024-02-02T13:24:36.756734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877004+0000","last_change":"2024-02-02T13:24:36.761943+0000","last_active":"2024-02-02T13:24:39.877004+0000","last_peered":"2024-02-02T13:24:39.877004+0000","last_clean":"2024-02-02T13:24:39.877004+0000","last_became_active":"2024-02-02T13:24:36.761718+0000","last_became_peered":"2024-02-02T13:24:36.761718+0000","last_unstale":"2024-02-02T13:24:39.877004+0000","last_undegraded":"2024-02-02T13:24:39.877004+0000","last_fullsized":"2024-02-02T13:24:39.877004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.760484+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.760484+0000","last_peered":"2024-02-02T13:24:36.760484+0000","last_clean":"2024-02-02T13:24:36.760484+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.760484+0000","last_undegraded":"2024-02-02T13:24:36.760484+0000","last_fullsized":"2024-02-02T13:24:36.760484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753283+0000","last_change":"2024-02-02T13:24:36.763372+0000","last_active":"2024-02-02T13:24:37.753283+0000","last_peered":"2024-02-02T13:24:37.753283+0000","last_clean":"2024-02-02T13:24:37.753283+0000","last_became_active":"2024-02-02T13:24:36.763132+0000","last_became_peered":"2024-02-02T13:24:36.763132+0000","last_unstale":"2024-02-02T13:24:37.753283+0000","last_undegraded":"2024-02-02T13:24:37.753283+0000","last_fullsized":"2024-02-02T13:24:37.753283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:03.055262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747812+0000","last_change":"2024-02-02T13:24:36.287910+0000","last_active":"2024-02-02T13:24:36.747812+0000","last_peered":"2024-02-02T13:24:36.747812+0000","last_clean":"2024-02-02T13:24:36.747812+0000","last_became_active":"2024-02-02T13:24:36.287683+0000","last_became_peered":"2024-02-02T13:24:36.287683+0000","last_unstale":"2024-02-02T13:24:36.747812+0000","last_undegraded":"2024-02-02T13:24:36.747812+0000","last_fullsized":"2024-02-02T13:24:36.747812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773408+0000","last_change":"2024-02-02T13:24:36.773275+0000","last_active":"2024-02-02T13:24:36.773408+0000","last_peered":"2024-02-02T13:24:36.773408+0000","last_clean":"2024-02-02T13:24:36.773408+0000","last_became_active":"2024-02-02T13:24:36.772646+0000","last_became_peered":"2024-02-02T13:24:36.772646+0000","last_unstale":"2024-02-02T13:24:36.773408+0000","last_undegraded":"2024-02-02T13:24:36.773408+0000","last_fullsized":"2024-02-02T13:24:36.773408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748406+0000","last_change":"2024-02-02T13:24:36.291402+0000","last_active":"2024-02-02T13:24:36.748406+0000","last_peered":"2024-02-02T13:24:36.748406+0000","last_clean":"2024-02-02T13:24:36.748406+0000","last_became_active":"2024-02-02T13:24:36.290274+0000","last_became_peered":"2024-02-02T13:24:36.290274+0000","last_unstale":"2024-02-02T13:24:36.748406+0000","last_undegraded":"2024-02-02T13:24:36.748406+0000","last_fullsized":"2024-02-02T13:24:36.748406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876617+0000","last_change":"2024-02-02T13:24:36.760028+0000","last_active":"2024-02-02T13:24:39.876617+0000","last_peered":"2024-02-02T13:24:39.876617+0000","last_clean":"2024-02-02T13:24:39.876617+0000","last_became_active":"2024-02-02T13:24:36.759641+0000","last_became_peered":"2024-02-02T13:24:36.759641+0000","last_unstale":"2024-02-02T13:24:39.876617+0000","last_undegraded":"2024-02-02T13:24:39.876617+0000","last_fullsized":"2024-02-02T13:24:39.876617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.746080+0000","last_change":"2024-02-02T13:24:36.281619+0000","last_active":"2024-02-02T13:24:36.746080+0000","last_peered":"2024-02-02T13:24:36.746080+0000","last_clean":"2024-02-02T13:24:36.746080+0000","last_became_active":"2024-02-02T13:24:36.281168+0000","last_became_peered":"2024-02-02T13:24:36.281168+0000","last_unstale":"2024-02-02T13:24:36.746080+0000","last_undegraded":"2024-02-02T13:24:36.746080+0000","last_fullsized":"2024-02-02T13:24:36.746080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211709+0000","last_change":"2024-02-02T13:24:36.750882+0000","last_active":"2024-02-02T13:24:41.211709+0000","last_peered":"2024-02-02T13:24:41.211709+0000","last_clean":"2024-02-02T13:24:41.211709+0000","last_became_active":"2024-02-02T13:24:36.750448+0000","last_became_peered":"2024-02-02T13:24:36.750448+0000","last_unstale":"2024-02-02T13:24:41.211709+0000","last_undegraded":"2024-02-02T13:24:41.211709+0000","last_fullsized":"2024-02-02T13:24:41.211709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774907+0000","last_change":"2024-02-02T13:24:36.289169+0000","last_active":"2024-02-02T13:24:36.774907+0000","last_peered":"2024-02-02T13:24:36.774907+0000","last_clean":"2024-02-02T13:24:36.774907+0000","last_became_active":"2024-02-02T13:24:36.288939+0000","last_became_peered":"2024-02-02T13:24:36.288939+0000","last_unstale":"2024-02-02T13:24:36.774907+0000","last_undegraded":"2024-02-02T13:24:36.774907+0000","last_fullsized":"2024-02-02T13:24:36.774907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093133+0000","last_change":"2024-02-02T13:24:36.756653+0000","last_active":"2024-02-02T13:24:41.093133+0000","last_peered":"2024-02-02T13:24:41.093133+0000","last_clean":"2024-02-02T13:24:41.093133+0000","last_became_active":"2024-02-02T13:24:36.756419+0000","last_became_peered":"2024-02-02T13:24:36.756419+0000","last_unstale":"2024-02-02T13:24:41.093133+0000","last_undegraded":"2024-02-02T13:24:41.093133+0000","last_fullsized":"2024-02-02T13:24:41.093133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.751195+0000","last_change":"2024-02-02T13:24:36.290423+0000","last_active":"2024-02-02T13:24:36.751195+0000","last_peered":"2024-02-02T13:24:36.751195+0000","last_clean":"2024-02-02T13:24:36.751195+0000","last_became_active":"2024-02-02T13:24:36.289863+0000","last_became_peered":"2024-02-02T13:24:36.289863+0000","last_unstale":"2024-02-02T13:24:36.751195+0000","last_undegraded":"2024-02-02T13:24:36.751195+0000","last_fullsized":"2024-02-02T13:24:36.751195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.754234+0000","last_change":"2024-02-02T13:24:36.759429+0000","last_active":"2024-02-02T13:24:37.754234+0000","last_peered":"2024-02-02T13:24:37.754234+0000","last_clean":"2024-02-02T13:24:37.754234+0000","last_became_active":"2024-02-02T13:24:36.759196+0000","last_became_peered":"2024-02-02T13:24:36.759196+0000","last_unstale":"2024-02-02T13:24:37.754234+0000","last_undegraded":"2024-02-02T13:24:37.754234+0000","last_fullsized":"2024-02-02T13:24:37.754234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747537+0000","last_change":"2024-02-02T13:24:36.288653+0000","last_active":"2024-02-02T13:24:36.747537+0000","last_peered":"2024-02-02T13:24:36.747537+0000","last_clean":"2024-02-02T13:24:36.747537+0000","last_became_active":"2024-02-02T13:24:36.288439+0000","last_became_peered":"2024-02-02T13:24:36.288439+0000","last_unstale":"2024-02-02T13:24:36.747537+0000","last_undegraded":"2024-02-02T13:24:36.747537+0000","last_fullsized":"2024-02-02T13:24:36.747537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211609+0000","last_change":"2024-02-02T13:24:36.751484+0000","last_active":"2024-02-02T13:24:41.211609+0000","last_peered":"2024-02-02T13:24:41.211609+0000","last_clean":"2024-02-02T13:24:41.211609+0000","last_became_active":"2024-02-02T13:24:36.751091+0000","last_became_peered":"2024-02-02T13:24:36.751091+0000","last_unstale":"2024-02-02T13:24:41.211609+0000","last_undegraded":"2024-02-02T13:24:41.211609+0000","last_fullsized":"2024-02-02T13:24:41.211609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747419+0000","last_change":"2024-02-02T13:24:36.280128+0000","last_active":"2024-02-02T13:24:36.747419+0000","last_peered":"2024-02-02T13:24:36.747419+0000","last_clean":"2024-02-02T13:24:36.747419+0000","last_became_active":"2024-02-02T13:24:36.279652+0000","last_became_peered":"2024-02-02T13:24:36.279652+0000","last_unstale":"2024-02-02T13:24:36.747419+0000","last_undegraded":"2024-02-02T13:24:36.747419+0000","last_fullsized":"2024-02-02T13:24:36.747419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212026+0000","last_change":"2024-02-02T13:24:36.763668+0000","last_active":"2024-02-02T13:24:41.212026+0000","last_peered":"2024-02-02T13:24:41.212026+0000","last_clean":"2024-02-02T13:24:41.212026+0000","last_became_active":"2024-02-02T13:24:36.762913+0000","last_became_peered":"2024-02-02T13:24:36.762913+0000","last_unstale":"2024-02-02T13:24:41.212026+0000","last_undegraded":"2024-02-02T13:24:41.212026+0000","last_fullsized":"2024-02-02T13:24:41.212026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774478+0000","last_change":"2024-02-02T13:24:36.289669+0000","last_active":"2024-02-02T13:24:36.774478+0000","last_peered":"2024-02-02T13:24:36.774478+0000","last_clean":"2024-02-02T13:24:36.774478+0000","last_became_active":"2024-02-02T13:24:36.289368+0000","last_became_peered":"2024-02-02T13:24:36.289368+0000","last_unstale":"2024-02-02T13:24:36.774478+0000","last_undegraded":"2024-02-02T13:24:36.774478+0000","last_fullsized":"2024-02-02T13:24:36.774478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770307+0000","last_change":"2024-02-02T13:24:36.769265+0000","last_active":"2024-02-02T13:24:36.770307+0000","last_peered":"2024-02-02T13:24:36.770307+0000","last_clean":"2024-02-02T13:24:36.770307+0000","last_became_active":"2024-02-02T13:24:36.769036+0000","last_became_peered":"2024-02-02T13:24:36.769036+0000","last_unstale":"2024-02-02T13:24:36.770307+0000","last_undegraded":"2024-02-02T13:24:36.770307+0000","last_fullsized":"2024-02-02T13:24:36.770307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770353+0000","last_change":"2024-02-02T13:24:36.765383+0000","last_active":"2024-02-02T13:24:36.770353+0000","last_peered":"2024-02-02T13:24:36.770353+0000","last_clean":"2024-02-02T13:24:36.770353+0000","last_became_active":"2024-02-02T13:24:36.765141+0000","last_became_peered":"2024-02-02T13:24:36.765141+0000","last_unstale":"2024-02-02T13:24:36.770353+0000","last_undegraded":"2024-02-02T13:24:36.770353+0000","last_fullsized":"2024-02-02T13:24:36.770353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778010+0000","last_change":"2024-02-02T13:24:36.290626+0000","last_active":"2024-02-02T13:24:36.778010+0000","last_peered":"2024-02-02T13:24:36.778010+0000","last_clean":"2024-02-02T13:24:36.778010+0000","last_became_active":"2024-02-02T13:24:36.290183+0000","last_became_peered":"2024-02-02T13:24:36.290183+0000","last_unstale":"2024-02-02T13:24:36.778010+0000","last_undegraded":"2024-02-02T13:24:36.778010+0000","last_fullsized":"2024-02-02T13:24:36.778010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777961+0000","last_change":"2024-02-02T13:24:36.777865+0000","last_active":"2024-02-02T13:24:36.777961+0000","last_peered":"2024-02-02T13:24:36.777961+0000","last_clean":"2024-02-02T13:24:36.777961+0000","last_became_active":"2024-02-02T13:24:36.777129+0000","last_became_peered":"2024-02-02T13:24:36.777129+0000","last_unstale":"2024-02-02T13:24:36.777961+0000","last_undegraded":"2024-02-02T13:24:36.777961+0000","last_fullsized":"2024-02-02T13:24:36.777961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772162+0000","last_change":"2024-02-02T13:24:36.290121+0000","last_active":"2024-02-02T13:24:36.772162+0000","last_peered":"2024-02-02T13:24:36.772162+0000","last_clean":"2024-02-02T13:24:36.772162+0000","last_became_active":"2024-02-02T13:24:36.289708+0000","last_became_peered":"2024-02-02T13:24:36.289708+0000","last_unstale":"2024-02-02T13:24:36.772162+0000","last_undegraded":"2024-02-02T13:24:36.772162+0000","last_fullsized":"2024-02-02T13:24:36.772162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093334+0000","last_change":"2024-02-02T13:24:36.759606+0000","last_active":"2024-02-02T13:24:41.093334+0000","last_peered":"2024-02-02T13:24:41.093334+0000","last_clean":"2024-02-02T13:24:41.093334+0000","last_became_active":"2024-02-02T13:24:36.759358+0000","last_became_peered":"2024-02-02T13:24:36.759358+0000","last_unstale":"2024-02-02T13:24:41.093334+0000","last_undegraded":"2024-02-02T13:24:41.093334+0000","last_fullsized":"2024-02-02T13:24:41.093334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771605+0000","last_change":"2024-02-02T13:24:36.286535+0000","last_active":"2024-02-02T13:24:36.771605+0000","last_peered":"2024-02-02T13:24:36.771605+0000","last_clean":"2024-02-02T13:24:36.771605+0000","last_became_active":"2024-02-02T13:24:36.286302+0000","last_became_peered":"2024-02-02T13:24:36.286302+0000","last_unstale":"2024-02-02T13:24:36.771605+0000","last_undegraded":"2024-02-02T13:24:36.771605+0000","last_fullsized":"2024-02-02T13:24:36.771605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212029+0000","last_change":"2024-02-02T13:24:36.769601+0000","last_active":"2024-02-02T13:24:41.212029+0000","last_peered":"2024-02-02T13:24:41.212029+0000","last_clean":"2024-02-02T13:24:41.212029+0000","last_became_active":"2024-02-02T13:24:36.768381+0000","last_became_peered":"2024-02-02T13:24:36.768381+0000","last_unstale":"2024-02-02T13:24:41.212029+0000","last_undegraded":"2024-02-02T13:24:41.212029+0000","last_fullsized":"2024-02-02T13:24:41.212029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774155+0000","last_change":"2024-02-02T13:24:36.289368+0000","last_active":"2024-02-02T13:24:36.774155+0000","last_peered":"2024-02-02T13:24:36.774155+0000","last_clean":"2024-02-02T13:24:36.774155+0000","last_became_active":"2024-02-02T13:24:36.289118+0000","last_became_peered":"2024-02-02T13:24:36.289118+0000","last_unstale":"2024-02-02T13:24:36.774155+0000","last_undegraded":"2024-02-02T13:24:36.774155+0000","last_fullsized":"2024-02-02T13:24:36.774155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":640,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194672,"statfs":{"total":200000000,"available":199344612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":63,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195249,"statfs":{"total":200000000,"available":199935017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:46.608 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:46.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.824+0000 7f61de600640 1 Processor -- start 2024-02-02T13:24:46.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.824+0000 7f61de600640 1 -- start start 2024-02-02T13:24:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61de600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 0x7f61d806ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61de600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 0x7f61d80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61de600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d8160e60 con 0x7f61d815d430 2024-02-02T13:24:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61de600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80cbca0 con 0x7f61d80ce4c0 2024-02-02T13:24:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61de600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80cee70 con 0x7f61d815d050 2024-02-02T13:24:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61d7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 0x7f61d80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61d7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 0x7f61d806ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61d7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 0x7f61d80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57162/0 (socket says 172.21.15.91:57162) 2024-02-02T13:24:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.826+0000 7f61d7e00640 1 -- 172.21.15.91:0/3170435570 learned_addr learned my addr 172.21.15.91:0/3170435570 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:46.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.827+0000 7f61d7400640 1 -- 172.21.15.91:0/3170435570 >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 msgr2=0x7f61d80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:46.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.827+0000 7f61d7400640 1 --2- 172.21.15.91:0/3170435570 >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 0x7f61d80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.827+0000 7f61d7400640 1 -- 172.21.15.91:0/3170435570 >> v1:172.21.15.113:6789/0 conn(0x7f61d815d430 legacy=0x7f61d80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:46.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.827+0000 7f61d7400640 1 -- 172.21.15.91:0/3170435570 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d81610b0 con 0x7f61d815d050 2024-02-02T13:24:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.829+0000 7f61d7400640 1 --2- 172.21.15.91:0/3170435570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 0x7f61d806ff30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ac6e7a9e619dfd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.829+0000 7f61d6a00640 1 -- 172.21.15.91:0/3170435570 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8031830 con 0x7f61d815d050 2024-02-02T13:24:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.829+0000 7f61d6a00640 1 -- 172.21.15.91:0/3170435570 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c8031990 con 0x7f61d815d050 2024-02-02T13:24:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.829+0000 7f61d6a00640 1 -- 172.21.15.91:0/3170435570 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8031d20 con 0x7f61d815d050 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 -- 172.21.15.91:0/3170435570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 msgr2=0x7f61d806ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 --2- 172.21.15.91:0/3170435570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 0x7f61d806ff30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 -- 172.21.15.91:0/3170435570 shutdown_connections 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 --2- 172.21.15.91:0/3170435570 >> v2:172.21.15.113:3300/0 conn(0x7f61d80ce4c0 0x7f61d80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 --2- 172.21.15.91:0/3170435570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d815d050 0x7f61d806ff30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 -- 172.21.15.91:0/3170435570 >> 172.21.15.91:0/3170435570 conn(0x7f61d80cbfc0 msgr2=0x7f61d80cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 -- 172.21.15.91:0/3170435570 shutdown_connections 2024-02-02T13:24:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.830+0000 7f61de600640 1 -- 172.21.15.91:0/3170435570 wait complete. 2024-02-02T13:24:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.832+0000 7f61de600640 1 Processor -- start 2024-02-02T13:24:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.832+0000 7f61de600640 1 -- start start 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d80ce4c0 0x7f61d80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d815d050 0x7f61d80c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80c8e70 con 0x7f61d815d430 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d815f4d0 con 0x7f61d815d050 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61de600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80c81f0 con 0x7f61d80ce4c0 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.833+0000 7f61d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53174/0 (socket says 172.21.15.91:53174) 2024-02-02T13:24:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 -- 172.21.15.91:0/3988377890 learned_addr learned my addr 172.21.15.91:0/3988377890 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7400640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d80ce4c0 0x7f61d80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61cfe00640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d815d050 0x7f61d80c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 -- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d80ce4c0 msgr2=0x7f61d80c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d80ce4c0 0x7f61d80c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 -- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d815d050 msgr2=0x7f61d80c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d815d050 0x7f61d80c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 -- 172.21.15.91:0/3988377890 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009400 con 0x7f61d815d430 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61d7e00640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61d000efc0 tx=0x7f61d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.834+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000cd20 con 0x7f61d815d430 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.835+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000f040 con 0x7f61d815d430 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.835+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0003bd0 con 0x7f61d815d430 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.835+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80c84d0 con 0x7f61d815d430 2024-02-02T13:24:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.835+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80c8a90 con 0x7f61d815d430 2024-02-02T13:24:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.836+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d8066980 con 0x7f61d815d430 2024-02-02T13:24:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.837+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f61d0003d30 con 0x7f61d815d430 2024-02-02T13:24:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.838+0000 7f61cf400640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 0x7f61b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.838+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61d0088f40 con 0x7f61d815d430 2024-02-02T13:24:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.838+0000 7f61d7400640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 0x7f61b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:46.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.838+0000 7f61d7400640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 0x7f61b006c8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:46.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:46.842+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f61d00545c0 con 0x7f61d815d430 2024-02-02T13:24:47.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.029+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61d8161670 con 0x7f61b006a430 2024-02-02T13:24:47.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.048+0000 7f61cf400640 1 -- 172.21.15.91:0/3988377890 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260054 (secure 0 0 0) 0x7f61c8045410 con 0x7f61b006a430 2024-02-02T13:24:47.053 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:47.055 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 msgr2=0x7f61b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 0x7f61b006c8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c803c040 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 msgr2=0x7f61d815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61d000efc0 tx=0x7f61d000c490 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 shutdown_connections 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f61b006a430 0x7f61b006c8f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61d815d430 0x7f61d815d920 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d815d050 0x7f61d80c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 --2- 172.21.15.91:0/3988377890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d80ce4c0 0x7f61d80c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 >> 172.21.15.91:0/3988377890 conn(0x7f61d80cbfc0 msgr2=0x7f61d81532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 shutdown_connections 2024-02-02T13:24:47.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.054+0000 7f61de600640 1 -- 172.21.15.91:0/3988377890 wait complete. 2024-02-02T13:24:47.076 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-02-02T13:24:46.238374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560812,"statfs":{"total":1600000000,"available":1598271612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.498488"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978392+0000","last_change":"2024-02-02T13:24:36.767566+0000","last_active":"2024-02-02T13:24:40.978392+0000","last_peered":"2024-02-02T13:24:40.978392+0000","last_clean":"2024-02-02T13:24:40.978392+0000","last_became_active":"2024-02-02T13:24:36.767342+0000","last_became_peered":"2024-02-02T13:24:36.767342+0000","last_unstale":"2024-02-02T13:24:40.978392+0000","last_undegraded":"2024-02-02T13:24:40.978392+0000","last_fullsized":"2024-02-02T13:24:40.978392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876609+0000","last_change":"2024-02-02T13:24:36.753974+0000","last_active":"2024-02-02T13:24:39.876609+0000","last_peered":"2024-02-02T13:24:39.876609+0000","last_clean":"2024-02-02T13:24:39.876609+0000","last_became_active":"2024-02-02T13:24:36.753603+0000","last_became_peered":"2024-02-02T13:24:36.753603+0000","last_unstale":"2024-02-02T13:24:39.876609+0000","last_undegraded":"2024-02-02T13:24:39.876609+0000","last_fullsized":"2024-02-02T13:24:39.876609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092881+0000","last_change":"2024-02-02T13:24:36.761233+0000","last_active":"2024-02-02T13:24:41.092881+0000","last_peered":"2024-02-02T13:24:41.092881+0000","last_clean":"2024-02-02T13:24:41.092881+0000","last_became_active":"2024-02-02T13:24:36.761020+0000","last_became_peered":"2024-02-02T13:24:36.761020+0000","last_unstale":"2024-02-02T13:24:41.092881+0000","last_undegraded":"2024-02-02T13:24:41.092881+0000","last_fullsized":"2024-02-02T13:24:41.092881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773368+0000","last_change":"2024-02-02T13:24:36.772920+0000","last_active":"2024-02-02T13:24:36.773368+0000","last_peered":"2024-02-02T13:24:36.773368+0000","last_clean":"2024-02-02T13:24:36.773368+0000","last_became_active":"2024-02-02T13:24:36.765145+0000","last_became_peered":"2024-02-02T13:24:36.765145+0000","last_unstale":"2024-02-02T13:24:36.773368+0000","last_undegraded":"2024-02-02T13:24:36.773368+0000","last_fullsized":"2024-02-02T13:24:36.773368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978323+0000","last_change":"2024-02-02T13:24:36.759645+0000","last_active":"2024-02-02T13:24:40.978323+0000","last_peered":"2024-02-02T13:24:40.978323+0000","last_clean":"2024-02-02T13:24:40.978323+0000","last_became_active":"2024-02-02T13:24:36.759399+0000","last_became_peered":"2024-02-02T13:24:36.759399+0000","last_unstale":"2024-02-02T13:24:40.978323+0000","last_undegraded":"2024-02-02T13:24:40.978323+0000","last_fullsized":"2024-02-02T13:24:40.978323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:52.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777056+0000","last_change":"2024-02-02T13:24:36.774617+0000","last_active":"2024-02-02T13:24:36.777056+0000","last_peered":"2024-02-02T13:24:36.777056+0000","last_clean":"2024-02-02T13:24:36.777056+0000","last_became_active":"2024-02-02T13:24:36.772658+0000","last_became_peered":"2024-02-02T13:24:36.772658+0000","last_unstale":"2024-02-02T13:24:36.777056+0000","last_undegraded":"2024-02-02T13:24:36.777056+0000","last_fullsized":"2024-02-02T13:24:36.777056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978756+0000","last_change":"2024-02-02T13:24:36.755776+0000","last_active":"2024-02-02T13:24:40.978756+0000","last_peered":"2024-02-02T13:24:40.978756+0000","last_clean":"2024-02-02T13:24:40.978756+0000","last_became_active":"2024-02-02T13:24:36.755523+0000","last_became_peered":"2024-02-02T13:24:36.755523+0000","last_unstale":"2024-02-02T13:24:40.978756+0000","last_undegraded":"2024-02-02T13:24:40.978756+0000","last_fullsized":"2024-02-02T13:24:40.978756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211535+0000","last_change":"2024-02-02T13:24:36.759777+0000","last_active":"2024-02-02T13:24:41.211535+0000","last_peered":"2024-02-02T13:24:41.211535+0000","last_clean":"2024-02-02T13:24:41.211535+0000","last_became_active":"2024-02-02T13:24:36.759170+0000","last_became_peered":"2024-02-02T13:24:36.759170+0000","last_unstale":"2024-02-02T13:24:41.211535+0000","last_undegraded":"2024-02-02T13:24:41.211535+0000","last_fullsized":"2024-02-02T13:24:41.211535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978056+0000","last_change":"2024-02-02T13:24:36.757480+0000","last_active":"2024-02-02T13:24:40.978056+0000","last_peered":"2024-02-02T13:24:40.978056+0000","last_clean":"2024-02-02T13:24:40.978056+0000","last_became_active":"2024-02-02T13:24:36.757254+0000","last_became_peered":"2024-02-02T13:24:36.757254+0000","last_unstale":"2024-02-02T13:24:40.978056+0000","last_undegraded":"2024-02-02T13:24:40.978056+0000","last_fullsized":"2024-02-02T13:24:40.978056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:41.311042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775838+0000","last_change":"2024-02-02T13:24:36.775765+0000","last_active":"2024-02-02T13:24:36.775838+0000","last_peered":"2024-02-02T13:24:36.775838+0000","last_clean":"2024-02-02T13:24:36.775838+0000","last_became_active":"2024-02-02T13:24:36.775523+0000","last_became_peered":"2024-02-02T13:24:36.775523+0000","last_unstale":"2024-02-02T13:24:36.775838+0000","last_undegraded":"2024-02-02T13:24:36.775838+0000","last_fullsized":"2024-02-02T13:24:36.775838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778319+0000","last_change":"2024-02-02T13:24:36.778240+0000","last_active":"2024-02-02T13:24:36.778319+0000","last_peered":"2024-02-02T13:24:36.778319+0000","last_clean":"2024-02-02T13:24:36.778319+0000","last_became_active":"2024-02-02T13:24:36.778019+0000","last_became_peered":"2024-02-02T13:24:36.778019+0000","last_unstale":"2024-02-02T13:24:36.778319+0000","last_undegraded":"2024-02-02T13:24:36.778319+0000","last_fullsized":"2024-02-02T13:24:36.778319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753463+0000","last_change":"2024-02-02T13:24:36.766203+0000","last_active":"2024-02-02T13:24:37.753463+0000","last_peered":"2024-02-02T13:24:37.753463+0000","last_clean":"2024-02-02T13:24:37.753463+0000","last_became_active":"2024-02-02T13:24:36.765708+0000","last_became_peered":"2024-02-02T13:24:36.765708+0000","last_unstale":"2024-02-02T13:24:37.753463+0000","last_undegraded":"2024-02-02T13:24:37.753463+0000","last_fullsized":"2024-02-02T13:24:37.753463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211785+0000","last_change":"2024-02-02T13:24:36.768024+0000","last_active":"2024-02-02T13:24:41.211785+0000","last_peered":"2024-02-02T13:24:41.211785+0000","last_clean":"2024-02-02T13:24:41.211785+0000","last_became_active":"2024-02-02T13:24:36.767819+0000","last_became_peered":"2024-02-02T13:24:36.767819+0000","last_unstale":"2024-02-02T13:24:41.211785+0000","last_undegraded":"2024-02-02T13:24:41.211785+0000","last_fullsized":"2024-02-02T13:24:41.211785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093008+0000","last_change":"2024-02-02T13:24:36.755455+0000","last_active":"2024-02-02T13:24:41.093008+0000","last_peered":"2024-02-02T13:24:41.093008+0000","last_clean":"2024-02-02T13:24:41.093008+0000","last_became_active":"2024-02-02T13:24:36.755245+0000","last_became_peered":"2024-02-02T13:24:36.755245+0000","last_unstale":"2024-02-02T13:24:41.093008+0000","last_undegraded":"2024-02-02T13:24:41.093008+0000","last_fullsized":"2024-02-02T13:24:41.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876981+0000","last_change":"2024-02-02T13:24:36.757288+0000","last_active":"2024-02-02T13:24:39.876981+0000","last_peered":"2024-02-02T13:24:39.876981+0000","last_clean":"2024-02-02T13:24:39.876981+0000","last_became_active":"2024-02-02T13:24:36.756687+0000","last_became_peered":"2024-02-02T13:24:36.756687+0000","last_unstale":"2024-02-02T13:24:39.876981+0000","last_undegraded":"2024-02-02T13:24:39.876981+0000","last_fullsized":"2024-02-02T13:24:39.876981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777000+0000","last_change":"2024-02-02T13:24:36.774416+0000","last_active":"2024-02-02T13:24:36.777000+0000","last_peered":"2024-02-02T13:24:36.777000+0000","last_clean":"2024-02-02T13:24:36.777000+0000","last_became_active":"2024-02-02T13:24:36.771102+0000","last_became_peered":"2024-02-02T13:24:36.771102+0000","last_unstale":"2024-02-02T13:24:36.777000+0000","last_undegraded":"2024-02-02T13:24:36.777000+0000","last_fullsized":"2024-02-02T13:24:36.777000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761526+0000","last_change":"2024-02-02T13:24:36.761450+0000","last_active":"2024-02-02T13:24:36.761526+0000","last_peered":"2024-02-02T13:24:36.761526+0000","last_clean":"2024-02-02T13:24:36.761526+0000","last_became_active":"2024-02-02T13:24:36.760875+0000","last_became_peered":"2024-02-02T13:24:36.760875+0000","last_unstale":"2024-02-02T13:24:36.761526+0000","last_undegraded":"2024-02-02T13:24:36.761526+0000","last_fullsized":"2024-02-02T13:24:36.761526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771249+0000","last_change":"2024-02-02T13:24:36.768404+0000","last_active":"2024-02-02T13:24:36.771249+0000","last_peered":"2024-02-02T13:24:36.771249+0000","last_clean":"2024-02-02T13:24:36.771249+0000","last_became_active":"2024-02-02T13:24:36.768048+0000","last_became_peered":"2024-02-02T13:24:36.768048+0000","last_unstale":"2024-02-02T13:24:36.771249+0000","last_undegraded":"2024-02-02T13:24:36.771249+0000","last_fullsized":"2024-02-02T13:24:36.771249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978685+0000","last_change":"2024-02-02T13:24:36.762446+0000","last_active":"2024-02-02T13:24:40.978685+0000","last_peered":"2024-02-02T13:24:40.978685+0000","last_clean":"2024-02-02T13:24:40.978685+0000","last_became_active":"2024-02-02T13:24:36.762021+0000","last_became_peered":"2024-02-02T13:24:36.762021+0000","last_unstale":"2024-02-02T13:24:40.978685+0000","last_undegraded":"2024-02-02T13:24:40.978685+0000","last_fullsized":"2024-02-02T13:24:40.978685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768525+0000","last_change":"2024-02-02T13:24:36.768424+0000","last_active":"2024-02-02T13:24:36.768525+0000","last_peered":"2024-02-02T13:24:36.768525+0000","last_clean":"2024-02-02T13:24:36.768525+0000","last_became_active":"2024-02-02T13:24:36.768203+0000","last_became_peered":"2024-02-02T13:24:36.768203+0000","last_unstale":"2024-02-02T13:24:36.768525+0000","last_undegraded":"2024-02-02T13:24:36.768525+0000","last_fullsized":"2024-02-02T13:24:36.768525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093333+0000","last_change":"2024-02-02T13:24:36.762245+0000","last_active":"2024-02-02T13:24:41.093333+0000","last_peered":"2024-02-02T13:24:41.093333+0000","last_clean":"2024-02-02T13:24:41.093333+0000","last_became_active":"2024-02-02T13:24:36.762030+0000","last_became_peered":"2024-02-02T13:24:36.762030+0000","last_unstale":"2024-02-02T13:24:41.093333+0000","last_undegraded":"2024-02-02T13:24:41.093333+0000","last_fullsized":"2024-02-02T13:24:41.093333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771962+0000","last_change":"2024-02-02T13:24:36.771888+0000","last_active":"2024-02-02T13:24:36.771962+0000","last_peered":"2024-02-02T13:24:36.771962+0000","last_clean":"2024-02-02T13:24:36.771962+0000","last_became_active":"2024-02-02T13:24:36.770589+0000","last_became_peered":"2024-02-02T13:24:36.770589+0000","last_unstale":"2024-02-02T13:24:36.771962+0000","last_undegraded":"2024-02-02T13:24:36.771962+0000","last_fullsized":"2024-02-02T13:24:36.771962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212147+0000","last_change":"2024-02-02T13:24:36.750480+0000","last_active":"2024-02-02T13:24:41.212147+0000","last_peered":"2024-02-02T13:24:41.212147+0000","last_clean":"2024-02-02T13:24:41.212147+0000","last_became_active":"2024-02-02T13:24:36.750125+0000","last_became_peered":"2024-02-02T13:24:36.750125+0000","last_unstale":"2024-02-02T13:24:41.212147+0000","last_undegraded":"2024-02-02T13:24:41.212147+0000","last_fullsized":"2024-02-02T13:24:41.212147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773388+0000","last_change":"2024-02-02T13:24:36.773077+0000","last_active":"2024-02-02T13:24:36.773388+0000","last_peered":"2024-02-02T13:24:36.773388+0000","last_clean":"2024-02-02T13:24:36.773388+0000","last_became_active":"2024-02-02T13:24:36.765475+0000","last_became_peered":"2024-02-02T13:24:36.765475+0000","last_unstale":"2024-02-02T13:24:36.773388+0000","last_undegraded":"2024-02-02T13:24:36.773388+0000","last_fullsized":"2024-02-02T13:24:36.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211624+0000","last_change":"2024-02-02T13:24:36.750736+0000","last_active":"2024-02-02T13:24:41.211624+0000","last_peered":"2024-02-02T13:24:41.211624+0000","last_clean":"2024-02-02T13:24:41.211624+0000","last_became_active":"2024-02-02T13:24:36.750520+0000","last_became_peered":"2024-02-02T13:24:36.750520+0000","last_unstale":"2024-02-02T13:24:41.211624+0000","last_undegraded":"2024-02-02T13:24:41.211624+0000","last_fullsized":"2024-02-02T13:24:41.211624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764120+0000","last_change":"2024-02-02T13:24:36.763039+0000","last_active":"2024-02-02T13:24:36.764120+0000","last_peered":"2024-02-02T13:24:36.764120+0000","last_clean":"2024-02-02T13:24:36.764120+0000","last_became_active":"2024-02-02T13:24:36.762632+0000","last_became_peered":"2024-02-02T13:24:36.762632+0000","last_unstale":"2024-02-02T13:24:36.764120+0000","last_undegraded":"2024-02-02T13:24:36.764120+0000","last_fullsized":"2024-02-02T13:24:36.764120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877008+0000","last_change":"2024-02-02T13:24:36.750634+0000","last_active":"2024-02-02T13:24:39.877008+0000","last_peered":"2024-02-02T13:24:39.877008+0000","last_clean":"2024-02-02T13:24:39.877008+0000","last_became_active":"2024-02-02T13:24:36.750023+0000","last_became_peered":"2024-02-02T13:24:36.750023+0000","last_unstale":"2024-02-02T13:24:39.877008+0000","last_undegraded":"2024-02-02T13:24:39.877008+0000","last_fullsized":"2024-02-02T13:24:39.877008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211911+0000","last_change":"2024-02-02T13:24:36.750339+0000","last_active":"2024-02-02T13:24:41.211911+0000","last_peered":"2024-02-02T13:24:41.211911+0000","last_clean":"2024-02-02T13:24:41.211911+0000","last_became_active":"2024-02-02T13:24:36.749819+0000","last_became_peered":"2024-02-02T13:24:36.749819+0000","last_unstale":"2024-02-02T13:24:41.211911+0000","last_undegraded":"2024-02-02T13:24:41.211911+0000","last_fullsized":"2024-02-02T13:24:41.211911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093260+0000","last_change":"2024-02-02T13:24:36.765687+0000","last_active":"2024-02-02T13:24:41.093260+0000","last_peered":"2024-02-02T13:24:41.093260+0000","last_clean":"2024-02-02T13:24:41.093260+0000","last_became_active":"2024-02-02T13:24:36.765417+0000","last_became_peered":"2024-02-02T13:24:36.765417+0000","last_unstale":"2024-02-02T13:24:41.093260+0000","last_undegraded":"2024-02-02T13:24:41.093260+0000","last_fullsized":"2024-02-02T13:24:41.093260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762953+0000","last_change":"2024-02-02T13:24:36.762822+0000","last_active":"2024-02-02T13:24:36.762953+0000","last_peered":"2024-02-02T13:24:36.762953+0000","last_clean":"2024-02-02T13:24:36.762953+0000","last_became_active":"2024-02-02T13:24:36.761559+0000","last_became_peered":"2024-02-02T13:24:36.761559+0000","last_unstale":"2024-02-02T13:24:36.762953+0000","last_undegraded":"2024-02-02T13:24:36.762953+0000","last_fullsized":"2024-02-02T13:24:36.762953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764632+0000","last_change":"2024-02-02T13:24:36.764552+0000","last_active":"2024-02-02T13:24:36.764632+0000","last_peered":"2024-02-02T13:24:36.764632+0000","last_clean":"2024-02-02T13:24:36.764632+0000","last_became_active":"2024-02-02T13:24:36.764330+0000","last_became_peered":"2024-02-02T13:24:36.764330+0000","last_unstale":"2024-02-02T13:24:36.764632+0000","last_undegraded":"2024-02-02T13:24:36.764632+0000","last_fullsized":"2024-02-02T13:24:36.764632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.765462+0000","last_change":"2024-02-02T13:24:36.765179+0000","last_active":"2024-02-02T13:24:36.765462+0000","last_peered":"2024-02-02T13:24:36.765462+0000","last_clean":"2024-02-02T13:24:36.765462+0000","last_became_active":"2024-02-02T13:24:36.764850+0000","last_became_peered":"2024-02-02T13:24:36.764850+0000","last_unstale":"2024-02-02T13:24:36.765462+0000","last_undegraded":"2024-02-02T13:24:36.765462+0000","last_fullsized":"2024-02-02T13:24:36.765462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753019+0000","last_change":"2024-02-02T13:24:36.755360+0000","last_active":"2024-02-02T13:24:37.753019+0000","last_peered":"2024-02-02T13:24:37.753019+0000","last_clean":"2024-02-02T13:24:37.753019+0000","last_became_active":"2024-02-02T13:24:36.754987+0000","last_became_peered":"2024-02-02T13:24:36.754987+0000","last_unstale":"2024-02-02T13:24:37.753019+0000","last_undegraded":"2024-02-02T13:24:37.753019+0000","last_fullsized":"2024-02-02T13:24:37.753019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772676+0000","last_change":"2024-02-02T13:24:36.288622+0000","last_active":"2024-02-02T13:24:36.772676+0000","last_peered":"2024-02-02T13:24:36.772676+0000","last_clean":"2024-02-02T13:24:36.772676+0000","last_became_active":"2024-02-02T13:24:36.288400+0000","last_became_peered":"2024-02-02T13:24:36.288400+0000","last_unstale":"2024-02-02T13:24:36.772676+0000","last_undegraded":"2024-02-02T13:24:36.772676+0000","last_fullsized":"2024-02-02T13:24:36.772676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092688+0000","last_change":"2024-02-02T13:24:36.765345+0000","last_active":"2024-02-02T13:24:41.092688+0000","last_peered":"2024-02-02T13:24:41.092688+0000","last_clean":"2024-02-02T13:24:41.092688+0000","last_became_active":"2024-02-02T13:24:36.765122+0000","last_became_peered":"2024-02-02T13:24:36.765122+0000","last_unstale":"2024-02-02T13:24:41.092688+0000","last_undegraded":"2024-02-02T13:24:41.092688+0000","last_fullsized":"2024-02-02T13:24:41.092688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774103+0000","last_change":"2024-02-02T13:24:36.284315+0000","last_active":"2024-02-02T13:24:36.774103+0000","last_peered":"2024-02-02T13:24:36.774103+0000","last_clean":"2024-02-02T13:24:36.774103+0000","last_became_active":"2024-02-02T13:24:36.283193+0000","last_became_peered":"2024-02-02T13:24:36.283193+0000","last_unstale":"2024-02-02T13:24:36.774103+0000","last_undegraded":"2024-02-02T13:24:36.774103+0000","last_fullsized":"2024-02-02T13:24:36.774103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.741346+0000","last_change":"2024-02-02T13:24:36.280943+0000","last_active":"2024-02-02T13:24:36.741346+0000","last_peered":"2024-02-02T13:24:36.741346+0000","last_clean":"2024-02-02T13:24:36.741346+0000","last_became_active":"2024-02-02T13:24:36.280565+0000","last_became_peered":"2024-02-02T13:24:36.280565+0000","last_unstale":"2024-02-02T13:24:36.741346+0000","last_undegraded":"2024-02-02T13:24:36.741346+0000","last_fullsized":"2024-02-02T13:24:36.741346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211701+0000","last_change":"2024-02-02T13:24:36.761849+0000","last_active":"2024-02-02T13:24:41.211701+0000","last_peered":"2024-02-02T13:24:41.211701+0000","last_clean":"2024-02-02T13:24:41.211701+0000","last_became_active":"2024-02-02T13:24:36.761460+0000","last_became_peered":"2024-02-02T13:24:36.761460+0000","last_unstale":"2024-02-02T13:24:41.211701+0000","last_undegraded":"2024-02-02T13:24:41.211701+0000","last_fullsized":"2024-02-02T13:24:41.211701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.753498+0000","last_change":"2024-02-02T13:24:36.286417+0000","last_active":"2024-02-02T13:24:36.753498+0000","last_peered":"2024-02-02T13:24:36.753498+0000","last_clean":"2024-02-02T13:24:36.753498+0000","last_became_active":"2024-02-02T13:24:36.286107+0000","last_became_peered":"2024-02-02T13:24:36.286107+0000","last_unstale":"2024-02-02T13:24:36.753498+0000","last_undegraded":"2024-02-02T13:24:36.753498+0000","last_fullsized":"2024-02-02T13:24:36.753498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876738+0000","last_change":"2024-02-02T13:24:36.750157+0000","last_active":"2024-02-02T13:24:39.876738+0000","last_peered":"2024-02-02T13:24:39.876738+0000","last_clean":"2024-02-02T13:24:39.876738+0000","last_became_active":"2024-02-02T13:24:36.749764+0000","last_became_peered":"2024-02-02T13:24:36.749764+0000","last_unstale":"2024-02-02T13:24:39.876738+0000","last_undegraded":"2024-02-02T13:24:39.876738+0000","last_fullsized":"2024-02-02T13:24:39.876738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762177+0000","last_change":"2024-02-02T13:24:36.290816+0000","last_active":"2024-02-02T13:24:36.762177+0000","last_peered":"2024-02-02T13:24:36.762177+0000","last_clean":"2024-02-02T13:24:36.762177+0000","last_became_active":"2024-02-02T13:24:36.290592+0000","last_became_peered":"2024-02-02T13:24:36.290592+0000","last_unstale":"2024-02-02T13:24:36.762177+0000","last_undegraded":"2024-02-02T13:24:36.762177+0000","last_fullsized":"2024-02-02T13:24:36.762177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761656+0000","last_change":"2024-02-02T13:24:36.761589+0000","last_active":"2024-02-02T13:24:36.761656+0000","last_peered":"2024-02-02T13:24:36.761656+0000","last_clean":"2024-02-02T13:24:36.761656+0000","last_became_active":"2024-02-02T13:24:36.761215+0000","last_became_peered":"2024-02-02T13:24:36.761215+0000","last_unstale":"2024-02-02T13:24:36.761656+0000","last_undegraded":"2024-02-02T13:24:36.761656+0000","last_fullsized":"2024-02-02T13:24:36.761656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.758412+0000","last_change":"2024-02-02T13:24:36.281944+0000","last_active":"2024-02-02T13:24:36.758412+0000","last_peered":"2024-02-02T13:24:36.758412+0000","last_clean":"2024-02-02T13:24:36.758412+0000","last_became_active":"2024-02-02T13:24:36.281546+0000","last_became_peered":"2024-02-02T13:24:36.281546+0000","last_unstale":"2024-02-02T13:24:36.758412+0000","last_undegraded":"2024-02-02T13:24:36.758412+0000","last_fullsized":"2024-02-02T13:24:36.758412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093550+0000","last_change":"2024-02-02T13:24:36.775310+0000","last_active":"2024-02-02T13:24:41.093550+0000","last_peered":"2024-02-02T13:24:41.093550+0000","last_clean":"2024-02-02T13:24:41.093550+0000","last_became_active":"2024-02-02T13:24:36.775021+0000","last_became_peered":"2024-02-02T13:24:36.775021+0000","last_unstale":"2024-02-02T13:24:41.093550+0000","last_undegraded":"2024-02-02T13:24:41.093550+0000","last_fullsized":"2024-02-02T13:24:41.093550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777366+0000","last_change":"2024-02-02T13:24:36.267935+0000","last_active":"2024-02-02T13:24:36.777366+0000","last_peered":"2024-02-02T13:24:36.777366+0000","last_clean":"2024-02-02T13:24:36.777366+0000","last_became_active":"2024-02-02T13:24:36.267120+0000","last_became_peered":"2024-02-02T13:24:36.267120+0000","last_unstale":"2024-02-02T13:24:36.777366+0000","last_undegraded":"2024-02-02T13:24:36.777366+0000","last_fullsized":"2024-02-02T13:24:36.777366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753018+0000","last_change":"2024-02-02T13:24:36.758353+0000","last_active":"2024-02-02T13:24:37.753018+0000","last_peered":"2024-02-02T13:24:37.753018+0000","last_clean":"2024-02-02T13:24:37.753018+0000","last_became_active":"2024-02-02T13:24:36.758074+0000","last_became_peered":"2024-02-02T13:24:36.758074+0000","last_unstale":"2024-02-02T13:24:37.753018+0000","last_undegraded":"2024-02-02T13:24:37.753018+0000","last_fullsized":"2024-02-02T13:24:37.753018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.776688+0000","last_change":"2024-02-02T13:24:36.283779+0000","last_active":"2024-02-02T13:24:36.776688+0000","last_peered":"2024-02-02T13:24:36.776688+0000","last_clean":"2024-02-02T13:24:36.776688+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.776688+0000","last_undegraded":"2024-02-02T13:24:36.776688+0000","last_fullsized":"2024-02-02T13:24:36.776688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978240+0000","last_change":"2024-02-02T13:24:36.751477+0000","last_active":"2024-02-02T13:24:40.978240+0000","last_peered":"2024-02-02T13:24:40.978240+0000","last_clean":"2024-02-02T13:24:40.978240+0000","last_became_active":"2024-02-02T13:24:36.751226+0000","last_became_peered":"2024-02-02T13:24:36.751226+0000","last_unstale":"2024-02-02T13:24:40.978240+0000","last_undegraded":"2024-02-02T13:24:40.978240+0000","last_fullsized":"2024-02-02T13:24:40.978240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773129+0000","last_change":"2024-02-02T13:24:36.286732+0000","last_active":"2024-02-02T13:24:36.773129+0000","last_peered":"2024-02-02T13:24:36.773129+0000","last_clean":"2024-02-02T13:24:36.773129+0000","last_became_active":"2024-02-02T13:24:36.286351+0000","last_became_peered":"2024-02-02T13:24:36.286351+0000","last_unstale":"2024-02-02T13:24:36.773129+0000","last_undegraded":"2024-02-02T13:24:36.773129+0000","last_fullsized":"2024-02-02T13:24:36.773129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093755+0000","last_change":"2024-02-02T13:24:36.756739+0000","last_active":"2024-02-02T13:24:41.093755+0000","last_peered":"2024-02-02T13:24:41.093755+0000","last_clean":"2024-02-02T13:24:41.093755+0000","last_became_active":"2024-02-02T13:24:36.756537+0000","last_became_peered":"2024-02-02T13:24:36.756537+0000","last_unstale":"2024-02-02T13:24:41.093755+0000","last_undegraded":"2024-02-02T13:24:41.093755+0000","last_fullsized":"2024-02-02T13:24:41.093755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755625+0000","last_change":"2024-02-02T13:24:36.286698+0000","last_active":"2024-02-02T13:24:36.755625+0000","last_peered":"2024-02-02T13:24:36.755625+0000","last_clean":"2024-02-02T13:24:36.755625+0000","last_became_active":"2024-02-02T13:24:36.282637+0000","last_became_peered":"2024-02-02T13:24:36.282637+0000","last_unstale":"2024-02-02T13:24:36.755625+0000","last_undegraded":"2024-02-02T13:24:36.755625+0000","last_fullsized":"2024-02-02T13:24:36.755625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978581+0000","last_change":"2024-02-02T13:24:36.766477+0000","last_active":"2024-02-02T13:24:40.978581+0000","last_peered":"2024-02-02T13:24:40.978581+0000","last_clean":"2024-02-02T13:24:40.978581+0000","last_became_active":"2024-02-02T13:24:36.765889+0000","last_became_peered":"2024-02-02T13:24:36.765889+0000","last_unstale":"2024-02-02T13:24:40.978581+0000","last_undegraded":"2024-02-02T13:24:40.978581+0000","last_fullsized":"2024-02-02T13:24:40.978581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764428+0000","last_change":"2024-02-02T13:24:36.286937+0000","last_active":"2024-02-02T13:24:36.764428+0000","last_peered":"2024-02-02T13:24:36.764428+0000","last_clean":"2024-02-02T13:24:36.764428+0000","last_became_active":"2024-02-02T13:24:36.286651+0000","last_became_peered":"2024-02-02T13:24:36.286651+0000","last_unstale":"2024-02-02T13:24:36.764428+0000","last_undegraded":"2024-02-02T13:24:36.764428+0000","last_fullsized":"2024-02-02T13:24:36.764428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.769009+0000","last_change":"2024-02-02T13:24:36.768901+0000","last_active":"2024-02-02T13:24:36.769009+0000","last_peered":"2024-02-02T13:24:36.769009+0000","last_clean":"2024-02-02T13:24:36.769009+0000","last_became_active":"2024-02-02T13:24:36.768565+0000","last_became_peered":"2024-02-02T13:24:36.768565+0000","last_unstale":"2024-02-02T13:24:36.769009+0000","last_undegraded":"2024-02-02T13:24:36.769009+0000","last_fullsized":"2024-02-02T13:24:36.769009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775361+0000","last_change":"2024-02-02T13:24:36.271772+0000","last_active":"2024-02-02T13:24:36.775361+0000","last_peered":"2024-02-02T13:24:36.775361+0000","last_clean":"2024-02-02T13:24:36.775361+0000","last_became_active":"2024-02-02T13:24:36.271163+0000","last_became_peered":"2024-02-02T13:24:36.271163+0000","last_unstale":"2024-02-02T13:24:36.775361+0000","last_undegraded":"2024-02-02T13:24:36.775361+0000","last_fullsized":"2024-02-02T13:24:36.775361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.979190+0000","last_change":"2024-02-02T13:24:36.756837+0000","last_active":"2024-02-02T13:24:40.979190+0000","last_peered":"2024-02-02T13:24:40.979190+0000","last_clean":"2024-02-02T13:24:40.979190+0000","last_became_active":"2024-02-02T13:24:36.756508+0000","last_became_peered":"2024-02-02T13:24:36.756508+0000","last_unstale":"2024-02-02T13:24:40.979190+0000","last_undegraded":"2024-02-02T13:24:40.979190+0000","last_fullsized":"2024-02-02T13:24:40.979190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771720+0000","last_change":"2024-02-02T13:24:36.287213+0000","last_active":"2024-02-02T13:24:36.771720+0000","last_peered":"2024-02-02T13:24:36.771720+0000","last_clean":"2024-02-02T13:24:36.771720+0000","last_became_active":"2024-02-02T13:24:36.286967+0000","last_became_peered":"2024-02-02T13:24:36.286967+0000","last_unstale":"2024-02-02T13:24:36.771720+0000","last_undegraded":"2024-02-02T13:24:36.771720+0000","last_fullsized":"2024-02-02T13:24:36.771720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978811+0000","last_change":"2024-02-02T13:24:36.766631+0000","last_active":"2024-02-02T13:24:40.978811+0000","last_peered":"2024-02-02T13:24:40.978811+0000","last_clean":"2024-02-02T13:24:40.978811+0000","last_became_active":"2024-02-02T13:24:36.766216+0000","last_became_peered":"2024-02-02T13:24:36.766216+0000","last_unstale":"2024-02-02T13:24:40.978811+0000","last_undegraded":"2024-02-02T13:24:40.978811+0000","last_fullsized":"2024-02-02T13:24:40.978811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772131+0000","last_change":"2024-02-02T13:24:36.275397+0000","last_active":"2024-02-02T13:24:36.772131+0000","last_peered":"2024-02-02T13:24:36.772131+0000","last_clean":"2024-02-02T13:24:36.772131+0000","last_became_active":"2024-02-02T13:24:36.274927+0000","last_became_peered":"2024-02-02T13:24:36.274927+0000","last_unstale":"2024-02-02T13:24:36.772131+0000","last_undegraded":"2024-02-02T13:24:36.772131+0000","last_fullsized":"2024-02-02T13:24:36.772131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978419+0000","last_change":"2024-02-02T13:24:36.757035+0000","last_active":"2024-02-02T13:24:40.978419+0000","last_peered":"2024-02-02T13:24:40.978419+0000","last_clean":"2024-02-02T13:24:40.978419+0000","last_became_active":"2024-02-02T13:24:36.756748+0000","last_became_peered":"2024-02-02T13:24:36.756748+0000","last_unstale":"2024-02-02T13:24:40.978419+0000","last_undegraded":"2024-02-02T13:24:40.978419+0000","last_fullsized":"2024-02-02T13:24:40.978419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:24:57.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768414+0000","last_change":"2024-02-02T13:24:36.283044+0000","last_active":"2024-02-02T13:24:36.768414+0000","last_peered":"2024-02-02T13:24:36.768414+0000","last_clean":"2024-02-02T13:24:36.768414+0000","last_became_active":"2024-02-02T13:24:36.282798+0000","last_became_peered":"2024-02-02T13:24:36.282798+0000","last_unstale":"2024-02-02T13:24:36.768414+0000","last_undegraded":"2024-02-02T13:24:36.768414+0000","last_fullsized":"2024-02-02T13:24:36.768414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770333+0000","last_change":"2024-02-02T13:24:36.764197+0000","last_active":"2024-02-02T13:24:36.770333+0000","last_peered":"2024-02-02T13:24:36.770333+0000","last_clean":"2024-02-02T13:24:36.770333+0000","last_became_active":"2024-02-02T13:24:36.763970+0000","last_became_peered":"2024-02-02T13:24:36.763970+0000","last_unstale":"2024-02-02T13:24:36.770333+0000","last_undegraded":"2024-02-02T13:24:36.770333+0000","last_fullsized":"2024-02-02T13:24:36.770333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.757774+0000","last_change":"2024-02-02T13:24:36.279681+0000","last_active":"2024-02-02T13:24:36.757774+0000","last_peered":"2024-02-02T13:24:36.757774+0000","last_clean":"2024-02-02T13:24:36.757774+0000","last_became_active":"2024-02-02T13:24:36.279342+0000","last_became_peered":"2024-02-02T13:24:36.279342+0000","last_unstale":"2024-02-02T13:24:36.757774+0000","last_undegraded":"2024-02-02T13:24:36.757774+0000","last_fullsized":"2024-02-02T13:24:36.757774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978828+0000","last_change":"2024-02-02T13:24:36.758113+0000","last_active":"2024-02-02T13:24:40.978828+0000","last_peered":"2024-02-02T13:24:40.978828+0000","last_clean":"2024-02-02T13:24:40.978828+0000","last_became_active":"2024-02-02T13:24:36.757859+0000","last_became_peered":"2024-02-02T13:24:36.757859+0000","last_unstale":"2024-02-02T13:24:40.978828+0000","last_undegraded":"2024-02-02T13:24:40.978828+0000","last_fullsized":"2024-02-02T13:24:40.978828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748840+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:36.748840+0000","last_peered":"2024-02-02T13:24:36.748840+0000","last_clean":"2024-02-02T13:24:36.748840+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:36.748840+0000","last_undegraded":"2024-02-02T13:24:36.748840+0000","last_fullsized":"2024-02-02T13:24:36.748840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755725+0000","last_change":"2024-02-02T13:24:36.285102+0000","last_active":"2024-02-02T13:24:36.755725+0000","last_peered":"2024-02-02T13:24:36.755725+0000","last_clean":"2024-02-02T13:24:36.755725+0000","last_became_active":"2024-02-02T13:24:36.280044+0000","last_became_peered":"2024-02-02T13:24:36.280044+0000","last_unstale":"2024-02-02T13:24:36.755725+0000","last_undegraded":"2024-02-02T13:24:36.755725+0000","last_fullsized":"2024-02-02T13:24:36.755725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211798+0000","last_change":"2024-02-02T13:24:36.762214+0000","last_active":"2024-02-02T13:24:41.211798+0000","last_peered":"2024-02-02T13:24:41.211798+0000","last_clean":"2024-02-02T13:24:41.211798+0000","last_became_active":"2024-02-02T13:24:36.761690+0000","last_became_peered":"2024-02-02T13:24:36.761690+0000","last_unstale":"2024-02-02T13:24:41.211798+0000","last_undegraded":"2024-02-02T13:24:41.211798+0000","last_fullsized":"2024-02-02T13:24:41.211798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748517+0000","last_change":"2024-02-02T13:24:36.291720+0000","last_active":"2024-02-02T13:24:36.748517+0000","last_peered":"2024-02-02T13:24:36.748517+0000","last_clean":"2024-02-02T13:24:36.748517+0000","last_became_active":"2024-02-02T13:24:36.290703+0000","last_became_peered":"2024-02-02T13:24:36.290703+0000","last_unstale":"2024-02-02T13:24:36.748517+0000","last_undegraded":"2024-02-02T13:24:36.748517+0000","last_fullsized":"2024-02-02T13:24:36.748517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093549+0000","last_change":"2024-02-02T13:24:36.766220+0000","last_active":"2024-02-02T13:24:41.093549+0000","last_peered":"2024-02-02T13:24:41.093549+0000","last_clean":"2024-02-02T13:24:41.093549+0000","last_became_active":"2024-02-02T13:24:36.765734+0000","last_became_peered":"2024-02-02T13:24:36.765734+0000","last_unstale":"2024-02-02T13:24:41.093549+0000","last_undegraded":"2024-02-02T13:24:41.093549+0000","last_fullsized":"2024-02-02T13:24:41.093549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.756734+0000","last_change":"2024-02-02T13:24:36.276562+0000","last_active":"2024-02-02T13:24:36.756734+0000","last_peered":"2024-02-02T13:24:36.756734+0000","last_clean":"2024-02-02T13:24:36.756734+0000","last_became_active":"2024-02-02T13:24:36.276189+0000","last_became_peered":"2024-02-02T13:24:36.276189+0000","last_unstale":"2024-02-02T13:24:36.756734+0000","last_undegraded":"2024-02-02T13:24:36.756734+0000","last_fullsized":"2024-02-02T13:24:36.756734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877004+0000","last_change":"2024-02-02T13:24:36.761943+0000","last_active":"2024-02-02T13:24:39.877004+0000","last_peered":"2024-02-02T13:24:39.877004+0000","last_clean":"2024-02-02T13:24:39.877004+0000","last_became_active":"2024-02-02T13:24:36.761718+0000","last_became_peered":"2024-02-02T13:24:36.761718+0000","last_unstale":"2024-02-02T13:24:39.877004+0000","last_undegraded":"2024-02-02T13:24:39.877004+0000","last_fullsized":"2024-02-02T13:24:39.877004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.760484+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.760484+0000","last_peered":"2024-02-02T13:24:36.760484+0000","last_clean":"2024-02-02T13:24:36.760484+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.760484+0000","last_undegraded":"2024-02-02T13:24:36.760484+0000","last_fullsized":"2024-02-02T13:24:36.760484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753283+0000","last_change":"2024-02-02T13:24:36.763372+0000","last_active":"2024-02-02T13:24:37.753283+0000","last_peered":"2024-02-02T13:24:37.753283+0000","last_clean":"2024-02-02T13:24:37.753283+0000","last_became_active":"2024-02-02T13:24:36.763132+0000","last_became_peered":"2024-02-02T13:24:36.763132+0000","last_unstale":"2024-02-02T13:24:37.753283+0000","last_undegraded":"2024-02-02T13:24:37.753283+0000","last_fullsized":"2024-02-02T13:24:37.753283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:03.055262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747812+0000","last_change":"2024-02-02T13:24:36.287910+0000","last_active":"2024-02-02T13:24:36.747812+0000","last_peered":"2024-02-02T13:24:36.747812+0000","last_clean":"2024-02-02T13:24:36.747812+0000","last_became_active":"2024-02-02T13:24:36.287683+0000","last_became_peered":"2024-02-02T13:24:36.287683+0000","last_unstale":"2024-02-02T13:24:36.747812+0000","last_undegraded":"2024-02-02T13:24:36.747812+0000","last_fullsized":"2024-02-02T13:24:36.747812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773408+0000","last_change":"2024-02-02T13:24:36.773275+0000","last_active":"2024-02-02T13:24:36.773408+0000","last_peered":"2024-02-02T13:24:36.773408+0000","last_clean":"2024-02-02T13:24:36.773408+0000","last_became_active":"2024-02-02T13:24:36.772646+0000","last_became_peered":"2024-02-02T13:24:36.772646+0000","last_unstale":"2024-02-02T13:24:36.773408+0000","last_undegraded":"2024-02-02T13:24:36.773408+0000","last_fullsized":"2024-02-02T13:24:36.773408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748406+0000","last_change":"2024-02-02T13:24:36.291402+0000","last_active":"2024-02-02T13:24:36.748406+0000","last_peered":"2024-02-02T13:24:36.748406+0000","last_clean":"2024-02-02T13:24:36.748406+0000","last_became_active":"2024-02-02T13:24:36.290274+0000","last_became_peered":"2024-02-02T13:24:36.290274+0000","last_unstale":"2024-02-02T13:24:36.748406+0000","last_undegraded":"2024-02-02T13:24:36.748406+0000","last_fullsized":"2024-02-02T13:24:36.748406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876617+0000","last_change":"2024-02-02T13:24:36.760028+0000","last_active":"2024-02-02T13:24:39.876617+0000","last_peered":"2024-02-02T13:24:39.876617+0000","last_clean":"2024-02-02T13:24:39.876617+0000","last_became_active":"2024-02-02T13:24:36.759641+0000","last_became_peered":"2024-02-02T13:24:36.759641+0000","last_unstale":"2024-02-02T13:24:39.876617+0000","last_undegraded":"2024-02-02T13:24:39.876617+0000","last_fullsized":"2024-02-02T13:24:39.876617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.746080+0000","last_change":"2024-02-02T13:24:36.281619+0000","last_active":"2024-02-02T13:24:36.746080+0000","last_peered":"2024-02-02T13:24:36.746080+0000","last_clean":"2024-02-02T13:24:36.746080+0000","last_became_active":"2024-02-02T13:24:36.281168+0000","last_became_peered":"2024-02-02T13:24:36.281168+0000","last_unstale":"2024-02-02T13:24:36.746080+0000","last_undegraded":"2024-02-02T13:24:36.746080+0000","last_fullsized":"2024-02-02T13:24:36.746080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211709+0000","last_change":"2024-02-02T13:24:36.750882+0000","last_active":"2024-02-02T13:24:41.211709+0000","last_peered":"2024-02-02T13:24:41.211709+0000","last_clean":"2024-02-02T13:24:41.211709+0000","last_became_active":"2024-02-02T13:24:36.750448+0000","last_became_peered":"2024-02-02T13:24:36.750448+0000","last_unstale":"2024-02-02T13:24:41.211709+0000","last_undegraded":"2024-02-02T13:24:41.211709+0000","last_fullsized":"2024-02-02T13:24:41.211709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774907+0000","last_change":"2024-02-02T13:24:36.289169+0000","last_active":"2024-02-02T13:24:36.774907+0000","last_peered":"2024-02-02T13:24:36.774907+0000","last_clean":"2024-02-02T13:24:36.774907+0000","last_became_active":"2024-02-02T13:24:36.288939+0000","last_became_peered":"2024-02-02T13:24:36.288939+0000","last_unstale":"2024-02-02T13:24:36.774907+0000","last_undegraded":"2024-02-02T13:24:36.774907+0000","last_fullsized":"2024-02-02T13:24:36.774907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093133+0000","last_change":"2024-02-02T13:24:36.756653+0000","last_active":"2024-02-02T13:24:41.093133+0000","last_peered":"2024-02-02T13:24:41.093133+0000","last_clean":"2024-02-02T13:24:41.093133+0000","last_became_active":"2024-02-02T13:24:36.756419+0000","last_became_peered":"2024-02-02T13:24:36.756419+0000","last_unstale":"2024-02-02T13:24:41.093133+0000","last_undegraded":"2024-02-02T13:24:41.093133+0000","last_fullsized":"2024-02-02T13:24:41.093133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.751195+0000","last_change":"2024-02-02T13:24:36.290423+0000","last_active":"2024-02-02T13:24:36.751195+0000","last_peered":"2024-02-02T13:24:36.751195+0000","last_clean":"2024-02-02T13:24:36.751195+0000","last_became_active":"2024-02-02T13:24:36.289863+0000","last_became_peered":"2024-02-02T13:24:36.289863+0000","last_unstale":"2024-02-02T13:24:36.751195+0000","last_undegraded":"2024-02-02T13:24:36.751195+0000","last_fullsized":"2024-02-02T13:24:36.751195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.754234+0000","last_change":"2024-02-02T13:24:36.759429+0000","last_active":"2024-02-02T13:24:37.754234+0000","last_peered":"2024-02-02T13:24:37.754234+0000","last_clean":"2024-02-02T13:24:37.754234+0000","last_became_active":"2024-02-02T13:24:36.759196+0000","last_became_peered":"2024-02-02T13:24:36.759196+0000","last_unstale":"2024-02-02T13:24:37.754234+0000","last_undegraded":"2024-02-02T13:24:37.754234+0000","last_fullsized":"2024-02-02T13:24:37.754234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747537+0000","last_change":"2024-02-02T13:24:36.288653+0000","last_active":"2024-02-02T13:24:36.747537+0000","last_peered":"2024-02-02T13:24:36.747537+0000","last_clean":"2024-02-02T13:24:36.747537+0000","last_became_active":"2024-02-02T13:24:36.288439+0000","last_became_peered":"2024-02-02T13:24:36.288439+0000","last_unstale":"2024-02-02T13:24:36.747537+0000","last_undegraded":"2024-02-02T13:24:36.747537+0000","last_fullsized":"2024-02-02T13:24:36.747537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211609+0000","last_change":"2024-02-02T13:24:36.751484+0000","last_active":"2024-02-02T13:24:41.211609+0000","last_peered":"2024-02-02T13:24:41.211609+0000","last_clean":"2024-02-02T13:24:41.211609+0000","last_became_active":"2024-02-02T13:24:36.751091+0000","last_became_peered":"2024-02-02T13:24:36.751091+0000","last_unstale":"2024-02-02T13:24:41.211609+0000","last_undegraded":"2024-02-02T13:24:41.211609+0000","last_fullsized":"2024-02-02T13:24:41.211609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747419+0000","last_change":"2024-02-02T13:24:36.280128+0000","last_active":"2024-02-02T13:24:36.747419+0000","last_peered":"2024-02-02T13:24:36.747419+0000","last_clean":"2024-02-02T13:24:36.747419+0000","last_became_active":"2024-02-02T13:24:36.279652+0000","last_became_peered":"2024-02-02T13:24:36.279652+0000","last_unstale":"2024-02-02T13:24:36.747419+0000","last_undegraded":"2024-02-02T13:24:36.747419+0000","last_fullsized":"2024-02-02T13:24:36.747419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212026+0000","last_change":"2024-02-02T13:24:36.763668+0000","last_active":"2024-02-02T13:24:41.212026+0000","last_peered":"2024-02-02T13:24:41.212026+0000","last_clean":"2024-02-02T13:24:41.212026+0000","last_became_active":"2024-02-02T13:24:36.762913+0000","last_became_peered":"2024-02-02T13:24:36.762913+0000","last_unstale":"2024-02-02T13:24:41.212026+0000","last_undegraded":"2024-02-02T13:24:41.212026+0000","last_fullsized":"2024-02-02T13:24:41.212026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774478+0000","last_change":"2024-02-02T13:24:36.289669+0000","last_active":"2024-02-02T13:24:36.774478+0000","last_peered":"2024-02-02T13:24:36.774478+0000","last_clean":"2024-02-02T13:24:36.774478+0000","last_became_active":"2024-02-02T13:24:36.289368+0000","last_became_peered":"2024-02-02T13:24:36.289368+0000","last_unstale":"2024-02-02T13:24:36.774478+0000","last_undegraded":"2024-02-02T13:24:36.774478+0000","last_fullsized":"2024-02-02T13:24:36.774478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770307+0000","last_change":"2024-02-02T13:24:36.769265+0000","last_active":"2024-02-02T13:24:36.770307+0000","last_peered":"2024-02-02T13:24:36.770307+0000","last_clean":"2024-02-02T13:24:36.770307+0000","last_became_active":"2024-02-02T13:24:36.769036+0000","last_became_peered":"2024-02-02T13:24:36.769036+0000","last_unstale":"2024-02-02T13:24:36.770307+0000","last_undegraded":"2024-02-02T13:24:36.770307+0000","last_fullsized":"2024-02-02T13:24:36.770307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770353+0000","last_change":"2024-02-02T13:24:36.765383+0000","last_active":"2024-02-02T13:24:36.770353+0000","last_peered":"2024-02-02T13:24:36.770353+0000","last_clean":"2024-02-02T13:24:36.770353+0000","last_became_active":"2024-02-02T13:24:36.765141+0000","last_became_peered":"2024-02-02T13:24:36.765141+0000","last_unstale":"2024-02-02T13:24:36.770353+0000","last_undegraded":"2024-02-02T13:24:36.770353+0000","last_fullsized":"2024-02-02T13:24:36.770353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778010+0000","last_change":"2024-02-02T13:24:36.290626+0000","last_active":"2024-02-02T13:24:36.778010+0000","last_peered":"2024-02-02T13:24:36.778010+0000","last_clean":"2024-02-02T13:24:36.778010+0000","last_became_active":"2024-02-02T13:24:36.290183+0000","last_became_peered":"2024-02-02T13:24:36.290183+0000","last_unstale":"2024-02-02T13:24:36.778010+0000","last_undegraded":"2024-02-02T13:24:36.778010+0000","last_fullsized":"2024-02-02T13:24:36.778010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777961+0000","last_change":"2024-02-02T13:24:36.777865+0000","last_active":"2024-02-02T13:24:36.777961+0000","last_peered":"2024-02-02T13:24:36.777961+0000","last_clean":"2024-02-02T13:24:36.777961+0000","last_became_active":"2024-02-02T13:24:36.777129+0000","last_became_peered":"2024-02-02T13:24:36.777129+0000","last_unstale":"2024-02-02T13:24:36.777961+0000","last_undegraded":"2024-02-02T13:24:36.777961+0000","last_fullsized":"2024-02-02T13:24:36.777961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772162+0000","last_change":"2024-02-02T13:24:36.290121+0000","last_active":"2024-02-02T13:24:36.772162+0000","last_peered":"2024-02-02T13:24:36.772162+0000","last_clean":"2024-02-02T13:24:36.772162+0000","last_became_active":"2024-02-02T13:24:36.289708+0000","last_became_peered":"2024-02-02T13:24:36.289708+0000","last_unstale":"2024-02-02T13:24:36.772162+0000","last_undegraded":"2024-02-02T13:24:36.772162+0000","last_fullsized":"2024-02-02T13:24:36.772162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093334+0000","last_change":"2024-02-02T13:24:36.759606+0000","last_active":"2024-02-02T13:24:41.093334+0000","last_peered":"2024-02-02T13:24:41.093334+0000","last_clean":"2024-02-02T13:24:41.093334+0000","last_became_active":"2024-02-02T13:24:36.759358+0000","last_became_peered":"2024-02-02T13:24:36.759358+0000","last_unstale":"2024-02-02T13:24:41.093334+0000","last_undegraded":"2024-02-02T13:24:41.093334+0000","last_fullsized":"2024-02-02T13:24:41.093334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771605+0000","last_change":"2024-02-02T13:24:36.286535+0000","last_active":"2024-02-02T13:24:36.771605+0000","last_peered":"2024-02-02T13:24:36.771605+0000","last_clean":"2024-02-02T13:24:36.771605+0000","last_became_active":"2024-02-02T13:24:36.286302+0000","last_became_peered":"2024-02-02T13:24:36.286302+0000","last_unstale":"2024-02-02T13:24:36.771605+0000","last_undegraded":"2024-02-02T13:24:36.771605+0000","last_fullsized":"2024-02-02T13:24:36.771605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212029+0000","last_change":"2024-02-02T13:24:36.769601+0000","last_active":"2024-02-02T13:24:41.212029+0000","last_peered":"2024-02-02T13:24:41.212029+0000","last_clean":"2024-02-02T13:24:41.212029+0000","last_became_active":"2024-02-02T13:24:36.768381+0000","last_became_peered":"2024-02-02T13:24:36.768381+0000","last_unstale":"2024-02-02T13:24:41.212029+0000","last_undegraded":"2024-02-02T13:24:41.212029+0000","last_fullsized":"2024-02-02T13:24:41.212029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774155+0000","last_change":"2024-02-02T13:24:36.289368+0000","last_active":"2024-02-02T13:24:36.774155+0000","last_peered":"2024-02-02T13:24:36.774155+0000","last_clean":"2024-02-02T13:24:36.774155+0000","last_became_active":"2024-02-02T13:24:36.289118+0000","last_became_peered":"2024-02-02T13:24:36.289118+0000","last_unstale":"2024-02-02T13:24:36.774155+0000","last_undegraded":"2024-02-02T13:24:36.774155+0000","last_fullsized":"2024-02-02T13:24:36.774155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":640,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194672,"statfs":{"total":200000000,"available":199344612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":63,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195249,"statfs":{"total":200000000,"available":199935017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:47.079 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:24:47.080 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-02T13:24:47.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-02-02T13:24:47.080 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-02-02T13:24:47.080 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-02-02T13:24:47.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.326+0000 7f5956600640 1 Processor -- start 2024-02-02T13:24:47.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.327+0000 7f5956600640 1 -- start start 2024-02-02T13:24:47.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5956600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59500d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5956600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500d19a0 con 0x7f59500d3840 2024-02-02T13:24:47.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5956600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500d1c80 con 0x7f595015d420 2024-02-02T13:24:47.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5956600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500cd960 con 0x7f59500b8b30 2024-02-02T13:24:47.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5946800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f59500d3840 0x7f59500b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34070/0 (socket says 172.21.15.91:34070) 2024-02-02T13:24:47.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.328+0000 7f5946800640 1 -- 172.21.15.91:0/4091530819 learned_addr learned my addr 172.21.15.91:0/4091530819 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594fe00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59500d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882623274 0 0) 0x7f59500d19a0 con 0x7f59500d3840 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5930003520 con 0x7f59500d3840 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784003771 0 0) 0x7f5930003520 con 0x7f59500d3840 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 msgr2=0x7f59500d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59500d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 >> v1:172.21.15.113:6789/0 conn(0x7f595015d420 legacy=0x7f59500d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500d1f60 con 0x7f59500d3840 2024-02-02T13:24:47.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5938001670 con 0x7f59500d3840 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 >> v1:172.21.15.91:6789/0 conn(0x7f59500d3840 legacy=0x7f59500b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59300039a0 0x7f5930003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5930004d00 0x7f5930005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500cd960 con 0x7f59300039a0 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593000ba30 con 0x7f5930004d00 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593000bbf0 con 0x7f59500b8b30 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.329+0000 7f594f400640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59300039a0 0x7f5930003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594fe00640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5930004d00 0x7f5930005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f5946800640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594f400640 1 -- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 msgr2=0x7f59300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594f400640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59500b8b30 0x7f59300047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594f400640 1 -- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5930004d00 msgr2=0x7f5930005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594f400640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5930004d00 0x7f5930005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.330+0000 7f594f400640 1 -- 172.21.15.91:0/4091530819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930003520 con 0x7f59300039a0 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.331+0000 7f594f400640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59300039a0 0x7f5930003dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5940001720 tx=0x7f59300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b6dbb06d472c7f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.331+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400052d0 con 0x7f59300039a0 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.331+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594002e050 con 0x7f59300039a0 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.331+0000 7f594ea00640 1 -- 172.21.15.91:0/4091530819 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400035f0 con 0x7f59300039a0 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 -- 172.21.15.91:0/4091530819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59300039a0 msgr2=0x7f5930003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 --2- 172.21.15.91:0/4091530819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59300039a0 0x7f5930003dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5940001720 tx=0x7f59300057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f594fe00640 1 -- 172.21.15.91:0/4091530819 reap_dead start 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 -- 172.21.15.91:0/4091530819 shutdown_connections 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 -- 172.21.15.91:0/4091530819 >> 172.21.15.91:0/4091530819 conn(0x7f5950070690 msgr2=0x7f5950070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 -- 172.21.15.91:0/4091530819 shutdown_connections 2024-02-02T13:24:47.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.332+0000 7f5956600640 1 -- 172.21.15.91:0/4091530819 wait complete. 2024-02-02T13:24:47.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.334+0000 7f5956600640 1 Processor -- start 2024-02-02T13:24:47.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.334+0000 7f5956600640 1 -- start start 2024-02-02T13:24:47.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59300039a0 0x7f59501501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 0x7f59501535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950153b00 con 0x7f5950155480 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950153f90 con 0x7f5950153110 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5956600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500b4fb0 con 0x7f59300039a0 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53192/0 (socket says 172.21.15.91:53192) 2024-02-02T13:24:47.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 -- 172.21.15.91:0/3686285833 learned_addr learned my addr 172.21.15.91:0/3686285833 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:47.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f594fe00640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 0x7f59501535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f594f400640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59300039a0 0x7f59501501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 -- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59300039a0 msgr2=0x7f59501501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59300039a0 0x7f59501501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.335+0000 7f5946800640 1 -- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 msgr2=0x7f59501535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5946800640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 0x7f59501535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5946800640 1 -- 172.21.15.91:0/3686285833 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59501564b0 con 0x7f5950155480 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f594fe00640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 0x7f59501535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5946800640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f593800d7a0 tx=0x7f593800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380074e0 con 0x7f5950155480 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938007640 con 0x7f5950155480 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500b5440 con 0x7f5950155480 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938004a00 con 0x7f5950155480 2024-02-02T13:24:47.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.336+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500b4680 con 0x7f5950155480 2024-02-02T13:24:47.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.337+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950073830 con 0x7f5950155480 2024-02-02T13:24:47.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.338+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5938004b60 con 0x7f5950155480 2024-02-02T13:24:47.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.339+0000 7f5947e00640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 0x7f592806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.339+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5938088240 con 0x7f5950155480 2024-02-02T13:24:47.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.339+0000 7f594f400640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 0x7f592806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.340+0000 7f594f400640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 0x7f592806c920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5940033000 tx=0x7f5940009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.343+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5938053820 con 0x7f5950155480 2024-02-02T13:24:47.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.582+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5950066cf0 con 0x7f5950155480 2024-02-02T13:24:47.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.591+0000 7f5947e00640 1 -- 172.21.15.91:0/3686285833 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169963 (secure 0 0 0) 0x7f5938056e40 con 0x7f5950155480 2024-02-02T13:24:47.595 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:47.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 msgr2=0x7f592806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 0x7f592806c920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5940033000 tx=0x7f5940009030 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 msgr2=0x7f59500b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f593800d7a0 tx=0x7f593800dc60 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 shutdown_connections 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f592806a460 0x7f592806c920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5950153110 0x7f59501535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5950155480 0x7f59500b3b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 --2- 172.21.15.91:0/3686285833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59300039a0 0x7f59501501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 >> 172.21.15.91:0/3686285833 conn(0x7f5950070690 msgr2=0x7f5950065910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 shutdown_connections 2024-02-02T13:24:47.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.596+0000 7f5956600640 1 -- 172.21.15.91:0/3686285833 wait complete. 2024-02-02T13:24:47.615 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":32622},{"type":"v1","addr":"172.21.15.91:6833","nonce":32622}]},"active_addr":"172.21.15.91:6833/32622","active_change":"2024-02-02T13:24:24.211768+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":2555686385}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":2858952338}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":1318217656}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":3732478566}]}]} 2024-02-02T13:24:47.618 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-02-02T13:24:47.618 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-02-02T13:24:47.618 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:47.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.809+0000 7f993a400640 1 Processor -- start 2024-02-02T13:24:47.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.809+0000 7f993a400640 1 -- start start 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f993a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f993415d490 0x7f993406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f993a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99340ac8d0 0x7f99340acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f993a400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993415d870 con 0x7f99340b0700 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f993a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340ad270 con 0x7f993415d490 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f993a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993406bce0 con 0x7f99340ac8d0 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f993415d490 0x7f993406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f993415d490 0x7f993406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53206/0 (socket says 172.21.15.91:53206) 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f99340b0700 0x7f99340c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34086/0 (socket says 172.21.15.91:34086) 2024-02-02T13:24:47.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933400640 1 -- 172.21.15.91:0/3408042843 learned_addr learned my addr 172.21.15.91:0/3408042843 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f992be00640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99340ac8d0 0x7f99340acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933400640 1 -- 172.21.15.91:0/3408042843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99340ac8d0 msgr2=0x7f99340acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.811+0000 7f9933400640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99340ac8d0 0x7f99340acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.812+0000 7f9933400640 1 -- 172.21.15.91:0/3408042843 >> v1:172.21.15.91:6789/0 conn(0x7f99340b0700 legacy=0x7f99340c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.812+0000 7f9932a00640 1 -- 172.21.15.91:0/3408042843 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822219822 0 0) 0x7f993415d870 con 0x7f99340b0700 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.812+0000 7f9933400640 1 -- 172.21.15.91:0/3408042843 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934161220 con 0x7f993415d490 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f9933400640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993415d490 0x7f993406c170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9924009870 tx=0x7f992402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99be608e5c733df0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f9932a00640 1 -- 172.21.15.91:0/3408042843 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924030840 con 0x7f993415d490 2024-02-02T13:24:47.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f9932a00640 1 -- 172.21.15.91:0/3408042843 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240309a0 con 0x7f993415d490 2024-02-02T13:24:47.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f9932a00640 1 -- 172.21.15.91:0/3408042843 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924030d30 con 0x7f993415d490 2024-02-02T13:24:47.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f993a400640 1 -- 172.21.15.91:0/3408042843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993415d490 msgr2=0x7f993406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.813+0000 7f993a400640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993415d490 0x7f993406c170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9924009870 tx=0x7f992402b980 comp rx=0 tx=0).stop 2024-02-02T13:24:47.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 -- 172.21.15.91:0/3408042843 shutdown_connections 2024-02-02T13:24:47.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99340ac8d0 0x7f99340acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 --2- 172.21.15.91:0/3408042843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993415d490 0x7f993406c170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 -- 172.21.15.91:0/3408042843 >> 172.21.15.91:0/3408042843 conn(0x7f99340cec80 msgr2=0x7f99340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:47.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 -- 172.21.15.91:0/3408042843 shutdown_connections 2024-02-02T13:24:47.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.814+0000 7f993a400640 1 -- 172.21.15.91:0/3408042843 wait complete. 2024-02-02T13:24:47.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.815+0000 7f993a400640 1 Processor -- start 2024-02-02T13:24:47.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.815+0000 7f993a400640 1 -- start start 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99340b0700 0x7f993415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f993415d490 0x7f99340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993415f2f0 con 0x7f99340ac8d0 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993415ed90 con 0x7f99340b0700 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.816+0000 7f993a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993415f010 con 0x7f993415d490 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53214/0 (socket says 172.21.15.91:53214) 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 -- 172.21.15.91:0/2518302698 learned_addr learned my addr 172.21.15.91:0/2518302698 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 -- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f993415d490 msgr2=0x7f99340b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f993415d490 0x7f99340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 -- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99340b0700 msgr2=0x7f993415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99340b0700 0x7f993415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 -- 172.21.15.91:0/2518302698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240094d0 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f9933400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9924000c00 tx=0x7f9924004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924039c90 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9924039df0 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.817+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924043640 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.818+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99340c3fa0 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.818+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99340c4560 con 0x7f99340ac8d0 2024-02-02T13:24:47.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.819+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99340c3e00 con 0x7f99340ac8d0 2024-02-02T13:24:47.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.820+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f99240437a0 con 0x7f99340ac8d0 2024-02-02T13:24:47.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.820+0000 7f992b400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 0x7f990c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:47.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.821+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f992403e070 con 0x7f99340ac8d0 2024-02-02T13:24:47.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.821+0000 7f992be00640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 0x7f990c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:47.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.821+0000 7f992be00640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 0x7f990c06c8f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f991c006fd0 tx=0x7f991c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:47.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:47.825+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f99340c3e00 con 0x7f99340ac8d0 2024-02-02T13:24:48.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.002+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99340c3e00 con 0x7f99340ac8d0 2024-02-02T13:24:48.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.004+0000 7f992b400640 1 -- 172.21.15.91:0/2518302698 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f99340c3e00 con 0x7f99340ac8d0 2024-02-02T13:24:48.005 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:48.006 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 msgr2=0x7f990c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 0x7f990c06c8f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f991c006fd0 tx=0x7f991c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 msgr2=0x7f99340b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9924000c00 tx=0x7f9924004730 comp rx=0 tx=0).stop 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 shutdown_connections 2024-02-02T13:24:48.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.006+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f990c06a430 0x7f990c06c8f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f993415d490 0x7f99340b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99340b0700 0x7f993415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 --2- 172.21.15.91:0/2518302698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99340ac8d0 0x7f99340b1870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 >> 172.21.15.91:0/2518302698 conn(0x7f99340cec80 msgr2=0x7f99340b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 shutdown_connections 2024-02-02T13:24:48.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.007+0000 7f993a400640 1 -- 172.21.15.91:0/2518302698 wait complete. 2024-02-02T13:24:48.026 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-02-02T13:24:48.026 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:48.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.225+0000 7f5b25000640 1 Processor -- start 2024-02-02T13:24:48.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.225+0000 7f5b25000640 1 -- start start 2024-02-02T13:24:48.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b25000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b200b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b25000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b200b0c40 0x7f5b200b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b25000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2015fa00 con 0x7f5b20151f90 2024-02-02T13:24:48.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b25000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b200b10d0 con 0x7f5b200b0c40 2024-02-02T13:24:48.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b25000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20070f80 con 0x7f5b2015f5a0 2024-02-02T13:24:48.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b1de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5b20151f90 0x7f5b2006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34094/0 (socket says 172.21.15.91:34094) 2024-02-02T13:24:48.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b1de00640 1 -- 172.21.15.91:0/2980269499 learned_addr learned my addr 172.21.15.91:0/2980269499 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b1e800640 1 --2- 172.21.15.91:0/2980269499 >> v2:172.21.15.113:3300/0 conn(0x7f5b200b0c40 0x7f5b200b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b17e00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b200b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676395499 0 0) 0x7f5b10001110 con 0x7f5b20151f90 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.227+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b00003520 con 0x7f5b20151f90 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24904432 0 0) 0x7f5b00003520 con 0x7f5b20151f90 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 msgr2=0x7f5b200b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b200b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 >> v2:172.21.15.113:3300/0 conn(0x7f5b200b0c40 msgr2=0x7f5b200b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 --2- 172.21.15.91:0/2980269499 >> v2:172.21.15.113:3300/0 conn(0x7f5b200b0c40 0x7f5b200b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20152370 con 0x7f5b20151f90 2024-02-02T13:24:48.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5b10001800 con 0x7f5b20151f90 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 >> v1:172.21.15.91:6789/0 conn(0x7f5b20151f90 legacy=0x7f5b2006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b00003f40 0x7f5b000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200b10d0 con 0x7f5b00003f40 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20070f80 con 0x7f5b00004d00 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0000b980 con 0x7f5b2015f5a0 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b17e00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b00003f40 0x7f5b000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.228+0000 7f5b1de00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b1e800640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b17e00640 1 -- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 msgr2=0x7f5b00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b17e00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015f5a0 0x7f5b00003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b17e00640 1 -- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00004d00 msgr2=0x7f5b00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b17e00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00004d00 0x7f5b00005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.229+0000 7f5b17e00640 1 -- 172.21.15.91:0/2980269499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00003520 con 0x7f5b00003f40 2024-02-02T13:24:48.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b17e00640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b00003f40 0x7f5b000047c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009af0 tx=0x7f5b000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e5872d889d2aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0802c6b0 con 0x7f5b00003f40 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0802c810 con 0x7f5b00003f40 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b1d400640 1 -- 172.21.15.91:0/2980269499 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0802cba0 con 0x7f5b00003f40 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b25000640 1 -- 172.21.15.91:0/2980269499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b00003f40 msgr2=0x7f5b000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b25000640 1 --2- 172.21.15.91:0/2980269499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b00003f40 0x7f5b000047c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009af0 tx=0x7f5b000057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.230+0000 7f5b1e800640 1 -- 172.21.15.91:0/2980269499 reap_dead start 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.231+0000 7f5b25000640 1 -- 172.21.15.91:0/2980269499 shutdown_connections 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.231+0000 7f5b25000640 1 -- 172.21.15.91:0/2980269499 >> 172.21.15.91:0/2980269499 conn(0x7f5b200cec80 msgr2=0x7f5b200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.231+0000 7f5b25000640 1 -- 172.21.15.91:0/2980269499 shutdown_connections 2024-02-02T13:24:48.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.231+0000 7f5b25000640 1 -- 172.21.15.91:0/2980269499 wait complete. 2024-02-02T13:24:48.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.232+0000 7f5b25000640 1 Processor -- start 2024-02-02T13:24:48.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 -- start start 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 0x7f5b2014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015df50 0x7f5b200c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2014e7e0 con 0x7f5b200b0170 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200ce140 con 0x7f5b00003f40 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.233+0000 7f5b25000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200b18a0 con 0x7f5b2015df50 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53234/0 (socket says 172.21.15.91:53234) 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 -- 172.21.15.91:0/3716624190 learned_addr learned my addr 172.21.15.91:0/3716624190 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b1de00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 0x7f5b2014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b1e800640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015df50 0x7f5b200c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 -- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015df50 msgr2=0x7f5b200c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015df50 0x7f5b200c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 -- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 msgr2=0x7f5b2014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 0x7f5b2014e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 -- 172.21.15.91:0/3716624190 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200c36b0 con 0x7f5b200b0170 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b1de00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 0x7f5b2014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.234+0000 7f5b17e00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b08037040 tx=0x7f5b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.235+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08003810 con 0x7f5b200b0170 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.235+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08003970 con 0x7f5b200b0170 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.235+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08003d00 con 0x7f5b200b0170 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.235+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b200c3990 con 0x7f5b200b0170 2024-02-02T13:24:48.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.235+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b200c3f50 con 0x7f5b200b0170 2024-02-02T13:24:48.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.236+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b200c40e0 con 0x7f5b200b0170 2024-02-02T13:24:48.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.237+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5b08040450 con 0x7f5b200b0170 2024-02-02T13:24:48.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.238+0000 7f5b16000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 0x7f5af806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.238+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b080aa790 con 0x7f5b200b0170 2024-02-02T13:24:48.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.239+0000 7f5b1de00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 0x7f5af806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.239+0000 7f5b1de00640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 0x7f5af806c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000fb0 tx=0x7f5b10001d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.243+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5b08075d70 con 0x7f5b200b0170 2024-02-02T13:24:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.434+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b20066cf0 con 0x7f5b200b0170 2024-02-02T13:24:48.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.437+0000 7f5b16000640 1 -- 172.21.15.91:0/3716624190 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f5b08003e60 con 0x7f5b200b0170 2024-02-02T13:24:48.439 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:48.439 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:48.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.439+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 msgr2=0x7f5af806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.439+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 0x7f5af806c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000fb0 tx=0x7f5b10001d40 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 msgr2=0x7f5b2015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b08037040 tx=0x7f5b08004730 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 shutdown_connections 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5af806a460 0x7f5af806c920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2015df50 0x7f5b200c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b200b0170 0x7f5b2015da10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 --2- 172.21.15.91:0/3716624190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b00003f40 0x7f5b2014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 >> 172.21.15.91:0/3716624190 conn(0x7f5b200cec80 msgr2=0x7f5b200c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 shutdown_connections 2024-02-02T13:24:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.440+0000 7f5b25000640 1 -- 172.21.15.91:0/3716624190 wait complete. 2024-02-02T13:24:48.462 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-02-02T13:24:48.462 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-02-02T13:24:48.462 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-02-02T13:24:48.463 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-02-02T13:24:48.463 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-02-02T13:24:48.464 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-02-02T13:24:48.464 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-02-02T13:24:48.464 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-02-02T13:24:48.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.751+0000 7f65af400640 1 Processor -- start 2024-02-02T13:24:48.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.752+0000 7f9676e00640 1 Processor -- start 2024-02-02T13:24:48.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.751+0000 7f65af400640 1 -- start start 2024-02-02T13:24:48.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.753+0000 7f65af400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65a80b0700 0x7f65a80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.753+0000 7f65af400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a80ccdd0 con 0x7f65a8161660 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.752+0000 7f9676e00640 1 -- start start 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.754+0000 7f9676e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f96700b0b90 0x7f96700cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.754+0000 7f9676e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700ccdd0 con 0x7f96700b0700 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.754+0000 7f9676e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700cc7c0 con 0x7f9670161660 2024-02-02T13:24:48.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.754+0000 7f9676e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700c9570 con 0x7f96700b0b90 2024-02-02T13:24:48.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.753+0000 7f65af400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a80cc7c0 con 0x7f65a80cd260 2024-02-02T13:24:48.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.753+0000 7f65af400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a80b10d0 con 0x7f65a80b0700 2024-02-02T13:24:48.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f9674e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f96700b0b90 0x7f96700cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f966f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f96700b0700 0x7f96700cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34120/0 (socket says 172.21.15.91:34120) 2024-02-02T13:24:48.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f65aca00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f65a8161660 0x7f65a806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34106/0 (socket says 172.21.15.91:34106) 2024-02-02T13:24:48.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f65aca00640 1 -- 172.21.15.91:0/2081960978 learned_addr learned my addr 172.21.15.91:0/2081960978 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:48.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f9674e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f96700b0b90 0x7f96700cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53236/0 (socket says 172.21.15.91:53236) 2024-02-02T13:24:48.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.755+0000 7f9674e00640 1 -- 172.21.15.91:0/2866187476 learned_addr learned my addr 172.21.15.91:0/2866187476 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f65a7e00640 1 --2- 172.21.15.91:0/2081960978 >> v2:172.21.15.113:3300/0 conn(0x7f65a80b0700 0x7f65a80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f9674e00640 1 -- 172.21.15.91:0/2866187476 >> v1:172.21.15.113:6789/0 conn(0x7f9670161660 legacy=0x7f967006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f9674e00640 1 -- 172.21.15.91:0/2866187476 >> v1:172.21.15.91:6789/0 conn(0x7f96700b0700 legacy=0x7f96700cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f9674e00640 1 -- 172.21.15.91:0/2866187476 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700c9850 con 0x7f96700b0b90 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f65a7e00640 1 -- 172.21.15.91:0/2081960978 >> v1:172.21.15.113:6789/0 conn(0x7f65a80cd260 legacy=0x7f65a80cd690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f65a7e00640 1 -- 172.21.15.91:0/2081960978 >> v1:172.21.15.91:6789/0 conn(0x7f65a8161660 legacy=0x7f65a806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.757+0000 7f65a7e00640 1 -- 172.21.15.91:0/2081960978 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a80c9570 con 0x7f65a80b0700 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.756+0000 7f65a7400640 1 -- 172.21.15.91:0/2081960978 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350969014 0 0) 0x7f65a80ccdd0 con 0x7f65a8161660 2024-02-02T13:24:48.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.757+0000 7f9674e00640 1 --2- 172.21.15.91:0/2866187476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96700b0b90 0x7f96700cd970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9658009870 tx=0x7f965802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e3fe3fd7039919 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f6376c00640 1 Processor -- start 2024-02-02T13:24:48.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f6376c00640 1 -- start start 2024-02-02T13:24:48.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f65a7e00640 1 --2- 172.21.15.91:0/2081960978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 0x7f65a80b0b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65900097a0 tx=0x7f659002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543e7d1354127f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f966ea00640 1 -- 172.21.15.91:0/2866187476 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658031870 con 0x7f96700b0b90 2024-02-02T13:24:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f966ea00640 1 -- 172.21.15.91:0/2866187476 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96580319d0 con 0x7f96700b0b90 2024-02-02T13:24:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.758+0000 7f966ea00640 1 -- 172.21.15.91:0/2866187476 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658031d60 con 0x7f96700b0b90 2024-02-02T13:24:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.759+0000 7fe66d200640 1 Processor -- start 2024-02-02T13:24:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.759+0000 7f65a7400640 1 -- 172.21.15.91:0/2081960978 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659002e070 con 0x7f65a80b0700 2024-02-02T13:24:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.759+0000 7fe66d200640 1 -- start start 2024-02-02T13:24:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.759+0000 7f65a7400640 1 -- 172.21.15.91:0/2081960978 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65900315a0 con 0x7f65a80b0700 2024-02-02T13:24:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.759+0000 7f65a7400640 1 -- 172.21.15.91:0/2081960978 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590031930 con 0x7f65a80b0700 2024-02-02T13:24:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f9676e00640 1 -- 172.21.15.91:0/2866187476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96700b0b90 msgr2=0x7f96700cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f9676e00640 1 --2- 172.21.15.91:0/2866187476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96700b0b90 0x7f96700cd970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9658009870 tx=0x7f965802b910 comp rx=0 tx=0).stop 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f6376c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6370070980 0x7f63700b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f6376c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63700ce210 con 0x7f6370160830 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f6376c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63700ce4f0 con 0x7f63700b1d70 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.760+0000 7f6376c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63700ce7d0 con 0x7f6370070980 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6600e78b0 0x7fe66004f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f9676e00640 1 -- 172.21.15.91:0/2866187476 shutdown_connections 2024-02-02T13:24:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f9676e00640 1 --2- 172.21.15.91:0/2866187476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96700b0b90 0x7f96700cd970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f9676e00640 1 -- 172.21.15.91:0/2866187476 >> 172.21.15.91:0/2866187476 conn(0x7f96700cec80 msgr2=0x7f96700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f636f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6370070980 0x7f63700b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f65af400640 1 -- 172.21.15.91:0/2081960978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 msgr2=0x7f65a80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f65af400640 1 --2- 172.21.15.91:0/2081960978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 0x7f65a80b0b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65900097a0 tx=0x7f659002b840 comp rx=0 tx=0).stop 2024-02-02T13:24:48.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f9676e00640 1 -- 172.21.15.91:0/2866187476 shutdown_connections 2024-02-02T13:24:48.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f636f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6370070980 0x7f63700b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53252/0 (socket says 172.21.15.91:53252) 2024-02-02T13:24:48.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7f636f400640 1 -- 172.21.15.91:0/3563294870 learned_addr learned my addr 172.21.15.91:0/3563294870 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 0x7fe6600eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe660052fe0 0x7fe660053470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe660050340 con 0x7fe660052fe0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe660006480 con 0x7fe6600e78b0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.761+0000 7fe66d200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600ebc40 con 0x7fe6600eb2c0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe667e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe660052fe0 0x7fe660053470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe666a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 0x7fe6600eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe666a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 0x7fe6600eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38604/0 (socket says 172.21.15.91:38604) 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe666a00640 1 -- 172.21.15.91:0/2139645422 learned_addr learned my addr 172.21.15.91:0/2139645422 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe667400640 1 --2- 172.21.15.91:0/2139645422 >> v2:172.21.15.113:3300/0 conn(0x7fe6600e78b0 0x7fe66004f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f636f400640 1 -- 172.21.15.91:0/3563294870 >> v1:172.21.15.113:6789/0 conn(0x7f63700b1d70 legacy=0x7f63700d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f636f400640 1 -- 172.21.15.91:0/3563294870 >> v1:172.21.15.91:6789/0 conn(0x7f6370160830 legacy=0x7f6370070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f9676e00640 1 -- 172.21.15.91:0/2866187476 wait complete. 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f65af400640 1 -- 172.21.15.91:0/2081960978 shutdown_connections 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f65af400640 1 --2- 172.21.15.91:0/2081960978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 0x7f65a80b0b90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f65af400640 1 -- 172.21.15.91:0/2081960978 >> 172.21.15.91:0/2081960978 conn(0x7f65a80cec80 msgr2=0x7f65a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe667e00640 1 -- 172.21.15.91:0/2139645422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 msgr2=0x7fe6600eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7fe667e00640 1 --2- 172.21.15.91:0/2139645422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 0x7fe6600eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.762+0000 7f636f400640 1 -- 172.21.15.91:0/3563294870 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700d17b0 con 0x7f6370070980 2024-02-02T13:24:48.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.763+0000 7f65af400640 1 -- 172.21.15.91:0/2081960978 shutdown_connections 2024-02-02T13:24:48.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.763+0000 7fe667e00640 1 -- 172.21.15.91:0/2139645422 >> v2:172.21.15.113:3300/0 conn(0x7fe6600e78b0 msgr2=0x7fe66004f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.763+0000 7fe667e00640 1 --2- 172.21.15.91:0/2139645422 >> v2:172.21.15.113:3300/0 conn(0x7fe6600e78b0 0x7fe66004f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.763+0000 7fe667e00640 1 -- 172.21.15.91:0/2139645422 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66004fd80 con 0x7fe660052fe0 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.763+0000 7f70fa200640 1 Processor -- start 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.764+0000 7f70fa200640 1 -- start start 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.764+0000 7fe667e00640 1 --2- 172.21.15.91:0/2139645422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe660052fe0 0x7fe660053470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bd70 tx=0x7fe65002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbc1f9db092d621 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f70fa200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f70fa200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f70f40b0700 0x7f70f40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f70fa200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40b10a0 con 0x7f70f4165200 2024-02-02T13:24:48.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f70fa200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40ccdd0 con 0x7f70f40b0700 2024-02-02T13:24:48.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f70fa200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40cc7c0 con 0x7f70f4161660 2024-02-02T13:24:48.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7fe666000640 1 -- 172.21.15.91:0/2139645422 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6500305f0 con 0x7fe660052fe0 2024-02-02T13:24:48.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7fe666000640 1 -- 172.21.15.91:0/2139645422 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650030750 con 0x7fe660052fe0 2024-02-02T13:24:48.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7fe666000640 1 -- 172.21.15.91:0/2139645422 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650030ae0 con 0x7fe660052fe0 2024-02-02T13:24:48.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f636f400640 1 --2- 172.21.15.91:0/3563294870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6370070980 0x7f63700b61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6358009870 tx=0x7f635802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6ffae503c26927 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f70f3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f70f3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38618/0 (socket says 172.21.15.91:38618) 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f70f3400640 1 -- 172.21.15.91:0/2502136317 learned_addr learned my addr 172.21.15.91:0/2502136317 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f70f2a00640 1 --2- 172.21.15.91:0/2502136317 >> v2:172.21.15.91:3300/0 conn(0x7f70f40b0700 0x7f70f40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.764+0000 7f65af400640 1 -- 172.21.15.91:0/2081960978 wait complete. 2024-02-02T13:24:48.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f9676e00640 1 Processor -- start 2024-02-02T13:24:48.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f65af400640 1 Processor -- start 2024-02-02T13:24:48.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f9676e00640 1 -- start start 2024-02-02T13:24:48.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f636ea00640 1 -- 172.21.15.91:0/3563294870 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358004230 con 0x7f6370070980 2024-02-02T13:24:48.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.765+0000 7f636ea00640 1 -- 172.21.15.91:0/3563294870 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635802e050 con 0x7f6370070980 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f636ea00640 1 -- 172.21.15.91:0/3563294870 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358004230 con 0x7f6370070980 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f6376c00640 1 -- 172.21.15.91:0/3563294870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6370070980 msgr2=0x7f63700b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.766+0000 7f6376c00640 1 --2- 172.21.15.91:0/3563294870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6370070980 0x7f63700b61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6358009870 tx=0x7f635802b7e0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f6376c00640 1 -- 172.21.15.91:0/3563294870 shutdown_connections 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f6376c00640 1 --2- 172.21.15.91:0/3563294870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6370070980 0x7f63700b61d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f6376c00640 1 -- 172.21.15.91:0/3563294870 >> 172.21.15.91:0/3563294870 conn(0x7f63700d3030 msgr2=0x7f63700d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f6376c00640 1 -- 172.21.15.91:0/3563294870 shutdown_connections 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f65af400640 1 -- start start 2024-02-02T13:24:48.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f6376c00640 1 -- 172.21.15.91:0/3563294870 wait complete. 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f70f3400640 1 -- 172.21.15.91:0/2502136317 >> v2:172.21.15.91:3300/0 conn(0x7f70f40b0700 msgr2=0x7f70f40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f70f3400640 1 --2- 172.21.15.91:0/2502136317 >> v2:172.21.15.91:3300/0 conn(0x7f70f40b0700 0x7f70f40b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f70f3400640 1 -- 172.21.15.91:0/2502136317 >> v1:172.21.15.113:6789/0 conn(0x7f70f4165200 legacy=0x7f70f4165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7f70f3400640 1 -- 172.21.15.91:0/2502136317 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f406c540 con 0x7f70f4161660 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 -- 172.21.15.91:0/2139645422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe660052fe0 msgr2=0x7fe660053470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 --2- 172.21.15.91:0/2139645422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe660052fe0 0x7fe660053470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bd70 tx=0x7fe65002b940 comp rx=0 tx=0).stop 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 -- 172.21.15.91:0/2139645422 shutdown_connections 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 --2- 172.21.15.91:0/2139645422 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe660052fe0 0x7fe660053470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 --2- 172.21.15.91:0/2139645422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600eb2c0 0x7fe6600eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 --2- 172.21.15.91:0/2139645422 >> v2:172.21.15.113:3300/0 conn(0x7fe6600e78b0 0x7fe66004f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.767+0000 7fe66d200640 1 -- 172.21.15.91:0/2139645422 >> 172.21.15.91:0/2139645422 conn(0x7fe660054c10 msgr2=0x7fe660054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 0x7f65a80c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a80cd260 0x7f65a80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 0x7f65a80b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b2db0 con 0x7f65a8161660 2024-02-02T13:24:48.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b3090 con 0x7f65a80b0700 2024-02-02T13:24:48.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.768+0000 7f65af400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80b3370 con 0x7f65a80cd260 2024-02-02T13:24:48.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7fe66d200640 1 -- 172.21.15.91:0/2139645422 shutdown_connections 2024-02-02T13:24:48.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f70f3400640 1 --2- 172.21.15.91:0/2502136317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009870 tx=0x7f70e402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a8cec6649d19d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7ff764200640 1 Processor -- start 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f65ad400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 0x7f65a80b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f65a7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a80cd260 0x7f65a80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f65a7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a80cd260 0x7f65a80b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38624/0 (socket says 172.21.15.91:38624) 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f65a7e00640 1 -- 172.21.15.91:0/2818272765 learned_addr learned my addr 172.21.15.91:0/2818272765 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f65ad400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 0x7f65a80b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53276/0 (socket says 172.21.15.91:53276) 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f6376c00640 1 Processor -- start 2024-02-02T13:24:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7ff764200640 1 -- start start 2024-02-02T13:24:48.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f6376c00640 1 -- start start 2024-02-02T13:24:48.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7f70f2000640 1 -- 172.21.15.91:0/2502136317 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4031640 con 0x7f70f4161660 2024-02-02T13:24:48.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f70f2000640 1 -- 172.21.15.91:0/2502136317 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e40317a0 con 0x7f70f4161660 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f65ad400640 1 -- 172.21.15.91:0/2818272765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a80cd260 msgr2=0x7f65a80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f65ad400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a80cd260 0x7f65a80b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f65ad400640 1 -- 172.21.15.91:0/2818272765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 msgr2=0x7f65a80c3f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f65ad400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a80b0700 0x7f65a80c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f9676e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96700b0700 0x7f967015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f65ad400640 1 -- 172.21.15.91:0/2818272765 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009400 con 0x7f65a8161660 2024-02-02T13:24:48.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f9676e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96700b0b90 0x7f96700c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f70f2000640 1 -- 172.21.15.91:0/2502136317 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4031b30 con 0x7f70f4161660 2024-02-02T13:24:48.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f9676e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9670161660 0x7f96701511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.769+0000 7fe66d200640 1 -- 172.21.15.91:0/2139645422 wait complete. 2024-02-02T13:24:48.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f9676e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967015da30 con 0x7f9670161660 2024-02-02T13:24:48.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f9676e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c6630 con 0x7f96700b0700 2024-02-02T13:24:48.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.770+0000 7f15fb200640 1 Processor -- start 2024-02-02T13:24:48.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f15fb200640 1 -- start start 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 -- 172.21.15.91:0/2502136317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 msgr2=0x7f70f406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 --2- 172.21.15.91:0/2502136317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009870 tx=0x7f70e402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f65ad400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 0x7f65a80b2740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65a000be20 tx=0x7f65a000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f966f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96700b0b90 0x7f96700c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7ff764200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7ff764200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0cd930 con 0x7ff75c0b6260 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7ff764200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0d1680 con 0x7ff75c0b2260 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7ff764200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0b8f90 con 0x7ff75c0d0070 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 -- 172.21.15.91:0/2502136317 shutdown_connections 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 --2- 172.21.15.91:0/2502136317 >> v2:172.21.15.91:3300/0 conn(0x7f70f40b0700 0x7f70f40b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 --2- 172.21.15.91:0/2502136317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4161660 0x7f70f406c000 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7fe66d200640 1 Processor -- start 2024-02-02T13:24:48.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f70fa200640 1 -- 172.21.15.91:0/2502136317 >> 172.21.15.91:0/2502136317 conn(0x7f70f40cec80 msgr2=0x7f70f40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000cdf0 con 0x7f65a8161660 2024-02-02T13:24:48.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0014440 con 0x7f65a8161660 2024-02-02T13:24:48.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0014830 con 0x7f65a8161660 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f966f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96700b0b90 0x7f96700c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38638/0 (socket says 172.21.15.91:38638) 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f966f400640 1 -- 172.21.15.91:0/1494379240 learned_addr learned my addr 172.21.15.91:0/1494379240 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.771+0000 7f9676e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c6910 con 0x7f96700b0b90 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f966fe00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96700b0700 0x7f967015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7ff761800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f9674e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9670161660 0x7f96701511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a80b3650 con 0x7f65a8161660 2024-02-02T13:24:48.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a80b43d0 con 0x7f65a8161660 2024-02-02T13:24:48.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7fe66d200640 1 -- start start 2024-02-02T13:24:48.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7f6376c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6370070980 0x7f637015e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7ff761800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38654/0 (socket says 172.21.15.91:38654) 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.772+0000 7ff761800640 1 -- 172.21.15.91:0/1458820173 learned_addr learned my addr 172.21.15.91:0/1458820173 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f15fb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f15fb200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f40d2cb0 con 0x7f15f4164d20 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f15fb200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f40b5b90 con 0x7f15f41652f0 2024-02-02T13:24:48.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f15fb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4165100 con 0x7f15f40d06e0 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f9674e00640 1 -- 172.21.15.91:0/1494379240 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96700b0b90 msgr2=0x7f96700c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f9674e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96700b0b90 0x7f96700c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f9674e00640 1 -- 172.21.15.91:0/1494379240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96700b0700 msgr2=0x7f967015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f9674e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96700b0700 0x7f967015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f9674e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580094d0 con 0x7f9670161660 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f636fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6370070980 0x7f637015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f636fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6370070980 0x7f637015e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38668/0 (socket says 172.21.15.91:38668) 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f6376c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63700b1d70 0x7f637015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7fe66d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe660052fe0 0x7fe6600e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7fe66d200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 0x7fe66017b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f15f9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f15f9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38670/0 (socket says 172.21.15.91:38670) 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f15f9200640 1 -- 172.21.15.91:0/2117205476 learned_addr learned my addr 172.21.15.91:0/2117205476 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7fe66d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6600eb2c0 0x7fe66017ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7fe66d200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600e5b70 con 0x7fe6600eb2c0 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7fe66d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600e5e50 con 0x7fe660052fe0 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7fe66d200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600e6130 con 0x7fe6600e78b0 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f6376c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6370160830 0x7f637015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f6376c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370158970 con 0x7f63700b1d70 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f636fe00640 1 -- 172.21.15.91:0/2218397387 learned_addr learned my addr 172.21.15.91:0/2218397387 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f6376c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637006be60 con 0x7f6370160830 2024-02-02T13:24:48.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637006ccc0 con 0x7f6370070980 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7ff75be00640 1 -- 172.21.15.91:0/1458820173 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569327304 0 0) 0x7ff74c001110 con 0x7ff75c0b6260 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7ff75be00640 1 -- 172.21.15.91:0/1458820173 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff73c003520 con 0x7ff75c0b6260 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f636f400640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63700b1d70 0x7f637015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f6374c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6370160830 0x7f637015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f9674e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9670161660 0x7f96701511b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9658033000 tx=0x7f96580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7fe666a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 0x7fe66017b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7ff761800640 1 -- 172.21.15.91:0/1458820173 >> v1:172.21.15.113:6789/0 conn(0x7ff75c0b2260 legacy=0x7ff75c0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7ff761800640 1 -- 172.21.15.91:0/1458820173 >> v1:172.21.15.91:6789/0 conn(0x7ff75c0b6260 legacy=0x7ff75c0d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7ff761800640 1 -- 172.21.15.91:0/1458820173 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0d1960 con 0x7ff75c0d0070 2024-02-02T13:24:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7fe666a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 0x7fe66017b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38682/0 (socket says 172.21.15.91:38682) 2024-02-02T13:24:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7fe666a00640 1 -- 172.21.15.91:0/420007934 learned_addr learned my addr 172.21.15.91:0/420007934 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f15f9200640 1 -- 172.21.15.91:0/2117205476 >> v1:172.21.15.113:6789/0 conn(0x7f15f41652f0 legacy=0x7f15f40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f15f9200640 1 -- 172.21.15.91:0/2117205476 >> v1:172.21.15.91:6789/0 conn(0x7f15f4164d20 legacy=0x7f15f40b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f15f9200640 1 -- 172.21.15.91:0/2117205476 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f41657d0 con 0x7f15f40d06e0 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965803add0 con 0x7f9670161660 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96580443f0 con 0x7f9670161660 2024-02-02T13:24:48.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658044780 con 0x7f9670161660 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7fe667400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe660052fe0 0x7fe6600e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7fe667e00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6600eb2c0 0x7fe66017ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 -- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6370070980 msgr2=0x7f637015e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6370070980 0x7f637015e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 -- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6370160830 msgr2=0x7f637015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6370160830 0x7f637015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580094d0 con 0x7f63700b1d70 2024-02-02T13:24:48.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.775+0000 7f636f400640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63700b1d70 0x7f637015c090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6358004000 tx=0x7f63580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.773+0000 7f70fa200640 1 -- 172.21.15.91:0/2502136317 shutdown_connections 2024-02-02T13:24:48.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.774+0000 7f70fa200640 1 -- 172.21.15.91:0/2502136317 wait complete. 2024-02-02T13:24:48.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fcc9a400640 1 Processor -- start 2024-02-02T13:24:48.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358030710 con 0x7f63700b1d70 2024-02-02T13:24:48.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe666a00640 1 -- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe660052fe0 msgr2=0x7fe6600e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe666a00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe660052fe0 0x7fe6600e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637015d2c0 con 0x7f63700b1d70 2024-02-02T13:24:48.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96700c6bf0 con 0x7f9670161660 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700c6200 con 0x7f9670161660 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe666a00640 1 -- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6600eb2c0 msgr2=0x7fe66017ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe666a00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6600eb2c0 0x7fe66017ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe666a00640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650009400 con 0x7fe6600e78b0 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637006c390 con 0x7f63700b1d70 2024-02-02T13:24:48.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f70fa200640 1 Processor -- start 2024-02-02T13:24:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358030870 con 0x7f63700b1d70 2024-02-02T13:24:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358030710 con 0x7f63700b1d70 2024-02-02T13:24:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fe667400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe660052fe0 0x7fe6600e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.777+0000 7fe666a00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 0x7fe66017b4c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe66804f7d0 tx=0x7fe6680741e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.777+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668092630 con 0x7fe6600e78b0 2024-02-02T13:24:48.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f65a0014990 con 0x7f65a8161660 2024-02-02T13:24:48.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6600e6410 con 0x7fe6600e78b0 2024-02-02T13:24:48.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668092790 con 0x7fe6600e78b0 2024-02-02T13:24:48.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6340000fc0 con 0x7f63700b1d70 2024-02-02T13:24:48.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7ff761800640 1 --2- 172.21.15.91:0/1458820173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff754009e80 tx=0x7ff75402bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f59a6fc705ddc4c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.776+0000 7fcc9a400640 1 -- start start 2024-02-02T13:24:48.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fcc9a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 0x7fcc9406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668092b40 con 0x7fe6600e78b0 2024-02-02T13:24:48.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fcc9a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcc940ce270 0x7fcc940b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fcc9a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc940b10a0 con 0x7fcc940b0700 2024-02-02T13:24:48.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fcc9a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940ccdd0 con 0x7fcc940ce270 2024-02-02T13:24:48.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fcc9a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940cc7c0 con 0x7fcc94161660 2024-02-02T13:24:48.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.778+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6600e3560 con 0x7fe6600e78b0 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7ff75be00640 1 -- 172.21.15.91:0/1458820173 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75402cb70 con 0x7ff75c0d0070 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7ff75be00640 1 -- 172.21.15.91:0/1458820173 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75402ccd0 con 0x7ff75c0d0070 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7ff75be00640 1 -- 172.21.15.91:0/1458820173 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754034610 con 0x7ff75c0d0070 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7fcc93e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcc940ce270 0x7fcc940b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7fcc93e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcc940ce270 0x7fcc940b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53324/0 (socket says 172.21.15.91:53324) 2024-02-02T13:24:48.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7fcc93e00640 1 -- 172.21.15.91:0/911029213 learned_addr learned my addr 172.21.15.91:0/911029213 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.777+0000 7f70fa200640 1 -- start start 2024-02-02T13:24:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7f15f9200640 1 --2- 172.21.15.91:0/2117205476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098a0 tx=0x7f15e402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255f01b5eb574a3e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93400640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 0x7fcc9406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96700b0fc0 con 0x7f9670161660 2024-02-02T13:24:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93e00640 1 -- 172.21.15.91:0/911029213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 msgr2=0x7fcc9406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7f65a4c00640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f658006a430 0x7f658006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7ff764200640 1 -- 172.21.15.91:0/1458820173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 msgr2=0x7ff75c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65a0088960 con 0x7f65a8161660 2024-02-02T13:24:48.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7f65af400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f65a82021b0 0x7f65a8202590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 --> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65a80c9850 con 0x7f65a82021b0 2024-02-02T13:24:48.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6600e5fc0 con 0x7fe6600e78b0 2024-02-02T13:24:48.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7ff764200640 1 --2- 172.21.15.91:0/1458820173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff754009e80 tx=0x7ff75402bfd0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93e00640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 0x7fcc9406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93e00640 1 -- 172.21.15.91:0/911029213 >> v1:172.21.15.113:6789/0 conn(0x7fcc940b0700 legacy=0x7fcc94165200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93e00640 1 -- 172.21.15.91:0/911029213 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9406c540 con 0x7fcc940ce270 2024-02-02T13:24:48.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.779+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6358039050 con 0x7f63700b1d70 2024-02-02T13:24:48.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7f65a7e00640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f65a82021b0 0x7f65a8202590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7fcc93400640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 0x7fcc9406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:48.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.780+0000 7f65aca00640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f658006a430 0x7f658006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70f40b0700 0x7f70f41634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f4161660 0x7f70f40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 0x7f70f4151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f15f2a00640 1 -- 172.21.15.91:0/2117205476 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4031520 con 0x7f15f40d06e0 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f4164440 con 0x7f70f40b0700 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f4163e80 con 0x7f70f4161660 2024-02-02T13:24:48.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70fa200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f4164160 con 0x7f70f4165200 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70f3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70f40b0700 0x7f70f41634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7ff764200640 1 -- 172.21.15.91:0/1458820173 shutdown_connections 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7ff764200640 1 --2- 172.21.15.91:0/1458820173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0d0070 0x7ff75c0b8a50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7ff764200640 1 -- 172.21.15.91:0/1458820173 >> 172.21.15.91:0/1458820173 conn(0x7ff75c070660 msgr2=0x7ff75c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 0x7f70f4151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 0x7f70f4151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38704/0 (socket says 172.21.15.91:38704) 2024-02-02T13:24:48.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f70f3e00640 1 -- 172.21.15.91:0/3588488665 learned_addr learned my addr 172.21.15.91:0/3588488665 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7ff764200640 1 -- 172.21.15.91:0/1458820173 shutdown_connections 2024-02-02T13:24:48.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f2a00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f4161660 0x7f70f40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7ff764200640 1 -- 172.21.15.91:0/1458820173 wait complete. 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f6367e00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f634806a490 0x7f634806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63580aa310 con 0x7f63700b1d70 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f6367e00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7f6348072630 0x7f6348074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f636fe00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f634806a490 0x7f634806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7fcc93e00640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc940ce270 0x7fcc940b0b60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009e50 tx=0x7fcc8c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c2e07545347b8e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe66809b3b0 con 0x7fe6600e78b0 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7fcc92000640 1 -- 172.21.15.91:0/911029213 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c02ca80 con 0x7fcc940ce270 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.781+0000 7f65a7e00640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f65a82021b0 0x7f65a8202590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f3e00640 1 -- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f4161660 msgr2=0x7f70f40c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f3e00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f4161660 0x7f70f40c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f3e00640 1 -- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70f40b0700 msgr2=0x7f70f41634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f3e00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70f40b0700 0x7f70f41634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.782+0000 7f70f3e00640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40094d0 con 0x7f70f4165200 2024-02-02T13:24:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7fcc92000640 1 -- 172.21.15.91:0/911029213 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c02cbe0 con 0x7fcc940ce270 2024-02-02T13:24:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7fcc92000640 1 -- 172.21.15.91:0/911029213 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0343f0 con 0x7fcc940ce270 2024-02-02T13:24:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f15f2a00640 1 -- 172.21.15.91:0/2117205476 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4031680 con 0x7f15f40d06e0 2024-02-02T13:24:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f65aca00640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f658006a430 0x7f658006c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6598000c00 tx=0x7f65980079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f6374c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7f6348072630 0x7f6348074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6348075100 con 0x7f6348072630 2024-02-02T13:24:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== osd.0 v2:172.21.15.91:6820/32877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f6590006e10 con 0x7f65a82021b0 2024-02-02T13:24:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63580ac8f0 con 0x7f63700b1d70 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f70f2a00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f4161660 0x7f70f40c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f70f3e00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 0x7f70f4151860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c730 tx=0x7f70ec00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f70f3400640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70f40b0700 0x7f70f41634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe65f400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe64006a430 0x7fe64006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fcc9a400640 1 -- 172.21.15.91:0/911029213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc940ce270 msgr2=0x7fcc940b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fcc9a400640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc940ce270 0x7fcc940b0b60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009e50 tx=0x7fcc8c02bee0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6681034a0 con 0x7fe6600e78b0 2024-02-02T13:24:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f636fe00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f634806a490 0x7f634806c950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63700cc9a0 tx=0x7f6360009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15fb200640 1 -- 172.21.15.91:0/2117205476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 msgr2=0x7f15f40d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15f2a00640 1 -- 172.21.15.91:0/2117205476 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4031520 con 0x7f15f40d06e0 2024-02-02T13:24:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15fb200640 1 --2- 172.21.15.91:0/2117205476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098a0 tx=0x7f15e402b840 comp rx=0 tx=0).stop 2024-02-02T13:24:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.783+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f965804d450 con 0x7f9670161660 2024-02-02T13:24:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f9667e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f964806a480 0x7f964806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe65f400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7fe640072650 0x7fe640074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f965802d040 con 0x7f9670161660 2024-02-02T13:24:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe640075120 con 0x7fe640072650 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe668103850 con 0x7fe6600e78b0 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec006d50 con 0x7f70f4165200 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec003970 con 0x7f70f4165200 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec003ce0 con 0x7f70f4165200 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15fb200640 1 -- 172.21.15.91:0/2117205476 shutdown_connections 2024-02-02T13:24:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15fb200640 1 --2- 172.21.15.91:0/2117205476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40d3120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f15fb200640 1 -- 172.21.15.91:0/2117205476 >> 172.21.15.91:0/2117205476 conn(0x7f15f4070660 msgr2=0x7f15f4070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe667400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe64006a430 0x7fe64006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7fe667e00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7fe640072650 0x7fe640074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f6374c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7f6348072630 0x7f6348074a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f415d100 con 0x7f70f4165200 2024-02-02T13:24:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 -- 172.21.15.91:0/911029213 shutdown_connections 2024-02-02T13:24:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7f15fb200640 1 -- 172.21.15.91:0/2117205476 shutdown_connections 2024-02-02T13:24:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7f15fb200640 1 -- 172.21.15.91:0/2117205476 wait complete. 2024-02-02T13:24:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f9667e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7f96480726a0 0x7f9648074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9648075170 con 0x7f96480726a0 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f41525b0 con 0x7f70f4165200 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc940ce270 0x7fcc940b0b60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 --2- 172.21.15.91:0/911029213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94161660 0x7fcc9406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 -- 172.21.15.91:0/911029213 >> 172.21.15.91:0/911029213 conn(0x7fcc940cec80 msgr2=0x7fcc940cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 -- 172.21.15.91:0/911029213 shutdown_connections 2024-02-02T13:24:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fcc9a400640 1 -- 172.21.15.91:0/911029213 wait complete. 2024-02-02T13:24:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.784+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96580af240 con 0x7f9670161660 2024-02-02T13:24:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7f966fe00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f964806a480 0x7f964806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.786+0000 7f966f400640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7f96480726a0 0x7f9648074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.786+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70f40c9850 con 0x7f70f4165200 2024-02-02T13:24:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== osd.4 v2:172.21.15.113:6823/33080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f6348075100 con 0x7f6348072630 2024-02-02T13:24:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.785+0000 7fe667e00640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7fe640072650 0x7fe640074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== osd.3 v2:172.21.15.91:6800/32876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fe640075120 con 0x7fe640072650 2024-02-02T13:24:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7fe667400640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe64006a430 0x7fe64006c8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe65800ba50 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7f966fe00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f964806a480 0x7f964806c940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9660000f60 tx=0x7f9660009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7ff764200640 1 Processor -- start 2024-02-02T13:24:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.787+0000 7ff764200640 1 -- start start 2024-02-02T13:24:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.788+0000 7f966f400640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7f96480726a0 0x7f9648074ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.788+0000 7f15fb200640 1 Processor -- start 2024-02-02T13:24:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.788+0000 7f15fb200640 1 -- start start 2024-02-02T13:24:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.788+0000 7fcc9a400640 1 Processor -- start 2024-02-02T13:24:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7fcc9a400640 1 -- start start 2024-02-02T13:24:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15f4164d20 0x7f15f40ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7ff764200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0b2260 0x7ff75c0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7ff764200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 0x7ff75c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f41652f0 0x7f15f4065910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40c70e0 con 0x7f15f4164d20 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40cca50 con 0x7f15f41652f0 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15fb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41550c0 con 0x7f15f40d06e0 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.789+0000 7f15f3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15f4164d20 0x7f15f40ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f15f3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15f4164d20 0x7f15f40ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53328/0 (socket says 172.21.15.91:53328) 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f15f3400640 1 -- 172.21.15.91:0/2722137771 learned_addr learned my addr 172.21.15.91:0/2722137771 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f15f3e00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f15f9200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f41652f0 0x7f15f4065910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== osd.1 v2:172.21.15.91:6805/32878 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f9648075170 con 0x7f96480726a0 2024-02-02T13:24:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7ff764200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff75c0d0070 0x7ff75c0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7ff764200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c8590 con 0x7ff75c0b6260 2024-02-02T13:24:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7ff764200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c7980 con 0x7ff75c0d0070 2024-02-02T13:24:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 -- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f41652f0 msgr2=0x7f15f4065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f41652f0 0x7f15f4065910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 -- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15f4164d20 msgr2=0x7f15f40ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15f4164d20 0x7f15f40ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40094d0 con 0x7f15f40d06e0 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff764200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c7c60 con 0x7ff75c0b2260 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff762200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff75c0d0070 0x7ff75c0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc940b0700 0x7fcc940c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 0x7fcc94070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc94161660 0x7fcc94149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415d3d0 con 0x7fcc94161660 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940c2720 con 0x7fcc940b0700 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7fcc9a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940c2ce0 con 0x7fcc940ce270 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7f15f3e00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40cc510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cb20 tx=0x7f15ec0020a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff760e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 0x7ff75c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff760e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 0x7ff75c0c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53342/0 (socket says 172.21.15.91:53342) 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff760e00640 1 -- 172.21.15.91:0/2814593391 learned_addr learned my addr 172.21.15.91:0/2814593391 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.791+0000 7ff762200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff75c0d0070 0x7ff75c0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57306/0 (socket says 172.21.15.91:57306) 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.790+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f70ec0050d0 con 0x7f70f4165200 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7fcc92a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 0x7fcc94070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7fcc92a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 0x7fcc94070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38722/0 (socket says 172.21.15.91:38722) 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7fcc92a00640 1 -- 172.21.15.91:0/3616620835 learned_addr learned my addr 172.21.15.91:0/3616620835 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7fcc93400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc940b0700 0x7fcc940c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec003da0 con 0x7f15f40d06e0 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f70eb400640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70cc06a460 0x7f70cc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70ec0886e0 con 0x7f70f4165200 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f70eb400640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f70cc072600 0x7f70cc074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70cc0750d0 con 0x7f70cc072600 2024-02-02T13:24:48.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70ec088b30 con 0x7f70f4165200 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f40cb1a0 con 0x7f15f40d06e0 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f40caec0 con 0x7f15f40d06e0 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7ff760e00640 1 -- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0b2260 msgr2=0x7ff75c0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7ff760e00640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff75c0b2260 0x7ff75c0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7ff760e00640 1 -- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff75c0d0070 msgr2=0x7ff75c0c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7ff760e00640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff75c0d0070 0x7ff75c0c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7ff760e00640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009b30 con 0x7ff75c0b6260 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.792+0000 7fcc93e00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc94161660 0x7fcc94149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc92a00640 1 -- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc940b0700 msgr2=0x7fcc940c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc92a00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc940b0700 0x7fcc940c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc92a00640 1 -- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc94161660 msgr2=0x7fcc94149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec0050f0 con 0x7f15f40d06e0 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7f70f3400640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70cc06a460 0x7f70cc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec005460 con 0x7f15f40d06e0 2024-02-02T13:24:48.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc92a00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc94161660 0x7fcc94149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc92a00640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009b30 con 0x7fcc940ce270 2024-02-02T13:24:48.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.793+0000 7fcc93e00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc94161660 0x7fcc94149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:48.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.794+0000 7f70f2a00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f70cc072600 0x7f70cc074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.795+0000 7f70f3400640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70cc06a460 0x7f70cc06c920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70e4036000 tx=0x7f70e403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.794+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15c4000fc0 con 0x7f15f40d06e0 2024-02-02T13:24:48.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.795+0000 7f70f2a00640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f70cc072600 0x7f70cc074a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.794+0000 7ff760e00640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 0x7ff75c0c8de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c400 tx=0x7ff74400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.795+0000 7fcc92a00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 0x7fcc94070370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c480 tx=0x7fcc7c00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0052d0 con 0x7fcc940ce270 2024-02-02T13:24:48.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440052d0 con 0x7ff75c0b6260 2024-02-02T13:24:48.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c005430 con 0x7fcc940ce270 2024-02-02T13:24:48.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c012870 con 0x7fcc940ce270 2024-02-02T13:24:48.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744005430 con 0x7ff75c0b6260 2024-02-02T13:24:48.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744003830 con 0x7ff75c0b6260 2024-02-02T13:24:48.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc940c8180 con 0x7fcc940ce270 2024-02-02T13:24:48.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc940c2970 con 0x7fcc940ce270 2024-02-02T13:24:48.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== osd.2 v2:172.21.15.91:6816/32875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f70cc0750d0 con 0x7f70cc072600 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f15ec0055c0 con 0x7f15f40d06e0 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0c7f40 con 0x7ff75c0b6260 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.796+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0c7160 con 0x7ff75c0b6260 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.799+0000 7f15ebe00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f15cc06a500 0x7f15cc06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.799+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15ec088240 con 0x7f15f40d06e0 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.799+0000 7f15ebe00640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7f15cc072720 0x7f15cc074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.799+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15cc0751f0 con 0x7f15cc072720 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.799+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15ec08a860 con 0x7f15f40d06e0 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.800+0000 7f15f3400640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f15cc06a500 0x7f15cc06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.801+0000 7f15f3400640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f15cc06a500 0x7f15cc06c9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15f40d2300 tx=0x7f15dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.798+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff75c0d1f20 con 0x7ff75c0b6260 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.800+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff744003990 con 0x7ff75c0b6260 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.798+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc940c9850 con 0x7fcc940ce270 2024-02-02T13:24:48.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.801+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcc7c0129d0 con 0x7fcc940ce270 2024-02-02T13:24:48.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.801+0000 7f15f9200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7f15cc072720 0x7f15cc074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.802+0000 7ff759600640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff73406a430 0x7ff73406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.802+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff744087dc0 con 0x7ff75c0b6260 2024-02-02T13:24:48.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.802+0000 7ff759600640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7ff734072650 0x7ff734074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff734075120 con 0x7ff734072650 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74408a3e0 con 0x7ff75c0b6260 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7ff761800640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff73406a430 0x7ff73406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7fcc8b400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc6c06a460 0x7fcc6c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7f15f9200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7f15cc072720 0x7f15cc074b40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc7c088e10 con 0x7fcc940ce270 2024-02-02T13:24:48.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7ff762200640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7ff734072650 0x7ff734074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.803+0000 7fcc8b400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7fcc6c072680 0x7fcc6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:48.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.804+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc6c075150 con 0x7fcc6c072680 2024-02-02T13:24:48.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.804+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc7c0891c0 con 0x7fcc940ce270 2024-02-02T13:24:48.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.804+0000 7ff761800640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff73406a430 0x7ff73406c8f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff754035000 tx=0x7ff75402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.804+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== osd.5 v2:172.21.15.113:6800/33078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f15cc0751f0 con 0x7f15cc072720 2024-02-02T13:24:48.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.804+0000 7ff762200640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7ff734072650 0x7ff734074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.805+0000 7fcc93400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc6c06a460 0x7fcc6c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.805+0000 7fcc93e00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7fcc6c072680 0x7fcc6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:48.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.806+0000 7fcc93400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc6c06a460 0x7fcc6c06c920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400ab10 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.806+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== osd.6 v2:172.21.15.113:6816/33081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7ff734075120 con 0x7ff734072650 2024-02-02T13:24:48.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.806+0000 7fcc93e00640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7fcc6c072680 0x7fcc6c074aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:48.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.808+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== osd.7 v2:172.21.15.113:6806/33079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fcc6c075150 con 0x7fcc6c072680 2024-02-02T13:24:48.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.815+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 --> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a80c9240 con 0x7f65a82021b0 2024-02-02T13:24:48.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.815+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 --> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6340002cb0 con 0x7f6348072630 2024-02-02T13:24:48.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.816+0000 7f65a4c00640 1 -- 172.21.15.91:0/2818272765 <== osd.0 v2:172.21.15.91:6820/32877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6590031710 con 0x7f65a82021b0 2024-02-02T13:24:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 --> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96700cca60 con 0x7f96480726a0 2024-02-02T13:24:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f65a82021b0 msgr2=0x7f65a8202590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f6367e00640 1 -- 172.21.15.91:0/2218397387 <== osd.4 v2:172.21.15.113:6823/33080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6340002cb0 con 0x7f6348072630 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f65a82021b0 0x7f65a8202590 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f658006a430 msgr2=0x7f658006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f658006a430 0x7f658006c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6598000c00 tx=0x7f65980079c0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 msgr2=0x7f65a80b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65af400640 1 --2- 172.21.15.91:0/2818272765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65a8161660 0x7f65a80b2740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65a000be20 tx=0x7f65a000bf10 comp rx=0 tx=0).stop 2024-02-02T13:24:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f9667e00640 1 -- 172.21.15.91:0/1494379240 <== osd.1 v2:172.21.15.91:6805/32878 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9668006e60 con 0x7f96480726a0 2024-02-02T13:24:48.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7f6348072630 msgr2=0x7f6348074a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f65ad400640 1 -- 172.21.15.91:0/2818272765 reap_dead start 2024-02-02T13:24:48.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.817+0000 7f6376c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7f6348072630 0x7f6348074a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 shutdown_connections 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f634806a490 msgr2=0x7f634806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f634806a490 0x7f634806c950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63700cc9a0 tx=0x7f6360009040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63700b1d70 msgr2=0x7f637015c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 --2- 172.21.15.91:0/2218397387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63700b1d70 0x7f637015c090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6358004000 tx=0x7f63580046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6374c00640 1 -- 172.21.15.91:0/2218397387 reap_dead start 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 shutdown_connections 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 >> 172.21.15.91:0/2218397387 conn(0x7f63700d3030 msgr2=0x7f63700b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 >> 172.21.15.91:0/2818272765 conn(0x7f65a80cec80 msgr2=0x7f65a80af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7f96480726a0 msgr2=0x7f9648074ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.818+0000 7f9676e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7f96480726a0 0x7f9648074ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 shutdown_connections 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f65af400640 1 -- 172.21.15.91:0/2818272765 wait complete. 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f964806a480 msgr2=0x7f964806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f964806a480 0x7f964806c940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9660000f60 tx=0x7f9660009040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 shutdown_connections 2024-02-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9670161660 msgr2=0x7f96701511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 --2- 172.21.15.91:0/1494379240 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9670161660 0x7f96701511b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9658033000 tx=0x7f96580046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f6376c00640 1 -- 172.21.15.91:0/2218397387 wait complete. 2024-02-02T13:24:48.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9674e00640 1 -- 172.21.15.91:0/1494379240 reap_dead start 2024-02-02T13:24:48.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 shutdown_connections 2024-02-02T13:24:48.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 >> 172.21.15.91:0/1494379240 conn(0x7f96700cec80 msgr2=0x7f9670165140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.819+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 --> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe660006110 con 0x7fe640072650 2024-02-02T13:24:48.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.821+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 shutdown_connections 2024-02-02T13:24:48.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.820+0000 7fe65f400640 1 -- 172.21.15.91:0/420007934 <== osd.3 v2:172.21.15.91:6800/32876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe640075120 con 0x7fe640072650 2024-02-02T13:24:48.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.821+0000 7f9676e00640 1 -- 172.21.15.91:0/1494379240 wait complete. 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.821+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7fe640072650 msgr2=0x7fe640074a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.821+0000 7fe66d200640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7fe640072650 0x7fe640074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe64006a430 msgr2=0x7fe64006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe64006a430 0x7fe64006c8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe65800ba50 tx=0x7fe658008040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 msgr2=0x7fe66017b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 --2- 172.21.15.91:0/420007934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6600e78b0 0x7fe66017b4c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe66804f7d0 tx=0x7fe6680741e0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe667e00640 1 -- 172.21.15.91:0/420007934 reap_dead start 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 shutdown_connections 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 >> 172.21.15.91:0/420007934 conn(0x7fe660054c10 msgr2=0x7fe660039940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.822+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 shutdown_connections 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.823+0000 7fe66d200640 1 -- 172.21.15.91:0/420007934 wait complete. 2024-02-02T13:24:48.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.825+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 --> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70f40c9570 con 0x7f70cc072600 2024-02-02T13:24:48.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.825+0000 7f70eb400640 1 -- 172.21.15.91:0/3588488665 <== osd.2 v2:172.21.15.91:6816/32875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70dc0076a0 con 0x7f70cc072600 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f70cc072600 msgr2=0x7f70cc074a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f70cc072600 0x7f70cc074a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70cc06a460 msgr2=0x7f70cc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70cc06a460 0x7f70cc06c920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70e4036000 tx=0x7f70e403c040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 msgr2=0x7f70f4151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.826+0000 7f70fa200640 1 --2- 172.21.15.91:0/3588488665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f4165200 0x7f70f4151860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c730 tx=0x7f70ec00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.827+0000 7f70f3e00640 1 -- 172.21.15.91:0/3588488665 reap_dead start 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.827+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 shutdown_connections 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.827+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 >> 172.21.15.91:0/3588488665 conn(0x7f70f40cec80 msgr2=0x7f70f4160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.827+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 shutdown_connections 2024-02-02T13:24:48.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.827+0000 7f70fa200640 1 -- 172.21.15.91:0/3588488665 wait complete. 2024-02-02T13:24:48.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.829+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 --> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15c4002cb0 con 0x7f15cc072720 2024-02-02T13:24:48.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.832+0000 7f15ebe00640 1 -- 172.21.15.91:0/2722137771 <== osd.5 v2:172.21.15.113:6800/33078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f15c4002cb0 con 0x7f15cc072720 2024-02-02T13:24:48.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.834+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 --> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff75c0d1c40 con 0x7ff734072650 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7f15cc072720 msgr2=0x7f15cc074b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7f15cc072720 0x7f15cc074b40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f15cc06a500 msgr2=0x7f15cc06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f15cc06a500 0x7f15cc06c9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15f40d2300 tx=0x7f15dc009040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 msgr2=0x7f15f40cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 --2- 172.21.15.91:0/2722137771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f40d06e0 0x7f15f40cc510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cb20 tx=0x7f15ec0020a0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15f9200640 1 -- 172.21.15.91:0/2722137771 reap_dead start 2024-02-02T13:24:48.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 shutdown_connections 2024-02-02T13:24:48.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 >> 172.21.15.91:0/2722137771 conn(0x7f15f4070660 msgr2=0x7f15f40cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.835+0000 7ff759600640 1 -- 172.21.15.91:0/2814593391 <== osd.6 v2:172.21.15.113:6816/33081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff74c007dc0 con 0x7ff734072650 2024-02-02T13:24:48.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.836+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 shutdown_connections 2024-02-02T13:24:48.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.836+0000 7f15fb200640 1 -- 172.21.15.91:0/2722137771 wait complete. 2024-02-02T13:24:48.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.836+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7ff734072650 msgr2=0x7ff734074a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.836+0000 7ff764200640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7ff734072650 0x7ff734074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.837+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff73406a430 msgr2=0x7ff73406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.837+0000 7ff764200640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff73406a430 0x7ff73406c8f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff754035000 tx=0x7ff75402e000 comp rx=0 tx=0).stop 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.837+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 msgr2=0x7ff75c0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.837+0000 7ff764200640 1 --2- 172.21.15.91:0/2814593391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff75c0b6260 0x7ff75c0c8de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c400 tx=0x7ff74400c8c0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.837+0000 7ff762200640 1 -- 172.21.15.91:0/2814593391 reap_dead start 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.838+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 shutdown_connections 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.838+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 >> 172.21.15.91:0/2814593391 conn(0x7ff75c070660 msgr2=0x7ff75c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.838+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 shutdown_connections 2024-02-02T13:24:48.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.838+0000 7ff764200640 1 -- 172.21.15.91:0/2814593391 wait complete. 2024-02-02T13:24:48.862 INFO:teuthology.orchestra.run.smithi091.stdout:55834574855 2024-02-02T13:24:48.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.841+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 --> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc940c9570 con 0x7fcc6c072680 2024-02-02T13:24:48.863 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-02T13:24:48.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.843+0000 7fcc8b400640 1 -- 172.21.15.91:0/3616620835 <== osd.7 v2:172.21.15.113:6806/33079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc8c0077a0 con 0x7fcc6c072680 2024-02-02T13:24:48.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.843+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7fcc6c072680 msgr2=0x7fcc6c074aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.843+0000 7fcc9a400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7fcc6c072680 0x7fcc6c074aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:48.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc6c06a460 msgr2=0x7fcc6c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc6c06a460 0x7fcc6c06c920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400ab10 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 msgr2=0x7fcc94070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 --2- 172.21.15.91:0/3616620835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc940ce270 0x7fcc94070370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c480 tx=0x7fcc7c00c940 comp rx=0 tx=0).stop 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc93e00640 1 -- 172.21.15.91:0/3616620835 reap_dead start 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 shutdown_connections 2024-02-02T13:24:48.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 >> 172.21.15.91:0/3616620835 conn(0x7fcc940cec80 msgr2=0x7fcc9414c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:48.866 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-02T13:24:48.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.844+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 shutdown_connections 2024-02-02T13:24:48.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:48.845+0000 7fcc9a400640 1 -- 172.21.15.91:0/3616620835 wait complete. 2024-02-02T13:24:48.867 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.867 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-02T13:24:48.868 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.868 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-02T13:24:48.869 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.869 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-02T13:24:48.870 INFO:teuthology.orchestra.run.smithi091.stdout:55834574855 2024-02-02T13:24:48.870 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-02T13:24:48.871 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.871 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-02T13:24:48.872 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:48.872 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-02T13:24:49.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.189+0000 7f9ccbe00640 1 Processor -- start 2024-02-02T13:24:49.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.190+0000 7f9ccbe00640 1 -- start start 2024-02-02T13:24:49.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f9ccbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cc4066020 0x7f9cc40705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f9ccbe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc40cf3b0 con 0x7f9cc4160740 2024-02-02T13:24:49.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f9ccbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc40cfba0 con 0x7f9cc40af230 2024-02-02T13:24:49.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f9ccbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc40cd8c0 con 0x7f9cc4066020 2024-02-02T13:24:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f48dd400640 1 Processor -- start 2024-02-02T13:24:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f9cc9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cc4066020 0x7f9cc40705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cc4066020 0x7f9cc40705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57310/0 (socket says 172.21.15.91:57310) 2024-02-02T13:24:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc8a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9cc40af230 0x7f9cc4065910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:34784/0 (socket says 172.21.15.91:34784) 2024-02-02T13:24:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc8a00640 1 -- 172.21.15.91:0/239479372 learned_addr learned my addr 172.21.15.91:0/239479372 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:49.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.192+0000 7f48dd400640 1 -- start start 2024-02-02T13:24:49.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.194+0000 7f48dd400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48d00e3190 0x7f48d003e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc9e00640 1 -- 172.21.15.91:0/239479372 >> v1:172.21.15.113:6789/0 conn(0x7f9cc40af230 legacy=0x7f9cc4065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc9e00640 1 -- 172.21.15.91:0/239479372 >> v1:172.21.15.91:6789/0 conn(0x7f9cc4160740 legacy=0x7f9cc406c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.194+0000 7f48dd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d0038180 0x7f48d0053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.194+0000 7f48dd400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d003ee60 con 0x7f48d00542d0 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48dd400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d003f100 con 0x7f48d00e3190 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48dd400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d0007c40 con 0x7f48d0038180 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.193+0000 7f9cc9e00640 1 -- 172.21.15.91:0/239479372 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40cdba0 con 0x7f9cc4066020 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48dca00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f48d00542d0 0x7f48d00e6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:34786/0 (socket says 172.21.15.91:34786) 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48dca00640 1 -- 172.21.15.91:0/2233407053 learned_addr learned my addr 172.21.15.91:0/2233407053 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48d7e00640 1 --2- 172.21.15.91:0/2233407053 >> v2:172.21.15.91:3300/0 conn(0x7f48d00e3190 0x7f48d003e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f48d7400640 1 --2- 172.21.15.91:0/2233407053 >> v2:172.21.15.113:3300/0 conn(0x7f48d0038180 0x7f48d0053d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f003d400640 1 Processor -- start 2024-02-02T13:24:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.195+0000 7f003d400640 1 -- start start 2024-02-02T13:24:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f9cc9e00640 1 --2- 172.21.15.91:0/239479372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc4066020 0x7f9cc40705b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0097a0 tx=0x7f9cac02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d289a20f49ba2f6e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d6a00640 1 -- 172.21.15.91:0/2233407053 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780706828 0 0) 0x7f48d003ee60 con 0x7f48d00542d0 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d6a00640 1 -- 172.21.15.91:0/2233407053 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003520 con 0x7f48d00542d0 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d7e00640 1 -- 172.21.15.91:0/2233407053 >> v2:172.21.15.113:3300/0 conn(0x7f48d0038180 msgr2=0x7f48d0053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d7e00640 1 --2- 172.21.15.91:0/2233407053 >> v2:172.21.15.113:3300/0 conn(0x7f48d0038180 0x7f48d0053d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d7e00640 1 -- 172.21.15.91:0/2233407053 >> v1:172.21.15.113:6789/0 conn(0x7f48d00542d0 legacy=0x7f48d00e6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f48d7e00640 1 -- 172.21.15.91:0/2233407053 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0054760 con 0x7f48d00e3190 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f48d7e00640 1 --2- 172.21.15.91:0/2233407053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d00e3190 0x7f48d003e920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f48d8068b00 tx=0x7f48d80a5ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c19c00aace1c24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f9cc3e00640 1 -- 172.21.15.91:0/239479372 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac02e070 con 0x7f9cc4066020 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f9cc3e00640 1 -- 172.21.15.91:0/239479372 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac0315a0 con 0x7f9cc4066020 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.196+0000 7f9cc3e00640 1 -- 172.21.15.91:0/239479372 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac031930 con 0x7f9cc4066020 2024-02-02T13:24:49.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f48d6a00640 1 -- 172.21.15.91:0/2233407053 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a6d10 con 0x7f48d00e3190 2024-02-02T13:24:49.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f9ccbe00640 1 -- 172.21.15.91:0/239479372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc4066020 msgr2=0x7f9cc40705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f48d6a00640 1 -- 172.21.15.91:0/2233407053 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80a6e70 con 0x7f48d00e3190 2024-02-02T13:24:49.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f48d6a00640 1 -- 172.21.15.91:0/2233407053 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a6d10 con 0x7f48d00e3190 2024-02-02T13:24:49.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f48dd400640 1 -- 172.21.15.91:0/2233407053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d00e3190 msgr2=0x7f48d003e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/239479372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc4066020 0x7f9cc40705b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0097a0 tx=0x7f9cac02b840 comp rx=0 tx=0).stop 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f48dd400640 1 --2- 172.21.15.91:0/2233407053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d00e3190 0x7f48d003e920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f48d8068b00 tx=0x7f48d80a5ea0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f003000c1f0 0x7f0030053150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f0030033d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00300e2e10 0x7f00300e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00300543f0 con 0x7f003000c1f0 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00300e7190 con 0x7f00300e2e10 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.197+0000 7f003d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0030053ac0 con 0x7f0030050230 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f0037e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f003000c1f0 0x7f0030053150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f0037e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f003000c1f0 0x7f0030053150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53358/0 (socket says 172.21.15.91:53358) 2024-02-02T13:24:49.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f0037e00640 1 -- 172.21.15.91:0/2978412949 learned_addr learned my addr 172.21.15.91:0/2978412949 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.198+0000 7f003ca00640 1 --2- 172.21.15.91:0/2978412949 >> v2:172.21.15.113:3300/0 conn(0x7f00300e2e10 0x7f00300e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037400640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f0030033d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f9ccbe00640 1 -- 172.21.15.91:0/239479372 shutdown_connections 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/239479372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc4066020 0x7f9cc40705b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f9ccbe00640 1 -- 172.21.15.91:0/239479372 >> 172.21.15.91:0/239479372 conn(0x7f9cc40b18f0 msgr2=0x7f9cc40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 -- 172.21.15.91:0/2233407053 shutdown_connections 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 --2- 172.21.15.91:0/2233407053 >> v2:172.21.15.113:3300/0 conn(0x7f48d0038180 0x7f48d0053d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 --2- 172.21.15.91:0/2233407053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d00e3190 0x7f48d003e920 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037e00640 1 -- 172.21.15.91:0/2978412949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 msgr2=0x7f0030033d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037e00640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f0030033d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037e00640 1 -- 172.21.15.91:0/2978412949 >> v2:172.21.15.113:3300/0 conn(0x7f00300e2e10 msgr2=0x7f00300e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037e00640 1 --2- 172.21.15.91:0/2978412949 >> v2:172.21.15.113:3300/0 conn(0x7f00300e2e10 0x7f00300e32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.199+0000 7f0037e00640 1 -- 172.21.15.91:0/2978412949 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0030053da0 con 0x7f003000c1f0 2024-02-02T13:24:49.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 -- 172.21.15.91:0/2233407053 >> 172.21.15.91:0/2233407053 conn(0x7f48d00590c0 msgr2=0x7f48d00594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f9ccbe00640 1 -- 172.21.15.91:0/239479372 shutdown_connections 2024-02-02T13:24:49.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 -- 172.21.15.91:0/2233407053 shutdown_connections 2024-02-02T13:24:49.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f48dd400640 1 -- 172.21.15.91:0/2233407053 wait complete. 2024-02-02T13:24:49.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.201+0000 7f0037e00640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f003000c1f0 0x7f0030053150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0038068250 tx=0x7f00380a6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a04c49756320a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.200+0000 7f9ccbe00640 1 -- 172.21.15.91:0/239479372 wait complete. 2024-02-02T13:24:49.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.202+0000 7f0036a00640 1 -- 172.21.15.91:0/2978412949 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038061050 con 0x7f003000c1f0 2024-02-02T13:24:49.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.202+0000 7f0036a00640 1 -- 172.21.15.91:0/2978412949 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038061230 con 0x7f003000c1f0 2024-02-02T13:24:49.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.202+0000 7f0036a00640 1 -- 172.21.15.91:0/2978412949 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038060690 con 0x7f003000c1f0 2024-02-02T13:24:49.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f9ccbe00640 1 Processor -- start 2024-02-02T13:24:49.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f48dd400640 1 Processor -- start 2024-02-02T13:24:49.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f48dd400640 1 -- start start 2024-02-02T13:24:49.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 -- 172.21.15.91:0/2978412949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f003000c1f0 msgr2=0x7f0030053150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f003000c1f0 0x7f0030053150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0038068250 tx=0x7f00380a6060 comp rx=0 tx=0).stop 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 -- 172.21.15.91:0/2978412949 shutdown_connections 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 --2- 172.21.15.91:0/2978412949 >> v2:172.21.15.113:3300/0 conn(0x7f00300e2e10 0x7f00300e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f0030033d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 --2- 172.21.15.91:0/2978412949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f003000c1f0 0x7f0030053150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f003d400640 1 -- 172.21.15.91:0/2978412949 >> 172.21.15.91:0/2978412949 conn(0x7f0030054c10 msgr2=0x7f0030054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.204+0000 7f003d400640 1 -- 172.21.15.91:0/2978412949 shutdown_connections 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.204+0000 7f7290000640 1 Processor -- start 2024-02-02T13:24:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.204+0000 7f003d400640 1 -- 172.21.15.91:0/2978412949 wait complete. 2024-02-02T13:24:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.203+0000 7f9ccbe00640 1 -- start start 2024-02-02T13:24:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.204+0000 7f7290000640 1 -- start start 2024-02-02T13:24:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.205+0000 7f48dd400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 0x7f48d0053030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.205+0000 7f48dd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.205+0000 7f48dd400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 0x7f48d0188990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.205+0000 7f48d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 0x7f48d0053030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.205+0000 7f48d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 0x7f48d0053030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53370/0 (socket says 172.21.15.91:53370) 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f48d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f48d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57324/0 (socket says 172.21.15.91:57324) 2024-02-02T13:24:49.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f48dca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 0x7f48d0188990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc40af230 0x7f9cc4150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f48d7e00640 1 -- 172.21.15.91:0/1305473969 learned_addr learned my addr 172.21.15.91:0/1305473969 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0053570 con 0x7f48d0038180 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f7290000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72881609b0 0x7f72880cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f7290000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 0x7f7288070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 0x7f9cc4150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4154110 con 0x7f9cc4160740 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4154530 con 0x7f9cc40af230 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9ccbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc415c7b0 con 0x7f9cc4066020 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f7290000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72880707b0 0x7f72880d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38748/0 (socket says 172.21.15.91:38748) 2024-02-02T13:24:49.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f7290000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7288070c40 con 0x7f72881609b0 2024-02-02T13:24:49.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f9cc9400640 1 -- 172.21.15.91:0/3233020079 learned_addr learned my addr 172.21.15.91:0/3233020079 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f7290000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7288160d90 con 0x7f72880707b0 2024-02-02T13:24:49.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f7290000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7288070fe0 con 0x7f72880ce530 2024-02-02T13:24:49.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f9cc9e00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 0x7f9cc4150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0050c70 con 0x7f48d00542d0 2024-02-02T13:24:49.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0050f50 con 0x7f48d00e3190 2024-02-02T13:24:49.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f003d400640 1 Processor -- start 2024-02-02T13:24:49.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.206+0000 7f003d400640 1 -- start start 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f9cc9400640 1 -- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc40af230 msgr2=0x7f9cc4150b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f9cc9400640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc40af230 0x7f9cc4150b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f9cc9400640 1 -- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 msgr2=0x7f9cc4150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.207+0000 7f9cc9400640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 0x7f9cc4150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 -- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 msgr2=0x7f48d0050190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 -- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 msgr2=0x7f48d0053030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 0x7f48d0053030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0051230 con 0x7f48d00e3190 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f9cc9400640 1 -- 172.21.15.91:0/3233020079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac009400 con 0x7f9cc4066020 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f003004af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f728d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72881609b0 0x7f72880cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f728e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72880707b0 0x7f72880d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f728e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72880707b0 0x7f72880d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57334/0 (socket says 172.21.15.91:57334) 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f728e000640 1 -- 172.21.15.91:0/3322561911 learned_addr learned my addr 172.21.15.91:0/3322561911 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00300e2e10 0x7f00300d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0030046ea0 con 0x7f00300e2e10 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00300489d0 con 0x7f003000c1f0 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f003d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00300486a0 con 0x7f0030050230 2024-02-02T13:24:49.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f9cc9e00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 0x7f9cc4150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:49.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48d7400640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:49.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7fb3e5000640 1 Processor -- start 2024-02-02T13:24:49.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f48dca00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 0x7f48d0188990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48c000c390 tx=0x7f48c000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f728cc00640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 0x7f7288070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f0037e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f0037e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57344/0 (socket says 172.21.15.91:57344) 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.208+0000 7f0037e00640 1 -- 172.21.15.91:0/2964664658 learned_addr learned my addr 172.21.15.91:0/2964664658 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7fb3e5000640 1 -- start start 2024-02-02T13:24:49.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f9cc9400640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00ba50 tx=0x7f9cbc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f0037e00640 1 -- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 msgr2=0x7f003004af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f0037e00640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f003004af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f0037e00640 1 -- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00300e2e10 msgr2=0x7f00300d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f0037e00640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00300e2e10 0x7f00300d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f0037e00640 1 -- 172.21.15.91:0/2964664658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003804eee0 con 0x7f003000c1f0 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0005ce0 con 0x7f48d00e3190 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0005e40 con 0x7f48d00e3190 2024-02-02T13:24:49.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00072f0 con 0x7f48d00e3190 2024-02-02T13:24:49.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d0051510 con 0x7f48d00e3190 2024-02-02T13:24:49.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d0051ad0 con 0x7f48d00e3190 2024-02-02T13:24:49.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f0037e00640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f003804f8f0 tx=0x7f0038067bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00cbe0 con 0x7f9cc4066020 2024-02-02T13:24:49.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00cd80 con 0x7f9cc4066020 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc415c1f0 con 0x7f9cc4066020 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038060e00 con 0x7f003000c1f0 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00380ae620 con 0x7f003000c1f0 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380aea10 con 0x7f003000c1f0 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f728e000640 1 -- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 msgr2=0x7f7288070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f728e000640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 0x7f7288070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f728e000640 1 -- 172.21.15.91:0/3322561911 >> v2:172.21.15.91:3300/0 conn(0x7f72881609b0 msgr2=0x7f72880cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f728e000640 1 --2- 172.21.15.91:0/3322561911 >> v2:172.21.15.91:3300/0 conn(0x7f72881609b0 0x7f72880cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.209+0000 7f728e000640 1 -- 172.21.15.91:0/3322561911 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880d2940 con 0x7f72880707b0 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc4153d30 con 0x7f9cc4066020 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc014680 con 0x7f9cc4066020 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7fb3e5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3e01609b0 0x7fb3e00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e00ce530 0x7fb3e0070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.210+0000 7f728cc00640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 0x7f7288070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3e5000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e0160d90 con 0x7fb3e00707b0 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3e5000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e0070c40 con 0x7fb3e01609b0 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3e5000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0070fe0 con 0x7fb3e00ce530 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00300470b0 con 0x7f003000c1f0 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0030047670 con 0x7f003000c1f0 2024-02-02T13:24:49.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7f728e000640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72880707b0 0x7f72880d0bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72880cd600 tx=0x7f727002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1591ad2e47bb32b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fefe1800640 1 Processor -- start 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fefe1800640 1 -- start start 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3dde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3e01609b0 0x7fb3e00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3dd400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e00ce530 0x7fb3e0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3dde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3e01609b0 0x7fb3e00cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:57356/0 (socket says 172.21.15.91:57356) 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.211+0000 7fb3dde00640 1 -- 172.21.15.91:0/1347087192 learned_addr learned my addr 172.21.15.91:0/1347087192 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fb3dde00640 1 -- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e00ce530 msgr2=0x7fb3e0070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fb3dde00640 1 --2- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e00ce530 0x7fb3e0070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fb3dde00640 1 -- 172.21.15.91:0/1347087192 >> v1:172.21.15.91:6789/0 conn(0x7fb3e00707b0 legacy=0x7fb3e00d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7fb3dde00640 1 -- 172.21.15.91:0/1347087192 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00d2940 con 0x7fb3e01609b0 2024-02-02T13:24:49.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8005250 con 0x7f48d00e3190 2024-02-02T13:24:49.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.212+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c94005250 con 0x7f9cc4066020 2024-02-02T13:24:49.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.213+0000 7f7287e00640 1 -- 172.21.15.91:0/3322561911 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727002e070 con 0x7f72880707b0 2024-02-02T13:24:49.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.213+0000 7f7287e00640 1 -- 172.21.15.91:0/3322561911 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72700315a0 con 0x7f72880707b0 2024-02-02T13:24:49.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.213+0000 7f7287e00640 1 -- 172.21.15.91:0/3322561911 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270031970 con 0x7f72880707b0 2024-02-02T13:24:49.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.213+0000 7f3a35c00640 1 Processor -- start 2024-02-02T13:24:49.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.213+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0030005490 con 0x7f003000c1f0 2024-02-02T13:24:49.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f3a35c00640 1 -- start start 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 -- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72880707b0 msgr2=0x7f72880d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72880707b0 0x7f72880d0bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72880cd600 tx=0x7f727002b840 comp rx=0 tx=0).stop 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 -- 172.21.15.91:0/3322561911 shutdown_connections 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72880707b0 0x7f72880d0bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 --2- 172.21.15.91:0/3322561911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880ce530 0x7f7288070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 --2- 172.21.15.91:0/3322561911 >> v2:172.21.15.91:3300/0 conn(0x7f72881609b0 0x7f72880cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7fefe1800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fefd40eb2f0 0x7fefd40eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7fefe1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd4006dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7f7290000640 1 -- 172.21.15.91:0/3322561911 >> 172.21.15.91:0/3322561911 conn(0x7f72880d3020 msgr2=0x7f72880d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f48c0035be0 con 0x7f48d00e3190 2024-02-02T13:24:49.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7fefe1800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd4050370 con 0x7fefd40e78e0 2024-02-02T13:24:49.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7fefe1800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd40064b0 con 0x7fefd40eb2f0 2024-02-02T13:24:49.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.214+0000 7fefe1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40ebc70 con 0x7fefd4006940 2024-02-02T13:24:49.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7fefdaa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fefd40e78e0 0x7fefd404f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34162/0 (socket says 172.21.15.91:34162) 2024-02-02T13:24:49.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7fefdaa00640 1 -- 172.21.15.91:0/1667950991 learned_addr learned my addr 172.21.15.91:0/1667950991 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:49.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f7290000640 1 -- 172.21.15.91:0/3322561911 shutdown_connections 2024-02-02T13:24:49.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7fefda000640 1 --2- 172.21.15.91:0/1667950991 >> v2:172.21.15.91:3300/0 conn(0x7fefd40eb2f0 0x7fefd40eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7fefdb400640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd4006dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f3a35c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3a300ce4c0 0x7f3a300b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f3a35c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f3a35c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3015d640 con 0x7f3a3015d260 2024-02-02T13:24:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f7290000640 1 -- 172.21.15.91:0/3322561911 wait complete. 2024-02-02T13:24:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f3a35c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a300d2530 con 0x7f3a300ce4c0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.215+0000 7f3a35c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300d2810 con 0x7f3a300d1dc0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f3a2e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3a300ce4c0 0x7f3a300b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38786/0 (socket says 172.21.15.91:38786) 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f3a2f400640 1 -- 172.21.15.91:0/2142655491 learned_addr learned my addr 172.21.15.91:0/2142655491 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f3a2e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3a300ce4c0 0x7f3a300b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53390/0 (socket says 172.21.15.91:53390) 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7fefd9600640 1 -- 172.21.15.91:0/1667950991 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924978948 0 0) 0x7fefd4050370 con 0x7fefd40e78e0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7fefd9600640 1 -- 172.21.15.91:0/1667950991 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefbc003520 con 0x7fefd40e78e0 2024-02-02T13:24:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9cbc01d3b0 con 0x7f9cc4066020 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.216+0000 7fb3dde00640 1 --2- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e01609b0 0x7fb3e00cdff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009870 tx=0x7fb3d002b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa87cfeaeaab2e0c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f48cfe00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 0x7f48b006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c0087e10 con 0x7f48d00e3190 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f3a2e000640 1 -- 172.21.15.91:0/2142655491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 msgr2=0x7f3a30070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f3a2e000640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f3a2e000640 1 -- 172.21.15.91:0/2142655491 >> v1:172.21.15.113:6789/0 conn(0x7f3a3015d260 legacy=0x7f3a300cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f3a2e000640 1 -- 172.21.15.91:0/2142655491 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300d2af0 con 0x7f3a300ce4c0 2024-02-02T13:24:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f3a2f400640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7fefda000640 1 -- 172.21.15.91:0/1667950991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 msgr2=0x7fefd4006dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7fefda000640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd4006dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7fefda000640 1 -- 172.21.15.91:0/1667950991 >> v1:172.21.15.91:6789/0 conn(0x7fefd40e78e0 legacy=0x7fefd404f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7fefda000640 1 -- 172.21.15.91:0/1667950991 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd404fdb0 con 0x7fefd40eb2f0 2024-02-02T13:24:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f48d7e00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 0x7f48b006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.217+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f00380aeb70 con 0x7f003000c1f0 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.218+0000 7f002fe00640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 0x7f001006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.218+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0038124e70 con 0x7f003000c1f0 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.218+0000 7f3a2e000640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a300b8b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a180097a0 tx=0x7f3a1802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1729a2cd023bb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.218+0000 7f9cc1600640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 0x7f9c9c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cbc088d00 con 0x7f9cc4066020 2024-02-02T13:24:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f0037400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 0x7f001006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f3a2d600640 1 -- 172.21.15.91:0/2142655491 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18031780 con 0x7f3a300ce4c0 2024-02-02T13:24:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7fefda000640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40eb2f0 0x7fefd40eb730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097a0 tx=0x7fefc402b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2630490ebf583429 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f48d7e00640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 0x7f48b006c9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48d8079d30 tx=0x7f48d806c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f7290000640 1 Processor -- start 2024-02-02T13:24:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f3a2d600640 1 -- 172.21.15.91:0/2142655491 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a180318e0 con 0x7f3a300ce4c0 2024-02-02T13:24:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f3a2d600640 1 -- 172.21.15.91:0/2142655491 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18031c70 con 0x7f3a300ce4c0 2024-02-02T13:24:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f9cc8a00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 0x7f9c9c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.219+0000 7f7290000640 1 -- start start 2024-02-02T13:24:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7fefd9600640 1 -- 172.21.15.91:0/1667950991 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4004230 con 0x7fefd40eb2f0 2024-02-02T13:24:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f0037400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 0x7f001006c920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0028006fd0 tx=0x7f0028008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 -- 172.21.15.91:0/2142655491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 msgr2=0x7f3a300b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7fefd9600640 1 -- 172.21.15.91:0/1667950991 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc402e050 con 0x7fefd40eb2f0 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7fefd9600640 1 -- 172.21.15.91:0/1667950991 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4003610 con 0x7fefd40eb2f0 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a300b8b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a180097a0 tx=0x7f3a1802b820 comp rx=0 tx=0).stop 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 -- 172.21.15.91:0/2142655491 shutdown_connections 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a30070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2142655491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a300b8b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 -- 172.21.15.91:0/2142655491 >> 172.21.15.91:0/2142655491 conn(0x7f3a300cfa60 msgr2=0x7f3a300cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 -- 172.21.15.91:0/2142655491 shutdown_connections 2024-02-02T13:24:49.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f9cc8a00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 0x7f9c9c06c9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ca340 tx=0x7f9cb4007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880707b0 0x7f72881f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.220+0000 7f3a35c00640 1 -- 172.21.15.91:0/2142655491 wait complete. 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72881609b0 0x7f72880ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288155590 con 0x7f72880ce530 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72881f7eb0 con 0x7f72881609b0 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f7290000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72880b5510 con 0x7f72880707b0 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 -- 172.21.15.91:0/1667950991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40eb2f0 msgr2=0x7fefd40eb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40eb2f0 0x7fefd40eb730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097a0 tx=0x7fefc402b760 comp rx=0 tx=0).stop 2024-02-02T13:24:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f728cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f728cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53396/0 (socket says 172.21.15.91:53396) 2024-02-02T13:24:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f728cc00640 1 -- 172.21.15.91:0/2066827972 learned_addr learned my addr 172.21.15.91:0/2066827972 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 -- 172.21.15.91:0/1667950991 shutdown_connections 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd4006dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 --2- 172.21.15.91:0/1667950991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40eb2f0 0x7fefd40eb730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7fefe1800640 1 -- 172.21.15.91:0/1667950991 >> 172.21.15.91:0/1667950991 conn(0x7fefd4054c40 msgr2=0x7fefd4055020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.221+0000 7f728e000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72881609b0 0x7f72880ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728d600640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880707b0 0x7f72881f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 -- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880707b0 msgr2=0x7f72881f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880707b0 0x7f72881f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 -- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72881609b0 msgr2=0x7f72880ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72881609b0 0x7f72880ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 -- 172.21.15.91:0/2066827972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700094d0 con 0x7f72880ce530 2024-02-02T13:24:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7fefe1800640 1 -- 172.21.15.91:0/1667950991 shutdown_connections 2024-02-02T13:24:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7fefe1800640 1 -- 172.21.15.91:0/1667950991 wait complete. 2024-02-02T13:24:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f728cc00640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f727800e7e0 tx=0x7f727800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278004030 con 0x7f72880ce530 2024-02-02T13:24:49.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278004190 con 0x7f72880ce530 2024-02-02T13:24:49.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278004500 con 0x7f72880ce530 2024-02-02T13:24:49.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.222+0000 7f3a35c00640 1 Processor -- start 2024-02-02T13:24:49.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.223+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72880b57f0 con 0x7f72880ce530 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.223+0000 7f3a35c00640 1 -- start start 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.223+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72881f6d30 con 0x7f72880ce530 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a3006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a3015d260 0x7f3a30153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3006be60 con 0x7f3a300ce4c0 2024-02-02T13:24:49.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3006c970 con 0x7f3a3015d260 2024-02-02T13:24:49.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a35c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3006c040 con 0x7f3a300d1dc0 2024-02-02T13:24:49.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7fefe1800640 1 Processor -- start 2024-02-02T13:24:49.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72881556d0 con 0x7f72880ce530 2024-02-02T13:24:49.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7fefe1800640 1 -- start start 2024-02-02T13:24:49.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53398/0 (socket says 172.21.15.91:53398) 2024-02-02T13:24:49.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.224+0000 7f3a2ea00640 1 -- 172.21.15.91:0/2782993403 learned_addr learned my addr 172.21.15.91:0/2782993403 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2e000640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a3006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2f400640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a3015d260 0x7f3a30153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 -- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 msgr2=0x7f3a3006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a3006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 -- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a3015d260 msgr2=0x7f3a30153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a3015d260 0x7f3a30153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 -- 172.21.15.91:0/2782993403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a28009b30 con 0x7f3a300ce4c0 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.225+0000 7f3a2ea00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000c400 tx=0x7f3a2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f2250600640 1 Processor -- start 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a200052d0 con 0x7f3a300ce4c0 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20005430 con 0x7f3a300ce4c0 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20003830 con 0x7f3a300ce4c0 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a18009400 con 0x7f3a300ce4c0 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd404a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f2250600640 1 -- start start 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3006c600 con 0x7f3a300ce4c0 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefd40eb2f0 0x7fefd40e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd404acd0 con 0x7fefd40e78e0 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40e8d10 con 0x7fefd40eb2f0 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefe1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40e3770 con 0x7fefd4006940 2024-02-02T13:24:49.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f727800c6d0 con 0x7f72880ce530 2024-02-02T13:24:49.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefda000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.226+0000 7fefda000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53406/0 (socket says 172.21.15.91:53406) 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 -- 172.21.15.91:0/3588107415 learned_addr learned my addr 172.21.15.91:0/3588107415 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefdb400640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefd40eb2f0 0x7fefd40e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefdaa00640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd404a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 -- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 msgr2=0x7fefd404a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd404a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 -- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefd40eb2f0 msgr2=0x7fefd40e5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefd40eb2f0 0x7fefd40e5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefda000640 1 -- 172.21.15.91:0/3588107415 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009400 con 0x7fefd40e78e0 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7fefdaa00640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd404a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:49.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.227+0000 7f7285600640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 0x7f726006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72780884f0 con 0x7f72880ce530 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7fefda000640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefc402bd70 tx=0x7fefc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f2250600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 0x7f22480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f2250600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f22480ce530 0x7f2248070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f2250600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2248160d90 con 0x7f22480707b0 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f2250600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2248070c40 con 0x7f22480ce530 2024-02-02T13:24:49.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f2250600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2248070fe0 con 0x7f22481609b0 2024-02-02T13:24:49.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f728d600640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 0x7f726006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f22480ce530 0x7f2248070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f22480ce530 0x7f2248070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53420/0 (socket says 172.21.15.91:53420) 2024-02-02T13:24:49.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 -- 172.21.15.91:0/1433031919 learned_addr learned my addr 172.21.15.91:0/1433031919 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a30067400 con 0x7f3a300ce4c0 2024-02-02T13:24:49.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224dc00640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 0x7f22480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 -- 172.21.15.91:0/1433031919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 msgr2=0x7f22480cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 0x7f22480cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.228+0000 7f224d200640 1 -- 172.21.15.91:0/1433031919 >> v1:172.21.15.113:6789/0 conn(0x7f22480707b0 legacy=0x7f22480d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:49.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.229+0000 7f224d200640 1 -- 172.21.15.91:0/1433031919 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480d2940 con 0x7f22480ce530 2024-02-02T13:24:49.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.229+0000 7f224dc00640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 0x7f22480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:49.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.229+0000 7f728d600640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 0x7f726006c8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72800073d0 tx=0x7f728000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.229+0000 7f224d200640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22480ce530 0x7f2248070270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2230009870 tx=0x7f223002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910abb06ead10558 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.232+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f00380f05d0 con 0x7f003000c1f0 2024-02-02T13:24:49.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.232+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f48c0053490 con 0x7f48d00e3190 2024-02-02T13:24:49.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.232+0000 7fb3dca00640 1 -- 172.21.15.91:0/1347087192 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0033620 con 0x7fb3e01609b0 2024-02-02T13:24:49.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.232+0000 7fb3dca00640 1 -- 172.21.15.91:0/1347087192 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0033780 con 0x7fb3e01609b0 2024-02-02T13:24:49.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.232+0000 7fb3dca00640 1 -- 172.21.15.91:0/1347087192 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0033b10 con 0x7fb3e01609b0 2024-02-02T13:24:49.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.233+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7278053a40 con 0x7f72880ce530 2024-02-02T13:24:49.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.233+0000 7fb3e5000640 1 -- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e01609b0 msgr2=0x7fb3e00cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.233+0000 7fb3e5000640 1 --2- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e01609b0 0x7fb3e00cdff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009870 tx=0x7fb3d002b7d0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.234+0000 7f2247e00640 1 -- 172.21.15.91:0/1433031919 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230031740 con 0x7f22480ce530 2024-02-02T13:24:49.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.234+0000 7f2247e00640 1 -- 172.21.15.91:0/1433031919 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22300318a0 con 0x7f22480ce530 2024-02-02T13:24:49.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.234+0000 7f2247e00640 1 -- 172.21.15.91:0/1433031919 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230031c30 con 0x7f22480ce530 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 -- 172.21.15.91:0/1347087192 shutdown_connections 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 --2- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e00ce530 0x7fb3e0070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 --2- 172.21.15.91:0/1347087192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e01609b0 0x7fb3e00cdff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 -- 172.21.15.91:0/1347087192 >> 172.21.15.91:0/1347087192 conn(0x7fb3e00d3020 msgr2=0x7fb3e00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 -- 172.21.15.91:0/1433031919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22480ce530 msgr2=0x7f2248070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22480ce530 0x7f2248070270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2230009870 tx=0x7f223002b7e0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3a20003990 con 0x7f3a300ce4c0 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 -- 172.21.15.91:0/1347087192 shutdown_connections 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fb3e5000640 1 -- 172.21.15.91:0/1347087192 wait complete. 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 -- 172.21.15.91:0/1433031919 shutdown_connections 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22480ce530 0x7f2248070270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 --2- 172.21.15.91:0/1433031919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22481609b0 0x7f22480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7f2250600640 1 -- 172.21.15.91:0/1433031919 >> 172.21.15.91:0/1433031919 conn(0x7f22480d3020 msgr2=0x7f22480d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4030710 con 0x7fefd40e78e0 2024-02-02T13:24:49.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc4030890 con 0x7fefd40e78e0 2024-02-02T13:24:49.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.235+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4030c40 con 0x7fefd40e78e0 2024-02-02T13:24:49.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7f2250600640 1 -- 172.21.15.91:0/1433031919 shutdown_connections 2024-02-02T13:24:49.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd40e46f0 con 0x7fefd40e78e0 2024-02-02T13:24:49.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7f3a26a00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 0x7f3a0806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a20088300 con 0x7f3a300ce4c0 2024-02-02T13:24:49.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7f3a2e000640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 0x7f3a0806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.236+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd40e4cb0 con 0x7fefd40e78e0 2024-02-02T13:24:49.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.237+0000 7f2250600640 1 -- 172.21.15.91:0/1433031919 wait complete. 2024-02-02T13:24:49.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.237+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd40e8ec0 con 0x7fefd40e78e0 2024-02-02T13:24:49.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.238+0000 7f3a2e000640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 0x7f3a0806c920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a18036000 tx=0x7f3a1803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.238+0000 7fb3e5000640 1 Processor -- start 2024-02-02T13:24:49.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.238+0000 7fb3e5000640 1 -- start start 2024-02-02T13:24:49.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.239+0000 7f2250600640 1 Processor -- start 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.239+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.239+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 0x7fb3e00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3e5000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e01609b0 0x7fb3e00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3e5000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00b3870 con 0x7fb3e00707b0 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3e5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00b3ac0 con 0x7fb3e00ce530 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3e5000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00b3da0 con 0x7fb3e01609b0 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3dde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3dde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53430/0 (socket says 172.21.15.91:53430) 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3dde00640 1 -- 172.21.15.91:0/3058669646 learned_addr learned my addr 172.21.15.91:0/3058669646 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3dd400640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 0x7fb3e00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7fb3de800640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e01609b0 0x7fb3e00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.240+0000 7f2250600640 1 -- start start 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dde00640 1 -- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e01609b0 msgr2=0x7fb3e00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dde00640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e01609b0 0x7fb3e00c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dde00640 1 -- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 msgr2=0x7fb3e00cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dde00640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 0x7fb3e00cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dde00640 1 -- 172.21.15.91:0/3058669646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00094d0 con 0x7fb3e00707b0 2024-02-02T13:24:49.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3a200538d0 con 0x7f3a300ce4c0 2024-02-02T13:24:49.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.241+0000 7fb3dd400640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 0x7fb3e00cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:49.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22480707b0 0x7f2248153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7fb3dde00640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009870 tx=0x7fb3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22480ce530 0x7f2248156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22481575c0 con 0x7f22481609b0 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22481544e0 con 0x7f22480ce530 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f2250600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2248154780 con 0x7f22480707b0 2024-02-02T13:24:49.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.242+0000 7f224e600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53442/0 (socket says 172.21.15.91:53442) 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 -- 172.21.15.91:0/2396927700 learned_addr learned my addr 172.21.15.91:0/2396927700 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0032910 con 0x7fb3e00707b0 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 -- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22480707b0 msgr2=0x7f2248153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22480707b0 0x7f2248153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 -- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22480ce530 msgr2=0x7f2248156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22480ce530 0x7f2248156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7f224e600640 1 -- 172.21.15.91:0/2396927700 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300094d0 con 0x7f22481609b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.244+0000 7f224e600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f224000c780 tx=0x7f224000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0032a70 con 0x7fb3e00707b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.243+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0032e20 con 0x7fb3e00707b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.244+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00b4080 con 0x7fb3e00707b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.244+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e00c9500 con 0x7fb3e00707b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.245+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e0066cf0 con 0x7fb3e00707b0 2024-02-02T13:24:49.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.247+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fefc4039050 con 0x7fefd40e78e0 2024-02-02T13:24:49.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.248+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9cbc054380 con 0x7f9cc4066020 2024-02-02T13:24:49.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.248+0000 7fefd2a00640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 0x7fefb406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.248+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefc40aa510 con 0x7fefd40e78e0 2024-02-02T13:24:49.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.249+0000 7fefdaa00640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 0x7fefb406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.250+0000 7fefdaa00640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 0x7fefb406c970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0011e0 tx=0x7fefcc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.254+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fefc4075b70 con 0x7fefd40e78e0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.254+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240004b60 con 0x7f22481609b0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.254+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248154a60 con 0x7f22481609b0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.255+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248154d40 con 0x7f22481609b0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.256+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248157aa0 con 0x7f22481609b0 2024-02-02T13:24:49.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.257+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224000ce90 con 0x7f22481609b0 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.257+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240006b50 con 0x7f22481609b0 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.258+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb3d003a040 con 0x7fb3e00707b0 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.259+0000 7fb3d6000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 0x7fb3b806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.259+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3d00aa340 con 0x7fb3e00707b0 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.261+0000 7fb3dd400640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 0x7fb3b806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.262+0000 7fb3dd400640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 0x7fb3b806c970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0159c80 tx=0x7fb3c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.261+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2240006cb0 con 0x7f22481609b0 2024-02-02T13:24:49.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.262+0000 7f2245600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 0x7f222006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:49.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.263+0000 7f224dc00640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 0x7f222006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:49.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.264+0000 7f224dc00640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 0x7f222006c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2238006fd0 tx=0x7f2238008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:49.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.268+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb3d00759a0 con 0x7fb3e00707b0 2024-02-02T13:24:49.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.269+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2240019350 con 0x7f22481609b0 2024-02-02T13:24:49.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.280+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2240054790 con 0x7f22481609b0 2024-02-02T13:24:49.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.634+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7288066cf0 con 0x7f72880ce530 2024-02-02T13:24:49.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.635+0000 7f7285600640 1 -- 172.21.15.91:0/2066827972 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7278057060 con 0x7f72880ce530 2024-02-02T13:24:49.638 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 msgr2=0x7f726006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f48dd400640 1 -- 172.21.15.91:0/1305473969 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48a80057d0 con 0x7f48d00e3190 2024-02-02T13:24:49.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 0x7f726006c8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72800073d0 tx=0x7f728000b040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 msgr2=0x7f72881f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f727800e7e0 tx=0x7f727800eca0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 shutdown_connections 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f726006a430 0x7f726006c8f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72881609b0 0x7f72880ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72880ce530 0x7f72881f67f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 --2- 172.21.15.91:0/2066827972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72880707b0 0x7f72881f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.639+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 >> 172.21.15.91:0/2066827972 conn(0x7f72880d3020 msgr2=0x7f72880cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.640+0000 7f48cfe00640 1 -- 172.21.15.91:0/1305473969 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48c0017090 con 0x7f48d00e3190 2024-02-02T13:24:49.642 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:49.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.643+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 shutdown_connections 2024-02-02T13:24:49.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.643+0000 7f7290000640 1 -- 172.21.15.91:0/2066827972 wait complete. 2024-02-02T13:24:49.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.644+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 msgr2=0x7f48b006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.644+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 0x7f48b006c9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48d8079d30 tx=0x7f48d806c040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.644+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 msgr2=0x7f48d0188990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.644+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 0x7f48d0188990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48c000c390 tx=0x7f48c000c850 comp rx=0 tx=0).stop 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 shutdown_connections 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f48b006a500 0x7f48b006c9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00e3190 0x7f48d0188990 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00542d0 0x7f48d0050190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 --2- 172.21.15.91:0/1305473969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48d0038180 0x7f48d0053030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 >> 172.21.15.91:0/1305473969 conn(0x7f48d00590c0 msgr2=0x7f48d0038a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 shutdown_connections 2024-02-02T13:24:49.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.645+0000 7f48cd600640 1 -- 172.21.15.91:0/1305473969 wait complete. 2024-02-02T13:24:49.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.648+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3a300d0c80 con 0x7f3a300ce4c0 2024-02-02T13:24:49.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.649+0000 7f3a26a00640 1 -- 172.21.15.91:0/2782993403 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a20017020 con 0x7f3a300ce4c0 2024-02-02T13:24:49.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.650+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0030052d00 con 0x7f003000c1f0 2024-02-02T13:24:49.652 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:49.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.652+0000 7f002fe00640 1 -- 172.21.15.91:0/2964664658 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00380bd020 con 0x7f003000c1f0 2024-02-02T13:24:49.656 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.652+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 msgr2=0x7f3a0806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 0x7f3a0806c920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a18036000 tx=0x7f3a1803d040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 msgr2=0x7f3a301f9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000c400 tx=0x7f3a2000c8c0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 shutdown_connections 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3a0806a460 0x7f3a0806c920 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a300d1dc0 0x7f3a3006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a3015d260 0x7f3a30153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 --2- 172.21.15.91:0/2782993403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a300ce4c0 0x7f3a301f9860 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.653+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 >> 172.21.15.91:0/2782993403 conn(0x7f3a300cfa60 msgr2=0x7f3a300d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.654+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 shutdown_connections 2024-02-02T13:24:49.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.654+0000 7f3a35c00640 1 -- 172.21.15.91:0/2782993403 wait complete. 2024-02-02T13:24:49.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.655+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 msgr2=0x7f001006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 0x7f001006c920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0028006fd0 tx=0x7f0028008040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 msgr2=0x7f00300e5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f003804f8f0 tx=0x7f0038067bb0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 shutdown_connections 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f001006a460 0x7f001006c920 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0030050230 0x7f003004af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00300e2e10 0x7f00300d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 --2- 172.21.15.91:0/2964664658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f003000c1f0 0x7f00300e5220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.656+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 >> 172.21.15.91:0/2964664658 conn(0x7f0030054c10 msgr2=0x7f00300e0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.657+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 shutdown_connections 2024-02-02T13:24:49.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.657+0000 7f003d400640 1 -- 172.21.15.91:0/2964664658 wait complete. 2024-02-02T13:24:49.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.664+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9c940057d0 con 0x7f9cc4066020 2024-02-02T13:24:49.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.665+0000 7f9cc1600640 1 -- 172.21.15.91:0/3233020079 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9cbc023090 con 0x7f9cc4066020 2024-02-02T13:24:49.666 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.669+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 msgr2=0x7f9c9c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.669+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 0x7f9c9c06c9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ca340 tx=0x7f9cb4007a40 comp rx=0 tx=0).stop 2024-02-02T13:24:49.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.669+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 msgr2=0x7f9cc4153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.669+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00ba50 tx=0x7f9cbc00bf10 comp rx=0 tx=0).stop 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 shutdown_connections 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c9c06a500 0x7f9c9c06c9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cc4160740 0x7f9cc4150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cc40af230 0x7f9cc4150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 --2- 172.21.15.91:0/3233020079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cc4066020 0x7f9cc4153010 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 >> 172.21.15.91:0/3233020079 conn(0x7f9cc40b18f0 msgr2=0x7f9cc406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 shutdown_connections 2024-02-02T13:24:49.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.670+0000 7f9ccbe00640 1 -- 172.21.15.91:0/3233020079 wait complete. 2024-02-02T13:24:49.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.673+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fefd40e8ec0 con 0x7fefd40e78e0 2024-02-02T13:24:49.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.673+0000 7fefd2a00640 1 -- 172.21.15.91:0/3588107415 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fefd40e8ec0 con 0x7fefd40e78e0 2024-02-02T13:24:49.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-02-02T13:24:49.676 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.677+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 msgr2=0x7fefb406c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.677+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 0x7fefb406c970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0011e0 tx=0x7fefcc009040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.677+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 msgr2=0x7fefd40e3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.677+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefc402bd70 tx=0x7fefc40046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 shutdown_connections 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fefb406a4b0 0x7fefb406c970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefd40eb2f0 0x7fefd40e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fefd40e78e0 0x7fefd40e3230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 --2- 172.21.15.91:0/3588107415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefd4006940 0x7fefd404a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.678+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 >> 172.21.15.91:0/3588107415 conn(0x7fefd4054c40 msgr2=0x7fefd404af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-02-02T13:24:49.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.679+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 shutdown_connections 2024-02-02T13:24:49.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.679+0000 7fefe1800640 1 -- 172.21.15.91:0/3588107415 wait complete. 2024-02-02T13:24:49.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-02-02T13:24:49.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.683+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb3e0160d90 con 0x7fb3e00707b0 2024-02-02T13:24:49.685 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.683+0000 7fb3d6000640 1 -- 172.21.15.91:0/3058669646 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3d0038020 con 0x7fb3e00707b0 2024-02-02T13:24:49.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.683+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2248157aa0 con 0x7f22481609b0 2024-02-02T13:24:49.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.684+0000 7f2245600640 1 -- 172.21.15.91:0/2396927700 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2248157aa0 con 0x7f22481609b0 2024-02-02T13:24:49.686 INFO:teuthology.orchestra.run.smithi091.stdout:55834574853 2024-02-02T13:24:49.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 msgr2=0x7f222006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 0x7f222006c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2238006fd0 tx=0x7f2238008040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 msgr2=0x7f22480c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f224000c780 tx=0x7f224000cc40 comp rx=0 tx=0).stop 2024-02-02T13:24:49.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 shutdown_connections 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222006a430 0x7f222006c8f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22481609b0 0x7f22480c8be0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22480ce530 0x7f2248156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 --2- 172.21.15.91:0/2396927700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22480707b0 0x7f2248153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 >> 172.21.15.91:0/2396927700 conn(0x7f22480d3020 msgr2=0x7f22480c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 shutdown_connections 2024-02-02T13:24:49.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.686+0000 7f2250600640 1 -- 172.21.15.91:0/2396927700 wait complete. 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 msgr2=0x7fb3b806c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 0x7fb3b806c970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0159c80 tx=0x7fb3c8008040 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 msgr2=0x7fb3e00b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009870 tx=0x7fb3d00046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 shutdown_connections 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb3b806a4b0 0x7fb3b806c970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3e01609b0 0x7fb3e00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e00ce530 0x7fb3e00cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 --2- 172.21.15.91:0/3058669646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3e00707b0 0x7fb3e00b4880 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:49.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 >> 172.21.15.91:0/3058669646 conn(0x7fb3e00d3020 msgr2=0x7fb3e015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:49.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 shutdown_connections 2024-02-02T13:24:49.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:49.687+0000 7fb3e5000640 1 -- 172.21.15.91:0/3058669646 wait complete. 2024-02-02T13:24:49.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-02-02T13:24:49.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-02-02T13:24:49.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-02-02T13:24:49.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-02-02T13:24:50.672 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-02T13:24:50.676 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-02T13:24:50.682 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-02T13:24:50.683 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-02T13:24:50.701 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-02T13:24:50.702 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-02T13:24:50.706 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-02T13:24:50.708 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-02T13:24:50.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.959+0000 7f3105800640 1 Processor -- start 2024-02-02T13:24:50.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.959+0000 7f3105800640 1 -- start start 2024-02-02T13:24:50.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.961+0000 7f3105800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3100158f10 0x7f31000ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.961+0000 7f3105800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f31000b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.961+0000 7f3105800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31000b0a00 0x7f310015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3104e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31000b0a00 0x7f310015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3104e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31000b0a00 0x7f310015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54368/0 (socket says 172.21.15.91:54368) 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f30ff400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f31000b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3104e00640 1 -- 172.21.15.91:0/1486763 learned_addr learned my addr 172.21.15.91:0/1486763 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000cb1e0 con 0x7f31000b0a00 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000b0e50 con 0x7f3100158f10 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.962+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c9090 con 0x7f310006be00 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.963+0000 7f3104e00640 1 -- 172.21.15.91:0/1486763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 msgr2=0x7f31000b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.963+0000 7f3104e00640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f31000b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.963+0000 7f3104e00640 1 -- 172.21.15.91:0/1486763 >> v2:172.21.15.113:3300/0 conn(0x7f3100158f10 msgr2=0x7f31000ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.963+0000 7f3104e00640 1 --2- 172.21.15.91:0/1486763 >> v2:172.21.15.113:3300/0 conn(0x7f3100158f10 0x7f31000ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.963+0000 7f3104e00640 1 -- 172.21.15.91:0/1486763 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310006c2a0 con 0x7f31000b0a00 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.965+0000 7f30ff400640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f31000b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:50.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.965+0000 7f3104e00640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f310015ca70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f310006c5c0 tx=0x7f30f8034a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fc216321f7f6cd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:50.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.966+0000 7f30fea00640 1 -- 172.21.15.91:0/1486763 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f80114e0 con 0x7f31000b0a00 2024-02-02T13:24:50.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.966+0000 7f30fea00640 1 -- 172.21.15.91:0/1486763 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f8011640 con 0x7f31000b0a00 2024-02-02T13:24:50.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.967+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 msgr2=0x7f310015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:50.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.967+0000 7f3105800640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f310015ca70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f310006c5c0 tx=0x7f30f8034a40 comp rx=0 tx=0).stop 2024-02-02T13:24:50.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 shutdown_connections 2024-02-02T13:24:50.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f310015ca70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 --2- 172.21.15.91:0/1486763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f31000b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 --2- 172.21.15.91:0/1486763 >> v2:172.21.15.113:3300/0 conn(0x7f3100158f10 0x7f31000ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 >> 172.21.15.91:0/1486763 conn(0x7f31000cb4c0 msgr2=0x7f31000cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:50.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 shutdown_connections 2024-02-02T13:24:50.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.968+0000 7f3105800640 1 -- 172.21.15.91:0/1486763 wait complete. 2024-02-02T13:24:50.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.972+0000 7f3105800640 1 Processor -- start 2024-02-02T13:24:50.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.972+0000 7f3105800640 1 -- start start 2024-02-02T13:24:50.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.974+0000 7f3105800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f3100156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.975+0000 7f3105800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.975+0000 7f30ff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.976+0000 7f30ff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54372/0 (socket says 172.21.15.91:54372) 2024-02-02T13:24:50.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.976+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f3100156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.976+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f3100156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39092/0 (socket says 172.21.15.91:39092) 2024-02-02T13:24:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.976+0000 7f30ff400640 1 -- 172.21.15.91:0/456927548 learned_addr learned my addr 172.21.15.91:0/456927548 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.976+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 0x7f31000ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.977+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c8e60 con 0x7f31000b0a00 2024-02-02T13:24:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.977+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31001f2530 con 0x7f3100158f10 2024-02-02T13:24:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.977+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31001f1640 con 0x7f310006be00 2024-02-02T13:24:50.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.977+0000 7f3104e00640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 0x7f31000ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.978+0000 7f30ff400640 1 -- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 msgr2=0x7f3100156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:50.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.979+0000 7f30ff400640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f3100156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.979+0000 7f30ff400640 1 -- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 msgr2=0x7f31000ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:50.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.979+0000 7f30ff400640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 0x7f31000ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:50.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.979+0000 7f30ff400640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8014660 con 0x7f31000b0a00 2024-02-02T13:24:50.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.979+0000 7f3104e00640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 0x7f31000ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:50.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.980+0000 7f30ff400640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c780 tx=0x7f30e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:50.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.981+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800a7c0 con 0x7f31000b0a00 2024-02-02T13:24:50.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.981+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e800a920 con 0x7f31000b0a00 2024-02-02T13:24:50.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.981+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31001f1920 con 0x7f31000b0a00 2024-02-02T13:24:50.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.982+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800acd0 con 0x7f31000b0a00 2024-02-02T13:24:50.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.984+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31001f1ee0 con 0x7f31000b0a00 2024-02-02T13:24:50.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.985+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310004dff0 con 0x7f31000b0a00 2024-02-02T13:24:50.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.988+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f30e8012420 con 0x7f31000b0a00 2024-02-02T13:24:50.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.989+0000 7f30f7e00640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 0x7f30d806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:50.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.990+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30e8090350 con 0x7f31000b0a00 2024-02-02T13:24:50.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.992+0000 7f30ffe00640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 0x7f30d806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:50.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.997+0000 7f30ffe00640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 0x7f30d806c940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30f00098e0 tx=0x7f30f0009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:50.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:50.998+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30e808d060 con 0x7f31000b0a00 2024-02-02T13:24:51.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.001+0000 7fc297800640 1 Processor -- start 2024-02-02T13:24:51.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.002+0000 7fc297800640 1 -- start start 2024-02-02T13:24:51.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.003+0000 7fc297800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2880e34d0 0x7fc288054420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.003+0000 7fc297800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc2880582f0 0x7fc288056d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.003+0000 7fc297800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc28803c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc295800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc28803c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc294e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2880e34d0 0x7fc288054420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.003+0000 7fc297800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc288058e00 con 0x7fc2880582f0 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc297800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc288057bc0 con 0x7fc2880e34d0 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc297800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288057ea0 con 0x7fc288057260 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc28fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc2880582f0 0x7fc288056d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc295800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc28803c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39098/0 (socket says 172.21.15.91:39098) 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc295800640 1 -- 172.21.15.91:0/2008213840 learned_addr learned my addr 172.21.15.91:0/2008213840 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.005+0000 7fc28fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc2880582f0 0x7fc288056d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54384/0 (socket says 172.21.15.91:54384) 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.006+0000 7fc28fe00640 1 -- 172.21.15.91:0/2008213840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 msgr2=0x7fc28803c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.006+0000 7fc28fe00640 1 --2- 172.21.15.91:0/2008213840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc28803c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.006+0000 7fc28fe00640 1 -- 172.21.15.91:0/2008213840 >> v2:172.21.15.113:3300/0 conn(0x7fc2880e34d0 msgr2=0x7fc288054420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.006+0000 7fc28fe00640 1 --2- 172.21.15.91:0/2008213840 >> v2:172.21.15.113:3300/0 conn(0x7fc2880e34d0 0x7fc288054420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.006+0000 7fc28fe00640 1 -- 172.21.15.91:0/2008213840 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288053840 con 0x7fc2880582f0 2024-02-02T13:24:51.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.008+0000 7fc28fe00640 1 --2- 172.21.15.91:0/2008213840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288056d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2800097a0 tx=0x7fc28002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5498e3dab0ff122 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.009+0000 7fc28f400640 1 -- 172.21.15.91:0/2008213840 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800317c0 con 0x7fc2880582f0 2024-02-02T13:24:51.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.009+0000 7fc28f400640 1 -- 172.21.15.91:0/2008213840 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280031920 con 0x7fc2880582f0 2024-02-02T13:24:51.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.009+0000 7fc28f400640 1 -- 172.21.15.91:0/2008213840 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280031cd0 con 0x7fc2880582f0 2024-02-02T13:24:51.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 -- 172.21.15.91:0/2008213840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 msgr2=0x7fc288056d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 --2- 172.21.15.91:0/2008213840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288056d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2800097a0 tx=0x7fc28002b860 comp rx=0 tx=0).stop 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 -- 172.21.15.91:0/2008213840 shutdown_connections 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 --2- 172.21.15.91:0/2008213840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc28803c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 --2- 172.21.15.91:0/2008213840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288056d20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 --2- 172.21.15.91:0/2008213840 >> v2:172.21.15.113:3300/0 conn(0x7fc2880e34d0 0x7fc288054420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 -- 172.21.15.91:0/2008213840 >> 172.21.15.91:0/2008213840 conn(0x7fc288059150 msgr2=0x7fc288059560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 -- 172.21.15.91:0/2008213840 shutdown_connections 2024-02-02T13:24:51.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.010+0000 7fc297800640 1 -- 172.21.15.91:0/2008213840 wait complete. 2024-02-02T13:24:51.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.013+0000 7fc297800640 1 Processor -- start 2024-02-02T13:24:51.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.013+0000 7fc297800640 1 -- start start 2024-02-02T13:24:51.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc2880d1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7f7d68000640 1 Processor -- start 2024-02-02T13:24:51.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.016+0000 7f7d68000640 1 -- start start 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288178350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28817c010 con 0x7fc2880582f0 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880d1850 con 0x7fc2880e34d0 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.015+0000 7fc297800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880d1b00 con 0x7fc288057260 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.016+0000 7fc295800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:49498/0 (socket says 172.21.15.91:49498) 2024-02-02T13:24:51.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 -- 172.21.15.91:0/3594351194 learned_addr learned my addr 172.21.15.91:0/3594351194 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 -- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 msgr2=0x7fc2880d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc2880d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 -- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 msgr2=0x7fc288178350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288178350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7fc295800640 1 -- 172.21.15.91:0/3594351194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280009400 con 0x7fc2880e34d0 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7f7d68000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d600cc740 0x7f7d60158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7f7d68000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d6006ff50 0x7f7d60070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7f7d68000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d600ccf20 con 0x7f7d600cf660 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7f7d68000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d600cd8c0 con 0x7f7d600cc740 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.017+0000 7f7d68000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d60070940 con 0x7f7d6006ff50 2024-02-02T13:24:51.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.018+0000 7fc295800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc29004f8f0 tx=0x7fc2900741e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.018+0000 7f7d64c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d600cc740 0x7f7d60158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.018+0000 7f7d64c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d600cc740 0x7f7d60158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54402/0 (socket says 172.21.15.91:54402) 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.018+0000 7f7d64c00640 1 -- 172.21.15.91:0/2817944384 learned_addr learned my addr 172.21.15.91:0/2817944384 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7f7d64c00640 1 -- 172.21.15.91:0/2817944384 >> v2:172.21.15.113:3300/0 conn(0x7f7d6006ff50 msgr2=0x7f7d60070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290067270 con 0x7fc2880e34d0 2024-02-02T13:24:51.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7f7d66000640 1 --2- 172.21.15.91:0/2817944384 >> v2:172.21.15.113:3300/0 conn(0x7f7d6006ff50 0x7f7d60070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880d1de0 con 0x7fc2880e34d0 2024-02-02T13:24:51.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7f7d64c00640 1 --2- 172.21.15.91:0/2817944384 >> v2:172.21.15.113:3300/0 conn(0x7f7d6006ff50 0x7f7d60070400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc290093670 con 0x7fc2880e34d0 2024-02-02T13:24:51.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28817c220 con 0x7fc2880e34d0 2024-02-02T13:24:51.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.020+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290067270 con 0x7fc2880e34d0 2024-02-02T13:24:51.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7f7d64c00640 1 -- 172.21.15.91:0/2817944384 >> v1:172.21.15.113:6789/0 conn(0x7f7d600cf660 legacy=0x7f7d600cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.019+0000 7f7d64c00640 1 -- 172.21.15.91:0/2817944384 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600cd200 con 0x7f7d600cc740 2024-02-02T13:24:51.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.021+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc260005250 con 0x7fc2880e34d0 2024-02-02T13:24:51.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.022+0000 7f7d64c00640 1 --2- 172.21.15.91:0/2817944384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cc740 0x7f7d60158fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d600cce50 tx=0x7f7d4802de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5c3d9872329a1d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.022+0000 7f7d5fe00640 1 -- 172.21.15.91:0/2817944384 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4802ebd0 con 0x7f7d600cc740 2024-02-02T13:24:51.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.023+0000 7f7d5fe00640 1 -- 172.21.15.91:0/2817944384 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d48032070 con 0x7f7d600cc740 2024-02-02T13:24:51.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.023+0000 7f7d5fe00640 1 -- 172.21.15.91:0/2817944384 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48038400 con 0x7f7d600cc740 2024-02-02T13:24:51.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.023+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc290090070 con 0x7fc2880e34d0 2024-02-02T13:24:51.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.024+0000 7f7d68000640 1 -- 172.21.15.91:0/2817944384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cc740 msgr2=0x7f7d60158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.024+0000 7fc28cc00640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 0x7fc26806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.024+0000 7f7d68000640 1 --2- 172.21.15.91:0/2817944384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cc740 0x7f7d60158fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d600cce50 tx=0x7f7d4802de20 comp rx=0 tx=0).stop 2024-02-02T13:24:51.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc290102910 con 0x7fc2880e34d0 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7fc294e00640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 0x7fc26806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 -- 172.21.15.91:0/2817944384 shutdown_connections 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 --2- 172.21.15.91:0/2817944384 >> v2:172.21.15.113:3300/0 conn(0x7f7d6006ff50 0x7f7d60070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 --2- 172.21.15.91:0/2817944384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cc740 0x7f7d60158fd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 -- 172.21.15.91:0/2817944384 >> 172.21.15.91:0/2817944384 conn(0x7f7d600ceb70 msgr2=0x7f7d600cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.024+0000 7f54fde00640 1 Processor -- start 2024-02-02T13:24:51.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.024+0000 7f54fde00640 1 -- start start 2024-02-02T13:24:51.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.026+0000 7fc294e00640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 0x7fc26806c950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc278009a40 tx=0x7fc278009500 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 -- 172.21.15.91:0/2817944384 shutdown_connections 2024-02-02T13:24:51.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.025+0000 7f7d68000640 1 -- 172.21.15.91:0/2817944384 wait complete. 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.027+0000 7f54fde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.027+0000 7f54fde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54f80cdbb0 0x7f54f80b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.027+0000 7f54fde00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f80b8da0 con 0x7f54f80b8000 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.027+0000 7f54fde00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f8071280 con 0x7f54f80cdbb0 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.027+0000 7f54fde00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80d16a0 con 0x7f54f80b6f80 2024-02-02T13:24:51.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f7d68000640 1 Processor -- start 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39116/0 (socket says 172.21.15.91:39116) 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f54f6a00640 1 -- 172.21.15.91:0/461405114 learned_addr learned my addr 172.21.15.91:0/461405114 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f7d68000640 1 -- start start 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.028+0000 7f54f7400640 1 --2- 172.21.15.91:0/461405114 >> v2:172.21.15.113:3300/0 conn(0x7f54f80cdbb0 0x7f54f80b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.029+0000 7f54f6a00640 1 -- 172.21.15.91:0/461405114 >> v2:172.21.15.113:3300/0 conn(0x7f54f80cdbb0 msgr2=0x7f54f80b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.029+0000 7f54f6a00640 1 --2- 172.21.15.91:0/461405114 >> v2:172.21.15.113:3300/0 conn(0x7f54f80cdbb0 0x7f54f80b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.029+0000 7f54f6a00640 1 -- 172.21.15.91:0/461405114 >> v1:172.21.15.113:6789/0 conn(0x7f54f80b8000 legacy=0x7f54f80d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.029+0000 7f54f6a00640 1 -- 172.21.15.91:0/461405114 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80d1980 con 0x7f54f80b6f80 2024-02-02T13:24:51.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.029+0000 7f6a50a00640 1 Processor -- start 2024-02-02T13:24:51.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f6a50a00640 1 -- start start 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d6006ff50 0x7f7d600b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d600cc740 0x7f7d6014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014b480 con 0x7f7d600cf660 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600c7ec0 con 0x7f7d600cc740 2024-02-02T13:24:51.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.030+0000 7f7d68000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600c72b0 con 0x7f7d6006ff50 2024-02-02T13:24:51.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f54f6a00640 1 --2- 172.21.15.91:0/461405114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e50 tx=0x7f54e002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e100d5ed77b9d670 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f2062e00640 1 Processor -- start 2024-02-02T13:24:51.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f2062e00640 1 -- start start 2024-02-02T13:24:51.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f54f5600640 1 -- 172.21.15.91:0/461405114 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e002c9f0 con 0x7f54f80b6f80 2024-02-02T13:24:51.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f54f5600640 1 -- 172.21.15.91:0/461405114 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e002cb50 con 0x7f54f80b6f80 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f54f5600640 1 -- 172.21.15.91:0/461405114 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00343f0 con 0x7f54f80b6f80 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a50a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15d1b0 0x7f6a4c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a50a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a50a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4c0d01c0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a50a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4c161c60 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a50a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c15dbe0 con 0x7f6a4c15d1b0 2024-02-02T13:24:51.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f3c35e00640 1 Processor -- start 2024-02-02T13:24:51.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f3c35e00640 1 -- start start 2024-02-02T13:24:51.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a4a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15d1b0 0x7f6a4c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 -- 172.21.15.91:0/461405114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 msgr2=0x7f54f80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 --2- 172.21.15.91:0/461405114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e50 tx=0x7f54e002be50 comp rx=0 tx=0).stop 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a49e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a49e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:49526/0 (socket says 172.21.15.91:49526) 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f6a49e00640 1 -- 172.21.15.91:0/1006409689 learned_addr learned my addr 172.21.15.91:0/1006409689 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f6a49e00640 1 -- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15d1b0 msgr2=0x7f6a4c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f3c35e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c280543f0 0x7f3c28056cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f3c35e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c28054880 con 0x7f3c28057230 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f3c35e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c28058dd0 con 0x7f3c280e34a0 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f3c35e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c28057b90 con 0x7f3c280543f0 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54406/0 (socket says 172.21.15.91:54406) 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 -- 172.21.15.91:0/1334725362 learned_addr learned my addr 172.21.15.91:0/1334725362 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d64c00640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d600cc740 0x7f7d6014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 -- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d6006ff50 msgr2=0x7f7d600b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d6006ff50 0x7f7d600b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 -- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d600cc740 msgr2=0x7f7d6014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.031+0000 7f7d66000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d600cc740 0x7f7d6014af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f7d66000640 1 -- 172.21.15.91:0/1334725362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4800b760 con 0x7f7d600cf660 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.032+0000 7f7d66000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800cc30 tx=0x7f7d5800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800ce40 con 0x7f7d600cf660 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58003990 con 0x7f7d600cf660 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003d00 con 0x7f7d600cf660 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d600c7590 con 0x7f7d600cf660 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 -- 172.21.15.91:0/461405114 shutdown_connections 2024-02-02T13:24:51.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 --2- 172.21.15.91:0/461405114 >> v2:172.21.15.113:3300/0 conn(0x7f54f80cdbb0 0x7f54f80b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 --2- 172.21.15.91:0/461405114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80b6f80 0x7f54f80cd670 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f54fde00640 1 -- 172.21.15.91:0/461405114 >> 172.21.15.91:0/461405114 conn(0x7f54f80d3020 msgr2=0x7f54f80d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d600c7b50 con 0x7f7d600cf660 2024-02-02T13:24:51.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f54fde00640 1 -- 172.21.15.91:0/461405114 shutdown_connections 2024-02-02T13:24:51.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f54fde00640 1 -- 172.21.15.91:0/461405114 wait complete. 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f2062e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f205c15d260 0x7f205c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f2062e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 0x7f205c070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f2062e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c15d640 con 0x7f205c0d1dc0 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f2062e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0d2530 con 0x7f205c15d260 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f2062e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0d2810 con 0x7f205c0ce4c0 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f6a49e00640 1 --2- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15d1b0 0x7f6a4c15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.033+0000 7f6a49e00640 1 -- 172.21.15.91:0/1006409689 >> v1:172.21.15.91:6789/0 conn(0x7f6a4c0cfd10 legacy=0x7f6a4c160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f6a49e00640 1 -- 172.21.15.91:0/1006409689 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c15dec0 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f3c34a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3c280e34a0 0x7f3c280582c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:49438/0 (socket says 172.21.15.91:49438) 2024-02-02T13:24:51.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f3c34a00640 1 -- 172.21.15.91:0/3420455758 learned_addr learned my addr 172.21.15.91:0/3420455758 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:51.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.034+0000 7f3c2fe00640 1 --2- 172.21.15.91:0/3420455758 >> v2:172.21.15.113:3300/0 conn(0x7f3c280543f0 0x7f3c28056cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.035+0000 7f205b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f205c0d1dc0 0x7f205c0cddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50176/0 (socket says 172.21.15.91:50176) 2024-02-02T13:24:51.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.035+0000 7f205b400640 1 -- 172.21.15.91:0/1213840185 learned_addr learned my addr 172.21.15.91:0/1213840185 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.035+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d60066cf0 con 0x7f7d600cf660 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f3c2fe00640 1 -- 172.21.15.91:0/3420455758 >> v1:172.21.15.113:6789/0 conn(0x7f3c280e34a0 legacy=0x7f3c280582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f3c2fe00640 1 -- 172.21.15.91:0/3420455758 >> v1:172.21.15.91:6789/0 conn(0x7f3c28057230 legacy=0x7f3c2803c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f3c2fe00640 1 -- 172.21.15.91:0/3420455758 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28057e70 con 0x7f3c280543f0 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f2060e00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 0x7f205c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f205be00640 1 --2- 172.21.15.91:0/1213840185 >> v2:172.21.15.113:3300/0 conn(0x7f205c15d260 0x7f205c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f6a49e00640 1 --2- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009870 tx=0x7f6a3402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=161405aaa0b2cba server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f54fde00640 1 Processor -- start 2024-02-02T13:24:51.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f205be00640 1 -- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 msgr2=0x7f205c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f205be00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 0x7f205c070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f205be00640 1 -- 172.21.15.91:0/1213840185 >> v1:172.21.15.91:6789/0 conn(0x7f205c0d1dc0 legacy=0x7f205c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f205aa00640 1 -- 172.21.15.91:0/1213840185 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611073111 0 0) 0x7f205c15d640 con 0x7f205c0d1dc0 2024-02-02T13:24:51.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f205be00640 1 -- 172.21.15.91:0/1213840185 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0d2af0 con 0x7f205c15d260 2024-02-02T13:24:51.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f2060e00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 0x7f205c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:51.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.036+0000 7f54fde00640 1 -- start start 2024-02-02T13:24:51.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.037+0000 7f3c2fe00640 1 --2- 172.21.15.91:0/3420455758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c28056cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c200097d0 tx=0x7f3c2002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7954947eda71ae69 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54f80b6f80 0x7f54f8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f205be00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c0b8b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097a0 tx=0x7f204c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e0c5943a14d1ea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80cdbb0 0x7f54f815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f815fa90 con 0x7f54f80b8000 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80c7f70 con 0x7f54f80b6f80 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54fde00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f815f4d0 con 0x7f54f80cdbb0 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54f6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54f6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54412/0 (socket says 172.21.15.91:54412) 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.038+0000 7f54f6000640 1 -- 172.21.15.91:0/676876956 learned_addr learned my addr 172.21.15.91:0/676876956 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6a00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54f80b6f80 0x7f54f8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f7400640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80cdbb0 0x7f54f815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc2900d1a00 con 0x7fc2880e34d0 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7d58004020 con 0x7f7d600cf660 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 -- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80cdbb0 msgr2=0x7f54f815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80cdbb0 0x7f54f815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 -- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54f80b6f80 msgr2=0x7f54f8148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54f80b6f80 0x7f54f8148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 -- 172.21.15.91:0/676876956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e0009b30 con 0x7f54f80b8000 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f6a49400640 1 -- 172.21.15.91:0/1006409689 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34031640 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f6a49400640 1 -- 172.21.15.91:0/1006409689 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a340317a0 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f6a49400640 1 -- 172.21.15.91:0/1006409689 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34031b30 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f205aa00640 1 -- 172.21.15.91:0/1213840185 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c031780 con 0x7f205c15d260 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f205aa00640 1 -- 172.21.15.91:0/1213840185 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0318e0 con 0x7f205c15d260 2024-02-02T13:24:51.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f205aa00640 1 -- 172.21.15.91:0/1213840185 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c031c70 con 0x7f205c15d260 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f3c2f400640 1 -- 172.21.15.91:0/3420455758 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2002d040 con 0x7f3c280543f0 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f3c2f400640 1 -- 172.21.15.91:0/3420455758 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20031810 con 0x7f3c280543f0 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f3c2f400640 1 -- 172.21.15.91:0/3420455758 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20031ba0 con 0x7f3c280543f0 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.039+0000 7f54f6000640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c430 tx=0x7f54e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.040+0000 7f6a50a00640 1 -- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15e6c0 msgr2=0x7f6a4c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.040+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009870 tx=0x7f6a3402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.040+0000 7f2062e00640 1 -- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 msgr2=0x7f205c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.040+0000 7f2062e00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c0b8b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097a0 tx=0x7f204c02b820 comp rx=0 tx=0).stop 2024-02-02T13:24:51.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f3c35e00640 1 -- 172.21.15.91:0/3420455758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 msgr2=0x7f3c28056cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f3c35e00640 1 --2- 172.21.15.91:0/3420455758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c28056cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c200097d0 tx=0x7f3c2002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.040+0000 7f7d5d600640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 0x7f7d3807d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d580890f0 con 0x7f7d600cf660 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f2062e00640 1 -- 172.21.15.91:0/1213840185 shutdown_connections 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f2062e00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0ce4c0 0x7f205c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f2062e00640 1 --2- 172.21.15.91:0/1213840185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c0b8b20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f2062e00640 1 -- 172.21.15.91:0/1213840185 >> 172.21.15.91:0/1213840185 conn(0x7f205c0cfa60 msgr2=0x7f205c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f3c35e00640 1 -- 172.21.15.91:0/3420455758 shutdown_connections 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f6a50a00640 1 -- 172.21.15.91:0/1006409689 shutdown_connections 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15e6c0 0x7f6a4c0cf7d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1006409689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15d1b0 0x7f6a4c15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f6a50a00640 1 -- 172.21.15.91:0/1006409689 >> 172.21.15.91:0/1006409689 conn(0x7f6a4c065910 msgr2=0x7f6a4c065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f3c35e00640 1 --2- 172.21.15.91:0/3420455758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c28056cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f3c35e00640 1 -- 172.21.15.91:0/3420455758 >> 172.21.15.91:0/3420455758 conn(0x7f3c28059120 msgr2=0x7f3c28059530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f2062e00640 1 -- 172.21.15.91:0/1213840185 shutdown_connections 2024-02-02T13:24:51.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.041+0000 7f6a50a00640 1 -- 172.21.15.91:0/1006409689 shutdown_connections 2024-02-02T13:24:51.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.042+0000 7f2062e00640 1 -- 172.21.15.91:0/1213840185 wait complete. 2024-02-02T13:24:51.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.042+0000 7f3c35e00640 1 -- 172.21.15.91:0/3420455758 shutdown_connections 2024-02-02T13:24:51.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.042+0000 7f7d65600640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 0x7f7d3807d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.042+0000 7f3c35e00640 1 -- 172.21.15.91:0/3420455758 wait complete. 2024-02-02T13:24:51.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.042+0000 7f6a50a00640 1 -- 172.21.15.91:0/1006409689 wait complete. 2024-02-02T13:24:51.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.043+0000 7f7d65600640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 0x7f7d3807d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d50003cc0 tx=0x7f7d500079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f3c35e00640 1 Processor -- start 2024-02-02T13:24:51.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f3c35e00640 1 -- start start 2024-02-02T13:24:51.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f2062e00640 1 Processor -- start 2024-02-02T13:24:51.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f6a50a00640 1 Processor -- start 2024-02-02T13:24:51.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f6a50a00640 1 -- start start 2024-02-02T13:24:51.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f09bc600640 1 Processor -- start 2024-02-02T13:24:51.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e80052d0 con 0x7f54f80b8000 2024-02-02T13:24:51.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7d58054770 con 0x7f7d600cf660 2024-02-02T13:24:51.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8005430 con 0x7f54f80b8000 2024-02-02T13:24:51.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f815f7b0 con 0x7f54f80b8000 2024-02-02T13:24:51.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 0x7f6a4c1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a4c15d1b0 0x7f6a4c155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 0x7f6a4c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1f2ac0 con 0x7f6a4c15d1b0 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0ca010 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f6a50a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0ca2f0 con 0x7f6a4c15e6c0 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.045+0000 7f2062e00640 1 -- start start 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f205c0ce4c0 0x7f205c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c06e6d0 con 0x7f205c0ce4c0 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c06d630 con 0x7f205c15d260 2024-02-02T13:24:51.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2062e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c06d910 con 0x7f205c0d1dc0 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f2060e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c281782a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280e34a0 0x7f3c28052760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280e3880 con 0x7f3c280e34a0 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280e3ad0 con 0x7f3c280543f0 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f3c35e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280e3db0 con 0x7f3c28057230 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.046+0000 7f09bc600640 1 -- start start 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f6a4b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 0x7f6a4c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f6a4b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 0x7f6a4c155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39182/0 (socket says 172.21.15.91:39182) 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f09bc600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09b4159910 0x7f09b40cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f09bc600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40b0f80 0x7f09b406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f09bc600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b40ccd00 con 0x7f09b40704a0 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f09bc600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b4159cf0 con 0x7f09b4159910 2024-02-02T13:24:51.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f09bc600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40b1de0 con 0x7f09b40b0f80 2024-02-02T13:24:51.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f6a4a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 0x7f6a4c1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f3c2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39174/0 (socket says 172.21.15.91:39174) 2024-02-02T13:24:51.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f3c2fe00640 1 -- 172.21.15.91:0/4268685203 learned_addr learned my addr 172.21.15.91:0/4268685203 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c35400640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280e34a0 0x7f3c28052760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f6a4b200640 1 -- 172.21.15.91:0/1730522953 learned_addr learned my addr 172.21.15.91:0/1730522953 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c34a00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c281782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f6a49e00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a4c15d1b0 0x7f6a4c155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c2fe00640 1 -- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 msgr2=0x7f3c281782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c2fe00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c281782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f09b9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09b4159910 0x7f09b40cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f09ba600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f09b40704a0 0x7f09b415d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50192/0 (socket says 172.21.15.91:50192) 2024-02-02T13:24:51.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003810 con 0x7f54f80b8000 2024-02-02T13:24:51.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.047+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f8148590 con 0x7f54f80b8000 2024-02-02T13:24:51.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c2fe00640 1 -- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280e34a0 msgr2=0x7f3c28052760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c2fe00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280e34a0 0x7f3c28052760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.048+0000 7f3c2fe00640 1 -- 172.21.15.91:0/4268685203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20009400 con 0x7f3c28057230 2024-02-02T13:24:51.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f09ba600640 1 -- 172.21.15.91:0/3210023926 learned_addr learned my addr 172.21.15.91:0/3210023926 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:51.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f3c34a00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c281782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:51.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f6a4a800640 1 -- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 msgr2=0x7f6a4c155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f09b9200640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40b0f80 0x7f09b406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f6a4a800640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 0x7f6a4c155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f6a4a800640 1 -- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a4c15d1b0 msgr2=0x7f6a4c155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f6a4a800640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a4c15d1b0 0x7f6a4c155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f6a4a800640 1 -- 172.21.15.91:0/1730522953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340094d0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f6a4a800640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 0x7f6a4c1f2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00d5f0 tx=0x7f6a3c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f09b9c00640 1 -- 172.21.15.91:0/3210023926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40b0f80 msgr2=0x7f09b406ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f09b9c00640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40b0f80 0x7f09b406ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f09b9c00640 1 -- 172.21.15.91:0/3210023926 >> v1:172.21.15.91:6789/0 conn(0x7f09b40704a0 legacy=0x7f09b415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f09b9c00640 1 -- 172.21.15.91:0/3210023926 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b40b1540 con 0x7f09b4159910 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0125f0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c012750 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c012b00 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f2060e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:49560/0 (socket says 172.21.15.91:49560) 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f205b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f205b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39178/0 (socket says 172.21.15.91:39178) 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.050+0000 7f205b400640 1 -- 172.21.15.91:0/600771095 learned_addr learned my addr 172.21.15.91:0/600771095 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.049+0000 7f205be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f205c0ce4c0 0x7f205c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f205b400640 1 -- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 msgr2=0x7f205c15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f205b400640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c15a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f205b400640 1 -- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f205c0ce4c0 msgr2=0x7f205c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f205b400640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f205c0ce4c0 0x7f205c0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f3c2fe00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3c200097d0 tx=0x7f3c200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c0ca5d0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f09b9c00640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b40cc7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c860 tx=0x7f09a402c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a11f8689b51a0e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f205b400640 1 -- 172.21.15.91:0/600771095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009a70 con 0x7f205c0d1dc0 2024-02-02T13:24:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f205b400640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f204400c4a0 tx=0x7f204400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f09b3e00640 1 -- 172.21.15.91:0/3210023926 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a402ce10 con 0x7f09b4159910 2024-02-02T13:24:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.051+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f80c7a60 con 0x7f54f80b8000 2024-02-02T13:24:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c157cb0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f09b3e00640 1 -- 172.21.15.91:0/3210023926 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4030070 con 0x7f09b4159910 2024-02-02T13:24:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2003ae60 con 0x7f3c28057230 2024-02-02T13:24:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c200443f0 con 0x7f3c28057230 2024-02-02T13:24:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f09b3e00640 1 -- 172.21.15.91:0/3210023926 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4036600 con 0x7f09b4159910 2024-02-02T13:24:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.052+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20044780 con 0x7f3c28057230 2024-02-02T13:24:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f09bc600640 1 -- 172.21.15.91:0/3210023926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 msgr2=0x7f09b40cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f09bc600640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b40cc7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c860 tx=0x7f09a402c060 comp rx=0 tx=0).stop 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20440075a0 con 0x7f205c0d1dc0 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20440047e0 con 0x7f205c0d1dc0 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044004c10 con 0x7f205c0d1dc0 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.054+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f54e8003970 con 0x7f54f80b8000 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c280e4090 con 0x7f3c28057230 2024-02-02T13:24:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.054+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c281787e0 con 0x7f3c28057230 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6a3c012c60 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f09bc600640 1 -- 172.21.15.91:0/3210023926 shutdown_connections 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f09bc600640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40b0f80 0x7f09b406ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f09bc600640 1 --2- 172.21.15.91:0/3210023926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b40cc7c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f09bc600640 1 -- 172.21.15.91:0/3210023926 >> 172.21.15.91:0/3210023926 conn(0x7f09b4069e30 msgr2=0x7f09b406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.053+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c009400 con 0x7f205c0d1dc0 2024-02-02T13:24:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.054+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c06ded0 con 0x7f205c0d1dc0 2024-02-02T13:24:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c0caa90 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c280e45b0 con 0x7f3c28057230 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.056+0000 7f6a42a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 0x7f6a2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.056+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a3c088dc0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.056+0000 7f09bc600640 1 -- 172.21.15.91:0/3210023926 shutdown_connections 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.056+0000 7f09bc600640 1 -- 172.21.15.91:0/3210023926 wait complete. 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.055+0000 7f54ef400640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 0x7f54d006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.056+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54e8087f50 con 0x7f54f80b8000 2024-02-02T13:24:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.057+0000 7f54f6a00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 0x7f54d006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.058+0000 7f09bc600640 1 Processor -- start 2024-02-02T13:24:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.058+0000 7f54f6a00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 0x7f54d006c920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54e0035000 tx=0x7f54e0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.058+0000 7f09bc600640 1 -- start start 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.058+0000 7f6a49e00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 0x7f6a2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09b40b0f80 0x7f09b40712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b41f2b20 con 0x7f09b4159910 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40c6540 con 0x7f09b40b0f80 2024-02-02T13:24:51.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f09bc600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b41f25f0 con 0x7f09b40704a0 2024-02-02T13:24:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.058+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2044012400 con 0x7f205c0d1dc0 2024-02-02T13:24:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f2053e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 0x7f203406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.059+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2044087bd0 con 0x7f205c0d1dc0 2024-02-02T13:24:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f09b9c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f09b9c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39190/0 (socket says 172.21.15.91:39190) 2024-02-02T13:24:51.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f09b9c00640 1 -- 172.21.15.91:0/3274731461 learned_addr learned my addr 172.21.15.91:0/3274731461 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c067400 con 0x7f205c0d1dc0 2024-02-02T13:24:51.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f6a49e00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 0x7f6a2406c8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a34036000 tx=0x7f6a3403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f09b9200640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09b40b0f80 0x7f09b40712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.060+0000 7f205be00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 0x7f203406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3c2004d450 con 0x7f3c28057230 2024-02-02T13:24:51.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09ba600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09b9c00640 1 -- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09b40b0f80 msgr2=0x7f09b40712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09b9c00640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09b40b0f80 0x7f09b40712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f205be00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 0x7f203406c8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f204c036000 tx=0x7f204c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f3c2cc00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 0x7f3c0806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09b9c00640 1 -- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 msgr2=0x7f09b415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09b9c00640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b415d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09b9c00640 1 -- 172.21.15.91:0/3274731461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a400c4c0 con 0x7f09b40704a0 2024-02-02T13:24:51.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.062+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c200aaaf0 con 0x7f3c28057230 2024-02-02T13:24:51.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.061+0000 7f09ba600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:24:51.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.062+0000 7f3c34a00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 0x7f3c0806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.062+0000 7f09b9c00640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.063+0000 7f3c34a00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 0x7f3c0806c940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c30068c20 tx=0x7f3c3006c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.065+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f54f80c7a60 con 0x7f54f80b8000 2024-02-02T13:24:51.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.071+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6a3c054310 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.075+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2044018020 con 0x7f205c0d1dc0 2024-02-02T13:24:51.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3c280e45b0 con 0x7f3c28057230 2024-02-02T13:24:51.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003dd0 con 0x7f09b40704a0 2024-02-02T13:24:51.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4041470 con 0x7f09b40704a0 2024-02-02T13:24:51.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4041820 con 0x7f09b40704a0 2024-02-02T13:24:51.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b40c6750 con 0x7f09b40704a0 2024-02-02T13:24:51.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.088+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b415c8d0 con 0x7f09b40704a0 2024-02-02T13:24:51.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.090+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b415cac0 con 0x7f09b40704a0 2024-02-02T13:24:51.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.092+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f09a4041980 con 0x7f09b40704a0 2024-02-02T13:24:51.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.094+0000 7f09b1600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 0x7f098c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.094+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09a40aba70 con 0x7f09b40704a0 2024-02-02T13:24:51.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.094+0000 7f09b9200640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 0x7f098c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.096+0000 7f09b9200640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 0x7f098c06c9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09b40c7c20 tx=0x7f099c00f040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.106+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f09a40770d0 con 0x7f09b40704a0 2024-02-02T13:24:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.403+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31000ad2f0 con 0x7f31000b0a00 2024-02-02T13:24:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.404+0000 7f30f7e00640 1 -- 172.21.15.91:0/456927548 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30e808d060 con 0x7f31000b0a00 2024-02-02T13:24:51.406 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.408+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 msgr2=0x7f30d806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.408+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 0x7f30d806c940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30f00098e0 tx=0x7f30f0009060 comp rx=0 tx=0).stop 2024-02-02T13:24:51.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.408+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 msgr2=0x7f31000c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.408+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c780 tx=0x7f30e800cc40 comp rx=0 tx=0).stop 2024-02-02T13:24:51.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.409+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 shutdown_connections 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.409+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d806a480 0x7f30d806c940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.409+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3100158f10 0x7f31000ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.409+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31000b0a00 0x7f31000c8920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.409+0000 7f3105800640 1 --2- 172.21.15.91:0/456927548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f310006be00 0x7f3100156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.410+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 >> 172.21.15.91:0/456927548 conn(0x7f31000cb4c0 msgr2=0x7f31000aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.410+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 shutdown_connections 2024-02-02T13:24:51.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.410+0000 7f3105800640 1 -- 172.21.15.91:0/456927548 wait complete. 2024-02-02T13:24:51.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-02-02T13:24:51.437 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.445+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7d600c94d0 con 0x7f7d600cf660 2024-02-02T13:24:51.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.446+0000 7f7d5d600640 1 -- 172.21.15.91:0/1334725362 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d5801d090 con 0x7f7d600cf660 2024-02-02T13:24:51.448 INFO:teuthology.orchestra.run.smithi091.stdout:55834574855 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.449+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 msgr2=0x7f7d3807d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.449+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 0x7f7d3807d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d50003cc0 tx=0x7f7d500079b0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.449+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 msgr2=0x7f7d600c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.449+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800cc30 tx=0x7f7d5800a5c0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 shutdown_connections 2024-02-02T13:24:51.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7d3807b490 0x7f7d3807d950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d600cf660 0x7f7d600c8350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d600cc740 0x7f7d6014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 --2- 172.21.15.91:0/1334725362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d6006ff50 0x7f7d600b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 >> 172.21.15.91:0/1334725362 conn(0x7f7d600ceb70 msgr2=0x7f7d60157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 shutdown_connections 2024-02-02T13:24:51.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.450+0000 7f7d68000640 1 -- 172.21.15.91:0/1334725362 wait complete. 2024-02-02T13:24:51.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.454+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f54f80c7a60 con 0x7f54f80b8000 2024-02-02T13:24:51.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.454+0000 7f54ef400640 1 -- 172.21.15.91:0/676876956 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54e8017020 con 0x7f54f80b8000 2024-02-02T13:24:51.456 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 msgr2=0x7f54d006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 0x7f54d006c920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54e0035000 tx=0x7f54e0030000 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 msgr2=0x7f54f815c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c430 tx=0x7f54e800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 shutdown_connections 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54d006a460 0x7f54d006c920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54f80cdbb0 0x7f54f815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54f80b8000 0x7f54f815c670 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 --2- 172.21.15.91:0/676876956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54f80b6f80 0x7f54f8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.457+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 >> 172.21.15.91:0/676876956 conn(0x7f54f80d3020 msgr2=0x7f54f81643f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.458+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 shutdown_connections 2024-02-02T13:24:51.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.458+0000 7f54fde00640 1 -- 172.21.15.91:0/676876956 wait complete. 2024-02-02T13:24:51.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.463+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2600057d0 con 0x7fc2880e34d0 2024-02-02T13:24:51.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.464+0000 7fc28cc00640 1 -- 172.21.15.91:0/3594351194 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc2900d1590 con 0x7fc2880e34d0 2024-02-02T13:24:51.466 INFO:teuthology.orchestra.run.smithi091.stdout:55834574855 2024-02-02T13:24:51.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 msgr2=0x7fc26806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 0x7fc26806c950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc278009a40 tx=0x7fc278009500 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 msgr2=0x7fc28817c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc29004f8f0 tx=0x7fc2900741e0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 shutdown_connections 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc26806a490 0x7fc26806c950 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2880e34d0 0x7fc28817c740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2880582f0 0x7fc288178350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 --2- 172.21.15.91:0/3594351194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc288057260 0x7fc2880d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.468+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 >> 172.21.15.91:0/3594351194 conn(0x7fc288059150 msgr2=0x7fc288005d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.469+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 shutdown_connections 2024-02-02T13:24:51.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.469+0000 7fc297800640 1 -- 172.21.15.91:0/3594351194 wait complete. 2024-02-02T13:24:51.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.470+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6a4c04dff0 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.470+0000 7f6a42a00640 1 -- 172.21.15.91:0/1730522953 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a3c057930 con 0x7f6a4c0cfd10 2024-02-02T13:24:51.472 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.474+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 msgr2=0x7f6a2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.474+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 0x7f6a2406c8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a34036000 tx=0x7f6a3403c040 comp rx=0 tx=0).stop 2024-02-02T13:24:51.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.474+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 msgr2=0x7f6a4c1f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.474+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 0x7f6a4c1f2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00d5f0 tx=0x7f6a3c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 shutdown_connections 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2406a430 0x7f6a2406c8f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c15e6c0 0x7f6a4c155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a4c15d1b0 0x7f6a4c155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 --2- 172.21.15.91:0/1730522953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cfd10 0x7f6a4c1f2580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 >> 172.21.15.91:0/1730522953 conn(0x7f6a4c065910 msgr2=0x7f6a4c0b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 shutdown_connections 2024-02-02T13:24:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.475+0000 7f6a50a00640 1 -- 172.21.15.91:0/1730522953 wait complete. 2024-02-02T13:24:51.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-02-02T13:24:51.479 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-02-02T13:24:51.480 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.481+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f205c0d0c80 con 0x7f205c0d1dc0 2024-02-02T13:24:51.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.482+0000 7f2053e00640 1 -- 172.21.15.91:0/600771095 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20440531d0 con 0x7f205c0d1dc0 2024-02-02T13:24:51.483 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 msgr2=0x7f203406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 0x7f203406c8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f204c036000 tx=0x7f204c03d040 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 msgr2=0x7f205c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f204400c4a0 tx=0x7f204400c960 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 shutdown_connections 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203406a430 0x7f203406c8f0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f204c036000 tx=0x7f204c03d040 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0d1dc0 0x7f205c159c80 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f204400c4a0 tx=0x7f204400c960 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c15d260 0x7f205c15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 --2- 172.21.15.91:0/600771095 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f205c0ce4c0 0x7f205c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.484+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 >> 172.21.15.91:0/600771095 conn(0x7f205c0cfa60 msgr2=0x7f205c0cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.485+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 shutdown_connections 2024-02-02T13:24:51.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.485+0000 7f2062e00640 1 -- 172.21.15.91:0/600771095 wait complete. 2024-02-02T13:24:51.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.487+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3c280e45b0 con 0x7f3c28057230 2024-02-02T13:24:51.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.489+0000 7f3c2cc00640 1 -- 172.21.15.91:0/4268685203 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c280e45b0 con 0x7f3c28057230 2024-02-02T13:24:51.490 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-02-02T13:24:51.492 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 msgr2=0x7f3c0806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 0x7f3c0806c940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c30068c20 tx=0x7f3c3006c040 comp rx=0 tx=0).stop 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 msgr2=0x7f3c28052220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3c200097d0 tx=0x7f3c200046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 shutdown_connections 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3c0806a480 0x7f3c0806c940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c28057230 0x7f3c28052220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.491+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c280e34a0 0x7f3c28052760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.492+0000 7f3c35e00640 1 --2- 172.21.15.91:0/4268685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c280543f0 0x7f3c281782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.492+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 >> 172.21.15.91:0/4268685203 conn(0x7f3c28059120 msgr2=0x7f3c2803abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.492+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 shutdown_connections 2024-02-02T13:24:51.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.492+0000 7f3c35e00640 1 -- 172.21.15.91:0/4268685203 wait complete. 2024-02-02T13:24:51.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-02-02T13:24:51.502 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-02-02T13:24:51.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.502+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f09b4066cf0 con 0x7f09b40704a0 2024-02-02T13:24:51.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.503+0000 7f09b1600640 1 -- 172.21.15.91:0/3274731461 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09a4045020 con 0x7f09b40704a0 2024-02-02T13:24:51.504 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.505 INFO:teuthology.orchestra.run.smithi091.stdout:55834574854 2024-02-02T13:24:51.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.505+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 msgr2=0x7f098c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.505+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 0x7f098c06c9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09b40c7c20 tx=0x7f099c00f040 comp rx=0 tx=0).stop 2024-02-02T13:24:51.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.505+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 msgr2=0x7f09b414e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.505+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a40046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 shutdown_connections 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098c06a500 0x7f098c06c9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09b4159910 0x7f09b415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09b40b0f80 0x7f09b40712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 --2- 172.21.15.91:0/3274731461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09b40704a0 0x7f09b414e5b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 >> 172.21.15.91:0/3274731461 conn(0x7f09b4069e30 msgr2=0x7f09b40b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 shutdown_connections 2024-02-02T13:24:51.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.506+0000 7f09bc600640 1 -- 172.21.15.91:0/3274731461 wait complete. 2024-02-02T13:24:51.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-02-02T13:24:51.509 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-02-02T13:24:51.525 DEBUG:teuthology.parallel:result is None 2024-02-02T13:24:51.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-02T13:24:51.526 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:51.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.760+0000 7fea37400640 1 Processor -- start 2024-02-02T13:24:51.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.760+0000 7fea37400640 1 -- start start 2024-02-02T13:24:51.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.761+0000 7fea37400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea3015a010 0x7fea300c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.761+0000 7fea37400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea300ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.761+0000 7fea37400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea300cc7c0 con 0x7fea300cd470 2024-02-02T13:24:51.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.761+0000 7fea37400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea300ce930 con 0x7fea3015a010 2024-02-02T13:24:51.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.761+0000 7fea37400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3006c890 con 0x7fea300ce130 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea27e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea300ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea3015a010 0x7fea300c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea27e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea300ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39196/0 (socket says 172.21.15.91:39196) 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea3015a010 0x7fea300c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:49594/0 (socket says 172.21.15.91:49594) 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea27e00640 1 -- 172.21.15.91:0/3620126307 learned_addr learned my addr 172.21.15.91:0/3620126307 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 -- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 msgr2=0x7fea300ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:51.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 --2- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea300ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 -- 172.21.15.91:0/3620126307 >> v1:172.21.15.113:6789/0 conn(0x7fea300cd470 legacy=0x7fea300b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.762+0000 7fea34a00640 1 -- 172.21.15.91:0/3620126307 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3006cb70 con 0x7fea3015a010 2024-02-02T13:24:51.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.764+0000 7fea34a00640 1 --2- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea3015a010 0x7fea300c9670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea200097a0 tx=0x7fea2002dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db38b4006283bb00 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.764+0000 7fea2fe00640 1 -- 172.21.15.91:0/3620126307 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2002f8a0 con 0x7fea3015a010 2024-02-02T13:24:51.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.764+0000 7fea2fe00640 1 -- 172.21.15.91:0/3620126307 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2002fa00 con 0x7fea3015a010 2024-02-02T13:24:51.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.764+0000 7fea2fe00640 1 -- 172.21.15.91:0/3620126307 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2002fd90 con 0x7fea3015a010 2024-02-02T13:24:51.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 -- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea3015a010 msgr2=0x7fea300c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 --2- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea3015a010 0x7fea300c9670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea200097a0 tx=0x7fea2002dae0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 -- 172.21.15.91:0/3620126307 shutdown_connections 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 --2- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea300ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 --2- 172.21.15.91:0/3620126307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea3015a010 0x7fea300c9670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 -- 172.21.15.91:0/3620126307 >> 172.21.15.91:0/3620126307 conn(0x7fea300cec80 msgr2=0x7fea300cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 -- 172.21.15.91:0/3620126307 shutdown_connections 2024-02-02T13:24:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.765+0000 7fea37400640 1 -- 172.21.15.91:0/3620126307 wait complete. 2024-02-02T13:24:51.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.767+0000 7fea37400640 1 Processor -- start 2024-02-02T13:24:51.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.767+0000 7fea37400640 1 -- start start 2024-02-02T13:24:51.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea3015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea300c7df0 0x7fea300c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea300c7100 con 0x7fea3015a010 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea300ad8c0 con 0x7fea300c7df0 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea37400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea300c7960 con 0x7fea300ce130 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54466/0 (socket says 172.21.15.91:54466) 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 -- 172.21.15.91:0/2787892305 learned_addr learned my addr 172.21.15.91:0/2787892305 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea34a00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea3015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea35400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea300c7df0 0x7fea300c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 -- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 msgr2=0x7fea3015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea3015c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 -- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea300c7df0 msgr2=0x7fea300c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea300c7df0 0x7fea300c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.768+0000 7fea27e00640 1 -- 172.21.15.91:0/2787892305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20009400 con 0x7fea3015a010 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea27e00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e8b0 tx=0x7fea1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800c940 con 0x7fea3015a010 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800caa0 con 0x7fea3015a010 2024-02-02T13:24:51.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800ce30 con 0x7fea3015a010 2024-02-02T13:24:51.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea300c87e0 con 0x7fea3015a010 2024-02-02T13:24:51.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.769+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea300c7310 con 0x7fea3015a010 2024-02-02T13:24:51.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.770+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea30066cf0 con 0x7fea3015a010 2024-02-02T13:24:51.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.771+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea18013470 con 0x7fea3015a010 2024-02-02T13:24:51.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.772+0000 7fea2d600640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 0x7fea0806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:51.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.772+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea18088a90 con 0x7fea3015a010 2024-02-02T13:24:51.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.772+0000 7fea34a00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 0x7fea0806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:51.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.773+0000 7fea34a00640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 0x7fea0806c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea20038000 tx=0x7fea20008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:51.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.776+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fea18054080 con 0x7fea3015a010 2024-02-02T13:24:51.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.946+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea300cca10 con 0x7fea0806a430 2024-02-02T13:24:51.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.954+0000 7fea2d600640 1 -- 172.21.15.91:0/2787892305 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259970 (secure 0 0 0) 0x7fea2002fb70 con 0x7fea0806a430 2024-02-02T13:24:51.959 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:51.960 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.959+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 msgr2=0x7fea0806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.959+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 0x7fea0806c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea20038000 tx=0x7fea20008030 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.959+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 msgr2=0x7fea3015c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.959+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e8b0 tx=0x7fea1800ed70 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 shutdown_connections 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fea0806a430 0x7fea0806c8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea300c7df0 0x7fea300c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea3015a010 0x7fea3015c8c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 --2- 172.21.15.91:0/2787892305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea300ce130 0x7fea3015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 >> 172.21.15.91:0/2787892305 conn(0x7fea300cec80 msgr2=0x7fea3006fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 shutdown_connections 2024-02-02T13:24:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:51.960+0000 7fea37400640 1 -- 172.21.15.91:0/2787892305 wait complete. 2024-02-02T13:24:51.978 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-02-02T13:24:50.239454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560793,"statfs":{"total":1600000000,"available":1598252712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003984"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978392+0000","last_change":"2024-02-02T13:24:36.767566+0000","last_active":"2024-02-02T13:24:40.978392+0000","last_peered":"2024-02-02T13:24:40.978392+0000","last_clean":"2024-02-02T13:24:40.978392+0000","last_became_active":"2024-02-02T13:24:36.767342+0000","last_became_peered":"2024-02-02T13:24:36.767342+0000","last_unstale":"2024-02-02T13:24:40.978392+0000","last_undegraded":"2024-02-02T13:24:40.978392+0000","last_fullsized":"2024-02-02T13:24:40.978392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876609+0000","last_change":"2024-02-02T13:24:36.753974+0000","last_active":"2024-02-02T13:24:39.876609+0000","last_peered":"2024-02-02T13:24:39.876609+0000","last_clean":"2024-02-02T13:24:39.876609+0000","last_became_active":"2024-02-02T13:24:36.753603+0000","last_became_peered":"2024-02-02T13:24:36.753603+0000","last_unstale":"2024-02-02T13:24:39.876609+0000","last_undegraded":"2024-02-02T13:24:39.876609+0000","last_fullsized":"2024-02-02T13:24:39.876609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092881+0000","last_change":"2024-02-02T13:24:36.761233+0000","last_active":"2024-02-02T13:24:41.092881+0000","last_peered":"2024-02-02T13:24:41.092881+0000","last_clean":"2024-02-02T13:24:41.092881+0000","last_became_active":"2024-02-02T13:24:36.761020+0000","last_became_peered":"2024-02-02T13:24:36.761020+0000","last_unstale":"2024-02-02T13:24:41.092881+0000","last_undegraded":"2024-02-02T13:24:41.092881+0000","last_fullsized":"2024-02-02T13:24:41.092881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773368+0000","last_change":"2024-02-02T13:24:36.772920+0000","last_active":"2024-02-02T13:24:36.773368+0000","last_peered":"2024-02-02T13:24:36.773368+0000","last_clean":"2024-02-02T13:24:36.773368+0000","last_became_active":"2024-02-02T13:24:36.765145+0000","last_became_peered":"2024-02-02T13:24:36.765145+0000","last_unstale":"2024-02-02T13:24:36.773368+0000","last_undegraded":"2024-02-02T13:24:36.773368+0000","last_fullsized":"2024-02-02T13:24:36.773368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978323+0000","last_change":"2024-02-02T13:24:36.759645+0000","last_active":"2024-02-02T13:24:40.978323+0000","last_peered":"2024-02-02T13:24:40.978323+0000","last_clean":"2024-02-02T13:24:40.978323+0000","last_became_active":"2024-02-02T13:24:36.759399+0000","last_became_peered":"2024-02-02T13:24:36.759399+0000","last_unstale":"2024-02-02T13:24:40.978323+0000","last_undegraded":"2024-02-02T13:24:40.978323+0000","last_fullsized":"2024-02-02T13:24:40.978323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:52.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777056+0000","last_change":"2024-02-02T13:24:36.774617+0000","last_active":"2024-02-02T13:24:36.777056+0000","last_peered":"2024-02-02T13:24:36.777056+0000","last_clean":"2024-02-02T13:24:36.777056+0000","last_became_active":"2024-02-02T13:24:36.772658+0000","last_became_peered":"2024-02-02T13:24:36.772658+0000","last_unstale":"2024-02-02T13:24:36.777056+0000","last_undegraded":"2024-02-02T13:24:36.777056+0000","last_fullsized":"2024-02-02T13:24:36.777056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978756+0000","last_change":"2024-02-02T13:24:36.755776+0000","last_active":"2024-02-02T13:24:40.978756+0000","last_peered":"2024-02-02T13:24:40.978756+0000","last_clean":"2024-02-02T13:24:40.978756+0000","last_became_active":"2024-02-02T13:24:36.755523+0000","last_became_peered":"2024-02-02T13:24:36.755523+0000","last_unstale":"2024-02-02T13:24:40.978756+0000","last_undegraded":"2024-02-02T13:24:40.978756+0000","last_fullsized":"2024-02-02T13:24:40.978756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211535+0000","last_change":"2024-02-02T13:24:36.759777+0000","last_active":"2024-02-02T13:24:41.211535+0000","last_peered":"2024-02-02T13:24:41.211535+0000","last_clean":"2024-02-02T13:24:41.211535+0000","last_became_active":"2024-02-02T13:24:36.759170+0000","last_became_peered":"2024-02-02T13:24:36.759170+0000","last_unstale":"2024-02-02T13:24:41.211535+0000","last_undegraded":"2024-02-02T13:24:41.211535+0000","last_fullsized":"2024-02-02T13:24:41.211535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978056+0000","last_change":"2024-02-02T13:24:36.757480+0000","last_active":"2024-02-02T13:24:40.978056+0000","last_peered":"2024-02-02T13:24:40.978056+0000","last_clean":"2024-02-02T13:24:40.978056+0000","last_became_active":"2024-02-02T13:24:36.757254+0000","last_became_peered":"2024-02-02T13:24:36.757254+0000","last_unstale":"2024-02-02T13:24:40.978056+0000","last_undegraded":"2024-02-02T13:24:40.978056+0000","last_fullsized":"2024-02-02T13:24:40.978056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:41.311042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775838+0000","last_change":"2024-02-02T13:24:36.775765+0000","last_active":"2024-02-02T13:24:36.775838+0000","last_peered":"2024-02-02T13:24:36.775838+0000","last_clean":"2024-02-02T13:24:36.775838+0000","last_became_active":"2024-02-02T13:24:36.775523+0000","last_became_peered":"2024-02-02T13:24:36.775523+0000","last_unstale":"2024-02-02T13:24:36.775838+0000","last_undegraded":"2024-02-02T13:24:36.775838+0000","last_fullsized":"2024-02-02T13:24:36.775838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778319+0000","last_change":"2024-02-02T13:24:36.778240+0000","last_active":"2024-02-02T13:24:36.778319+0000","last_peered":"2024-02-02T13:24:36.778319+0000","last_clean":"2024-02-02T13:24:36.778319+0000","last_became_active":"2024-02-02T13:24:36.778019+0000","last_became_peered":"2024-02-02T13:24:36.778019+0000","last_unstale":"2024-02-02T13:24:36.778319+0000","last_undegraded":"2024-02-02T13:24:36.778319+0000","last_fullsized":"2024-02-02T13:24:36.778319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753463+0000","last_change":"2024-02-02T13:24:36.766203+0000","last_active":"2024-02-02T13:24:37.753463+0000","last_peered":"2024-02-02T13:24:37.753463+0000","last_clean":"2024-02-02T13:24:37.753463+0000","last_became_active":"2024-02-02T13:24:36.765708+0000","last_became_peered":"2024-02-02T13:24:36.765708+0000","last_unstale":"2024-02-02T13:24:37.753463+0000","last_undegraded":"2024-02-02T13:24:37.753463+0000","last_fullsized":"2024-02-02T13:24:37.753463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211785+0000","last_change":"2024-02-02T13:24:36.768024+0000","last_active":"2024-02-02T13:24:41.211785+0000","last_peered":"2024-02-02T13:24:41.211785+0000","last_clean":"2024-02-02T13:24:41.211785+0000","last_became_active":"2024-02-02T13:24:36.767819+0000","last_became_peered":"2024-02-02T13:24:36.767819+0000","last_unstale":"2024-02-02T13:24:41.211785+0000","last_undegraded":"2024-02-02T13:24:41.211785+0000","last_fullsized":"2024-02-02T13:24:41.211785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093008+0000","last_change":"2024-02-02T13:24:36.755455+0000","last_active":"2024-02-02T13:24:41.093008+0000","last_peered":"2024-02-02T13:24:41.093008+0000","last_clean":"2024-02-02T13:24:41.093008+0000","last_became_active":"2024-02-02T13:24:36.755245+0000","last_became_peered":"2024-02-02T13:24:36.755245+0000","last_unstale":"2024-02-02T13:24:41.093008+0000","last_undegraded":"2024-02-02T13:24:41.093008+0000","last_fullsized":"2024-02-02T13:24:41.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876981+0000","last_change":"2024-02-02T13:24:36.757288+0000","last_active":"2024-02-02T13:24:39.876981+0000","last_peered":"2024-02-02T13:24:39.876981+0000","last_clean":"2024-02-02T13:24:39.876981+0000","last_became_active":"2024-02-02T13:24:36.756687+0000","last_became_peered":"2024-02-02T13:24:36.756687+0000","last_unstale":"2024-02-02T13:24:39.876981+0000","last_undegraded":"2024-02-02T13:24:39.876981+0000","last_fullsized":"2024-02-02T13:24:39.876981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777000+0000","last_change":"2024-02-02T13:24:36.774416+0000","last_active":"2024-02-02T13:24:36.777000+0000","last_peered":"2024-02-02T13:24:36.777000+0000","last_clean":"2024-02-02T13:24:36.777000+0000","last_became_active":"2024-02-02T13:24:36.771102+0000","last_became_peered":"2024-02-02T13:24:36.771102+0000","last_unstale":"2024-02-02T13:24:36.777000+0000","last_undegraded":"2024-02-02T13:24:36.777000+0000","last_fullsized":"2024-02-02T13:24:36.777000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761526+0000","last_change":"2024-02-02T13:24:36.761450+0000","last_active":"2024-02-02T13:24:36.761526+0000","last_peered":"2024-02-02T13:24:36.761526+0000","last_clean":"2024-02-02T13:24:36.761526+0000","last_became_active":"2024-02-02T13:24:36.760875+0000","last_became_peered":"2024-02-02T13:24:36.760875+0000","last_unstale":"2024-02-02T13:24:36.761526+0000","last_undegraded":"2024-02-02T13:24:36.761526+0000","last_fullsized":"2024-02-02T13:24:36.761526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771249+0000","last_change":"2024-02-02T13:24:36.768404+0000","last_active":"2024-02-02T13:24:36.771249+0000","last_peered":"2024-02-02T13:24:36.771249+0000","last_clean":"2024-02-02T13:24:36.771249+0000","last_became_active":"2024-02-02T13:24:36.768048+0000","last_became_peered":"2024-02-02T13:24:36.768048+0000","last_unstale":"2024-02-02T13:24:36.771249+0000","last_undegraded":"2024-02-02T13:24:36.771249+0000","last_fullsized":"2024-02-02T13:24:36.771249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978685+0000","last_change":"2024-02-02T13:24:36.762446+0000","last_active":"2024-02-02T13:24:40.978685+0000","last_peered":"2024-02-02T13:24:40.978685+0000","last_clean":"2024-02-02T13:24:40.978685+0000","last_became_active":"2024-02-02T13:24:36.762021+0000","last_became_peered":"2024-02-02T13:24:36.762021+0000","last_unstale":"2024-02-02T13:24:40.978685+0000","last_undegraded":"2024-02-02T13:24:40.978685+0000","last_fullsized":"2024-02-02T13:24:40.978685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768525+0000","last_change":"2024-02-02T13:24:36.768424+0000","last_active":"2024-02-02T13:24:36.768525+0000","last_peered":"2024-02-02T13:24:36.768525+0000","last_clean":"2024-02-02T13:24:36.768525+0000","last_became_active":"2024-02-02T13:24:36.768203+0000","last_became_peered":"2024-02-02T13:24:36.768203+0000","last_unstale":"2024-02-02T13:24:36.768525+0000","last_undegraded":"2024-02-02T13:24:36.768525+0000","last_fullsized":"2024-02-02T13:24:36.768525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093333+0000","last_change":"2024-02-02T13:24:36.762245+0000","last_active":"2024-02-02T13:24:41.093333+0000","last_peered":"2024-02-02T13:24:41.093333+0000","last_clean":"2024-02-02T13:24:41.093333+0000","last_became_active":"2024-02-02T13:24:36.762030+0000","last_became_peered":"2024-02-02T13:24:36.762030+0000","last_unstale":"2024-02-02T13:24:41.093333+0000","last_undegraded":"2024-02-02T13:24:41.093333+0000","last_fullsized":"2024-02-02T13:24:41.093333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771962+0000","last_change":"2024-02-02T13:24:36.771888+0000","last_active":"2024-02-02T13:24:36.771962+0000","last_peered":"2024-02-02T13:24:36.771962+0000","last_clean":"2024-02-02T13:24:36.771962+0000","last_became_active":"2024-02-02T13:24:36.770589+0000","last_became_peered":"2024-02-02T13:24:36.770589+0000","last_unstale":"2024-02-02T13:24:36.771962+0000","last_undegraded":"2024-02-02T13:24:36.771962+0000","last_fullsized":"2024-02-02T13:24:36.771962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212147+0000","last_change":"2024-02-02T13:24:36.750480+0000","last_active":"2024-02-02T13:24:41.212147+0000","last_peered":"2024-02-02T13:24:41.212147+0000","last_clean":"2024-02-02T13:24:41.212147+0000","last_became_active":"2024-02-02T13:24:36.750125+0000","last_became_peered":"2024-02-02T13:24:36.750125+0000","last_unstale":"2024-02-02T13:24:41.212147+0000","last_undegraded":"2024-02-02T13:24:41.212147+0000","last_fullsized":"2024-02-02T13:24:41.212147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773388+0000","last_change":"2024-02-02T13:24:36.773077+0000","last_active":"2024-02-02T13:24:36.773388+0000","last_peered":"2024-02-02T13:24:36.773388+0000","last_clean":"2024-02-02T13:24:36.773388+0000","last_became_active":"2024-02-02T13:24:36.765475+0000","last_became_peered":"2024-02-02T13:24:36.765475+0000","last_unstale":"2024-02-02T13:24:36.773388+0000","last_undegraded":"2024-02-02T13:24:36.773388+0000","last_fullsized":"2024-02-02T13:24:36.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211624+0000","last_change":"2024-02-02T13:24:36.750736+0000","last_active":"2024-02-02T13:24:41.211624+0000","last_peered":"2024-02-02T13:24:41.211624+0000","last_clean":"2024-02-02T13:24:41.211624+0000","last_became_active":"2024-02-02T13:24:36.750520+0000","last_became_peered":"2024-02-02T13:24:36.750520+0000","last_unstale":"2024-02-02T13:24:41.211624+0000","last_undegraded":"2024-02-02T13:24:41.211624+0000","last_fullsized":"2024-02-02T13:24:41.211624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764120+0000","last_change":"2024-02-02T13:24:36.763039+0000","last_active":"2024-02-02T13:24:36.764120+0000","last_peered":"2024-02-02T13:24:36.764120+0000","last_clean":"2024-02-02T13:24:36.764120+0000","last_became_active":"2024-02-02T13:24:36.762632+0000","last_became_peered":"2024-02-02T13:24:36.762632+0000","last_unstale":"2024-02-02T13:24:36.764120+0000","last_undegraded":"2024-02-02T13:24:36.764120+0000","last_fullsized":"2024-02-02T13:24:36.764120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877008+0000","last_change":"2024-02-02T13:24:36.750634+0000","last_active":"2024-02-02T13:24:39.877008+0000","last_peered":"2024-02-02T13:24:39.877008+0000","last_clean":"2024-02-02T13:24:39.877008+0000","last_became_active":"2024-02-02T13:24:36.750023+0000","last_became_peered":"2024-02-02T13:24:36.750023+0000","last_unstale":"2024-02-02T13:24:39.877008+0000","last_undegraded":"2024-02-02T13:24:39.877008+0000","last_fullsized":"2024-02-02T13:24:39.877008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211911+0000","last_change":"2024-02-02T13:24:36.750339+0000","last_active":"2024-02-02T13:24:41.211911+0000","last_peered":"2024-02-02T13:24:41.211911+0000","last_clean":"2024-02-02T13:24:41.211911+0000","last_became_active":"2024-02-02T13:24:36.749819+0000","last_became_peered":"2024-02-02T13:24:36.749819+0000","last_unstale":"2024-02-02T13:24:41.211911+0000","last_undegraded":"2024-02-02T13:24:41.211911+0000","last_fullsized":"2024-02-02T13:24:41.211911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093260+0000","last_change":"2024-02-02T13:24:36.765687+0000","last_active":"2024-02-02T13:24:41.093260+0000","last_peered":"2024-02-02T13:24:41.093260+0000","last_clean":"2024-02-02T13:24:41.093260+0000","last_became_active":"2024-02-02T13:24:36.765417+0000","last_became_peered":"2024-02-02T13:24:36.765417+0000","last_unstale":"2024-02-02T13:24:41.093260+0000","last_undegraded":"2024-02-02T13:24:41.093260+0000","last_fullsized":"2024-02-02T13:24:41.093260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762953+0000","last_change":"2024-02-02T13:24:36.762822+0000","last_active":"2024-02-02T13:24:36.762953+0000","last_peered":"2024-02-02T13:24:36.762953+0000","last_clean":"2024-02-02T13:24:36.762953+0000","last_became_active":"2024-02-02T13:24:36.761559+0000","last_became_peered":"2024-02-02T13:24:36.761559+0000","last_unstale":"2024-02-02T13:24:36.762953+0000","last_undegraded":"2024-02-02T13:24:36.762953+0000","last_fullsized":"2024-02-02T13:24:36.762953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764632+0000","last_change":"2024-02-02T13:24:36.764552+0000","last_active":"2024-02-02T13:24:36.764632+0000","last_peered":"2024-02-02T13:24:36.764632+0000","last_clean":"2024-02-02T13:24:36.764632+0000","last_became_active":"2024-02-02T13:24:36.764330+0000","last_became_peered":"2024-02-02T13:24:36.764330+0000","last_unstale":"2024-02-02T13:24:36.764632+0000","last_undegraded":"2024-02-02T13:24:36.764632+0000","last_fullsized":"2024-02-02T13:24:36.764632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.765462+0000","last_change":"2024-02-02T13:24:36.765179+0000","last_active":"2024-02-02T13:24:36.765462+0000","last_peered":"2024-02-02T13:24:36.765462+0000","last_clean":"2024-02-02T13:24:36.765462+0000","last_became_active":"2024-02-02T13:24:36.764850+0000","last_became_peered":"2024-02-02T13:24:36.764850+0000","last_unstale":"2024-02-02T13:24:36.765462+0000","last_undegraded":"2024-02-02T13:24:36.765462+0000","last_fullsized":"2024-02-02T13:24:36.765462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753019+0000","last_change":"2024-02-02T13:24:36.755360+0000","last_active":"2024-02-02T13:24:37.753019+0000","last_peered":"2024-02-02T13:24:37.753019+0000","last_clean":"2024-02-02T13:24:37.753019+0000","last_became_active":"2024-02-02T13:24:36.754987+0000","last_became_peered":"2024-02-02T13:24:36.754987+0000","last_unstale":"2024-02-02T13:24:37.753019+0000","last_undegraded":"2024-02-02T13:24:37.753019+0000","last_fullsized":"2024-02-02T13:24:37.753019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772676+0000","last_change":"2024-02-02T13:24:36.288622+0000","last_active":"2024-02-02T13:24:36.772676+0000","last_peered":"2024-02-02T13:24:36.772676+0000","last_clean":"2024-02-02T13:24:36.772676+0000","last_became_active":"2024-02-02T13:24:36.288400+0000","last_became_peered":"2024-02-02T13:24:36.288400+0000","last_unstale":"2024-02-02T13:24:36.772676+0000","last_undegraded":"2024-02-02T13:24:36.772676+0000","last_fullsized":"2024-02-02T13:24:36.772676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092688+0000","last_change":"2024-02-02T13:24:36.765345+0000","last_active":"2024-02-02T13:24:41.092688+0000","last_peered":"2024-02-02T13:24:41.092688+0000","last_clean":"2024-02-02T13:24:41.092688+0000","last_became_active":"2024-02-02T13:24:36.765122+0000","last_became_peered":"2024-02-02T13:24:36.765122+0000","last_unstale":"2024-02-02T13:24:41.092688+0000","last_undegraded":"2024-02-02T13:24:41.092688+0000","last_fullsized":"2024-02-02T13:24:41.092688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774103+0000","last_change":"2024-02-02T13:24:36.284315+0000","last_active":"2024-02-02T13:24:36.774103+0000","last_peered":"2024-02-02T13:24:36.774103+0000","last_clean":"2024-02-02T13:24:36.774103+0000","last_became_active":"2024-02-02T13:24:36.283193+0000","last_became_peered":"2024-02-02T13:24:36.283193+0000","last_unstale":"2024-02-02T13:24:36.774103+0000","last_undegraded":"2024-02-02T13:24:36.774103+0000","last_fullsized":"2024-02-02T13:24:36.774103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.741346+0000","last_change":"2024-02-02T13:24:36.280943+0000","last_active":"2024-02-02T13:24:36.741346+0000","last_peered":"2024-02-02T13:24:36.741346+0000","last_clean":"2024-02-02T13:24:36.741346+0000","last_became_active":"2024-02-02T13:24:36.280565+0000","last_became_peered":"2024-02-02T13:24:36.280565+0000","last_unstale":"2024-02-02T13:24:36.741346+0000","last_undegraded":"2024-02-02T13:24:36.741346+0000","last_fullsized":"2024-02-02T13:24:36.741346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211701+0000","last_change":"2024-02-02T13:24:36.761849+0000","last_active":"2024-02-02T13:24:41.211701+0000","last_peered":"2024-02-02T13:24:41.211701+0000","last_clean":"2024-02-02T13:24:41.211701+0000","last_became_active":"2024-02-02T13:24:36.761460+0000","last_became_peered":"2024-02-02T13:24:36.761460+0000","last_unstale":"2024-02-02T13:24:41.211701+0000","last_undegraded":"2024-02-02T13:24:41.211701+0000","last_fullsized":"2024-02-02T13:24:41.211701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.753498+0000","last_change":"2024-02-02T13:24:36.286417+0000","last_active":"2024-02-02T13:24:36.753498+0000","last_peered":"2024-02-02T13:24:36.753498+0000","last_clean":"2024-02-02T13:24:36.753498+0000","last_became_active":"2024-02-02T13:24:36.286107+0000","last_became_peered":"2024-02-02T13:24:36.286107+0000","last_unstale":"2024-02-02T13:24:36.753498+0000","last_undegraded":"2024-02-02T13:24:36.753498+0000","last_fullsized":"2024-02-02T13:24:36.753498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876738+0000","last_change":"2024-02-02T13:24:36.750157+0000","last_active":"2024-02-02T13:24:39.876738+0000","last_peered":"2024-02-02T13:24:39.876738+0000","last_clean":"2024-02-02T13:24:39.876738+0000","last_became_active":"2024-02-02T13:24:36.749764+0000","last_became_peered":"2024-02-02T13:24:36.749764+0000","last_unstale":"2024-02-02T13:24:39.876738+0000","last_undegraded":"2024-02-02T13:24:39.876738+0000","last_fullsized":"2024-02-02T13:24:39.876738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762177+0000","last_change":"2024-02-02T13:24:36.290816+0000","last_active":"2024-02-02T13:24:36.762177+0000","last_peered":"2024-02-02T13:24:36.762177+0000","last_clean":"2024-02-02T13:24:36.762177+0000","last_became_active":"2024-02-02T13:24:36.290592+0000","last_became_peered":"2024-02-02T13:24:36.290592+0000","last_unstale":"2024-02-02T13:24:36.762177+0000","last_undegraded":"2024-02-02T13:24:36.762177+0000","last_fullsized":"2024-02-02T13:24:36.762177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761656+0000","last_change":"2024-02-02T13:24:36.761589+0000","last_active":"2024-02-02T13:24:36.761656+0000","last_peered":"2024-02-02T13:24:36.761656+0000","last_clean":"2024-02-02T13:24:36.761656+0000","last_became_active":"2024-02-02T13:24:36.761215+0000","last_became_peered":"2024-02-02T13:24:36.761215+0000","last_unstale":"2024-02-02T13:24:36.761656+0000","last_undegraded":"2024-02-02T13:24:36.761656+0000","last_fullsized":"2024-02-02T13:24:36.761656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.758412+0000","last_change":"2024-02-02T13:24:36.281944+0000","last_active":"2024-02-02T13:24:36.758412+0000","last_peered":"2024-02-02T13:24:36.758412+0000","last_clean":"2024-02-02T13:24:36.758412+0000","last_became_active":"2024-02-02T13:24:36.281546+0000","last_became_peered":"2024-02-02T13:24:36.281546+0000","last_unstale":"2024-02-02T13:24:36.758412+0000","last_undegraded":"2024-02-02T13:24:36.758412+0000","last_fullsized":"2024-02-02T13:24:36.758412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093550+0000","last_change":"2024-02-02T13:24:36.775310+0000","last_active":"2024-02-02T13:24:41.093550+0000","last_peered":"2024-02-02T13:24:41.093550+0000","last_clean":"2024-02-02T13:24:41.093550+0000","last_became_active":"2024-02-02T13:24:36.775021+0000","last_became_peered":"2024-02-02T13:24:36.775021+0000","last_unstale":"2024-02-02T13:24:41.093550+0000","last_undegraded":"2024-02-02T13:24:41.093550+0000","last_fullsized":"2024-02-02T13:24:41.093550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777366+0000","last_change":"2024-02-02T13:24:36.267935+0000","last_active":"2024-02-02T13:24:36.777366+0000","last_peered":"2024-02-02T13:24:36.777366+0000","last_clean":"2024-02-02T13:24:36.777366+0000","last_became_active":"2024-02-02T13:24:36.267120+0000","last_became_peered":"2024-02-02T13:24:36.267120+0000","last_unstale":"2024-02-02T13:24:36.777366+0000","last_undegraded":"2024-02-02T13:24:36.777366+0000","last_fullsized":"2024-02-02T13:24:36.777366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753018+0000","last_change":"2024-02-02T13:24:36.758353+0000","last_active":"2024-02-02T13:24:37.753018+0000","last_peered":"2024-02-02T13:24:37.753018+0000","last_clean":"2024-02-02T13:24:37.753018+0000","last_became_active":"2024-02-02T13:24:36.758074+0000","last_became_peered":"2024-02-02T13:24:36.758074+0000","last_unstale":"2024-02-02T13:24:37.753018+0000","last_undegraded":"2024-02-02T13:24:37.753018+0000","last_fullsized":"2024-02-02T13:24:37.753018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.776688+0000","last_change":"2024-02-02T13:24:36.283779+0000","last_active":"2024-02-02T13:24:36.776688+0000","last_peered":"2024-02-02T13:24:36.776688+0000","last_clean":"2024-02-02T13:24:36.776688+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.776688+0000","last_undegraded":"2024-02-02T13:24:36.776688+0000","last_fullsized":"2024-02-02T13:24:36.776688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978240+0000","last_change":"2024-02-02T13:24:36.751477+0000","last_active":"2024-02-02T13:24:40.978240+0000","last_peered":"2024-02-02T13:24:40.978240+0000","last_clean":"2024-02-02T13:24:40.978240+0000","last_became_active":"2024-02-02T13:24:36.751226+0000","last_became_peered":"2024-02-02T13:24:36.751226+0000","last_unstale":"2024-02-02T13:24:40.978240+0000","last_undegraded":"2024-02-02T13:24:40.978240+0000","last_fullsized":"2024-02-02T13:24:40.978240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773129+0000","last_change":"2024-02-02T13:24:36.286732+0000","last_active":"2024-02-02T13:24:36.773129+0000","last_peered":"2024-02-02T13:24:36.773129+0000","last_clean":"2024-02-02T13:24:36.773129+0000","last_became_active":"2024-02-02T13:24:36.286351+0000","last_became_peered":"2024-02-02T13:24:36.286351+0000","last_unstale":"2024-02-02T13:24:36.773129+0000","last_undegraded":"2024-02-02T13:24:36.773129+0000","last_fullsized":"2024-02-02T13:24:36.773129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093755+0000","last_change":"2024-02-02T13:24:36.756739+0000","last_active":"2024-02-02T13:24:41.093755+0000","last_peered":"2024-02-02T13:24:41.093755+0000","last_clean":"2024-02-02T13:24:41.093755+0000","last_became_active":"2024-02-02T13:24:36.756537+0000","last_became_peered":"2024-02-02T13:24:36.756537+0000","last_unstale":"2024-02-02T13:24:41.093755+0000","last_undegraded":"2024-02-02T13:24:41.093755+0000","last_fullsized":"2024-02-02T13:24:41.093755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755625+0000","last_change":"2024-02-02T13:24:36.286698+0000","last_active":"2024-02-02T13:24:36.755625+0000","last_peered":"2024-02-02T13:24:36.755625+0000","last_clean":"2024-02-02T13:24:36.755625+0000","last_became_active":"2024-02-02T13:24:36.282637+0000","last_became_peered":"2024-02-02T13:24:36.282637+0000","last_unstale":"2024-02-02T13:24:36.755625+0000","last_undegraded":"2024-02-02T13:24:36.755625+0000","last_fullsized":"2024-02-02T13:24:36.755625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978581+0000","last_change":"2024-02-02T13:24:36.766477+0000","last_active":"2024-02-02T13:24:40.978581+0000","last_peered":"2024-02-02T13:24:40.978581+0000","last_clean":"2024-02-02T13:24:40.978581+0000","last_became_active":"2024-02-02T13:24:36.765889+0000","last_became_peered":"2024-02-02T13:24:36.765889+0000","last_unstale":"2024-02-02T13:24:40.978581+0000","last_undegraded":"2024-02-02T13:24:40.978581+0000","last_fullsized":"2024-02-02T13:24:40.978581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764428+0000","last_change":"2024-02-02T13:24:36.286937+0000","last_active":"2024-02-02T13:24:36.764428+0000","last_peered":"2024-02-02T13:24:36.764428+0000","last_clean":"2024-02-02T13:24:36.764428+0000","last_became_active":"2024-02-02T13:24:36.286651+0000","last_became_peered":"2024-02-02T13:24:36.286651+0000","last_unstale":"2024-02-02T13:24:36.764428+0000","last_undegraded":"2024-02-02T13:24:36.764428+0000","last_fullsized":"2024-02-02T13:24:36.764428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.769009+0000","last_change":"2024-02-02T13:24:36.768901+0000","last_active":"2024-02-02T13:24:36.769009+0000","last_peered":"2024-02-02T13:24:36.769009+0000","last_clean":"2024-02-02T13:24:36.769009+0000","last_became_active":"2024-02-02T13:24:36.768565+0000","last_became_peered":"2024-02-02T13:24:36.768565+0000","last_unstale":"2024-02-02T13:24:36.769009+0000","last_undegraded":"2024-02-02T13:24:36.769009+0000","last_fullsized":"2024-02-02T13:24:36.769009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775361+0000","last_change":"2024-02-02T13:24:36.271772+0000","last_active":"2024-02-02T13:24:36.775361+0000","last_peered":"2024-02-02T13:24:36.775361+0000","last_clean":"2024-02-02T13:24:36.775361+0000","last_became_active":"2024-02-02T13:24:36.271163+0000","last_became_peered":"2024-02-02T13:24:36.271163+0000","last_unstale":"2024-02-02T13:24:36.775361+0000","last_undegraded":"2024-02-02T13:24:36.775361+0000","last_fullsized":"2024-02-02T13:24:36.775361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.979190+0000","last_change":"2024-02-02T13:24:36.756837+0000","last_active":"2024-02-02T13:24:40.979190+0000","last_peered":"2024-02-02T13:24:40.979190+0000","last_clean":"2024-02-02T13:24:40.979190+0000","last_became_active":"2024-02-02T13:24:36.756508+0000","last_became_peered":"2024-02-02T13:24:36.756508+0000","last_unstale":"2024-02-02T13:24:40.979190+0000","last_undegraded":"2024-02-02T13:24:40.979190+0000","last_fullsized":"2024-02-02T13:24:40.979190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771720+0000","last_change":"2024-02-02T13:24:36.287213+0000","last_active":"2024-02-02T13:24:36.771720+0000","last_peered":"2024-02-02T13:24:36.771720+0000","last_clean":"2024-02-02T13:24:36.771720+0000","last_became_active":"2024-02-02T13:24:36.286967+0000","last_became_peered":"2024-02-02T13:24:36.286967+0000","last_unstale":"2024-02-02T13:24:36.771720+0000","last_undegraded":"2024-02-02T13:24:36.771720+0000","last_fullsized":"2024-02-02T13:24:36.771720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978811+0000","last_change":"2024-02-02T13:24:36.766631+0000","last_active":"2024-02-02T13:24:40.978811+0000","last_peered":"2024-02-02T13:24:40.978811+0000","last_clean":"2024-02-02T13:24:40.978811+0000","last_became_active":"2024-02-02T13:24:36.766216+0000","last_became_peered":"2024-02-02T13:24:36.766216+0000","last_unstale":"2024-02-02T13:24:40.978811+0000","last_undegraded":"2024-02-02T13:24:40.978811+0000","last_fullsized":"2024-02-02T13:24:40.978811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772131+0000","last_change":"2024-02-02T13:24:36.275397+0000","last_active":"2024-02-02T13:24:36.772131+0000","last_peered":"2024-02-02T13:24:36.772131+0000","last_clean":"2024-02-02T13:24:36.772131+0000","last_became_active":"2024-02-02T13:24:36.274927+0000","last_became_peered":"2024-02-02T13:24:36.274927+0000","last_unstale":"2024-02-02T13:24:36.772131+0000","last_undegraded":"2024-02-02T13:24:36.772131+0000","last_fullsized":"2024-02-02T13:24:36.772131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978419+0000","last_change":"2024-02-02T13:24:36.757035+0000","last_active":"2024-02-02T13:24:40.978419+0000","last_peered":"2024-02-02T13:24:40.978419+0000","last_clean":"2024-02-02T13:24:40.978419+0000","last_became_active":"2024-02-02T13:24:36.756748+0000","last_became_peered":"2024-02-02T13:24:36.756748+0000","last_unstale":"2024-02-02T13:24:40.978419+0000","last_undegraded":"2024-02-02T13:24:40.978419+0000","last_fullsized":"2024-02-02T13:24:40.978419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:24:57.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768414+0000","last_change":"2024-02-02T13:24:36.283044+0000","last_active":"2024-02-02T13:24:36.768414+0000","last_peered":"2024-02-02T13:24:36.768414+0000","last_clean":"2024-02-02T13:24:36.768414+0000","last_became_active":"2024-02-02T13:24:36.282798+0000","last_became_peered":"2024-02-02T13:24:36.282798+0000","last_unstale":"2024-02-02T13:24:36.768414+0000","last_undegraded":"2024-02-02T13:24:36.768414+0000","last_fullsized":"2024-02-02T13:24:36.768414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770333+0000","last_change":"2024-02-02T13:24:36.764197+0000","last_active":"2024-02-02T13:24:36.770333+0000","last_peered":"2024-02-02T13:24:36.770333+0000","last_clean":"2024-02-02T13:24:36.770333+0000","last_became_active":"2024-02-02T13:24:36.763970+0000","last_became_peered":"2024-02-02T13:24:36.763970+0000","last_unstale":"2024-02-02T13:24:36.770333+0000","last_undegraded":"2024-02-02T13:24:36.770333+0000","last_fullsized":"2024-02-02T13:24:36.770333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.757774+0000","last_change":"2024-02-02T13:24:36.279681+0000","last_active":"2024-02-02T13:24:36.757774+0000","last_peered":"2024-02-02T13:24:36.757774+0000","last_clean":"2024-02-02T13:24:36.757774+0000","last_became_active":"2024-02-02T13:24:36.279342+0000","last_became_peered":"2024-02-02T13:24:36.279342+0000","last_unstale":"2024-02-02T13:24:36.757774+0000","last_undegraded":"2024-02-02T13:24:36.757774+0000","last_fullsized":"2024-02-02T13:24:36.757774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978828+0000","last_change":"2024-02-02T13:24:36.758113+0000","last_active":"2024-02-02T13:24:40.978828+0000","last_peered":"2024-02-02T13:24:40.978828+0000","last_clean":"2024-02-02T13:24:40.978828+0000","last_became_active":"2024-02-02T13:24:36.757859+0000","last_became_peered":"2024-02-02T13:24:36.757859+0000","last_unstale":"2024-02-02T13:24:40.978828+0000","last_undegraded":"2024-02-02T13:24:40.978828+0000","last_fullsized":"2024-02-02T13:24:40.978828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748840+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:36.748840+0000","last_peered":"2024-02-02T13:24:36.748840+0000","last_clean":"2024-02-02T13:24:36.748840+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:36.748840+0000","last_undegraded":"2024-02-02T13:24:36.748840+0000","last_fullsized":"2024-02-02T13:24:36.748840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755725+0000","last_change":"2024-02-02T13:24:36.285102+0000","last_active":"2024-02-02T13:24:36.755725+0000","last_peered":"2024-02-02T13:24:36.755725+0000","last_clean":"2024-02-02T13:24:36.755725+0000","last_became_active":"2024-02-02T13:24:36.280044+0000","last_became_peered":"2024-02-02T13:24:36.280044+0000","last_unstale":"2024-02-02T13:24:36.755725+0000","last_undegraded":"2024-02-02T13:24:36.755725+0000","last_fullsized":"2024-02-02T13:24:36.755725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211798+0000","last_change":"2024-02-02T13:24:36.762214+0000","last_active":"2024-02-02T13:24:41.211798+0000","last_peered":"2024-02-02T13:24:41.211798+0000","last_clean":"2024-02-02T13:24:41.211798+0000","last_became_active":"2024-02-02T13:24:36.761690+0000","last_became_peered":"2024-02-02T13:24:36.761690+0000","last_unstale":"2024-02-02T13:24:41.211798+0000","last_undegraded":"2024-02-02T13:24:41.211798+0000","last_fullsized":"2024-02-02T13:24:41.211798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748517+0000","last_change":"2024-02-02T13:24:36.291720+0000","last_active":"2024-02-02T13:24:36.748517+0000","last_peered":"2024-02-02T13:24:36.748517+0000","last_clean":"2024-02-02T13:24:36.748517+0000","last_became_active":"2024-02-02T13:24:36.290703+0000","last_became_peered":"2024-02-02T13:24:36.290703+0000","last_unstale":"2024-02-02T13:24:36.748517+0000","last_undegraded":"2024-02-02T13:24:36.748517+0000","last_fullsized":"2024-02-02T13:24:36.748517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093549+0000","last_change":"2024-02-02T13:24:36.766220+0000","last_active":"2024-02-02T13:24:41.093549+0000","last_peered":"2024-02-02T13:24:41.093549+0000","last_clean":"2024-02-02T13:24:41.093549+0000","last_became_active":"2024-02-02T13:24:36.765734+0000","last_became_peered":"2024-02-02T13:24:36.765734+0000","last_unstale":"2024-02-02T13:24:41.093549+0000","last_undegraded":"2024-02-02T13:24:41.093549+0000","last_fullsized":"2024-02-02T13:24:41.093549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.756734+0000","last_change":"2024-02-02T13:24:36.276562+0000","last_active":"2024-02-02T13:24:36.756734+0000","last_peered":"2024-02-02T13:24:36.756734+0000","last_clean":"2024-02-02T13:24:36.756734+0000","last_became_active":"2024-02-02T13:24:36.276189+0000","last_became_peered":"2024-02-02T13:24:36.276189+0000","last_unstale":"2024-02-02T13:24:36.756734+0000","last_undegraded":"2024-02-02T13:24:36.756734+0000","last_fullsized":"2024-02-02T13:24:36.756734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877004+0000","last_change":"2024-02-02T13:24:36.761943+0000","last_active":"2024-02-02T13:24:39.877004+0000","last_peered":"2024-02-02T13:24:39.877004+0000","last_clean":"2024-02-02T13:24:39.877004+0000","last_became_active":"2024-02-02T13:24:36.761718+0000","last_became_peered":"2024-02-02T13:24:36.761718+0000","last_unstale":"2024-02-02T13:24:39.877004+0000","last_undegraded":"2024-02-02T13:24:39.877004+0000","last_fullsized":"2024-02-02T13:24:39.877004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.760484+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.760484+0000","last_peered":"2024-02-02T13:24:36.760484+0000","last_clean":"2024-02-02T13:24:36.760484+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.760484+0000","last_undegraded":"2024-02-02T13:24:36.760484+0000","last_fullsized":"2024-02-02T13:24:36.760484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753283+0000","last_change":"2024-02-02T13:24:36.763372+0000","last_active":"2024-02-02T13:24:37.753283+0000","last_peered":"2024-02-02T13:24:37.753283+0000","last_clean":"2024-02-02T13:24:37.753283+0000","last_became_active":"2024-02-02T13:24:36.763132+0000","last_became_peered":"2024-02-02T13:24:36.763132+0000","last_unstale":"2024-02-02T13:24:37.753283+0000","last_undegraded":"2024-02-02T13:24:37.753283+0000","last_fullsized":"2024-02-02T13:24:37.753283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:03.055262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747812+0000","last_change":"2024-02-02T13:24:36.287910+0000","last_active":"2024-02-02T13:24:36.747812+0000","last_peered":"2024-02-02T13:24:36.747812+0000","last_clean":"2024-02-02T13:24:36.747812+0000","last_became_active":"2024-02-02T13:24:36.287683+0000","last_became_peered":"2024-02-02T13:24:36.287683+0000","last_unstale":"2024-02-02T13:24:36.747812+0000","last_undegraded":"2024-02-02T13:24:36.747812+0000","last_fullsized":"2024-02-02T13:24:36.747812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773408+0000","last_change":"2024-02-02T13:24:36.773275+0000","last_active":"2024-02-02T13:24:36.773408+0000","last_peered":"2024-02-02T13:24:36.773408+0000","last_clean":"2024-02-02T13:24:36.773408+0000","last_became_active":"2024-02-02T13:24:36.772646+0000","last_became_peered":"2024-02-02T13:24:36.772646+0000","last_unstale":"2024-02-02T13:24:36.773408+0000","last_undegraded":"2024-02-02T13:24:36.773408+0000","last_fullsized":"2024-02-02T13:24:36.773408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748406+0000","last_change":"2024-02-02T13:24:36.291402+0000","last_active":"2024-02-02T13:24:36.748406+0000","last_peered":"2024-02-02T13:24:36.748406+0000","last_clean":"2024-02-02T13:24:36.748406+0000","last_became_active":"2024-02-02T13:24:36.290274+0000","last_became_peered":"2024-02-02T13:24:36.290274+0000","last_unstale":"2024-02-02T13:24:36.748406+0000","last_undegraded":"2024-02-02T13:24:36.748406+0000","last_fullsized":"2024-02-02T13:24:36.748406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876617+0000","last_change":"2024-02-02T13:24:36.760028+0000","last_active":"2024-02-02T13:24:39.876617+0000","last_peered":"2024-02-02T13:24:39.876617+0000","last_clean":"2024-02-02T13:24:39.876617+0000","last_became_active":"2024-02-02T13:24:36.759641+0000","last_became_peered":"2024-02-02T13:24:36.759641+0000","last_unstale":"2024-02-02T13:24:39.876617+0000","last_undegraded":"2024-02-02T13:24:39.876617+0000","last_fullsized":"2024-02-02T13:24:39.876617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877230+0000","last_change":"2024-02-02T13:24:36.281619+0000","last_active":"2024-02-02T13:24:39.877230+0000","last_peered":"2024-02-02T13:24:39.877230+0000","last_clean":"2024-02-02T13:24:39.877230+0000","last_became_active":"2024-02-02T13:24:36.281168+0000","last_became_peered":"2024-02-02T13:24:36.281168+0000","last_unstale":"2024-02-02T13:24:39.877230+0000","last_undegraded":"2024-02-02T13:24:39.877230+0000","last_fullsized":"2024-02-02T13:24:39.877230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211709+0000","last_change":"2024-02-02T13:24:36.750882+0000","last_active":"2024-02-02T13:24:41.211709+0000","last_peered":"2024-02-02T13:24:41.211709+0000","last_clean":"2024-02-02T13:24:41.211709+0000","last_became_active":"2024-02-02T13:24:36.750448+0000","last_became_peered":"2024-02-02T13:24:36.750448+0000","last_unstale":"2024-02-02T13:24:41.211709+0000","last_undegraded":"2024-02-02T13:24:41.211709+0000","last_fullsized":"2024-02-02T13:24:41.211709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774907+0000","last_change":"2024-02-02T13:24:36.289169+0000","last_active":"2024-02-02T13:24:36.774907+0000","last_peered":"2024-02-02T13:24:36.774907+0000","last_clean":"2024-02-02T13:24:36.774907+0000","last_became_active":"2024-02-02T13:24:36.288939+0000","last_became_peered":"2024-02-02T13:24:36.288939+0000","last_unstale":"2024-02-02T13:24:36.774907+0000","last_undegraded":"2024-02-02T13:24:36.774907+0000","last_fullsized":"2024-02-02T13:24:36.774907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093133+0000","last_change":"2024-02-02T13:24:36.756653+0000","last_active":"2024-02-02T13:24:41.093133+0000","last_peered":"2024-02-02T13:24:41.093133+0000","last_clean":"2024-02-02T13:24:41.093133+0000","last_became_active":"2024-02-02T13:24:36.756419+0000","last_became_peered":"2024-02-02T13:24:36.756419+0000","last_unstale":"2024-02-02T13:24:41.093133+0000","last_undegraded":"2024-02-02T13:24:41.093133+0000","last_fullsized":"2024-02-02T13:24:41.093133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.751195+0000","last_change":"2024-02-02T13:24:36.290423+0000","last_active":"2024-02-02T13:24:36.751195+0000","last_peered":"2024-02-02T13:24:36.751195+0000","last_clean":"2024-02-02T13:24:36.751195+0000","last_became_active":"2024-02-02T13:24:36.289863+0000","last_became_peered":"2024-02-02T13:24:36.289863+0000","last_unstale":"2024-02-02T13:24:36.751195+0000","last_undegraded":"2024-02-02T13:24:36.751195+0000","last_fullsized":"2024-02-02T13:24:36.751195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.754234+0000","last_change":"2024-02-02T13:24:36.759429+0000","last_active":"2024-02-02T13:24:37.754234+0000","last_peered":"2024-02-02T13:24:37.754234+0000","last_clean":"2024-02-02T13:24:37.754234+0000","last_became_active":"2024-02-02T13:24:36.759196+0000","last_became_peered":"2024-02-02T13:24:36.759196+0000","last_unstale":"2024-02-02T13:24:37.754234+0000","last_undegraded":"2024-02-02T13:24:37.754234+0000","last_fullsized":"2024-02-02T13:24:37.754234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747537+0000","last_change":"2024-02-02T13:24:36.288653+0000","last_active":"2024-02-02T13:24:36.747537+0000","last_peered":"2024-02-02T13:24:36.747537+0000","last_clean":"2024-02-02T13:24:36.747537+0000","last_became_active":"2024-02-02T13:24:36.288439+0000","last_became_peered":"2024-02-02T13:24:36.288439+0000","last_unstale":"2024-02-02T13:24:36.747537+0000","last_undegraded":"2024-02-02T13:24:36.747537+0000","last_fullsized":"2024-02-02T13:24:36.747537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211609+0000","last_change":"2024-02-02T13:24:36.751484+0000","last_active":"2024-02-02T13:24:41.211609+0000","last_peered":"2024-02-02T13:24:41.211609+0000","last_clean":"2024-02-02T13:24:41.211609+0000","last_became_active":"2024-02-02T13:24:36.751091+0000","last_became_peered":"2024-02-02T13:24:36.751091+0000","last_unstale":"2024-02-02T13:24:41.211609+0000","last_undegraded":"2024-02-02T13:24:41.211609+0000","last_fullsized":"2024-02-02T13:24:41.211609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747419+0000","last_change":"2024-02-02T13:24:36.280128+0000","last_active":"2024-02-02T13:24:36.747419+0000","last_peered":"2024-02-02T13:24:36.747419+0000","last_clean":"2024-02-02T13:24:36.747419+0000","last_became_active":"2024-02-02T13:24:36.279652+0000","last_became_peered":"2024-02-02T13:24:36.279652+0000","last_unstale":"2024-02-02T13:24:36.747419+0000","last_undegraded":"2024-02-02T13:24:36.747419+0000","last_fullsized":"2024-02-02T13:24:36.747419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212026+0000","last_change":"2024-02-02T13:24:36.763668+0000","last_active":"2024-02-02T13:24:41.212026+0000","last_peered":"2024-02-02T13:24:41.212026+0000","last_clean":"2024-02-02T13:24:41.212026+0000","last_became_active":"2024-02-02T13:24:36.762913+0000","last_became_peered":"2024-02-02T13:24:36.762913+0000","last_unstale":"2024-02-02T13:24:41.212026+0000","last_undegraded":"2024-02-02T13:24:41.212026+0000","last_fullsized":"2024-02-02T13:24:41.212026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774478+0000","last_change":"2024-02-02T13:24:36.289669+0000","last_active":"2024-02-02T13:24:36.774478+0000","last_peered":"2024-02-02T13:24:36.774478+0000","last_clean":"2024-02-02T13:24:36.774478+0000","last_became_active":"2024-02-02T13:24:36.289368+0000","last_became_peered":"2024-02-02T13:24:36.289368+0000","last_unstale":"2024-02-02T13:24:36.774478+0000","last_undegraded":"2024-02-02T13:24:36.774478+0000","last_fullsized":"2024-02-02T13:24:36.774478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770307+0000","last_change":"2024-02-02T13:24:36.769265+0000","last_active":"2024-02-02T13:24:36.770307+0000","last_peered":"2024-02-02T13:24:36.770307+0000","last_clean":"2024-02-02T13:24:36.770307+0000","last_became_active":"2024-02-02T13:24:36.769036+0000","last_became_peered":"2024-02-02T13:24:36.769036+0000","last_unstale":"2024-02-02T13:24:36.770307+0000","last_undegraded":"2024-02-02T13:24:36.770307+0000","last_fullsized":"2024-02-02T13:24:36.770307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770353+0000","last_change":"2024-02-02T13:24:36.765383+0000","last_active":"2024-02-02T13:24:36.770353+0000","last_peered":"2024-02-02T13:24:36.770353+0000","last_clean":"2024-02-02T13:24:36.770353+0000","last_became_active":"2024-02-02T13:24:36.765141+0000","last_became_peered":"2024-02-02T13:24:36.765141+0000","last_unstale":"2024-02-02T13:24:36.770353+0000","last_undegraded":"2024-02-02T13:24:36.770353+0000","last_fullsized":"2024-02-02T13:24:36.770353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778010+0000","last_change":"2024-02-02T13:24:36.290626+0000","last_active":"2024-02-02T13:24:36.778010+0000","last_peered":"2024-02-02T13:24:36.778010+0000","last_clean":"2024-02-02T13:24:36.778010+0000","last_became_active":"2024-02-02T13:24:36.290183+0000","last_became_peered":"2024-02-02T13:24:36.290183+0000","last_unstale":"2024-02-02T13:24:36.778010+0000","last_undegraded":"2024-02-02T13:24:36.778010+0000","last_fullsized":"2024-02-02T13:24:36.778010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777961+0000","last_change":"2024-02-02T13:24:36.777865+0000","last_active":"2024-02-02T13:24:36.777961+0000","last_peered":"2024-02-02T13:24:36.777961+0000","last_clean":"2024-02-02T13:24:36.777961+0000","last_became_active":"2024-02-02T13:24:36.777129+0000","last_became_peered":"2024-02-02T13:24:36.777129+0000","last_unstale":"2024-02-02T13:24:36.777961+0000","last_undegraded":"2024-02-02T13:24:36.777961+0000","last_fullsized":"2024-02-02T13:24:36.777961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772162+0000","last_change":"2024-02-02T13:24:36.290121+0000","last_active":"2024-02-02T13:24:36.772162+0000","last_peered":"2024-02-02T13:24:36.772162+0000","last_clean":"2024-02-02T13:24:36.772162+0000","last_became_active":"2024-02-02T13:24:36.289708+0000","last_became_peered":"2024-02-02T13:24:36.289708+0000","last_unstale":"2024-02-02T13:24:36.772162+0000","last_undegraded":"2024-02-02T13:24:36.772162+0000","last_fullsized":"2024-02-02T13:24:36.772162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093334+0000","last_change":"2024-02-02T13:24:36.759606+0000","last_active":"2024-02-02T13:24:41.093334+0000","last_peered":"2024-02-02T13:24:41.093334+0000","last_clean":"2024-02-02T13:24:41.093334+0000","last_became_active":"2024-02-02T13:24:36.759358+0000","last_became_peered":"2024-02-02T13:24:36.759358+0000","last_unstale":"2024-02-02T13:24:41.093334+0000","last_undegraded":"2024-02-02T13:24:41.093334+0000","last_fullsized":"2024-02-02T13:24:41.093334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771605+0000","last_change":"2024-02-02T13:24:36.286535+0000","last_active":"2024-02-02T13:24:36.771605+0000","last_peered":"2024-02-02T13:24:36.771605+0000","last_clean":"2024-02-02T13:24:36.771605+0000","last_became_active":"2024-02-02T13:24:36.286302+0000","last_became_peered":"2024-02-02T13:24:36.286302+0000","last_unstale":"2024-02-02T13:24:36.771605+0000","last_undegraded":"2024-02-02T13:24:36.771605+0000","last_fullsized":"2024-02-02T13:24:36.771605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212029+0000","last_change":"2024-02-02T13:24:36.769601+0000","last_active":"2024-02-02T13:24:41.212029+0000","last_peered":"2024-02-02T13:24:41.212029+0000","last_clean":"2024-02-02T13:24:41.212029+0000","last_became_active":"2024-02-02T13:24:36.768381+0000","last_became_peered":"2024-02-02T13:24:36.768381+0000","last_unstale":"2024-02-02T13:24:41.212029+0000","last_undegraded":"2024-02-02T13:24:41.212029+0000","last_fullsized":"2024-02-02T13:24:41.212029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774155+0000","last_change":"2024-02-02T13:24:36.289368+0000","last_active":"2024-02-02T13:24:36.774155+0000","last_peered":"2024-02-02T13:24:36.774155+0000","last_clean":"2024-02-02T13:24:36.774155+0000","last_became_active":"2024-02-02T13:24:36.289118+0000","last_became_peered":"2024-02-02T13:24:36.289118+0000","last_unstale":"2024-02-02T13:24:36.774155+0000","last_undegraded":"2024-02-02T13:24:36.774155+0000","last_fullsized":"2024-02-02T13:24:36.774155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.893}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928717,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8089999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8490000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.8410000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2050000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.651}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9210000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.74}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:51.982 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:24:52.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.222+0000 7f8939400640 1 Processor -- start 2024-02-02T13:24:52.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.223+0000 7f8939400640 1 -- start start 2024-02-02T13:24:52.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8939400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f893415f5a0 0x7f89340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8939400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89340b0c40 0x7f89340b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8939400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340b1070 con 0x7f8934151f90 2024-02-02T13:24:52.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8939400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893415fa30 con 0x7f89340b0c40 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8939400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8934070f80 con 0x7f893415f5a0 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.224+0000 7f8933200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89340b0c40 0x7f89340b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89340b0c40 0x7f89340b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54476/0 (socket says 172.21.15.91:54476) 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 -- 172.21.15.91:0/855829053 learned_addr learned my addr 172.21.15.91:0/855829053 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8929c00640 1 --2- 172.21.15.91:0/855829053 >> v2:172.21.15.113:3300/0 conn(0x7f893415f5a0 0x7f89340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 -- 172.21.15.91:0/855829053 >> v2:172.21.15.113:3300/0 conn(0x7f893415f5a0 msgr2=0x7f89340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 --2- 172.21.15.91:0/855829053 >> v2:172.21.15.113:3300/0 conn(0x7f893415f5a0 0x7f89340b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 -- 172.21.15.91:0/855829053 >> v1:172.21.15.113:6789/0 conn(0x7f8934151f90 legacy=0x7f893406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:52.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.225+0000 7f8933200640 1 -- 172.21.15.91:0/855829053 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934152370 con 0x7f89340b0c40 2024-02-02T13:24:52.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.226+0000 7f8933200640 1 --2- 172.21.15.91:0/855829053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89340b0c40 0x7f89340b3af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8934071a00 tx=0x7f892c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14d8c5a8f407a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.226+0000 7f8931e00640 1 -- 172.21.15.91:0/855829053 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c02ca80 con 0x7f89340b0c40 2024-02-02T13:24:52.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.226+0000 7f8931e00640 1 -- 172.21.15.91:0/855829053 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c02cbe0 con 0x7f89340b0c40 2024-02-02T13:24:52.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.226+0000 7f8931e00640 1 -- 172.21.15.91:0/855829053 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c0343f0 con 0x7f89340b0c40 2024-02-02T13:24:52.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 -- 172.21.15.91:0/855829053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89340b0c40 msgr2=0x7f89340b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 --2- 172.21.15.91:0/855829053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89340b0c40 0x7f89340b3af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8934071a00 tx=0x7f892c02bee0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 -- 172.21.15.91:0/855829053 shutdown_connections 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 --2- 172.21.15.91:0/855829053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89340b0c40 0x7f89340b3af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 --2- 172.21.15.91:0/855829053 >> v2:172.21.15.113:3300/0 conn(0x7f893415f5a0 0x7f89340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 -- 172.21.15.91:0/855829053 >> 172.21.15.91:0/855829053 conn(0x7f89340cec80 msgr2=0x7f89340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 -- 172.21.15.91:0/855829053 shutdown_connections 2024-02-02T13:24:52.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.227+0000 7f8939400640 1 -- 172.21.15.91:0/855829053 wait complete. 2024-02-02T13:24:52.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.228+0000 7f8939400640 1 Processor -- start 2024-02-02T13:24:52.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.229+0000 7f8939400640 1 -- start start 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.229+0000 7f8939400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 0x7f893415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8939400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8939400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893415f5a0 0x7f893415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8939400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934155c30 con 0x7f8934151f90 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8939400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934155f10 con 0x7f89340b0c40 2024-02-02T13:24:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8939400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89341561f0 con 0x7f893415f5a0 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54480/0 (socket says 172.21.15.91:54480) 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 -- 172.21.15.91:0/2279176818 learned_addr learned my addr 172.21.15.91:0/2279176818 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 -- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893415f5a0 msgr2=0x7f893415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893415f5a0 0x7f893415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8932800640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 0x7f893415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 -- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 msgr2=0x7f893415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 0x7f893415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 -- 172.21.15.91:0/2279176818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c009b30 con 0x7f8934151f90 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8932800640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 0x7f893415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.230+0000 7f8929c00640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c7f0 tx=0x7f891c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.231+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c004220 con 0x7f8934151f90 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.231+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c004380 con 0x7f8934151f90 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.231+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c007840 con 0x7f8934151f90 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.231+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89341564d0 con 0x7f8934151f90 2024-02-02T13:24:52.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.231+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893415da20 con 0x7f8934151f90 2024-02-02T13:24:52.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.232+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89341569f0 con 0x7f8934151f90 2024-02-02T13:24:52.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.233+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f891c007bb0 con 0x7f8934151f90 2024-02-02T13:24:52.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.233+0000 7f892b400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 0x7f890c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.234+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f891c0888f0 con 0x7f8934151f90 2024-02-02T13:24:52.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.234+0000 7f8932800640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 0x7f890c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.234+0000 7f8932800640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 0x7f890c06c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f892400a810 tx=0x7f8924008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.238+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f89341569f0 con 0x7f8934151f90 2024-02-02T13:24:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.418+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89340cb2f0 con 0x7f890c06a430 2024-02-02T13:24:52.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.426+0000 7f892b400640 1 -- 172.21.15.91:0/2279176818 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260155 (secure 0 0 0) 0x7f892400ad90 con 0x7f890c06a430 2024-02-02T13:24:52.431 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:52.433 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 msgr2=0x7f890c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 0x7f890c06c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f892400a810 tx=0x7f8924008040 comp rx=0 tx=0).stop 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 msgr2=0x7f893415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c7f0 tx=0x7f891c00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 shutdown_connections 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f890c06a430 0x7f890c06c8f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893415f5a0 0x7f893415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8934151f90 0x7f893415ba50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 --2- 172.21.15.91:0/2279176818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89340b0c40 0x7f893415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 >> 172.21.15.91:0/2279176818 conn(0x7f89340cec80 msgr2=0x7f89340af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 shutdown_connections 2024-02-02T13:24:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.432+0000 7f8939400640 1 -- 172.21.15.91:0/2279176818 wait complete. 2024-02-02T13:24:52.450 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-02-02T13:24:52.240333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560793,"statfs":{"total":1600000000,"available":1598252712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004284"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978392+0000","last_change":"2024-02-02T13:24:36.767566+0000","last_active":"2024-02-02T13:24:40.978392+0000","last_peered":"2024-02-02T13:24:40.978392+0000","last_clean":"2024-02-02T13:24:40.978392+0000","last_became_active":"2024-02-02T13:24:36.767342+0000","last_became_peered":"2024-02-02T13:24:36.767342+0000","last_unstale":"2024-02-02T13:24:40.978392+0000","last_undegraded":"2024-02-02T13:24:40.978392+0000","last_fullsized":"2024-02-02T13:24:40.978392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876609+0000","last_change":"2024-02-02T13:24:36.753974+0000","last_active":"2024-02-02T13:24:39.876609+0000","last_peered":"2024-02-02T13:24:39.876609+0000","last_clean":"2024-02-02T13:24:39.876609+0000","last_became_active":"2024-02-02T13:24:36.753603+0000","last_became_peered":"2024-02-02T13:24:36.753603+0000","last_unstale":"2024-02-02T13:24:39.876609+0000","last_undegraded":"2024-02-02T13:24:39.876609+0000","last_fullsized":"2024-02-02T13:24:39.876609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092881+0000","last_change":"2024-02-02T13:24:36.761233+0000","last_active":"2024-02-02T13:24:41.092881+0000","last_peered":"2024-02-02T13:24:41.092881+0000","last_clean":"2024-02-02T13:24:41.092881+0000","last_became_active":"2024-02-02T13:24:36.761020+0000","last_became_peered":"2024-02-02T13:24:36.761020+0000","last_unstale":"2024-02-02T13:24:41.092881+0000","last_undegraded":"2024-02-02T13:24:41.092881+0000","last_fullsized":"2024-02-02T13:24:41.092881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773368+0000","last_change":"2024-02-02T13:24:36.772920+0000","last_active":"2024-02-02T13:24:36.773368+0000","last_peered":"2024-02-02T13:24:36.773368+0000","last_clean":"2024-02-02T13:24:36.773368+0000","last_became_active":"2024-02-02T13:24:36.765145+0000","last_became_peered":"2024-02-02T13:24:36.765145+0000","last_unstale":"2024-02-02T13:24:36.773368+0000","last_undegraded":"2024-02-02T13:24:36.773368+0000","last_fullsized":"2024-02-02T13:24:36.773368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978323+0000","last_change":"2024-02-02T13:24:36.759645+0000","last_active":"2024-02-02T13:24:40.978323+0000","last_peered":"2024-02-02T13:24:40.978323+0000","last_clean":"2024-02-02T13:24:40.978323+0000","last_became_active":"2024-02-02T13:24:36.759399+0000","last_became_peered":"2024-02-02T13:24:36.759399+0000","last_unstale":"2024-02-02T13:24:40.978323+0000","last_undegraded":"2024-02-02T13:24:40.978323+0000","last_fullsized":"2024-02-02T13:24:40.978323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:52.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777056+0000","last_change":"2024-02-02T13:24:36.774617+0000","last_active":"2024-02-02T13:24:36.777056+0000","last_peered":"2024-02-02T13:24:36.777056+0000","last_clean":"2024-02-02T13:24:36.777056+0000","last_became_active":"2024-02-02T13:24:36.772658+0000","last_became_peered":"2024-02-02T13:24:36.772658+0000","last_unstale":"2024-02-02T13:24:36.777056+0000","last_undegraded":"2024-02-02T13:24:36.777056+0000","last_fullsized":"2024-02-02T13:24:36.777056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978756+0000","last_change":"2024-02-02T13:24:36.755776+0000","last_active":"2024-02-02T13:24:40.978756+0000","last_peered":"2024-02-02T13:24:40.978756+0000","last_clean":"2024-02-02T13:24:40.978756+0000","last_became_active":"2024-02-02T13:24:36.755523+0000","last_became_peered":"2024-02-02T13:24:36.755523+0000","last_unstale":"2024-02-02T13:24:40.978756+0000","last_undegraded":"2024-02-02T13:24:40.978756+0000","last_fullsized":"2024-02-02T13:24:40.978756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:39:14.180157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211535+0000","last_change":"2024-02-02T13:24:36.759777+0000","last_active":"2024-02-02T13:24:41.211535+0000","last_peered":"2024-02-02T13:24:41.211535+0000","last_clean":"2024-02-02T13:24:41.211535+0000","last_became_active":"2024-02-02T13:24:36.759170+0000","last_became_peered":"2024-02-02T13:24:36.759170+0000","last_unstale":"2024-02-02T13:24:41.211535+0000","last_undegraded":"2024-02-02T13:24:41.211535+0000","last_fullsized":"2024-02-02T13:24:41.211535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978056+0000","last_change":"2024-02-02T13:24:36.757480+0000","last_active":"2024-02-02T13:24:40.978056+0000","last_peered":"2024-02-02T13:24:40.978056+0000","last_clean":"2024-02-02T13:24:40.978056+0000","last_became_active":"2024-02-02T13:24:36.757254+0000","last_became_peered":"2024-02-02T13:24:36.757254+0000","last_unstale":"2024-02-02T13:24:40.978056+0000","last_undegraded":"2024-02-02T13:24:40.978056+0000","last_fullsized":"2024-02-02T13:24:40.978056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:41.311042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775838+0000","last_change":"2024-02-02T13:24:36.775765+0000","last_active":"2024-02-02T13:24:36.775838+0000","last_peered":"2024-02-02T13:24:36.775838+0000","last_clean":"2024-02-02T13:24:36.775838+0000","last_became_active":"2024-02-02T13:24:36.775523+0000","last_became_peered":"2024-02-02T13:24:36.775523+0000","last_unstale":"2024-02-02T13:24:36.775838+0000","last_undegraded":"2024-02-02T13:24:36.775838+0000","last_fullsized":"2024-02-02T13:24:36.775838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778319+0000","last_change":"2024-02-02T13:24:36.778240+0000","last_active":"2024-02-02T13:24:36.778319+0000","last_peered":"2024-02-02T13:24:36.778319+0000","last_clean":"2024-02-02T13:24:36.778319+0000","last_became_active":"2024-02-02T13:24:36.778019+0000","last_became_peered":"2024-02-02T13:24:36.778019+0000","last_unstale":"2024-02-02T13:24:36.778319+0000","last_undegraded":"2024-02-02T13:24:36.778319+0000","last_fullsized":"2024-02-02T13:24:36.778319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753463+0000","last_change":"2024-02-02T13:24:36.766203+0000","last_active":"2024-02-02T13:24:37.753463+0000","last_peered":"2024-02-02T13:24:37.753463+0000","last_clean":"2024-02-02T13:24:37.753463+0000","last_became_active":"2024-02-02T13:24:36.765708+0000","last_became_peered":"2024-02-02T13:24:36.765708+0000","last_unstale":"2024-02-02T13:24:37.753463+0000","last_undegraded":"2024-02-02T13:24:37.753463+0000","last_fullsized":"2024-02-02T13:24:37.753463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211785+0000","last_change":"2024-02-02T13:24:36.768024+0000","last_active":"2024-02-02T13:24:41.211785+0000","last_peered":"2024-02-02T13:24:41.211785+0000","last_clean":"2024-02-02T13:24:41.211785+0000","last_became_active":"2024-02-02T13:24:36.767819+0000","last_became_peered":"2024-02-02T13:24:36.767819+0000","last_unstale":"2024-02-02T13:24:41.211785+0000","last_undegraded":"2024-02-02T13:24:41.211785+0000","last_fullsized":"2024-02-02T13:24:41.211785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093008+0000","last_change":"2024-02-02T13:24:36.755455+0000","last_active":"2024-02-02T13:24:41.093008+0000","last_peered":"2024-02-02T13:24:41.093008+0000","last_clean":"2024-02-02T13:24:41.093008+0000","last_became_active":"2024-02-02T13:24:36.755245+0000","last_became_peered":"2024-02-02T13:24:36.755245+0000","last_unstale":"2024-02-02T13:24:41.093008+0000","last_undegraded":"2024-02-02T13:24:41.093008+0000","last_fullsized":"2024-02-02T13:24:41.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876981+0000","last_change":"2024-02-02T13:24:36.757288+0000","last_active":"2024-02-02T13:24:39.876981+0000","last_peered":"2024-02-02T13:24:39.876981+0000","last_clean":"2024-02-02T13:24:39.876981+0000","last_became_active":"2024-02-02T13:24:36.756687+0000","last_became_peered":"2024-02-02T13:24:36.756687+0000","last_unstale":"2024-02-02T13:24:39.876981+0000","last_undegraded":"2024-02-02T13:24:39.876981+0000","last_fullsized":"2024-02-02T13:24:39.876981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777000+0000","last_change":"2024-02-02T13:24:36.774416+0000","last_active":"2024-02-02T13:24:36.777000+0000","last_peered":"2024-02-02T13:24:36.777000+0000","last_clean":"2024-02-02T13:24:36.777000+0000","last_became_active":"2024-02-02T13:24:36.771102+0000","last_became_peered":"2024-02-02T13:24:36.771102+0000","last_unstale":"2024-02-02T13:24:36.777000+0000","last_undegraded":"2024-02-02T13:24:36.777000+0000","last_fullsized":"2024-02-02T13:24:36.777000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761526+0000","last_change":"2024-02-02T13:24:36.761450+0000","last_active":"2024-02-02T13:24:36.761526+0000","last_peered":"2024-02-02T13:24:36.761526+0000","last_clean":"2024-02-02T13:24:36.761526+0000","last_became_active":"2024-02-02T13:24:36.760875+0000","last_became_peered":"2024-02-02T13:24:36.760875+0000","last_unstale":"2024-02-02T13:24:36.761526+0000","last_undegraded":"2024-02-02T13:24:36.761526+0000","last_fullsized":"2024-02-02T13:24:36.761526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771249+0000","last_change":"2024-02-02T13:24:36.768404+0000","last_active":"2024-02-02T13:24:36.771249+0000","last_peered":"2024-02-02T13:24:36.771249+0000","last_clean":"2024-02-02T13:24:36.771249+0000","last_became_active":"2024-02-02T13:24:36.768048+0000","last_became_peered":"2024-02-02T13:24:36.768048+0000","last_unstale":"2024-02-02T13:24:36.771249+0000","last_undegraded":"2024-02-02T13:24:36.771249+0000","last_fullsized":"2024-02-02T13:24:36.771249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978685+0000","last_change":"2024-02-02T13:24:36.762446+0000","last_active":"2024-02-02T13:24:40.978685+0000","last_peered":"2024-02-02T13:24:40.978685+0000","last_clean":"2024-02-02T13:24:40.978685+0000","last_became_active":"2024-02-02T13:24:36.762021+0000","last_became_peered":"2024-02-02T13:24:36.762021+0000","last_unstale":"2024-02-02T13:24:40.978685+0000","last_undegraded":"2024-02-02T13:24:40.978685+0000","last_fullsized":"2024-02-02T13:24:40.978685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768525+0000","last_change":"2024-02-02T13:24:36.768424+0000","last_active":"2024-02-02T13:24:36.768525+0000","last_peered":"2024-02-02T13:24:36.768525+0000","last_clean":"2024-02-02T13:24:36.768525+0000","last_became_active":"2024-02-02T13:24:36.768203+0000","last_became_peered":"2024-02-02T13:24:36.768203+0000","last_unstale":"2024-02-02T13:24:36.768525+0000","last_undegraded":"2024-02-02T13:24:36.768525+0000","last_fullsized":"2024-02-02T13:24:36.768525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093333+0000","last_change":"2024-02-02T13:24:36.762245+0000","last_active":"2024-02-02T13:24:41.093333+0000","last_peered":"2024-02-02T13:24:41.093333+0000","last_clean":"2024-02-02T13:24:41.093333+0000","last_became_active":"2024-02-02T13:24:36.762030+0000","last_became_peered":"2024-02-02T13:24:36.762030+0000","last_unstale":"2024-02-02T13:24:41.093333+0000","last_undegraded":"2024-02-02T13:24:41.093333+0000","last_fullsized":"2024-02-02T13:24:41.093333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771962+0000","last_change":"2024-02-02T13:24:36.771888+0000","last_active":"2024-02-02T13:24:36.771962+0000","last_peered":"2024-02-02T13:24:36.771962+0000","last_clean":"2024-02-02T13:24:36.771962+0000","last_became_active":"2024-02-02T13:24:36.770589+0000","last_became_peered":"2024-02-02T13:24:36.770589+0000","last_unstale":"2024-02-02T13:24:36.771962+0000","last_undegraded":"2024-02-02T13:24:36.771962+0000","last_fullsized":"2024-02-02T13:24:36.771962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212147+0000","last_change":"2024-02-02T13:24:36.750480+0000","last_active":"2024-02-02T13:24:41.212147+0000","last_peered":"2024-02-02T13:24:41.212147+0000","last_clean":"2024-02-02T13:24:41.212147+0000","last_became_active":"2024-02-02T13:24:36.750125+0000","last_became_peered":"2024-02-02T13:24:36.750125+0000","last_unstale":"2024-02-02T13:24:41.212147+0000","last_undegraded":"2024-02-02T13:24:41.212147+0000","last_fullsized":"2024-02-02T13:24:41.212147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773388+0000","last_change":"2024-02-02T13:24:36.773077+0000","last_active":"2024-02-02T13:24:36.773388+0000","last_peered":"2024-02-02T13:24:36.773388+0000","last_clean":"2024-02-02T13:24:36.773388+0000","last_became_active":"2024-02-02T13:24:36.765475+0000","last_became_peered":"2024-02-02T13:24:36.765475+0000","last_unstale":"2024-02-02T13:24:36.773388+0000","last_undegraded":"2024-02-02T13:24:36.773388+0000","last_fullsized":"2024-02-02T13:24:36.773388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211624+0000","last_change":"2024-02-02T13:24:36.750736+0000","last_active":"2024-02-02T13:24:41.211624+0000","last_peered":"2024-02-02T13:24:41.211624+0000","last_clean":"2024-02-02T13:24:41.211624+0000","last_became_active":"2024-02-02T13:24:36.750520+0000","last_became_peered":"2024-02-02T13:24:36.750520+0000","last_unstale":"2024-02-02T13:24:41.211624+0000","last_undegraded":"2024-02-02T13:24:41.211624+0000","last_fullsized":"2024-02-02T13:24:41.211624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764120+0000","last_change":"2024-02-02T13:24:36.763039+0000","last_active":"2024-02-02T13:24:36.764120+0000","last_peered":"2024-02-02T13:24:36.764120+0000","last_clean":"2024-02-02T13:24:36.764120+0000","last_became_active":"2024-02-02T13:24:36.762632+0000","last_became_peered":"2024-02-02T13:24:36.762632+0000","last_unstale":"2024-02-02T13:24:36.764120+0000","last_undegraded":"2024-02-02T13:24:36.764120+0000","last_fullsized":"2024-02-02T13:24:36.764120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877008+0000","last_change":"2024-02-02T13:24:36.750634+0000","last_active":"2024-02-02T13:24:39.877008+0000","last_peered":"2024-02-02T13:24:39.877008+0000","last_clean":"2024-02-02T13:24:39.877008+0000","last_became_active":"2024-02-02T13:24:36.750023+0000","last_became_peered":"2024-02-02T13:24:36.750023+0000","last_unstale":"2024-02-02T13:24:39.877008+0000","last_undegraded":"2024-02-02T13:24:39.877008+0000","last_fullsized":"2024-02-02T13:24:39.877008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211911+0000","last_change":"2024-02-02T13:24:36.750339+0000","last_active":"2024-02-02T13:24:41.211911+0000","last_peered":"2024-02-02T13:24:41.211911+0000","last_clean":"2024-02-02T13:24:41.211911+0000","last_became_active":"2024-02-02T13:24:36.749819+0000","last_became_peered":"2024-02-02T13:24:36.749819+0000","last_unstale":"2024-02-02T13:24:41.211911+0000","last_undegraded":"2024-02-02T13:24:41.211911+0000","last_fullsized":"2024-02-02T13:24:41.211911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093260+0000","last_change":"2024-02-02T13:24:36.765687+0000","last_active":"2024-02-02T13:24:41.093260+0000","last_peered":"2024-02-02T13:24:41.093260+0000","last_clean":"2024-02-02T13:24:41.093260+0000","last_became_active":"2024-02-02T13:24:36.765417+0000","last_became_peered":"2024-02-02T13:24:36.765417+0000","last_unstale":"2024-02-02T13:24:41.093260+0000","last_undegraded":"2024-02-02T13:24:41.093260+0000","last_fullsized":"2024-02-02T13:24:41.093260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:17:34.755365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762953+0000","last_change":"2024-02-02T13:24:36.762822+0000","last_active":"2024-02-02T13:24:36.762953+0000","last_peered":"2024-02-02T13:24:36.762953+0000","last_clean":"2024-02-02T13:24:36.762953+0000","last_became_active":"2024-02-02T13:24:36.761559+0000","last_became_peered":"2024-02-02T13:24:36.761559+0000","last_unstale":"2024-02-02T13:24:36.762953+0000","last_undegraded":"2024-02-02T13:24:36.762953+0000","last_fullsized":"2024-02-02T13:24:36.762953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764632+0000","last_change":"2024-02-02T13:24:36.764552+0000","last_active":"2024-02-02T13:24:36.764632+0000","last_peered":"2024-02-02T13:24:36.764632+0000","last_clean":"2024-02-02T13:24:36.764632+0000","last_became_active":"2024-02-02T13:24:36.764330+0000","last_became_peered":"2024-02-02T13:24:36.764330+0000","last_unstale":"2024-02-02T13:24:36.764632+0000","last_undegraded":"2024-02-02T13:24:36.764632+0000","last_fullsized":"2024-02-02T13:24:36.764632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:16.248831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.765462+0000","last_change":"2024-02-02T13:24:36.765179+0000","last_active":"2024-02-02T13:24:36.765462+0000","last_peered":"2024-02-02T13:24:36.765462+0000","last_clean":"2024-02-02T13:24:36.765462+0000","last_became_active":"2024-02-02T13:24:36.764850+0000","last_became_peered":"2024-02-02T13:24:36.764850+0000","last_unstale":"2024-02-02T13:24:36.765462+0000","last_undegraded":"2024-02-02T13:24:36.765462+0000","last_fullsized":"2024-02-02T13:24:36.765462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753019+0000","last_change":"2024-02-02T13:24:36.755360+0000","last_active":"2024-02-02T13:24:37.753019+0000","last_peered":"2024-02-02T13:24:37.753019+0000","last_clean":"2024-02-02T13:24:37.753019+0000","last_became_active":"2024-02-02T13:24:36.754987+0000","last_became_peered":"2024-02-02T13:24:36.754987+0000","last_unstale":"2024-02-02T13:24:37.753019+0000","last_undegraded":"2024-02-02T13:24:37.753019+0000","last_fullsized":"2024-02-02T13:24:37.753019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:49.562083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772676+0000","last_change":"2024-02-02T13:24:36.288622+0000","last_active":"2024-02-02T13:24:36.772676+0000","last_peered":"2024-02-02T13:24:36.772676+0000","last_clean":"2024-02-02T13:24:36.772676+0000","last_became_active":"2024-02-02T13:24:36.288400+0000","last_became_peered":"2024-02-02T13:24:36.288400+0000","last_unstale":"2024-02-02T13:24:36.772676+0000","last_undegraded":"2024-02-02T13:24:36.772676+0000","last_fullsized":"2024-02-02T13:24:36.772676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.092688+0000","last_change":"2024-02-02T13:24:36.765345+0000","last_active":"2024-02-02T13:24:41.092688+0000","last_peered":"2024-02-02T13:24:41.092688+0000","last_clean":"2024-02-02T13:24:41.092688+0000","last_became_active":"2024-02-02T13:24:36.765122+0000","last_became_peered":"2024-02-02T13:24:36.765122+0000","last_unstale":"2024-02-02T13:24:41.092688+0000","last_undegraded":"2024-02-02T13:24:41.092688+0000","last_fullsized":"2024-02-02T13:24:41.092688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:39.598077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774103+0000","last_change":"2024-02-02T13:24:36.284315+0000","last_active":"2024-02-02T13:24:36.774103+0000","last_peered":"2024-02-02T13:24:36.774103+0000","last_clean":"2024-02-02T13:24:36.774103+0000","last_became_active":"2024-02-02T13:24:36.283193+0000","last_became_peered":"2024-02-02T13:24:36.283193+0000","last_unstale":"2024-02-02T13:24:36.774103+0000","last_undegraded":"2024-02-02T13:24:36.774103+0000","last_fullsized":"2024-02-02T13:24:36.774103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.741346+0000","last_change":"2024-02-02T13:24:36.280943+0000","last_active":"2024-02-02T13:24:36.741346+0000","last_peered":"2024-02-02T13:24:36.741346+0000","last_clean":"2024-02-02T13:24:36.741346+0000","last_became_active":"2024-02-02T13:24:36.280565+0000","last_became_peered":"2024-02-02T13:24:36.280565+0000","last_unstale":"2024-02-02T13:24:36.741346+0000","last_undegraded":"2024-02-02T13:24:36.741346+0000","last_fullsized":"2024-02-02T13:24:36.741346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211701+0000","last_change":"2024-02-02T13:24:36.761849+0000","last_active":"2024-02-02T13:24:41.211701+0000","last_peered":"2024-02-02T13:24:41.211701+0000","last_clean":"2024-02-02T13:24:41.211701+0000","last_became_active":"2024-02-02T13:24:36.761460+0000","last_became_peered":"2024-02-02T13:24:36.761460+0000","last_unstale":"2024-02-02T13:24:41.211701+0000","last_undegraded":"2024-02-02T13:24:41.211701+0000","last_fullsized":"2024-02-02T13:24:41.211701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:29:55.608720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.753498+0000","last_change":"2024-02-02T13:24:36.286417+0000","last_active":"2024-02-02T13:24:36.753498+0000","last_peered":"2024-02-02T13:24:36.753498+0000","last_clean":"2024-02-02T13:24:36.753498+0000","last_became_active":"2024-02-02T13:24:36.286107+0000","last_became_peered":"2024-02-02T13:24:36.286107+0000","last_unstale":"2024-02-02T13:24:36.753498+0000","last_undegraded":"2024-02-02T13:24:36.753498+0000","last_fullsized":"2024-02-02T13:24:36.753498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876738+0000","last_change":"2024-02-02T13:24:36.750157+0000","last_active":"2024-02-02T13:24:39.876738+0000","last_peered":"2024-02-02T13:24:39.876738+0000","last_clean":"2024-02-02T13:24:39.876738+0000","last_became_active":"2024-02-02T13:24:36.749764+0000","last_became_peered":"2024-02-02T13:24:36.749764+0000","last_unstale":"2024-02-02T13:24:39.876738+0000","last_undegraded":"2024-02-02T13:24:39.876738+0000","last_fullsized":"2024-02-02T13:24:39.876738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.762177+0000","last_change":"2024-02-02T13:24:36.290816+0000","last_active":"2024-02-02T13:24:36.762177+0000","last_peered":"2024-02-02T13:24:36.762177+0000","last_clean":"2024-02-02T13:24:36.762177+0000","last_became_active":"2024-02-02T13:24:36.290592+0000","last_became_peered":"2024-02-02T13:24:36.290592+0000","last_unstale":"2024-02-02T13:24:36.762177+0000","last_undegraded":"2024-02-02T13:24:36.762177+0000","last_fullsized":"2024-02-02T13:24:36.762177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.761656+0000","last_change":"2024-02-02T13:24:36.761589+0000","last_active":"2024-02-02T13:24:36.761656+0000","last_peered":"2024-02-02T13:24:36.761656+0000","last_clean":"2024-02-02T13:24:36.761656+0000","last_became_active":"2024-02-02T13:24:36.761215+0000","last_became_peered":"2024-02-02T13:24:36.761215+0000","last_unstale":"2024-02-02T13:24:36.761656+0000","last_undegraded":"2024-02-02T13:24:36.761656+0000","last_fullsized":"2024-02-02T13:24:36.761656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.758412+0000","last_change":"2024-02-02T13:24:36.281944+0000","last_active":"2024-02-02T13:24:36.758412+0000","last_peered":"2024-02-02T13:24:36.758412+0000","last_clean":"2024-02-02T13:24:36.758412+0000","last_became_active":"2024-02-02T13:24:36.281546+0000","last_became_peered":"2024-02-02T13:24:36.281546+0000","last_unstale":"2024-02-02T13:24:36.758412+0000","last_undegraded":"2024-02-02T13:24:36.758412+0000","last_fullsized":"2024-02-02T13:24:36.758412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093550+0000","last_change":"2024-02-02T13:24:36.775310+0000","last_active":"2024-02-02T13:24:41.093550+0000","last_peered":"2024-02-02T13:24:41.093550+0000","last_clean":"2024-02-02T13:24:41.093550+0000","last_became_active":"2024-02-02T13:24:36.775021+0000","last_became_peered":"2024-02-02T13:24:36.775021+0000","last_unstale":"2024-02-02T13:24:41.093550+0000","last_undegraded":"2024-02-02T13:24:41.093550+0000","last_fullsized":"2024-02-02T13:24:41.093550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:45:32.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777366+0000","last_change":"2024-02-02T13:24:36.267935+0000","last_active":"2024-02-02T13:24:36.777366+0000","last_peered":"2024-02-02T13:24:36.777366+0000","last_clean":"2024-02-02T13:24:36.777366+0000","last_became_active":"2024-02-02T13:24:36.267120+0000","last_became_peered":"2024-02-02T13:24:36.267120+0000","last_unstale":"2024-02-02T13:24:36.777366+0000","last_undegraded":"2024-02-02T13:24:36.777366+0000","last_fullsized":"2024-02-02T13:24:36.777366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753018+0000","last_change":"2024-02-02T13:24:36.758353+0000","last_active":"2024-02-02T13:24:37.753018+0000","last_peered":"2024-02-02T13:24:37.753018+0000","last_clean":"2024-02-02T13:24:37.753018+0000","last_became_active":"2024-02-02T13:24:36.758074+0000","last_became_peered":"2024-02-02T13:24:36.758074+0000","last_unstale":"2024-02-02T13:24:37.753018+0000","last_undegraded":"2024-02-02T13:24:37.753018+0000","last_fullsized":"2024-02-02T13:24:37.753018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:44.279594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.776688+0000","last_change":"2024-02-02T13:24:36.283779+0000","last_active":"2024-02-02T13:24:36.776688+0000","last_peered":"2024-02-02T13:24:36.776688+0000","last_clean":"2024-02-02T13:24:36.776688+0000","last_became_active":"2024-02-02T13:24:36.280495+0000","last_became_peered":"2024-02-02T13:24:36.280495+0000","last_unstale":"2024-02-02T13:24:36.776688+0000","last_undegraded":"2024-02-02T13:24:36.776688+0000","last_fullsized":"2024-02-02T13:24:36.776688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978240+0000","last_change":"2024-02-02T13:24:36.751477+0000","last_active":"2024-02-02T13:24:40.978240+0000","last_peered":"2024-02-02T13:24:40.978240+0000","last_clean":"2024-02-02T13:24:40.978240+0000","last_became_active":"2024-02-02T13:24:36.751226+0000","last_became_peered":"2024-02-02T13:24:36.751226+0000","last_unstale":"2024-02-02T13:24:40.978240+0000","last_undegraded":"2024-02-02T13:24:40.978240+0000","last_fullsized":"2024-02-02T13:24:40.978240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:40:29.586938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773129+0000","last_change":"2024-02-02T13:24:36.286732+0000","last_active":"2024-02-02T13:24:36.773129+0000","last_peered":"2024-02-02T13:24:36.773129+0000","last_clean":"2024-02-02T13:24:36.773129+0000","last_became_active":"2024-02-02T13:24:36.286351+0000","last_became_peered":"2024-02-02T13:24:36.286351+0000","last_unstale":"2024-02-02T13:24:36.773129+0000","last_undegraded":"2024-02-02T13:24:36.773129+0000","last_fullsized":"2024-02-02T13:24:36.773129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:55:32.650288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093755+0000","last_change":"2024-02-02T13:24:36.756739+0000","last_active":"2024-02-02T13:24:41.093755+0000","last_peered":"2024-02-02T13:24:41.093755+0000","last_clean":"2024-02-02T13:24:41.093755+0000","last_became_active":"2024-02-02T13:24:36.756537+0000","last_became_peered":"2024-02-02T13:24:36.756537+0000","last_unstale":"2024-02-02T13:24:41.093755+0000","last_undegraded":"2024-02-02T13:24:41.093755+0000","last_fullsized":"2024-02-02T13:24:41.093755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755625+0000","last_change":"2024-02-02T13:24:36.286698+0000","last_active":"2024-02-02T13:24:36.755625+0000","last_peered":"2024-02-02T13:24:36.755625+0000","last_clean":"2024-02-02T13:24:36.755625+0000","last_became_active":"2024-02-02T13:24:36.282637+0000","last_became_peered":"2024-02-02T13:24:36.282637+0000","last_unstale":"2024-02-02T13:24:36.755625+0000","last_undegraded":"2024-02-02T13:24:36.755625+0000","last_fullsized":"2024-02-02T13:24:36.755625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978581+0000","last_change":"2024-02-02T13:24:36.766477+0000","last_active":"2024-02-02T13:24:40.978581+0000","last_peered":"2024-02-02T13:24:40.978581+0000","last_clean":"2024-02-02T13:24:40.978581+0000","last_became_active":"2024-02-02T13:24:36.765889+0000","last_became_peered":"2024-02-02T13:24:36.765889+0000","last_unstale":"2024-02-02T13:24:40.978581+0000","last_undegraded":"2024-02-02T13:24:40.978581+0000","last_fullsized":"2024-02-02T13:24:40.978581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:04:37.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.764428+0000","last_change":"2024-02-02T13:24:36.286937+0000","last_active":"2024-02-02T13:24:36.764428+0000","last_peered":"2024-02-02T13:24:36.764428+0000","last_clean":"2024-02-02T13:24:36.764428+0000","last_became_active":"2024-02-02T13:24:36.286651+0000","last_became_peered":"2024-02-02T13:24:36.286651+0000","last_unstale":"2024-02-02T13:24:36.764428+0000","last_undegraded":"2024-02-02T13:24:36.764428+0000","last_fullsized":"2024-02-02T13:24:36.764428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.769009+0000","last_change":"2024-02-02T13:24:36.768901+0000","last_active":"2024-02-02T13:24:36.769009+0000","last_peered":"2024-02-02T13:24:36.769009+0000","last_clean":"2024-02-02T13:24:36.769009+0000","last_became_active":"2024-02-02T13:24:36.768565+0000","last_became_peered":"2024-02-02T13:24:36.768565+0000","last_unstale":"2024-02-02T13:24:36.769009+0000","last_undegraded":"2024-02-02T13:24:36.769009+0000","last_fullsized":"2024-02-02T13:24:36.769009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.775361+0000","last_change":"2024-02-02T13:24:36.271772+0000","last_active":"2024-02-02T13:24:36.775361+0000","last_peered":"2024-02-02T13:24:36.775361+0000","last_clean":"2024-02-02T13:24:36.775361+0000","last_became_active":"2024-02-02T13:24:36.271163+0000","last_became_peered":"2024-02-02T13:24:36.271163+0000","last_unstale":"2024-02-02T13:24:36.775361+0000","last_undegraded":"2024-02-02T13:24:36.775361+0000","last_fullsized":"2024-02-02T13:24:36.775361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.979190+0000","last_change":"2024-02-02T13:24:36.756837+0000","last_active":"2024-02-02T13:24:40.979190+0000","last_peered":"2024-02-02T13:24:40.979190+0000","last_clean":"2024-02-02T13:24:40.979190+0000","last_became_active":"2024-02-02T13:24:36.756508+0000","last_became_peered":"2024-02-02T13:24:36.756508+0000","last_unstale":"2024-02-02T13:24:40.979190+0000","last_undegraded":"2024-02-02T13:24:40.979190+0000","last_fullsized":"2024-02-02T13:24:40.979190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771720+0000","last_change":"2024-02-02T13:24:36.287213+0000","last_active":"2024-02-02T13:24:36.771720+0000","last_peered":"2024-02-02T13:24:36.771720+0000","last_clean":"2024-02-02T13:24:36.771720+0000","last_became_active":"2024-02-02T13:24:36.286967+0000","last_became_peered":"2024-02-02T13:24:36.286967+0000","last_unstale":"2024-02-02T13:24:36.771720+0000","last_undegraded":"2024-02-02T13:24:36.771720+0000","last_fullsized":"2024-02-02T13:24:36.771720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:19.826795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978811+0000","last_change":"2024-02-02T13:24:36.766631+0000","last_active":"2024-02-02T13:24:40.978811+0000","last_peered":"2024-02-02T13:24:40.978811+0000","last_clean":"2024-02-02T13:24:40.978811+0000","last_became_active":"2024-02-02T13:24:36.766216+0000","last_became_peered":"2024-02-02T13:24:36.766216+0000","last_unstale":"2024-02-02T13:24:40.978811+0000","last_undegraded":"2024-02-02T13:24:40.978811+0000","last_fullsized":"2024-02-02T13:24:40.978811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:08:16.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772131+0000","last_change":"2024-02-02T13:24:36.275397+0000","last_active":"2024-02-02T13:24:36.772131+0000","last_peered":"2024-02-02T13:24:36.772131+0000","last_clean":"2024-02-02T13:24:36.772131+0000","last_became_active":"2024-02-02T13:24:36.274927+0000","last_became_peered":"2024-02-02T13:24:36.274927+0000","last_unstale":"2024-02-02T13:24:36.772131+0000","last_undegraded":"2024-02-02T13:24:36.772131+0000","last_fullsized":"2024-02-02T13:24:36.772131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978419+0000","last_change":"2024-02-02T13:24:36.757035+0000","last_active":"2024-02-02T13:24:40.978419+0000","last_peered":"2024-02-02T13:24:40.978419+0000","last_clean":"2024-02-02T13:24:40.978419+0000","last_became_active":"2024-02-02T13:24:36.756748+0000","last_became_peered":"2024-02-02T13:24:36.756748+0000","last_unstale":"2024-02-02T13:24:40.978419+0000","last_undegraded":"2024-02-02T13:24:40.978419+0000","last_fullsized":"2024-02-02T13:24:40.978419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:24:57.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.768414+0000","last_change":"2024-02-02T13:24:36.283044+0000","last_active":"2024-02-02T13:24:36.768414+0000","last_peered":"2024-02-02T13:24:36.768414+0000","last_clean":"2024-02-02T13:24:36.768414+0000","last_became_active":"2024-02-02T13:24:36.282798+0000","last_became_peered":"2024-02-02T13:24:36.282798+0000","last_unstale":"2024-02-02T13:24:36.768414+0000","last_undegraded":"2024-02-02T13:24:36.768414+0000","last_fullsized":"2024-02-02T13:24:36.768414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770333+0000","last_change":"2024-02-02T13:24:36.764197+0000","last_active":"2024-02-02T13:24:36.770333+0000","last_peered":"2024-02-02T13:24:36.770333+0000","last_clean":"2024-02-02T13:24:36.770333+0000","last_became_active":"2024-02-02T13:24:36.763970+0000","last_became_peered":"2024-02-02T13:24:36.763970+0000","last_unstale":"2024-02-02T13:24:36.770333+0000","last_undegraded":"2024-02-02T13:24:36.770333+0000","last_fullsized":"2024-02-02T13:24:36.770333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:23.630759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.757774+0000","last_change":"2024-02-02T13:24:36.279681+0000","last_active":"2024-02-02T13:24:36.757774+0000","last_peered":"2024-02-02T13:24:36.757774+0000","last_clean":"2024-02-02T13:24:36.757774+0000","last_became_active":"2024-02-02T13:24:36.279342+0000","last_became_peered":"2024-02-02T13:24:36.279342+0000","last_unstale":"2024-02-02T13:24:36.757774+0000","last_undegraded":"2024-02-02T13:24:36.757774+0000","last_fullsized":"2024-02-02T13:24:36.757774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:15:29.035104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:40.978828+0000","last_change":"2024-02-02T13:24:36.758113+0000","last_active":"2024-02-02T13:24:40.978828+0000","last_peered":"2024-02-02T13:24:40.978828+0000","last_clean":"2024-02-02T13:24:40.978828+0000","last_became_active":"2024-02-02T13:24:36.757859+0000","last_became_peered":"2024-02-02T13:24:36.757859+0000","last_unstale":"2024-02-02T13:24:40.978828+0000","last_undegraded":"2024-02-02T13:24:40.978828+0000","last_fullsized":"2024-02-02T13:24:40.978828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:58.279967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748840+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:24:36.748840+0000","last_peered":"2024-02-02T13:24:36.748840+0000","last_clean":"2024-02-02T13:24:36.748840+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:24:36.748840+0000","last_undegraded":"2024-02-02T13:24:36.748840+0000","last_fullsized":"2024-02-02T13:24:36.748840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.755725+0000","last_change":"2024-02-02T13:24:36.285102+0000","last_active":"2024-02-02T13:24:36.755725+0000","last_peered":"2024-02-02T13:24:36.755725+0000","last_clean":"2024-02-02T13:24:36.755725+0000","last_became_active":"2024-02-02T13:24:36.280044+0000","last_became_peered":"2024-02-02T13:24:36.280044+0000","last_unstale":"2024-02-02T13:24:36.755725+0000","last_undegraded":"2024-02-02T13:24:36.755725+0000","last_fullsized":"2024-02-02T13:24:36.755725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:10:18.715647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211798+0000","last_change":"2024-02-02T13:24:36.762214+0000","last_active":"2024-02-02T13:24:41.211798+0000","last_peered":"2024-02-02T13:24:41.211798+0000","last_clean":"2024-02-02T13:24:41.211798+0000","last_became_active":"2024-02-02T13:24:36.761690+0000","last_became_peered":"2024-02-02T13:24:36.761690+0000","last_unstale":"2024-02-02T13:24:41.211798+0000","last_undegraded":"2024-02-02T13:24:41.211798+0000","last_fullsized":"2024-02-02T13:24:41.211798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:50.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748517+0000","last_change":"2024-02-02T13:24:36.291720+0000","last_active":"2024-02-02T13:24:36.748517+0000","last_peered":"2024-02-02T13:24:36.748517+0000","last_clean":"2024-02-02T13:24:36.748517+0000","last_became_active":"2024-02-02T13:24:36.290703+0000","last_became_peered":"2024-02-02T13:24:36.290703+0000","last_unstale":"2024-02-02T13:24:36.748517+0000","last_undegraded":"2024-02-02T13:24:36.748517+0000","last_fullsized":"2024-02-02T13:24:36.748517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093549+0000","last_change":"2024-02-02T13:24:36.766220+0000","last_active":"2024-02-02T13:24:41.093549+0000","last_peered":"2024-02-02T13:24:41.093549+0000","last_clean":"2024-02-02T13:24:41.093549+0000","last_became_active":"2024-02-02T13:24:36.765734+0000","last_became_peered":"2024-02-02T13:24:36.765734+0000","last_unstale":"2024-02-02T13:24:41.093549+0000","last_undegraded":"2024-02-02T13:24:41.093549+0000","last_fullsized":"2024-02-02T13:24:41.093549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.756734+0000","last_change":"2024-02-02T13:24:36.276562+0000","last_active":"2024-02-02T13:24:36.756734+0000","last_peered":"2024-02-02T13:24:36.756734+0000","last_clean":"2024-02-02T13:24:36.756734+0000","last_became_active":"2024-02-02T13:24:36.276189+0000","last_became_peered":"2024-02-02T13:24:36.276189+0000","last_unstale":"2024-02-02T13:24:36.756734+0000","last_undegraded":"2024-02-02T13:24:36.756734+0000","last_fullsized":"2024-02-02T13:24:36.756734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877004+0000","last_change":"2024-02-02T13:24:36.761943+0000","last_active":"2024-02-02T13:24:39.877004+0000","last_peered":"2024-02-02T13:24:39.877004+0000","last_clean":"2024-02-02T13:24:39.877004+0000","last_became_active":"2024-02-02T13:24:36.761718+0000","last_became_peered":"2024-02-02T13:24:36.761718+0000","last_unstale":"2024-02-02T13:24:39.877004+0000","last_undegraded":"2024-02-02T13:24:39.877004+0000","last_fullsized":"2024-02-02T13:24:39.877004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.760484+0000","last_change":"2024-02-02T13:24:36.275569+0000","last_active":"2024-02-02T13:24:36.760484+0000","last_peered":"2024-02-02T13:24:36.760484+0000","last_clean":"2024-02-02T13:24:36.760484+0000","last_became_active":"2024-02-02T13:24:36.273104+0000","last_became_peered":"2024-02-02T13:24:36.273104+0000","last_unstale":"2024-02-02T13:24:36.760484+0000","last_undegraded":"2024-02-02T13:24:36.760484+0000","last_fullsized":"2024-02-02T13:24:36.760484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:02.045199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.753283+0000","last_change":"2024-02-02T13:24:36.763372+0000","last_active":"2024-02-02T13:24:37.753283+0000","last_peered":"2024-02-02T13:24:37.753283+0000","last_clean":"2024-02-02T13:24:37.753283+0000","last_became_active":"2024-02-02T13:24:36.763132+0000","last_became_peered":"2024-02-02T13:24:36.763132+0000","last_unstale":"2024-02-02T13:24:37.753283+0000","last_undegraded":"2024-02-02T13:24:37.753283+0000","last_fullsized":"2024-02-02T13:24:37.753283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:03.055262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747812+0000","last_change":"2024-02-02T13:24:36.287910+0000","last_active":"2024-02-02T13:24:36.747812+0000","last_peered":"2024-02-02T13:24:36.747812+0000","last_clean":"2024-02-02T13:24:36.747812+0000","last_became_active":"2024-02-02T13:24:36.287683+0000","last_became_peered":"2024-02-02T13:24:36.287683+0000","last_unstale":"2024-02-02T13:24:36.747812+0000","last_undegraded":"2024-02-02T13:24:36.747812+0000","last_fullsized":"2024-02-02T13:24:36.747812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.773408+0000","last_change":"2024-02-02T13:24:36.773275+0000","last_active":"2024-02-02T13:24:36.773408+0000","last_peered":"2024-02-02T13:24:36.773408+0000","last_clean":"2024-02-02T13:24:36.773408+0000","last_became_active":"2024-02-02T13:24:36.772646+0000","last_became_peered":"2024-02-02T13:24:36.772646+0000","last_unstale":"2024-02-02T13:24:36.773408+0000","last_undegraded":"2024-02-02T13:24:36.773408+0000","last_fullsized":"2024-02-02T13:24:36.773408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:28.845264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.748406+0000","last_change":"2024-02-02T13:24:36.291402+0000","last_active":"2024-02-02T13:24:36.748406+0000","last_peered":"2024-02-02T13:24:36.748406+0000","last_clean":"2024-02-02T13:24:36.748406+0000","last_became_active":"2024-02-02T13:24:36.290274+0000","last_became_peered":"2024-02-02T13:24:36.290274+0000","last_unstale":"2024-02-02T13:24:36.748406+0000","last_undegraded":"2024-02-02T13:24:36.748406+0000","last_fullsized":"2024-02-02T13:24:36.748406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.876617+0000","last_change":"2024-02-02T13:24:36.760028+0000","last_active":"2024-02-02T13:24:39.876617+0000","last_peered":"2024-02-02T13:24:39.876617+0000","last_clean":"2024-02-02T13:24:39.876617+0000","last_became_active":"2024-02-02T13:24:36.759641+0000","last_became_peered":"2024-02-02T13:24:36.759641+0000","last_unstale":"2024-02-02T13:24:39.876617+0000","last_undegraded":"2024-02-02T13:24:39.876617+0000","last_fullsized":"2024-02-02T13:24:39.876617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:39.877230+0000","last_change":"2024-02-02T13:24:36.281619+0000","last_active":"2024-02-02T13:24:39.877230+0000","last_peered":"2024-02-02T13:24:39.877230+0000","last_clean":"2024-02-02T13:24:39.877230+0000","last_became_active":"2024-02-02T13:24:36.281168+0000","last_became_peered":"2024-02-02T13:24:36.281168+0000","last_unstale":"2024-02-02T13:24:39.877230+0000","last_undegraded":"2024-02-02T13:24:39.877230+0000","last_fullsized":"2024-02-02T13:24:39.877230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:07:15.238768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211709+0000","last_change":"2024-02-02T13:24:36.750882+0000","last_active":"2024-02-02T13:24:41.211709+0000","last_peered":"2024-02-02T13:24:41.211709+0000","last_clean":"2024-02-02T13:24:41.211709+0000","last_became_active":"2024-02-02T13:24:36.750448+0000","last_became_peered":"2024-02-02T13:24:36.750448+0000","last_unstale":"2024-02-02T13:24:41.211709+0000","last_undegraded":"2024-02-02T13:24:41.211709+0000","last_fullsized":"2024-02-02T13:24:41.211709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774907+0000","last_change":"2024-02-02T13:24:36.289169+0000","last_active":"2024-02-02T13:24:36.774907+0000","last_peered":"2024-02-02T13:24:36.774907+0000","last_clean":"2024-02-02T13:24:36.774907+0000","last_became_active":"2024-02-02T13:24:36.288939+0000","last_became_peered":"2024-02-02T13:24:36.288939+0000","last_unstale":"2024-02-02T13:24:36.774907+0000","last_undegraded":"2024-02-02T13:24:36.774907+0000","last_fullsized":"2024-02-02T13:24:36.774907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:12.610878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093133+0000","last_change":"2024-02-02T13:24:36.756653+0000","last_active":"2024-02-02T13:24:41.093133+0000","last_peered":"2024-02-02T13:24:41.093133+0000","last_clean":"2024-02-02T13:24:41.093133+0000","last_became_active":"2024-02-02T13:24:36.756419+0000","last_became_peered":"2024-02-02T13:24:36.756419+0000","last_unstale":"2024-02-02T13:24:41.093133+0000","last_undegraded":"2024-02-02T13:24:41.093133+0000","last_fullsized":"2024-02-02T13:24:41.093133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:36:01.592707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.751195+0000","last_change":"2024-02-02T13:24:36.290423+0000","last_active":"2024-02-02T13:24:36.751195+0000","last_peered":"2024-02-02T13:24:36.751195+0000","last_clean":"2024-02-02T13:24:36.751195+0000","last_became_active":"2024-02-02T13:24:36.289863+0000","last_became_peered":"2024-02-02T13:24:36.289863+0000","last_unstale":"2024-02-02T13:24:36.751195+0000","last_undegraded":"2024-02-02T13:24:36.751195+0000","last_fullsized":"2024-02-02T13:24:36.751195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:43.269531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:37.754234+0000","last_change":"2024-02-02T13:24:36.759429+0000","last_active":"2024-02-02T13:24:37.754234+0000","last_peered":"2024-02-02T13:24:37.754234+0000","last_clean":"2024-02-02T13:24:37.754234+0000","last_became_active":"2024-02-02T13:24:36.759196+0000","last_became_peered":"2024-02-02T13:24:36.759196+0000","last_unstale":"2024-02-02T13:24:37.754234+0000","last_undegraded":"2024-02-02T13:24:37.754234+0000","last_fullsized":"2024-02-02T13:24:37.754234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:32.467879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747537+0000","last_change":"2024-02-02T13:24:36.288653+0000","last_active":"2024-02-02T13:24:36.747537+0000","last_peered":"2024-02-02T13:24:36.747537+0000","last_clean":"2024-02-02T13:24:36.747537+0000","last_became_active":"2024-02-02T13:24:36.288439+0000","last_became_peered":"2024-02-02T13:24:36.288439+0000","last_unstale":"2024-02-02T13:24:36.747537+0000","last_undegraded":"2024-02-02T13:24:36.747537+0000","last_fullsized":"2024-02-02T13:24:36.747537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:48.552020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.211609+0000","last_change":"2024-02-02T13:24:36.751484+0000","last_active":"2024-02-02T13:24:41.211609+0000","last_peered":"2024-02-02T13:24:41.211609+0000","last_clean":"2024-02-02T13:24:41.211609+0000","last_became_active":"2024-02-02T13:24:36.751091+0000","last_became_peered":"2024-02-02T13:24:36.751091+0000","last_unstale":"2024-02-02T13:24:41.211609+0000","last_undegraded":"2024-02-02T13:24:41.211609+0000","last_fullsized":"2024-02-02T13:24:41.211609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:00.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.747419+0000","last_change":"2024-02-02T13:24:36.280128+0000","last_active":"2024-02-02T13:24:36.747419+0000","last_peered":"2024-02-02T13:24:36.747419+0000","last_clean":"2024-02-02T13:24:36.747419+0000","last_became_active":"2024-02-02T13:24:36.279652+0000","last_became_peered":"2024-02-02T13:24:36.279652+0000","last_unstale":"2024-02-02T13:24:36.747419+0000","last_undegraded":"2024-02-02T13:24:36.747419+0000","last_fullsized":"2024-02-02T13:24:36.747419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:05:31.457816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212026+0000","last_change":"2024-02-02T13:24:36.763668+0000","last_active":"2024-02-02T13:24:41.212026+0000","last_peered":"2024-02-02T13:24:41.212026+0000","last_clean":"2024-02-02T13:24:41.212026+0000","last_became_active":"2024-02-02T13:24:36.762913+0000","last_became_peered":"2024-02-02T13:24:36.762913+0000","last_unstale":"2024-02-02T13:24:41.212026+0000","last_undegraded":"2024-02-02T13:24:41.212026+0000","last_fullsized":"2024-02-02T13:24:41.212026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:10.384620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774478+0000","last_change":"2024-02-02T13:24:36.289669+0000","last_active":"2024-02-02T13:24:36.774478+0000","last_peered":"2024-02-02T13:24:36.774478+0000","last_clean":"2024-02-02T13:24:36.774478+0000","last_became_active":"2024-02-02T13:24:36.289368+0000","last_became_peered":"2024-02-02T13:24:36.289368+0000","last_unstale":"2024-02-02T13:24:36.774478+0000","last_undegraded":"2024-02-02T13:24:36.774478+0000","last_fullsized":"2024-02-02T13:24:36.774478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:34.260369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770307+0000","last_change":"2024-02-02T13:24:36.769265+0000","last_active":"2024-02-02T13:24:36.770307+0000","last_peered":"2024-02-02T13:24:36.770307+0000","last_clean":"2024-02-02T13:24:36.770307+0000","last_became_active":"2024-02-02T13:24:36.769036+0000","last_became_peered":"2024-02-02T13:24:36.769036+0000","last_unstale":"2024-02-02T13:24:36.770307+0000","last_undegraded":"2024-02-02T13:24:36.770307+0000","last_fullsized":"2024-02-02T13:24:36.770307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:29.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.770353+0000","last_change":"2024-02-02T13:24:36.765383+0000","last_active":"2024-02-02T13:24:36.770353+0000","last_peered":"2024-02-02T13:24:36.770353+0000","last_clean":"2024-02-02T13:24:36.770353+0000","last_became_active":"2024-02-02T13:24:36.765141+0000","last_became_peered":"2024-02-02T13:24:36.765141+0000","last_unstale":"2024-02-02T13:24:36.770353+0000","last_undegraded":"2024-02-02T13:24:36.770353+0000","last_fullsized":"2024-02-02T13:24:36.770353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:52.887977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.778010+0000","last_change":"2024-02-02T13:24:36.290626+0000","last_active":"2024-02-02T13:24:36.778010+0000","last_peered":"2024-02-02T13:24:36.778010+0000","last_clean":"2024-02-02T13:24:36.778010+0000","last_became_active":"2024-02-02T13:24:36.290183+0000","last_became_peered":"2024-02-02T13:24:36.290183+0000","last_unstale":"2024-02-02T13:24:36.778010+0000","last_undegraded":"2024-02-02T13:24:36.778010+0000","last_fullsized":"2024-02-02T13:24:36.778010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:08:43.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.777961+0000","last_change":"2024-02-02T13:24:36.777865+0000","last_active":"2024-02-02T13:24:36.777961+0000","last_peered":"2024-02-02T13:24:36.777961+0000","last_clean":"2024-02-02T13:24:36.777961+0000","last_became_active":"2024-02-02T13:24:36.777129+0000","last_became_peered":"2024-02-02T13:24:36.777129+0000","last_unstale":"2024-02-02T13:24:36.777961+0000","last_undegraded":"2024-02-02T13:24:36.777961+0000","last_fullsized":"2024-02-02T13:24:36.777961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:03.933391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.772162+0000","last_change":"2024-02-02T13:24:36.290121+0000","last_active":"2024-02-02T13:24:36.772162+0000","last_peered":"2024-02-02T13:24:36.772162+0000","last_clean":"2024-02-02T13:24:36.772162+0000","last_became_active":"2024-02-02T13:24:36.289708+0000","last_became_peered":"2024-02-02T13:24:36.289708+0000","last_unstale":"2024-02-02T13:24:36.772162+0000","last_undegraded":"2024-02-02T13:24:36.772162+0000","last_fullsized":"2024-02-02T13:24:36.772162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:28:58.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.093334+0000","last_change":"2024-02-02T13:24:36.759606+0000","last_active":"2024-02-02T13:24:41.093334+0000","last_peered":"2024-02-02T13:24:41.093334+0000","last_clean":"2024-02-02T13:24:41.093334+0000","last_became_active":"2024-02-02T13:24:36.759358+0000","last_became_peered":"2024-02-02T13:24:36.759358+0000","last_unstale":"2024-02-02T13:24:41.093334+0000","last_undegraded":"2024-02-02T13:24:41.093334+0000","last_fullsized":"2024-02-02T13:24:41.093334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:38.702532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.771605+0000","last_change":"2024-02-02T13:24:36.286535+0000","last_active":"2024-02-02T13:24:36.771605+0000","last_peered":"2024-02-02T13:24:36.771605+0000","last_clean":"2024-02-02T13:24:36.771605+0000","last_became_active":"2024-02-02T13:24:36.286302+0000","last_became_peered":"2024-02-02T13:24:36.286302+0000","last_unstale":"2024-02-02T13:24:36.771605+0000","last_undegraded":"2024-02-02T13:24:36.771605+0000","last_fullsized":"2024-02-02T13:24:36.771605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:16:01.685074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-02-02T13:24:41.212029+0000","last_change":"2024-02-02T13:24:36.769601+0000","last_active":"2024-02-02T13:24:41.212029+0000","last_peered":"2024-02-02T13:24:41.212029+0000","last_clean":"2024-02-02T13:24:41.212029+0000","last_became_active":"2024-02-02T13:24:36.768381+0000","last_became_peered":"2024-02-02T13:24:36.768381+0000","last_unstale":"2024-02-02T13:24:41.212029+0000","last_undegraded":"2024-02-02T13:24:41.212029+0000","last_fullsized":"2024-02-02T13:24:41.212029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:36.243291+0000","last_clean_scrub_stamp":"2024-02-02T13:24:36.243291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:07.871595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-02-02T13:24:36.774155+0000","last_change":"2024-02-02T13:24:36.289368+0000","last_active":"2024-02-02T13:24:36.774155+0000","last_peered":"2024-02-02T13:24:36.774155+0000","last_clean":"2024-02-02T13:24:36.774155+0000","last_became_active":"2024-02-02T13:24:36.289118+0000","last_became_peered":"2024-02-02T13:24:36.289118+0000","last_unstale":"2024-02-02T13:24:36.774155+0000","last_undegraded":"2024-02-02T13:24:36.774155+0000","last_fullsized":"2024-02-02T13:24:36.774155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:35.233228+0000","last_clean_scrub_stamp":"2024-02-02T13:24:35.233228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:23.128595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.823}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7609999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928717,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5840000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6110000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.988}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8490000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.8410000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2050000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.651}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9210000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.74}]}]}],"pool_statfs":[]}} 2024-02-02T13:24:52.454 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-02T13:24:52.454 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-02T13:24:52.455 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-02-02T13:24:52.455 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-02-02T13:24:52.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.693+0000 7f485f200640 1 Processor -- start 2024-02-02T13:24:52.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.693+0000 7f485f200640 1 -- start start 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.694+0000 7f485f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f4860070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48600cdbc0 0x7f48600ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600d1bf0 con 0x7f48600d2ad0 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485f200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600cd730 con 0x7f48600cdbc0 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600d1e00 con 0x7f4860070270 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48600cdbc0 0x7f48600ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48600cdbc0 0x7f48600ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54494/0 (socket says 172.21.15.91:54494) 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 -- 172.21.15.91:0/3443549250 learned_addr learned my addr 172.21.15.91:0/3443549250 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:52.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 -- 172.21.15.91:0/3443549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 msgr2=0x7f4860070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:52.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 --2- 172.21.15.91:0/3443549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f4860070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 -- 172.21.15.91:0/3443549250 >> v1:172.21.15.113:6789/0 conn(0x7f48600d2ad0 legacy=0x7f48600b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:52.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.695+0000 7f485e800640 1 -- 172.21.15.91:0/3443549250 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48600b6880 con 0x7f48600cdbc0 2024-02-02T13:24:52.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.696+0000 7f485e800640 1 --2- 172.21.15.91:0/3443549250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f48600ce070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4858009870 tx=0x7f485802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7324c20097e813ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485ca00640 1 -- 172.21.15.91:0/3443549250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858031640 con 0x7f48600cdbc0 2024-02-02T13:24:52.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485ca00640 1 -- 172.21.15.91:0/3443549250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48580317a0 con 0x7f48600cdbc0 2024-02-02T13:24:52.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485ca00640 1 -- 172.21.15.91:0/3443549250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858031b30 con 0x7f48600cdbc0 2024-02-02T13:24:52.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 -- 172.21.15.91:0/3443549250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 msgr2=0x7f48600ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 --2- 172.21.15.91:0/3443549250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f48600ce070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4858009870 tx=0x7f485802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 -- 172.21.15.91:0/3443549250 shutdown_connections 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 --2- 172.21.15.91:0/3443549250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f48600ce070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 --2- 172.21.15.91:0/3443549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f4860070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 -- 172.21.15.91:0/3443549250 >> 172.21.15.91:0/3443549250 conn(0x7f48600cfa30 msgr2=0x7f48600cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 -- 172.21.15.91:0/3443549250 shutdown_connections 2024-02-02T13:24:52.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.697+0000 7f485f200640 1 -- 172.21.15.91:0/3443549250 wait complete. 2024-02-02T13:24:52.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.699+0000 7f485f200640 1 Processor -- start 2024-02-02T13:24:52.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.699+0000 7f485f200640 1 -- start start 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f48600ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48600d2ad0 0x7f48600a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600ab810 con 0x7f48600cdbc0 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600aa230 con 0x7f48600d2ad0 2024-02-02T13:24:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600aa510 con 0x7f4860070270 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54500/0 (socket says 172.21.15.91:54500) 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485d400640 1 -- 172.21.15.91:0/588193754 learned_addr learned my addr 172.21.15.91:0/588193754 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485e800640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48600d2ad0 0x7f48600a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485de00640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f48600ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.700+0000 7f485d400640 1 -- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 msgr2=0x7f48600ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485d400640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f48600ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485d400640 1 -- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48600d2ad0 msgr2=0x7f48600a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485d400640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48600d2ad0 0x7f48600a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485d400640 1 -- 172.21.15.91:0/588193754 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580094d0 con 0x7f48600cdbc0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485d400640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f484800b3c0 tx=0x7f484800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011830 con 0x7f48600cdbc0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4848011990 con 0x7f48600cdbc0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011d20 con 0x7f48600cdbc0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48600aa7f0 con 0x7f48600cdbc0 2024-02-02T13:24:52.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.701+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600aadb0 con 0x7f48600cdbc0 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.702+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600aafd0 con 0x7f48600cdbc0 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.703+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f484801b460 con 0x7f48600cdbc0 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.704+0000 7f4856000640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 0x7f483806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.704+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4848088600 con 0x7f48600cdbc0 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.705+0000 7f485de00640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 0x7f483806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.705+0000 7f485de00640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 0x7f483806c8f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4850007dc0 tx=0x7f48500072f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:52.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.709+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4848053c80 con 0x7f48600cdbc0 2024-02-02T13:24:52.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.934+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f486004dff0 con 0x7f48600cdbc0 2024-02-02T13:24:52.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.935+0000 7f4856000640 1 -- 172.21.15.91:0/588193754 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f48480572a0 con 0x7f48600cdbc0 2024-02-02T13:24:52.937 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:52.937 INFO:teuthology.orchestra.run.smithi091.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.937+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 msgr2=0x7f483806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.937+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 0x7f483806c8f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4850007dc0 tx=0x7f48500072f0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.937+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 msgr2=0x7f486014e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.937+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f484800b3c0 tx=0x7f484800b880 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 shutdown_connections 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f483806a430 0x7f483806c8f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48600d2ad0 0x7f48600a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48600cdbc0 0x7f486014e2d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 --2- 172.21.15.91:0/588193754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4860070270 0x7f48600ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 >> 172.21.15.91:0/588193754 conn(0x7f48600cfa30 msgr2=0x7f48600ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 shutdown_connections 2024-02-02T13:24:52.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:52.938+0000 7f485f200640 1 -- 172.21.15.91:0/588193754 wait complete. 2024-02-02T13:24:52.956 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-02-02T13:24:52.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:53.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.158+0000 7ff285e00640 1 Processor -- start 2024-02-02T13:24:53.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.158+0000 7ff285e00640 1 -- start start 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff285e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff280151f90 0x7ff28015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff285e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff28006f4f0 0x7ff2800b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff285e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800b1070 con 0x7ff2800b0c40 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff285e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff280070f80 con 0x7ff280151f90 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff285e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff280152370 con 0x7ff28006f4f0 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff27ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff280151f90 0x7ff28015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff27ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff280151f90 0x7ff28015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54506/0 (socket says 172.21.15.91:54506) 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.160+0000 7ff27ea00640 1 -- 172.21.15.91:0/3059148547 learned_addr learned my addr 172.21.15.91:0/3059148547 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.161+0000 7ff275e00640 1 --2- 172.21.15.91:0/3059148547 >> v2:172.21.15.113:3300/0 conn(0x7ff28006f4f0 0x7ff2800b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.161+0000 7ff27ea00640 1 -- 172.21.15.91:0/3059148547 >> v2:172.21.15.113:3300/0 conn(0x7ff28006f4f0 msgr2=0x7ff2800b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.161+0000 7ff27ea00640 1 --2- 172.21.15.91:0/3059148547 >> v2:172.21.15.113:3300/0 conn(0x7ff28006f4f0 0x7ff2800b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.161+0000 7ff27ea00640 1 -- 172.21.15.91:0/3059148547 >> v1:172.21.15.113:6789/0 conn(0x7ff2800b0c40 legacy=0x7ff2800b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.161+0000 7ff27ea00640 1 -- 172.21.15.91:0/3059148547 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800cb010 con 0x7ff280151f90 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.162+0000 7ff27ea00640 1 --2- 172.21.15.91:0/3059148547 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff280151f90 0x7ff28015f5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff280071a00 tx=0x7ff27002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840248f8540252a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.162+0000 7ff27e000640 1 -- 172.21.15.91:0/3059148547 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270031870 con 0x7ff280151f90 2024-02-02T13:24:53.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.162+0000 7ff27e000640 1 -- 172.21.15.91:0/3059148547 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2700319d0 con 0x7ff280151f90 2024-02-02T13:24:53.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.162+0000 7ff27e000640 1 -- 172.21.15.91:0/3059148547 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270031d60 con 0x7ff280151f90 2024-02-02T13:24:53.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 -- 172.21.15.91:0/3059148547 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff280151f90 msgr2=0x7ff28015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 --2- 172.21.15.91:0/3059148547 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff280151f90 0x7ff28015f5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff280071a00 tx=0x7ff27002b910 comp rx=0 tx=0).stop 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 -- 172.21.15.91:0/3059148547 shutdown_connections 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 --2- 172.21.15.91:0/3059148547 >> v2:172.21.15.113:3300/0 conn(0x7ff28006f4f0 0x7ff2800b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 --2- 172.21.15.91:0/3059148547 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff280151f90 0x7ff28015f5a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 -- 172.21.15.91:0/3059148547 >> 172.21.15.91:0/3059148547 conn(0x7ff2800cec80 msgr2=0x7ff2800cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 -- 172.21.15.91:0/3059148547 shutdown_connections 2024-02-02T13:24:53.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.163+0000 7ff285e00640 1 -- 172.21.15.91:0/3059148547 wait complete. 2024-02-02T13:24:53.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.165+0000 7ff285e00640 1 Processor -- start 2024-02-02T13:24:53.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.165+0000 7ff285e00640 1 -- start start 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2800b0c40 0x7ff2800c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff280151f90 0x7ff28015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28014e200 con 0x7ff28006f4f0 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c6510 con 0x7ff280151f90 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff285e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c67f0 con 0x7ff2800b0c40 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54510/0 (socket says 172.21.15.91:54510) 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 -- 172.21.15.91:0/2500364435 learned_addr learned my addr 172.21.15.91:0/2500364435 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:53.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 -- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2800b0c40 msgr2=0x7ff2800c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2800b0c40 0x7ff2800c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 -- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff280151f90 msgr2=0x7ff28015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff280151f90 0x7ff28015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.166+0000 7ff27ea00640 1 -- 172.21.15.91:0/2500364435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700094d0 con 0x7ff28006f4f0 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff27ea00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff270000c00 tx=0x7ff2700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270031840 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2700443f0 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700447c0 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2800c6ad0 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.167+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2800c6200 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.168+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2801524b0 con 0x7ff28006f4f0 2024-02-02T13:24:53.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.169+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff27004d450 con 0x7ff28006f4f0 2024-02-02T13:24:53.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.170+0000 7ff277e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 0x7ff25806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.170+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff27002d040 con 0x7ff28006f4f0 2024-02-02T13:24:53.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.170+0000 7ff275e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 0x7ff25806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.170+0000 7ff275e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 0x7ff25806c8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2800c8cc0 tx=0x7ff268008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.174+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff270076710 con 0x7ff28006f4f0 2024-02-02T13:24:53.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.365+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff280066cf0 con 0x7ff28006f4f0 2024-02-02T13:24:53.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.366+0000 7ff277e00640 1 -- 172.21.15.91:0/2500364435 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7ff270079d30 con 0x7ff28006f4f0 2024-02-02T13:24:53.368 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:53.368 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 msgr2=0x7ff25806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 0x7ff25806c8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2800c8cc0 tx=0x7ff268008040 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 msgr2=0x7ff28014dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff270000c00 tx=0x7ff2700046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 shutdown_connections 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff25806a430 0x7ff25806c8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2800b0c40 0x7ff2800c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff280151f90 0x7ff28015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 --2- 172.21.15.91:0/2500364435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff28006f4f0 0x7ff28014dcc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 >> 172.21.15.91:0/2500364435 conn(0x7ff2800cec80 msgr2=0x7ff2800acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 shutdown_connections 2024-02-02T13:24:53.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.369+0000 7ff285e00640 1 -- 172.21.15.91:0/2500364435 wait complete. 2024-02-02T13:24:53.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:24:53.390 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:24:53.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.591+0000 7f9c2b800640 1 Processor -- start 2024-02-02T13:24:53.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.591+0000 7f9c2b800640 1 -- start start 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.593+0000 7f9c2b800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c2415f5a0 0x7f9c240b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.593+0000 7f9c2b800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240b1070 con 0x7f9c24151f90 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.593+0000 7f9c2b800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2406c600 con 0x7f9c240b0c40 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.593+0000 7f9c2b800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2415fa30 con 0x7f9c2415f5a0 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c28e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9c24151f90 0x7f9c2406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50196/0 (socket says 172.21.15.91:50196) 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c28e00640 1 -- 172.21.15.91:0/4288147468 learned_addr learned my addr 172.21.15.91:0/4288147468 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:53.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4288147468 >> v2:172.21.15.91:3300/0 conn(0x7f9c2415f5a0 0x7f9c240b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c23e00640 1 -- 172.21.15.91:0/4288147468 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988545073 0 0) 0x7f9c240b1070 con 0x7f9c24151f90 2024-02-02T13:24:53.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c23e00640 1 -- 172.21.15.91:0/4288147468 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c04003520 con 0x7f9c24151f90 2024-02-02T13:24:53.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c1bc00640 1 -- 172.21.15.91:0/4288147468 >> v1:172.21.15.113:6789/0 conn(0x7f9c240b0c40 legacy=0x7f9c240b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c1bc00640 1 -- 172.21.15.91:0/4288147468 >> v1:172.21.15.91:6789/0 conn(0x7f9c24151f90 legacy=0x7f9c2406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.594+0000 7f9c1bc00640 1 -- 172.21.15.91:0/4288147468 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c24070f80 con 0x7f9c2415f5a0 2024-02-02T13:24:53.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.596+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4288147468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415f5a0 0x7f9c240b0700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097a0 tx=0x7f9c0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94edb877b21f1eff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.596+0000 7f9c23e00640 1 -- 172.21.15.91:0/4288147468 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c030610 con 0x7f9c2415f5a0 2024-02-02T13:24:53.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.596+0000 7f9c23e00640 1 -- 172.21.15.91:0/4288147468 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c030770 con 0x7f9c2415f5a0 2024-02-02T13:24:53.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.596+0000 7f9c23e00640 1 -- 172.21.15.91:0/4288147468 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c030b00 con 0x7f9c2415f5a0 2024-02-02T13:24:53.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 -- 172.21.15.91:0/4288147468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415f5a0 msgr2=0x7f9c240b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4288147468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415f5a0 0x7f9c240b0700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097a0 tx=0x7f9c0c02b960 comp rx=0 tx=0).stop 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 -- 172.21.15.91:0/4288147468 shutdown_connections 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4288147468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415f5a0 0x7f9c240b0700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 -- 172.21.15.91:0/4288147468 >> 172.21.15.91:0/4288147468 conn(0x7f9c240cec80 msgr2=0x7f9c240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 -- 172.21.15.91:0/4288147468 shutdown_connections 2024-02-02T13:24:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.597+0000 7f9c2b800640 1 -- 172.21.15.91:0/4288147468 wait complete. 2024-02-02T13:24:53.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.598+0000 7f9c2b800640 1 Processor -- start 2024-02-02T13:24:53.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 -- start start 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 0x7f9c240c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415f5a0 0x7f9c24152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240c2720 con 0x7f9c240b0c40 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.599+0000 7f9c2b800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240b4130 con 0x7f9c24151f90 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c2b800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240c29e0 con 0x7f9c2415f5a0 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54520/0 (socket says 172.21.15.91:54520) 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 -- 172.21.15.91:0/4104141842 learned_addr learned my addr 172.21.15.91:0/4104141842 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 0x7f9c240c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 -- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415f5a0 msgr2=0x7f9c24152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415f5a0 0x7f9c24152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 -- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 msgr2=0x7f9c240c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 0x7f9c240c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 -- 172.21.15.91:0/4104141842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009400 con 0x7f9c240b0c40 2024-02-02T13:24:53.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 0x7f9c240c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:53.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.600+0000 7f9c28e00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c410 tx=0x7f9c1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.601+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14005ce0 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.601+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c14005e40 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.601+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c240c2cc0 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.601+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c140072f0 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.601+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c240c8a10 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.602+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c240c31e0 con 0x7f9c240b0c40 2024-02-02T13:24:53.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.603+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c14007510 con 0x7f9c240b0c40 2024-02-02T13:24:53.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.604+0000 7f9c21600640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:53.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.604+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c14087f80 con 0x7f9c240b0c40 2024-02-02T13:24:53.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.604+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:53.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.605+0000 7f9c1bc00640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c036000 tx=0x7f9c0c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.608+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c140baca0 con 0x7f9c240b0c40 2024-02-02T13:24:53.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.797+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c24073830 con 0x7f9c240b0c40 2024-02-02T13:24:53.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.798+0000 7f9c21600640 1 -- 172.21.15.91:0/4104141842 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c14017020 con 0x7f9c240b0c40 2024-02-02T13:24:53.800 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 msgr2=0x7f9bfc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c036000 tx=0x7f9c0c03b040 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 msgr2=0x7f9c24154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c410 tx=0x7f9c1400c8d0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 shutdown_connections 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415f5a0 0x7f9c24152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c24151f90 0x7f9c240c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 --2- 172.21.15.91:0/4104141842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b0c40 0x7f9c24154f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 >> 172.21.15.91:0/4104141842 conn(0x7f9c240cec80 msgr2=0x7f9c240af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 shutdown_connections 2024-02-02T13:24:53.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:53.801+0000 7f9c2b800640 1 -- 172.21.15.91:0/4104141842 wait complete. 2024-02-02T13:24:53.819 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:24:54.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.020+0000 7fb95a600640 1 Processor -- start 2024-02-02T13:24:54.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.020+0000 7fb95a600640 1 -- start start 2024-02-02T13:24:54.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb95a600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb95415d3a0 0x7fb95406b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb95a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9540c9450 0x7fb9540c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb95a600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9540ce3b0 con 0x7fb9540add80 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb95a600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540adbd0 con 0x7fb95415d3a0 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb95a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540c59d0 con 0x7fb9540c9450 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb959200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb95415d3a0 0x7fb95406b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.021+0000 7fb959200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb95415d3a0 0x7fb95406b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54530/0 (socket says 172.21.15.91:54530) 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959200640 1 -- 172.21.15.91:0/3183277946 learned_addr learned my addr 172.21.15.91:0/3183277946 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959c00640 1 --2- 172.21.15.91:0/3183277946 >> v2:172.21.15.113:3300/0 conn(0x7fb9540c9450 0x7fb9540c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959200640 1 -- 172.21.15.91:0/3183277946 >> v2:172.21.15.113:3300/0 conn(0x7fb9540c9450 msgr2=0x7fb9540c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959200640 1 --2- 172.21.15.91:0/3183277946 >> v2:172.21.15.113:3300/0 conn(0x7fb9540c9450 0x7fb9540c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959200640 1 -- 172.21.15.91:0/3183277946 >> v1:172.21.15.113:6789/0 conn(0x7fb9540add80 legacy=0x7fb9540706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.022+0000 7fb959200640 1 -- 172.21.15.91:0/3183277946 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540ce8e0 con 0x7fb95415d3a0 2024-02-02T13:24:54.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.023+0000 7fb959200640 1 --2- 172.21.15.91:0/3183277946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb95415d3a0 0x7fb95406b450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f700 tx=0x7fb94402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341969aa108fac0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb953400640 1 -- 172.21.15.91:0/3183277946 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944031750 con 0x7fb95415d3a0 2024-02-02T13:24:54.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb953400640 1 -- 172.21.15.91:0/3183277946 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9440318b0 con 0x7fb95415d3a0 2024-02-02T13:24:54.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb953400640 1 -- 172.21.15.91:0/3183277946 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944031c40 con 0x7fb95415d3a0 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 -- 172.21.15.91:0/3183277946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb95415d3a0 msgr2=0x7fb95406b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 --2- 172.21.15.91:0/3183277946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb95415d3a0 0x7fb95406b450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb95406f700 tx=0x7fb94402b7f0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 -- 172.21.15.91:0/3183277946 shutdown_connections 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 --2- 172.21.15.91:0/3183277946 >> v2:172.21.15.113:3300/0 conn(0x7fb9540c9450 0x7fb9540c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 --2- 172.21.15.91:0/3183277946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb95415d3a0 0x7fb95406b450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 -- 172.21.15.91:0/3183277946 >> 172.21.15.91:0/3183277946 conn(0x7fb9540cea90 msgr2=0x7fb9540ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:54.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 -- 172.21.15.91:0/3183277946 shutdown_connections 2024-02-02T13:24:54.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.024+0000 7fb95a600640 1 -- 172.21.15.91:0/3183277946 wait complete. 2024-02-02T13:24:54.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.026+0000 7fb95a600640 1 Processor -- start 2024-02-02T13:24:54.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.026+0000 7fb95a600640 1 -- start start 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9540add80 0x7fb95414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 0x7fb9540aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540c7500 con 0x7fb9540c9450 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540c6ed0 con 0x7fb95415d3a0 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb95a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540b13a0 con 0x7fb9540add80 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb953e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb953e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54532/0 (socket says 172.21.15.91:54532) 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.027+0000 7fb953e00640 1 -- 172.21.15.91:0/3103541389 learned_addr learned my addr 172.21.15.91:0/3103541389 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:54.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb959c00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 0x7fb9540aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb959200640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9540add80 0x7fb95414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 -- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9540add80 msgr2=0x7fb95414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9540add80 0x7fb95414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 -- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 msgr2=0x7fb9540aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 0x7fb9540aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 -- 172.21.15.91:0/3103541389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440094d0 con 0x7fb9540c9450 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb959c00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 0x7fb9540aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb953e00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c3f0 tx=0x7fb93c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c016070 con 0x7fb9540c9450 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c007020 con 0x7fb9540c9450 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0073b0 con 0x7fb9540c9450 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9540af620 con 0x7fb9540c9450 2024-02-02T13:24:54.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.028+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9540c7120 con 0x7fb9540c9450 2024-02-02T13:24:54.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.030+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9540c7640 con 0x7fb9540c9450 2024-02-02T13:24:54.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.030+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb93c007510 con 0x7fb9540c9450 2024-02-02T13:24:54.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.031+0000 7fb950c00640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 0x7fb92c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.031+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb93c088120 con 0x7fb9540c9450 2024-02-02T13:24:54.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.031+0000 7fb959200640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 0x7fb92c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.032+0000 7fb959200640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 0x7fb92c06c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb944035000 tx=0x7fb944008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.036+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb9540c7640 con 0x7fb9540c9450 2024-02-02T13:24:54.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.208+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9540c7640 con 0x7fb9540c9450 2024-02-02T13:24:54.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.208+0000 7fb950c00640 1 -- 172.21.15.91:0/3103541389 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9540c7640 con 0x7fb9540c9450 2024-02-02T13:24:54.210 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.210+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 msgr2=0x7fb92c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.210+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 0x7fb92c06c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb944035000 tx=0x7fb944008030 comp rx=0 tx=0).stop 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.210+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 msgr2=0x7fb9540aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.210+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c3f0 tx=0x7fb93c00c8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 shutdown_connections 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb92c06a430 0x7fb92c06c8f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb95415d3a0 0x7fb9540aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9540c9450 0x7fb9540aea80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 --2- 172.21.15.91:0/3103541389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9540add80 0x7fb95414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 >> 172.21.15.91:0/3103541389 conn(0x7fb9540cea90 msgr2=0x7fb9540c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:54.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 shutdown_connections 2024-02-02T13:24:54.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.211+0000 7fb95a600640 1 -- 172.21.15.91:0/3103541389 wait complete. 2024-02-02T13:24:54.229 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:24:54.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.437+0000 7fe92b200640 1 Processor -- start 2024-02-02T13:24:54.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.437+0000 7fe92b200640 1 -- start start 2024-02-02T13:24:54.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe92b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe92c0b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe92b200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0d25f0 con 0x7fe92c0d2a80 2024-02-02T13:24:54.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe92b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0d19e0 con 0x7fe92c15d110 2024-02-02T13:24:54.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe92b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0d1cc0 con 0x7fe92c0cff00 2024-02-02T13:24:54.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe921200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe92c0d2a80 0x7fe92c0b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50212/0 (socket says 172.21.15.91:50212) 2024-02-02T13:24:54.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe921200640 1 -- 172.21.15.91:0/534698036 learned_addr learned my addr 172.21.15.91:0/534698036 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe92a800640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe92c0b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.439+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973529126 0 0) 0x7fe92c0d25f0 con 0x7fe92c0d2a80 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c003520 con 0x7fe92c0d2a80 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3159567307 0 0) 0x7fe90c003520 con 0x7fe92c0d2a80 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 msgr2=0x7fe92c0b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe92c0b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 >> v1:172.21.15.113:6789/0 conn(0x7fe92c15d110 legacy=0x7fe92c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0d1fa0 con 0x7fe92c0d2a80 2024-02-02T13:24:54.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe914001670 con 0x7fe92c0d2a80 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 >> v1:172.21.15.91:6789/0 conn(0x7fe92c0d2a80 legacy=0x7fe92c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe90c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c003f70 0x7fe90c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe90c004d00 0x7fe90c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0d1cc0 con 0x7fe90c004d00 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00bb50 con 0x7fe90c003f70 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00bd10 con 0x7fe92c0cff00 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.440+0000 7fe92a800640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe90c004d00 0x7fe90c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe921200640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c003f70 0x7fe90c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe929e00640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe90c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe92a800640 1 -- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 msgr2=0x7fe90c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe92a800640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c0cff00 0x7fe90c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe92a800640 1 -- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c003f70 msgr2=0x7fe90c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe92a800640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c003f70 0x7fe90c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.441+0000 7fe92a800640 1 -- 172.21.15.91:0/534698036 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c003520 con 0x7fe90c004d00 2024-02-02T13:24:54.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.442+0000 7fe92a800640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe90c004d00 0x7fe90c005180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f90 tx=0x7fe90c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e365a4923e9c0397 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.442+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92402cbd0 con 0x7fe90c004d00 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.442+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92400f040 con 0x7fe90c004d00 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.442+0000 7fe929400640 1 -- 172.21.15.91:0/534698036 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924034440 con 0x7fe90c004d00 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.442+0000 7fe92b200640 1 -- 172.21.15.91:0/534698036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe90c004d00 msgr2=0x7fe90c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92b200640 1 --2- 172.21.15.91:0/534698036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe90c004d00 0x7fe90c005180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f90 tx=0x7fe90c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92a800640 1 -- 172.21.15.91:0/534698036 reap_dead start 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92b200640 1 -- 172.21.15.91:0/534698036 shutdown_connections 2024-02-02T13:24:54.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92b200640 1 -- 172.21.15.91:0/534698036 >> 172.21.15.91:0/534698036 conn(0x7fe92c070690 msgr2=0x7fe92c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92b200640 1 -- 172.21.15.91:0/534698036 shutdown_connections 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.443+0000 7fe92b200640 1 -- 172.21.15.91:0/534698036 wait complete. 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.444+0000 7fe92b200640 1 Processor -- start 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.444+0000 7fe92b200640 1 -- start start 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c004d00 0x7fe92c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c1570b0 0x7fe92c0cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c8030 con 0x7fe92c157510 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0ccaa0 con 0x7fe90c004d00 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.445+0000 7fe92b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c15a620 con 0x7fe92c1570b0 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54558/0 (socket says 172.21.15.91:54558) 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 -- 172.21.15.91:0/2031438557 learned_addr learned my addr 172.21.15.91:0/2031438557 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 -- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c1570b0 msgr2=0x7fe92c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c1570b0 0x7fe92c0cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 -- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c004d00 msgr2=0x7fe92c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c004d00 0x7fe92c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 -- 172.21.15.91:0/2031438557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c15d670 con 0x7fe92c157510 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe92a800640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f90 tx=0x7fe924009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003b90 con 0x7fe92c157510 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924003cf0 con 0x7fe92c157510 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.446+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924034b60 con 0x7fe92c157510 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.447+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0c73b0 con 0x7fe92c157510 2024-02-02T13:24:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.447+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0c7970 con 0x7fe92c157510 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.448+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0c8170 con 0x7fe92c157510 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.449+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe924034cc0 con 0x7fe92c157510 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.449+0000 7fe922a00640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 0x7fe90406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.449+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9240aa1a0 con 0x7fe92c157510 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.450+0000 7fe929e00640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 0x7fe90406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.450+0000 7fe929e00640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 0x7fe90406c8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.454+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe92c0c8170 con 0x7fe92c157510 2024-02-02T13:24:54.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.646+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe92c0c8170 con 0x7fe92c157510 2024-02-02T13:24:54.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.647+0000 7fe922a00640 1 -- 172.21.15.91:0/2031438557 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe92c0c8170 con 0x7fe92c157510 2024-02-02T13:24:54.649 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 msgr2=0x7fe90406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 0x7fe90406c8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 msgr2=0x7fe92c0cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe924009f90 tx=0x7fe924009750 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 shutdown_connections 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe90406a430 0x7fe90406c8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c1570b0 0x7fe92c0cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe92c157510 0x7fe92c0cbc10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 --2- 172.21.15.91:0/2031438557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe90c004d00 0x7fe92c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 >> 172.21.15.91:0/2031438557 conn(0x7fe92c070690 msgr2=0x7fe92c15aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 shutdown_connections 2024-02-02T13:24:54.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.650+0000 7fe92b200640 1 -- 172.21.15.91:0/2031438557 wait complete. 2024-02-02T13:24:54.669 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:24:54.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.870+0000 7f7dd7600640 1 Processor -- start 2024-02-02T13:24:54.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.870+0000 7f7dd7600640 1 -- start start 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd7600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7dd00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd7600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7dd00b0700 0x7f7dd00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd7600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd00cc7c0 con 0x7f7dd015a010 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd7600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00b10d0 con 0x7f7dd00b0700 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd7600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00ce930 con 0x7f7dd00ce130 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd4c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7dd015a010 0x7f7dd00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50222/0 (socket says 172.21.15.91:50222) 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.872+0000 7f7dd4c00640 1 -- 172.21.15.91:0/1599469617 learned_addr learned my addr 172.21.15.91:0/1599469617 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dd5600640 1 --2- 172.21.15.91:0/1599469617 >> v2:172.21.15.113:3300/0 conn(0x7f7dd00b0700 0x7f7dd00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7dd00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985250212 0 0) 0x7f7dd00cc7c0 con 0x7f7dd015a010 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db0003520 con 0x7f7dd015a010 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1332440656 0 0) 0x7f7db0003520 con 0x7f7dd015a010 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 msgr2=0x7f7dd00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7dd00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 >> v2:172.21.15.113:3300/0 conn(0x7f7dd00b0700 msgr2=0x7f7dd00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 --2- 172.21.15.91:0/1599469617 >> v2:172.21.15.113:3300/0 conn(0x7f7dd00b0700 0x7f7dd00b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00cd640 con 0x7f7dd015a010 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f7dc0001670 con 0x7f7dd015a010 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 >> v1:172.21.15.91:6789/0 conn(0x7f7dd015a010 legacy=0x7f7dd00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0003970 0x7f7db0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7db00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7db0004d00 0x7f7db0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b10d0 con 0x7f7db0004d00 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00ce930 con 0x7f7db0003970 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.873+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db000b980 con 0x7f7dd00ce130 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7db0004d00 0x7f7db0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd4c00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0003970 0x7f7db0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7db00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 -- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 msgr2=0x7f7db00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7db00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 -- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0003970 msgr2=0x7f7db0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0003970 0x7f7db0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd5600640 1 -- 172.21.15.91:0/1599469617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0003520 con 0x7f7db0004d00 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00ce130 0x7f7db00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:54.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.874+0000 7f7dd4c00640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0003970 0x7f7db0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.875+0000 7f7dd5600640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7db0004d00 0x7f7db0005180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009fc0 tx=0x7f7db00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccd70ca541d9c9a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.875+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc802cc60 con 0x7f7db0004d00 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.875+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc800f040 con 0x7f7db0004d00 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.875+0000 7f7dcfe00640 1 -- 172.21.15.91:0/1599469617 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8034440 con 0x7f7db0004d00 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 -- 172.21.15.91:0/1599469617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7db0004d00 msgr2=0x7f7db0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 --2- 172.21.15.91:0/1599469617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7db0004d00 0x7f7db0005180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009fc0 tx=0x7f7db00057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd5600640 1 -- 172.21.15.91:0/1599469617 reap_dead start 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 -- 172.21.15.91:0/1599469617 shutdown_connections 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 -- 172.21.15.91:0/1599469617 >> 172.21.15.91:0/1599469617 conn(0x7f7dd00cec80 msgr2=0x7f7dd00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:54.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 -- 172.21.15.91:0/1599469617 shutdown_connections 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.876+0000 7f7dd7600640 1 -- 172.21.15.91:0/1599469617 wait complete. 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.878+0000 7f7dd7600640 1 Processor -- start 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.878+0000 7f7dd7600640 1 -- start start 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0004d00 0x7f7dd00c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 0x7f7dd00c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0159500 con 0x7f7dd0158570 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c4ea0 con 0x7f7db0004d00 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dd7600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c5180 con 0x7f7dd00c4190 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dc7c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dc7c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54582/0 (socket says 172.21.15.91:54582) 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.879+0000 7f7dc7c00640 1 -- 172.21.15.91:0/3685032309 learned_addr learned my addr 172.21.15.91:0/3685032309 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dd4c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0004d00 0x7f7dd00c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dd5600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 0x7f7dd00c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 -- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 msgr2=0x7f7dd00c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 0x7f7dd00c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 -- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0004d00 msgr2=0x7f7dd00c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0004d00 0x7f7dd00c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 -- 172.21.15.91:0/3685032309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00c5460 con 0x7f7dd0158570 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dd5600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 0x7f7dd00c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dc7c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7db80108b0 tx=0x7f7db8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db800eb60 con 0x7f7dd0158570 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800ecc0 con 0x7f7dd0158570 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db801c400 con 0x7f7dd0158570 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd00c5740 con 0x7f7dd0158570 2024-02-02T13:24:54.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.880+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd00c5a20 con 0x7f7dd0158570 2024-02-02T13:24:54.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.881+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0066cf0 con 0x7f7dd0158570 2024-02-02T13:24:54.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.882+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7db801c560 con 0x7f7dd0158570 2024-02-02T13:24:54.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.883+0000 7f7dcd600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 0x7f7da806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:54.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.883+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7db8088ff0 con 0x7f7dd0158570 2024-02-02T13:24:54.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.883+0000 7f7dd4c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 0x7f7da806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:54.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.884+0000 7f7dd4c00640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 0x7f7da806c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001eb0 tx=0x7f7dc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:54.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:54.887+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7db80bbca0 con 0x7f7dd0158570 2024-02-02T13:24:55.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.098+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dd00b3b50 con 0x7f7dd0158570 2024-02-02T13:24:55.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.099+0000 7f7dcd600640 1 -- 172.21.15.91:0/3685032309 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f7db8054630 con 0x7f7dd0158570 2024-02-02T13:24:55.101 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:55.101 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:24:55.101 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:24:55.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.101+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 msgr2=0x7f7da806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.101+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 0x7f7da806c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001eb0 tx=0x7f7dc0009040 comp rx=0 tx=0).stop 2024-02-02T13:24:55.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.101+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 msgr2=0x7f7dd01589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.101+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7db80108b0 tx=0x7f7db8010d70 comp rx=0 tx=0).stop 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 shutdown_connections 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7da806a430 0x7f7da806c8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dd00c4190 0x7f7dd00c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7dd0158570 0x7f7dd01589f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 --2- 172.21.15.91:0/3685032309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7db0004d00 0x7f7dd00c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 >> 172.21.15.91:0/3685032309 conn(0x7f7dd00cec80 msgr2=0x7f7dd015b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 shutdown_connections 2024-02-02T13:24:55.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.102+0000 7f7dd7600640 1 -- 172.21.15.91:0/3685032309 wait complete. 2024-02-02T13:24:55.121 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:55.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.327+0000 7f5303200640 1 Processor -- start 2024-02-02T13:24:55.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.327+0000 7f5303200640 1 -- start start 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.328+0000 7f5303200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 0x7f5304159b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.328+0000 7f5303200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 0x7f53040ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.328+0000 7f5303200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040ae2e0 con 0x7f53040c9170 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.328+0000 7f5303200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530415a4b0 con 0x7f530406c3f0 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.328+0000 7f5303200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530415a790 con 0x7f53040adf00 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5301400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f53040c9170 0x7f530406bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50230/0 (socket says 172.21.15.91:50230) 2024-02-02T13:24:55.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5301400640 1 -- 172.21.15.91:0/1725586814 learned_addr learned my addr 172.21.15.91:0/1725586814 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:55.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5301e00640 1 --2- 172.21.15.91:0/1725586814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 0x7f5304159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5302800640 1 --2- 172.21.15.91:0/1725586814 >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 0x7f53040ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5300a00640 1 -- 172.21.15.91:0/1725586814 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995463120 0 0) 0x7f53040ae2e0 con 0x7f53040c9170 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.329+0000 7f5300a00640 1 -- 172.21.15.91:0/1725586814 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e4003440 con 0x7f53040c9170 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.330+0000 7f5301e00640 1 -- 172.21.15.91:0/1725586814 >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 msgr2=0x7f53040ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.330+0000 7f5301e00640 1 --2- 172.21.15.91:0/1725586814 >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 0x7f53040ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.330+0000 7f5301e00640 1 -- 172.21.15.91:0/1725586814 >> v1:172.21.15.91:6789/0 conn(0x7f53040c9170 legacy=0x7f530406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.330+0000 7f5301e00640 1 -- 172.21.15.91:0/1725586814 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530415aa70 con 0x7f53040adf00 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.330+0000 7f5302800640 1 --2- 172.21.15.91:0/1725586814 >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 0x7f53040ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:55.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.331+0000 7f5301e00640 1 --2- 172.21.15.91:0/1725586814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 0x7f5304159b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097a0 tx=0x7f52f402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8871da67b4689d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.335+0000 7f5300a00640 1 -- 172.21.15.91:0/1725586814 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f402d040 con 0x7f53040adf00 2024-02-02T13:24:55.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5300a00640 1 -- 172.21.15.91:0/1725586814 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4031810 con 0x7f53040adf00 2024-02-02T13:24:55.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5300a00640 1 -- 172.21.15.91:0/1725586814 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4031ba0 con 0x7f53040adf00 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 -- 172.21.15.91:0/1725586814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 msgr2=0x7f5304159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 --2- 172.21.15.91:0/1725586814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 0x7f5304159b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097a0 tx=0x7f52f402b8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 -- 172.21.15.91:0/1725586814 shutdown_connections 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 --2- 172.21.15.91:0/1725586814 >> v2:172.21.15.113:3300/0 conn(0x7f530406c3f0 0x7f53040ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 --2- 172.21.15.91:0/1725586814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040adf00 0x7f5304159b40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 -- 172.21.15.91:0/1725586814 >> 172.21.15.91:0/1725586814 conn(0x7f53040cea90 msgr2=0x7f53040ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:55.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 -- 172.21.15.91:0/1725586814 shutdown_connections 2024-02-02T13:24:55.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.336+0000 7f5303200640 1 -- 172.21.15.91:0/1725586814 wait complete. 2024-02-02T13:24:55.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.338+0000 7f5303200640 1 Processor -- start 2024-02-02T13:24:55.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.338+0000 7f5303200640 1 -- start start 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 0x7f53040c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040c9170 0x7f53040c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040c8c40 con 0x7f530406c3f0 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304159560 con 0x7f53040adf00 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5303200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b2680 con 0x7f53040c9170 2024-02-02T13:24:55.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5301e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5301e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54590/0 (socket says 172.21.15.91:54590) 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.339+0000 7f5301e00640 1 -- 172.21.15.91:0/121891665 learned_addr learned my addr 172.21.15.91:0/121891665 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301400640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 0x7f53040c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 -- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040c9170 msgr2=0x7f53040c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040c9170 0x7f53040c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 -- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 msgr2=0x7f53040c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 0x7f53040c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 -- 172.21.15.91:0/121891665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc009a70 con 0x7f530406c3f0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301400640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 0x7f53040c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5301e00640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000c00 tx=0x7f52f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f403ace0 con 0x7f530406c3f0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f403ae60 con 0x7f530406c3f0 2024-02-02T13:24:55.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40443f0 con 0x7f530406c3f0 2024-02-02T13:24:55.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f4009400 con 0x7f530406c3f0 2024-02-02T13:24:55.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.340+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53040b2c40 con 0x7f530406c3f0 2024-02-02T13:24:55.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.342+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530404dff0 con 0x7f530406c3f0 2024-02-02T13:24:55.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.342+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f52f404d450 con 0x7f530406c3f0 2024-02-02T13:24:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.344+0000 7f52fa000640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 0x7f52dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.344+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52f40aac00 con 0x7f530406c3f0 2024-02-02T13:24:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.344+0000 7f5301400640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 0x7f52dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.344+0000 7f5301400640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 0x7f52dc06c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f53040cdb60 tx=0x7f52ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.348+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f52f40760c0 con 0x7f530406c3f0 2024-02-02T13:24:55.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.522+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53040ad850 con 0x7f530406c3f0 2024-02-02T13:24:55.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.525+0000 7f52fa000640 1 -- 172.21.15.91:0/121891665 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f52f40796e0 con 0x7f530406c3f0 2024-02-02T13:24:55.527 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:55.527 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 msgr2=0x7f52dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 0x7f52dc06c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f53040cdb60 tx=0x7f52ec009040 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 msgr2=0x7f5304159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000c00 tx=0x7f52f40046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 shutdown_connections 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f52dc06a430 0x7f52dc06c8f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53040c9170 0x7f53040c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53040adf00 0x7f53040c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 --2- 172.21.15.91:0/121891665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f530406c3f0 0x7f5304159020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 >> 172.21.15.91:0/121891665 conn(0x7f53040cea90 msgr2=0x7f53040ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 shutdown_connections 2024-02-02T13:24:55.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.528+0000 7f5303200640 1 -- 172.21.15.91:0/121891665 wait complete. 2024-02-02T13:24:55.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:24:55.548 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:24:55.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.754+0000 7f1357e00640 1 Processor -- start 2024-02-02T13:24:55.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.755+0000 7f1357e00640 1 -- start start 2024-02-02T13:24:55.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.756+0000 7f1357e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f135815b2a0 0x7f13580cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.756+0000 7f1357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.756+0000 7f1357e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135806c1c0 con 0x7f135815aec0 2024-02-02T13:24:55.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.756+0000 7f1357e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135806b5f0 con 0x7f135815b2a0 2024-02-02T13:24:55.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.756+0000 7f1357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135806cd60 con 0x7f135806f3f0 2024-02-02T13:24:55.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.757+0000 7f1357400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.757+0000 7f134de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f135815b2a0 0x7f13580cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.757+0000 7f134de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f135815b2a0 0x7f13580cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:49744/0 (socket says 172.21.15.91:49744) 2024-02-02T13:24:55.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.757+0000 7f1357400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:39334/0 (socket says 172.21.15.91:39334) 2024-02-02T13:24:55.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.757+0000 7f134de00640 1 -- 172.21.15.91:0/4124419367 learned_addr learned my addr 172.21.15.91:0/4124419367 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:55.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.758+0000 7f134de00640 1 -- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 msgr2=0x7f13580ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:55.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.758+0000 7f134de00640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.758+0000 7f134de00640 1 -- 172.21.15.91:0/4124419367 >> v1:172.21.15.113:6789/0 conn(0x7f135815aec0 legacy=0x7f135806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.758+0000 7f134de00640 1 -- 172.21.15.91:0/4124419367 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135806b890 con 0x7f135815b2a0 2024-02-02T13:24:55.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.758+0000 7f1357400640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:24:55.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.759+0000 7f134de00640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13580cf5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1340009870 tx=0x7f134002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a8744656375a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.760+0000 7f1356000640 1 -- 172.21.15.91:0/4124419367 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134002f830 con 0x7f135815b2a0 2024-02-02T13:24:55.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.760+0000 7f1356000640 1 -- 172.21.15.91:0/4124419367 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134002f990 con 0x7f135815b2a0 2024-02-02T13:24:55.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.760+0000 7f1356000640 1 -- 172.21.15.91:0/4124419367 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134002fd20 con 0x7f135815b2a0 2024-02-02T13:24:55.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 -- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 msgr2=0x7f13580cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13580cf5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1340009870 tx=0x7f134002ba10 comp rx=0 tx=0).stop 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 -- 172.21.15.91:0/4124419367 shutdown_connections 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 --2- 172.21.15.91:0/4124419367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13580cf5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 -- 172.21.15.91:0/4124419367 >> 172.21.15.91:0/4124419367 conn(0x7f13580cea90 msgr2=0x7f13580ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 -- 172.21.15.91:0/4124419367 shutdown_connections 2024-02-02T13:24:55.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.761+0000 7f1357e00640 1 -- 172.21.15.91:0/4124419367 wait complete. 2024-02-02T13:24:55.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.763+0000 7f1357e00640 1 Processor -- start 2024-02-02T13:24:55.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.763+0000 7f1357e00640 1 -- start start 2024-02-02T13:24:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13581592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580b5150 con 0x7f135815aec0 2024-02-02T13:24:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580ad320 con 0x7f135815b2a0 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.764+0000 7f1357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580ae740 con 0x7f135806f3f0 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54604/0 (socket says 172.21.15.91:54604) 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 -- 172.21.15.91:0/275148232 learned_addr learned my addr 172.21.15.91:0/275148232 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f1356a00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f1357400640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13581592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 -- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 msgr2=0x7f13580b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 -- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 msgr2=0x7f13581592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13581592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.765+0000 7f134de00640 1 -- 172.21.15.91:0/275148232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1350009a70 con 0x7f135815aec0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f134de00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340037c40 con 0x7f135815aec0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340037da0 con 0x7f135815aec0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13400094d0 con 0x7f135815aec0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340041640 con 0x7f135815aec0 2024-02-02T13:24:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.766+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13580aed00 con 0x7f135815aec0 2024-02-02T13:24:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.767+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135804dff0 con 0x7f135815aec0 2024-02-02T13:24:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.768+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f13400417a0 con 0x7f135815aec0 2024-02-02T13:24:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.769+0000 7f134f400640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 0x7f133006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:55.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.769+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f134003c070 con 0x7f135815aec0 2024-02-02T13:24:55.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.769+0000 7f1356a00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 0x7f133006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:55.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.770+0000 7f1356a00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 0x7f133006c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1348000f60 tx=0x7f1348009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.774+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1340076ab0 con 0x7f135815aec0 2024-02-02T13:24:55.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.964+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13580b0c00 con 0x7f135815aec0 2024-02-02T13:24:55.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.965+0000 7f134f400640 1 -- 172.21.15.91:0/275148232 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f134007a0d0 con 0x7f135815aec0 2024-02-02T13:24:55.966 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 msgr2=0x7f133006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 0x7f133006c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1348000f60 tx=0x7f1348009040 comp rx=0 tx=0).stop 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 msgr2=0x7f13580acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004730 comp rx=0 tx=0).stop 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 shutdown_connections 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f133006a430 0x7f133006c8f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f135815b2a0 0x7f13581592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f135815aec0 0x7f13580acde0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 --2- 172.21.15.91:0/275148232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f135806f3f0 0x7f13580b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:55.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 >> 172.21.15.91:0/275148232 conn(0x7f13580cea90 msgr2=0x7f135815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:55.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 shutdown_connections 2024-02-02T13:24:55.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:55.967+0000 7f1357e00640 1 -- 172.21.15.91:0/275148232 wait complete. 2024-02-02T13:24:55.988 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:24:56.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.191+0000 7f4162800640 1 Processor -- start 2024-02-02T13:24:56.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.191+0000 7f4162800640 1 -- start start 2024-02-02T13:24:56.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f4162800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f415c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f4162800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f415c06f4f0 0x7f415c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f4162800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c070f80 con 0x7f415c0b0c40 2024-02-02T13:24:56.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f4162800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c0b10d0 con 0x7f415c06f4f0 2024-02-02T13:24:56.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f4162800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c152370 con 0x7f415c151f90 2024-02-02T13:24:56.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f415c0b0c40 0x7f415c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50234/0 (socket says 172.21.15.91:50234) 2024-02-02T13:24:56.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415be00640 1 -- 172.21.15.91:0/4068610745 learned_addr learned my addr 172.21.15.91:0/4068610745 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415aa00640 1 --2- 172.21.15.91:0/4068610745 >> v2:172.21.15.113:3300/0 conn(0x7f415c06f4f0 0x7f415c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415b400640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f415c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185122929 0 0) 0x7f415c070f80 con 0x7f415c0b0c40 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.193+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f413c003520 con 0x7f415c0b0c40 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4269528639 0 0) 0x7f413c003520 con 0x7f415c0b0c40 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 msgr2=0x7f415c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f415c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 >> v2:172.21.15.113:3300/0 conn(0x7f415c06f4f0 msgr2=0x7f415c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 --2- 172.21.15.91:0/4068610745 >> v2:172.21.15.113:3300/0 conn(0x7f415c06f4f0 0x7f415c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0cb010 con 0x7f415c0b0c40 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f41540019f0 con 0x7f415c0b0c40 2024-02-02T13:24:56.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 >> v1:172.21.15.91:6789/0 conn(0x7f415c0b0c40 legacy=0x7f415c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f413c003970 0x7f413c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f413c0047c0 0x7f413c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f413c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0b10d0 con 0x7f413c0047c0 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c152370 con 0x7f413c003970 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c00b980 con 0x7f415c151f90 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415aa00640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f413c0047c0 0x7f413c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.194+0000 7f415b400640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f413c003970 0x7f413c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415be00640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f413c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415aa00640 1 -- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 msgr2=0x7f413c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415aa00640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f415c151f90 0x7f413c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415aa00640 1 -- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f413c003970 msgr2=0x7f413c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415aa00640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f413c003970 0x7f413c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.195+0000 7f415aa00640 1 -- 172.21.15.91:0/4068610745 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c003520 con 0x7f413c0047c0 2024-02-02T13:24:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f415aa00640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f413c0047c0 0x7f413c004c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4144009ac0 tx=0x7f413c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c6dd32bab0d5ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414402c6d0 con 0x7f413c0047c0 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414402c830 con 0x7f413c0047c0 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f415a000640 1 -- 172.21.15.91:0/4068610745 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414402cbc0 con 0x7f413c0047c0 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f4162800640 1 -- 172.21.15.91:0/4068610745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f413c0047c0 msgr2=0x7f413c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f4162800640 1 --2- 172.21.15.91:0/4068610745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f413c0047c0 0x7f413c004c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4144009ac0 tx=0x7f413c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.196+0000 7f415be00640 1 -- 172.21.15.91:0/4068610745 reap_dead start 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.197+0000 7f4162800640 1 -- 172.21.15.91:0/4068610745 shutdown_connections 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.197+0000 7f4162800640 1 -- 172.21.15.91:0/4068610745 >> 172.21.15.91:0/4068610745 conn(0x7f415c0cec80 msgr2=0x7f415c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:56.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.197+0000 7f4162800640 1 -- 172.21.15.91:0/4068610745 shutdown_connections 2024-02-02T13:24:56.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.197+0000 7f4162800640 1 -- 172.21.15.91:0/4068610745 wait complete. 2024-02-02T13:24:56.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.198+0000 7f4162800640 1 Processor -- start 2024-02-02T13:24:56.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.198+0000 7f4162800640 1 -- start start 2024-02-02T13:24:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f413c0047c0 0x7f415c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 0x7f415c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0add00 con 0x7f415c14e2a0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0ae240 con 0x7f415c0ac8d0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f4162800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0adf60 con 0x7f413c0047c0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.199+0000 7f415be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54614/0 (socket says 172.21.15.91:54614) 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 -- 172.21.15.91:0/2406933975 learned_addr learned my addr 172.21.15.91:0/2406933975 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415aa00640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 0x7f415c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415b400640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f413c0047c0 0x7f415c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 -- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f413c0047c0 msgr2=0x7f415c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f413c0047c0 0x7f415c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 -- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 msgr2=0x7f415c0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 0x7f415c0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 -- 172.21.15.91:0/2406933975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0b2c80 con 0x7f415c14e2a0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415aa00640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 0x7f415c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f415be00640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f415c0cc770 tx=0x7f4154007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400ea40 con 0x7f415c14e2a0 2024-02-02T13:24:56.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400eba0 con 0x7f415c14e2a0 2024-02-02T13:24:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c0b2f60 con 0x7f415c14e2a0 2024-02-02T13:24:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.200+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154013420 con 0x7f415c14e2a0 2024-02-02T13:24:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.201+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c14e720 con 0x7f415c14e2a0 2024-02-02T13:24:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.202+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c066cf0 con 0x7f415c14e2a0 2024-02-02T13:24:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.203+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4154013580 con 0x7f415c14e2a0 2024-02-02T13:24:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.203+0000 7f4153400640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 0x7f413406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.203+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4154088c10 con 0x7f415c14e2a0 2024-02-02T13:24:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.204+0000 7f415b400640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 0x7f413406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.204+0000 7f415b400640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 0x7f413406c920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f414c009870 tx=0x7f414c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.208+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4154054200 con 0x7f415c14e2a0 2024-02-02T13:24:56.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.401+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f415c06c7b0 con 0x7f415c14e2a0 2024-02-02T13:24:56.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.402+0000 7f4153400640 1 -- 172.21.15.91:0/2406933975 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4154057820 con 0x7f415c14e2a0 2024-02-02T13:24:56.403 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:24:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 msgr2=0x7f413406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 0x7f413406c920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f414c009870 tx=0x7f414c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 msgr2=0x7f415c0cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f415c0cc770 tx=0x7f4154007af0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 shutdown_connections 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f413406a460 0x7f413406c920 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415c14e2a0 0x7f415c0cd520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415c0ac8d0 0x7f415c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 --2- 172.21.15.91:0/2406933975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f413c0047c0 0x7f415c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.404+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 >> 172.21.15.91:0/2406933975 conn(0x7f415c0cec80 msgr2=0x7f415c15ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.405+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 shutdown_connections 2024-02-02T13:24:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.405+0000 7f4162800640 1 -- 172.21.15.91:0/2406933975 wait complete. 2024-02-02T13:24:56.424 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:24:56.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.616+0000 7fa8a6c00640 1 Processor -- start 2024-02-02T13:24:56.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.617+0000 7fa8a6c00640 1 -- start start 2024-02-02T13:24:56.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa8a6c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa8a00b0e10 0x7fa8a006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa8a6c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015d870 con 0x7fa8a00acfe0 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa8a6c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a006bce0 con 0x7fa8a015d490 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa8a6c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a0161220 con 0x7fa8a00b0e10 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa89f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa8a00acfe0 0x7fa8a00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50244/0 (socket says 172.21.15.91:50244) 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa8a4c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa8a00b0e10 0x7fa8a006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.618+0000 7fa89f400640 1 -- 172.21.15.91:0/1767320647 learned_addr learned my addr 172.21.15.91:0/1767320647 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.619+0000 7fa89ea00640 1 -- 172.21.15.91:0/1767320647 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109430550 0 0) 0x7fa8a015d870 con 0x7fa8a00acfe0 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.619+0000 7fa89ea00640 1 -- 172.21.15.91:0/1767320647 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa880003520 con 0x7fa8a00acfe0 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.619+0000 7fa8a4c00640 1 -- 172.21.15.91:0/1767320647 >> v1:172.21.15.113:6789/0 conn(0x7fa8a015d490 legacy=0x7fa8a00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.619+0000 7fa8a4c00640 1 -- 172.21.15.91:0/1767320647 >> v1:172.21.15.91:6789/0 conn(0x7fa8a00acfe0 legacy=0x7fa8a00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.619+0000 7fa8a4c00640 1 -- 172.21.15.91:0/1767320647 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00ad490 con 0x7fa8a00b0e10 2024-02-02T13:24:56.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.620+0000 7fa8a4c00640 1 --2- 172.21.15.91:0/1767320647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a006c170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa898009d90 tx=0x7fa89802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4e4e00fe917b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.620+0000 7fa89ea00640 1 -- 172.21.15.91:0/1767320647 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89802cd00 con 0x7fa8a00b0e10 2024-02-02T13:24:56.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa89ea00640 1 -- 172.21.15.91:0/1767320647 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89802ce60 con 0x7fa8a00b0e10 2024-02-02T13:24:56.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa89ea00640 1 -- 172.21.15.91:0/1767320647 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898034660 con 0x7fa8a00b0e10 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 -- 172.21.15.91:0/1767320647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 msgr2=0x7fa8a006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/1767320647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a006c170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa898009d90 tx=0x7fa89802bf50 comp rx=0 tx=0).stop 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 -- 172.21.15.91:0/1767320647 shutdown_connections 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/1767320647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a006c170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 -- 172.21.15.91:0/1767320647 >> 172.21.15.91:0/1767320647 conn(0x7fa8a00cec80 msgr2=0x7fa8a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 -- 172.21.15.91:0/1767320647 shutdown_connections 2024-02-02T13:24:56.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.621+0000 7fa8a6c00640 1 -- 172.21.15.91:0/1767320647 wait complete. 2024-02-02T13:24:56.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.623+0000 7fa8a6c00640 1 Processor -- start 2024-02-02T13:24:56.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.623+0000 7fa8a6c00640 1 -- start start 2024-02-02T13:24:56.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a00acfe0 0x7fa8a00c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8a015d490 0x7fa8a0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c3da0 con 0x7fa8a00b0e10 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c27c0 con 0x7fa8a00acfe0 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa8a6c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c2aa0 con 0x7fa8a015d490 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.624+0000 7fa89f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54626/0 (socket says 172.21.15.91:54626) 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 -- 172.21.15.91:0/831417493 learned_addr learned my addr 172.21.15.91:0/831417493 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89fe00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a00acfe0 0x7fa8a00c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa8a4c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8a015d490 0x7fa8a0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 -- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8a015d490 msgr2=0x7fa8a0156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8a015d490 0x7fa8a0156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 -- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a00acfe0 msgr2=0x7fa8a00c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a00acfe0 0x7fa8a00c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 -- 172.21.15.91:0/831417493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa898009a70 con 0x7fa8a00b0e10 2024-02-02T13:24:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa89f400640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c400 tx=0x7fa88800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.625+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888005ce0 con 0x7fa8a00b0e10 2024-02-02T13:24:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.626+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888005e40 con 0x7fa8a00b0e10 2024-02-02T13:24:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.626+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880072d0 con 0x7fa8a00b0e10 2024-02-02T13:24:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.626+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00c2d80 con 0x7fa8a00b0e10 2024-02-02T13:24:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.626+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00c3340 con 0x7fa8a00b0e10 2024-02-02T13:24:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.627+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00c3560 con 0x7fa8a00b0e10 2024-02-02T13:24:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.628+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa8880074b0 con 0x7fa8a00b0e10 2024-02-02T13:24:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.629+0000 7fa897e00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 0x7fa87806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.629+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa888087d30 con 0x7fa8a00b0e10 2024-02-02T13:24:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.629+0000 7fa89fe00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 0x7fa87806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.629+0000 7fa89fe00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 0x7fa87806c8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa890000f60 tx=0x7fa890009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:56.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.635+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa888053460 con 0x7fa8a00b0e10 2024-02-02T13:24:56.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.821+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8a006ca50 con 0x7fa8a00b0e10 2024-02-02T13:24:56.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.822+0000 7fa897e00640 1 -- 172.21.15.91:0/831417493 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa888056a80 con 0x7fa8a00b0e10 2024-02-02T13:24:56.824 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:24:56.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.824+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 msgr2=0x7fa87806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.824+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 0x7fa87806c8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa890000f60 tx=0x7fa890009040 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.824+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 msgr2=0x7fa8a015c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.824+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c400 tx=0x7fa88800c8c0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 shutdown_connections 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa87806a430 0x7fa87806c8f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8a015d490 0x7fa8a0156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8a00b0e10 0x7fa8a015c750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 --2- 172.21.15.91:0/831417493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a00acfe0 0x7fa8a00c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 >> 172.21.15.91:0/831417493 conn(0x7fa8a00cec80 msgr2=0x7fa8a00caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 shutdown_connections 2024-02-02T13:24:56.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:56.825+0000 7fa8a6c00640 1 -- 172.21.15.91:0/831417493 wait complete. 2024-02-02T13:24:56.844 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:24:57.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.049+0000 7fe6e7e00640 1 Processor -- start 2024-02-02T13:24:57.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.049+0000 7fe6e7e00640 1 -- start start 2024-02-02T13:24:57.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 0x7fe6e015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe6e00b0e10 0x7fe6e00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e0070f80 con 0x7fe6e006f4f0 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e7e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e0152370 con 0x7fe6e00b0e10 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00cb010 con 0x7fe6e0151f90 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e5e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe6e00b0e10 0x7fe6e00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e5e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe6e00b0e10 0x7fe6e00b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54638/0 (socket says 172.21.15.91:54638) 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.050+0000 7fe6e5e00640 1 -- 172.21.15.91:0/959563367 learned_addr learned my addr 172.21.15.91:0/959563367 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5400640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 0x7fe6e015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5e00640 1 -- 172.21.15.91:0/959563367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 msgr2=0x7fe6e015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5e00640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 0x7fe6e015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5e00640 1 -- 172.21.15.91:0/959563367 >> v1:172.21.15.113:6789/0 conn(0x7fe6e006f4f0 legacy=0x7fe6e00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5e00640 1 -- 172.21.15.91:0/959563367 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e00cb2f0 con 0x7fe6e00b0e10 2024-02-02T13:24:57.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.051+0000 7fe6e5400640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 0x7fe6e015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:57.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.052+0000 7fe6e5e00640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e00b0e10 0x7fe6e00b3af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009870 tx=0x7fe6d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d111bb80857d7c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.052+0000 7fe6dfe00640 1 -- 172.21.15.91:0/959563367 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8031640 con 0x7fe6e00b0e10 2024-02-02T13:24:57.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.052+0000 7fe6dfe00640 1 -- 172.21.15.91:0/959563367 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d80317a0 con 0x7fe6e00b0e10 2024-02-02T13:24:57.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.052+0000 7fe6dfe00640 1 -- 172.21.15.91:0/959563367 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8031b30 con 0x7fe6e00b0e10 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 -- 172.21.15.91:0/959563367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e00b0e10 msgr2=0x7fe6e00b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e00b0e10 0x7fe6e00b3af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009870 tx=0x7fe6d802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 -- 172.21.15.91:0/959563367 shutdown_connections 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e00b0e10 0x7fe6e00b3af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/959563367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e0151f90 0x7fe6e015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 -- 172.21.15.91:0/959563367 >> 172.21.15.91:0/959563367 conn(0x7fe6e00cec80 msgr2=0x7fe6e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 -- 172.21.15.91:0/959563367 shutdown_connections 2024-02-02T13:24:57.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.053+0000 7fe6e7e00640 1 -- 172.21.15.91:0/959563367 wait complete. 2024-02-02T13:24:57.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.055+0000 7fe6e7e00640 1 Processor -- start 2024-02-02T13:24:57.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.055+0000 7fe6e7e00640 1 -- start start 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 0x7fe6e01f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 0x7fe6e0150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01f7320 con 0x7fe6e006f4f0 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00c5cc0 con 0x7fe6e0151f90 2024-02-02T13:24:57.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0157a00 con 0x7fe6e00b0e10 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54656/0 (socket says 172.21.15.91:54656) 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.056+0000 7fe6e5400640 1 -- 172.21.15.91:0/1203036996 learned_addr learned my addr 172.21.15.91:0/1203036996 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 0x7fe6e0150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e4a00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 0x7fe6e01f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 -- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 msgr2=0x7fe6e01f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 0x7fe6e01f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 -- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 msgr2=0x7fe6e0150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 0x7fe6e0150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 -- 172.21.15.91:0/1203036996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d80094d0 con 0x7fe6e006f4f0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e4a00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 0x7fe6e01f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 0x7fe6e0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e5400640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000ce00 tx=0x7fe6d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000f890 con 0x7fe6e006f4f0 2024-02-02T13:24:57.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d000f9f0 con 0x7fe6e006f4f0 2024-02-02T13:24:57.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000fdc0 con 0x7fe6e006f4f0 2024-02-02T13:24:57.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0157ce0 con 0x7fe6e006f4f0 2024-02-02T13:24:57.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.057+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e00c5f10 con 0x7fe6e006f4f0 2024-02-02T13:24:57.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.058+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0070c10 con 0x7fe6e006f4f0 2024-02-02T13:24:57.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.059+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe6d000b040 con 0x7fe6e006f4f0 2024-02-02T13:24:57.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.060+0000 7fe6dd600640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 0x7fe6b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.060+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6d0089310 con 0x7fe6e006f4f0 2024-02-02T13:24:57.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.061+0000 7fe6e4a00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 0x7fe6b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.061+0000 7fe6e4a00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 0x7fe6b806c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00c20a0 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.065+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6d00548f0 con 0x7fe6e006f4f0 2024-02-02T13:24:57.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.295+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6e006f930 con 0x7fe6e006f4f0 2024-02-02T13:24:57.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.296+0000 7fe6dd600640 1 -- 172.21.15.91:0/1203036996 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fe6e006f930 con 0x7fe6e006f4f0 2024-02-02T13:24:57.298 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:57.298 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:24:57.298 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:24:57.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.298+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 msgr2=0x7fe6b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 0x7fe6b806c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00c20a0 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 msgr2=0x7fe6e01585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000ce00 tx=0x7fe6d000a650 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 shutdown_connections 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6b806a430 0x7fe6b806c8f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e00b0e10 0x7fe6e01f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e0151f90 0x7fe6e0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 --2- 172.21.15.91:0/1203036996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6e006f4f0 0x7fe6e01585b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 >> 172.21.15.91:0/1203036996 conn(0x7fe6e00cec80 msgr2=0x7fe6e006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 shutdown_connections 2024-02-02T13:24:57.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.299+0000 7fe6e7e00640 1 -- 172.21.15.91:0/1203036996 wait complete. 2024-02-02T13:24:57.318 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:24:57.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.518+0000 7f8bee800640 1 Processor -- start 2024-02-02T13:24:57.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.518+0000 7f8bee800640 1 -- start start 2024-02-02T13:24:57.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8bee800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8be815a010 0x7f8be80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8bee800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8be80ce130 0x7f8be80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8bee800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80cc7c0 con 0x7f8be80cd470 2024-02-02T13:24:57.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8bee800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be80ce930 con 0x7f8be80ce130 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8bee800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be806c890 con 0x7f8be815a010 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8be80ce130 0x7f8be80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8be80ce130 0x7f8be80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54658/0 (socket says 172.21.15.91:54658) 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 -- 172.21.15.91:0/2874702692 learned_addr learned my addr 172.21.15.91:0/2874702692 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be7e00640 1 --1- 172.21.15.91:0/2874702692 >> v1:172.21.15.91:6789/0 conn(0x7f8be80cd470 0x7f8be80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50254/0 (socket says 172.21.15.91:50254) 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be7400640 1 --2- 172.21.15.91:0/2874702692 >> v2:172.21.15.113:3300/0 conn(0x7f8be815a010 0x7f8be80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 -- 172.21.15.91:0/2874702692 >> v2:172.21.15.113:3300/0 conn(0x7f8be815a010 msgr2=0x7f8be80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:57.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 --2- 172.21.15.91:0/2874702692 >> v2:172.21.15.113:3300/0 conn(0x7f8be815a010 0x7f8be80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 -- 172.21.15.91:0/2874702692 >> v1:172.21.15.91:6789/0 conn(0x7f8be80cd470 legacy=0x7f8be80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6000640 1 -- 172.21.15.91:0/2874702692 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646868920 0 0) 0x7f8be80cc7c0 con 0x7f8be80cd470 2024-02-02T13:24:57.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.520+0000 7f8be6a00640 1 -- 172.21.15.91:0/2874702692 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be806cb70 con 0x7f8be80ce130 2024-02-02T13:24:57.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.521+0000 7f8be6a00640 1 --2- 172.21.15.91:0/2874702692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80ccf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8be80cfaa0 tx=0x7f8bd002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5489c95af1e0ac9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8be6000640 1 -- 172.21.15.91:0/2874702692 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd002ca80 con 0x7f8be80ce130 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8be6000640 1 -- 172.21.15.91:0/2874702692 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd002cbe0 con 0x7f8be80ce130 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8be6000640 1 -- 172.21.15.91:0/2874702692 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00343f0 con 0x7f8be80ce130 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 -- 172.21.15.91:0/2874702692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 msgr2=0x7f8be80ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 --2- 172.21.15.91:0/2874702692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80ccf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8be80cfaa0 tx=0x7f8bd002bee0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 -- 172.21.15.91:0/2874702692 shutdown_connections 2024-02-02T13:24:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 --2- 172.21.15.91:0/2874702692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80ccf30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 --2- 172.21.15.91:0/2874702692 >> v2:172.21.15.113:3300/0 conn(0x7f8be815a010 0x7f8be80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.522+0000 7f8bee800640 1 -- 172.21.15.91:0/2874702692 >> 172.21.15.91:0/2874702692 conn(0x7f8be80cec80 msgr2=0x7f8be80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.523+0000 7f8bee800640 1 -- 172.21.15.91:0/2874702692 shutdown_connections 2024-02-02T13:24:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.523+0000 7f8bee800640 1 -- 172.21.15.91:0/2874702692 wait complete. 2024-02-02T13:24:57.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.524+0000 7f8bee800640 1 Processor -- start 2024-02-02T13:24:57.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.524+0000 7f8bee800640 1 -- start start 2024-02-02T13:24:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be815a010 0x7f8be80b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be815dbb0 0x7f8be80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80b37f0 con 0x7f8be80ce130 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80b2f50 con 0x7f8be815dbb0 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.525+0000 7f8bee800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80b3230 con 0x7f8be815a010 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54674/0 (socket says 172.21.15.91:54674) 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 -- 172.21.15.91:0/1152712109 learned_addr learned my addr 172.21.15.91:0/1152712109 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 -- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be815a010 msgr2=0x7f8be80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be815a010 0x7f8be80b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 -- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be815dbb0 msgr2=0x7f8be80c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be815dbb0 0x7f8be80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 -- 172.21.15.91:0/1152712109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009b30 con 0x7f8be80ce130 2024-02-02T13:24:57.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8be7400640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800d5f0 tx=0x7f8bd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.526+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80125f0 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.527+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8012750 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.527+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8012b00 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.527+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be80b3510 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.527+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be815e060 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.528+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be80ccf30 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.529+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8bd8012c60 con 0x7f8be80ce130 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.529+0000 7f8bdf400640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 0x7f8bc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.530+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bd8088e60 con 0x7f8be80ce130 2024-02-02T13:24:57.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.530+0000 7f8be6a00640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 0x7f8bc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.530+0000 7f8be6a00640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 0x7f8bc006c920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0035000 tx=0x7f8bd002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.534+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8bd80543b0 con 0x7f8be80ce130 2024-02-02T13:24:57.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.729+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8be806ce20 con 0x7f8be80ce130 2024-02-02T13:24:57.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.732+0000 7f8bdf400640 1 -- 172.21.15.91:0/1152712109 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f8bd80579d0 con 0x7f8be80ce130 2024-02-02T13:24:57.734 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:57.734 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:24:57.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 msgr2=0x7f8bc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 0x7f8bc006c920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0035000 tx=0x7f8bd002e000 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 msgr2=0x7f8be80b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800d5f0 tx=0x7f8bd800dab0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 shutdown_connections 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8bc006a460 0x7f8bc006c920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be815a010 0x7f8be80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be815dbb0 0x7f8be80c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 --2- 172.21.15.91:0/1152712109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8be80ce130 0x7f8be80b28c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 >> 172.21.15.91:0/1152712109 conn(0x7f8be80cec80 msgr2=0x7f8be80af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 shutdown_connections 2024-02-02T13:24:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.735+0000 7f8bee800640 1 -- 172.21.15.91:0/1152712109 wait complete. 2024-02-02T13:24:57.756 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:24:57.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.960+0000 7f493f200640 1 Processor -- start 2024-02-02T13:24:57.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.960+0000 7f493f200640 1 -- start start 2024-02-02T13:24:57.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494015b2a0 0x7f49400cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 0x7f49400ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494006c1c0 con 0x7f494015aec0 2024-02-02T13:24:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493f200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494006b5f0 con 0x7f494015b2a0 2024-02-02T13:24:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494006cd60 con 0x7f494006f3f0 2024-02-02T13:24:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f494015aec0 0x7f494006c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50266/0 (socket says 172.21.15.91:50266) 2024-02-02T13:24:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493de00640 1 -- 172.21.15.91:0/2046541418 learned_addr learned my addr 172.21.15.91:0/2046541418 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f4935200640 1 --2- 172.21.15.91:0/2046541418 >> v2:172.21.15.91:3300/0 conn(0x7f494015b2a0 0x7f49400cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493e800640 1 --2- 172.21.15.91:0/2046541418 >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 0x7f49400ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.962+0000 7f493d400640 1 -- 172.21.15.91:0/2046541418 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700789001 0 0) 0x7f494006c1c0 con 0x7f494015aec0 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f493d400640 1 -- 172.21.15.91:0/2046541418 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4920003520 con 0x7f494015aec0 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f4935200640 1 -- 172.21.15.91:0/2046541418 >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 msgr2=0x7f49400ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f4935200640 1 --2- 172.21.15.91:0/2046541418 >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 0x7f49400ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f4935200640 1 -- 172.21.15.91:0/2046541418 >> v1:172.21.15.91:6789/0 conn(0x7f494015aec0 legacy=0x7f494006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f4935200640 1 -- 172.21.15.91:0/2046541418 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494006b890 con 0x7f494015b2a0 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.963+0000 7f493e800640 1 --2- 172.21.15.91:0/2046541418 >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 0x7f49400ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f4935200640 1 --2- 172.21.15.91:0/2046541418 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015b2a0 0x7f49400cf5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f49400b46d0 tx=0x7f492802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbe93a6f2a6ee8d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493d400640 1 -- 172.21.15.91:0/2046541418 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928030610 con 0x7f494015b2a0 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493d400640 1 -- 172.21.15.91:0/2046541418 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928030770 con 0x7f494015b2a0 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493d400640 1 -- 172.21.15.91:0/2046541418 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928030b00 con 0x7f494015b2a0 2024-02-02T13:24:57.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 -- 172.21.15.91:0/2046541418 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015b2a0 msgr2=0x7f49400cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 --2- 172.21.15.91:0/2046541418 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015b2a0 0x7f49400cf5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f49400b46d0 tx=0x7f492802b960 comp rx=0 tx=0).stop 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 -- 172.21.15.91:0/2046541418 shutdown_connections 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 --2- 172.21.15.91:0/2046541418 >> v2:172.21.15.113:3300/0 conn(0x7f494006f3f0 0x7f49400ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 --2- 172.21.15.91:0/2046541418 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015b2a0 0x7f49400cf5b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.964+0000 7f493f200640 1 -- 172.21.15.91:0/2046541418 >> 172.21.15.91:0/2046541418 conn(0x7f49400cea90 msgr2=0x7f49400ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.965+0000 7f493f200640 1 -- 172.21.15.91:0/2046541418 shutdown_connections 2024-02-02T13:24:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.965+0000 7f493f200640 1 -- 172.21.15.91:0/2046541418 wait complete. 2024-02-02T13:24:57.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.966+0000 7f493f200640 1 Processor -- start 2024-02-02T13:24:57.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.966+0000 7f493f200640 1 -- start start 2024-02-02T13:24:57.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 0x7f49400c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494015b2a0 0x7f49400ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c8200 con 0x7f494015aec0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c8d40 con 0x7f494006f3f0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.967+0000 7f493f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c9020 con 0x7f494015b2a0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54684/0 (socket says 172.21.15.91:54684) 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 -- 172.21.15.91:0/524271459 learned_addr learned my addr 172.21.15.91:0/524271459 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f493de00640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 0x7f49400c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f493e800640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494015b2a0 0x7f49400ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 -- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494015b2a0 msgr2=0x7f49400ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494015b2a0 0x7f49400ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 -- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 msgr2=0x7f49400c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 0x7f49400c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 -- 172.21.15.91:0/524271459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009400 con 0x7f494015aec0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f493de00640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 0x7f49400c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.968+0000 7f4935200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4928009890 tx=0x7f49280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.969+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492803e070 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.969+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928030e10 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.969+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400c9300 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.969+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928039c30 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.969+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400c98c0 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.970+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494004dff0 con 0x7f494015aec0 2024-02-02T13:24:57.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.971+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f492804b400 con 0x7f494015aec0 2024-02-02T13:24:57.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.972+0000 7f4936a00640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 0x7f491806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:57.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.972+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49280aaa70 con 0x7f494015aec0 2024-02-02T13:24:57.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.972+0000 7f493de00640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 0x7f491806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.972+0000 7f493de00640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 0x7f491806c8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49300011e0 tx=0x7f4930009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:57.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:57.976+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4928076150 con 0x7f494015aec0 2024-02-02T13:24:58.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.200+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49400af030 con 0x7f494015aec0 2024-02-02T13:24:58.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.201+0000 7f4936a00640 1 -- 172.21.15.91:0/524271459 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f4928034090 con 0x7f494015aec0 2024-02-02T13:24:58.203 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:24:58.203 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:24:58.203 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:24:58.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 msgr2=0x7f491806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 0x7f491806c8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49300011e0 tx=0x7f4930009040 comp rx=0 tx=0).stop 2024-02-02T13:24:58.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 msgr2=0x7f49400cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4928009890 tx=0x7f49280046c0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 shutdown_connections 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491806a430 0x7f491806c8f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494015b2a0 0x7f49400ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494015aec0 0x7f49400cd980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 --2- 172.21.15.91:0/524271459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494006f3f0 0x7f49400c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 >> 172.21.15.91:0/524271459 conn(0x7f49400cea90 msgr2=0x7f494015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 shutdown_connections 2024-02-02T13:24:58.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.204+0000 7f493f200640 1 -- 172.21.15.91:0/524271459 wait complete. 2024-02-02T13:24:58.325 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:24:58.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:24:58.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:24:36.734786+0000', 'modified': '2024-02-02T13:24:40.234617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4255': {'gid': 4255, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6834/1768842703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 1768842703}, {'type': 'v1', 'addr': '172.21.15.113:6834', 'nonce': 1768842703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:24:58.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:24:58.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-02-02T13:24:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.842+0000 7f163a600640 1 Processor -- start 2024-02-02T13:24:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.842+0000 7f163a600640 1 -- start start 2024-02-02T13:24:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f163a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 0x7f1634071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f163a600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f16340b1c40 0x7f16340d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f163a600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b8890 con 0x7f163415d3b0 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f163a600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1634161b10 con 0x7f16340b1c40 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f163a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340b8b30 con 0x7f1634160d00 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f1633e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f16340b1c40 0x7f16340d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f1632a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f163415d3b0 0x7f16341610e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50280/0 (socket says 172.21.15.91:50280) 2024-02-02T13:24:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.844+0000 7f1632a00640 1 -- 172.21.15.91:0/3565322780 learned_addr learned my addr 172.21.15.91:0/3565322780 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633400640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 0x7f1634071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633e00640 1 -- 172.21.15.91:0/3565322780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 msgr2=0x7f1634071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633e00640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 0x7f1634071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633e00640 1 -- 172.21.15.91:0/3565322780 >> v1:172.21.15.91:6789/0 conn(0x7f163415d3b0 legacy=0x7f16341610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633e00640 1 -- 172.21.15.91:0/3565322780 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340b8e10 con 0x7f16340b1c40 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.845+0000 7f1633400640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 0x7f1634071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.846+0000 7f1633e00640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16340b1c40 0x7f16340d0be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f162c0097a0 tx=0x7f162c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bba9e09e2d2212b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.846+0000 7f1632000640 1 -- 172.21.15.91:0/3565322780 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c02d040 con 0x7f16340b1c40 2024-02-02T13:24:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.846+0000 7f1632000640 1 -- 172.21.15.91:0/3565322780 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c031810 con 0x7f16340b1c40 2024-02-02T13:24:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.846+0000 7f1632000640 1 -- 172.21.15.91:0/3565322780 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c031ba0 con 0x7f16340b1c40 2024-02-02T13:24:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 -- 172.21.15.91:0/3565322780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16340b1c40 msgr2=0x7f16340d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16340b1c40 0x7f16340d0be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f162c0097a0 tx=0x7f162c02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 -- 172.21.15.91:0/3565322780 shutdown_connections 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16340b1c40 0x7f16340d0be0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 --2- 172.21.15.91:0/3565322780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1634160d00 0x7f1634071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 -- 172.21.15.91:0/3565322780 >> 172.21.15.91:0/3565322780 conn(0x7f16340d3010 msgr2=0x7f16340d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 -- 172.21.15.91:0/3565322780 shutdown_connections 2024-02-02T13:24:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.847+0000 7f163a600640 1 -- 172.21.15.91:0/3565322780 wait complete. 2024-02-02T13:24:58.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.848+0000 7f163a600640 1 Processor -- start 2024-02-02T13:24:58.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.849+0000 7f163a600640 1 -- start start 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.849+0000 7f163a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16340b1c40 0x7f16340caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f163a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f163a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1634160d00 0x7f16340d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f163a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16341f3770 con 0x7f163415d3b0 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f163a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16341f3a50 con 0x7f1634160d00 2024-02-02T13:24:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f163a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340ca2a0 con 0x7f16340b1c40 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54714/0 (socket says 172.21.15.91:54714) 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 -- 172.21.15.91:0/3434971013 learned_addr learned my addr 172.21.15.91:0/3434971013 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1633400640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16340b1c40 0x7f16340caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1633e00640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1634160d00 0x7f16340d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 -- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16340b1c40 msgr2=0x7f16340caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16340b1c40 0x7f16340caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 -- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1634160d00 msgr2=0x7f16340d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1634160d00 0x7f16340d33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 -- 172.21.15.91:0/3434971013 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c009400 con 0x7f163415d3b0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.850+0000 7f1632a00640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f162400cbc0 tx=0x7f162400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.851+0000 7f162b400640 1 -- 172.21.15.91:0/3434971013 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400ad40 con 0x7f163415d3b0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.851+0000 7f162b400640 1 -- 172.21.15.91:0/3434971013 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624012400 con 0x7f163415d3b0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.851+0000 7f162b400640 1 -- 172.21.15.91:0/3434971013 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400aea0 con 0x7f163415d3b0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.851+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340ca580 con 0x7f163415d3b0 2024-02-02T13:24:58.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.851+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16340cafe0 con 0x7f163415d3b0 2024-02-02T13:24:58.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.853+0000 7f162b400640 1 -- 172.21.15.91:0/3434971013 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f16240127b0 con 0x7f163415d3b0 2024-02-02T13:24:58.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.854+0000 7f162b400640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 0x7f160c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.854+0000 7f162b400640 1 -- 172.21.15.91:0/3434971013 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1624088800 con 0x7f163415d3b0 2024-02-02T13:24:58.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.854+0000 7f1633400640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 0x7f160c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.855+0000 7f1633400640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 0x7f160c06c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b840 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.855+0000 7f163d3ac740 1 Processor -- start 2024-02-02T13:24:58.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.855+0000 7f163d3ac740 1 -- start start 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ebef50 0x564e88ebf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88e92ad0 0x564e88ebc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88e6a650 con 0x564e88e8eed0 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88dff000 con 0x564e88ebef50 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e88dfc790 con 0x564e88e92ad0 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54720/0 (socket says 172.21.15.91:54720) 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 -- 172.21.15.91:0/3669315449 learned_addr learned my addr 172.21.15.91:0/3669315449 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:58.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 -- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88e92ad0 msgr2=0x564e88ebc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88e92ad0 0x564e88ebc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 -- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ebef50 msgr2=0x564e88ebf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ebef50 0x564e88ebf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 -- 172.21.15.91:0/3669315449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e88dbf710 con 0x564e88e8eed0 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.856+0000 7f1632a00640 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1624057ae0 tx=0x7f1624057c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c4b403a6fa7f9a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f1623e00640 1 -- 172.21.15.91:0/3669315449 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624055ef0 con 0x564e88e8eed0 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f1623e00640 1 -- 172.21.15.91:0/3669315449 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624054260 con 0x564e88e8eed0 2024-02-02T13:24:58.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f1623e00640 1 -- 172.21.15.91:0/3669315449 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624057260 con 0x564e88e8eed0 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 -- 172.21.15.91:0/3669315449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 msgr2=0x564e88ebc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1624057ae0 tx=0x7f1624057c90 comp rx=0 tx=0).stop 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 -- 172.21.15.91:0/3669315449 shutdown_connections 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88e92ad0 0x564e88ebc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e8eed0 0x564e88ebc1c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 --2- 172.21.15.91:0/3669315449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ebef50 0x564e88ebf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 -- 172.21.15.91:0/3669315449 >> 172.21.15.91:0/3669315449 conn(0x564e88e911e0 msgr2=0x564e88ed9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 -- 172.21.15.91:0/3669315449 shutdown_connections 2024-02-02T13:24:58.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.857+0000 7f163d3ac740 1 -- 172.21.15.91:0/3669315449 wait complete. 2024-02-02T13:24:58.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.858+0000 7f163d3ac740 1 -- start start 2024-02-02T13:24:58.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.858+0000 7f163d3ac740 1 Processor -- start 2024-02-02T13:24:58.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ea5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88df8080 con 0x564e88e92ad0 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88d99850 con 0x564e88ea5690 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e88d994f0 con 0x564e88ebef50 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.859+0000 7f163d3ac740 10 client.0 _ll_register_callbacks cb 0x564e88df19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54734/0 (socket says 172.21.15.91:54734) 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 -- 172.21.15.91:0/2264385630 learned_addr learned my addr 172.21.15.91:0/2264385630 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 -- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 msgr2=0x564e88ea5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ea5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 -- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 msgr2=0x564e88edc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633e00640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 -- 172.21.15.91:0/2264385630 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16340cafe0 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1633400640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c012750 tx=0x7f161c012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22987bb1075ffaf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.860+0000 7f1620c00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00b050 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f161c013800 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e88e225d0 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f161c012e70 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 10 client.4644 fetch_fsmap learned FSMap version 8 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564e88df6700 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e88e225d0 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f161c0139c0 con 0x564e88e92ad0 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 10 client.4644 fetch_fsmap finished waiting for FSMap version 8 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 10 client.4644 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-02T13:24:58.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 20 client.4644 populate_metadata read hostname 'smithi091' 2024-02-02T13:24:58.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88eb72a0 0x564e88eb7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 4 client.4644 mds_command: new command op to 4255 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:24:58.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.861+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 --> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e88dea8d0 con 0x564e88eb72a0 2024-02-02T13:24:58.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.862+0000 7f1632a00640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88eb72a0 0x564e88eb7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.863+0000 7f1632a00640 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88eb72a0 0x564e88eb7680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf3b5c04a746978 server_cookie=685616ad367ec846 in_seq=0 out_seq=0 2024-02-02T13:24:58.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.863+0000 7f1620c00640 10 client.4644 ms_handle_connect on v2:172.21.15.113:6832/1768842703 2024-02-02T13:24:58.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f1620c00640 1 -- 172.21.15.91:0/2264385630 <== mds.0 v2:172.21.15.113:6832/1768842703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564e88dea8d0 con 0x564e88eb72a0 2024-02-02T13:24:58.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f1620c00640 10 client.4644 handle_command_reply: tid=0 2024-02-02T13:24:58.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 client.4644 shutdown 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 msgr2=0x564e88eb4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c012750 tx=0x7f161c012840 comp rx=0 tx=0).stop 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 shutdown_connections 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88eb72a0 0x564e88eb7680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ea5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f1620c00640 0 client.4644 ms_handle_reset on v2:172.21.15.113:6832/1768842703 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2264385630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88eb4130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 >> 172.21.15.91:0/2264385630 conn(0x564e88e8eed0 msgr2=0x564e88ec2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 shutdown_connections 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.865+0000 7f163d3ac740 1 -- 172.21.15.91:0/2264385630 wait complete. 2024-02-02T13:24:58.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.866+0000 7f163d3ac740 20 client.4644 trim_cache size 0 max 16384 2024-02-02T13:24:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 Processor -- start 2024-02-02T13:24:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 -- start start 2024-02-02T13:24:58.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ef2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88b5ff20 con 0x564e88e92ad0 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88ea3270 con 0x564e88ea5690 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.882+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e88ea29d0 con 0x564e88ebef50 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54748/0 (socket says 172.21.15.91:54748) 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 -- 172.21.15.91:0/2616268073 learned_addr learned my addr 172.21.15.91:0/2616268073 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 -- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 msgr2=0x564e88ef2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ef2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 -- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 msgr2=0x564e88edd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 -- 172.21.15.91:0/2616268073 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e88e2c280 con 0x564e88e92ad0 2024-02-02T13:24:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.883+0000 7f1633400640 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f161c012880 tx=0x7f161c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46356c6421e9cf85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f1628c00640 1 -- 172.21.15.91:0/2616268073 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c01e400 con 0x564e88e92ad0 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f1628c00640 1 -- 172.21.15.91:0/2616268073 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c017070 con 0x564e88e92ad0 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f1628c00640 1 -- 172.21.15.91:0/2616268073 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c01e760 con 0x564e88e92ad0 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 -- 172.21.15.91:0/2616268073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 msgr2=0x564e88edd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f161c012880 tx=0x7f161c01c000 comp rx=0 tx=0).stop 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 -- 172.21.15.91:0/2616268073 shutdown_connections 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88ef2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88ea5690 0x564e88edd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 --2- 172.21.15.91:0/2616268073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88e92ad0 0x564e88edd270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 -- 172.21.15.91:0/2616268073 >> 172.21.15.91:0/2616268073 conn(0x564e88e8eed0 msgr2=0x564e88e8e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 -- 172.21.15.91:0/2616268073 shutdown_connections 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.884+0000 7f163d3ac740 1 -- 172.21.15.91:0/2616268073 wait complete. 2024-02-02T13:24:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.885+0000 7f163d3ac740 1 -- start start 2024-02-02T13:24:58.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.885+0000 7f163d3ac740 1 Processor -- start 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88edd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88f32140 0x564e88edd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88e95380 con 0x564e88ea5690 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88f49f30 con 0x564e88f32140 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e88f48b60 con 0x564e88ebef50 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f163d3ac740 10 client.0 _ll_register_callbacks cb 0x7f161c00de30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f1633400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54764/0 (socket says 172.21.15.91:54764) 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.886+0000 7f1633400640 1 -- 172.21.15.91:0/1437244679 learned_addr learned my addr 172.21.15.91:0/1437244679 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1632a00640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88edd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633e00640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88f32140 0x564e88edd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 -- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 msgr2=0x564e88edd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88edd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 -- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88f32140 msgr2=0x564e88edd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88f32140 0x564e88edd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 -- 172.21.15.91:0/1437244679 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e88df6700 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1633400640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f161c00df70 tx=0x7f161c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b8a9123be81d6a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1620c00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c01e810 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f161c013ad0 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1624054260 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f161c004640 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f163d3ac740 10 client.4650 fetch_fsmap learned FSMap version 8 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.887+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564e88d98a60 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624054260 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f161c03d070 con 0x564e88ea5690 2024-02-02T13:24:58.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 20 client.4650 populate_metadata read hostname 'smithi091' 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88ef3200 0x564e88ef35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 4 client.4650 mds_command: new command op to 4255 tid=0[{"prefix": "status"}] 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 --> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564e88df1100 con 0x564e88ef3200 2024-02-02T13:24:58.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.888+0000 7f1632a00640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88ef3200 0x564e88ef35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:58.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.889+0000 7f1620c00640 10 client.4650 ms_handle_connect on v2:172.21.15.113:6832/1768842703 2024-02-02T13:24:58.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.889+0000 7f1632a00640 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88ef3200 0x564e88ef35e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c8a115ce6d2641 server_cookie=b31cbc30fa9e3e14 in_seq=0 out_seq=0 2024-02-02T13:24:58.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f1620c00640 1 -- 172.21.15.91:0/1437244679 <== mds.0 v2:172.21.15.113:6832/1768842703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x564e88df1100 con 0x564e88ef3200 2024-02-02T13:24:58.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f1620c00640 10 client.4650 handle_command_reply: tid=0 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 client.4650 shutdown 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 msgr2=0x564e88e94e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f161c00df70 tx=0x7f161c013000 comp rx=0 tx=0).stop 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 shutdown_connections 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:6832/1768842703,v1:172.21.15.113:6834/1768842703] conn(0x564e88ef3200 0x564e88ef35e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564e88ebef50 0x564e88edd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564e88f32140 0x564e88edd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 --2- 172.21.15.91:0/1437244679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564e88ea5690 0x564e88e94e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f1620c00640 0 client.4650 ms_handle_reset on v2:172.21.15.113:6832/1768842703 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 >> 172.21.15.91:0/1437244679 conn(0x564e88e8eed0 msgr2=0x564e88e93420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 shutdown_connections 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 1 -- 172.21.15.91:0/1437244679 wait complete. 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.890+0000 7f163d3ac740 20 client.4650 trim_cache size 0 max 16384 2024-02-02T13:24:58.894 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 4255, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 22.147347869000001, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 8, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 20, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 20, 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 27.294518139000001 2024-02-02T13:24:58.895 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 msgr2=0x7f160c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 0x7f160c06c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b840 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 msgr2=0x7f16341f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f162400cbc0 tx=0x7f162400a430 comp rx=0 tx=0).stop 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 shutdown_connections 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f160c06a430 0x7f160c06c8f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1634160d00 0x7f16340d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f163415d3b0 0x7f16341f2cb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 --2- 172.21.15.91:0/3434971013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16340b1c40 0x7f16340caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:58.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 >> 172.21.15.91:0/3434971013 conn(0x7f16340d3010 msgr2=0x7f16340b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:58.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 shutdown_connections 2024-02-02T13:24:58.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:58.891+0000 7f163a600640 1 -- 172.21.15.91:0/3434971013 wait complete. 2024-02-02T13:24:58.908 INFO:teuthology.run_tasks:Running task kclient... 2024-02-02T13:24:58.931 INFO:tasks.kclient:Mounting kernel clients... 2024-02-02T13:24:58.931 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-02-02T13:24:58.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:58.932 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:24:58.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:58.983 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2024-02-02T13:24:59.080 INFO:teuthology.orchestra.run.smithi091.stderr:Cannot find device "ceph-brx" 2024-02-02T13:24:59.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:24:59.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.084 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:24:59.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.133 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2024-02-02T13:24:59.227 INFO:teuthology.orchestra.run.smithi091.stderr:Cannot find device "ceph-brx" 2024-02-02T13:24:59.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:24:59.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.232 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:24:59.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.289 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-02-02T13:24:59.383 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-02-02T13:24:59.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:24:59.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.387 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:24:59.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:24:59.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-02-02T13:24:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-02-02T13:24:59.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:24:59.529 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-02-02T13:24:59.529 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T13:24:59.529 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:24:59.529 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:24:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.745+0000 7f69a7800640 1 Processor -- start 2024-02-02T13:24:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.745+0000 7f69a7800640 1 -- start start 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a7800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69a015a010 0x7f69a00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a00b0700 0x7f69a00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a7800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a00cc7c0 con 0x7f69a00ce130 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a7800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00b10d0 con 0x7f69a015a010 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00ce930 con 0x7f69a00b0700 2024-02-02T13:24:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a4e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69a015a010 0x7f69a00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a4e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69a015a010 0x7f69a00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54770/0 (socket says 172.21.15.91:54770) 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a4e00640 1 -- 172.21.15.91:0/2832000115 learned_addr learned my addr 172.21.15.91:0/2832000115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a4e00640 1 -- 172.21.15.91:0/2832000115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a00b0700 msgr2=0x7f69a00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.747+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2832000115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a00b0700 0x7f69a00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.748+0000 7f69a4e00640 1 -- 172.21.15.91:0/2832000115 >> v1:172.21.15.113:6789/0 conn(0x7f69a00ce130 legacy=0x7f69a00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:24:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.748+0000 7f69a4e00640 1 -- 172.21.15.91:0/2832000115 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00cd640 con 0x7f69a015a010 2024-02-02T13:24:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2832000115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a015a010 0x7f69a00c9670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6990009870 tx=0x7f699002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ebfc656c60a918 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f699fe00640 1 -- 172.21.15.91:0/2832000115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990031640 con 0x7f69a015a010 2024-02-02T13:24:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f699fe00640 1 -- 172.21.15.91:0/2832000115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69900317a0 con 0x7f69a015a010 2024-02-02T13:24:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f699fe00640 1 -- 172.21.15.91:0/2832000115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990031b30 con 0x7f69a015a010 2024-02-02T13:24:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f69a7800640 1 -- 172.21.15.91:0/2832000115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a015a010 msgr2=0x7f69a00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.749+0000 7f69a7800640 1 --2- 172.21.15.91:0/2832000115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a015a010 0x7f69a00c9670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6990009870 tx=0x7f699002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 -- 172.21.15.91:0/2832000115 shutdown_connections 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 --2- 172.21.15.91:0/2832000115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a00b0700 0x7f69a00b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 --2- 172.21.15.91:0/2832000115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a015a010 0x7f69a00c9670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 -- 172.21.15.91:0/2832000115 >> 172.21.15.91:0/2832000115 conn(0x7f69a00cec80 msgr2=0x7f69a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 -- 172.21.15.91:0/2832000115 shutdown_connections 2024-02-02T13:24:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.750+0000 7f69a7800640 1 -- 172.21.15.91:0/2832000115 wait complete. 2024-02-02T13:24:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.751+0000 7f69a7800640 1 Processor -- start 2024-02-02T13:24:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.751+0000 7f69a7800640 1 -- start start 2024-02-02T13:24:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 0x7f69a014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015a010 0x7f69a00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01505b0 con 0x7f69a00ce130 2024-02-02T13:24:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a0149e60 con 0x7f69a00b0700 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.752+0000 7f69a7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00ad4b0 con 0x7f69a015a010 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54776/0 (socket says 172.21.15.91:54776) 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 -- 172.21.15.91:0/2328151858 learned_addr learned my addr 172.21.15.91:0/2328151858 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 0x7f69a014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f69a5800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015a010 0x7f69a00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 -- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015a010 msgr2=0x7f69a00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015a010 0x7f69a00ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 -- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 msgr2=0x7f69a014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 0x7f69a014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 -- 172.21.15.91:0/2328151858 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900094d0 con 0x7f69a00ce130 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 0x7f69a014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.753+0000 7f6997c00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f698800c7c0 tx=0x7f698800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:59.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.754+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800aa50 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.754+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698800abb0 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.754+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988012420 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.754+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a00ad7f0 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.754+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a014c800 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.755+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a014ca20 con 0x7f69a00ce130 2024-02-02T13:24:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.756+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6988012580 con 0x7f69a00ce130 2024-02-02T13:24:59.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.757+0000 7f699d600640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 0x7f697806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:24:59.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.757+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6988088870 con 0x7f69a00ce130 2024-02-02T13:24:59.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.757+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 0x7f697806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:24:59.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.758+0000 7f69a4e00640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 0x7f697806c8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6990036000 tx=0x7f699003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:24:59.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.762+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6988053e60 con 0x7f69a00ce130 2024-02-02T13:24:59.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.990+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69a0066cf0 con 0x7f69a00ce130 2024-02-02T13:24:59.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.991+0000 7f699d600640 1 -- 172.21.15.91:0/2328151858 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6988016070 con 0x7f69a00ce130 2024-02-02T13:24:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:24:59.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.995+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 msgr2=0x7f697806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:59.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.995+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 0x7f697806c8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6990036000 tx=0x7f699003c040 comp rx=0 tx=0).stop 2024-02-02T13:24:59.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.995+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 msgr2=0x7f69a00afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:24:59.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.995+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f698800c7c0 tx=0x7f698800cc80 comp rx=0 tx=0).stop 2024-02-02T13:24:59.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 shutdown_connections 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f697806a430 0x7f697806c8f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015a010 0x7f69a00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a00ce130 0x7f69a00afaa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 --2- 172.21.15.91:0/2328151858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a00b0700 0x7f69a014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 >> 172.21.15.91:0/2328151858 conn(0x7f69a00cec80 msgr2=0x7f69a00b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 shutdown_connections 2024-02-02T13:24:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:24:59.996+0000 7f69a7800640 1 -- 172.21.15.91:0/2328151858 wait complete. 2024-02-02T13:25:00.016 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-02T13:25:00.016 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:00.017 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:25:00.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:00.017 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:00.063 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42625sec preferred_lft 42625sec 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:25:00.064 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:00.065 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-02T13:25:00.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add name ceph-brx type bridge 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip addr flush dev ceph-brx 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set ceph-brx up 2024-02-02T13:25:00.065 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-02T13:25:00.066 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:00.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:00.535 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:00.612 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:25:00.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:00.616 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:25:00.669 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:25:00.669 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:00.670 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:00.670 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:00.670 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:00.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:00.670 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:00.670 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:00.670 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:00.670 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:00.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:00.671 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:01.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:01.400 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:25:01.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:01.439 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list-id 2024-02-02T13:25:01.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:01.527 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:01.527 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:01.527 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:25:01.527 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-02T13:25:01.527 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:01.697 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-02T13:25:01.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:01.697 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-02T13:25:01.698 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:01.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:01.950 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:01.950 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:01.950 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:25:01.950 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set dev brx.0 master ceph-brx 2024-02-02T13:25:01.950 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:02.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:02.096 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:02.138 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:25:02.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:02.139 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:02.225 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:25:02.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:02.225 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:02.424 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:02.424 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:03.097 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:03.097 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:03.097 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:03.097 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:03.097 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:03.101 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:25:03.101 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:03.101 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:03.101 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:03.102 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:03.105 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:03.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:03.105 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:03.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:03.204 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:03.278 INFO:teuthology.orchestra.run.smithi091.stdout:client4522 2024-02-02T13:25:03.278 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:03.278 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:03.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:03.333 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:03.403 INFO:teuthology.orchestra.run.smithi091.stdout:client4522 2024-02-02T13:25:03.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:03.403 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:03.479 INFO:teuthology.orchestra.run.smithi091.stdout:client4522 2024-02-02T13:25:03.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:03.479 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client4522/status') 2024-02-02T13:25:03.589 INFO:teuthology.orchestra.run.smithi091.stdout:instance: client.4522 (3)172.21.15.91:0/2114330543 2024-02-02T13:25:03.590 INFO:teuthology.orchestra.run.smithi091.stdout:blocklisted: false 2024-02-02T13:25:03.590 INFO:teuthology.orchestra.run.smithi091.stderr:0+1 records in 2024-02-02T13:25:03.590 INFO:teuthology.orchestra.run.smithi091.stderr:0+1 records out 2024-02-02T13:25:03.590 INFO:teuthology.orchestra.run.smithi091.stderr:70 bytes copied, 6.962e-05 s, 1.0 MB/s 2024-02-02T13:25:03.594 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-02-02T13:25:03.594 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T13:25:03.595 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-02-02T13:25:03.595 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:25:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.805+0000 7efdc8a00640 1 Processor -- start 2024-02-02T13:25:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.806+0000 7efdc8a00640 1 -- start start 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc8a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efdc0151f90 0x7efdc015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc00b0e10 0x7efdc00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc8a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc0070f80 con 0x7efdc006f4f0 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc8a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc0152370 con 0x7efdc0151f90 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00cb010 con 0x7efdc00b0e10 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc6000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efdc0151f90 0x7efdc015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc6000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efdc0151f90 0x7efdc015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56044/0 (socket says 172.21.15.91:56044) 2024-02-02T13:25:03.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc6000640 1 -- 172.21.15.91:0/1378129552 learned_addr learned my addr 172.21.15.91:0/1378129552 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.807+0000 7efdc5600640 1 --1- 172.21.15.91:0/1378129552 >> v1:172.21.15.91:6789/0 conn(0x7efdc006f4f0 0x7efdc00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41996/0 (socket says 172.21.15.91:41996) 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.808+0000 7efdc6a00640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc00b0e10 0x7efdc00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.808+0000 7efdc6000640 1 -- 172.21.15.91:0/1378129552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc00b0e10 msgr2=0x7efdc00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.808+0000 7efdc6000640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc00b0e10 0x7efdc00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.808+0000 7efdc6000640 1 -- 172.21.15.91:0/1378129552 >> v1:172.21.15.91:6789/0 conn(0x7efdc006f4f0 legacy=0x7efdc00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.808+0000 7efdc6000640 1 -- 172.21.15.91:0/1378129552 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00cb2f0 con 0x7efdc0151f90 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.809+0000 7efdc6000640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc0151f90 0x7efdc015f5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c315566b4f671486 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.809+0000 7efdc4c00640 1 -- 172.21.15.91:0/1378129552 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8030840 con 0x7efdc0151f90 2024-02-02T13:25:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.809+0000 7efdc4c00640 1 -- 172.21.15.91:0/1378129552 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb80309a0 con 0x7efdc0151f90 2024-02-02T13:25:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.809+0000 7efdc4c00640 1 -- 172.21.15.91:0/1378129552 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8030d30 con 0x7efdc0151f90 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 -- 172.21.15.91:0/1378129552 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc0151f90 msgr2=0x7efdc015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc0151f90 0x7efdc015f5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802b980 comp rx=0 tx=0).stop 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 -- 172.21.15.91:0/1378129552 shutdown_connections 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc00b0e10 0x7efdc00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 --2- 172.21.15.91:0/1378129552 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc0151f90 0x7efdc015f5a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 -- 172.21.15.91:0/1378129552 >> 172.21.15.91:0/1378129552 conn(0x7efdc00cec80 msgr2=0x7efdc00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 -- 172.21.15.91:0/1378129552 shutdown_connections 2024-02-02T13:25:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.810+0000 7efdc8a00640 1 -- 172.21.15.91:0/1378129552 wait complete. 2024-02-02T13:25:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.812+0000 7efdc8a00640 1 Processor -- start 2024-02-02T13:25:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.812+0000 7efdc8a00640 1 -- start start 2024-02-02T13:25:03.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc00b0e10 0x7efdc00af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc0151f90 0x7efdc00afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc0155d70 con 0x7efdc006f4f0 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00c3200 con 0x7efdc00b0e10 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc015aee0 con 0x7efdc0151f90 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56060/0 (socket says 172.21.15.91:56060) 2024-02-02T13:25:03.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 -- 172.21.15.91:0/480181314 learned_addr learned my addr 172.21.15.91:0/480181314 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 -- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc0151f90 msgr2=0x7efdc00afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc0151f90 0x7efdc00afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 -- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc00b0e10 msgr2=0x7efdc00af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc00b0e10 0x7efdc00af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.813+0000 7efdc6000640 1 -- 172.21.15.91:0/480181314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80094d0 con 0x7efdc006f4f0 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdc6000640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdb802b9c0 tx=0x7efdb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8039c90 con 0x7efdc006f4f0 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb8039df0 con 0x7efdc006f4f0 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb80446c0 con 0x7efdc006f4f0 2024-02-02T13:25:03.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc015a640 con 0x7efdc006f4f0 2024-02-02T13:25:03.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.814+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc015ac00 con 0x7efdc006f4f0 2024-02-02T13:25:03.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.815+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc0073830 con 0x7efdc006f4f0 2024-02-02T13:25:03.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.816+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efdb802f390 con 0x7efdc006f4f0 2024-02-02T13:25:03.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.817+0000 7efdbe000640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 0x7efd9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:03.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.818+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdb803e070 con 0x7efdc006f4f0 2024-02-02T13:25:03.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.818+0000 7efdc5600640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 0x7efd9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:03.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.818+0000 7efdc5600640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 0x7efd9806c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efdc00cc030 tx=0x7efdb00079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:03.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:03.822+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efdb8076aa0 con 0x7efdc006f4f0 2024-02-02T13:25:04.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.049+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efdc00711d0 con 0x7efdc006f4f0 2024-02-02T13:25:04.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.050+0000 7efdbe000640 1 -- 172.21.15.91:0/480181314 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7efdb807a0c0 con 0x7efdc006f4f0 2024-02-02T13:25:04.052 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 msgr2=0x7efd9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 0x7efd9806c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efdc00cc030 tx=0x7efdb00079c0 comp rx=0 tx=0).stop 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 msgr2=0x7efdc00c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdb802b9c0 tx=0x7efdb8004730 comp rx=0 tx=0).stop 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 shutdown_connections 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.052+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd9806a430 0x7efd9806c8f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdc0151f90 0x7efdc00afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:04.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc00b0e10 0x7efdc00af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:04.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 --2- 172.21.15.91:0/480181314 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc006f4f0 0x7efdc00c2cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:04.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 >> 172.21.15.91:0/480181314 conn(0x7efdc00cec80 msgr2=0x7efdc00cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:04.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 shutdown_connections 2024-02-02T13:25:04.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:04.053+0000 7efdc8a00640 1 -- 172.21.15.91:0/480181314 wait complete. 2024-02-02T13:25:04.075 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:04.076 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-02-02T13:25:04.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.076 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:25:04.126 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:04.127 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42621sec preferred_lft 42621sec 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:04.128 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:25:04.129 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:25:04.129 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.130 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:04.215 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:25:04.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.220 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:25:04.272 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:25:04.272 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:04.272 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:04.272 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:04.272 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:04.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:04.273 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:04.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.445 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:25:04.493 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:25:04.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.494 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list-id 2024-02-02T13:25:04.540 INFO:teuthology.orchestra.run.smithi091.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-02-02T13:25:04.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.541 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:04.542 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:04.542 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:25:04.542 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-02-02T13:25:04.542 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:04.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.679 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-02-02T13:25:04.764 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host proto kernel_lo 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 92:de:ce:16:f7:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::90de:ceff:fe16:f7ff/64 scope link proto kernel_ll 2024-02-02T13:25:04.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.769 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host proto kernel_lo 2024-02-02T13:25:04.855 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 92:de:ce:16:f7:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::90de:ceff:fe16:f7ff/64 scope link proto kernel_ll 2024-02-02T13:25:04.856 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:04.860 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-02-02T13:25:04.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-02-02T13:25:04.860 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-02-02T13:25:04.861 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:05.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.123 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:05.124 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:05.124 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.1 up 2024-02-02T13:25:05.124 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set dev brx.1 master ceph-brx 2024-02-02T13:25:05.124 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:05.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:05.288 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-02-02T13:25:05.337 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-02-02T13:25:05.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:05.338 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-02-02T13:25:05.388 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-02-02T13:25:05.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.388 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:05.518 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:05.519 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:05.552 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:05.552 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:05.552 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:05.552 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 2@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:05.552 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:05.555 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-02-02T13:25:05.555 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:05.555 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:05.555 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:05.555 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:05.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:05.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.560 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-02-02T13:25:05.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.637 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:05.690 INFO:teuthology.orchestra.run.smithi091.stdout:client4668 2024-02-02T13:25:05.690 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:05.690 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:05.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.739 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:05.839 INFO:teuthology.orchestra.run.smithi091.stdout:client4668 2024-02-02T13:25:05.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.839 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:05.894 INFO:teuthology.orchestra.run.smithi091.stdout:client4668 2024-02-02T13:25:05.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:05.895 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client4668/status') 2024-02-02T13:25:05.983 INFO:teuthology.orchestra.run.smithi091.stdout:instance: client.4668 (3)192.168.0.2:0/3964231809 2024-02-02T13:25:05.984 INFO:teuthology.orchestra.run.smithi091.stdout:blocklisted: false 2024-02-02T13:25:05.984 INFO:teuthology.orchestra.run.smithi091.stderr:0+1 records in 2024-02-02T13:25:05.984 INFO:teuthology.orchestra.run.smithi091.stderr:0+1 records out 2024-02-02T13:25:05.984 INFO:teuthology.orchestra.run.smithi091.stderr:69 bytes copied, 6.5524e-05 s, 1.1 MB/s 2024-02-02T13:25:05.988 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-02-02T13:25:05.988 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T13:25:05.988 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:05.988 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:25:06.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.198+0000 7fe989a00640 1 Processor -- start 2024-02-02T13:25:06.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.199+0000 7fe989a00640 1 -- start start 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe989a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9840b8a50 0x7fe9840d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe989a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9840cd930 con 0x7fe9840d0070 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe989a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9840d1680 con 0x7fe9840b2260 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe989a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9840d1960 con 0x7fe9840b8a50 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe982a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe9840d0070 0x7fe9840d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41998/0 (socket says 172.21.15.91:41998) 2024-02-02T13:25:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.200+0000 7fe982a00640 1 -- 172.21.15.91:0/2831927274 learned_addr learned my addr 172.21.15.91:0/2831927274 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176136798 0 0) 0x7fe974001110 con 0x7fe9840d0070 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe964003520 con 0x7fe9840d0070 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe983400640 1 --2- 172.21.15.91:0/2831927274 >> v2:172.21.15.113:3300/0 conn(0x7fe9840b8a50 0x7fe9840d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337747408 0 0) 0x7fe964003520 con 0x7fe9840d0070 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 >> v2:172.21.15.113:3300/0 conn(0x7fe9840b8a50 msgr2=0x7fe9840d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 --2- 172.21.15.91:0/2831927274 >> v2:172.21.15.113:3300/0 conn(0x7fe9840b8a50 0x7fe9840d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 >> v1:172.21.15.113:6789/0 conn(0x7fe9840b2260 legacy=0x7fe9840b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9840d1c40 con 0x7fe9840d0070 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe974001800 con 0x7fe9840d0070 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 >> v1:172.21.15.91:6789/0 conn(0x7fe9840d0070 legacy=0x7fe9840d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 0x7fe964003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.201+0000 7fe981600640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9640047c0 0x7fe964004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe981600640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964005180 0x7fe96400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840d1960 con 0x7fe964003970 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96400c0d0 con 0x7fe964005180 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96400c290 con 0x7fe9640047c0 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 0x7fe964003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe983400640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964005180 0x7fe96400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 -- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9640047c0 msgr2=0x7fe964004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982000640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9640047c0 0x7fe964004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9640047c0 0x7fe964004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 -- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964005180 msgr2=0x7fe96400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964005180 0x7fe96400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982a00640 1 -- 172.21.15.91:0/2831927274 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964003520 con 0x7fe964003970 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe983400640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964005180 0x7fe96400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe982000640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9640047c0 0x7fe964004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.202+0000 7fe983400640 1 -- 172.21.15.91:0/2831927274 reap_dead start 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.203+0000 7fe982a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 0x7fe964003d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c240 tx=0x7fe9640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b0adcc3803a49b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974005ad0 con 0x7fe964003970 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974005c50 con 0x7fe964003970 2024-02-02T13:25:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe981600640 1 -- 172.21.15.91:0/2831927274 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97402c8b0 con 0x7fe964003970 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 -- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 msgr2=0x7fe964003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 0x7fe964003d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c240 tx=0x7fe9640057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 -- 172.21.15.91:0/2831927274 shutdown_connections 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 --2- 172.21.15.91:0/2831927274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe964003970 0x7fe964003d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 -- 172.21.15.91:0/2831927274 >> 172.21.15.91:0/2831927274 conn(0x7fe984070660 msgr2=0x7fe984070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 -- 172.21.15.91:0/2831927274 shutdown_connections 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.204+0000 7fe989a00640 1 -- 172.21.15.91:0/2831927274 wait complete. 2024-02-02T13:25:06.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.206+0000 7fe989a00640 1 Processor -- start 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.206+0000 7fe989a00640 1 -- start start 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 0x7fe98415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9841f3920 0x7fe98415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840cb9b0 con 0x7fe98415e550 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984157f50 con 0x7fe964003970 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe989a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98415f790 con 0x7fe9841f3920 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.207+0000 7fe983400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56082/0 (socket says 172.21.15.91:56082) 2024-02-02T13:25:06.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 -- 172.21.15.91:0/2082183642 learned_addr learned my addr 172.21.15.91:0/2082183642 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:06.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe982000640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9841f3920 0x7fe98415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe982a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 0x7fe98415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 -- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9841f3920 msgr2=0x7fe98415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9841f3920 0x7fe98415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 -- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 msgr2=0x7fe98415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 0x7fe98415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 -- 172.21.15.91:0/2082183642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98415fa70 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe982a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 0x7fe98415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.208+0000 7fe983400640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9840cd710 tx=0x7fe97c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.209+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004230 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.209+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c0043b0 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.209+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004520 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.209+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9840cb2f0 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.209+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98415f480 con 0x7fe98415e550 2024-02-02T13:25:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.210+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9840cbaf0 con 0x7fe98415e550 2024-02-02T13:25:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.211+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe97c003850 con 0x7fe98415e550 2024-02-02T13:25:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.212+0000 7fe97aa00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 0x7fe95c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.212+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe97c087f00 con 0x7fe98415e550 2024-02-02T13:25:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.212+0000 7fe982a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 0x7fe95c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.213+0000 7fe982a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 0x7fe95c06c920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe974034000 tx=0x7fe974009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.216+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe9840cbaf0 con 0x7fe98415e550 2024-02-02T13:25:06.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.448+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9840cbaf0 con 0x7fe98415e550 2024-02-02T13:25:06.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.449+0000 7fe97aa00640 1 -- 172.21.15.91:0/2082183642 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe9840cbaf0 con 0x7fe98415e550 2024-02-02T13:25:06.451 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 msgr2=0x7fe95c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 0x7fe95c06c920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe974034000 tx=0x7fe974009030 comp rx=0 tx=0).stop 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 msgr2=0x7fe9840b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9840cd710 tx=0x7fe97c009860 comp rx=0 tx=0).stop 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 shutdown_connections 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe95c06a460 0x7fe95c06c920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9841f3920 0x7fe98415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98415e550 0x7fe9840b7c00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 --2- 172.21.15.91:0/2082183642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe964003970 0x7fe98415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:06.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.451+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 >> 172.21.15.91:0/2082183642 conn(0x7fe984070660 msgr2=0x7fe9840b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:06.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.452+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 shutdown_connections 2024-02-02T13:25:06.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:06.452+0000 7fe989a00640 1 -- 172.21.15.91:0/2082183642 wait complete. 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:06.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:25:06.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:06.471 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42659sec preferred_lft 42659sec 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:25:06.528 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:06.531 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-02T13:25:06.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-02-02T13:25:06.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-02T13:25:06.532 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:07.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.005 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:07.086 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:25:07.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.090 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:07.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:07.145 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:07.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.820 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:25:07.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.874 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-02-02T13:25:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:07.926 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:07.927 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:07.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:25:07.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-02T13:25:07.927 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:08.063 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-02T13:25:08.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:08.064 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:08.064 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:08.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-02T13:25:08.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-02T13:25:08.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-02T13:25:08.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-02T13:25:08.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-02T13:25:08.065 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:08.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:08.351 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:08.351 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:08.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:25:08.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-02-02T13:25:08.351 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:08.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:08.512 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:25:08.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:08.559 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:08.606 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:25:08.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:08.606 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:09.479 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:09.479 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:09.479 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:09.479 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:09.479 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:09.487 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:25:09.487 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:09.487 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:09.488 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:09.488 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:09.491 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:09.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:09.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:09.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:09.577 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:09.630 INFO:teuthology.orchestra.run.smithi113.stdout:client4421 2024-02-02T13:25:09.630 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:09.630 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:09.678 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:09.775 INFO:teuthology.orchestra.run.smithi113.stdout:client4421 2024-02-02T13:25:09.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:09.775 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:09.833 INFO:teuthology.orchestra.run.smithi113.stdout:client4421 2024-02-02T13:25:09.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:09.833 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client4421/status') 2024-02-02T13:25:09.923 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4421 (3)192.168.0.1:0/3320044294 2024-02-02T13:25:09.923 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-02-02T13:25:09.923 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-02-02T13:25:09.923 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-02-02T13:25:09.923 INFO:teuthology.orchestra.run.smithi113.stderr:69 bytes copied, 7.2249e-05 s, 955 kB/s 2024-02-02T13:25:09.927 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-02-02T13:25:09.927 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T13:25:09.927 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-02-02T13:25:09.928 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:25:10.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.143+0000 7fe8d5800640 1 Processor -- start 2024-02-02T13:25:10.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.143+0000 7fe8d5800640 1 -- start start 2024-02-02T13:25:10.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.144+0000 7fe8d5800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8d00ce130 0x7fe8d00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:10.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8d5800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00cc7c0 con 0x7fe8d00b0700 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8d5800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00ce930 con 0x7fe8d015a010 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8d5800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d00cd640 con 0x7fe8d00ce130 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8d00ce130 0x7fe8d00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8d00ce130 0x7fe8d00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47418/0 (socket says 172.21.15.91:47418) 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8cf400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe8d00b0700 0x7fe8d006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34134/0 (socket says 172.21.15.91:34134) 2024-02-02T13:25:10.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 -- 172.21.15.91:0/1109619546 learned_addr learned my addr 172.21.15.91:0/1109619546 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 -- 172.21.15.91:0/1109619546 >> v1:172.21.15.113:6789/0 conn(0x7fe8d015a010 legacy=0x7fe8d00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 -- 172.21.15.91:0/1109619546 >> v1:172.21.15.91:6789/0 conn(0x7fe8d00b0700 legacy=0x7fe8d006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8ce000640 1 -- 172.21.15.91:0/1109619546 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d006c890 con 0x7fe8d00ce130 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.145+0000 7fe8cd600640 1 -- 172.21.15.91:0/1109619546 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824646728 0 0) 0x7fe8d00cc7c0 con 0x7fe8d00b0700 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.146+0000 7fe8ce000640 1 --2- 172.21.15.91:0/1109619546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d00c9670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009870 tx=0x7fe8b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee7dbce3480a6f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8cd600640 1 -- 172.21.15.91:0/1109619546 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8030860 con 0x7fe8d00ce130 2024-02-02T13:25:10.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8cd600640 1 -- 172.21.15.91:0/1109619546 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80309c0 con 0x7fe8d00ce130 2024-02-02T13:25:10.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8cd600640 1 -- 172.21.15.91:0/1109619546 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8030d50 con 0x7fe8d00ce130 2024-02-02T13:25:10.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 -- 172.21.15.91:0/1109619546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 msgr2=0x7fe8d00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 --2- 172.21.15.91:0/1109619546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d00c9670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009870 tx=0x7fe8b802b9a0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 -- 172.21.15.91:0/1109619546 shutdown_connections 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 --2- 172.21.15.91:0/1109619546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d00c9670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 -- 172.21.15.91:0/1109619546 >> 172.21.15.91:0/1109619546 conn(0x7fe8d00cec80 msgr2=0x7fe8d00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 -- 172.21.15.91:0/1109619546 shutdown_connections 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.147+0000 7fe8d5800640 1 -- 172.21.15.91:0/1109619546 wait complete. 2024-02-02T13:25:10.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.149+0000 7fe8d5800640 1 Processor -- start 2024-02-02T13:25:10.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.149+0000 7fe8d5800640 1 -- start start 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8d00b0700 0x7fe8d01f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8d015a010 0x7fe8d00c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c36d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c5df0 con 0x7fe8d015a010 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8d5800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c60d0 con 0x7fe8d00b0700 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.150+0000 7fe8ce000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47434/0 (socket says 172.21.15.91:47434) 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 -- 172.21.15.91:0/2796926788 learned_addr learned my addr 172.21.15.91:0/2796926788 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 -- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8d00b0700 msgr2=0x7fe8d01f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8d00b0700 0x7fe8d01f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 -- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8d015a010 msgr2=0x7fe8d00c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8d015a010 0x7fe8d00c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 -- 172.21.15.91:0/2796926788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80094d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.151+0000 7fe8ce000640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009870 tx=0x7fe8b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:10.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.152+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8030810 con 0x7fe8d00ce130 2024-02-02T13:25:10.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.152+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8043400 con 0x7fe8d00ce130 2024-02-02T13:25:10.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.152+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d00c63b0 con 0x7fe8d00ce130 2024-02-02T13:25:10.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.152+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80437b0 con 0x7fe8d00ce130 2024-02-02T13:25:10.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.152+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d01f0980 con 0x7fe8d00ce130 2024-02-02T13:25:10.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.153+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d00c68d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.154+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe8b8043910 con 0x7fe8d00ce130 2024-02-02T13:25:10.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.155+0000 7fe8c6a00640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:10.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.155+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8b803e070 con 0x7fe8d00ce130 2024-02-02T13:25:10.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.155+0000 7fe8cea00640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:10.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.155+0000 7fe8cea00640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 0x7fe8a806c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0006fd0 tx=0x7fe8c0006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.159+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe8d00c68d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.386+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8d00c68d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.387+0000 7fe8c6a00640 1 -- 172.21.15.91:0/2796926788 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8d00c68d0 con 0x7fe8d00ce130 2024-02-02T13:25:10.389 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:25:10.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.390+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 msgr2=0x7fe8a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.390+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 0x7fe8a806c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0006fd0 tx=0x7fe8c0006bc0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.390+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 msgr2=0x7fe8d0158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:10.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.390+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009870 tx=0x7fe8b8004730 comp rx=0 tx=0).stop 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 shutdown_connections 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8d015a010 0x7fe8d00c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8d00ce130 0x7fe8d0158340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 --2- 172.21.15.91:0/2796926788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8d00b0700 0x7fe8d01f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 >> 172.21.15.91:0/2796926788 conn(0x7fe8d00cec80 msgr2=0x7fe8d015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 shutdown_connections 2024-02-02T13:25:10.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:10.391+0000 7fe8d5800640 1 -- 172.21.15.91:0/2796926788 wait complete. 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:10.410 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-02-02T13:25:10.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.411 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:25:10.465 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:25:10.466 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42655sec preferred_lft 42655sec 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.467 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:10.468 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:25:10.468 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:25:10.468 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:10.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.468 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:10.551 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:25:10.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.555 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:25:10.608 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:25:10.608 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:10.608 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:10.609 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:10.609 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:10.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:10.610 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:10.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.790 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:25:10.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:25:10.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.841 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-02-02T13:25:10.893 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-02-02T13:25:10.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:10.895 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:10.895 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:10.895 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:25:10.895 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-02-02T13:25:10.895 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:11.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.038 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-02-02T13:25:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host proto kernel_lo 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether de:7e:5e:47:6a:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::dc7e:5eff:fe47:6a15/64 scope link proto kernel_ll 2024-02-02T13:25:11.128 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.132 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-02-02T13:25:11.225 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:11.225 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host proto kernel_lo 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether de:7e:5e:47:6a:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::dc7e:5eff:fe47:6a15/64 scope link proto kernel_ll 2024-02-02T13:25:11.226 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:11.230 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-02-02T13:25:11.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.230 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:11.230 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:11.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-02-02T13:25:11.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-02-02T13:25:11.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-02-02T13:25:11.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-02-02T13:25:11.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-02-02T13:25:11.231 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:11.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.502 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:11.502 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:11.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2024-02-02T13:25:11.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2024-02-02T13:25:11.502 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:11.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:11.668 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-02-02T13:25:11.714 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-02-02T13:25:11.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:11.714 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-02-02T13:25:11.807 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-02-02T13:25:11.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:11.935 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:11.935 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:11.967 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:11.967 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:11.967 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:11.967 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 3@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:11.968 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:11.969 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-02-02T13:25:11.969 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:11.969 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:11.970 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:11.970 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:11.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:11.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:11.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-02-02T13:25:12.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:12.051 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:12.102 INFO:teuthology.orchestra.run.smithi113.stdout:client4424 2024-02-02T13:25:12.102 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:12.103 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:12.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:12.151 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:12.249 INFO:teuthology.orchestra.run.smithi113.stdout:client4424 2024-02-02T13:25:12.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:12.249 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:12.306 INFO:teuthology.orchestra.run.smithi113.stdout:client4424 2024-02-02T13:25:12.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:12.306 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client4424/status') 2024-02-02T13:25:12.397 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4424 (3)192.168.0.2:0/3644353690 2024-02-02T13:25:12.397 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-02-02T13:25:12.398 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-02-02T13:25:12.398 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-02-02T13:25:12.398 INFO:teuthology.orchestra.run.smithi113.stderr:69 bytes copied, 6.3741e-05 s, 1.1 MB/s 2024-02-02T13:25:12.402 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-02-02T13:25:12.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.659+0000 7f274f400640 1 Processor -- start 2024-02-02T13:25:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.659+0000 7f274f400640 1 -- start start 2024-02-02T13:25:12.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.660+0000 7f274f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f274806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.660+0000 7f274f400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274815d870 con 0x7f27480ac8d0 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f274f400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274806bce0 con 0x7f27480b0e10 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f274f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2748161220 con 0x7f274815d490 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f273fc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f27480ac8d0 0x7f27480b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34146/0 (socket says 172.21.15.91:34146) 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f273fc00640 1 -- 172.21.15.91:0/3936766139 learned_addr learned my addr 172.21.15.91:0/3936766139 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f274806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472944914 0 0) 0x7f274815d870 con 0x7f27480ac8d0 2024-02-02T13:25:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.661+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2728003520 con 0x7f27480ac8d0 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 282996686 0 0) 0x7f2728003520 con 0x7f27480ac8d0 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 msgr2=0x7f274806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f274806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 >> v1:172.21.15.113:6789/0 conn(0x7f27480b0e10 legacy=0x7f27480c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480ad490 con 0x7f27480ac8d0 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2730001670 con 0x7f27480ac8d0 2024-02-02T13:25:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f274806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 >> v1:172.21.15.91:6789/0 conn(0x7f27480ac8d0 legacy=0x7f27480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27280039a0 0x7f2728003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f27280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728004d00 0x7f2728005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748161220 con 0x7f27280039a0 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272800ba30 con 0x7f2728004d00 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272800bba0 con 0x7f274815d490 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.662+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27280039a0 0x7f2728003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274ca00640 1 -- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 msgr2=0x7f27280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f273fc00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f27280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f27280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274ca00640 1 -- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728004d00 msgr2=0x7f2728005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274d400640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728004d00 0x7f2728005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728004d00 0x7f2728005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274ca00640 1 -- 172.21.15.91:0/3936766139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009400 con 0x7f27280039a0 2024-02-02T13:25:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f273fc00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f274815d490 0x7f27280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.663+0000 7f274d400640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728004d00 0x7f2728005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.664+0000 7f274ca00640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27280039a0 0x7f2728003dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2738009c90 tx=0x7f27280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a72113f63e3585 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.664+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273802c8f0 con 0x7f27280039a0 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.664+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273802ca50 con 0x7f27280039a0 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.664+0000 7f2747e00640 1 -- 172.21.15.91:0/3936766139 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273802ce00 con 0x7f27280039a0 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 -- 172.21.15.91:0/3936766139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27280039a0 msgr2=0x7f2728003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 --2- 172.21.15.91:0/3936766139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27280039a0 0x7f2728003dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2738009c90 tx=0x7f27280057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274d400640 1 -- 172.21.15.91:0/3936766139 reap_dead start 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 -- 172.21.15.91:0/3936766139 shutdown_connections 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 -- 172.21.15.91:0/3936766139 >> 172.21.15.91:0/3936766139 conn(0x7f27480cec80 msgr2=0x7f27480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 -- 172.21.15.91:0/3936766139 shutdown_connections 2024-02-02T13:25:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.665+0000 7f274f400640 1 -- 172.21.15.91:0/3936766139 wait complete. 2024-02-02T13:25:12.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.667+0000 7f274f400640 1 Processor -- start 2024-02-02T13:25:12.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.667+0000 7f274f400640 1 -- start start 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27280039a0 0x7f27480c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 0x7f27480af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480afa50 con 0x7f27480b2230 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480c6e30 con 0x7f27280039a0 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27480c7110 con 0x7f27480af090 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f273fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f273fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47454/0 (socket says 172.21.15.91:47454) 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f273fc00640 1 -- 172.21.15.91:0/1189562969 learned_addr learned my addr 172.21.15.91:0/1189562969 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274ca00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27280039a0 0x7f27480c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.668+0000 7f274d400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 0x7f27480af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 -- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 msgr2=0x7f27480af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 0x7f27480af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 -- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27280039a0 msgr2=0x7f27480c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27280039a0 0x7f27480c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 -- 172.21.15.91:0/1189562969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480c73f0 con 0x7f27480b2230 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f274d400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 0x7f27480af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f273fc00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f273000d8b0 tx=0x7f273000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300074e0 con 0x7f27480b2230 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2730007640 con 0x7f27480b2230 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300049e0 con 0x7f27480b2230 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27480b2f40 con 0x7f27480b2230 2024-02-02T13:25:12.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.669+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274815d870 con 0x7f27480b2230 2024-02-02T13:25:12.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.671+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2748066cf0 con 0x7f27480b2230 2024-02-02T13:25:12.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.672+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2730004b40 con 0x7f27480b2230 2024-02-02T13:25:12.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.672+0000 7f2745600640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 0x7f272006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:12.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.672+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2730088410 con 0x7f27480b2230 2024-02-02T13:25:12.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.673+0000 7f274ca00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 0x7f272006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:12.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.673+0000 7f274ca00640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 0x7f272006c8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f2738030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:12.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.677+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f27300574b0 con 0x7f27480b2230 2024-02-02T13:25:12.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.868+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27480b4c00 con 0x7f27480b2230 2024-02-02T13:25:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.869+0000 7f2745600640 1 -- 172.21.15.91:0/1189562969 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f2730057040 con 0x7f27480b2230 2024-02-02T13:25:12.871 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:12.872 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 msgr2=0x7f272006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 0x7f272006c8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f2738030000 comp rx=0 tx=0).stop 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 msgr2=0x7f274815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f273000d8b0 tx=0x7f273000dd70 comp rx=0 tx=0).stop 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 shutdown_connections 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f272006a430 0x7f272006c8f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27480af090 0x7f27480af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27480b2230 0x7f274815c680 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 --2- 172.21.15.91:0/1189562969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27280039a0 0x7f27480c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 >> 172.21.15.91:0/1189562969 conn(0x7f27480cec80 msgr2=0x7f2748158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 shutdown_connections 2024-02-02T13:25:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:12.872+0000 7f274f400640 1 -- 172.21.15.91:0/1189562969 wait complete. 2024-02-02T13:25:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:12.893 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:13.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.103+0000 7f73ce800640 1 Processor -- start 2024-02-02T13:25:13.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.103+0000 7f73ce800640 1 -- start start 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.104+0000 7f73ce800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 0x7f73c80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.104+0000 7f73ce800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f73c80ce130 0x7f73c80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73ce800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80cc7c0 con 0x7f73c80cd470 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73ce800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80ce930 con 0x7f73c80ce130 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73ce800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c806c890 con 0x7f73c815a010 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f73c80ce130 0x7f73c80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f73c80ce130 0x7f73c80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47462/0 (socket says 172.21.15.91:47462) 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 -- 172.21.15.91:0/2903525091 learned_addr learned my addr 172.21.15.91:0/2903525091 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73c7400640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 0x7f73c80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 -- 172.21.15.91:0/2903525091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 msgr2=0x7f73c80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 0x7f73c80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 -- 172.21.15.91:0/2903525091 >> v1:172.21.15.113:6789/0 conn(0x7f73c80cd470 legacy=0x7f73c80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:13.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73be800640 1 -- 172.21.15.91:0/2903525091 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c806cb70 con 0x7f73c80ce130 2024-02-02T13:25:13.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.105+0000 7f73c7400640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 0x7f73c80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:13.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.106+0000 7f73be800640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c80ce130 0x7f73c80ccf30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e50 tx=0x7f73b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305b90c0d5a4f49 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.107+0000 7f73c6a00640 1 -- 172.21.15.91:0/2903525091 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b002ca70 con 0x7f73c80ce130 2024-02-02T13:25:13.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.107+0000 7f73c6a00640 1 -- 172.21.15.91:0/2903525091 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b002cbd0 con 0x7f73c80ce130 2024-02-02T13:25:13.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.107+0000 7f73c6a00640 1 -- 172.21.15.91:0/2903525091 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b00343f0 con 0x7f73c80ce130 2024-02-02T13:25:13.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 -- 172.21.15.91:0/2903525091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c80ce130 msgr2=0x7f73c80ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c80ce130 0x7f73c80ccf30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e50 tx=0x7f73b002bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 -- 172.21.15.91:0/2903525091 shutdown_connections 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c80ce130 0x7f73c80ccf30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 --2- 172.21.15.91:0/2903525091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c815a010 0x7f73c80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 -- 172.21.15.91:0/2903525091 >> 172.21.15.91:0/2903525091 conn(0x7f73c80cec80 msgr2=0x7f73c80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 -- 172.21.15.91:0/2903525091 shutdown_connections 2024-02-02T13:25:13.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.108+0000 7f73ce800640 1 -- 172.21.15.91:0/2903525091 wait complete. 2024-02-02T13:25:13.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.109+0000 7f73ce800640 1 Processor -- start 2024-02-02T13:25:13.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 -- start start 2024-02-02T13:25:13.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c80ce130 0x7f73c80c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c8147740 0x7f73c80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81472b0 con 0x7f73c815a010 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81f2ba0 con 0x7f73c8147740 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.110+0000 7f73ce800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8146c60 con 0x7f73c80ce130 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47474/0 (socket says 172.21.15.91:47474) 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 -- 172.21.15.91:0/3906604136 learned_addr learned my addr 172.21.15.91:0/3906604136 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 -- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c80ce130 msgr2=0x7f73c80c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73c7400640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c80ce130 0x7f73c80c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c80ce130 0x7f73c80c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73c7e00640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c8147740 0x7f73c80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 -- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c8147740 msgr2=0x7f73c80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c8147740 0x7f73c80aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.111+0000 7f73be800640 1 -- 172.21.15.91:0/3906604136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b0009b30 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73be800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e50 tx=0x7f73b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b003e450 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b003e5f0 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b003e9a0 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c8146f40 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.112+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81f1e90 con 0x7f73c815a010 2024-02-02T13:25:13.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.113+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8066cf0 con 0x7f73c815a010 2024-02-02T13:25:13.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.114+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f73b003eb00 con 0x7f73c815a010 2024-02-02T13:25:13.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.115+0000 7f73bfe00640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 0x7f73a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.115+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73b00ab810 con 0x7f73c815a010 2024-02-02T13:25:13.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.115+0000 7f73c7400640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 0x7f73a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.116+0000 7f73c7400640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 0x7f73a006c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b840 tx=0x7f73b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.119+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f73b0076f20 con 0x7f73c815a010 2024-02-02T13:25:13.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.306+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73c80b4c00 con 0x7f73c815a010 2024-02-02T13:25:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.307+0000 7f73bfe00640 1 -- 172.21.15.91:0/3906604136 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73b003edb0 con 0x7f73c815a010 2024-02-02T13:25:13.309 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 msgr2=0x7f73a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 0x7f73a006c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b840 tx=0x7f73b8008040 comp rx=0 tx=0).stop 2024-02-02T13:25:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 msgr2=0x7f73c80ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e50 tx=0x7f73b0004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 shutdown_connections 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73a006a430 0x7f73a006c8f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c8147740 0x7f73c80aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73c815a010 0x7f73c80ae770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 --2- 172.21.15.91:0/3906604136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c80ce130 0x7f73c80c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 >> 172.21.15.91:0/3906604136 conn(0x7f73c80cec80 msgr2=0x7f73c815d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.310+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 shutdown_connections 2024-02-02T13:25:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.311+0000 7f73ce800640 1 -- 172.21.15.91:0/3906604136 wait complete. 2024-02-02T13:25:13.331 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:13.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.542+0000 7f6272800640 1 Processor -- start 2024-02-02T13:25:13.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.542+0000 7f6272800640 1 -- start start 2024-02-02T13:25:13.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f6272800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f626c15d490 0x7f626c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f6272800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c0b0e10 0x7f626c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f6272800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c15d870 con 0x7f626c0ac8d0 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f6272800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c0ad2a0 con 0x7f626c15d490 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f6272800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c06bce0 con 0x7f626c0b0e10 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f626b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f626c15d490 0x7f626c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f626aa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f626c0ac8d0 0x7f626c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34148/0 (socket says 172.21.15.91:34148) 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f626b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f626c15d490 0x7f626c06c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47490/0 (socket says 172.21.15.91:47490) 2024-02-02T13:25:13.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.543+0000 7f626aa00640 1 -- 172.21.15.91:0/3350320932 learned_addr learned my addr 172.21.15.91:0/3350320932 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626be00640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c0b0e10 0x7f626c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626b400640 1 -- 172.21.15.91:0/3350320932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c0b0e10 msgr2=0x7f626c0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626b400640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c0b0e10 0x7f626c0acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626b400640 1 -- 172.21.15.91:0/3350320932 >> v1:172.21.15.91:6789/0 conn(0x7f626c0ac8d0 legacy=0x7f626c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626b400640 1 -- 172.21.15.91:0/3350320932 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c161220 con 0x7f626c15d490 2024-02-02T13:25:13.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.544+0000 7f626a000640 1 -- 172.21.15.91:0/3350320932 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733929636 0 0) 0x7f626c15d870 con 0x7f626c0ac8d0 2024-02-02T13:25:13.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.545+0000 7f626b400640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c15d490 0x7f626c06c170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f625c0097a0 tx=0x7f625c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b240a28719dcd40e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.545+0000 7f626a000640 1 -- 172.21.15.91:0/3350320932 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c031830 con 0x7f626c15d490 2024-02-02T13:25:13.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.545+0000 7f626a000640 1 -- 172.21.15.91:0/3350320932 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c031990 con 0x7f626c15d490 2024-02-02T13:25:13.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.545+0000 7f626a000640 1 -- 172.21.15.91:0/3350320932 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c031d20 con 0x7f626c15d490 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 -- 172.21.15.91:0/3350320932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c15d490 msgr2=0x7f626c06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c15d490 0x7f626c06c170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f625c0097a0 tx=0x7f625c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 -- 172.21.15.91:0/3350320932 shutdown_connections 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c0b0e10 0x7f626c0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 --2- 172.21.15.91:0/3350320932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c15d490 0x7f626c06c170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 -- 172.21.15.91:0/3350320932 >> 172.21.15.91:0/3350320932 conn(0x7f626c0cec80 msgr2=0x7f626c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 -- 172.21.15.91:0/3350320932 shutdown_connections 2024-02-02T13:25:13.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.546+0000 7f6272800640 1 -- 172.21.15.91:0/3350320932 wait complete. 2024-02-02T13:25:13.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.548+0000 7f6272800640 1 Processor -- start 2024-02-02T13:25:13.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.548+0000 7f6272800640 1 -- start start 2024-02-02T13:25:13.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f626c0b0e10 0x7f626c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c15d490 0x7f626c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c0c3b10 con 0x7f626c0ac8d0 2024-02-02T13:25:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c0c2f00 con 0x7f626c0b0e10 2024-02-02T13:25:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f6272800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c0c31e0 con 0x7f626c15d490 2024-02-02T13:25:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47498/0 (socket says 172.21.15.91:47498) 2024-02-02T13:25:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 -- 172.21.15.91:0/2128986848 learned_addr learned my addr 172.21.15.91:0/2128986848 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 -- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c15d490 msgr2=0x7f626c14e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c15d490 0x7f626c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 -- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f626c0b0e10 msgr2=0x7f626c0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f626c0b0e10 0x7f626c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.549+0000 7f626b400640 1 -- 172.21.15.91:0/2128986848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c009400 con 0x7f626c0ac8d0 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f626b400640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f625c0098c0 tx=0x7f625c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c03ba70 con 0x7f626c0ac8d0 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c03bbf0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0453f0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c0c34c0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.550+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c0c2720 con 0x7f626c0ac8d0 2024-02-02T13:25:13.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.551+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c066cf0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.552+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f625c04e450 con 0x7f626c0ac8d0 2024-02-02T13:25:13.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.553+0000 7f6263400640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 0x7f624406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.553+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f625c0aad70 con 0x7f626c0ac8d0 2024-02-02T13:25:13.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.553+0000 7f626aa00640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 0x7f624406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.554+0000 7f626aa00640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 0x7f624406c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6254007fd0 tx=0x7f6254007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.557+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f625c0763e0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.748+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f626c0718b0 con 0x7f626c0ac8d0 2024-02-02T13:25:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.750+0000 7f6263400640 1 -- 172.21.15.91:0/2128986848 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f625c035020 con 0x7f626c0ac8d0 2024-02-02T13:25:13.751 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 msgr2=0x7f624406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 0x7f624406c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6254007fd0 tx=0x7f6254007960 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 msgr2=0x7f626c0c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f625c0098c0 tx=0x7f625c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 shutdown_connections 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f624406a430 0x7f624406c8f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f626c15d490 0x7f626c14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f626c0b0e10 0x7f626c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 --2- 172.21.15.91:0/2128986848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f626c0ac8d0 0x7f626c0c3fa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 >> 172.21.15.91:0/2128986848 conn(0x7f626c0cec80 msgr2=0x7f626c157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:13.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 shutdown_connections 2024-02-02T13:25:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.752+0000 7f6272800640 1 -- 172.21.15.91:0/2128986848 wait complete. 2024-02-02T13:25:13.772 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:13.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.977+0000 7f4e2be00640 1 Processor -- start 2024-02-02T13:25:13.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.977+0000 7f4e2be00640 1 -- start start 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e2be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4e240ce130 0x7f4e240c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e2be00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240cc7c0 con 0x7f4e2415a010 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e2be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240ce930 con 0x7f4e240b0700 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e2be00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e240cd640 con 0x7f4e240ce130 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e29400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4e2415a010 0x7f4e240ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34152/0 (socket says 172.21.15.91:34152) 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.978+0000 7f4e29400640 1 -- 172.21.15.91:0/118115331 learned_addr learned my addr 172.21.15.91:0/118115331 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:13.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e28a00640 1 --2- 172.21.15.91:0/118115331 >> v2:172.21.15.91:3300/0 conn(0x7f4e240ce130 0x7f4e240c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e23e00640 1 -- 172.21.15.91:0/118115331 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168367502 0 0) 0x7f4e240cc7c0 con 0x7f4e2415a010 2024-02-02T13:25:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e23e00640 1 -- 172.21.15.91:0/118115331 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e04003520 con 0x7f4e2415a010 2024-02-02T13:25:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e28a00640 1 -- 172.21.15.91:0/118115331 >> v1:172.21.15.113:6789/0 conn(0x7f4e240b0700 legacy=0x7f4e2406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e28a00640 1 -- 172.21.15.91:0/118115331 >> v1:172.21.15.91:6789/0 conn(0x7f4e2415a010 legacy=0x7f4e240ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.979+0000 7f4e28a00640 1 -- 172.21.15.91:0/118115331 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2406c890 con 0x7f4e240ce130 2024-02-02T13:25:13.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.980+0000 7f4e28a00640 1 --2- 172.21.15.91:0/118115331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240c9670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d965e15479c4cf69 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e23e00640 1 -- 172.21.15.91:0/118115331 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c030610 con 0x7f4e240ce130 2024-02-02T13:25:13.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e23e00640 1 -- 172.21.15.91:0/118115331 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c030770 con 0x7f4e240ce130 2024-02-02T13:25:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e23e00640 1 -- 172.21.15.91:0/118115331 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c030b00 con 0x7f4e240ce130 2024-02-02T13:25:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e2be00640 1 -- 172.21.15.91:0/118115331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 msgr2=0x7f4e240c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e2be00640 1 --2- 172.21.15.91:0/118115331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240c9670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02b960 comp rx=0 tx=0).stop 2024-02-02T13:25:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e2be00640 1 -- 172.21.15.91:0/118115331 shutdown_connections 2024-02-02T13:25:13.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e2be00640 1 --2- 172.21.15.91:0/118115331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240c9670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.981+0000 7f4e2be00640 1 -- 172.21.15.91:0/118115331 >> 172.21.15.91:0/118115331 conn(0x7f4e240cec80 msgr2=0x7f4e240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:13.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.982+0000 7f4e2be00640 1 -- 172.21.15.91:0/118115331 shutdown_connections 2024-02-02T13:25:13.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.982+0000 7f4e2be00640 1 -- 172.21.15.91:0/118115331 wait complete. 2024-02-02T13:25:13.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.984+0000 7f4e2be00640 1 Processor -- start 2024-02-02T13:25:13.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.984+0000 7f4e2be00640 1 -- start start 2024-02-02T13:25:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e240b0700 0x7f4e24157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e2415a010 0x7f4e241f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240aca50 con 0x7f4e240ce130 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240acd30 con 0x7f4e240b0700 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e2be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240ad010 con 0x7f4e2415a010 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e28a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e28a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47522/0 (socket says 172.21.15.91:47522) 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e28a00640 1 -- 172.21.15.91:0/3861151578 learned_addr learned my addr 172.21.15.91:0/3861151578 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.985+0000 7f4e29400640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e240b0700 0x7f4e24157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e29e00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e2415a010 0x7f4e241f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 -- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e2415a010 msgr2=0x7f4e241f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e2415a010 0x7f4e241f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 -- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e240b0700 msgr2=0x7f4e24157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e240b0700 0x7f4e24157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 -- 172.21.15.91:0/3861151578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009400 con 0x7f4e240ce130 2024-02-02T13:25:13.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e28a00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c02bf70 tx=0x7f4e0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c03e040 con 0x7f4e240ce130 2024-02-02T13:25:13.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c039cb0 con 0x7f4e240ce130 2024-02-02T13:25:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c044630 con 0x7f4e240ce130 2024-02-02T13:25:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.986+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e240ad2f0 con 0x7f4e240ce130 2024-02-02T13:25:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.987+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e241f3c40 con 0x7f4e240ce130 2024-02-02T13:25:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.988+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e24066cf0 con 0x7f4e240ce130 2024-02-02T13:25:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.989+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4e0c043390 con 0x7f4e240ce130 2024-02-02T13:25:13.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.989+0000 7f4e21600640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 0x7f4dfc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:13.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.990+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e0c0aab10 con 0x7f4e240ce130 2024-02-02T13:25:13.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.990+0000 7f4e29400640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 0x7f4dfc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.991+0000 7f4e29400640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 0x7f4dfc06c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4e140011e0 tx=0x7f4e14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:13.994+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4e0c076100 con 0x7f4e240ce130 2024-02-02T13:25:14.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.181+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e240b4c00 con 0x7f4e240ce130 2024-02-02T13:25:14.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.182+0000 7f4e21600640 1 -- 172.21.15.91:0/3861151578 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e0c079720 con 0x7f4e240ce130 2024-02-02T13:25:14.183 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 msgr2=0x7f4dfc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 0x7f4dfc06c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4e140011e0 tx=0x7f4e14009040 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 msgr2=0x7f4e240af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c02bf70 tx=0x7f4e0c004730 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 shutdown_connections 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4dfc06a430 0x7f4dfc06c8f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e2415a010 0x7f4e241f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e240ce130 0x7f4e240af720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 --2- 172.21.15.91:0/3861151578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e240b0700 0x7f4e24157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 >> 172.21.15.91:0/3861151578 conn(0x7f4e240cec80 msgr2=0x7f4e24073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 shutdown_connections 2024-02-02T13:25:14.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.185+0000 7f4e2be00640 1 -- 172.21.15.91:0/3861151578 wait complete. 2024-02-02T13:25:14.204 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:14.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.406+0000 7f660b400640 1 Processor -- start 2024-02-02T13:25:14.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.406+0000 7f660b400640 1 -- start start 2024-02-02T13:25:14.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f660b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f66040b61b0 0x7f66040cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f660b400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040b7280 con 0x7f660415d0a0 2024-02-02T13:25:14.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f660b400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040b6670 con 0x7f660415d480 2024-02-02T13:25:14.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f660b400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040b6950 con 0x7f66040b61b0 2024-02-02T13:25:14.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f6608a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f660415d0a0 0x7f66041617c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34156/0 (socket says 172.21.15.91:34156) 2024-02-02T13:25:14.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.408+0000 7f6608a00640 1 -- 172.21.15.91:0/1566853708 learned_addr learned my addr 172.21.15.91:0/1566853708 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603e00640 1 --2- 172.21.15.91:0/1566853708 >> v2:172.21.15.113:3300/0 conn(0x7f66040b61b0 0x7f66040cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191933129 0 0) 0x7f66040b7280 con 0x7f660415d0a0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003520 con 0x7f660415d0a0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2789976643 0 0) 0x7f65e4003520 con 0x7f660415d0a0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 >> v2:172.21.15.113:3300/0 conn(0x7f66040b61b0 msgr2=0x7f66040cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 --2- 172.21.15.91:0/1566853708 >> v2:172.21.15.113:3300/0 conn(0x7f66040b61b0 0x7f66040cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 >> v1:172.21.15.113:6789/0 conn(0x7f660415d480 legacy=0x7f66040b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040b6c30 con 0x7f660415d0a0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f65ec001670 con 0x7f660415d0a0 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.409+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 >> v1:172.21.15.91:6789/0 conn(0x7f660415d0a0 legacy=0x7f66041617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040b6950 con 0x7f65e4005180 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c020 con 0x7f65e40047c0 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c1e0 con 0x7f65e4003970 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6608a00640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6603e00640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 -- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4003970 msgr2=0x7f65e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 -- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65e40047c0 msgr2=0x7f65e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65e40047c0 0x7f65e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.410+0000 7f6609400640 1 -- 172.21.15.91:0/1566853708 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003520 con 0x7f65e4005180 2024-02-02T13:25:14.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.411+0000 7f6609400640 1 -- 172.21.15.91:0/1566853708 reap_dead start 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.411+0000 7f6609400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 0x7f65e400b980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f66040b7230 tx=0x7f65e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c3a381cf398821 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0052d0 con 0x7f65e4005180 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc02e050 con 0x7f65e4005180 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f6603400640 1 -- 172.21.15.91:0/1566853708 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0035f0 con 0x7f65e4005180 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 -- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 msgr2=0x7f65e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 0x7f65e400b980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f66040b7230 tx=0x7f65e40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 -- 172.21.15.91:0/1566853708 shutdown_connections 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 --2- 172.21.15.91:0/1566853708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65e4005180 0x7f65e400b980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 -- 172.21.15.91:0/1566853708 >> 172.21.15.91:0/1566853708 conn(0x7f6604070690 msgr2=0x7f6604070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.412+0000 7f660b400640 1 -- 172.21.15.91:0/1566853708 shutdown_connections 2024-02-02T13:25:14.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.413+0000 7f660b400640 1 -- 172.21.15.91:0/1566853708 wait complete. 2024-02-02T13:25:14.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.414+0000 7f660b400640 1 Processor -- start 2024-02-02T13:25:14.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.415+0000 7f660b400640 1 -- start start 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.415+0000 7f660b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4005180 0x7f66040cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.415+0000 7f660b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.415+0000 7f660b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66041f33d0 0x7f6604151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f660b400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c8610 con 0x7f66041f2f50 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f660b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c88f0 con 0x7f66041f33d0 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f660b400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c8bd0 con 0x7f65e4005180 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47548/0 (socket says 172.21.15.91:47548) 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 -- 172.21.15.91:0/761523186 learned_addr learned my addr 172.21.15.91:0/761523186 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:14.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6608a00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4005180 0x7f66040cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6609400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66041f33d0 0x7f6604151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 -- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4005180 msgr2=0x7f66040cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4005180 0x7f66040cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 -- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66041f33d0 msgr2=0x7f6604151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66041f33d0 0x7f6604151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.416+0000 7f6603e00640 1 -- 172.21.15.91:0/761523186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040c8eb0 con 0x7f66041f2f50 2024-02-02T13:25:14.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.417+0000 7f6603e00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009f10 tx=0x7f65f40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.418+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f400e070 con 0x7f66041f2f50 2024-02-02T13:25:14.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.418+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4003990 con 0x7f66041f2f50 2024-02-02T13:25:14.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.418+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4003b40 con 0x7f66041f2f50 2024-02-02T13:25:14.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.418+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040c9190 con 0x7f66041f2f50 2024-02-02T13:25:14.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.418+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040ccc40 con 0x7f66041f2f50 2024-02-02T13:25:14.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.420+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6604066cf0 con 0x7f66041f2f50 2024-02-02T13:25:14.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.420+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f65f4003ca0 con 0x7f66041f2f50 2024-02-02T13:25:14.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.421+0000 7f6600c00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 0x7f65dc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.421+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f4089180 con 0x7f66041f2f50 2024-02-02T13:25:14.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.422+0000 7f6608a00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 0x7f65dc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.422+0000 7f6608a00640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 0x7f65dc06c920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66040cc0e0 tx=0x7f65ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.426+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f65f4054800 con 0x7f66041f2f50 2024-02-02T13:25:14.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.618+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66040d0d30 con 0x7f66041f2f50 2024-02-02T13:25:14.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.621+0000 7f6600c00640 1 -- 172.21.15.91:0/761523186 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f65f4057e20 con 0x7f66041f2f50 2024-02-02T13:25:14.622 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:14.622 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 msgr2=0x7f65dc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 0x7f65dc06c920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66040cc0e0 tx=0x7f65ec009040 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 msgr2=0x7f6604151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009f10 tx=0x7f65f40115a0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 shutdown_connections 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65dc06a460 0x7f65dc06c920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66041f33d0 0x7f6604151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66041f2f50 0x7f6604151220 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 --2- 172.21.15.91:0/761523186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65e4005180 0x7f66040cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 >> 172.21.15.91:0/761523186 conn(0x7f6604070690 msgr2=0x7f66040b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 shutdown_connections 2024-02-02T13:25:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.624+0000 7f660b400640 1 -- 172.21.15.91:0/761523186 wait complete. 2024-02-02T13:25:14.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:14.642 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:14.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.847+0000 7ff854c00640 1 Processor -- start 2024-02-02T13:25:14.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.847+0000 7ff854c00640 1 -- start start 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.848+0000 7ff854c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff85015a010 0x7ff8500c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.848+0000 7ff854c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8500b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.848+0000 7ff854c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8500cc7c0 con 0x7ff8500ce130 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.848+0000 7ff854c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8500b10d0 con 0x7ff85015a010 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.848+0000 7ff854c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500ce930 con 0x7ff8500b0700 2024-02-02T13:25:14.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff845200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff8500ce130 0x7ff8500ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34164/0 (socket says 172.21.15.91:34164) 2024-02-02T13:25:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff845200640 1 -- 172.21.15.91:0/1382605232 learned_addr learned my addr 172.21.15.91:0/1382605232 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350229096 0 0) 0x7ff8500cc7c0 con 0x7ff8500ce130 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff830003520 con 0x7ff8500ce130 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84e800640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8500b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84de00640 1 --2- 172.21.15.91:0/1382605232 >> v2:172.21.15.113:3300/0 conn(0x7ff85015a010 0x7ff8500c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928652664 0 0) 0x7ff830003520 con 0x7ff8500ce130 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 msgr2=0x7ff8500b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8500b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 >> v2:172.21.15.113:3300/0 conn(0x7ff85015a010 msgr2=0x7ff8500c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 --2- 172.21.15.91:0/1382605232 >> v2:172.21.15.113:3300/0 conn(0x7ff85015a010 0x7ff8500c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500cd640 con 0x7ff8500ce130 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff838001670 con 0x7ff8500ce130 2024-02-02T13:25:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.849+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 >> v1:172.21.15.91:6789/0 conn(0x7ff8500ce130 legacy=0x7ff8500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff830003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff830004d00 0x7ff830005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b10d0 con 0x7ff830003970 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500ce930 con 0x7ff830004d00 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83000ba30 con 0x7ff8500b0700 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff830003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff845200640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 -- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 msgr2=0x7ff8300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84e800640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff830004d00 0x7ff830005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 -- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff830004d00 msgr2=0x7ff830005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff830004d00 0x7ff830005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff84de00640 1 -- 172.21.15.91:0/1382605232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830003520 con 0x7ff830003970 2024-02-02T13:25:14.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.850+0000 7ff845200640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500b0700 0x7ff8300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:14.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.851+0000 7ff84de00640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff830003d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff840007390 tx=0x7ff8300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce80c8f5983def1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84002ccf0 con 0x7ff830003970 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84002ce50 con 0x7ff830003970 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff84d400640 1 -- 172.21.15.91:0/1382605232 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840034660 con 0x7ff830003970 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff854c00640 1 -- 172.21.15.91:0/1382605232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 msgr2=0x7ff830003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff854c00640 1 --2- 172.21.15.91:0/1382605232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff830003d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff840007390 tx=0x7ff8300057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff84e800640 1 -- 172.21.15.91:0/1382605232 reap_dead start 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff854c00640 1 -- 172.21.15.91:0/1382605232 shutdown_connections 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.852+0000 7ff854c00640 1 -- 172.21.15.91:0/1382605232 >> 172.21.15.91:0/1382605232 conn(0x7ff8500cec80 msgr2=0x7ff8500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.853+0000 7ff854c00640 1 -- 172.21.15.91:0/1382605232 shutdown_connections 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.853+0000 7ff854c00640 1 -- 172.21.15.91:0/1382605232 wait complete. 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.854+0000 7ff854c00640 1 Processor -- start 2024-02-02T13:25:14.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.854+0000 7ff854c00640 1 -- start start 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850152000 0x7ff8500c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500adab0 0x7ff85015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c2b40 con 0x7ff830003970 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c24c0 con 0x7ff850152000 2024-02-02T13:25:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff854c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c4320 con 0x7ff8500adab0 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff84de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff84de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47554/0 (socket says 172.21.15.91:47554) 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.855+0000 7ff84de00640 1 -- 172.21.15.91:0/1083752129 learned_addr learned my addr 172.21.15.91:0/1083752129 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 -- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500adab0 msgr2=0x7ff85015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500adab0 0x7ff85015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 -- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850152000 msgr2=0x7ff8500c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850152000 0x7ff8500c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 -- 172.21.15.91:0/1083752129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500c2760 con 0x7ff830003970 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff84de00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff840037040 tx=0x7ff840004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840003b90 con 0x7ff830003970 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff840003cf0 con 0x7ff830003970 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8500c34a0 con 0x7ff830003970 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.856+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840004260 con 0x7ff830003970 2024-02-02T13:25:14.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.857+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8500c3f60 con 0x7ff830003970 2024-02-02T13:25:14.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.858+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff850066cf0 con 0x7ff830003970 2024-02-02T13:25:14.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.859+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff84003b040 con 0x7ff830003970 2024-02-02T13:25:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.860+0000 7ff846a00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 0x7ff82806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.860+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8400aa7d0 con 0x7ff830003970 2024-02-02T13:25:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.860+0000 7ff845200640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 0x7ff82806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.861+0000 7ff845200640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 0x7ff82806c8f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8380017a0 tx=0x7ff838009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:14.865+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff840075e50 con 0x7ff830003970 2024-02-02T13:25:15.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.053+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8501679e0 con 0x7ff830003970 2024-02-02T13:25:15.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.053+0000 7ff846a00640 1 -- 172.21.15.91:0/1083752129 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff840079470 con 0x7ff830003970 2024-02-02T13:25:15.054 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:15.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 msgr2=0x7ff82806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 0x7ff82806c8f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8380017a0 tx=0x7ff838009040 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 msgr2=0x7ff8500c2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff840037040 tx=0x7ff840004ab0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 shutdown_connections 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff82806a430 0x7ff82806c8f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8500adab0 0x7ff85015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850152000 0x7ff8500c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 --2- 172.21.15.91:0/1083752129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff830003970 0x7ff8500c2f60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 >> 172.21.15.91:0/1083752129 conn(0x7ff8500cec80 msgr2=0x7ff8500ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 shutdown_connections 2024-02-02T13:25:15.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.056+0000 7ff854c00640 1 -- 172.21.15.91:0/1083752129 wait complete. 2024-02-02T13:25:15.075 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:15.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.282+0000 7f5b11600640 1 Processor -- start 2024-02-02T13:25:15.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.282+0000 7f5b11600640 1 -- start start 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b11600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b0c0b8990 0x7f5b0c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b11600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c0b8e00 con 0x7f5b0c15cee0 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b11600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c0b7d40 con 0x7f5b0c0ce1d0 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b11600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b6ed0 con 0x7f5b0c0b8990 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b0be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5b0c15cee0 0x7f5b0c0cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34180/0 (socket says 172.21.15.91:34180) 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b0be00640 1 -- 172.21.15.91:0/1098970728 learned_addr learned my addr 172.21.15.91:0/1098970728 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b10c00640 1 --2- 172.21.15.91:0/1098970728 >> v2:172.21.15.91:3300/0 conn(0x7f5b0c0b8990 0x7f5b0c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b0b400640 1 -- 172.21.15.91:0/1098970728 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763211429 0 0) 0x7f5b0c0b8e00 con 0x7f5b0c15cee0 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b0b400640 1 -- 172.21.15.91:0/1098970728 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aec003520 con 0x7f5b0c15cee0 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b10c00640 1 -- 172.21.15.91:0/1098970728 >> v1:172.21.15.113:6789/0 conn(0x7f5b0c0ce1d0 legacy=0x7f5b0c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b10c00640 1 -- 172.21.15.91:0/1098970728 >> v1:172.21.15.91:6789/0 conn(0x7f5b0c15cee0 legacy=0x7f5b0c0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.284+0000 7f5b10c00640 1 -- 172.21.15.91:0/1098970728 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0b81d0 con 0x7f5b0c0b8990 2024-02-02T13:25:15.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b10c00640 1 --2- 172.21.15.91:0/1098970728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c0b8990 0x7f5b0c0b6990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009d90 tx=0x7f5b0402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669490b52d0024c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b0b400640 1 -- 172.21.15.91:0/1098970728 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0402cc60 con 0x7f5b0c0b8990 2024-02-02T13:25:15.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b0b400640 1 -- 172.21.15.91:0/1098970728 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0402cdc0 con 0x7f5b0c0b8990 2024-02-02T13:25:15.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b0b400640 1 -- 172.21.15.91:0/1098970728 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04034660 con 0x7f5b0c0b8990 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b11600640 1 -- 172.21.15.91:0/1098970728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c0b8990 msgr2=0x7f5b0c0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.286+0000 7f5b11600640 1 --2- 172.21.15.91:0/1098970728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c0b8990 0x7f5b0c0b6990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009d90 tx=0x7f5b0402beb0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.287+0000 7f5b11600640 1 -- 172.21.15.91:0/1098970728 shutdown_connections 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.287+0000 7f5b11600640 1 --2- 172.21.15.91:0/1098970728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c0b8990 0x7f5b0c0b6990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.287+0000 7f5b11600640 1 -- 172.21.15.91:0/1098970728 >> 172.21.15.91:0/1098970728 conn(0x7f5b0c070690 msgr2=0x7f5b0c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.287+0000 7f5b11600640 1 -- 172.21.15.91:0/1098970728 shutdown_connections 2024-02-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.287+0000 7f5b11600640 1 -- 172.21.15.91:0/1098970728 wait complete. 2024-02-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.288+0000 7f5b11600640 1 Processor -- start 2024-02-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.288+0000 7f5b11600640 1 -- start start 2024-02-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0c0b8990 0x7f5b0c1f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 0x7f5b0c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0cb910 con 0x7f5b0c15cee0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c15eeb0 con 0x7f5b0c0ce1d0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.289+0000 7f5b11600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c15eac0 con 0x7f5b0c0b8990 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47570/0 (socket says 172.21.15.91:47570) 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 -- 172.21.15.91:0/1917618788 learned_addr learned my addr 172.21.15.91:0/1917618788 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b03200640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 0x7f5b0c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 -- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0c0b8990 msgr2=0x7f5b0c1f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0c0b8990 0x7f5b0c1f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 -- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 msgr2=0x7f5b0c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 0x7f5b0c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 -- 172.21.15.91:0/1917618788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009a70 con 0x7f5b0c15cee0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b03200640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 0x7f5b0c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.290+0000 7f5b10c00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b04037040 tx=0x7f5b04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.291+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04040650 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.291+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040407b0 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.291+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04040b60 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.291+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c0cb070 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.291+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c0cb630 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.292+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c04dff0 con 0x7f5b0c15cee0 2024-02-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.293+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5b04040cc0 con 0x7f5b0c15cee0 2024-02-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.294+0000 7f5b08c00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 0x7f5ae406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.294+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b040ab6b0 con 0x7f5b0c15cee0 2024-02-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.294+0000 7f5b0be00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 0x7f5ae406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.295+0000 7f5b0be00640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 0x7f5ae406c8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001610 tx=0x7f5afc007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.299+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5b04076d50 con 0x7f5b0c15cee0 2024-02-02T13:25:15.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.492+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b0c15f020 con 0x7f5b0c15cee0 2024-02-02T13:25:15.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.493+0000 7f5b08c00640 1 -- 172.21.15.91:0/1917618788 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b04009450 con 0x7f5b0c15cee0 2024-02-02T13:25:15.494 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 msgr2=0x7f5ae406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 0x7f5ae406c8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001610 tx=0x7f5afc007930 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 msgr2=0x7f5b0c15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b04037040 tx=0x7f5b04004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 shutdown_connections 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae406a430 0x7f5ae406c8f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0c15cee0 0x7f5b0c15df00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0c0ce1d0 0x7f5b0c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 --2- 172.21.15.91:0/1917618788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0c0b8990 0x7f5b0c1f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 >> 172.21.15.91:0/1917618788 conn(0x7f5b0c070690 msgr2=0x7f5b0c0b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 shutdown_connections 2024-02-02T13:25:15.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.497+0000 7f5b11600640 1 -- 172.21.15.91:0/1917618788 wait complete. 2024-02-02T13:25:15.515 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:15.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.719+0000 7f66ff600640 1 Processor -- start 2024-02-02T13:25:15.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.719+0000 7f66ff600640 1 -- start start 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.720+0000 7f66ff600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f8151f90 0x7f66f815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.720+0000 7f66ff600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66f806f4f0 0x7f66f80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.720+0000 7f66ff600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f8070f80 con 0x7f66f80b0c40 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.720+0000 7f66ff600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f80b10d0 con 0x7f66f806f4f0 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.720+0000 7f66ff600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8152370 con 0x7f66f8151f90 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66f806f4f0 0x7f66f80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66f806f4f0 0x7f66f80b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47572/0 (socket says 172.21.15.91:47572) 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 -- 172.21.15.91:0/75787677 learned_addr learned my addr 172.21.15.91:0/75787677 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66fcc00640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f8151f90 0x7f66f815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 -- 172.21.15.91:0/75787677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f8151f90 msgr2=0x7f66f815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f8151f90 0x7f66f815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 -- 172.21.15.91:0/75787677 >> v1:172.21.15.113:6789/0 conn(0x7f66f80b0c40 legacy=0x7f66f80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:15.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.721+0000 7f66f7e00640 1 -- 172.21.15.91:0/75787677 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80cb010 con 0x7f66f806f4f0 2024-02-02T13:25:15.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.722+0000 7f66f7e00640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f806f4f0 0x7f66f80b0700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009870 tx=0x7f66e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba4cbd7b218f7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.723+0000 7f66f7400640 1 -- 172.21.15.91:0/75787677 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0031640 con 0x7f66f806f4f0 2024-02-02T13:25:15.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.723+0000 7f66f7400640 1 -- 172.21.15.91:0/75787677 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00317a0 con 0x7f66f806f4f0 2024-02-02T13:25:15.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.723+0000 7f66f7400640 1 -- 172.21.15.91:0/75787677 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0031b30 con 0x7f66f806f4f0 2024-02-02T13:25:15.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.723+0000 7f66ff600640 1 -- 172.21.15.91:0/75787677 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f806f4f0 msgr2=0x7f66f80b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.723+0000 7f66ff600640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f806f4f0 0x7f66f80b0700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009870 tx=0x7f66e002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 -- 172.21.15.91:0/75787677 shutdown_connections 2024-02-02T13:25:15.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f806f4f0 0x7f66f80b0700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 --2- 172.21.15.91:0/75787677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f8151f90 0x7f66f815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 -- 172.21.15.91:0/75787677 >> 172.21.15.91:0/75787677 conn(0x7f66f80cec80 msgr2=0x7f66f80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:15.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 -- 172.21.15.91:0/75787677 shutdown_connections 2024-02-02T13:25:15.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.724+0000 7f66ff600640 1 -- 172.21.15.91:0/75787677 wait complete. 2024-02-02T13:25:15.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.725+0000 7f66ff600640 1 Processor -- start 2024-02-02T13:25:15.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.725+0000 7f66ff600640 1 -- start start 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f806f4f0 0x7f66f80c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f80b0c40 0x7f66f814c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80c6780 con 0x7f66f8151f90 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80c6a60 con 0x7f66f80b0c40 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.726+0000 7f66ff600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80c74b0 con 0x7f66f806f4f0 2024-02-02T13:25:15.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47588/0 (socket says 172.21.15.91:47588) 2024-02-02T13:25:15.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 -- 172.21.15.91:0/3263152087 learned_addr learned my addr 172.21.15.91:0/3263152087 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 -- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f806f4f0 msgr2=0x7f66f80c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f806f4f0 0x7f66f80c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 -- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f80b0c40 msgr2=0x7f66f814c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f80b0c40 0x7f66f814c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 -- 172.21.15.91:0/3263152087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00094d0 con 0x7f66f8151f90 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.727+0000 7f66fd600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b9e0 tx=0x7f66f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.728+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003b70 con 0x7f66f8151f90 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.728+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0003cd0 con 0x7f66f8151f90 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.728+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000cae0 con 0x7f66f8151f90 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.728+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f80c7790 con 0x7f66f8151f90 2024-02-02T13:25:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.728+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f814ca50 con 0x7f66f8151f90 2024-02-02T13:25:15.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.729+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f80c7310 con 0x7f66f8151f90 2024-02-02T13:25:15.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.730+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f66f000cc40 con 0x7f66f8151f90 2024-02-02T13:25:15.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.731+0000 7f66f4c00640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 0x7f66d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:15.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.731+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f00886c0 con 0x7f66f8151f90 2024-02-02T13:25:15.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.731+0000 7f66fcc00640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 0x7f66d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:15.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.732+0000 7f66fcc00640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 0x7f66d006c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098f0 tx=0x7f66e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:15.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.735+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f66f80c7310 con 0x7f66f8151f90 2024-02-02T13:25:15.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.909+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66f80c79d0 con 0x7f66f8151f90 2024-02-02T13:25:15.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.910+0000 7f66f4c00640 1 -- 172.21.15.91:0/3263152087 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66f80c79d0 con 0x7f66f8151f90 2024-02-02T13:25:15.911 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 msgr2=0x7f66d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 0x7f66d006c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098f0 tx=0x7f66e8008040 comp rx=0 tx=0).stop 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 msgr2=0x7f66f80c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b9e0 tx=0x7f66f000bea0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 shutdown_connections 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66d006a430 0x7f66d006c8f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f66f8151f90 0x7f66f80c5cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f80b0c40 0x7f66f814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 --2- 172.21.15.91:0/3263152087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66f806f4f0 0x7f66f80c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:15.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.912+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 >> 172.21.15.91:0/3263152087 conn(0x7f66f80cec80 msgr2=0x7f66f806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:15.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.913+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 shutdown_connections 2024-02-02T13:25:15.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:15.913+0000 7f66ff600640 1 -- 172.21.15.91:0/3263152087 wait complete. 2024-02-02T13:25:15.931 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:16.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.130+0000 7f2b83000640 1 Processor -- start 2024-02-02T13:25:16.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.130+0000 7f2b83000640 1 -- start start 2024-02-02T13:25:16.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b83000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b7c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b83000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c15d870 con 0x7f2b7c15d490 2024-02-02T13:25:16.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b83000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c06bce0 con 0x7f2b7c0acfe0 2024-02-02T13:25:16.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b83000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c161220 con 0x7f2b7c0b0e10 2024-02-02T13:25:16.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b7be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2b7c15d490 0x7f2b7c0ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34192/0 (socket says 172.21.15.91:34192) 2024-02-02T13:25:16.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b7be00640 1 -- 172.21.15.91:0/2886181075 learned_addr learned my addr 172.21.15.91:0/2886181075 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:16.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b81000640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b7c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633704128 0 0) 0x7f2b7c15d870 con 0x7f2b7c15d490 2024-02-02T13:25:16.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.132+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b5c003520 con 0x7f2b7c15d490 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1534400392 0 0) 0x7f2b5c003520 con 0x7f2b7c15d490 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 msgr2=0x7f2b7c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b7c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 >> v1:172.21.15.113:6789/0 conn(0x7f2b7c0acfe0 legacy=0x7f2b7c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0ad490 con 0x7f2b7c15d490 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2b6c001670 con 0x7f2b7c15d490 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 >> v1:172.21.15.91:6789/0 conn(0x7f2b7c15d490 legacy=0x7f2b7c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5c0039a0 0x7f2b5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c161220 con 0x7f2b5c0047c0 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c00bae0 con 0x7f2b5c0039a0 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c00bca0 con 0x7f2b7c0b0e10 2024-02-02T13:25:16.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b81000640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 -- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 msgr2=0x7f2b5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 -- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5c0039a0 msgr2=0x7f2b5c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5c0039a0 0x7f2b5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b7b400640 1 -- 172.21.15.91:0/2886181075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c003520 con 0x7f2b5c0047c0 2024-02-02T13:25:16.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.133+0000 7f2b81000640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0b0e10 0x7f2b5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.134+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b5c004c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b64001750 tx=0x7f2b5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6053be72b614bf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640052d0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6402e050 con 0x7f2b5c0047c0 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b7aa00640 1 -- 172.21.15.91:0/2886181075 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640035f0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 -- 172.21.15.91:0/2886181075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 msgr2=0x7f2b5c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 --2- 172.21.15.91:0/2886181075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b5c004c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b64001750 tx=0x7f2b5c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b81000640 1 -- 172.21.15.91:0/2886181075 reap_dead start 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 -- 172.21.15.91:0/2886181075 shutdown_connections 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 -- 172.21.15.91:0/2886181075 >> 172.21.15.91:0/2886181075 conn(0x7f2b7c0cec80 msgr2=0x7f2b7c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 -- 172.21.15.91:0/2886181075 shutdown_connections 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.135+0000 7f2b83000640 1 -- 172.21.15.91:0/2886181075 wait complete. 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.137+0000 7f2b83000640 1 Processor -- start 2024-02-02T13:25:16.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.137+0000 7f2b83000640 1 -- start start 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b3cd0 0x7f2b7c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0c8890 0x7f2b7c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c159380 con 0x7f2b5c0047c0 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1f28b0 con 0x7f2b7c0b3cd0 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b83000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0b2c30 con 0x7f2b7c0c8890 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47616/0 (socket says 172.21.15.91:47616) 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 -- 172.21.15.91:0/2289563474 learned_addr learned my addr 172.21.15.91:0/2289563474 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 -- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0c8890 msgr2=0x7f2b7c148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0c8890 0x7f2b7c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 -- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b3cd0 msgr2=0x7f2b7c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b3cd0 0x7f2b7c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.138+0000 7f2b7be00640 1 -- 172.21.15.91:0/2289563474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0b2f10 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b7be00640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c700 tx=0x7f2b6c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0074e0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c007640 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004a00 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c0b31f0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.139+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c0b37b0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.140+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c066cf0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.141+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2b6c004b60 con 0x7f2b5c0047c0 2024-02-02T13:25:16.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.142+0000 7f2b73e00640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 0x7f2b5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.142+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b6c087e90 con 0x7f2b5c0047c0 2024-02-02T13:25:16.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.142+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 0x7f2b5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.143+0000 7f2b7b400640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 0x7f2b5406c8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b64033000 tx=0x7f2b6402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.146+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2b6c0533e0 con 0x7f2b5c0047c0 2024-02-02T13:25:16.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.340+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b7c0b4c00 con 0x7f2b5c0047c0 2024-02-02T13:25:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.342+0000 7f2b73e00640 1 -- 172.21.15.91:0/2289563474 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f2b6c018020 con 0x7f2b5c0047c0 2024-02-02T13:25:16.344 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:16.344 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.344+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 msgr2=0x7f2b5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.344+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 0x7f2b5406c8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b64033000 tx=0x7f2b6402f000 comp rx=0 tx=0).stop 2024-02-02T13:25:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.344+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 msgr2=0x7f2b7c1f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.344+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c700 tx=0x7f2b6c00cbc0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 shutdown_connections 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b5406a430 0x7f2b5406c8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7c0c8890 0x7f2b7c148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b3cd0 0x7f2b7c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 --2- 172.21.15.91:0/2289563474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b5c0047c0 0x7f2b7c1f2370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 >> 172.21.15.91:0/2289563474 conn(0x7f2b7c0cec80 msgr2=0x7f2b7c0af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 shutdown_connections 2024-02-02T13:25:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.345+0000 7f2b83000640 1 -- 172.21.15.91:0/2289563474 wait complete. 2024-02-02T13:25:16.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:16.364 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:16.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.567+0000 7f532ec00640 1 Processor -- start 2024-02-02T13:25:16.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.567+0000 7f532ec00640 1 -- start start 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53280adf00 0x7f5328159b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53280c9170 0x7f532806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f532806c220 0x7f532815d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532806c6d0 con 0x7f53280adf00 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280ae2e0 con 0x7f532806c220 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532815a4b0 con 0x7f53280c9170 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53280adf00 0x7f5328159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53280adf00 0x7f5328159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47630/0 (socket says 172.21.15.91:47630) 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.569+0000 7f532d800640 1 -- 172.21.15.91:0/457575806 learned_addr learned my addr 172.21.15.91:0/457575806 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:16.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532e200640 1 --2- 172.21.15.91:0/457575806 >> v2:172.21.15.113:3300/0 conn(0x7f532806c220 0x7f532815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532ce00640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53280c9170 0x7f532806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532d800640 1 -- 172.21.15.91:0/457575806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53280c9170 msgr2=0x7f532806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532d800640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53280c9170 0x7f532806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532d800640 1 -- 172.21.15.91:0/457575806 >> v2:172.21.15.113:3300/0 conn(0x7f532806c220 msgr2=0x7f532815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532d800640 1 --2- 172.21.15.91:0/457575806 >> v2:172.21.15.113:3300/0 conn(0x7f532806c220 0x7f532815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.570+0000 7f532d800640 1 -- 172.21.15.91:0/457575806 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532815a790 con 0x7f53280adf00 2024-02-02T13:25:16.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.571+0000 7f532d800640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f5328159b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f532806f380 tx=0x7f531802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f78446ba3d9a24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f5327e00640 1 -- 172.21.15.91:0/457575806 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802ea80 con 0x7f53280adf00 2024-02-02T13:25:16.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f5327e00640 1 -- 172.21.15.91:0/457575806 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531802ebe0 con 0x7f53280adf00 2024-02-02T13:25:16.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f5327e00640 1 -- 172.21.15.91:0/457575806 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318009600 con 0x7f53280adf00 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 -- 172.21.15.91:0/457575806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 msgr2=0x7f5328159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f5328159b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f532806f380 tx=0x7f531802e0e0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 -- 172.21.15.91:0/457575806 shutdown_connections 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 --2- 172.21.15.91:0/457575806 >> v2:172.21.15.113:3300/0 conn(0x7f532806c220 0x7f532815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53280c9170 0x7f532806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.572+0000 7f532ec00640 1 --2- 172.21.15.91:0/457575806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f5328159b40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.573+0000 7f532ec00640 1 -- 172.21.15.91:0/457575806 >> 172.21.15.91:0/457575806 conn(0x7f53280cea90 msgr2=0x7f53280ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.573+0000 7f532ec00640 1 -- 172.21.15.91:0/457575806 shutdown_connections 2024-02-02T13:25:16.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.573+0000 7f532ec00640 1 -- 172.21.15.91:0/457575806 wait complete. 2024-02-02T13:25:16.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.575+0000 7f532ec00640 1 Processor -- start 2024-02-02T13:25:16.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.575+0000 7f532ec00640 1 -- start start 2024-02-02T13:25:16.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 0x7f5328150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53280c9170 0x7f53280ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328151500 con 0x7f53280adf00 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280b3630 con 0x7f53280c9170 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53281f2d20 con 0x7f532806c220 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47632/0 (socket says 172.21.15.91:47632) 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.576+0000 7f532ce00640 1 -- 172.21.15.91:0/2033841139 learned_addr learned my addr 172.21.15.91:0/2033841139 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532e200640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53280c9170 0x7f53280ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532d800640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 0x7f5328150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 -- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 msgr2=0x7f5328150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 0x7f5328150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 -- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53280c9170 msgr2=0x7f53280ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53280c9170 0x7f53280ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 -- 172.21.15.91:0/2033841139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531800bd40 con 0x7f53280adf00 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532d800640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 0x7f5328150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ce00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f531000d5c0 tx=0x7f531000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53100125f0 con 0x7f53280adf00 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310012750 con 0x7f53280adf00 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310012b00 con 0x7f53280adf00 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.577+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53280b2780 con 0x7f53280adf00 2024-02-02T13:25:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.578+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53280b2d00 con 0x7f53280adf00 2024-02-02T13:25:16.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.579+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532804dff0 con 0x7f53280adf00 2024-02-02T13:25:16.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.579+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5310012c60 con 0x7f53280adf00 2024-02-02T13:25:16.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.581+0000 7f5325600640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 0x7f530006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:16.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.581+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5310088ec0 con 0x7f53280adf00 2024-02-02T13:25:16.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.581+0000 7f532d800640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 0x7f530006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:16.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.582+0000 7f532d800640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 0x7f530006c8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5318037000 tx=0x7f5318032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.585+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5310054380 con 0x7f53280adf00 2024-02-02T13:25:16.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.778+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53280b3770 con 0x7f53280adf00 2024-02-02T13:25:16.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.779+0000 7f5325600640 1 -- 172.21.15.91:0/2033841139 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53100579a0 con 0x7f53280adf00 2024-02-02T13:25:16.780 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:16.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 msgr2=0x7f530006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 0x7f530006c8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5318037000 tx=0x7f5318032000 comp rx=0 tx=0).stop 2024-02-02T13:25:16.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 msgr2=0x7f53281f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:16.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f531000d5c0 tx=0x7f531000da80 comp rx=0 tx=0).stop 2024-02-02T13:25:16.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 shutdown_connections 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f530006a430 0x7f530006c8f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53280c9170 0x7f53280ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53280adf00 0x7f53281f27e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 --2- 172.21.15.91:0/2033841139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f532806c220 0x7f5328150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.781+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 >> 172.21.15.91:0/2033841139 conn(0x7f53280cea90 msgr2=0x7f5328158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.782+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 shutdown_connections 2024-02-02T13:25:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:16.782+0000 7f532ec00640 1 -- 172.21.15.91:0/2033841139 wait complete. 2024-02-02T13:25:16.800 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:17.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.000+0000 7ff80cc00640 1 Processor -- start 2024-02-02T13:25:17.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.000+0000 7ff80cc00640 1 -- start start 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff80cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8081595a0 0x7ff8080c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff80cc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8080c9a80 0x7ff80806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff80cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff808070420 con 0x7ff80806fc50 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff80cc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff808070fa0 con 0x7ff8081595a0 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff80cc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8080c9f10 con 0x7ff8080c9a80 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff805e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8081595a0 0x7ff8080c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff805e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8081595a0 0x7ff8080c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47636/0 (socket says 172.21.15.91:47636) 2024-02-02T13:25:17.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.001+0000 7ff805e00640 1 -- 172.21.15.91:0/1768539664 learned_addr learned my addr 172.21.15.91:0/1768539664 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:17.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.002+0000 7ff805e00640 1 -- 172.21.15.91:0/1768539664 >> v2:172.21.15.113:3300/0 conn(0x7ff8080c9a80 msgr2=0x7ff80806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:17.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.002+0000 7ff805e00640 1 --2- 172.21.15.91:0/1768539664 >> v2:172.21.15.113:3300/0 conn(0x7ff8080c9a80 0x7ff80806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.002+0000 7ff805e00640 1 -- 172.21.15.91:0/1768539664 >> v1:172.21.15.113:6789/0 conn(0x7ff80806fc50 legacy=0x7ff80815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:17.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.002+0000 7ff805e00640 1 -- 172.21.15.91:0/1768539664 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808070700 con 0x7ff8081595a0 2024-02-02T13:25:17.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.003+0000 7ff805e00640 1 --2- 172.21.15.91:0/1768539664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8081595a0 0x7ff8080c9540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff8080cb350 tx=0x7ff7f802ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dfcdd31707d687 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.003+0000 7ff805400640 1 -- 172.21.15.91:0/1768539664 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8030840 con 0x7ff8081595a0 2024-02-02T13:25:17.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.003+0000 7ff805400640 1 -- 172.21.15.91:0/1768539664 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80309a0 con 0x7ff8081595a0 2024-02-02T13:25:17.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.003+0000 7ff805400640 1 -- 172.21.15.91:0/1768539664 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8030d30 con 0x7ff8081595a0 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 -- 172.21.15.91:0/1768539664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8081595a0 msgr2=0x7ff8080c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 --2- 172.21.15.91:0/1768539664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8081595a0 0x7ff8080c9540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff8080cb350 tx=0x7ff7f802ca20 comp rx=0 tx=0).stop 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 -- 172.21.15.91:0/1768539664 shutdown_connections 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 --2- 172.21.15.91:0/1768539664 >> v2:172.21.15.113:3300/0 conn(0x7ff8080c9a80 0x7ff80806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 --2- 172.21.15.91:0/1768539664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8081595a0 0x7ff8080c9540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 -- 172.21.15.91:0/1768539664 >> 172.21.15.91:0/1768539664 conn(0x7ff8080ceca0 msgr2=0x7ff8080cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 -- 172.21.15.91:0/1768539664 shutdown_connections 2024-02-02T13:25:17.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.004+0000 7ff80cc00640 1 -- 172.21.15.91:0/1768539664 wait complete. 2024-02-02T13:25:17.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.006+0000 7ff80cc00640 1 Processor -- start 2024-02-02T13:25:17.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.006+0000 7ff80cc00640 1 -- start start 2024-02-02T13:25:17.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8080c9a80 0x7ff8081e4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 0x7ff8081e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081dfb10 con 0x7ff80806fc50 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff808153500 con 0x7ff8081595a0 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff80cc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080c37f0 con 0x7ff8080c9a80 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47646/0 (socket says 172.21.15.91:47646) 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 -- 172.21.15.91:0/2834157141 learned_addr learned my addr 172.21.15.91:0/2834157141 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 -- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8080c9a80 msgr2=0x7ff8081e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8080c9a80 0x7ff8081e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff806800640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 0x7ff8081e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 -- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 msgr2=0x7ff8081e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 0x7ff8081e51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff805e00640 1 -- 172.21.15.91:0/2834157141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80094d0 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.007+0000 7ff806800640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 0x7ff8081e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff805e00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8038bd0 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8038d50 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8042630 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8081e3a70 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.008+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8081e4030 con 0x7ff80806fc50 2024-02-02T13:25:17.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.009+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff808066cf0 con 0x7ff80806fc50 2024-02-02T13:25:17.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.010+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff7f802e3d0 con 0x7ff80806fc50 2024-02-02T13:25:17.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.012+0000 7ff7fea00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 0x7ff7e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.012+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7f80aaad0 con 0x7ff80806fc50 2024-02-02T13:25:17.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.012+0000 7ff7fd200640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 0x7ff7e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.012+0000 7ff7fd200640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 0x7ff7e006c8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8081e60e0 tx=0x7ff7f0006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.015+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff7f8076020 con 0x7ff80806fc50 2024-02-02T13:25:17.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.202+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8081e41c0 con 0x7ff80806fc50 2024-02-02T13:25:17.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.203+0000 7ff7fea00640 1 -- 172.21.15.91:0/2834157141 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7f80470a0 con 0x7ff80806fc50 2024-02-02T13:25:17.205 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 msgr2=0x7ff7e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 0x7ff7e006c8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8081e60e0 tx=0x7ff7f0006c90 comp rx=0 tx=0).stop 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 msgr2=0x7ff808152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8004730 comp rx=0 tx=0).stop 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 shutdown_connections 2024-02-02T13:25:17.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff7e006a430 0x7ff7e006c8f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8080c9a80 0x7ff8081e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081595a0 0x7ff8081e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 --2- 172.21.15.91:0/2834157141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff80806fc50 0x7ff808152fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 >> 172.21.15.91:0/2834157141 conn(0x7ff8080ceca0 msgr2=0x7ff8080cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 shutdown_connections 2024-02-02T13:25:17.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.206+0000 7ff80cc00640 1 -- 172.21.15.91:0/2834157141 wait complete. 2024-02-02T13:25:17.225 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:17.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.431+0000 7fd3e6e00640 1 Processor -- start 2024-02-02T13:25:17.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.431+0000 7fd3e6e00640 1 -- start start 2024-02-02T13:25:17.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3e0151f90 0x7fd3e015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0e10 0x7fd3e00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e6e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e0070f80 con 0x7fd3e006f4f0 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e0152370 con 0x7fd3e0151f90 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00cb010 con 0x7fd3e00b0e10 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.433+0000 7fd3e4e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0e10 0x7fd3e00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3df400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd3e006f4f0 0x7fd3e00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:50524/0 (socket says 172.21.15.91:50524) 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3df400640 1 -- 172.21.15.91:0/3108283718 learned_addr learned my addr 172.21.15.91:0/3108283718 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:17.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/3108283718 >> v2:172.21.15.113:3300/0 conn(0x7fd3e0151f90 0x7fd3e015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3dfe00640 1 -- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0e10 msgr2=0x7fd3e00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:17.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0e10 0x7fd3e00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3dfe00640 1 -- 172.21.15.91:0/3108283718 >> v1:172.21.15.113:6789/0 conn(0x7fd3e006f4f0 legacy=0x7fd3e00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.434+0000 7fd3dfe00640 1 -- 172.21.15.91:0/3108283718 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e00cb2f0 con 0x7fd3e0151f90 2024-02-02T13:25:17.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.436+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e0151f90 0x7fd3e015f5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00097a0 tx=0x7fd3d002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66dd2fd5eb6c079e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.436+0000 7fd3dea00640 1 -- 172.21.15.91:0/3108283718 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d002d040 con 0x7fd3e0151f90 2024-02-02T13:25:17.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.436+0000 7fd3dea00640 1 -- 172.21.15.91:0/3108283718 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d0031810 con 0x7fd3e0151f90 2024-02-02T13:25:17.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.436+0000 7fd3dea00640 1 -- 172.21.15.91:0/3108283718 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0031ba0 con 0x7fd3e0151f90 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 -- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e0151f90 msgr2=0x7fd3e015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e0151f90 0x7fd3e015f5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00097a0 tx=0x7fd3d002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 -- 172.21.15.91:0/3108283718 shutdown_connections 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0e10 0x7fd3e00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/3108283718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e0151f90 0x7fd3e015f5a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 -- 172.21.15.91:0/3108283718 >> 172.21.15.91:0/3108283718 conn(0x7fd3e00cec80 msgr2=0x7fd3e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 -- 172.21.15.91:0/3108283718 shutdown_connections 2024-02-02T13:25:17.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.437+0000 7fd3e6e00640 1 -- 172.21.15.91:0/3108283718 wait complete. 2024-02-02T13:25:17.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.439+0000 7fd3e6e00640 1 Processor -- start 2024-02-02T13:25:17.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.440+0000 7fd3e6e00640 1 -- start start 2024-02-02T13:25:17.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e006f4f0 0x7fd3e00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e00b0e10 0x7fd3e00b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c2e70 con 0x7fd3e0151f90 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e01f07c0 con 0x7fd3e00b0e10 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e015d610 con 0x7fd3e006f4f0 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47648/0 (socket says 172.21.15.91:47648) 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3e4e00640 1 -- 172.21.15.91:0/4083026248 learned_addr learned my addr 172.21.15.91:0/4083026248 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3df400640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e00b0e10 0x7fd3e00b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.441+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e006f4f0 0x7fd3e00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 -- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e006f4f0 msgr2=0x7fd3e00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e006f4f0 0x7fd3e00b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 -- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e00b0e10 msgr2=0x7fd3e00b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e00b0e10 0x7fd3e00b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 -- 172.21.15.91:0/4083026248 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d0009400 con 0x7fd3e0151f90 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e4e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800ebe0 tx=0x7fd3d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8009a70 con 0x7fd3e0151f90 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d800f040 con 0x7fd3e0151f90 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8014480 con 0x7fd3e0151f90 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e015d8f0 con 0x7fd3e0151f90 2024-02-02T13:25:17.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.442+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e015deb0 con 0x7fd3e0151f90 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.444+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e00c2fe0 con 0x7fd3e0151f90 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.444+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3d80145e0 con 0x7fd3e0151f90 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.445+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 0x7fd3b806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.445+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3d8089520 con 0x7fd3e0151f90 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.446+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 0x7fd3b806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.446+0000 7fd3dfe00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 0x7fd3b806c920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0036000 tx=0x7fd3d003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.450+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd3d8054ba0 con 0x7fd3e0151f90 2024-02-02T13:25:17.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.638+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3e00714b0 con 0x7fd3e0151f90 2024-02-02T13:25:17.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.639+0000 7fd3d7e00640 1 -- 172.21.15.91:0/4083026248 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3d8018020 con 0x7fd3e0151f90 2024-02-02T13:25:17.641 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:17.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 msgr2=0x7fd3b806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 0x7fd3b806c920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0036000 tx=0x7fd3d003c040 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 msgr2=0x7fd3e015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800ebe0 tx=0x7fd3d800c490 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 shutdown_connections 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a460 0x7fd3b806c920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e015c8a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e00b0e10 0x7fd3e00b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 --2- 172.21.15.91:0/4083026248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e006f4f0 0x7fd3e00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 >> 172.21.15.91:0/4083026248 conn(0x7fd3e00cec80 msgr2=0x7fd3e0159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 shutdown_connections 2024-02-02T13:25:17.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.642+0000 7fd3e6e00640 1 -- 172.21.15.91:0/4083026248 wait complete. 2024-02-02T13:25:17.663 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:17.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.863+0000 7fc981c00640 1 Processor -- start 2024-02-02T13:25:17.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.863+0000 7fc981c00640 1 -- start start 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc97c160830 0x7fc97c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 0x7fc97c0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c070c40 con 0x7fc97c160830 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0ce210 con 0x7fc97c0b1d70 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.864+0000 7fc981c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0ce4f0 con 0x7fc97c070270 2024-02-02T13:25:17.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc97c160830 0x7fc97c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc97c160830 0x7fc97c0b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47650/0 (socket says 172.21.15.91:47650) 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 -- 172.21.15.91:0/1733713672 learned_addr learned my addr 172.21.15.91:0/1733713672 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc973e00640 1 --2- 172.21.15.91:0/1733713672 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 0x7fc97c0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97b400640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 -- 172.21.15.91:0/1733713672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 msgr2=0x7fc97c070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:17.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 -- 172.21.15.91:0/1733713672 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 msgr2=0x7fc97c0711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:17.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 --2- 172.21.15.91:0/1733713672 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 0x7fc97c0711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc97aa00640 1 -- 172.21.15.91:0/1733713672 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0ce7d0 con 0x7fc97c160830 2024-02-02T13:25:17.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.865+0000 7fc973e00640 1 --2- 172.21.15.91:0/1733713672 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 0x7fc97c0711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.866+0000 7fc97aa00640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c160830 0x7fc97c0b61d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0b5fc0 tx=0x7fc96c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934fea45f9fdece2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc97a000640 1 -- 172.21.15.91:0/1733713672 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0317e0 con 0x7fc97c160830 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc97a000640 1 -- 172.21.15.91:0/1733713672 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c031940 con 0x7fc97c160830 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc97a000640 1 -- 172.21.15.91:0/1733713672 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c031cd0 con 0x7fc97c160830 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 -- 172.21.15.91:0/1733713672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c160830 msgr2=0x7fc97c0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c160830 0x7fc97c0b61d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0b5fc0 tx=0x7fc96c02b880 comp rx=0 tx=0).stop 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 -- 172.21.15.91:0/1733713672 shutdown_connections 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 --2- 172.21.15.91:0/1733713672 >> v2:172.21.15.113:3300/0 conn(0x7fc97c0b1d70 0x7fc97c0711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 --2- 172.21.15.91:0/1733713672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c160830 0x7fc97c0b61d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 -- 172.21.15.91:0/1733713672 >> 172.21.15.91:0/1733713672 conn(0x7fc97c0d3030 msgr2=0x7fc97c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:17.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 -- 172.21.15.91:0/1733713672 shutdown_connections 2024-02-02T13:25:17.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.867+0000 7fc981c00640 1 -- 172.21.15.91:0/1733713672 wait complete. 2024-02-02T13:25:17.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.869+0000 7fc981c00640 1 Processor -- start 2024-02-02T13:25:17.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.869+0000 7fc981c00640 1 -- start start 2024-02-02T13:25:17.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c160830 0x7fc97c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0cbc80 con 0x7fc97c0b1d70 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c7a60 con 0x7fc97c160830 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc981c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0cc5e0 con 0x7fc97c070270 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.870+0000 7fc973e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47666/0 (socket says 172.21.15.91:47666) 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 -- 172.21.15.91:0/840555870 learned_addr learned my addr 172.21.15.91:0/840555870 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc97aa00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc97b400640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c160830 0x7fc97c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 -- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 msgr2=0x7fc97c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 -- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c160830 msgr2=0x7fc97c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c160830 0x7fc97c159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 -- 172.21.15.91:0/840555870 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0094d0 con 0x7fc97c0b1d70 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc973e00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c7f0 tx=0x7fc96400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964007a30 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964007b90 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400ce40 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.871+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c0c92d0 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.872+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0cb2e0 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.873+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c066cf0 con 0x7fc97c0b1d70 2024-02-02T13:25:17.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.874+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc9640113f0 con 0x7fc97c0b1d70 2024-02-02T13:25:17.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.874+0000 7fc972a00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:17.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.875+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc964088a60 con 0x7fc97c0b1d70 2024-02-02T13:25:17.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.875+0000 7fc97aa00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:17.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.875+0000 7fc97aa00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc96c036000 tx=0x7fc96c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:17.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:17.879+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc964054050 con 0x7fc97c0b1d70 2024-02-02T13:25:18.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.072+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc97c0d3f40 con 0x7fc97c0b1d70 2024-02-02T13:25:18.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.075+0000 7fc972a00640 1 -- 172.21.15.91:0/840555870 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fc96401f090 con 0x7fc97c0b1d70 2024-02-02T13:25:18.077 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:18.077 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 msgr2=0x7fc95406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc96c036000 tx=0x7fc96c03d040 comp rx=0 tx=0).stop 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 msgr2=0x7fc97c0cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c7f0 tx=0x7fc96400cbf0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 shutdown_connections 2024-02-02T13:25:18.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc95406a430 0x7fc95406c8f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc97c160830 0x7fc97c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc97c0b1d70 0x7fc97c0cc0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 --2- 172.21.15.91:0/840555870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97c070270 0x7fc97c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.077+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 >> 172.21.15.91:0/840555870 conn(0x7fc97c0d3030 msgr2=0x7fc97c160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.078+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 shutdown_connections 2024-02-02T13:25:18.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.078+0000 7fc981c00640 1 -- 172.21.15.91:0/840555870 wait complete. 2024-02-02T13:25:18.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:18.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:18.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.306+0000 7fdcf5000640 1 Processor -- start 2024-02-02T13:25:18.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.306+0000 7fdcf5000640 1 -- start start 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.307+0000 7fdcf5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdcf00acfe0 0x7fdcf006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.307+0000 7fdcf5000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdcf00b0700 0x7fdcf00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.307+0000 7fdcf5000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf015d870 con 0x7fdcf015d490 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.307+0000 7fdcf5000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf006bce0 con 0x7fdcf00b0700 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.307+0000 7fdcf5000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf0161220 con 0x7fdcf00acfe0 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcede00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdcf015d490 0x7fdcf00ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34206/0 (socket says 172.21.15.91:34206) 2024-02-02T13:25:18.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcede00640 1 -- 172.21.15.91:0/2397139850 learned_addr learned my addr 172.21.15.91:0/2397139850 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcee800640 1 --2- 172.21.15.91:0/2397139850 >> v2:172.21.15.91:3300/0 conn(0x7fdcf00b0700 0x7fdcf00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdced400640 1 --2- 172.21.15.91:0/2397139850 >> v2:172.21.15.113:3300/0 conn(0x7fdcf00acfe0 0x7fdcf006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdceca00640 1 -- 172.21.15.91:0/2397139850 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876346413 0 0) 0x7fdcf015d870 con 0x7fdcf015d490 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcee800640 1 -- 172.21.15.91:0/2397139850 >> v2:172.21.15.113:3300/0 conn(0x7fdcf00acfe0 msgr2=0x7fdcf006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcee800640 1 --2- 172.21.15.91:0/2397139850 >> v2:172.21.15.113:3300/0 conn(0x7fdcf00acfe0 0x7fdcf006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcee800640 1 -- 172.21.15.91:0/2397139850 >> v1:172.21.15.91:6789/0 conn(0x7fdcf015d490 legacy=0x7fdcf00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.308+0000 7fdcee800640 1 -- 172.21.15.91:0/2397139850 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00b1050 con 0x7fdcf00b0700 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.309+0000 7fdcee800640 1 --2- 172.21.15.91:0/2397139850 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf00b0700 0x7fdcf00b0b10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b1870 tx=0x7fdce802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bc6fddddb95969 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdceca00640 1 -- 172.21.15.91:0/2397139850 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce802cc80 con 0x7fdcf00b0700 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdceca00640 1 -- 172.21.15.91:0/2397139850 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce802cde0 con 0x7fdcf00b0700 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdceca00640 1 -- 172.21.15.91:0/2397139850 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8034660 con 0x7fdcf00b0700 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 -- 172.21.15.91:0/2397139850 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf00b0700 msgr2=0x7fdcf00b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 --2- 172.21.15.91:0/2397139850 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf00b0700 0x7fdcf00b0b10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b1870 tx=0x7fdce802bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 -- 172.21.15.91:0/2397139850 shutdown_connections 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 --2- 172.21.15.91:0/2397139850 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf00b0700 0x7fdcf00b0b10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 --2- 172.21.15.91:0/2397139850 >> v2:172.21.15.113:3300/0 conn(0x7fdcf00acfe0 0x7fdcf006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 -- 172.21.15.91:0/2397139850 >> 172.21.15.91:0/2397139850 conn(0x7fdcf00cec80 msgr2=0x7fdcf00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 -- 172.21.15.91:0/2397139850 shutdown_connections 2024-02-02T13:25:18.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.310+0000 7fdcf5000640 1 -- 172.21.15.91:0/2397139850 wait complete. 2024-02-02T13:25:18.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.312+0000 7fdcf5000640 1 Processor -- start 2024-02-02T13:25:18.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.312+0000 7fdcf5000640 1 -- start start 2024-02-02T13:25:18.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 0x7fdcf01496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b0700 0x7fdcf01462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf0149be0 con 0x7fdcf015d490 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00c8720 con 0x7fdcf00acfe0 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcf5000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00c3cd0 con 0x7fdcf00b0700 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcee800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcee800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47692/0 (socket says 172.21.15.91:47692) 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.313+0000 7fdcee800640 1 -- 172.21.15.91:0/273437191 learned_addr learned my addr 172.21.15.91:0/273437191 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdced400640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b0700 0x7fdcf01462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcede00640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 0x7fdcf01496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 -- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b0700 msgr2=0x7fdcf01462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b0700 0x7fdcf01462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 -- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 msgr2=0x7fdcf01496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 0x7fdcf01496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 -- 172.21.15.91:0/273437191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009a70 con 0x7fdcf015d490 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcede00640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 0x7fdcf01496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdcee800640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009e80 tx=0x7fdce8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8040650 con 0x7fdcf015d490 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80407b0 con 0x7fdcf015d490 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.314+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8040b60 con 0x7fdcf015d490 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.315+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf00c3fb0 con 0x7fdcf015d490 2024-02-02T13:25:18.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.315+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf00c4570 con 0x7fdcf015d490 2024-02-02T13:25:18.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.316+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf00b2ef0 con 0x7fdcf015d490 2024-02-02T13:25:18.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.317+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdce8040cc0 con 0x7fdcf015d490 2024-02-02T13:25:18.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.317+0000 7fdce6000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.318+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdce80ab5b0 con 0x7fdcf015d490 2024-02-02T13:25:18.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.318+0000 7fdcede00640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.318+0000 7fdcede00640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdce0007fd0 tx=0x7fdce0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.324+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdce8076c10 con 0x7fdcf015d490 2024-02-02T13:25:18.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.513+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcf0066cf0 con 0x7fdcf015d490 2024-02-02T13:25:18.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.514+0000 7fdce6000640 1 -- 172.21.15.91:0/273437191 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdce8009450 con 0x7fdcf015d490 2024-02-02T13:25:18.515 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 msgr2=0x7fdcc806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdce0007fd0 tx=0x7fdce0009040 comp rx=0 tx=0).stop 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 msgr2=0x7fdcf01613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009e80 tx=0x7fdce8004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 shutdown_connections 2024-02-02T13:25:18.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b0700 0x7fdcf01462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf015d490 0x7fdcf01613c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 --2- 172.21.15.91:0/273437191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00acfe0 0x7fdcf01496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 >> 172.21.15.91:0/273437191 conn(0x7fdcf00cec80 msgr2=0x7fdcf00afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 shutdown_connections 2024-02-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.516+0000 7fdcf5000640 1 -- 172.21.15.91:0/273437191 wait complete. 2024-02-02T13:25:18.537 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:18.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.748+0000 7fe69f200640 1 Processor -- start 2024-02-02T13:25:18.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.748+0000 7fe69f200640 1 -- start start 2024-02-02T13:25:18.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.749+0000 7fe69f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe69815aec0 0x7fe6980cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.749+0000 7fe69f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69815b2a0 0x7fe69806c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.749+0000 7fe69f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69806c1c0 con 0x7fe69806cb90 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.749+0000 7fe69f200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69806b5f0 con 0x7fe69815aec0 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.749+0000 7fe69f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69806b890 con 0x7fe69815b2a0 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe69815aec0 0x7fe6980cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe69815aec0 0x7fe6980cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47696/0 (socket says 172.21.15.91:47696) 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 -- 172.21.15.91:0/733782316 learned_addr learned my addr 172.21.15.91:0/733782316 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69d400640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69815b2a0 0x7fe69806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 -- 172.21.15.91:0/733782316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69815b2a0 msgr2=0x7fe69806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69815b2a0 0x7fe69806c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 -- 172.21.15.91:0/733782316 >> v1:172.21.15.91:6789/0 conn(0x7fe69806cb90 legacy=0x7fe6980ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69de00640 1 -- 172.21.15.91:0/733782316 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69806bb70 con 0x7fe69815aec0 2024-02-02T13:25:18.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.750+0000 7fe69ca00640 1 -- 172.21.15.91:0/733782316 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090262203 0 0) 0x7fe69806c1c0 con 0x7fe69806cb90 2024-02-02T13:25:18.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.751+0000 7fe69de00640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980cf5b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe688009870 tx=0x7fe68802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5855587135184ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69ca00640 1 -- 172.21.15.91:0/733782316 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68802f840 con 0x7fe69815aec0 2024-02-02T13:25:18.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69ca00640 1 -- 172.21.15.91:0/733782316 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68802f9a0 con 0x7fe69815aec0 2024-02-02T13:25:18.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69ca00640 1 -- 172.21.15.91:0/733782316 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68802fd30 con 0x7fe69815aec0 2024-02-02T13:25:18.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 -- 172.21.15.91:0/733782316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 msgr2=0x7fe6980cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980cf5b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe688009870 tx=0x7fe68802ba20 comp rx=0 tx=0).stop 2024-02-02T13:25:18.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 -- 172.21.15.91:0/733782316 shutdown_connections 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69815b2a0 0x7fe69806c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 --2- 172.21.15.91:0/733782316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980cf5b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.752+0000 7fe69f200640 1 -- 172.21.15.91:0/733782316 >> 172.21.15.91:0/733782316 conn(0x7fe6980cea90 msgr2=0x7fe6980ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.753+0000 7fe69f200640 1 -- 172.21.15.91:0/733782316 shutdown_connections 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.753+0000 7fe69f200640 1 -- 172.21.15.91:0/733782316 wait complete. 2024-02-02T13:25:18.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.754+0000 7fe69f200640 1 Processor -- start 2024-02-02T13:25:18.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.754+0000 7fe69f200640 1 -- start start 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 0x7fe6980b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69815b2a0 0x7fe6980ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980ac650 con 0x7fe69815aec0 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980ab5b0 con 0x7fe69815b2a0 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.755+0000 7fe69f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980ab890 con 0x7fe69806cb90 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47704/0 (socket says 172.21.15.91:47704) 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 -- 172.21.15.91:0/1077499464 learned_addr learned my addr 172.21.15.91:0/1077499464 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69de00640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 0x7fe6980b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 -- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 msgr2=0x7fe6980b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 0x7fe6980b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69e800640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69815b2a0 0x7fe6980ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 -- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69815b2a0 msgr2=0x7fe6980ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69815b2a0 0x7fe6980ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 -- 172.21.15.91:0/1077499464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880094d0 con 0x7fe69815aec0 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69de00640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 0x7fe6980b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:18.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.756+0000 7fe69d400640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe68000d5f0 tx=0x7fe68000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.757+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800125f0 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.757+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680012750 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.757+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680012b00 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.757+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6980abb70 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.757+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6980ac130 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.758+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6980ac2f0 con 0x7fe69815aec0 2024-02-02T13:25:18.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.759+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe680012c60 con 0x7fe69815aec0 2024-02-02T13:25:18.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.760+0000 7fe696000640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 0x7fe67006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:18.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.760+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe680088e30 con 0x7fe69815aec0 2024-02-02T13:25:18.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.760+0000 7fe69de00640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 0x7fe67006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:18.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.761+0000 7fe69de00640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 0x7fe67006c8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe688036000 tx=0x7fe688008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:18.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.765+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe680054380 con 0x7fe69815aec0 2024-02-02T13:25:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.956+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe69804dff0 con 0x7fe69815aec0 2024-02-02T13:25:18.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.957+0000 7fe696000640 1 -- 172.21.15.91:0/1077499464 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6800579a0 con 0x7fe69815aec0 2024-02-02T13:25:18.959 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 msgr2=0x7fe67006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 0x7fe67006c8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe688036000 tx=0x7fe688008030 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 msgr2=0x7fe6980acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe68000d5f0 tx=0x7fe68000dab0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 shutdown_connections 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe67006a430 0x7fe67006c8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69815b2a0 0x7fe6980ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe69815aec0 0x7fe6980acb40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 --2- 172.21.15.91:0/1077499464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806cb90 0x7fe6980b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 >> 172.21.15.91:0/1077499464 conn(0x7fe6980cea90 msgr2=0x7fe6980b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:18.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 shutdown_connections 2024-02-02T13:25:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:18.960+0000 7fe69f200640 1 -- 172.21.15.91:0/1077499464 wait complete. 2024-02-02T13:25:18.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:19.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.190+0000 7fc61ee00640 1 Processor -- start 2024-02-02T13:25:19.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.190+0000 7fc61ee00640 1 -- start start 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc61ee00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc618070980 0x7fc6180b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc61ee00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180ce210 con 0x7fc618160830 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc61ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180ce4f0 con 0x7fc6180b1d70 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc61ee00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180ce7d0 con 0x7fc618070980 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc617400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc618070980 0x7fc6180b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc617e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc618160830 0x7fc618070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34210/0 (socket says 172.21.15.91:34210) 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.192+0000 7fc617e00640 1 -- 172.21.15.91:0/4266395972 learned_addr learned my addr 172.21.15.91:0/4266395972 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.193+0000 7fc617400640 1 -- 172.21.15.91:0/4266395972 >> v1:172.21.15.113:6789/0 conn(0x7fc6180b1d70 legacy=0x7fc6180d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.193+0000 7fc617400640 1 -- 172.21.15.91:0/4266395972 >> v1:172.21.15.91:6789/0 conn(0x7fc618160830 legacy=0x7fc618070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.193+0000 7fc617400640 1 -- 172.21.15.91:0/4266395972 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180d17b0 con 0x7fc618070980 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.194+0000 7fc617400640 1 --2- 172.21.15.91:0/4266395972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618070980 0x7fc6180b61d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6000097a0 tx=0x7fc60002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aa633bcc08b71e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.194+0000 7fc616a00640 1 -- 172.21.15.91:0/4266395972 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60002e070 con 0x7fc618070980 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.194+0000 7fc616a00640 1 -- 172.21.15.91:0/4266395972 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6000315a0 con 0x7fc618070980 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.194+0000 7fc616a00640 1 -- 172.21.15.91:0/4266395972 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600031930 con 0x7fc618070980 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 -- 172.21.15.91:0/4266395972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618070980 msgr2=0x7fc6180b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 --2- 172.21.15.91:0/4266395972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618070980 0x7fc6180b61d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6000097a0 tx=0x7fc60002b840 comp rx=0 tx=0).stop 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 -- 172.21.15.91:0/4266395972 shutdown_connections 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 --2- 172.21.15.91:0/4266395972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618070980 0x7fc6180b61d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 -- 172.21.15.91:0/4266395972 >> 172.21.15.91:0/4266395972 conn(0x7fc6180d3030 msgr2=0x7fc6180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 -- 172.21.15.91:0/4266395972 shutdown_connections 2024-02-02T13:25:19.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.195+0000 7fc61ee00640 1 -- 172.21.15.91:0/4266395972 wait complete. 2024-02-02T13:25:19.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.197+0000 7fc61ee00640 1 Processor -- start 2024-02-02T13:25:19.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.197+0000 7fc61ee00640 1 -- start start 2024-02-02T13:25:19.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc618070980 0x7fc6180b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6180b1d70 0x7fc61815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61814cc70 con 0x7fc618160830 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61814c060 con 0x7fc6180b1d70 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61814c340 con 0x7fc618070980 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47710/0 (socket says 172.21.15.91:47710) 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 -- 172.21.15.91:0/1648915246 learned_addr learned my addr 172.21.15.91:0/1648915246 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc617e00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc618070980 0x7fc6180b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 -- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc618070980 msgr2=0x7fc6180b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc617400640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6180b1d70 0x7fc61815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc618070980 0x7fc6180b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 -- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6180b1d70 msgr2=0x7fc61815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6180b1d70 0x7fc61815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.198+0000 7fc61ce00640 1 -- 172.21.15.91:0/1648915246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009400 con 0x7fc618160830 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc61ce00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c780 tx=0x7fc61000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610004b60 con 0x7fc618160830 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61000ce90 con 0x7fc618160830 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000a850 con 0x7fc618160830 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61814c620 con 0x7fc618160830 2024-02-02T13:25:19.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.199+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61814eca0 con 0x7fc618160830 2024-02-02T13:25:19.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.200+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc618066cf0 con 0x7fc618160830 2024-02-02T13:25:19.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.201+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc61000a9b0 con 0x7fc618160830 2024-02-02T13:25:19.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.202+0000 7fc60fe00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 0x7fc5f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.202+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc610088b50 con 0x7fc618160830 2024-02-02T13:25:19.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.202+0000 7fc617e00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 0x7fc5f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.203+0000 7fc617e00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 0x7fc5f006c920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc608005880 tx=0x7fc608009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.207+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc610054130 con 0x7fc618160830 2024-02-02T13:25:19.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.399+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6180d3f40 con 0x7fc618160830 2024-02-02T13:25:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.400+0000 7fc60fe00640 1 -- 172.21.15.91:0/1648915246 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc610057750 con 0x7fc618160830 2024-02-02T13:25:19.401 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 msgr2=0x7fc5f006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 0x7fc5f006c920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc608005880 tx=0x7fc608009040 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 msgr2=0x7fc618160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c780 tx=0x7fc61000cc40 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 shutdown_connections 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5f006a460 0x7fc5f006c920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc618160830 0x7fc618160f40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6180b1d70 0x7fc61815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 --2- 172.21.15.91:0/1648915246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc618070980 0x7fc6180b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 >> 172.21.15.91:0/1648915246 conn(0x7fc6180d3030 msgr2=0x7fc6180d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 shutdown_connections 2024-02-02T13:25:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.402+0000 7fc61ee00640 1 -- 172.21.15.91:0/1648915246 wait complete. 2024-02-02T13:25:19.432 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-02-02T13:25:19.432 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-02-02T13:25:19.432 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-02-02T13:25:19.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.644+0000 7f0593200640 1 Processor -- start 2024-02-02T13:25:19.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.644+0000 7f0593200640 1 -- start start 2024-02-02T13:25:19.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0593200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f059415b2a0 0x7f05940cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0593200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f05940ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0593200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059406c1c0 con 0x7f059415aec0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0593200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406b5f0 con 0x7f059415b2a0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0593200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059406cd60 con 0x7f059406f3f0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0591e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f059415aec0 0x7f059406c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34218/0 (socket says 172.21.15.91:34218) 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0591e00640 1 -- 172.21.15.91:0/2780462195 learned_addr learned my addr 172.21.15.91:0/2780462195 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0592800640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f05940ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f058be00640 1 --2- 172.21.15.91:0/2780462195 >> v2:172.21.15.113:3300/0 conn(0x7f059415b2a0 0x7f05940cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.646+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175380858 0 0) 0x7f059406c1c0 con 0x7f059415aec0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0574003520 con 0x7f059415aec0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186671978 0 0) 0x7f0574003520 con 0x7f059415aec0 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 msgr2=0x7f05940ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f05940ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 >> v2:172.21.15.113:3300/0 conn(0x7f059415b2a0 msgr2=0x7f05940cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 --2- 172.21.15.91:0/2780462195 >> v2:172.21.15.113:3300/0 conn(0x7f059415b2a0 0x7f05940cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059406b890 con 0x7f059415aec0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0584001670 con 0x7f059415aec0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 >> v1:172.21.15.91:6789/0 conn(0x7f059415aec0 legacy=0x7f059406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f0574003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0574003f40 0x7f05740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0574004d00 0x7f0574005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059406b5f0 con 0x7f0574004d00 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059406cd60 con 0x7f0574003f40 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.647+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057400ba00 con 0x7f059406f3f0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0574004d00 0x7f0574005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0591e00640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f0574003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f058be00640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0574003f40 0x7f05740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 -- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 msgr2=0x7f0574003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059406f3f0 0x7f0574003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 -- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0574003f40 msgr2=0x7f05740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0574003f40 0x7f05740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.648+0000 7f0592800640 1 -- 172.21.15.91:0/2780462195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574003520 con 0x7f0574004d00 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.649+0000 7f0592800640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0574004d00 0x7f0574005180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f058c009fc0 tx=0x7f05740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e716275d2452cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.649+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c02cc20 con 0x7f0574004d00 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.649+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00f040 con 0x7f0574004d00 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.649+0000 7f0591400640 1 -- 172.21.15.91:0/2780462195 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c034440 con 0x7f0574004d00 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 -- 172.21.15.91:0/2780462195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0574004d00 msgr2=0x7f0574005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 --2- 172.21.15.91:0/2780462195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0574004d00 0x7f0574005180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f058c009fc0 tx=0x7f05740057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0592800640 1 -- 172.21.15.91:0/2780462195 reap_dead start 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 -- 172.21.15.91:0/2780462195 shutdown_connections 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 -- 172.21.15.91:0/2780462195 >> 172.21.15.91:0/2780462195 conn(0x7f05940cea90 msgr2=0x7f05940ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 -- 172.21.15.91:0/2780462195 shutdown_connections 2024-02-02T13:25:19.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.650+0000 7f0593200640 1 -- 172.21.15.91:0/2780462195 wait complete. 2024-02-02T13:25:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.652+0000 7f0593200640 1 Processor -- start 2024-02-02T13:25:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.652+0000 7f0593200640 1 -- start start 2024-02-02T13:25:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0574004d00 0x7f05940c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05940c6f30 0x7f059414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594156d00 con 0x7f059414f0a0 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059414f500 con 0x7f05940c6f30 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0593200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05940c7560 con 0x7f0574004d00 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f058be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f058be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47728/0 (socket says 172.21.15.91:47728) 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f058be00640 1 -- 172.21.15.91:0/3038360913 learned_addr learned my addr 172.21.15.91:0/3038360913 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.653+0000 7f0591e00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0574004d00 0x7f05940c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 -- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0574004d00 msgr2=0x7f05940c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0574004d00 0x7f05940c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 -- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05940c6f30 msgr2=0x7f059414ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05940c6f30 0x7f059414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:19.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 -- 172.21.15.91:0/3038360913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940c7840 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058be00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f057c011600 tx=0x7f057c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c015870 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c0159d0 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c015d40 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05940c7b20 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.654+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05940c80e0 con 0x7f059414f0a0 2024-02-02T13:25:19.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.655+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059404dff0 con 0x7f059414f0a0 2024-02-02T13:25:19.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.656+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f057c00f490 con 0x7f059414f0a0 2024-02-02T13:25:19.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.657+0000 7f058a000640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 0x7f056c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:19.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.657+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f057c088e00 con 0x7f059414f0a0 2024-02-02T13:25:19.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.658+0000 7f0591e00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 0x7f056c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:19.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.658+0000 7f0591e00640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 0x7f056c06c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0584001eb0 tx=0x7f0584000f50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:19.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.662+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f057c0543f0 con 0x7f059414f0a0 2024-02-02T13:25:19.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:19.920+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f05940c4f20 con 0x7f059414f0a0 2024-02-02T13:25:20.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.459+0000 7f058a000640 1 -- 172.21.15.91:0/3038360913 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v55) v1 ==== 133+0+0 (secure 0 0 0) 0x7f057c057a10 con 0x7f059414f0a0 2024-02-02T13:25:20.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 msgr2=0x7f056c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 0x7f056c06c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0584001eb0 tx=0x7f0584000f50 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 msgr2=0x7f05940c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f057c011600 tx=0x7f057c011ac0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 shutdown_connections 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f056c06a430 0x7f056c06c8f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05940c6f30 0x7f059414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f059414f0a0 0x7f05940c69f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 --2- 172.21.15.91:0/3038360913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0574004d00 0x7f05940c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 >> 172.21.15.91:0/3038360913 conn(0x7f05940cea90 msgr2=0x7f059406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 shutdown_connections 2024-02-02T13:25:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.463+0000 7f0593200640 1 -- 172.21.15.91:0/3038360913 wait complete. 2024-02-02T13:25:20.484 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-02-02T13:25:20.700 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_objectstore":"memstore"} 2024-02-02T13:25:20.700 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-02-02T13:25:20.700 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:25:20.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.906+0000 7fa274c00640 1 Processor -- start 2024-02-02T13:25:20.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.906+0000 7fa274c00640 1 -- start start 2024-02-02T13:25:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa274c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2700b2260 0x7fa2700b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa274c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2700cd930 con 0x7fa2700b6260 2024-02-02T13:25:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa274c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2700d1680 con 0x7fa2700d0070 2024-02-02T13:25:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa274c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2700b8f90 con 0x7fa2700b2260 2024-02-02T13:25:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa26e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa2700b6260 0x7fa2700d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39310/0 (socket says 172.21.15.91:39310) 2024-02-02T13:25:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa26e800640 1 -- 172.21.15.91:0/927439328 learned_addr learned my addr 172.21.15.91:0/927439328 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:20.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267842065 0 0) 0x7fa268001490 con 0x7fa2700b6260 2024-02-02T13:25:20.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa250003520 con 0x7fa2700b6260 2024-02-02T13:25:20.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.908+0000 7fa26d400640 1 --2- 172.21.15.91:0/927439328 >> v2:172.21.15.113:3300/0 conn(0x7fa2700b2260 0x7fa2700b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2716134805 0 0) 0x7fa250003520 con 0x7fa2700b6260 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 >> v2:172.21.15.113:3300/0 conn(0x7fa2700b2260 msgr2=0x7fa2700b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 --2- 172.21.15.91:0/927439328 >> v2:172.21.15.113:3300/0 conn(0x7fa2700b2260 0x7fa2700b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 >> v1:172.21.15.113:6789/0 conn(0x7fa2700d0070 legacy=0x7fa2700d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700d1960 con 0x7fa2700b6260 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa268001b80 con 0x7fa2700b6260 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 >> v1:172.21.15.91:6789/0 conn(0x7fa2700b6260 legacy=0x7fa2700d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 0x7fa250003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500047c0 0x7fa250004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2500051d0 0x7fa25000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700b8f90 con 0x7fa2500039c0 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25000c120 con 0x7fa2500047c0 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25000c2e0 con 0x7fa2500051d0 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.909+0000 7fa26de00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 0x7fa250003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26d400640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500047c0 0x7fa250004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26de00640 1 -- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2500051d0 msgr2=0x7fa25000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26de00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2500051d0 0x7fa25000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26de00640 1 -- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500047c0 msgr2=0x7fa250004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26de00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500047c0 0x7fa250004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26e800640 1 -- 172.21.15.91:0/927439328 reap_dead start 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26de00640 1 -- 172.21.15.91:0/927439328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250003520 con 0x7fa2500039c0 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.910+0000 7fa26d400640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500047c0 0x7fa250004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.911+0000 7fa26de00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 0x7fa250003de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa260001750 tx=0x7fa2500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3a35e090667a32 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.911+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600052d0 con 0x7fa2500039c0 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.911+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26002e050 con 0x7fa2500039c0 2024-02-02T13:25:20.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.911+0000 7fa26ca00640 1 -- 172.21.15.91:0/927439328 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260003830 con 0x7fa2500039c0 2024-02-02T13:25:20.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 -- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 msgr2=0x7fa250003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 0x7fa250003de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa260001750 tx=0x7fa2500057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 -- 172.21.15.91:0/927439328 shutdown_connections 2024-02-02T13:25:20.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 --2- 172.21.15.91:0/927439328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2500039c0 0x7fa250003de0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 -- 172.21.15.91:0/927439328 >> 172.21.15.91:0/927439328 conn(0x7fa270070660 msgr2=0x7fa270070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:20.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 -- 172.21.15.91:0/927439328 shutdown_connections 2024-02-02T13:25:20.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.912+0000 7fa274c00640 1 -- 172.21.15.91:0/927439328 wait complete. 2024-02-02T13:25:20.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.913+0000 7fa274c00640 1 Processor -- start 2024-02-02T13:25:20.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.914+0000 7fa274c00640 1 -- start start 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.914+0000 7fa274c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500039c0 0x7fa27014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa274c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2700c83a0 0x7fa270160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa274c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa274c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270149300 con 0x7fa27014b5e0 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa274c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27014bfa0 con 0x7fa2500039c0 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa274c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c8c80 con 0x7fa2700c83a0 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52534/0 (socket says 172.21.15.91:52534) 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 -- 172.21.15.91:0/3685410871 learned_addr learned my addr 172.21.15.91:0/3685410871 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 -- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2700c83a0 msgr2=0x7fa270160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2700c83a0 0x7fa270160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 -- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500039c0 msgr2=0x7fa27014c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500039c0 0x7fa27014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.915+0000 7fa26e800640 1 -- 172.21.15.91:0/3685410871 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700c8f60 con 0x7fa27014b5e0 2024-02-02T13:25:20.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa26e800640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2680015f0 tx=0x7fa26800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268004d60 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800a5b0 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2700c9240 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800a960 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.916+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2700c9800 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.917+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2700cb250 con 0x7fa27014b5e0 2024-02-02T13:25:20.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.918+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa26800aac0 con 0x7fa27014b5e0 2024-02-02T13:25:20.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.919+0000 7fa266000640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 0x7fa24806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:20.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.919+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa268087c80 con 0x7fa27014b5e0 2024-02-02T13:25:20.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.919+0000 7fa26de00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 0x7fa24806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:20.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.920+0000 7fa26de00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 0x7fa24806c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa260033000 tx=0x7fa26002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:20.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:20.923+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa2700cb250 con 0x7fa27014b5e0 2024-02-02T13:25:21.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.113+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2700cd930 con 0x7fa27014b5e0 2024-02-02T13:25:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.121+0000 7fa266000640 1 -- 172.21.15.91:0/3685410871 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2700cd930 con 0x7fa27014b5e0 2024-02-02T13:25:21.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 msgr2=0x7fa24806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 0x7fa24806c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa260033000 tx=0x7fa26002f000 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 msgr2=0x7fa27014ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2680015f0 tx=0x7fa26800cc00 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 shutdown_connections 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa24806a460 0x7fa24806c920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2700c83a0 0x7fa270160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa27014b5e0 0x7fa27014ba60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 --2- 172.21.15.91:0/3685410871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2500039c0 0x7fa27014c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 >> 172.21.15.91:0/3685410871 conn(0x7fa270070660 msgr2=0x7fa270074020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 shutdown_connections 2024-02-02T13:25:21.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.124+0000 7fa274c00640 1 -- 172.21.15.91:0/3685410871 wait complete. 2024-02-02T13:25:21.144 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:25:21.144 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:25:21.248 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:25:21.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:25:21.374 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:21.374 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:21.432 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:21.432 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:21.484 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:21.484 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:21.534 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:21.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.791+0000 7f0780200640 1 Processor -- start 2024-02-02T13:25:21.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.791+0000 7f0780200640 1 -- start start 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.792+0000 7f0780200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f07780b0e10 0x7f07780b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.792+0000 7f0780200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0778070f80 con 0x7f0778151f90 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.792+0000 7f0780200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077806c600 con 0x7f077815f5a0 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.792+0000 7f0780200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077815fa30 con 0x7f07780b0e10 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f077d800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0778151f90 0x7f077806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39312/0 (socket says 172.21.15.91:39312) 2024-02-02T13:25:21.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f077d800640 1 -- 172.21.15.91:0/3589438772 learned_addr learned my addr 172.21.15.91:0/3589438772 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:21.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148371212 0 0) 0x7f0778070f80 con 0x7f0778151f90 2024-02-02T13:25:21.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f077e200640 1 --2- 172.21.15.91:0/3589438772 >> v2:172.21.15.113:3300/0 conn(0x7f07780b0e10 0x7f07780b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:21.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0758003520 con 0x7f0778151f90 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987252763 0 0) 0x7f0758003520 con 0x7f0778151f90 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 >> v2:172.21.15.113:3300/0 conn(0x7f07780b0e10 msgr2=0x7f07780b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.793+0000 7f0777e00640 1 --2- 172.21.15.91:0/3589438772 >> v2:172.21.15.113:3300/0 conn(0x7f07780b0e10 0x7f07780b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 >> v1:172.21.15.113:6789/0 conn(0x7f077815f5a0 legacy=0x7f07780b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778152370 con 0x7f0778151f90 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0768001670 con 0x7f0778151f90 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 >> v1:172.21.15.91:6789/0 conn(0x7f0778151f90 legacy=0x7f077806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07580039c0 0x7f0758003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07580047c0 0x7f0758004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f075800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077815fa30 con 0x7f0758005180 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075800c080 con 0x7f07580039c0 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075800c240 con 0x7f07580047c0 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f075800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:21.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077d800640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07580039c0 0x7f0758003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:21.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 -- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07580047c0 msgr2=0x7f0758004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:21.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07580047c0 0x7f0758004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 -- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07580039c0 msgr2=0x7f0758003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:21.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07580039c0 0x7f0758003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.794+0000 7f077e200640 1 -- 172.21.15.91:0/3589438772 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0758003520 con 0x7f0758005180 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.795+0000 7f077e200640 1 -- 172.21.15.91:0/3589438772 reap_dead start 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f077e200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f075800b980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0770009f90 tx=0x7f07580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b90b86f8636bca8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077002cc20 con 0x7f0758005180 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077000f040 con 0x7f0758005180 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0777e00640 1 -- 172.21.15.91:0/3589438772 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770034440 con 0x7f0758005180 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 -- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 msgr2=0x7f075800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f075800b980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0770009f90 tx=0x7f07580057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 -- 172.21.15.91:0/3589438772 shutdown_connections 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 --2- 172.21.15.91:0/3589438772 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f075800b980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 -- 172.21.15.91:0/3589438772 >> 172.21.15.91:0/3589438772 conn(0x7f07780cec80 msgr2=0x7f07780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.796+0000 7f0780200640 1 -- 172.21.15.91:0/3589438772 shutdown_connections 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.797+0000 7f0780200640 1 -- 172.21.15.91:0/3589438772 wait complete. 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.798+0000 7f0780200640 1 Processor -- start 2024-02-02T13:25:21.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.798+0000 7f0780200640 1 -- start start 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0778150c40 0x7f0778153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0778156df0 0x7f0778157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07781577c0 con 0x7f0758005180 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778154d10 con 0x7f0778156df0 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.799+0000 7f0780200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07781510c0 con 0x7f0778150c40 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52548/0 (socket says 172.21.15.91:52548) 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 -- 172.21.15.91:0/3908994548 learned_addr learned my addr 172.21.15.91:0/3908994548 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:21.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 -- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0778150c40 msgr2=0x7f0778153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0778150c40 0x7f0778153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 -- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0778156df0 msgr2=0x7f0778157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0778156df0 0x7f0778157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 -- 172.21.15.91:0/3908994548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778153c70 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f077d800640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f076800d7a0 tx=0x7f076800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.800+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680074e0 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.801+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768007640 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.801+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680049e0 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.801+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0778153f50 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.801+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0778154510 con 0x7f0758005180 2024-02-02T13:25:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.802+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0778155990 con 0x7f0758005180 2024-02-02T13:25:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.803+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0768004b40 con 0x7f0758005180 2024-02-02T13:25:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.803+0000 7f0775600640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 0x7f075006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.804+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0768088110 con 0x7f0758005180 2024-02-02T13:25:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.804+0000 7f077ce00640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 0x7f075006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.804+0000 7f077ce00640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 0x7f075006c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0760001590 tx=0x7f0760007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:21.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.808+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0768053720 con 0x7f0758005180 2024-02-02T13:25:21.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.997+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0778066cf0 con 0x7f0758005180 2024-02-02T13:25:22.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:21.999+0000 7f0775600640 1 -- 172.21.15.91:0/3908994548 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0768019090 con 0x7f0758005180 2024-02-02T13:25:22.001 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:22.001 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":20,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:24:37.739263+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 msgr2=0x7f075006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 0x7f075006c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0760001590 tx=0x7f0760007990 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 msgr2=0x7f07780c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f076800d7a0 tx=0x7f076800dc60 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 shutdown_connections 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f075006a430 0x7f075006c8f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0778150c40 0x7f0778153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0778156df0 0x7f0778157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 --2- 172.21.15.91:0/3908994548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0758005180 0x7f07780c51e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 >> 172.21.15.91:0/3908994548 conn(0x7f07780cec80 msgr2=0x7f07780cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:22.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 shutdown_connections 2024-02-02T13:25:22.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.002+0000 7f0780200640 1 -- 172.21.15.91:0/3908994548 wait complete. 2024-02-02T13:25:22.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:22.022 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:22.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.226+0000 7f1d40000640 1 Processor -- start 2024-02-02T13:25:22.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.226+0000 7f1d40000640 1 -- start start 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.227+0000 7f1d40000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d3815a010 0x7f1d380c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.227+0000 7f1d40000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d380ce130 0x7f1d380ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.227+0000 7f1d40000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380cc7c0 con 0x7f1d380cd470 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.227+0000 7f1d40000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380ce930 con 0x7f1d3815a010 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.227+0000 7f1d40000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3806c890 con 0x7f1d380ce130 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d3815a010 0x7f1d380c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d3815a010 0x7f1d380c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52562/0 (socket says 172.21.15.91:52562) 2024-02-02T13:25:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 -- 172.21.15.91:0/3615255100 learned_addr learned my addr 172.21.15.91:0/3615255100 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:22.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 -- 172.21.15.91:0/3615255100 >> v2:172.21.15.113:3300/0 conn(0x7f1d380ce130 msgr2=0x7f1d380ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:22.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 --2- 172.21.15.91:0/3615255100 >> v2:172.21.15.113:3300/0 conn(0x7f1d380ce130 0x7f1d380ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 -- 172.21.15.91:0/3615255100 >> v1:172.21.15.113:6789/0 conn(0x7f1d380cd470 legacy=0x7f1d380b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:22.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.228+0000 7f1d3d600640 1 -- 172.21.15.91:0/3615255100 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3806cb70 con 0x7f1d3815a010 2024-02-02T13:25:22.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.230+0000 7f1d3d600640 1 --2- 172.21.15.91:0/3615255100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380c9670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1d380b1870 tx=0x7f1d2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ca090b8709e0e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.230+0000 7f1d37e00640 1 -- 172.21.15.91:0/3615255100 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280317e0 con 0x7f1d3815a010 2024-02-02T13:25:22.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.230+0000 7f1d37e00640 1 -- 172.21.15.91:0/3615255100 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28031940 con 0x7f1d3815a010 2024-02-02T13:25:22.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.230+0000 7f1d37e00640 1 -- 172.21.15.91:0/3615255100 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031cd0 con 0x7f1d3815a010 2024-02-02T13:25:22.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 -- 172.21.15.91:0/3615255100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 msgr2=0x7f1d380c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 --2- 172.21.15.91:0/3615255100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380c9670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1d380b1870 tx=0x7f1d2802b880 comp rx=0 tx=0).stop 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 -- 172.21.15.91:0/3615255100 shutdown_connections 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 --2- 172.21.15.91:0/3615255100 >> v2:172.21.15.113:3300/0 conn(0x7f1d380ce130 0x7f1d380ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 --2- 172.21.15.91:0/3615255100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380c9670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 -- 172.21.15.91:0/3615255100 >> 172.21.15.91:0/3615255100 conn(0x7f1d380cec80 msgr2=0x7f1d380cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 -- 172.21.15.91:0/3615255100 shutdown_connections 2024-02-02T13:25:22.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.231+0000 7f1d40000640 1 -- 172.21.15.91:0/3615255100 wait complete. 2024-02-02T13:25:22.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.233+0000 7f1d40000640 1 Processor -- start 2024-02-02T13:25:22.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.233+0000 7f1d40000640 1 -- start start 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 0x7f1d380b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 0x7f1d380c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380c3e80 con 0x7f1d3815a010 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380c4120 con 0x7f1d380b2130 2024-02-02T13:25:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d40000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380c29e0 con 0x7f1d380ce130 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52564/0 (socket says 172.21.15.91:52564) 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 -- 172.21.15.91:0/1257294135 learned_addr learned my addr 172.21.15.91:0/1257294135 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3d600640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 0x7f1d380b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 -- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 msgr2=0x7f1d380b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 0x7f1d380b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 -- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 msgr2=0x7f1d380c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3e000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 0x7f1d380c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.234+0000 7f1d3cc00640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 0x7f1d380c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d3cc00640 1 -- 172.21.15.91:0/1257294135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d280094d0 con 0x7f1d3815a010 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d3d600640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 0x7f1d380b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d3e000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 0x7f1d380c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:22.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d3cc00640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000b4a0 tx=0x7f1d2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20010610 con 0x7f1d3815a010 2024-02-02T13:25:22.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20010770 con 0x7f1d3815a010 2024-02-02T13:25:22.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20010b20 con 0x7f1d3815a010 2024-02-02T13:25:22.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d380c2700 con 0x7f1d3815a010 2024-02-02T13:25:22.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.235+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d380c46a0 con 0x7f1d3815a010 2024-02-02T13:25:22.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.237+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d380c4890 con 0x7f1d3815a010 2024-02-02T13:25:22.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.237+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1d20010c80 con 0x7f1d3815a010 2024-02-02T13:25:22.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.238+0000 7f1d35600640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 0x7f1d1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.238+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d200885b0 con 0x7f1d3815a010 2024-02-02T13:25:22.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.238+0000 7f1d3d600640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 0x7f1d1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.239+0000 7f1d3d600640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 0x7f1d1006c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d28036000 tx=0x7f1d2803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.243+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1d20053b00 con 0x7f1d3815a010 2024-02-02T13:25:22.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.429+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d38066cf0 con 0x7f1d3815a010 2024-02-02T13:25:22.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.430+0000 7f1d35600640 1 -- 172.21.15.91:0/1257294135 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d20086e60 con 0x7f1d3815a010 2024-02-02T13:25:22.432 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:22.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 msgr2=0x7f1d1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 0x7f1d1006c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d28036000 tx=0x7f1d2803d040 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 msgr2=0x7f1d380b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000b4a0 tx=0x7f1d2000b960 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 shutdown_connections 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1d1006a430 0x7f1d1006c8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380b2130 0x7f1d380c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d3815a010 0x7f1d380b1bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 --2- 172.21.15.91:0/1257294135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ce130 0x7f1d380b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.434+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 >> 172.21.15.91:0/1257294135 conn(0x7f1d380cec80 msgr2=0x7f1d38070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.435+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 shutdown_connections 2024-02-02T13:25:22.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.435+0000 7f1d40000640 1 -- 172.21.15.91:0/1257294135 wait complete. 2024-02-02T13:25:22.453 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:22.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.657+0000 7fd9bec00640 1 Processor -- start 2024-02-02T13:25:22.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.657+0000 7fd9bec00640 1 -- start start 2024-02-02T13:25:22.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.658+0000 7fd9bec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.658+0000 7fd9bec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9b80b2260 0x7fd9b80d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9bec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b80cd930 con 0x7fd9b80b6260 2024-02-02T13:25:22.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9bec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80b8f90 con 0x7fd9b80b2260 2024-02-02T13:25:22.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9bec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80d1680 con 0x7fd9b80d0070 2024-02-02T13:25:22.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:51472/0 (socket says 172.21.15.91:51472) 2024-02-02T13:25:22.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9b7e00640 1 -- 172.21.15.91:0/3958583763 learned_addr learned my addr 172.21.15.91:0/3958583763 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:22.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.659+0000 7fd9b7400640 1 --2- 172.21.15.91:0/3958583763 >> v2:172.21.15.113:3300/0 conn(0x7fd9b80b2260 0x7fd9b80d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.660+0000 7fd9b7e00640 1 -- 172.21.15.91:0/3958583763 >> v2:172.21.15.113:3300/0 conn(0x7fd9b80b2260 msgr2=0x7fd9b80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:22.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.660+0000 7fd9b7e00640 1 --2- 172.21.15.91:0/3958583763 >> v2:172.21.15.113:3300/0 conn(0x7fd9b80b2260 0x7fd9b80d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.660+0000 7fd9b7e00640 1 -- 172.21.15.91:0/3958583763 >> v1:172.21.15.113:6789/0 conn(0x7fd9b80b6260 legacy=0x7fd9b80d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.660+0000 7fd9b7e00640 1 -- 172.21.15.91:0/3958583763 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80d1960 con 0x7fd9b80d0070 2024-02-02T13:25:22.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.662+0000 7fd9b7e00640 1 --2- 172.21.15.91:0/3958583763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009e50 tx=0x7fd9a802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a592277e26779b72 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.662+0000 7fd9b6a00640 1 -- 172.21.15.91:0/3958583763 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a802cb00 con 0x7fd9b80d0070 2024-02-02T13:25:22.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.662+0000 7fd9b6a00640 1 -- 172.21.15.91:0/3958583763 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a802cc60 con 0x7fd9b80d0070 2024-02-02T13:25:22.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.662+0000 7fd9b6a00640 1 -- 172.21.15.91:0/3958583763 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80343f0 con 0x7fd9b80d0070 2024-02-02T13:25:22.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 -- 172.21.15.91:0/3958583763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 msgr2=0x7fd9b80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 --2- 172.21.15.91:0/3958583763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009e50 tx=0x7fd9a802bf60 comp rx=0 tx=0).stop 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 -- 172.21.15.91:0/3958583763 shutdown_connections 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 --2- 172.21.15.91:0/3958583763 >> v2:172.21.15.113:3300/0 conn(0x7fd9b80b2260 0x7fd9b80d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 --2- 172.21.15.91:0/3958583763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b80b8a50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 -- 172.21.15.91:0/3958583763 >> 172.21.15.91:0/3958583763 conn(0x7fd9b8070660 msgr2=0x7fd9b8070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 -- 172.21.15.91:0/3958583763 shutdown_connections 2024-02-02T13:25:22.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.663+0000 7fd9bec00640 1 -- 172.21.15.91:0/3958583763 wait complete. 2024-02-02T13:25:22.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.664+0000 7fd9bec00640 1 Processor -- start 2024-02-02T13:25:22.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 -- start start 2024-02-02T13:25:22.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b80b2260 0x7fd9b80cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80ca570 con 0x7fd9b80b6260 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80ccc00 con 0x7fd9b80b2260 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.665+0000 7fd9bec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80ccec0 con 0x7fd9b80d0070 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52578/0 (socket says 172.21.15.91:52578) 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 -- 172.21.15.91:0/2933427260 learned_addr learned my addr 172.21.15.91:0/2933427260 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9bcc00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7e00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b80b2260 0x7fd9b80cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 -- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 msgr2=0x7fd9b81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b81594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 -- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b80b2260 msgr2=0x7fd9b80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b80b2260 0x7fd9b80cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.666+0000 7fd9b7400640 1 -- 172.21.15.91:0/2933427260 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8009b30 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9b7400640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000e870 tx=0x7fd9a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00094f0 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a000c940 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000ccf0 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b80cd1a0 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.667+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b815d5e0 con 0x7fd9b80b6260 2024-02-02T13:25:22.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.668+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b8066cf0 con 0x7fd9b80b6260 2024-02-02T13:25:22.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.669+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd9a0013470 con 0x7fd9b80b6260 2024-02-02T13:25:22.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.670+0000 7fd9afe00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 0x7fd99006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:22.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.670+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9a0088a50 con 0x7fd9b80b6260 2024-02-02T13:25:22.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.670+0000 7fd9b7e00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 0x7fd99006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:22.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.671+0000 7fd9b7e00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 0x7fd99006c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8035000 tx=0x7fd9a8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.674+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd9a0054030 con 0x7fd9b80b6260 2024-02-02T13:25:22.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.864+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9b80744e0 con 0x7fd9b80b6260 2024-02-02T13:25:22.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.865+0000 7fd9afe00640 1 -- 172.21.15.91:0/2933427260 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9a0021020 con 0x7fd9b80b6260 2024-02-02T13:25:22.867 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:22.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.867+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 msgr2=0x7fd99006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 0x7fd99006c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8035000 tx=0x7fd9a8030000 comp rx=0 tx=0).stop 2024-02-02T13:25:22.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 msgr2=0x7fd9b80ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:22.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000e870 tx=0x7fd9a000ed30 comp rx=0 tx=0).stop 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 shutdown_connections 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd99006a430 0x7fd99006c8f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80d0070 0x7fd9b81594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9b80b6260 0x7fd9b80ca9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 --2- 172.21.15.91:0/2933427260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b80b2260 0x7fd9b80cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 >> 172.21.15.91:0/2933427260 conn(0x7fd9b8070660 msgr2=0x7fd9b80b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 shutdown_connections 2024-02-02T13:25:22.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:22.868+0000 7fd9bec00640 1 -- 172.21.15.91:0/2933427260 wait complete. 2024-02-02T13:25:22.887 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:23.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.097+0000 7f94db400640 1 Processor -- start 2024-02-02T13:25:23.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.097+0000 7f94db400640 1 -- start start 2024-02-02T13:25:23.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94db400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94d40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94db400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40cc7c0 con 0x7f94d40ce130 2024-02-02T13:25:23.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94db400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40ce930 con 0x7f94d40b0700 2024-02-02T13:25:23.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94db400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cd640 con 0x7f94d415a010 2024-02-02T13:25:23.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94cbc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f94d40ce130 0x7f94d40ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39314/0 (socket says 172.21.15.91:39314) 2024-02-02T13:25:23.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.099+0000 7f94cbc00640 1 -- 172.21.15.91:0/697429231 learned_addr learned my addr 172.21.15.91:0/697429231 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d8a00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94d40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791005262 0 0) 0x7f94d40cc7c0 con 0x7f94d40ce130 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b4003520 con 0x7f94d40ce130 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2955283017 0 0) 0x7f94b4003520 con 0x7f94d40ce130 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 msgr2=0x7f94d40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94d40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 >> v1:172.21.15.113:6789/0 conn(0x7f94d40b0700 legacy=0x7f94d406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d406c890 con 0x7f94d40ce130 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f94bc001670 con 0x7f94d40ce130 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 >> v1:172.21.15.91:6789/0 conn(0x7f94d40ce130 legacy=0x7f94d40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b40039a0 0x7f94b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b4004d00 0x7f94b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cd640 con 0x7f94b40039a0 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b400dc40 con 0x7f94b4004d00 2024-02-02T13:25:23.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b400de00 con 0x7f94d415a010 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.100+0000 7f94d8a00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b40039a0 0x7f94b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94cbc00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94d8a00640 1 -- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 msgr2=0x7f94b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94d8a00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94d8a00640 1 -- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b4004d00 msgr2=0x7f94b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94d8a00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b4004d00 0x7f94b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94d8a00640 1 -- 172.21.15.91:0/697429231 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4003520 con 0x7f94b40039a0 2024-02-02T13:25:23.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.101+0000 7f94cbc00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d415a010 0x7f94b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:23.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.102+0000 7f94d8a00640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b40039a0 0x7f94b4003dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009a90 tx=0x7f94b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0cebfea48117ae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.102+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402c680 con 0x7f94b40039a0 2024-02-02T13:25:23.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.102+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c402c7e0 con 0x7f94b40039a0 2024-02-02T13:25:23.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.102+0000 7f94d3e00640 1 -- 172.21.15.91:0/697429231 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402cb70 con 0x7f94b40039a0 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 -- 172.21.15.91:0/697429231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b40039a0 msgr2=0x7f94b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 --2- 172.21.15.91:0/697429231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b40039a0 0x7f94b4003dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009a90 tx=0x7f94b40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94d9400640 1 -- 172.21.15.91:0/697429231 reap_dead start 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 -- 172.21.15.91:0/697429231 shutdown_connections 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 -- 172.21.15.91:0/697429231 >> 172.21.15.91:0/697429231 conn(0x7f94d40cec80 msgr2=0x7f94d40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 -- 172.21.15.91:0/697429231 shutdown_connections 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.103+0000 7f94db400640 1 -- 172.21.15.91:0/697429231 wait complete. 2024-02-02T13:25:23.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.105+0000 7f94db400640 1 Processor -- start 2024-02-02T13:25:23.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.105+0000 7f94db400640 1 -- start start 2024-02-02T13:25:23.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 0x7f94d40b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 0x7f94d4155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415e330 con 0x7f94d415d940 2024-02-02T13:25:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40c2390 con 0x7f94b40039a0 2024-02-02T13:25:23.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94db400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4156410 con 0x7f94d4157000 2024-02-02T13:25:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94d9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94d9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52598/0 (socket says 172.21.15.91:52598) 2024-02-02T13:25:23.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.106+0000 7f94d9400640 1 -- 172.21.15.91:0/3143721557 learned_addr learned my addr 172.21.15.91:0/3143721557 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94cbc00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 0x7f94d4155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d8a00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 0x7f94d40b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 -- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 msgr2=0x7f94d4155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 0x7f94d4155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 -- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 msgr2=0x7f94d40b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 0x7f94d40b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 -- 172.21.15.91:0/3143721557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4157770 con 0x7f94d415d940 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94cbc00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 0x7f94d4155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d8a00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 0x7f94d40b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d9400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d415ec80 tx=0x7f94cc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003970 con 0x7f94d415d940 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc003b10 con 0x7f94d415d940 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004020 con 0x7f94d415d940 2024-02-02T13:25:23.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d40c25e0 con 0x7f94d415d940 2024-02-02T13:25:23.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.107+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d40c2ba0 con 0x7f94d415d940 2024-02-02T13:25:23.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.108+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4066cf0 con 0x7f94d415d940 2024-02-02T13:25:23.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.109+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f94cc004180 con 0x7f94d415d940 2024-02-02T13:25:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.110+0000 7f94d1600640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 0x7f94ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.110+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94cc087b30 con 0x7f94d415d940 2024-02-02T13:25:23.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.111+0000 7f94d8a00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 0x7f94ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.111+0000 7f94d8a00640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 0x7f94ac06c8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94c4036000 tx=0x7f94c4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.115+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f94cc053150 con 0x7f94d415d940 2024-02-02T13:25:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.302+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94d40b4c00 con 0x7f94d415d940 2024-02-02T13:25:23.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.303+0000 7f94d1600640 1 -- 172.21.15.91:0/3143721557 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94cc056770 con 0x7f94d415d940 2024-02-02T13:25:23.305 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 msgr2=0x7f94ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 0x7f94ac06c8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94c4036000 tx=0x7f94c4030000 comp rx=0 tx=0).stop 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 msgr2=0x7f94d415ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d415ec80 tx=0x7f94cc00c960 comp rx=0 tx=0).stop 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 shutdown_connections 2024-02-02T13:25:23.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94ac06a430 0x7f94ac06c8f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d4157000 0x7f94d4155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d415d940 0x7f94d415ddf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 --2- 172.21.15.91:0/3143721557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b40039a0 0x7f94d40b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 >> 172.21.15.91:0/3143721557 conn(0x7f94d40cec80 msgr2=0x7f94d4071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.306+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 shutdown_connections 2024-02-02T13:25:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.307+0000 7f94db400640 1 -- 172.21.15.91:0/3143721557 wait complete. 2024-02-02T13:25:23.327 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:25:23.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.535+0000 7fc0ce400640 1 Processor -- start 2024-02-02T13:25:23.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.535+0000 7fc0ce400640 1 -- start start 2024-02-02T13:25:23.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.536+0000 7fc0ce400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0c815ad10 0x7fc0c806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.536+0000 7fc0ce400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.536+0000 7fc0ce400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0c80cb580 0x7fc0c80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.536+0000 7fc0ce400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c80cbf50 con 0x7fc0c80cb580 2024-02-02T13:25:23.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0ce400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c80cd610 con 0x7fc0c815ad10 2024-02-02T13:25:23.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0ce400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c806d910 con 0x7fc0c815b0f0 2024-02-02T13:25:23.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0c80cb580 0x7fc0c80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0c80cb580 0x7fc0c80cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52602/0 (socket says 172.21.15.91:52602) 2024-02-02T13:25:23.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 -- 172.21.15.91:0/3810560571 learned_addr learned my addr 172.21.15.91:0/3810560571 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:23.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7400640 1 --2- 172.21.15.91:0/3810560571 >> v2:172.21.15.113:3300/0 conn(0x7fc0c815ad10 0x7fc0c806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c6a00640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 -- 172.21.15.91:0/3810560571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 msgr2=0x7fc0c806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 -- 172.21.15.91:0/3810560571 >> v2:172.21.15.113:3300/0 conn(0x7fc0c815ad10 msgr2=0x7fc0c806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 --2- 172.21.15.91:0/3810560571 >> v2:172.21.15.113:3300/0 conn(0x7fc0c815ad10 0x7fc0c806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.537+0000 7fc0c7e00640 1 -- 172.21.15.91:0/3810560571 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80cc850 con 0x7fc0c80cb580 2024-02-02T13:25:23.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0c7e00640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c80cba10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80cc6f0 tx=0x7fc0c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc24b5c46bc293 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0c6000640 1 -- 172.21.15.91:0/3810560571 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c002ca70 con 0x7fc0c80cb580 2024-02-02T13:25:23.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0c6000640 1 -- 172.21.15.91:0/3810560571 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c002cbd0 con 0x7fc0c80cb580 2024-02-02T13:25:23.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0c6000640 1 -- 172.21.15.91:0/3810560571 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00343f0 con 0x7fc0c80cb580 2024-02-02T13:25:23.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0ce400640 1 -- 172.21.15.91:0/3810560571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 msgr2=0x7fc0c80cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.539+0000 7fc0ce400640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c80cba10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80cc6f0 tx=0x7fc0c002bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 -- 172.21.15.91:0/3810560571 shutdown_connections 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c80cba10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 --2- 172.21.15.91:0/3810560571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 --2- 172.21.15.91:0/3810560571 >> v2:172.21.15.113:3300/0 conn(0x7fc0c815ad10 0x7fc0c806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 -- 172.21.15.91:0/3810560571 >> 172.21.15.91:0/3810560571 conn(0x7fc0c80b0700 msgr2=0x7fc0c80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 -- 172.21.15.91:0/3810560571 shutdown_connections 2024-02-02T13:25:23.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.540+0000 7fc0ce400640 1 -- 172.21.15.91:0/3810560571 wait complete. 2024-02-02T13:25:23.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.541+0000 7fc0ce400640 1 Processor -- start 2024-02-02T13:25:23.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.542+0000 7fc0ce400640 1 -- start start 2024-02-02T13:25:23.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.542+0000 7fc0ce400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0ce400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c815ad10 0x7fc0c8155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0ce400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c80c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0ce400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c6cf0 con 0x7fc0c80cb580 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0ce400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c6fd0 con 0x7fc0c815ad10 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0ce400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c72b0 con 0x7fc0c815b0f0 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52618/0 (socket says 172.21.15.91:52618) 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 -- 172.21.15.91:0/4035405920 learned_addr learned my addr 172.21.15.91:0/4035405920 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7e00640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c80c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 -- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 msgr2=0x7fc0c80c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c80c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 -- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c815ad10 msgr2=0x7fc0c8155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c815ad10 0x7fc0c8155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7400640 1 -- 172.21.15.91:0/4035405920 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009b30 con 0x7fc0c80cb580 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.543+0000 7fc0c7e00640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c80c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:23.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0c7400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800d800 tx=0x7fc0b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800b6c0 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b800b820 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800bbb0 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c80c7590 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.544+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c8155940 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.545+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c8066cf0 con 0x7fc0c80cb580 2024-02-02T13:25:23.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.546+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0b800bd10 con 0x7fc0c80cb580 2024-02-02T13:25:23.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.547+0000 7fc0bf400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 0x7fc0a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:23.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.547+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0b8088e70 con 0x7fc0c80cb580 2024-02-02T13:25:23.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.547+0000 7fc0c6a00640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 0x7fc0a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:23.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.548+0000 7fc0c6a00640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 0x7fc0a006c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80c4680 tx=0x7fc0b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:23.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.551+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0b8054450 con 0x7fc0c80cb580 2024-02-02T13:25:23.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.746+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc0c80c6e30 con 0x7fc0c80cb580 2024-02-02T13:25:23.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.747+0000 7fc0bf400640 1 -- 172.21.15.91:0/4035405920 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc0c80c6e30 con 0x7fc0c80cb580 2024-02-02T13:25:23.748 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:25:23.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 msgr2=0x7fc0a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 0x7fc0a006c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80c4680 tx=0x7fc0b0008040 comp rx=0 tx=0).stop 2024-02-02T13:25:23.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 msgr2=0x7fc0c8159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:23.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800d800 tx=0x7fc0b800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 shutdown_connections 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0a006a430 0x7fc0a006c8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c815b0f0 0x7fc0c80c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c815ad10 0x7fc0c8155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 --2- 172.21.15.91:0/4035405920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c80cb580 0x7fc0c8159710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.749+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 >> 172.21.15.91:0/4035405920 conn(0x7fc0c80b0700 msgr2=0x7fc0c80c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.750+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 shutdown_connections 2024-02-02T13:25:23.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:23.750+0000 7fc0ce400640 1 -- 172.21.15.91:0/4035405920 wait complete. 2024-02-02T13:25:23.769 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:25:23.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:23.769 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-02-02T13:25:23.862 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:25:23.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:23.862 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:25:23.917 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:23.917 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:23.973 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:23.974 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:24.026 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:24.026 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:24.081 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:24.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.296+0000 7f0c80a00640 1 Processor -- start 2024-02-02T13:25:24.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.297+0000 7f0c80a00640 1 -- start start 2024-02-02T13:25:24.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c80a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7815a010 0x7f0c780c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c80a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c780ce130 0x7f0c780ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c80a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780cc7c0 con 0x7f0c780cd470 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c80a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780ce930 con 0x7f0c780ce130 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c80a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7806c890 con 0x7f0c7815a010 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.298+0000 7f0c7d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c780ce130 0x7f0c780ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c780ce130 0x7f0c780ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52630/0 (socket says 172.21.15.91:52630) 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 -- 172.21.15.91:0/2915764614 learned_addr learned my addr 172.21.15.91:0/2915764614 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 -- 172.21.15.91:0/2915764614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7815a010 msgr2=0x7f0c780c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 --2- 172.21.15.91:0/2915764614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7815a010 0x7f0c780c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 -- 172.21.15.91:0/2915764614 >> v1:172.21.15.113:6789/0 conn(0x7f0c780cd470 legacy=0x7f0c780b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.299+0000 7f0c7d600640 1 -- 172.21.15.91:0/2915764614 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7806cb70 con 0x7f0c780ce130 2024-02-02T13:25:24.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.300+0000 7f0c7d600640 1 --2- 172.21.15.91:0/2915764614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780ce130 0x7f0c780ccf30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1975f08c362fc9e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.300+0000 7f0c7cc00640 1 -- 172.21.15.91:0/2915764614 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60031640 con 0x7f0c780ce130 2024-02-02T13:25:24.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.300+0000 7f0c7cc00640 1 -- 172.21.15.91:0/2915764614 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c600317a0 con 0x7f0c780ce130 2024-02-02T13:25:24.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.300+0000 7f0c7cc00640 1 -- 172.21.15.91:0/2915764614 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60031b30 con 0x7f0c780ce130 2024-02-02T13:25:24.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 -- 172.21.15.91:0/2915764614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780ce130 msgr2=0x7f0c780ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2915764614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780ce130 0x7f0c780ccf30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 -- 172.21.15.91:0/2915764614 shutdown_connections 2024-02-02T13:25:24.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2915764614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780ce130 0x7f0c780ccf30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2915764614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7815a010 0x7f0c780c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 -- 172.21.15.91:0/2915764614 >> 172.21.15.91:0/2915764614 conn(0x7f0c780cec80 msgr2=0x7f0c780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:24.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 -- 172.21.15.91:0/2915764614 shutdown_connections 2024-02-02T13:25:24.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.301+0000 7f0c80a00640 1 -- 172.21.15.91:0/2915764614 wait complete. 2024-02-02T13:25:24.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.304+0000 7f0c80a00640 1 Processor -- start 2024-02-02T13:25:24.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.304+0000 7f0c80a00640 1 -- start start 2024-02-02T13:25:24.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c780ce130 0x7f0c780b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 0x7f0c78070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781f2b30 con 0x7f0c780c8990 2024-02-02T13:25:24.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780c6750 con 0x7f0c7815a010 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c80a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780c6a30 con 0x7f0c780ce130 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c7ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c7ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52632/0 (socket says 172.21.15.91:52632) 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.305+0000 7f0c7ea00640 1 -- 172.21.15.91:0/2828470988 learned_addr learned my addr 172.21.15.91:0/2828470988 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7d600640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 0x7f0c78070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7e000640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c780ce130 0x7f0c780b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 -- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c780ce130 msgr2=0x7f0c780b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c780ce130 0x7f0c780b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 -- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 msgr2=0x7f0c78070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 0x7f0c78070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 -- 172.21.15.91:0/2828470988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600094d0 con 0x7f0c780c8990 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7d600640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 0x7f0c78070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:24.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c7ea00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c780 tx=0x7f0c7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70003b70 con 0x7f0c780c8990 2024-02-02T13:25:24.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70003cf0 con 0x7f0c780c8990 2024-02-02T13:25:24.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c780c8470 con 0x7f0c780c8990 2024-02-02T13:25:24.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700052b0 con 0x7f0c780c8990 2024-02-02T13:25:24.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.306+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c780c6d10 con 0x7f0c780c8990 2024-02-02T13:25:24.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.308+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c780c82d0 con 0x7f0c780c8990 2024-02-02T13:25:24.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.309+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c70005410 con 0x7f0c780c8990 2024-02-02T13:25:24.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.309+0000 7f0c76000640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 0x7f0c5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.310+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c70087ff0 con 0x7f0c780c8990 2024-02-02T13:25:24.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.310+0000 7f0c7e000640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 0x7f0c5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.310+0000 7f0c7e000640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 0x7f0c5006c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800a770 tx=0x7f0c68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.314+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c780c82d0 con 0x7f0c780c8990 2024-02-02T13:25:24.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.509+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c780c82d0 con 0x7f0c780c8990 2024-02-02T13:25:24.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.511+0000 7f0c76000640 1 -- 172.21.15.91:0/2828470988 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f0c780c82d0 con 0x7f0c780c8990 2024-02-02T13:25:24.513 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:24.513 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":21,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:24.470503+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:24.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.513+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 msgr2=0x7f0c5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.513+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 0x7f0c5006c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800a770 tx=0x7f0c68008040 comp rx=0 tx=0).stop 2024-02-02T13:25:24.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.513+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 msgr2=0x7f0c780711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.513+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c780 tx=0x7f0c7000cc40 comp rx=0 tx=0).stop 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 shutdown_connections 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c5006a430 0x7f0c5006c8f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c780c8990 0x7f0c780711e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7815a010 0x7f0c78070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 --2- 172.21.15.91:0/2828470988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c780ce130 0x7f0c780b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 >> 172.21.15.91:0/2828470988 conn(0x7f0c780cec80 msgr2=0x7f0c781591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 shutdown_connections 2024-02-02T13:25:24.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.514+0000 7f0c80a00640 1 -- 172.21.15.91:0/2828470988 wait complete. 2024-02-02T13:25:24.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:24.534 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:24.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.729+0000 7f9170c00640 1 Processor -- start 2024-02-02T13:25:24.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.730+0000 7f9170c00640 1 -- start start 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9170c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 0x7f916c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9170c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f916c0b0700 0x7f916c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9170c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0cc7c0 con 0x7f916c15a010 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9170c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0b10d0 con 0x7f916c0b0700 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9170c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ce930 con 0x7f916c0ce130 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9169e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f916c15a010 0x7f916c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39318/0 (socket says 172.21.15.91:39318) 2024-02-02T13:25:24.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.731+0000 7f9169e00640 1 -- 172.21.15.91:0/1112207323 learned_addr learned my addr 172.21.15.91:0/1112207323 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:24.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910771667 0 0) 0x7f916c0cc7c0 con 0x7f916c15a010 2024-02-02T13:25:24.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f914c003520 con 0x7f916c15a010 2024-02-02T13:25:24.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f916a800640 1 --2- 172.21.15.91:0/1112207323 >> v2:172.21.15.113:3300/0 conn(0x7f916c0b0700 0x7f916c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9161200640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 0x7f916c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242200381 0 0) 0x7f914c003520 con 0x7f916c15a010 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 msgr2=0x7f916c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 0x7f916c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 >> v2:172.21.15.113:3300/0 conn(0x7f916c0b0700 msgr2=0x7f916c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 --2- 172.21.15.91:0/1112207323 >> v2:172.21.15.113:3300/0 conn(0x7f916c0b0700 0x7f916c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0cd640 con 0x7f916c15a010 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f915c001670 con 0x7f916c15a010 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f916a800640 1 --2- 172.21.15.91:0/1112207323 >> v2:172.21.15.113:3300/0 conn(0x7f916c0b0700 0x7f916c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 >> v1:172.21.15.91:6789/0 conn(0x7f916c15a010 legacy=0x7f916c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 0x7f914c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f914c003fa0 0x7f914c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c004d00 0x7f914c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b10d0 con 0x7f914c003fa0 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.732+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ce930 con 0x7f914c004d00 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914c00dc90 con 0x7f916c0ce130 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f914c003fa0 0x7f914c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f916a800640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c004d00 0x7f914c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 -- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 msgr2=0x7f914c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c0ce130 0x7f914c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 -- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c004d00 msgr2=0x7f914c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c004d00 0x7f914c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.733+0000 7f9161200640 1 -- 172.21.15.91:0/1112207323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c003520 con 0x7f914c003fa0 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.734+0000 7f9161200640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f914c003fa0 0x7f914c0047c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f915400ba30 tx=0x7f914c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fed6cae3966d7e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154030660 con 0x7f914c003fa0 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91540307c0 con 0x7f914c003fa0 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9169400640 1 -- 172.21.15.91:0/1112207323 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154030b50 con 0x7f914c003fa0 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 -- 172.21.15.91:0/1112207323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f914c003fa0 msgr2=0x7f914c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 --2- 172.21.15.91:0/1112207323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f914c003fa0 0x7f914c0047c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f915400ba30 tx=0x7f914c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f916a800640 1 -- 172.21.15.91:0/1112207323 reap_dead start 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 -- 172.21.15.91:0/1112207323 shutdown_connections 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 -- 172.21.15.91:0/1112207323 >> 172.21.15.91:0/1112207323 conn(0x7f916c0cec80 msgr2=0x7f916c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 -- 172.21.15.91:0/1112207323 shutdown_connections 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.735+0000 7f9170c00640 1 -- 172.21.15.91:0/1112207323 wait complete. 2024-02-02T13:25:24.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.737+0000 7f9170c00640 1 Processor -- start 2024-02-02T13:25:24.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.737+0000 7f9170c00640 1 -- start start 2024-02-02T13:25:24.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 0x7f916c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c15b070 0x7f916c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b4380 con 0x7f916c0c81e0 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c8bb0 con 0x7f914c003fa0 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.738+0000 7f9170c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c15ba10 con 0x7f916c15b070 2024-02-02T13:25:24.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52650/0 (socket says 172.21.15.91:52650) 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 -- 172.21.15.91:0/2430693258 learned_addr learned my addr 172.21.15.91:0/2430693258 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f9169e00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 0x7f916c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 -- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c15b070 msgr2=0x7f916c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c15b070 0x7f916c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 -- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 msgr2=0x7f916c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 0x7f916c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 -- 172.21.15.91:0/2430693258 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0b34b0 con 0x7f916c0c81e0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f9169e00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 0x7f916c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.739+0000 7f916a800640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f916c15ec80 tx=0x7f9164004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.740+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164007cf0 con 0x7f916c0c81e0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.740+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164007e50 con 0x7f916c0c81e0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.740+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c1515f0 con 0x7f916c0c81e0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.740+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164014640 con 0x7f916c0c81e0 2024-02-02T13:25:24.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.740+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0b39e0 con 0x7f916c0c81e0 2024-02-02T13:25:24.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.741+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0b44c0 con 0x7f916c0c81e0 2024-02-02T13:25:24.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.742+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f916401d3b0 con 0x7f916c0c81e0 2024-02-02T13:25:24.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.743+0000 7f9162a00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 0x7f914406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:24.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.743+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9164088660 con 0x7f916c0c81e0 2024-02-02T13:25:24.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.743+0000 7f9169e00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 0x7f914406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:24.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.743+0000 7f9169e00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 0x7f914406c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f915c007a90 tx=0x7f915c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:24.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.747+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f916c0b44c0 con 0x7f916c0c81e0 2024-02-02T13:25:24.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.937+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f916c0b44c0 con 0x7f916c0c81e0 2024-02-02T13:25:24.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.938+0000 7f9162a00640 1 -- 172.21.15.91:0/2430693258 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f916c0b44c0 con 0x7f916c0c81e0 2024-02-02T13:25:24.940 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:24.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 msgr2=0x7f914406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 0x7f914406c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f915c007a90 tx=0x7f915c009040 comp rx=0 tx=0).stop 2024-02-02T13:25:24.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 msgr2=0x7f916c0c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:24.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f916c15ec80 tx=0x7f9164004700 comp rx=0 tx=0).stop 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 shutdown_connections 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914406a430 0x7f914406c8f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f916c15b070 0x7f916c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f916c0c81e0 0x7f916c0c8670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 --2- 172.21.15.91:0/2430693258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914c003fa0 0x7f916c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 >> 172.21.15.91:0/2430693258 conn(0x7f916c0cec80 msgr2=0x7f916c0acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 shutdown_connections 2024-02-02T13:25:24.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:24.941+0000 7f9170c00640 1 -- 172.21.15.91:0/2430693258 wait complete. 2024-02-02T13:25:24.961 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:25.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.175+0000 7fcb87e00640 1 Processor -- start 2024-02-02T13:25:25.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.176+0000 7fcb87e00640 1 -- start start 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.177+0000 7fcb87e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb800b2260 0x7fcb800b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.177+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.177+0000 7fcb87e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800cd930 con 0x7fcb800d0070 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.177+0000 7fcb87e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800b8f90 con 0x7fcb800b2260 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.177+0000 7fcb87e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800d1680 con 0x7fcb800b6260 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb800b2260 0x7fcb800b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb800b2260 0x7fcb800b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52654/0 (socket says 172.21.15.91:52654) 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 -- 172.21.15.91:0/377860106 learned_addr learned my addr 172.21.15.91:0/377860106 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb85e00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 -- 172.21.15.91:0/377860106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 msgr2=0x7fcb800b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 -- 172.21.15.91:0/377860106 >> v1:172.21.15.113:6789/0 conn(0x7fcb800d0070 legacy=0x7fcb800d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb84a00640 1 -- 172.21.15.91:0/377860106 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800d1960 con 0x7fcb800b2260 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.178+0000 7fcb85e00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.179+0000 7fcb84a00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800b8a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009870 tx=0x7fcb6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216a884fac8cbb0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.180+0000 7fcb7fe00640 1 -- 172.21.15.91:0/377860106 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68031640 con 0x7fcb800b2260 2024-02-02T13:25:25.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.180+0000 7fcb7fe00640 1 -- 172.21.15.91:0/377860106 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680317a0 con 0x7fcb800b2260 2024-02-02T13:25:25.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.180+0000 7fcb7fe00640 1 -- 172.21.15.91:0/377860106 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68031b30 con 0x7fcb800b2260 2024-02-02T13:25:25.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 -- 172.21.15.91:0/377860106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 msgr2=0x7fcb800b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800b8a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009870 tx=0x7fcb6802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 -- 172.21.15.91:0/377860106 shutdown_connections 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 --2- 172.21.15.91:0/377860106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800b8a50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 -- 172.21.15.91:0/377860106 >> 172.21.15.91:0/377860106 conn(0x7fcb80070660 msgr2=0x7fcb80070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 -- 172.21.15.91:0/377860106 shutdown_connections 2024-02-02T13:25:25.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.181+0000 7fcb87e00640 1 -- 172.21.15.91:0/377860106 wait complete. 2024-02-02T13:25:25.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.182+0000 7fcb87e00640 1 Processor -- start 2024-02-02T13:25:25.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.183+0000 7fcb87e00640 1 -- start start 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb800d0070 0x7fcb800ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800abcf0 con 0x7fcb800b2260 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800a9a70 con 0x7fcb800d0070 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb87e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800abf80 con 0x7fcb800b6260 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52670/0 (socket says 172.21.15.91:52670) 2024-02-02T13:25:25.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 -- 172.21.15.91:0/2874045043 learned_addr learned my addr 172.21.15.91:0/2874045043 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 -- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 msgr2=0x7fcb800ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 -- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb800d0070 msgr2=0x7fcb800ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb800d0070 0x7fcb800ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb800d0070 0x7fcb800ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.184+0000 7fcb85400640 1 -- 172.21.15.91:0/2874045043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680094d0 con 0x7fcb800b2260 2024-02-02T13:25:25.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb85400640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000b3f0 tx=0x7fcb7000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70011830 con 0x7fcb800b2260 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70011990 con 0x7fcb800b2260 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70011d40 con 0x7fcb800b2260 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb800a9d10 con 0x7fcb800b2260 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.185+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb800aa2d0 con 0x7fcb800b2260 2024-02-02T13:25:25.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.186+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb80066cf0 con 0x7fcb800b2260 2024-02-02T13:25:25.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.187+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcb7001a400 con 0x7fcb800b2260 2024-02-02T13:25:25.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.188+0000 7fcb7d600640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 0x7fcb5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.188+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcb70088820 con 0x7fcb800b2260 2024-02-02T13:25:25.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.188+0000 7fcb84a00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 0x7fcb5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.189+0000 7fcb84a00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 0x7fcb5806c8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036000 tx=0x7fcb68030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.193+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcb70053d90 con 0x7fcb800b2260 2024-02-02T13:25:25.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.381+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb8006a6f0 con 0x7fcb800b2260 2024-02-02T13:25:25.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.382+0000 7fcb7d600640 1 -- 172.21.15.91:0/2874045043 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb70015020 con 0x7fcb800b2260 2024-02-02T13:25:25.384 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 msgr2=0x7fcb5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 0x7fcb5806c8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036000 tx=0x7fcb68030000 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 msgr2=0x7fcb800cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000b3f0 tx=0x7fcb7000b8b0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 shutdown_connections 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb5806a430 0x7fcb5806c8f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb800b6260 0x7fcb800ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb800d0070 0x7fcb800ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 --2- 172.21.15.91:0/2874045043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb800b2260 0x7fcb800cc480 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 >> 172.21.15.91:0/2874045043 conn(0x7fcb80070660 msgr2=0x7fcb80073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:25.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 shutdown_connections 2024-02-02T13:25:25.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.385+0000 7fcb87e00640 1 -- 172.21.15.91:0/2874045043 wait complete. 2024-02-02T13:25:25.406 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:25.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.613+0000 7f54b9a00640 1 Processor -- start 2024-02-02T13:25:25.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.613+0000 7f54b9a00640 1 -- start start 2024-02-02T13:25:25.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54b406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b40b0700 0x7f54b40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b9a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b40b10a0 con 0x7f54b4165200 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b9a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b40ccdd0 con 0x7f54b40b0700 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40cc7c0 con 0x7f54b4161660 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b3400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f54b4165200 0x7f54b4165650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39334/0 (socket says 172.21.15.91:39334) 2024-02-02T13:25:25.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.615+0000 7f54b3400640 1 -- 172.21.15.91:0/1071896284 learned_addr learned my addr 172.21.15.91:0/1071896284 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:25.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b2a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54b406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b2000640 1 --2- 172.21.15.91:0/1071896284 >> v2:172.21.15.113:3300/0 conn(0x7f54b40b0700 0x7f54b40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562209870 0 0) 0x7f54b40b10a0 con 0x7f54b4165200 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5494003520 con 0x7f54b4165200 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513481334 0 0) 0x7f5494003520 con 0x7f54b4165200 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 msgr2=0x7f54b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54b406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 >> v2:172.21.15.113:3300/0 conn(0x7f54b40b0700 msgr2=0x7f54b40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 --2- 172.21.15.91:0/1071896284 >> v2:172.21.15.113:3300/0 conn(0x7f54b40b0700 0x7f54b40b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b406c540 con 0x7f54b4165200 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f54ac001670 con 0x7f54b4165200 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 >> v1:172.21.15.91:6789/0 conn(0x7f54b4165200 legacy=0x7f54b4165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f5494003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5494004d00 0x7f5494005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40ccdd0 con 0x7f5494003970 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40cc7c0 con 0x7f5494004d00 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549400bb10 con 0x7f54b4161660 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.616+0000 7f54b2a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f5494003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2000640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2a00640 1 -- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 msgr2=0x7f54940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b3400640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5494004d00 0x7f5494005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b4161660 0x7f54940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2a00640 1 -- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5494004d00 msgr2=0x7f5494005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5494004d00 0x7f5494005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b2a00640 1 -- 172.21.15.91:0/1071896284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494003520 con 0x7f5494003970 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.617+0000 7f54b3400640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5494004d00 0x7f5494005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.618+0000 7f54b2a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f5494003d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f549c00ba30 tx=0x7f54940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef748385d701ad6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.618+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c032640 con 0x7f5494003970 2024-02-02T13:25:25.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.618+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c0327a0 con 0x7f5494003970 2024-02-02T13:25:25.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.618+0000 7f54b1600640 1 -- 172.21.15.91:0/1071896284 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c032b30 con 0x7f5494003970 2024-02-02T13:25:25.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 -- 172.21.15.91:0/1071896284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 msgr2=0x7f5494003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 --2- 172.21.15.91:0/1071896284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f5494003d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f549c00ba30 tx=0x7f54940057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b3400640 1 -- 172.21.15.91:0/1071896284 reap_dead start 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 -- 172.21.15.91:0/1071896284 shutdown_connections 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 -- 172.21.15.91:0/1071896284 >> 172.21.15.91:0/1071896284 conn(0x7f54b40cec80 msgr2=0x7f54b40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 -- 172.21.15.91:0/1071896284 shutdown_connections 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.619+0000 7f54b9a00640 1 -- 172.21.15.91:0/1071896284 wait complete. 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.621+0000 7f54b9a00640 1 Processor -- start 2024-02-02T13:25:25.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.621+0000 7f54b9a00640 1 -- start start 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b40b3ac0 0x7f54b40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b40b2880 0x7f54b40b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4153930 con 0x7f5494003970 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b3270 con 0x7f54b40b2880 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4152db0 con 0x7f54b40b3ac0 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52690/0 (socket says 172.21.15.91:52690) 2024-02-02T13:25:25.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 -- 172.21.15.91:0/3264751103 learned_addr learned my addr 172.21.15.91:0/3264751103 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 -- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b40b3ac0 msgr2=0x7f54b40c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b40b3ac0 0x7f54b40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b3400640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b40b2880 0x7f54b40b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 -- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b40b2880 msgr2=0x7f54b40b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b40b2880 0x7f54b40b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.622+0000 7f54b2a00640 1 -- 172.21.15.91:0/3264751103 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4153090 con 0x7f5494003970 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54b2a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f549c00bb50 tx=0x7f549c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c03ca80 con 0x7f5494003970 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c03cbe0 con 0x7f5494003970 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c030410 con 0x7f5494003970 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b4153370 con 0x7f5494003970 2024-02-02T13:25:25.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.623+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4152010 con 0x7f5494003970 2024-02-02T13:25:25.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.624+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4066cf0 con 0x7f5494003970 2024-02-02T13:25:25.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.625+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f549c030570 con 0x7f5494003970 2024-02-02T13:25:25.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.626+0000 7f54aaa00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 0x7f548c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:25.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.626+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f549c0ab330 con 0x7f5494003970 2024-02-02T13:25:25.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.626+0000 7f54b2000640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 0x7f548c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:25.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.627+0000 7f54b2000640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 0x7f548c06c920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54b4164d70 tx=0x7f54a4003a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:25.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.630+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f549c076800 con 0x7f5494003970 2024-02-02T13:25:25.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.817+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54b40c26f0 con 0x7f5494003970 2024-02-02T13:25:25.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.818+0000 7f54aaa00640 1 -- 172.21.15.91:0/3264751103 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f549c079e20 con 0x7f5494003970 2024-02-02T13:25:25.820 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 msgr2=0x7f548c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 0x7f548c06c920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54b4164d70 tx=0x7f54a4003a10 comp rx=0 tx=0).stop 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 msgr2=0x7f54b4151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f549c00bb50 tx=0x7f549c004730 comp rx=0 tx=0).stop 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 shutdown_connections 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f548c06a460 0x7f548c06c920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b40b3ac0 0x7f54b40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b40b2880 0x7f54b40b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 --2- 172.21.15.91:0/3264751103 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5494003970 0x7f54b4151ad0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:25.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 >> 172.21.15.91:0/3264751103 conn(0x7f54b40cec80 msgr2=0x7f54b40b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:25.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 shutdown_connections 2024-02-02T13:25:25.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:25.821+0000 7f54b9a00640 1 -- 172.21.15.91:0/3264751103 wait complete. 2024-02-02T13:25:25.844 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:25:26.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.054+0000 7fe40ee00640 1 Processor -- start 2024-02-02T13:25:26.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.054+0000 7fe40ee00640 1 -- start start 2024-02-02T13:25:26.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40ee00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe40815dfb0 0x7fe4080c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40ee00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40806faa0 con 0x7fe4080ce4d0 2024-02-02T13:25:26.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4080ca420 con 0x7fe4080c9a80 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40ee00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40815e800 con 0x7fe40815dfb0 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40da00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe4080ce4d0 0x7fe4080707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39348/0 (socket says 172.21.15.91:39348) 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40da00640 1 -- 172.21.15.91:0/1141060377 learned_addr learned my addr 172.21.15.91:0/1141060377 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe40e400640 1 --2- 172.21.15.91:0/1141060377 >> v2:172.21.15.91:3300/0 conn(0x7fe40815dfb0 0x7fe4080c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe407e00640 1 -- 172.21.15.91:0/1141060377 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539849291 0 0) 0x7fe40806faa0 con 0x7fe4080ce4d0 2024-02-02T13:25:26.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.056+0000 7fe407e00640 1 -- 172.21.15.91:0/1141060377 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e8003520 con 0x7fe4080ce4d0 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.057+0000 7fe40e400640 1 -- 172.21.15.91:0/1141060377 >> v1:172.21.15.113:6789/0 conn(0x7fe4080c9a80 legacy=0x7fe40815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.057+0000 7fe40e400640 1 -- 172.21.15.91:0/1141060377 >> v1:172.21.15.91:6789/0 conn(0x7fe4080ce4d0 legacy=0x7fe4080707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.057+0000 7fe40e400640 1 -- 172.21.15.91:0/1141060377 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408070320 con 0x7fe40815dfb0 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe40e400640 1 --2- 172.21.15.91:0/1141060377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c9ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe400009d90 tx=0x7fe40002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1992c0927cf6fdbb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe407e00640 1 -- 172.21.15.91:0/1141060377 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40002cd00 con 0x7fe40815dfb0 2024-02-02T13:25:26.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe407e00640 1 -- 172.21.15.91:0/1141060377 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40002ce60 con 0x7fe40815dfb0 2024-02-02T13:25:26.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe407e00640 1 -- 172.21.15.91:0/1141060377 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400034660 con 0x7fe40815dfb0 2024-02-02T13:25:26.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe40ee00640 1 -- 172.21.15.91:0/1141060377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 msgr2=0x7fe4080c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.058+0000 7fe40ee00640 1 --2- 172.21.15.91:0/1141060377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c9ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe400009d90 tx=0x7fe40002bf50 comp rx=0 tx=0).stop 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.059+0000 7fe40ee00640 1 -- 172.21.15.91:0/1141060377 shutdown_connections 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.059+0000 7fe40ee00640 1 --2- 172.21.15.91:0/1141060377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c9ee0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.059+0000 7fe40ee00640 1 -- 172.21.15.91:0/1141060377 >> 172.21.15.91:0/1141060377 conn(0x7fe4080cb4a0 msgr2=0x7fe4080cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.059+0000 7fe40ee00640 1 -- 172.21.15.91:0/1141060377 shutdown_connections 2024-02-02T13:25:26.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.059+0000 7fe40ee00640 1 -- 172.21.15.91:0/1141060377 wait complete. 2024-02-02T13:25:26.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.060+0000 7fe40ee00640 1 Processor -- start 2024-02-02T13:25:26.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.061+0000 7fe40ee00640 1 -- start start 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 0x7fe408157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 0x7fe4080aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe408070ec0 con 0x7fe40815dfb0 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080c2a40 con 0x7fe4080ce4d0 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080aeba0 con 0x7fe4080c9a80 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52714/0 (socket says 172.21.15.91:52714) 2024-02-02T13:25:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 -- 172.21.15.91:0/3389104776 learned_addr learned my addr 172.21.15.91:0/3389104776 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40d000640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 0x7fe4080aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40da00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 0x7fe408157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 -- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 msgr2=0x7fe408157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 0x7fe408157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 -- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 msgr2=0x7fe4080aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 0x7fe4080aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40e400640 1 -- 172.21.15.91:0/3389104776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400009a70 con 0x7fe40815dfb0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40d000640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 0x7fe4080aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.062+0000 7fe40da00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 0x7fe408157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe40e400640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe400009d90 tx=0x7fe400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40003d600 con 0x7fe40815dfb0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40003d780 con 0x7fe40815dfb0 2024-02-02T13:25:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40003db30 con 0x7fe40815dfb0 2024-02-02T13:25:26.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe408157b60 con 0x7fe40815dfb0 2024-02-02T13:25:26.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.063+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080ae8c0 con 0x7fe40815dfb0 2024-02-02T13:25:26.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.065+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4080c2b80 con 0x7fe40815dfb0 2024-02-02T13:25:26.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.065+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe40003b040 con 0x7fe40815dfb0 2024-02-02T13:25:26.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.066+0000 7fe405600640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 0x7fe3e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.066+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe4000aae60 con 0x7fe40815dfb0 2024-02-02T13:25:26.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.066+0000 7fe40da00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 0x7fe3e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.067+0000 7fe40da00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 0x7fe3e006c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005880 tx=0x7fe3f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.071+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe4080c2b80 con 0x7fe40815dfb0 2024-02-02T13:25:26.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.276+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe4080c2b80 con 0x7fe40815dfb0 2024-02-02T13:25:26.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.276+0000 7fe405600640 1 -- 172.21.15.91:0/3389104776 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe4080c2b80 con 0x7fe40815dfb0 2024-02-02T13:25:26.278 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 msgr2=0x7fe3e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 0x7fe3e006c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005880 tx=0x7fe3f8009040 comp rx=0 tx=0).stop 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 msgr2=0x7fe4080c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe400009d90 tx=0x7fe400004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 shutdown_connections 2024-02-02T13:25:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe3e006a460 0x7fe3e006c920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe40815dfb0 0x7fe4080c7aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4080ce4d0 0x7fe4080aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 --2- 172.21.15.91:0/3389104776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4080c9a80 0x7fe408157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 >> 172.21.15.91:0/3389104776 conn(0x7fe4080cb4a0 msgr2=0x7fe4080c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 shutdown_connections 2024-02-02T13:25:26.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.279+0000 7fe40ee00640 1 -- 172.21.15.91:0/3389104776 wait complete. 2024-02-02T13:25:26.300 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:25:26.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:26.300 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-02-02T13:25:26.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:25:26.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:26.405 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:25:26.467 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:26.467 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:26.523 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:26.523 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:26.571 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:26.572 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:26.667 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:26.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.874+0000 7f0ef7200640 1 Processor -- start 2024-02-02T13:25:26.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.874+0000 7f0ef7200640 1 -- start start 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.875+0000 7f0ef7200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ef815b2a0 0x7f0ef80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.875+0000 7f0ef7200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef806c1c0 con 0x7f0ef806f3f0 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.875+0000 7f0ef7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef806b5f0 con 0x7f0ef815aec0 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.875+0000 7f0ef7200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef806cd60 con 0x7f0ef815b2a0 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.876+0000 7f0ef6800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0ef806f3f0 0x7f0ef80ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39358/0 (socket says 172.21.15.91:39358) 2024-02-02T13:25:26.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.876+0000 7f0ef6800640 1 -- 172.21.15.91:0/2980401593 learned_addr learned my addr 172.21.15.91:0/2980401593 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:26.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.876+0000 7f0eed200640 1 --2- 172.21.15.91:0/2980401593 >> v2:172.21.15.113:3300/0 conn(0x7f0ef815b2a0 0x7f0ef80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.876+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841157002 0 0) 0x7f0ef806c1c0 con 0x7f0ef806f3f0 2024-02-02T13:25:26.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.876+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed8003520 con 0x7f0ef806f3f0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771575129 0 0) 0x7f0ed8003520 con 0x7f0ef806f3f0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 >> v2:172.21.15.113:3300/0 conn(0x7f0ef815b2a0 msgr2=0x7f0ef80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 --2- 172.21.15.91:0/2980401593 >> v2:172.21.15.113:3300/0 conn(0x7f0ef815b2a0 0x7f0ef80cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 >> v1:172.21.15.113:6789/0 conn(0x7f0ef815aec0 legacy=0x7f0ef806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef806b890 con 0x7f0ef806f3f0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0ef00019f0 con 0x7f0ef806f3f0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 >> v1:172.21.15.91:6789/0 conn(0x7f0ef806f3f0 legacy=0x7f0ef80ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed8003970 0x7f0ed8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 0x7f0ed8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8005180 0x7f0ed800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef806cd60 con 0x7f0ed80047c0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800c030 con 0x7f0ed8005180 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800c1f0 con 0x7f0ed8003970 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 0x7f0ed8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 -- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed8003970 msgr2=0x7f0ed8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:26.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed8003970 0x7f0ed8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 -- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8005180 msgr2=0x7f0ed800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:26.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8005180 0x7f0ed800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.877+0000 7f0eed200640 1 -- 172.21.15.91:0/2980401593 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8003520 con 0x7f0ed80047c0 2024-02-02T13:25:26.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.878+0000 7f0ef6800640 1 -- 172.21.15.91:0/2980401593 reap_dead start 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0eed200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 0x7f0ed8004c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009ac0 tx=0x7f0ed80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d41606629417c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee002c710 con 0x7f0ed80047c0 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee002c870 con 0x7f0ed80047c0 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0ef5400640 1 -- 172.21.15.91:0/2980401593 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee002cc00 con 0x7f0ed80047c0 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0ef7200640 1 -- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 msgr2=0x7f0ed8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.879+0000 7f0ef7200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 0x7f0ed8004c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009ac0 tx=0x7f0ed80057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.880+0000 7f0ef7200640 1 -- 172.21.15.91:0/2980401593 shutdown_connections 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.880+0000 7f0ef7200640 1 --2- 172.21.15.91:0/2980401593 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ed80047c0 0x7f0ed8004c40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.880+0000 7f0ef7200640 1 -- 172.21.15.91:0/2980401593 >> 172.21.15.91:0/2980401593 conn(0x7f0ef80cea90 msgr2=0x7f0ef80ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.880+0000 7f0ef7200640 1 -- 172.21.15.91:0/2980401593 shutdown_connections 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.880+0000 7f0ef7200640 1 -- 172.21.15.91:0/2980401593 wait complete. 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.881+0000 7f0ef7200640 1 Processor -- start 2024-02-02T13:25:26.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.882+0000 7f0ef7200640 1 -- start start 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.882+0000 7f0ef7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 0x7f0ef80b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 0x7f0ef8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef7200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef7200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80b3820 con 0x7f0ef81e46b0 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80b3b00 con 0x7f0ef81e3e00 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80b3de0 con 0x7f0ed80047c0 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52732/0 (socket says 172.21.15.91:52732) 2024-02-02T13:25:26.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 -- 172.21.15.91:0/3040242767 learned_addr learned my addr 172.21.15.91:0/3040242767 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0eed200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 0x7f0ef8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef5e00640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 0x7f0ef80b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 -- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 msgr2=0x7f0ef80b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 0x7f0ef80b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 -- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 msgr2=0x7f0ef8154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 0x7f0ef8154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef6800640 1 -- 172.21.15.91:0/3040242767 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80aef70 con 0x7f0ef81e46b0 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0eed200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 0x7f0ef8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.883+0000 7f0ef5e00640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 0x7f0ef80b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:26.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0ef6800640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001560 tx=0x7f0ef000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0004d60 con 0x7f0ef81e46b0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef000a610 con 0x7f0ef81e46b0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef80af250 con 0x7f0ef81e46b0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000a9c0 con 0x7f0ef81e46b0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.884+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef81553d0 con 0x7f0ef81e46b0 2024-02-02T13:25:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.885+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef804dff0 con 0x7f0ef81e46b0 2024-02-02T13:25:26.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.886+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0ef000ab20 con 0x7f0ef81e46b0 2024-02-02T13:25:26.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.887+0000 7f0eeea00640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 0x7f0ed006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:26.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.887+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0ef00885f0 con 0x7f0ef81e46b0 2024-02-02T13:25:26.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.887+0000 7f0ef5e00640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 0x7f0ed006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:26.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.888+0000 7f0ef5e00640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 0x7f0ed006c8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ef804f7d0 tx=0x7f0ee8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:26.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:26.891+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0ef0053b50 con 0x7f0ef81e46b0 2024-02-02T13:25:27.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.086+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ef81e4490 con 0x7f0ef81e46b0 2024-02-02T13:25:27.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.089+0000 7f0eeea00640 1 -- 172.21.15.91:0/3040242767 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f0ef0057170 con 0x7f0ef81e46b0 2024-02-02T13:25:27.091 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:27.091 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":22,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:25.481699+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 msgr2=0x7f0ed006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 0x7f0ed006c8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ef804f7d0 tx=0x7f0ee8009040 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 msgr2=0x7f0ef81e4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001560 tx=0x7f0ef000cb90 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 shutdown_connections 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ed006a430 0x7f0ed006c8f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ef81e46b0 0x7f0ef81e4b60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ef81e3e00 0x7f0ef8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 --2- 172.21.15.91:0/3040242767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed80047c0 0x7f0ef80b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 >> 172.21.15.91:0/3040242767 conn(0x7f0ef80cea90 msgr2=0x7f0ef81e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 shutdown_connections 2024-02-02T13:25:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.092+0000 7f0ef7200640 1 -- 172.21.15.91:0/3040242767 wait complete. 2024-02-02T13:25:27.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:27.113 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:27.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.320+0000 7fc370800640 1 Processor -- start 2024-02-02T13:25:27.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.320+0000 7fc370800640 1 -- start start 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.321+0000 7fc370800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.321+0000 7fc370800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc3680b0c40 0x7fc3680b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.321+0000 7fc370800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36815fa00 con 0x7fc368151f90 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.321+0000 7fc370800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3680b10d0 con 0x7fc3680b0c40 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.321+0000 7fc370800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc368070f80 con 0x7fc36815f5a0 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc368151f90 0x7fc36806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39366/0 (socket says 172.21.15.91:39366) 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36de00640 1 -- 172.21.15.91:0/3980369086 learned_addr learned my addr 172.21.15.91:0/3980369086 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36e800640 1 --2- 172.21.15.91:0/3980369086 >> v2:172.21.15.91:3300/0 conn(0x7fc3680b0c40 0x7fc3680b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36d400640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36ca00640 1 -- 172.21.15.91:0/3980369086 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580660259 0 0) 0x7fc358001110 con 0x7fc368151f90 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36ca00640 1 -- 172.21.15.91:0/3980369086 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc348003520 con 0x7fc368151f90 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36e800640 1 -- 172.21.15.91:0/3980369086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 msgr2=0x7fc3680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36e800640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36e800640 1 -- 172.21.15.91:0/3980369086 >> v1:172.21.15.91:6789/0 conn(0x7fc368151f90 legacy=0x7fc36806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.322+0000 7fc36e800640 1 -- 172.21.15.91:0/3980369086 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368152370 con 0x7fc3680b0c40 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.323+0000 7fc36e800640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3680b0c40 0x7fc3680b3af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc360009d90 tx=0x7fc36002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5a20ea9ce8e356 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc36ca00640 1 -- 172.21.15.91:0/3980369086 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36002cc50 con 0x7fc3680b0c40 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc36ca00640 1 -- 172.21.15.91:0/3980369086 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36002cdb0 con 0x7fc3680b0c40 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc36ca00640 1 -- 172.21.15.91:0/3980369086 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360034660 con 0x7fc3680b0c40 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 -- 172.21.15.91:0/3980369086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3680b0c40 msgr2=0x7fc3680b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3680b0c40 0x7fc3680b3af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc360009d90 tx=0x7fc36002bea0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 -- 172.21.15.91:0/3980369086 shutdown_connections 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3680b0c40 0x7fc3680b3af0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 --2- 172.21.15.91:0/3980369086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 -- 172.21.15.91:0/3980369086 >> 172.21.15.91:0/3980369086 conn(0x7fc3680cec80 msgr2=0x7fc3680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:27.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 -- 172.21.15.91:0/3980369086 shutdown_connections 2024-02-02T13:25:27.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.324+0000 7fc370800640 1 -- 172.21.15.91:0/3980369086 wait complete. 2024-02-02T13:25:27.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.326+0000 7fc370800640 1 Processor -- start 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.326+0000 7fc370800640 1 -- start start 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 0x7fc368150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368150cc0 con 0x7fc368151f90 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680c82b0 con 0x7fc3680b0c40 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc370800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680c5cc0 con 0x7fc36815f5a0 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc36d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc36d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52754/0 (socket says 172.21.15.91:52754) 2024-02-02T13:25:27.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.327+0000 7fc36d400640 1 -- 172.21.15.91:0/843837960 learned_addr learned my addr 172.21.15.91:0/843837960 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36e800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36de00640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 0x7fc368150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 -- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 msgr2=0x7fc3680afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 -- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 msgr2=0x7fc368150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 0x7fc368150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 -- 172.21.15.91:0/843837960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360009a70 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36de00640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 0x7fc368150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc36d400640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d800 tx=0x7fc35000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.328+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000b6c0 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.329+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35000b840 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.329+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000bbf0 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.329+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3680c5fa0 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.329+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3680c6560 con 0x7fc368151f90 2024-02-02T13:25:27.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.330+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368066cf0 con 0x7fc368151f90 2024-02-02T13:25:27.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.331+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc35000bd50 con 0x7fc368151f90 2024-02-02T13:25:27.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.332+0000 7fc366000640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 0x7fc34006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.332+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc350088e90 con 0x7fc368151f90 2024-02-02T13:25:27.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.332+0000 7fc36de00640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 0x7fc34006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.332+0000 7fc36de00640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 0x7fc34006c8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc358001370 tx=0x7fc358009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.336+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc350054490 con 0x7fc368151f90 2024-02-02T13:25:27.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.521+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc36806c7b0 con 0x7fc368151f90 2024-02-02T13:25:27.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.522+0000 7fc366000640 1 -- 172.21.15.91:0/843837960 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc350057ab0 con 0x7fc368151f90 2024-02-02T13:25:27.524 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 msgr2=0x7fc34006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 0x7fc34006c8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc358001370 tx=0x7fc358009040 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 msgr2=0x7fc3680af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d800 tx=0x7fc35000dcc0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 shutdown_connections 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc34006a430 0x7fc34006c8f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc36815f5a0 0x7fc3680afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc368151f90 0x7fc3680af860 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 --2- 172.21.15.91:0/843837960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3680b0c40 0x7fc368150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.524+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 >> 172.21.15.91:0/843837960 conn(0x7fc3680cec80 msgr2=0x7fc3680adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:27.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.525+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 shutdown_connections 2024-02-02T13:25:27.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.525+0000 7fc370800640 1 -- 172.21.15.91:0/843837960 wait complete. 2024-02-02T13:25:27.544 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:27.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.746+0000 7f9339000640 1 Processor -- start 2024-02-02T13:25:27.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.746+0000 7f9339000640 1 -- start start 2024-02-02T13:25:27.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.747+0000 7f9339000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.747+0000 7f9339000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 0x7f93340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9339000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9334070f80 con 0x7f93340b0c40 2024-02-02T13:25:27.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9339000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340b10d0 con 0x7f933406f4f0 2024-02-02T13:25:27.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9339000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9334152370 con 0x7f9334151f90 2024-02-02T13:25:27.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9331e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f932be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 0x7f93340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9331e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:51632/0 (socket says 172.21.15.91:51632) 2024-02-02T13:25:27.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.748+0000 7f9331e00640 1 -- 172.21.15.91:0/1397899084 learned_addr learned my addr 172.21.15.91:0/1397899084 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:27.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.749+0000 7f9331e00640 1 -- 172.21.15.91:0/1397899084 >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 msgr2=0x7f93340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:27.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.749+0000 7f9331e00640 1 --2- 172.21.15.91:0/1397899084 >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 0x7f93340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.749+0000 7f9331e00640 1 -- 172.21.15.91:0/1397899084 >> v1:172.21.15.113:6789/0 conn(0x7f93340b0c40 legacy=0x7f93340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.749+0000 7f9331e00640 1 -- 172.21.15.91:0/1397899084 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340cb010 con 0x7f9334151f90 2024-02-02T13:25:27.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.749+0000 7f932be00640 1 --2- 172.21.15.91:0/1397899084 >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 0x7f93340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:25:27.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.751+0000 7f9331e00640 1 --2- 172.21.15.91:0/1397899084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9324009870 tx=0x7f932402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4839fd868da7260 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.751+0000 7f9331400640 1 -- 172.21.15.91:0/1397899084 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324030840 con 0x7f9334151f90 2024-02-02T13:25:27.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.751+0000 7f9331400640 1 -- 172.21.15.91:0/1397899084 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240309a0 con 0x7f9334151f90 2024-02-02T13:25:27.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.751+0000 7f9331400640 1 -- 172.21.15.91:0/1397899084 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324030d30 con 0x7f9334151f90 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 -- 172.21.15.91:0/1397899084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 msgr2=0x7f933415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 --2- 172.21.15.91:0/1397899084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9324009870 tx=0x7f932402b980 comp rx=0 tx=0).stop 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 -- 172.21.15.91:0/1397899084 shutdown_connections 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 --2- 172.21.15.91:0/1397899084 >> v2:172.21.15.113:3300/0 conn(0x7f933406f4f0 0x7f93340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 --2- 172.21.15.91:0/1397899084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f933415f5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 -- 172.21.15.91:0/1397899084 >> 172.21.15.91:0/1397899084 conn(0x7f93340cec80 msgr2=0x7f93340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 -- 172.21.15.91:0/1397899084 shutdown_connections 2024-02-02T13:25:27.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.752+0000 7f9339000640 1 -- 172.21.15.91:0/1397899084 wait complete. 2024-02-02T13:25:27.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.754+0000 7f9339000640 1 Processor -- start 2024-02-02T13:25:27.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.754+0000 7f9339000640 1 -- start start 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 0x7f93341f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f93341f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c67e0 con 0x7f933406f4f0 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c5e50 con 0x7f93340b0c40 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9339000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b1750 con 0x7f9334151f90 2024-02-02T13:25:27.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9331e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9331e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52758/0 (socket says 172.21.15.91:52758) 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.755+0000 7f9331e00640 1 -- 172.21.15.91:0/3066353223 learned_addr learned my addr 172.21.15.91:0/3066353223 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f932be00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 0x7f93341f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9332800640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f93341f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 -- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 msgr2=0x7f93341f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f93341f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 -- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 msgr2=0x7f93341f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 0x7f93341f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 -- 172.21.15.91:0/3066353223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009400 con 0x7f933406f4f0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f932be00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 0x7f93341f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f9331e00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93240090b0 tx=0x7f9324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.756+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932403e040 con 0x7f933406f4f0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.757+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324039cb0 con 0x7f933406f4f0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.757+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324043680 con 0x7f933406f4f0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.757+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93240094d0 con 0x7f933406f4f0 2024-02-02T13:25:27.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.757+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340c60a0 con 0x7f933406f4f0 2024-02-02T13:25:27.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.758+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9334066cf0 con 0x7f933406f4f0 2024-02-02T13:25:27.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.758+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f93240423e0 con 0x7f933406f4f0 2024-02-02T13:25:27.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.759+0000 7f932a000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:27.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.759+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f93240aadc0 con 0x7f933406f4f0 2024-02-02T13:25:27.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.760+0000 7f932be00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:27.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.760+0000 7f932be00640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93340ca8d0 tx=0x7f931c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:27.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.764+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9324076320 con 0x7f933406f4f0 2024-02-02T13:25:27.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.949+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93340c5550 con 0x7f933406f4f0 2024-02-02T13:25:27.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.950+0000 7f932a000640 1 -- 172.21.15.91:0/3066353223 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93240340a0 con 0x7f933406f4f0 2024-02-02T13:25:27.952 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:27.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.954+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 msgr2=0x7f930c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.954+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93340ca8d0 tx=0x7f931c008040 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 msgr2=0x7f93340c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93240090b0 tx=0x7f9324004730 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 shutdown_connections 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9334151f90 0x7f93341f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93340b0c40 0x7f93341f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 --2- 172.21.15.91:0/3066353223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933406f4f0 0x7f93340c8060 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 >> 172.21.15.91:0/3066353223 conn(0x7f93340cec80 msgr2=0x7f93341598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 shutdown_connections 2024-02-02T13:25:27.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:27.955+0000 7f9339000640 1 -- 172.21.15.91:0/3066353223 wait complete. 2024-02-02T13:25:27.974 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:28.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.180+0000 7fe61a600640 1 Processor -- start 2024-02-02T13:25:28.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.180+0000 7fe61a600640 1 -- start start 2024-02-02T13:25:28.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe61a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe61406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe61a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6140ac8d0 0x7fe6140acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe61a600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61415d870 con 0x7fe6140b0700 2024-02-02T13:25:28.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe61a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6140ad270 con 0x7fe6140ac8d0 2024-02-02T13:25:28.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe61a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61406bce0 con 0x7fe61415d490 2024-02-02T13:25:28.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe613e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe6140b0700 0x7fe6140c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39374/0 (socket says 172.21.15.91:39374) 2024-02-02T13:25:28.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe613e00640 1 -- 172.21.15.91:0/4272505816 learned_addr learned my addr 172.21.15.91:0/4272505816 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe60be00640 1 --2- 172.21.15.91:0/4272505816 >> v2:172.21.15.113:3300/0 conn(0x7fe6140ac8d0 0x7fe6140acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.182+0000 7fe613400640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe61406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864062080 0 0) 0x7fe61415d870 con 0x7fe6140b0700 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f4003520 con 0x7fe6140b0700 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909773343 0 0) 0x7fe5f4003520 con 0x7fe6140b0700 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 msgr2=0x7fe61406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe61406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 >> v2:172.21.15.113:3300/0 conn(0x7fe6140ac8d0 msgr2=0x7fe6140acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 --2- 172.21.15.91:0/4272505816 >> v2:172.21.15.113:3300/0 conn(0x7fe6140ac8d0 0x7fe6140acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614161220 con 0x7fe6140b0700 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe60c0019f0 con 0x7fe6140b0700 2024-02-02T13:25:28.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 >> v1:172.21.15.91:6789/0 conn(0x7fe6140b0700 legacy=0x7fe6140c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5f4003970 0x7fe5f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f40047c0 0x7fe5f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe5f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140ad270 con 0x7fe5f4003970 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61406bce0 con 0x7fe5f40047c0 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f400b980 con 0x7fe61415d490 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.183+0000 7fe613400640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5f4003970 0x7fe5f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613e00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe5f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe60be00640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f40047c0 0x7fe5f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613400640 1 -- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 msgr2=0x7fe5f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613400640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61415d490 0x7fe5f4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613400640 1 -- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f40047c0 msgr2=0x7fe5f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613400640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f40047c0 0x7fe5f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.184+0000 7fe613400640 1 -- 172.21.15.91:0/4272505816 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4003520 con 0x7fe5f4003970 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.185+0000 7fe613400640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5f4003970 0x7fe5f4003d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ac0 tx=0x7fe5f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4d947675690245 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60402c6b0 con 0x7fe5f4003970 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60402c810 con 0x7fe5f4003970 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe612a00640 1 -- 172.21.15.91:0/4272505816 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60402cba0 con 0x7fe5f4003970 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 -- 172.21.15.91:0/4272505816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5f4003970 msgr2=0x7fe5f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 --2- 172.21.15.91:0/4272505816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5f4003970 0x7fe5f4003d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ac0 tx=0x7fe5f40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe613e00640 1 -- 172.21.15.91:0/4272505816 reap_dead start 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 -- 172.21.15.91:0/4272505816 shutdown_connections 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 -- 172.21.15.91:0/4272505816 >> 172.21.15.91:0/4272505816 conn(0x7fe6140cec80 msgr2=0x7fe6140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 -- 172.21.15.91:0/4272505816 shutdown_connections 2024-02-02T13:25:28.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.186+0000 7fe61a600640 1 -- 172.21.15.91:0/4272505816 wait complete. 2024-02-02T13:25:28.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.188+0000 7fe61a600640 1 Processor -- start 2024-02-02T13:25:28.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.188+0000 7fe61a600640 1 -- start start 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6141f23e0 0x7fe6140b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61406f970 0x7fe61406fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61415f440 con 0x7fe61415f860 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140b4690 con 0x7fe6141f23e0 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe61a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140b43b0 con 0x7fe61406f970 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe60be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe60be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52778/0 (socket says 172.21.15.91:52778) 2024-02-02T13:25:28.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.189+0000 7fe60be00640 1 -- 172.21.15.91:0/3508968464 learned_addr learned my addr 172.21.15.91:0/3508968464 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 -- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61406f970 msgr2=0x7fe61406fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe613e00640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61406f970 0x7fe61406fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe613400640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6141f23e0 0x7fe6140b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61406f970 0x7fe61406fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 -- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6141f23e0 msgr2=0x7fe6140b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6141f23e0 0x7fe6140b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 -- 172.21.15.91:0/3508968464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61415ed80 con 0x7fe61415f860 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60be00640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0116d0 tx=0x7fe5fc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00f9c0 con 0x7fe61415f860 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc00fb20 con 0x7fe61415f860 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00fe90 con 0x7fe61415f860 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.190+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61415f060 con 0x7fe61415f860 2024-02-02T13:25:28.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.191+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140b29d0 con 0x7fe61415f860 2024-02-02T13:25:28.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.192+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61406be50 con 0x7fe61415f860 2024-02-02T13:25:28.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.192+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5fc015460 con 0x7fe61415f860 2024-02-02T13:25:28.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.194+0000 7fe60b400640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 0x7fe5ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.194+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe5fc088d40 con 0x7fe61415f860 2024-02-02T13:25:28.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.194+0000 7fe613400640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 0x7fe5ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.195+0000 7fe613400640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 0x7fe5ec06c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe604036000 tx=0x7fe604030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.198+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe5fc054320 con 0x7fe61415f860 2024-02-02T13:25:28.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.392+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe614066cf0 con 0x7fe61415f860 2024-02-02T13:25:28.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.393+0000 7fe60b400640 1 -- 172.21.15.91:0/3508968464 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5fc057940 con 0x7fe61415f860 2024-02-02T13:25:28.394 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:28.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.395+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 msgr2=0x7fe5ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.395+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 0x7fe5ec06c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe604036000 tx=0x7fe604030000 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.395+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 msgr2=0x7fe61415fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.395+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0116d0 tx=0x7fe5fc011b90 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 shutdown_connections 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5ec06a460 0x7fe5ec06c920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe61406f970 0x7fe61406fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe61415f860 0x7fe61415fce0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 --2- 172.21.15.91:0/3508968464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6141f23e0 0x7fe6140b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 >> 172.21.15.91:0/3508968464 conn(0x7fe6140cec80 msgr2=0x7fe614157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 shutdown_connections 2024-02-02T13:25:28.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.396+0000 7fe61a600640 1 -- 172.21.15.91:0/3508968464 wait complete. 2024-02-02T13:25:28.417 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:25:28.514 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:25:28.522 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:25:28.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.630+0000 7f311b200640 1 Processor -- start 2024-02-02T13:25:28.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.630+0000 7f311b200640 1 -- start start 2024-02-02T13:25:28.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.631+0000 7f311b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f311406ae70 0x7f31140cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.631+0000 7f311b200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140d0480 con 0x7f311415d480 2024-02-02T13:25:28.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f311b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140cfec0 con 0x7f31140cde90 2024-02-02T13:25:28.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f311b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31140d01a0 con 0x7f311406ae70 2024-02-02T13:25:28.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f3113e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f311415d480 0x7f31140cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39388/0 (socket says 172.21.15.91:39388) 2024-02-02T13:25:28.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f3113e00640 1 -- 172.21.15.91:0/2699946109 learned_addr learned my addr 172.21.15.91:0/2699946109 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:28.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074720573 0 0) 0x7f31140d0480 con 0x7f311415d480 2024-02-02T13:25:28.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f4003520 con 0x7f311415d480 2024-02-02T13:25:28.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.632+0000 7f310be00640 1 --2- 172.21.15.91:0/2699946109 >> v2:172.21.15.113:3300/0 conn(0x7f311406ae70 0x7f31140cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 349954004 0 0) 0x7f30f4003520 con 0x7f311415d480 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 >> v2:172.21.15.113:3300/0 conn(0x7f311406ae70 msgr2=0x7f31140cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 --2- 172.21.15.91:0/2699946109 >> v2:172.21.15.113:3300/0 conn(0x7f311406ae70 0x7f31140cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 >> v1:172.21.15.113:6789/0 conn(0x7f31140cde90 legacy=0x7f31140cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140cb580 con 0x7f311415d480 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3104001670 con 0x7f311415d480 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 >> v1:172.21.15.91:6789/0 conn(0x7f311415d480 legacy=0x7f31140cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f4003970 0x7f30f4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f40047c0 0x7f30f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 0x7f30f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140d01a0 con 0x7f30f4005180 2024-02-02T13:25:28.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f400c0b0 con 0x7f30f4003970 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f400c270 con 0x7f30f40047c0 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3119200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 0x7f30f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3113e00640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f4003970 0x7f30f4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f310be00640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f40047c0 0x7f30f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3119200640 1 -- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f40047c0 msgr2=0x7f30f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3119200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f40047c0 0x7f30f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3119200640 1 -- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f4003970 msgr2=0x7f30f4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.633+0000 7f3119200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f4003970 0x7f30f4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.634+0000 7f3119200640 1 -- 172.21.15.91:0/2699946109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f4003520 con 0x7f30f4005180 2024-02-02T13:25:28.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.634+0000 7f310be00640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f40047c0 0x7f30f4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:28.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.635+0000 7f3119200640 1 -- 172.21.15.91:0/2699946109 reap_dead start 2024-02-02T13:25:28.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.635+0000 7f3119200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 0x7f30f400b980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f310c004480 tx=0x7f30f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aa26f1291f9fe5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.635+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c006b50 con 0x7f30f4005180 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.635+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c006cd0 con 0x7f30f4005180 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.635+0000 7f3113400640 1 -- 172.21.15.91:0/2699946109 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c007260 con 0x7f30f4005180 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 -- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 msgr2=0x7f30f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 0x7f30f400b980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f310c004480 tx=0x7f30f40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 -- 172.21.15.91:0/2699946109 shutdown_connections 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 --2- 172.21.15.91:0/2699946109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f4005180 0x7f30f400b980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 -- 172.21.15.91:0/2699946109 >> 172.21.15.91:0/2699946109 conn(0x7f311409a160 msgr2=0x7f31140d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 -- 172.21.15.91:0/2699946109 shutdown_connections 2024-02-02T13:25:28.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.636+0000 7f311b200640 1 -- 172.21.15.91:0/2699946109 wait complete. 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.638+0000 7f311b200640 1 Processor -- start 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.638+0000 7f311b200640 1 -- start start 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 0x7f3114157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f311406e0c0 0x7f31140c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140c5570 con 0x7f31141583e0 2024-02-02T13:25:28.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3114158260 con 0x7f311406e0c0 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f311b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311406e550 con 0x7f30f4005180 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52798/0 (socket says 172.21.15.91:52798) 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 -- 172.21.15.91:0/2767060076 learned_addr learned my addr 172.21.15.91:0/2767060076 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f3113e00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 0x7f3114157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 -- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 msgr2=0x7f3114157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 0x7f3114157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 -- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f311406e0c0 msgr2=0x7f31140c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f3119200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f311406e0c0 0x7f31140c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.639+0000 7f310be00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f311406e0c0 0x7f31140c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f310be00640 1 -- 172.21.15.91:0/2767060076 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140b66e0 con 0x7f31141583e0 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f3113e00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 0x7f3114157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:28.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f310be00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e7e0 tx=0x7f30fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00c6d0 con 0x7f31141583e0 2024-02-02T13:25:28.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc00c830 con 0x7f31141583e0 2024-02-02T13:25:28.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00cba0 con 0x7f31141583e0 2024-02-02T13:25:28.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31140c48f0 con 0x7f31141583e0 2024-02-02T13:25:28.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.640+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140c4eb0 con 0x7f31141583e0 2024-02-02T13:25:28.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.642+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3114066cf0 con 0x7f31141583e0 2024-02-02T13:25:28.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.642+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f30fc00cd00 con 0x7f31141583e0 2024-02-02T13:25:28.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.643+0000 7f3110c00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 0x7f30ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:28.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.643+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f30fc0886a0 con 0x7f31141583e0 2024-02-02T13:25:28.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.643+0000 7f3113e00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 0x7f30ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:28.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.644+0000 7f3113e00640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 0x7f30ec06c8f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3104001050 tx=0x7f3104000fa0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:28.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.647+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30fc053ca0 con 0x7f31141583e0 2024-02-02T13:25:28.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.849+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f31140b2490 con 0x7f31141583e0 2024-02-02T13:25:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.850+0000 7f3110c00640 1 -- 172.21.15.91:0/2767060076 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f30fc0572c0 con 0x7f31141583e0 2024-02-02T13:25:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:25:28.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.852+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 msgr2=0x7f30ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.852+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 0x7f30ec06c8f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3104001050 tx=0x7f3104000fa0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.852+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 msgr2=0x7f311406db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.852+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e7e0 tx=0x7f30fc00eca0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 shutdown_connections 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30ec06a430 0x7f30ec06c8f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f311406e0c0 0x7f31140c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31141583e0 0x7f311406db80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 --2- 172.21.15.91:0/2767060076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f4005180 0x7f3114157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 >> 172.21.15.91:0/2767060076 conn(0x7f311409a160 msgr2=0x7f3114069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 shutdown_connections 2024-02-02T13:25:28.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:28.853+0000 7f311b200640 1 -- 172.21.15.91:0/2767060076 wait complete. 2024-02-02T13:25:28.873 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-02-02T13:25:28.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:28.873 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-02-02T13:25:29.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:25:29.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:29.018 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-02-02T13:25:29.067 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:29.067 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:29.122 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:29.122 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:29.171 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:29.171 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:29.220 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:29.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.431+0000 7f80d2600640 1 Processor -- start 2024-02-02T13:25:29.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.432+0000 7f80d2600640 1 -- start start 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80d2600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80cc15f5a0 0x7f80cc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80d2600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0b1070 con 0x7f80cc151f90 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80d2600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc06c600 con 0x7f80cc0b0c40 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80d2600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc15fa30 con 0x7f80cc15f5a0 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80cb400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f80cc151f90 0x7f80cc06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39402/0 (socket says 172.21.15.91:39402) 2024-02-02T13:25:29.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.433+0000 7f80cb400640 1 -- 172.21.15.91:0/1014076193 learned_addr learned my addr 172.21.15.91:0/1014076193 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:29.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80c2800640 1 --2- 172.21.15.91:0/1014076193 >> v2:172.21.15.113:3300/0 conn(0x7f80cc15f5a0 0x7f80cc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195361321 0 0) 0x7f80cc0b1070 con 0x7f80cc151f90 2024-02-02T13:25:29.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80ac003520 con 0x7f80cc151f90 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3731156324 0 0) 0x7f80ac003520 con 0x7f80cc151f90 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 >> v2:172.21.15.113:3300/0 conn(0x7f80cc15f5a0 msgr2=0x7f80cc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 --2- 172.21.15.91:0/1014076193 >> v2:172.21.15.113:3300/0 conn(0x7f80cc15f5a0 0x7f80cc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 >> v1:172.21.15.113:6789/0 conn(0x7f80cc0b0c40 legacy=0x7f80cc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc070f80 con 0x7f80cc151f90 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f80bc001670 con 0x7f80cc151f90 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 >> v1:172.21.15.91:6789/0 conn(0x7f80cc151f90 legacy=0x7f80cc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80ac0047c0 0x7f80ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac0051d0 0x7f80ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15fa30 con 0x7f80ac0039c0 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac00c100 con 0x7f80ac0051d0 2024-02-02T13:25:29.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.434+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac00c2c0 con 0x7f80ac0047c0 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 -- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80ac0047c0 msgr2=0x7f80ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80ac0047c0 0x7f80ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 -- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac0051d0 msgr2=0x7f80ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cbe00640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac0051d0 0x7f80ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac0051d0 0x7f80ac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cb400640 1 -- 172.21.15.91:0/1014076193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac003520 con 0x7f80ac0039c0 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.435+0000 7f80cbe00640 1 -- 172.21.15.91:0/1014076193 reap_dead start 2024-02-02T13:25:29.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.436+0000 7f80cb400640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80ac003dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00c160 tx=0x7f80ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc65610df5fb9a32 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.436+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc005ad0 con 0x7f80ac0039c0 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.436+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc005c30 con 0x7f80ac0039c0 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.436+0000 7f80caa00640 1 -- 172.21.15.91:0/1014076193 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc02c7e0 con 0x7f80ac0039c0 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 -- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 msgr2=0x7f80ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80ac003dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00c160 tx=0x7f80ac0057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 -- 172.21.15.91:0/1014076193 shutdown_connections 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 --2- 172.21.15.91:0/1014076193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80ac003dc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 -- 172.21.15.91:0/1014076193 >> 172.21.15.91:0/1014076193 conn(0x7f80cc0cec80 msgr2=0x7f80cc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 -- 172.21.15.91:0/1014076193 shutdown_connections 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.437+0000 7f80d2600640 1 -- 172.21.15.91:0/1014076193 wait complete. 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.439+0000 7f80d2600640 1 Processor -- start 2024-02-02T13:25:29.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.439+0000 7f80d2600640 1 -- start start 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80cc0c8620 0x7f80cc15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80cc15a950 0x7f80cc0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c7d00 con 0x7f80ac0039c0 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15ae00 con 0x7f80cc0c8620 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80d2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15c270 con 0x7f80cc15a950 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52816/0 (socket says 172.21.15.91:52816) 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.440+0000 7f80cb400640 1 -- 172.21.15.91:0/2117471446 learned_addr learned my addr 172.21.15.91:0/2117471446 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80c2800640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80cc0c8620 0x7f80cc15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 -- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80cc15a950 msgr2=0x7f80cc0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cbe00640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80cc15a950 0x7f80cc0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80cc15a950 0x7f80cc0c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 -- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80cc0c8620 msgr2=0x7f80cc15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80cc0c8620 0x7f80cc15a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 -- 172.21.15.91:0/2117471446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0c8aa0 con 0x7f80ac0039c0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80cb400640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80bc02c380 tx=0x7f80bc001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc03c070 con 0x7f80ac0039c0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc005a40 con 0x7f80ac0039c0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc007070 con 0x7f80ac0039c0 2024-02-02T13:25:29.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.441+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc0c70f0 con 0x7f80ac0039c0 2024-02-02T13:25:29.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.442+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc0c76b0 con 0x7f80ac0039c0 2024-02-02T13:25:29.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.443+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc066cf0 con 0x7f80ac0039c0 2024-02-02T13:25:29.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.444+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f80bc032070 con 0x7f80ac0039c0 2024-02-02T13:25:29.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.445+0000 7f80c3e00640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 0x7f80a406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.445+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f80bc0a9570 con 0x7f80ac0039c0 2024-02-02T13:25:29.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.445+0000 7f80c2800640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 0x7f80a406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.446+0000 7f80c2800640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 0x7f80a406c920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006a90 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.450+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f80bc074c10 con 0x7f80ac0039c0 2024-02-02T13:25:29.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.642+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80cc155900 con 0x7f80ac0039c0 2024-02-02T13:25:29.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.645+0000 7f80c3e00640 1 -- 172.21.15.91:0/2117471446 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f80bc078230 con 0x7f80ac0039c0 2024-02-02T13:25:29.647 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:29.647 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":22,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:25.481699+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 msgr2=0x7f80a406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 0x7f80a406c920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006a90 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 msgr2=0x7f80cc14d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80bc02c380 tx=0x7f80bc001930 comp rx=0 tx=0).stop 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 shutdown_connections 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80a406a460 0x7f80a406c920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80cc15a950 0x7f80cc0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80cc0c8620 0x7f80cc15a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 --2- 172.21.15.91:0/2117471446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80ac0039c0 0x7f80cc14d270 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.648+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 >> 172.21.15.91:0/2117471446 conn(0x7f80cc0cec80 msgr2=0x7f80cc159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:29.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.649+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 shutdown_connections 2024-02-02T13:25:29.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.649+0000 7f80d2600640 1 -- 172.21.15.91:0/2117471446 wait complete. 2024-02-02T13:25:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:29.670 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:29.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.881+0000 7f0955200640 1 Processor -- start 2024-02-02T13:25:29.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.881+0000 7f0955200640 1 -- start start 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0955200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f095015d2b0 0x7f09500ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0955200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09500b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0955200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09500d1e80 con 0x7f095015d690 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0955200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09500d2120 con 0x7f095015d2b0 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0955200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500d2400 con 0x7f09500d0250 2024-02-02T13:25:29.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0947e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f095015d690 0x7f09500cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46732/0 (socket says 172.21.15.91:46732) 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.883+0000 7f0947e00640 1 -- 172.21.15.91:0/1229248536 learned_addr learned my addr 172.21.15.91:0/1229248536 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094de00640 1 --2- 172.21.15.91:0/1229248536 >> v2:172.21.15.113:3300/0 conn(0x7f095015d2b0 0x7f09500ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094e800640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09500b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839494927 0 0) 0x7f09500d1e80 con 0x7f095015d690 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0930003520 con 0x7f095015d690 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234346066 0 0) 0x7f0930003520 con 0x7f095015d690 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 msgr2=0x7f09500b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09500b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 >> v2:172.21.15.113:3300/0 conn(0x7f095015d2b0 msgr2=0x7f09500ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 --2- 172.21.15.91:0/1229248536 >> v2:172.21.15.113:3300/0 conn(0x7f095015d2b0 0x7f09500ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500d26e0 con 0x7f095015d690 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0938001670 con 0x7f095015d690 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 >> v1:172.21.15.91:6789/0 conn(0x7f095015d690 legacy=0x7f09500cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0930003970 0x7f0930003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930004d00 0x7f0930005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500d2120 con 0x7f0930003970 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500d2400 con 0x7f0930004d00 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.884+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093000b980 con 0x7f09500d0250 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0930003970 0x7f0930003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f0947e00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 -- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 msgr2=0x7f09300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094e800640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930004d00 0x7f0930005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 -- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930004d00 msgr2=0x7f0930005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930004d00 0x7f0930005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f094de00640 1 -- 172.21.15.91:0/1229248536 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930003520 con 0x7f0930003970 2024-02-02T13:25:29.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.885+0000 7f0947e00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500d0250 0x7f09300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.886+0000 7f094de00640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0930003970 0x7f0930003d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0940009ac0 tx=0x7f09300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1529b36874dcad11 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.886+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094002c710 con 0x7f0930003970 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.886+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094002c870 con 0x7f0930003970 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.886+0000 7f094d400640 1 -- 172.21.15.91:0/1229248536 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094002cc00 con 0x7f0930003970 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 -- 172.21.15.91:0/1229248536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0930003970 msgr2=0x7f0930003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 --2- 172.21.15.91:0/1229248536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0930003970 0x7f0930003d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0940009ac0 tx=0x7f09300057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f094e800640 1 -- 172.21.15.91:0/1229248536 reap_dead start 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 -- 172.21.15.91:0/1229248536 shutdown_connections 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 -- 172.21.15.91:0/1229248536 >> 172.21.15.91:0/1229248536 conn(0x7f0950065910 msgr2=0x7f0950065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 -- 172.21.15.91:0/1229248536 shutdown_connections 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.887+0000 7f0955200640 1 -- 172.21.15.91:0/1229248536 wait complete. 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.889+0000 7f0955200640 1 Processor -- start 2024-02-02T13:25:29.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.889+0000 7f0955200640 1 -- start start 2024-02-02T13:25:29.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 0x7f09500b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f095014ccc0 0x7f0950160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09501fb310 con 0x7f095014c840 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09501fa700 con 0x7f0930003970 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0955200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09501fa9e0 con 0x7f095014ccc0 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0947e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0947e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51552/0 (socket says 172.21.15.91:51552) 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.890+0000 7f0947e00640 1 -- 172.21.15.91:0/3796693256 learned_addr learned my addr 172.21.15.91:0/3796693256 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f094de00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 0x7f09500b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f094e800640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f095014ccc0 0x7f0950160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 -- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f095014ccc0 msgr2=0x7f0950160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f095014ccc0 0x7f0950160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 -- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 msgr2=0x7f09500b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 0x7f09500b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 -- 172.21.15.91:0/3796693256 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09501facc0 con 0x7f095014c840 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f094de00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 0x7f09500b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0947e00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f093800d8b0 tx=0x7f093800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380074e0 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938007640 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09501fafa0 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.891+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380049e0 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.892+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0950160560 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.893+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950066cf0 con 0x7f095014c840 2024-02-02T13:25:29.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.894+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0938004b40 con 0x7f095014c840 2024-02-02T13:25:29.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.894+0000 7f0946000640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 0x7f092806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:29.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.895+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f09380886c0 con 0x7f095014c840 2024-02-02T13:25:29.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.895+0000 7f094de00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 0x7f092806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:29.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.896+0000 7f094de00640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 0x7f092806c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0940036000 tx=0x7f0940030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:29.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:29.899+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0938053b90 con 0x7f095014c840 2024-02-02T13:25:30.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.086+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09501fb450 con 0x7f095014c840 2024-02-02T13:25:30.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.087+0000 7f0946000640 1 -- 172.21.15.91:0/3796693256 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09501fb450 con 0x7f095014c840 2024-02-02T13:25:30.089 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 msgr2=0x7f092806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 0x7f092806c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0940036000 tx=0x7f0940030000 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 msgr2=0x7f095015fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f093800d8b0 tx=0x7f093800dd70 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 shutdown_connections 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f092806a430 0x7f092806c8f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f095014ccc0 0x7f0950160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f095014c840 0x7f095015fae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 --2- 172.21.15.91:0/3796693256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930003970 0x7f09500b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 >> 172.21.15.91:0/3796693256 conn(0x7f0950065910 msgr2=0x7f095015bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 shutdown_connections 2024-02-02T13:25:30.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.090+0000 7f0955200640 1 -- 172.21.15.91:0/3796693256 wait complete. 2024-02-02T13:25:30.111 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:30.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.321+0000 7f0c4a000640 1 Processor -- start 2024-02-02T13:25:30.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.321+0000 7f0c4a000640 1 -- start start 2024-02-02T13:25:30.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.322+0000 7f0c4a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440ce130 0x7f0c440c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.322+0000 7f0c4a000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c440b0700 0x7f0c440b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.322+0000 7f0c4a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440cc7c0 con 0x7f0c4415a010 2024-02-02T13:25:30.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.322+0000 7f0c4a000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440b10d0 con 0x7f0c440b0700 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.322+0000 7f0c4a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440ce930 con 0x7f0c440ce130 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c440b0700 0x7f0c440b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c440b0700 0x7f0c440b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51558/0 (socket says 172.21.15.91:51558) 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 -- 172.21.15.91:0/2898387382 learned_addr learned my addr 172.21.15.91:0/2898387382 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c3be00640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440ce130 0x7f0c440c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 -- 172.21.15.91:0/2898387382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440ce130 msgr2=0x7f0c440c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:30.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440ce130 0x7f0c440c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 -- 172.21.15.91:0/2898387382 >> v1:172.21.15.113:6789/0 conn(0x7f0c4415a010 legacy=0x7f0c440ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:30.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.323+0000 7f0c43e00640 1 -- 172.21.15.91:0/2898387382 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440cd640 con 0x7f0c440b0700 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.324+0000 7f0c43e00640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c440b0700 0x7f0c440b0b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef60e113c5628b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c42a00640 1 -- 172.21.15.91:0/2898387382 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c02ca70 con 0x7f0c440b0700 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c42a00640 1 -- 172.21.15.91:0/2898387382 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c02cbd0 con 0x7f0c440b0700 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c42a00640 1 -- 172.21.15.91:0/2898387382 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0343f0 con 0x7f0c440b0700 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 -- 172.21.15.91:0/2898387382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c440b0700 msgr2=0x7f0c440b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c440b0700 0x7f0c440b0b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 -- 172.21.15.91:0/2898387382 shutdown_connections 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c440b0700 0x7f0c440b0b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 --2- 172.21.15.91:0/2898387382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440ce130 0x7f0c440c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 -- 172.21.15.91:0/2898387382 >> 172.21.15.91:0/2898387382 conn(0x7f0c440cec80 msgr2=0x7f0c440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:30.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 -- 172.21.15.91:0/2898387382 shutdown_connections 2024-02-02T13:25:30.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.325+0000 7f0c4a000640 1 -- 172.21.15.91:0/2898387382 wait complete. 2024-02-02T13:25:30.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.327+0000 7f0c4a000640 1 Processor -- start 2024-02-02T13:25:30.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.327+0000 7f0c4a000640 1 -- start start 2024-02-02T13:25:30.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 0x7f0c4414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 0x7f0c440b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44152ae0 con 0x7f0c4415a010 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b3fb0 con 0x7f0c440ce130 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c4a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4414f210 con 0x7f0c440b0700 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c43e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c43e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51572/0 (socket says 172.21.15.91:51572) 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.328+0000 7f0c43e00640 1 -- 172.21.15.91:0/666729913 learned_addr learned my addr 172.21.15.91:0/666729913 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43400640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 0x7f0c4414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 -- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 msgr2=0x7f0c4414ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 0x7f0c4414ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 -- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 msgr2=0x7f0c440b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c3be00640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 0x7f0c440b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 0x7f0c440b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 -- 172.21.15.91:0/666729913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c009b30 con 0x7f0c4415a010 2024-02-02T13:25:30.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43400640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 0x7f0c4414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:30.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c3be00640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 0x7f0c440b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:30.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c43e00640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c036040 tx=0x7f0c3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.329+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c009510 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.330+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c03e450 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.330+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c441502f0 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.330+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c03e800 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.330+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c44152700 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.331+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c44152c20 con 0x7f0c4415a010 2024-02-02T13:25:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.332+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c3c03e960 con 0x7f0c4415a010 2024-02-02T13:25:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.332+0000 7f0c3b400640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 0x7f0c1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.333+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0c3c0ab770 con 0x7f0c4415a010 2024-02-02T13:25:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.333+0000 7f0c43400640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 0x7f0c1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.333+0000 7f0c43400640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 0x7f0c1c06c8f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a830 tx=0x7f0c34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.337+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c44152c20 con 0x7f0c4415a010 2024-02-02T13:25:30.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.527+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c44152c20 con 0x7f0c4415a010 2024-02-02T13:25:30.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.528+0000 7f0c3b400640 1 -- 172.21.15.91:0/666729913 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c44152c20 con 0x7f0c4415a010 2024-02-02T13:25:30.529 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:30.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.530+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 msgr2=0x7f0c1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.530+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 0x7f0c1c06c8f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a830 tx=0x7f0c34008040 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.530+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 msgr2=0x7f0c4415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.530+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c036040 tx=0x7f0c3c004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 shutdown_connections 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1c06a430 0x7f0c1c06c8f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c4415a010 0x7f0c4415dc20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c440ce130 0x7f0c440b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 --2- 172.21.15.91:0/666729913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c440b0700 0x7f0c4414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 >> 172.21.15.91:0/666729913 conn(0x7f0c440cec80 msgr2=0x7f0c440ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 shutdown_connections 2024-02-02T13:25:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.531+0000 7f0c4a000640 1 -- 172.21.15.91:0/666729913 wait complete. 2024-02-02T13:25:30.550 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:30.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.757+0000 7fed72600640 1 Processor -- start 2024-02-02T13:25:30.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.757+0000 7fed72600640 1 -- start start 2024-02-02T13:25:30.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed72600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c151f90 0x7fed6c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed72600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fed6c06f4f0 0x7fed6c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed72600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c070f80 con 0x7fed6c0b0c40 2024-02-02T13:25:30.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed72600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0b10d0 con 0x7fed6c06f4f0 2024-02-02T13:25:30.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed72600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c152370 con 0x7fed6c151f90 2024-02-02T13:25:30.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed6aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fed6c06f4f0 0x7fed6c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed6aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fed6c06f4f0 0x7fed6c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51584/0 (socket says 172.21.15.91:51584) 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.759+0000 7fed6aa00640 1 -- 172.21.15.91:0/996759209 learned_addr learned my addr 172.21.15.91:0/996759209 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.760+0000 7fed6b400640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c151f90 0x7fed6c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.760+0000 7fed6aa00640 1 -- 172.21.15.91:0/996759209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c151f90 msgr2=0x7fed6c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.760+0000 7fed6aa00640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c151f90 0x7fed6c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.760+0000 7fed6aa00640 1 -- 172.21.15.91:0/996759209 >> v1:172.21.15.113:6789/0 conn(0x7fed6c0b0c40 legacy=0x7fed6c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.760+0000 7fed6aa00640 1 -- 172.21.15.91:0/996759209 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0cb010 con 0x7fed6c06f4f0 2024-02-02T13:25:30.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.761+0000 7fed6aa00640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0b0700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d71a549ea6f67b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.761+0000 7fed6a000640 1 -- 172.21.15.91:0/996759209 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54031640 con 0x7fed6c06f4f0 2024-02-02T13:25:30.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.761+0000 7fed6a000640 1 -- 172.21.15.91:0/996759209 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540317a0 con 0x7fed6c06f4f0 2024-02-02T13:25:30.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.761+0000 7fed6a000640 1 -- 172.21.15.91:0/996759209 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54031b30 con 0x7fed6c06f4f0 2024-02-02T13:25:30.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 -- 172.21.15.91:0/996759209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c06f4f0 msgr2=0x7fed6c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0b0700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 -- 172.21.15.91:0/996759209 shutdown_connections 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0b0700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 --2- 172.21.15.91:0/996759209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c151f90 0x7fed6c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 -- 172.21.15.91:0/996759209 >> 172.21.15.91:0/996759209 conn(0x7fed6c0cec80 msgr2=0x7fed6c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 -- 172.21.15.91:0/996759209 shutdown_connections 2024-02-02T13:25:30.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.762+0000 7fed72600640 1 -- 172.21.15.91:0/996759209 wait complete. 2024-02-02T13:25:30.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.763+0000 7fed72600640 1 Processor -- start 2024-02-02T13:25:30.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 -- start start 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c0b0c40 0x7fed6c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15e370 con 0x7fed6c151f90 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0c5dd0 con 0x7fed6c06f4f0 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.764+0000 7fed72600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0c60b0 con 0x7fed6c0b0c40 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51588/0 (socket says 172.21.15.91:51588) 2024-02-02T13:25:30.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 -- 172.21.15.91:0/3723048448 learned_addr learned my addr 172.21.15.91:0/3723048448 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6aa00640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c0b0c40 0x7fed6c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 -- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c0b0c40 msgr2=0x7fed6c15d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c0b0c40 0x7fed6c15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 -- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed6c06f4f0 msgr2=0x7fed6c0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 -- 172.21.15.91:0/3723048448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540094d0 con 0x7fed6c151f90 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.765+0000 7fed6be00640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b9e0 tx=0x7fed6400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.766+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64003b70 con 0x7fed6c151f90 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.766+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64003cd0 con 0x7fed6c151f90 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.766+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400cae0 con 0x7fed6c151f90 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.766+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0c6390 con 0x7fed6c151f90 2024-02-02T13:25:30.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.766+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0c8ab0 con 0x7fed6c151f90 2024-02-02T13:25:30.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.767+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c0c68b0 con 0x7fed6c151f90 2024-02-02T13:25:30.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.768+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fed6400cc40 con 0x7fed6c151f90 2024-02-02T13:25:30.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.769+0000 7fed63400640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:30.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.769+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fed640887a0 con 0x7fed6c151f90 2024-02-02T13:25:30.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.769+0000 7fed6b400640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:30.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.770+0000 7fed6b400640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 0x7fed4406c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098f0 tx=0x7fed5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:30.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.773+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fed6c0c68b0 con 0x7fed6c151f90 2024-02-02T13:25:30.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.959+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed6c0c68b0 con 0x7fed6c151f90 2024-02-02T13:25:30.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.960+0000 7fed63400640 1 -- 172.21.15.91:0/3723048448 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed6c0c68b0 con 0x7fed6c151f90 2024-02-02T13:25:30.962 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 msgr2=0x7fed4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 0x7fed4406c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098f0 tx=0x7fed5c008040 comp rx=0 tx=0).stop 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 msgr2=0x7fed6c15dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b9e0 tx=0x7fed6400bea0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 shutdown_connections 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed6c0b0c40 0x7fed6c15d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed6c151f90 0x7fed6c15dd40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 --2- 172.21.15.91:0/3723048448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed6c06f4f0 0x7fed6c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.962+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 >> 172.21.15.91:0/3723048448 conn(0x7fed6c0cec80 msgr2=0x7fed6c0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.963+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 shutdown_connections 2024-02-02T13:25:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:30.963+0000 7fed72600640 1 -- 172.21.15.91:0/3723048448 wait complete. 2024-02-02T13:25:30.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:25:31.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.180+0000 7fa102200640 1 Processor -- start 2024-02-02T13:25:31.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.180+0000 7fa102200640 1 -- start start 2024-02-02T13:25:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.181+0000 7fa102200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 0x7fa0fc0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.181+0000 7fa102200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 0x7fa0fc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.181+0000 7fa102200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc0b0540 con 0x7fa0fc0adf20 2024-02-02T13:25:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.181+0000 7fa102200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc15b2d0 con 0x7fa0fc15aef0 2024-02-02T13:25:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.181+0000 7fa102200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0ca410 con 0x7fa0fc06ce50 2024-02-02T13:25:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.182+0000 7fa100e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 0x7fa0fc0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.182+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 0x7fa0fc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.182+0000 7fa100e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 0x7fa0fc0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:59472/0 (socket says 172.21.15.91:59472) 2024-02-02T13:25:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.182+0000 7fa100e00640 1 -- 172.21.15.91:0/1564127612 learned_addr learned my addr 172.21.15.91:0/1564127612 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.183+0000 7fa0fbe00640 1 -- 172.21.15.91:0/1564127612 >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 msgr2=0x7fa0fc0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.183+0000 7fa0fbe00640 1 --2- 172.21.15.91:0/1564127612 >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 0x7fa0fc0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.183+0000 7fa0fbe00640 1 -- 172.21.15.91:0/1564127612 >> v1:172.21.15.113:6789/0 conn(0x7fa0fc0adf20 legacy=0x7fa0fc06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.183+0000 7fa0fbe00640 1 -- 172.21.15.91:0/1564127612 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0b07e0 con 0x7fa0fc06ce50 2024-02-02T13:25:31.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.185+0000 7fa0fbe00640 1 --2- 172.21.15.91:0/1564127612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 0x7fa0fc070b30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40097a0 tx=0x7fa0e402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83bcedb0c2405931 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.185+0000 7fa0fb400640 1 -- 172.21.15.91:0/1564127612 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e402d040 con 0x7fa0fc06ce50 2024-02-02T13:25:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.185+0000 7fa0fb400640 1 -- 172.21.15.91:0/1564127612 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4031810 con 0x7fa0fc06ce50 2024-02-02T13:25:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.185+0000 7fa0fb400640 1 -- 172.21.15.91:0/1564127612 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4031ba0 con 0x7fa0fc06ce50 2024-02-02T13:25:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 -- 172.21.15.91:0/1564127612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 msgr2=0x7fa0fc070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 --2- 172.21.15.91:0/1564127612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 0x7fa0fc070b30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40097a0 tx=0x7fa0e402b8b0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 -- 172.21.15.91:0/1564127612 shutdown_connections 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 --2- 172.21.15.91:0/1564127612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc06ce50 0x7fa0fc070b30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 --2- 172.21.15.91:0/1564127612 >> v2:172.21.15.113:3300/0 conn(0x7fa0fc15aef0 0x7fa0fc0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 -- 172.21.15.91:0/1564127612 >> 172.21.15.91:0/1564127612 conn(0x7fa0fc0cb4a0 msgr2=0x7fa0fc0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 -- 172.21.15.91:0/1564127612 shutdown_connections 2024-02-02T13:25:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.186+0000 7fa102200640 1 -- 172.21.15.91:0/1564127612 wait complete. 2024-02-02T13:25:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.188+0000 7fa102200640 1 Processor -- start 2024-02-02T13:25:31.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.188+0000 7fa102200640 1 -- start start 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0fc06ce50 0x7fa0fc14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 0x7fa0fc0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0c4690 con 0x7fa0fc0adf20 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc15cec0 con 0x7fa0fc06ce50 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.189+0000 7fa102200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc1502e0 con 0x7fa0fc15aef0 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51596/0 (socket says 172.21.15.91:51596) 2024-02-02T13:25:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 -- 172.21.15.91:0/947859144 learned_addr learned my addr 172.21.15.91:0/947859144 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa100e00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0fc06ce50 0x7fa0fc14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa101800640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 0x7fa0fc0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 -- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 msgr2=0x7fa0fc0c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 0x7fa0fc0c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 -- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0fc06ce50 msgr2=0x7fa0fc14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0fc06ce50 0x7fa0fc14fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa0fbe00640 1 -- 172.21.15.91:0/947859144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009400 con 0x7fa0fc0adf20 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.190+0000 7fa101800640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 0x7fa0fc0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa0fbe00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098c0 tx=0x7fa0e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4006e10 con 0x7fa0fc0adf20 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e403ad60 con 0x7fa0fc0adf20 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4044610 con 0x7fa0fc0adf20 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc15c2b0 con 0x7fa0fc0adf20 2024-02-02T13:25:31.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.191+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc15c870 con 0x7fa0fc0adf20 2024-02-02T13:25:31.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.192+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc04dff0 con 0x7fa0fc0adf20 2024-02-02T13:25:31.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.193+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0e4044770 con 0x7fa0fc0adf20 2024-02-02T13:25:31.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.194+0000 7fa0f8c00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 0x7fa0d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.194+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa0e40aacf0 con 0x7fa0fc0adf20 2024-02-02T13:25:31.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.194+0000 7fa100e00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 0x7fa0d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.195+0000 7fa100e00640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 0x7fa0d406c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc153e00 tx=0x7fa0ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.199+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0e40762e0 con 0x7fa0fc0adf20 2024-02-02T13:25:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.388+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0fc0c5300 con 0x7fa0fc0adf20 2024-02-02T13:25:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.389+0000 7fa0f8c00640 1 -- 172.21.15.91:0/947859144 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa0e4079900 con 0x7fa0fc0adf20 2024-02-02T13:25:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:25:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 msgr2=0x7fa0d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 0x7fa0d406c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc153e00 tx=0x7fa0ec008040 comp rx=0 tx=0).stop 2024-02-02T13:25:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 msgr2=0x7fa0fc0c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098c0 tx=0x7fa0e40046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 shutdown_connections 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0d406a430 0x7fa0d406c8f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0fc15aef0 0x7fa0fc0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0fc0adf20 0x7fa0fc0c4dc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 --2- 172.21.15.91:0/947859144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0fc06ce50 0x7fa0fc14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 >> 172.21.15.91:0/947859144 conn(0x7fa0fc0cb4a0 msgr2=0x7fa0fc070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 shutdown_connections 2024-02-02T13:25:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.391+0000 7fa102200640 1 -- 172.21.15.91:0/947859144 wait complete. 2024-02-02T13:25:31.409 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-02-02T13:25:31.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:31.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-02-02T13:25:31.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:25:31.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:31.512 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-02-02T13:25:31.560 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:25:31.561 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:31.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.765+0000 7f26f2400640 1 Processor -- start 2024-02-02T13:25:31.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.765+0000 7f26f2400640 1 -- start start 2024-02-02T13:25:31.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.766+0000 7f26f2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26ec15f5a0 0x7f26ec0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26f2400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f26ec0b0c40 0x7f26ec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26f2400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec0b1070 con 0x7f26ec151f90 2024-02-02T13:25:31.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26f2400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec15fa30 con 0x7f26ec0b0c40 2024-02-02T13:25:31.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26f2400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec070f80 con 0x7f26ec15f5a0 2024-02-02T13:25:31.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26eb400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f26ec151f90 0x7f26ec06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46746/0 (socket says 172.21.15.91:46746) 2024-02-02T13:25:31.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26eb400640 1 -- 172.21.15.91:0/1860576232 learned_addr learned my addr 172.21.15.91:0/1860576232 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:31.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ebe00640 1 --2- 172.21.15.91:0/1860576232 >> v2:172.21.15.91:3300/0 conn(0x7f26ec0b0c40 0x7f26ec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ea000640 1 -- 172.21.15.91:0/1860576232 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210233385 0 0) 0x7f26ec0b1070 con 0x7f26ec151f90 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ea000640 1 -- 172.21.15.91:0/1860576232 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26cc003520 con 0x7f26ec151f90 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26eaa00640 1 --2- 172.21.15.91:0/1860576232 >> v2:172.21.15.113:3300/0 conn(0x7f26ec15f5a0 0x7f26ec0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ebe00640 1 -- 172.21.15.91:0/1860576232 >> v2:172.21.15.113:3300/0 conn(0x7f26ec15f5a0 msgr2=0x7f26ec0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ebe00640 1 --2- 172.21.15.91:0/1860576232 >> v2:172.21.15.113:3300/0 conn(0x7f26ec15f5a0 0x7f26ec0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ebe00640 1 -- 172.21.15.91:0/1860576232 >> v1:172.21.15.91:6789/0 conn(0x7f26ec151f90 legacy=0x7f26ec06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.767+0000 7f26ebe00640 1 -- 172.21.15.91:0/1860576232 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec152370 con 0x7f26ec0b0c40 2024-02-02T13:25:31.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26ebe00640 1 --2- 172.21.15.91:0/1860576232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec0b3af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26ec071a00 tx=0x7f26e402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4c49fc35eb1140 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26ea000640 1 -- 172.21.15.91:0/1860576232 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e402cc70 con 0x7f26ec0b0c40 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26ea000640 1 -- 172.21.15.91:0/1860576232 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e402cdd0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26ea000640 1 -- 172.21.15.91:0/1860576232 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4034660 con 0x7f26ec0b0c40 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26f2400640 1 -- 172.21.15.91:0/1860576232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 msgr2=0x7f26ec0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.769+0000 7f26f2400640 1 --2- 172.21.15.91:0/1860576232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec0b3af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26ec071a00 tx=0x7f26e402bec0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 -- 172.21.15.91:0/1860576232 shutdown_connections 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 --2- 172.21.15.91:0/1860576232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec0b3af0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 --2- 172.21.15.91:0/1860576232 >> v2:172.21.15.113:3300/0 conn(0x7f26ec15f5a0 0x7f26ec0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 -- 172.21.15.91:0/1860576232 >> 172.21.15.91:0/1860576232 conn(0x7f26ec0cec80 msgr2=0x7f26ec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 -- 172.21.15.91:0/1860576232 shutdown_connections 2024-02-02T13:25:31.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.770+0000 7f26f2400640 1 -- 172.21.15.91:0/1860576232 wait complete. 2024-02-02T13:25:31.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.771+0000 7f26f2400640 1 Processor -- start 2024-02-02T13:25:31.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.771+0000 7f26f2400640 1 -- start start 2024-02-02T13:25:31.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26ec151f90 0x7f26ec1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26ec15f5a0 0x7f26ec1f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec1f3ea0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec1f1cb0 con 0x7f26ec151f90 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.772+0000 7f26f2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec154de0 con 0x7f26ec15f5a0 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51604/0 (socket says 172.21.15.91:51604) 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 -- 172.21.15.91:0/1583920052 learned_addr learned my addr 172.21.15.91:0/1583920052 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 -- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26ec15f5a0 msgr2=0x7f26ec1f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26ec15f5a0 0x7f26ec1f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 -- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26ec151f90 msgr2=0x7f26ec1f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26ec151f90 0x7f26ec1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 -- 172.21.15.91:0/1583920052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4009a70 con 0x7f26ec0b0c40 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.773+0000 7f26eb400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c410 tx=0x7f26dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.774+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc005ce0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.774+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc005e40 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.774+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0072f0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.774+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec1550c0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.774+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0c6db0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.775+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec066cf0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.776+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f26dc007510 con 0x7f26ec0b0c40 2024-02-02T13:25:31.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.777+0000 7f26e3400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 0x7f26c406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:31.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.777+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f26dc087fd0 con 0x7f26ec0b0c40 2024-02-02T13:25:31.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.778+0000 7f26eaa00640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 0x7f26c406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:31.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.778+0000 7f26eaa00640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 0x7f26c406c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26d4006fd0 tx=0x7f26d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:31.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:31.782+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f26dc053600 con 0x7f26ec0b0c40 2024-02-02T13:25:32.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.009+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26ec073f10 con 0x7f26ec0b0c40 2024-02-02T13:25:32.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.010+0000 7f26e3400640 1 -- 172.21.15.91:0/1583920052 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f26dc056c20 con 0x7f26ec0b0c40 2024-02-02T13:25:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:32.012 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:25:32.012 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 msgr2=0x7f26c406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 0x7f26c406c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26d4006fd0 tx=0x7f26d4008040 comp rx=0 tx=0).stop 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 msgr2=0x7f26ec1546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c410 tx=0x7f26dc00c8d0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 shutdown_connections 2024-02-02T13:25:32.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f26c406a430 0x7f26c406c8f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26ec15f5a0 0x7f26ec1f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26ec151f90 0x7f26ec1f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.012+0000 7f26f2400640 1 --2- 172.21.15.91:0/1583920052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26ec0b0c40 0x7f26ec1546e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.013+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 >> 172.21.15.91:0/1583920052 conn(0x7f26ec0cec80 msgr2=0x7f26ec159280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.013+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 shutdown_connections 2024-02-02T13:25:32.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.013+0000 7f26f2400640 1 -- 172.21.15.91:0/1583920052 wait complete. 2024-02-02T13:25:32.033 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:32.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.216+0000 7ff094200640 1 Processor -- start 2024-02-02T13:25:32.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.217+0000 7ff094200640 1 -- start start 2024-02-02T13:25:32.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff094200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff08c15d3d0 0x7ff08c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff094200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0d2e80 0x7ff08c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff094200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08c15d7b0 con 0x7ff08c0b8760 2024-02-02T13:25:32.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff094200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08c0d3310 con 0x7ff08c15d3d0 2024-02-02T13:25:32.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff094200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0d3530 con 0x7ff08c0d2e80 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff091800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff08c15d3d0 0x7ff08c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff091800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff08c15d3d0 0x7ff08c0cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51606/0 (socket says 172.21.15.91:51606) 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.218+0000 7ff091800640 1 -- 172.21.15.91:0/3260480761 learned_addr learned my addr 172.21.15.91:0/3260480761 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.219+0000 7ff092200640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0d2e80 0x7ff08c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.219+0000 7ff091800640 1 -- 172.21.15.91:0/3260480761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0d2e80 msgr2=0x7ff08c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.219+0000 7ff091800640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0d2e80 0x7ff08c0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.219+0000 7ff091800640 1 -- 172.21.15.91:0/3260480761 >> v1:172.21.15.113:6789/0 conn(0x7ff08c0b8760 legacy=0x7ff08c0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:32.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.219+0000 7ff091800640 1 -- 172.21.15.91:0/3260480761 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c0d3810 con 0x7ff08c15d3d0 2024-02-02T13:25:32.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.220+0000 7ff091800640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0cfec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009e50 tx=0x7ff07c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d64238ebd24d423 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.220+0000 7ff08be00640 1 -- 172.21.15.91:0/3260480761 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02ca70 con 0x7ff08c15d3d0 2024-02-02T13:25:32.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.220+0000 7ff08be00640 1 -- 172.21.15.91:0/3260480761 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c02cbd0 con 0x7ff08c15d3d0 2024-02-02T13:25:32.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.220+0000 7ff08be00640 1 -- 172.21.15.91:0/3260480761 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0343f0 con 0x7ff08c15d3d0 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 -- 172.21.15.91:0/3260480761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c15d3d0 msgr2=0x7ff08c0cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0cfec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009e50 tx=0x7ff07c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 -- 172.21.15.91:0/3260480761 shutdown_connections 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0d2e80 0x7ff08c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 --2- 172.21.15.91:0/3260480761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0cfec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 -- 172.21.15.91:0/3260480761 >> 172.21.15.91:0/3260480761 conn(0x7ff08c070450 msgr2=0x7ff08c070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 -- 172.21.15.91:0/3260480761 shutdown_connections 2024-02-02T13:25:32.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.221+0000 7ff094200640 1 -- 172.21.15.91:0/3260480761 wait complete. 2024-02-02T13:25:32.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.223+0000 7ff094200640 1 Processor -- start 2024-02-02T13:25:32.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.223+0000 7ff094200640 1 -- start start 2024-02-02T13:25:32.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0b8760 0x7ff08c1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c151df0 con 0x7ff08c0d2e80 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0c7760 con 0x7ff08c15d3d0 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff094200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0c7a40 con 0x7ff08c0b8760 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51612/0 (socket says 172.21.15.91:51612) 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 -- 172.21.15.91:0/3943696905 learned_addr learned my addr 172.21.15.91:0/3943696905 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff091800640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0b8760 0x7ff08c1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 -- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0b8760 msgr2=0x7ff08c1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff092200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0b8760 0x7ff08c1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 -- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c15d3d0 msgr2=0x7ff08c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.224+0000 7ff090e00640 1 -- 172.21.15.91:0/3943696905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009b30 con 0x7ff08c0d2e80 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff090e00640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b3f0 tx=0x7ff07400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074011830 con 0x7ff08c0d2e80 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff074011990 con 0x7ff08c0d2e80 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074011d40 con 0x7ff08c0d2e80 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c0c7d20 con 0x7ff08c0d2e80 2024-02-02T13:25:32.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.225+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c0c82e0 con 0x7ff08c0d2e80 2024-02-02T13:25:32.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.226+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c0c8500 con 0x7ff08c0d2e80 2024-02-02T13:25:32.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.227+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff07401a400 con 0x7ff08c0d2e80 2024-02-02T13:25:32.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.228+0000 7ff089600640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 0x7ff06406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.229+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff074088940 con 0x7ff08c0d2e80 2024-02-02T13:25:32.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.229+0000 7ff091800640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 0x7ff06406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.229+0000 7ff091800640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 0x7ff06406c8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff07c035000 tx=0x7ff07c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.233+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff074053e10 con 0x7ff08c0d2e80 2024-02-02T13:25:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.420+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff08c067400 con 0x7ff08c0d2e80 2024-02-02T13:25:32.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.422+0000 7ff089600640 1 -- 172.21.15.91:0/3943696905 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16287 (secure 0 0 0) 0x7ff074015090 con 0x7ff08c0d2e80 2024-02-02T13:25:32.424 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:32.424 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":22,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:25.481699+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 msgr2=0x7ff06406c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 0x7ff06406c8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff07c035000 tx=0x7ff07c030000 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 msgr2=0x7ff08c0c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b3f0 tx=0x7ff07400b8b0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 shutdown_connections 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff06406a400 0x7ff06406c8c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c15d3d0 0x7ff08c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff08c0d2e80 0x7ff08c0c6bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 --2- 172.21.15.91:0/3943696905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0b8760 0x7ff08c1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 >> 172.21.15.91:0/3943696905 conn(0x7ff08c070450 msgr2=0x7ff08c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 shutdown_connections 2024-02-02T13:25:32.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.425+0000 7ff094200640 1 -- 172.21.15.91:0/3943696905 wait complete. 2024-02-02T13:25:32.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:24:34.438237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:24:35.744074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:32.443 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.635+0000 7f2a61800640 1 Processor -- start 2024-02-02T13:25:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.635+0000 7f2a61800640 1 -- start start 2024-02-02T13:25:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a61800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a5c0ce890 0x7f2a5c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a61800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c1611f0 con 0x7f2a5c06c270 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a61800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0cf010 con 0x7f2a5c0cb2a0 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a61800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0ced30 con 0x7f2a5c0ce890 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a5b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a5c0ce890 0x7f2a5c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a60e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2a5c06c270 0x7f2a5c0c9900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46756/0 (socket says 172.21.15.91:46756) 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a60e00640 1 -- 172.21.15.91:0/736235332 learned_addr learned my addr 172.21.15.91:0/736235332 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a5b400640 1 -- 172.21.15.91:0/736235332 >> v1:172.21.15.113:6789/0 conn(0x7f2a5c0cb2a0 legacy=0x7f2a5c06bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a5b400640 1 -- 172.21.15.91:0/736235332 >> v1:172.21.15.91:6789/0 conn(0x7f2a5c06c270 legacy=0x7f2a5c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.637+0000 7f2a5aa00640 1 -- 172.21.15.91:0/736235332 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180908690 0 0) 0x7f2a5c1611f0 con 0x7f2a5c06c270 2024-02-02T13:25:32.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.638+0000 7f2a5b400640 1 -- 172.21.15.91:0/736235332 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c160ba0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.639+0000 7f2a5b400640 1 --2- 172.21.15.91:0/736235332 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c15df00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a440097a0 tx=0x7f2a4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84614c640e39943d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.639+0000 7f2a5aa00640 1 -- 172.21.15.91:0/736235332 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402e070 con 0x7f2a5c0ce890 2024-02-02T13:25:32.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.639+0000 7f2a5aa00640 1 -- 172.21.15.91:0/736235332 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a440315a0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.639+0000 7f2a5aa00640 1 -- 172.21.15.91:0/736235332 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44031930 con 0x7f2a5c0ce890 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 -- 172.21.15.91:0/736235332 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 msgr2=0x7f2a5c15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 --2- 172.21.15.91:0/736235332 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c15df00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a440097a0 tx=0x7f2a4402b840 comp rx=0 tx=0).stop 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 -- 172.21.15.91:0/736235332 shutdown_connections 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 --2- 172.21.15.91:0/736235332 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c15df00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 -- 172.21.15.91:0/736235332 >> 172.21.15.91:0/736235332 conn(0x7f2a5c0cf2b0 msgr2=0x7f2a5c0cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 -- 172.21.15.91:0/736235332 shutdown_connections 2024-02-02T13:25:32.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.640+0000 7f2a61800640 1 -- 172.21.15.91:0/736235332 wait complete. 2024-02-02T13:25:32.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.643+0000 7f2a61800640 1 Processor -- start 2024-02-02T13:25:32.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.643+0000 7f2a61800640 1 -- start start 2024-02-02T13:25:32.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.644+0000 7f2a61800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a5c06c270 0x7f2a5c14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a61800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a5c0cb2a0 0x7f2a5c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a61800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a61800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15a840 con 0x7f2a5c0ce890 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a61800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c9510 con 0x7f2a5c0cb2a0 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a61800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c68c0 con 0x7f2a5c06c270 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51618/0 (socket says 172.21.15.91:51618) 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 -- 172.21.15.91:0/3385362485 learned_addr learned my addr 172.21.15.91:0/3385362485 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a5b400640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a5c0cb2a0 0x7f2a5c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a5be00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a5c06c270 0x7f2a5c14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 -- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a5c06c270 msgr2=0x7f2a5c14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a5c06c270 0x7f2a5c14a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 -- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a5c0cb2a0 msgr2=0x7f2a5c15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a5c0cb2a0 0x7f2a5c15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.645+0000 7f2a60e00640 1 -- 172.21.15.91:0/3385362485 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009400 con 0x7f2a5c0ce890 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a60e00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c750 tx=0x7f2a5400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400ce60 con 0x7f2a5c0ce890 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400a640 con 0x7f2a5c0ce890 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400a9d0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c0b2760 con 0x7f2a5c0ce890 2024-02-02T13:25:32.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.646+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c0c8ec0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.647+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c0b28f0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.648+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2a5400ab30 con 0x7f2a5c0ce890 2024-02-02T13:25:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.649+0000 7f2a53e00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 0x7f2a3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.649+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2a54087de0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.649+0000 7f2a5be00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 0x7f2a3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.650+0000 7f2a5be00640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 0x7f2a3406c8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c004ea0 tx=0x7f2a4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:32.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.653+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2a54053350 con 0x7f2a5c0ce890 2024-02-02T13:25:32.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.844+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a5c04dff0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.844+0000 7f2a53e00640 1 -- 172.21.15.91:0/3385362485 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a540180a0 con 0x7f2a5c0ce890 2024-02-02T13:25:32.846 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 msgr2=0x7f2a3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 0x7f2a3406c8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c004ea0 tx=0x7f2a4c009040 comp rx=0 tx=0).stop 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 msgr2=0x7f2a5c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c750 tx=0x7f2a5400cc10 comp rx=0 tx=0).stop 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 shutdown_connections 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a3406a430 0x7f2a3406c8f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a5c0ce890 0x7f2a5c0c6380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a5c0cb2a0 0x7f2a5c15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 --2- 172.21.15.91:0/3385362485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a5c06c270 0x7f2a5c14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 >> 172.21.15.91:0/3385362485 conn(0x7f2a5c0cf2b0 msgr2=0x7f2a5c15e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 shutdown_connections 2024-02-02T13:25:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:32.847+0000 7f2a61800640 1 -- 172.21.15.91:0/3385362485 wait complete. 2024-02-02T13:25:32.865 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:25:33.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.061+0000 7fa655800640 1 Processor -- start 2024-02-02T13:25:33.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.061+0000 7fa655800640 1 -- start start 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.062+0000 7fa655800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa6500cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.062+0000 7fa655800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65006b5f0 con 0x7fa6500c9240 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.062+0000 7fa655800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65006c450 con 0x7fa6501597a0 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.062+0000 7fa655800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65006cff0 con 0x7fa6500cd690 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa6500c9240 0x7fa65006f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46762/0 (socket says 172.21.15.91:46762) 2024-02-02T13:25:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64f400640 1 -- 172.21.15.91:0/3304940857 learned_addr learned my addr 172.21.15.91:0/3304940857 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa645e00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa6500cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421103137 0 0) 0x7fa65006b5f0 con 0x7fa6500c9240 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa630003520 con 0x7fa6500c9240 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398775630 0 0) 0x7fa630003520 con 0x7fa6500c9240 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 msgr2=0x7fa6500cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa6500cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.063+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 >> v1:172.21.15.113:6789/0 conn(0x7fa6501597a0 legacy=0x7fa65006c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650159b80 con 0x7fa6500c9240 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa6480019f0 con 0x7fa6500c9240 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 >> v1:172.21.15.91:6789/0 conn(0x7fa6500c9240 legacy=0x7fa65006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6300039a0 0x7fa630003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6300047c0 0x7fa630004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa630005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65006cff0 con 0x7fa6300047c0 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630005a50 con 0x7fa6300039a0 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa630005bc0 con 0x7fa6500cd690 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6300047c0 0x7fa630004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64f400640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa630005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 -- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 msgr2=0x7fa630005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6500cd690 0x7fa630005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 -- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6300039a0 msgr2=0x7fa630003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa64ea00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6300039a0 0x7fa630003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6300039a0 0x7fa630003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.064+0000 7fa645e00640 1 -- 172.21.15.91:0/3304940857 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630003520 con 0x7fa6300047c0 2024-02-02T13:25:33.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.065+0000 7fa64ea00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6300039a0 0x7fa630003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:33.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.065+0000 7fa645e00640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6300047c0 0x7fa630004c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa638009ac0 tx=0x7fa6300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b32d33a1ed23b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63802c6b0 con 0x7fa6300047c0 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63802c810 con 0x7fa6300047c0 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa64e000640 1 -- 172.21.15.91:0/3304940857 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63802cba0 con 0x7fa6300047c0 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 -- 172.21.15.91:0/3304940857 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6300047c0 msgr2=0x7fa630004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 --2- 172.21.15.91:0/3304940857 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6300047c0 0x7fa630004c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa638009ac0 tx=0x7fa6300057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa64f400640 1 -- 172.21.15.91:0/3304940857 reap_dead start 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 -- 172.21.15.91:0/3304940857 shutdown_connections 2024-02-02T13:25:33.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 -- 172.21.15.91:0/3304940857 >> 172.21.15.91:0/3304940857 conn(0x7fa6500cec80 msgr2=0x7fa6500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:33.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 -- 172.21.15.91:0/3304940857 shutdown_connections 2024-02-02T13:25:33.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.066+0000 7fa655800640 1 -- 172.21.15.91:0/3304940857 wait complete. 2024-02-02T13:25:33.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.068+0000 7fa655800640 1 Processor -- start 2024-02-02T13:25:33.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.068+0000 7fa655800640 1 -- start start 2024-02-02T13:25:33.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6300047c0 0x7fa65014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 0x7fa6500c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500ce030 con 0x7fa650158070 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65014b600 con 0x7fa6500b4130 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa655800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500b3090 con 0x7fa6300047c0 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa645e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa645e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51636/0 (socket says 172.21.15.91:51636) 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.069+0000 7fa645e00640 1 -- 172.21.15.91:0/3427803298 learned_addr learned my addr 172.21.15.91:0/3427803298 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa64f400640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 0x7fa6500c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 -- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6300047c0 msgr2=0x7fa65014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa64ea00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6300047c0 0x7fa65014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6300047c0 0x7fa65014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 -- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 msgr2=0x7fa6500c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 0x7fa6500c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 -- 172.21.15.91:0/3427803298 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500b3370 con 0x7fa650158070 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa64f400640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 0x7fa6500c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:33.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa645e00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa63802c250 tx=0x7fa638004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638003810 con 0x7fa650158070 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638003970 con 0x7fa650158070 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.070+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500b3650 con 0x7fa650158070 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.071+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638003ce0 con 0x7fa650158070 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.071+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500b3c10 con 0x7fa650158070 2024-02-02T13:25:33.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.072+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6500ce030 con 0x7fa650158070 2024-02-02T13:25:33.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.073+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa638034440 con 0x7fa650158070 2024-02-02T13:25:33.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.074+0000 7fa647400640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 0x7fa62806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.074+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa6380aac70 con 0x7fa650158070 2024-02-02T13:25:33.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.074+0000 7fa64ea00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 0x7fa62806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.075+0000 7fa64ea00640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 0x7fa62806c920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa640007fd0 tx=0x7fa640009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.078+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa638076310 con 0x7fa650158070 2024-02-02T13:25:33.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.269+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6500b3dd0 con 0x7fa650158070 2024-02-02T13:25:33.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.270+0000 7fa647400640 1 -- 172.21.15.91:0/3427803298 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa638039020 con 0x7fa650158070 2024-02-02T13:25:33.271 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:25:33.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.272+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 msgr2=0x7fa62806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.272+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 0x7fa62806c920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa640007fd0 tx=0x7fa640009040 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.272+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 msgr2=0x7fa6500b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.272+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa63802c250 tx=0x7fa638004730 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 shutdown_connections 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa62806a460 0x7fa62806c920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6500b4130 0x7fa6500c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa650158070 0x7fa6500b27b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 --2- 172.21.15.91:0/3427803298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6300047c0 0x7fa65014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 >> 172.21.15.91:0/3427803298 conn(0x7fa6500cec80 msgr2=0x7fa650070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 shutdown_connections 2024-02-02T13:25:33.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.273+0000 7fa655800640 1 -- 172.21.15.91:0/3427803298 wait complete. 2024-02-02T13:25:33.291 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:25:33.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.496+0000 7f451f200640 1 Processor -- start 2024-02-02T13:25:33.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.496+0000 7f451f200640 1 -- start start 2024-02-02T13:25:33.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f451f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f45180b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f451f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4518070f80 con 0x7f451815f5a0 2024-02-02T13:25:33.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f451f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451806c600 con 0x7f4518151f90 2024-02-02T13:25:33.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f451f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451815fa30 con 0x7f45180b0e10 2024-02-02T13:25:33.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f4517400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f451815f5a0 0x7f45180b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46764/0 (socket says 172.21.15.91:46764) 2024-02-02T13:25:33.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f4517400640 1 -- 172.21.15.91:0/191438824 learned_addr learned my addr 172.21.15.91:0/191438824 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f451d200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f45180b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240340340 0 0) 0x7f4518070f80 con 0x7f451815f5a0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.498+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f8003520 con 0x7f451815f5a0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2269504624 0 0) 0x7f44f8003520 con 0x7f451815f5a0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 msgr2=0x7f45180b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f45180b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 >> v1:172.21.15.113:6789/0 conn(0x7f4518151f90 legacy=0x7f451806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518152370 con 0x7f451815f5a0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f45080019f0 con 0x7f451815f5a0 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 >> v1:172.21.15.91:6789/0 conn(0x7f451815f5a0 legacy=0x7f45180b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44f80039a0 0x7f44f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f44f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f44f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451815fa30 con 0x7f44f8004d00 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f800ba90 con 0x7f44f80039a0 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f800bc50 con 0x7f45180b0e10 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.499+0000 7f451d200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f44f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f4517400640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f44f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f451d200640 1 -- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 msgr2=0x7f44f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f451d200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180b0e10 0x7f44f80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f451d200640 1 -- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44f80039a0 msgr2=0x7f44f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f451d200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44f80039a0 0x7f44f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.500+0000 7f451d200640 1 -- 172.21.15.91:0/191438824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8003520 con 0x7f44f8004d00 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f451d200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f44f8005180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45180cec30 tx=0x7f44f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9736c2fabd53f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451002c630 con 0x7f44f8004d00 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451002c790 con 0x7f44f8004d00 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f4516a00640 1 -- 172.21.15.91:0/191438824 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451002cb40 con 0x7f44f8004d00 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f451f200640 1 -- 172.21.15.91:0/191438824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 msgr2=0x7f44f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f451f200640 1 --2- 172.21.15.91:0/191438824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f44f8005180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45180cec30 tx=0x7f44f80057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.501+0000 7f451d200640 1 -- 172.21.15.91:0/191438824 reap_dead start 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.502+0000 7f451f200640 1 -- 172.21.15.91:0/191438824 shutdown_connections 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.502+0000 7f451f200640 1 -- 172.21.15.91:0/191438824 >> 172.21.15.91:0/191438824 conn(0x7f45180cec80 msgr2=0x7f45180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.502+0000 7f451f200640 1 -- 172.21.15.91:0/191438824 shutdown_connections 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.502+0000 7f451f200640 1 -- 172.21.15.91:0/191438824 wait complete. 2024-02-02T13:25:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.503+0000 7f451f200640 1 Processor -- start 2024-02-02T13:25:33.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.503+0000 7f451f200640 1 -- start start 2024-02-02T13:25:33.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45180c83f0 0x7f4518150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180cdd80 0x7f45180ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180ce770 con 0x7f44f8004d00 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451815d8c0 con 0x7f45180c83f0 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f451f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180c88f0 con 0x7f45180cdd80 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.504+0000 7f4517e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51650/0 (socket says 172.21.15.91:51650) 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 -- 172.21.15.91:0/2158241121 learned_addr learned my addr 172.21.15.91:0/2158241121 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517400640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45180c83f0 0x7f4518150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 -- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180cdd80 msgr2=0x7f45180ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180cdd80 0x7f45180ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 -- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45180c83f0 msgr2=0x7f4518150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45180c83f0 0x7f4518150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 -- 172.21.15.91:0/2158241121 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451815d270 con 0x7f44f8004d00 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f4517e00640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f450000c3d0 tx=0x7f450000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000052d0 con 0x7f44f8004d00 2024-02-02T13:25:33.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500005430 con 0x7f44f8004d00 2024-02-02T13:25:33.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500003830 con 0x7f44f8004d00 2024-02-02T13:25:33.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.505+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451815d550 con 0x7f44f8004d00 2024-02-02T13:25:33.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.506+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4518150c50 con 0x7f44f8004d00 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.507+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4518066cf0 con 0x7f44f8004d00 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.507+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4500003990 con 0x7f44f8004d00 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.508+0000 7f450fe00640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 0x7f44f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.508+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4500087f20 con 0x7f44f8004d00 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.508+0000 7f4517400640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 0x7f44f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.509+0000 7f4517400640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 0x7f44f006c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4508001350 tx=0x7f4508009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.512+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4500053480 con 0x7f44f8004d00 2024-02-02T13:25:33.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.701+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f451806c7b0 con 0x7f44f8004d00 2024-02-02T13:25:33.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.701+0000 7f450fe00640 1 -- 172.21.15.91:0/2158241121 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4500056aa0 con 0x7f44f8004d00 2024-02-02T13:25:33.703 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 msgr2=0x7f44f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 0x7f44f006c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4508001350 tx=0x7f4508009040 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 msgr2=0x7f451815ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f450000c3d0 tx=0x7f450000c890 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 shutdown_connections 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f44f006a430 0x7f44f006c8f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45180cdd80 0x7f45180ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45180c83f0 0x7f4518150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 --2- 172.21.15.91:0/2158241121 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44f8004d00 0x7f451815ddc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 >> 172.21.15.91:0/2158241121 conn(0x7f45180cec80 msgr2=0x7f45180cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:33.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.704+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 shutdown_connections 2024-02-02T13:25:33.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.705+0000 7f451f200640 1 -- 172.21.15.91:0/2158241121 wait complete. 2024-02-02T13:25:33.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:33.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.927+0000 7fe8eba00640 1 Processor -- start 2024-02-02T13:25:33.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.927+0000 7fe8eba00640 1 -- start start 2024-02-02T13:25:33.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.928+0000 7fe8eba00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8e415a010 0x7fe8e40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.928+0000 7fe8eba00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40cc7c0 con 0x7fe8e40b0700 2024-02-02T13:25:33.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.928+0000 7fe8eba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40ce930 con 0x7fe8e40ce130 2024-02-02T13:25:33.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.928+0000 7fe8eba00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e40cd640 con 0x7fe8e415a010 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe8e415a010 0x7fe8e40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe8e40b0700 0x7fe8e406ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46776/0 (socket says 172.21.15.91:46776) 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9a00640 1 -- 172.21.15.91:0/393681581 learned_addr learned my addr 172.21.15.91:0/393681581 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e3400640 1 -- 172.21.15.91:0/393681581 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077319257 0 0) 0x7fe8e40cc7c0 con 0x7fe8e40b0700 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9000640 1 -- 172.21.15.91:0/393681581 >> v1:172.21.15.113:6789/0 conn(0x7fe8e40ce130 legacy=0x7fe8e40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9000640 1 -- 172.21.15.91:0/393681581 >> v1:172.21.15.91:6789/0 conn(0x7fe8e40b0700 legacy=0x7fe8e406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.929+0000 7fe8e9000640 1 -- 172.21.15.91:0/393681581 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e406c890 con 0x7fe8e415a010 2024-02-02T13:25:33.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.930+0000 7fe8e9000640 1 --2- 172.21.15.91:0/393681581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e415a010 0x7fe8e40c9670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f559747db27f49b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.930+0000 7fe8e3400640 1 -- 172.21.15.91:0/393681581 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4031830 con 0x7fe8e415a010 2024-02-02T13:25:33.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8e3400640 1 -- 172.21.15.91:0/393681581 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4031990 con 0x7fe8e415a010 2024-02-02T13:25:33.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8e3400640 1 -- 172.21.15.91:0/393681581 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4031d20 con 0x7fe8e415a010 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 -- 172.21.15.91:0/393681581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e415a010 msgr2=0x7fe8e40c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 --2- 172.21.15.91:0/393681581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e415a010 0x7fe8e40c9670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 -- 172.21.15.91:0/393681581 shutdown_connections 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 --2- 172.21.15.91:0/393681581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e415a010 0x7fe8e40c9670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 -- 172.21.15.91:0/393681581 >> 172.21.15.91:0/393681581 conn(0x7fe8e40cec80 msgr2=0x7fe8e40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 -- 172.21.15.91:0/393681581 shutdown_connections 2024-02-02T13:25:33.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.931+0000 7fe8eba00640 1 -- 172.21.15.91:0/393681581 wait complete. 2024-02-02T13:25:33.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.933+0000 7fe8eba00640 1 Processor -- start 2024-02-02T13:25:33.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.933+0000 7fe8eba00640 1 -- start start 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8e40ce130 0x7fe8e4150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 0x7fe8e40af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e41f6ea0 con 0x7fe8e40b0700 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e40c5390 con 0x7fe8e415a010 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8eba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e41f60e0 con 0x7fe8e40ce130 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8e9000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8e9000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51670/0 (socket says 172.21.15.91:51670) 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.934+0000 7fe8e9000640 1 -- 172.21.15.91:0/1429341376 learned_addr learned my addr 172.21.15.91:0/1429341376 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9a00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 0x7fe8e40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 -- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8e40ce130 msgr2=0x7fe8e4150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8e40ce130 0x7fe8e4150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 -- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 msgr2=0x7fe8e40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 0x7fe8e40af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 -- 172.21.15.91:0/1429341376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4009400 con 0x7fe8e40b0700 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9a00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 0x7fe8e40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e9000640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d403aa60 con 0x7fe8e40b0700 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d403abc0 con 0x7fe8e40b0700 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40443f0 con 0x7fe8e40b0700 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.935+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e41f63c0 con 0x7fe8e40b0700 2024-02-02T13:25:33.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.936+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e41f6980 con 0x7fe8e40b0700 2024-02-02T13:25:33.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.937+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e41f6b70 con 0x7fe8e40b0700 2024-02-02T13:25:33.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.937+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe8d404d450 con 0x7fe8e40b0700 2024-02-02T13:25:33.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.938+0000 7fe8e0c00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 0x7fe8bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:33.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.938+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe8d40aab60 con 0x7fe8e40b0700 2024-02-02T13:25:33.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.939+0000 7fe8e3e00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 0x7fe8bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:33.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.939+0000 7fe8e3e00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 0x7fe8bc06c8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40c28d0 tx=0x7fe8cc0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:33.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:33.942+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe8d4076280 con 0x7fe8e40b0700 2024-02-02T13:25:34.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.173+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8e4066cf0 con 0x7fe8e40b0700 2024-02-02T13:25:34.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.174+0000 7fe8e0c00640 1 -- 172.21.15.91:0/1429341376 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fe8d40798a0 con 0x7fe8e40b0700 2024-02-02T13:25:34.176 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:34.176 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:25:34.176 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 msgr2=0x7fe8bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 0x7fe8bc06c8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40c28d0 tx=0x7fe8cc0079b0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 msgr2=0x7fe8e40c2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d40046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 shutdown_connections 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe8bc06a430 0x7fe8bc06c8f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8e40ce130 0x7fe8e4150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e415a010 0x7fe8e40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 --2- 172.21.15.91:0/1429341376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8e40b0700 0x7fe8e40c2390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 >> 172.21.15.91:0/1429341376 conn(0x7fe8e40cec80 msgr2=0x7fe8e415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 shutdown_connections 2024-02-02T13:25:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.177+0000 7fe8eba00640 1 -- 172.21.15.91:0/1429341376 wait complete. 2024-02-02T13:25:34.196 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:34.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.406+0000 7f8769e00640 1 Processor -- start 2024-02-02T13:25:34.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.407+0000 7f8769e00640 1 -- start start 2024-02-02T13:25:34.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8769e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87640b0e10 0x7f87640c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8769e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640cc7c0 con 0x7f876415a010 2024-02-02T13:25:34.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8769e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640ce930 con 0x7f87640ce130 2024-02-02T13:25:34.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8769e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640cd640 con 0x7f87640b0e10 2024-02-02T13:25:34.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8762a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f876415a010 0x7f87640ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46790/0 (socket says 172.21.15.91:46790) 2024-02-02T13:25:34.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8762a00640 1 -- 172.21.15.91:0/3529846582 learned_addr learned my addr 172.21.15.91:0/3529846582 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.408+0000 7f8763400640 1 --2- 172.21.15.91:0/3529846582 >> v2:172.21.15.113:3300/0 conn(0x7f87640b0e10 0x7f87640c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626421155 0 0) 0x7f87640cc7c0 con 0x7f876415a010 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8744003520 con 0x7f876415a010 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230828611 0 0) 0x7f8744003520 con 0x7f876415a010 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 >> v2:172.21.15.113:3300/0 conn(0x7f87640b0e10 msgr2=0x7f87640c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 --2- 172.21.15.91:0/3529846582 >> v2:172.21.15.113:3300/0 conn(0x7f87640b0e10 0x7f87640c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 >> v1:172.21.15.113:6789/0 conn(0x7f87640ce130 legacy=0x7f87640b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876406c890 con 0x7f876415a010 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8754001670 con 0x7f876415a010 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 >> v1:172.21.15.91:6789/0 conn(0x7f876415a010 legacy=0x7f87640ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8744003970 0x7f8744003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8744004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8744005180 0x7f874400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640cd640 con 0x7f87440047c0 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874400c120 con 0x7f8744005180 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874400c2e0 con 0x7f8744003970 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.409+0000 7f8759e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8744004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8762a00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8744003970 0x7f8744003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8763400640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8744005180 0x7f874400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8759e00640 1 -- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8744003970 msgr2=0x7f8744003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8759e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8744003970 0x7f8744003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8759e00640 1 -- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8744005180 msgr2=0x7f874400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8759e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8744005180 0x7f874400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8759e00640 1 -- 172.21.15.91:0/3529846582 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744003520 con 0x7f87440047c0 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8763400640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8744005180 0x7f874400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:34.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.410+0000 7f8763400640 1 -- 172.21.15.91:0/3529846582 reap_dead start 2024-02-02T13:25:34.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.411+0000 7f8759e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8744004c40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f874c001750 tx=0x7f87440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f71a2d9dd2f54fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.411+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c0052d0 con 0x7f87440047c0 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.411+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c02e050 con 0x7f87440047c0 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.411+0000 7f8762000640 1 -- 172.21.15.91:0/3529846582 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c003830 con 0x7f87440047c0 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 -- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 msgr2=0x7f8744004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8744004c40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f874c001750 tx=0x7f87440057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 -- 172.21.15.91:0/3529846582 shutdown_connections 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 --2- 172.21.15.91:0/3529846582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8744004c40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 -- 172.21.15.91:0/3529846582 >> 172.21.15.91:0/3529846582 conn(0x7f87640cec80 msgr2=0x7f87640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 -- 172.21.15.91:0/3529846582 shutdown_connections 2024-02-02T13:25:34.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.412+0000 7f8769e00640 1 -- 172.21.15.91:0/3529846582 wait complete. 2024-02-02T13:25:34.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.414+0000 7f8769e00640 1 Processor -- start 2024-02-02T13:25:34.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.414+0000 7f8769e00640 1 -- start start 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 0x7f87641f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 0x7f87641f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641f3480 con 0x7f87440047c0 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c75e0 con 0x7f876415b010 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8769e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876415b9f0 con 0x7f876415b490 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8762a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8762a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51698/0 (socket says 172.21.15.91:51698) 2024-02-02T13:25:34.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8762a00640 1 -- 172.21.15.91:0/1244441511 learned_addr learned my addr 172.21.15.91:0/1244441511 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8759e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 0x7f87641f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.415+0000 7f8763400640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 0x7f87641f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 -- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 msgr2=0x7f87641f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 0x7f87641f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 -- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 msgr2=0x7f87641f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 0x7f87641f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 -- 172.21.15.91:0/1244441511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876415bcd0 con 0x7f87440047c0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8759e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 0x7f87641f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8763400640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 0x7f87641f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8762a00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8754000ff0 tx=0x7f875400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540072d0 con 0x7f87440047c0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754007430 con 0x7f87440047c0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876415bfb0 con 0x7f87440047c0 2024-02-02T13:25:34.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540047e0 con 0x7f87440047c0 2024-02-02T13:25:34.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.416+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8764156860 con 0x7f87440047c0 2024-02-02T13:25:34.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.417+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640c7350 con 0x7f87440047c0 2024-02-02T13:25:34.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.418+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8754004940 con 0x7f87440047c0 2024-02-02T13:25:34.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.419+0000 7f875be00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 0x7f873c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.419+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8754088410 con 0x7f87440047c0 2024-02-02T13:25:34.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.419+0000 7f8759e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 0x7f873c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.420+0000 7f8759e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 0x7f873c06c920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f874c033000 tx=0x7f874c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.424+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f87540574b0 con 0x7f87440047c0 2024-02-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.613+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8764066cf0 con 0x7f87440047c0 2024-02-02T13:25:34.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.616+0000 7f875be00640 1 -- 172.21.15.91:0/1244441511 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f8754057040 con 0x7f87440047c0 2024-02-02T13:25:34.618 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:34.618 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":22,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:25.481699+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 msgr2=0x7f873c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 0x7f873c06c920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f874c033000 tx=0x7f874c009030 comp rx=0 tx=0).stop 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 msgr2=0x7f8764156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8754000ff0 tx=0x7f875400dc00 comp rx=0 tx=0).stop 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 shutdown_connections 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f873c06a460 0x7f873c06c920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f876415b490 0x7f87641f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f876415b010 0x7f87641f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 --2- 172.21.15.91:0/1244441511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87440047c0 0x7f8764156320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.620+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 >> 172.21.15.91:0/1244441511 conn(0x7f87640cec80 msgr2=0x7f87640b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:34.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.621+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 shutdown_connections 2024-02-02T13:25:34.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.621+0000 7f8769e00640 1 -- 172.21.15.91:0/1244441511 wait complete. 2024-02-02T13:25:34.640 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:25:34.640 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:34.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.846+0000 7fdb71000640 1 Processor -- start 2024-02-02T13:25:34.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.846+0000 7fdb71000640 1 -- start start 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb71000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb6c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb71000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb6c0b0700 0x7fdb6c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb71000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6c0cc7c0 con 0x7fdb6c15a010 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb71000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6c0b10d0 con 0x7fdb6c0b0700 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb71000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0ce930 con 0x7fdb6c0ce130 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb69e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdb6c15a010 0x7fdb6c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46804/0 (socket says 172.21.15.91:46804) 2024-02-02T13:25:34.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.848+0000 7fdb69e00640 1 -- 172.21.15.91:0/3059004922 learned_addr learned my addr 172.21.15.91:0/3059004922 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:34.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb6a800640 1 --2- 172.21.15.91:0/3059004922 >> v2:172.21.15.113:3300/0 conn(0x7fdb6c0b0700 0x7fdb6c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209954704 0 0) 0x7fdb6c0cc7c0 con 0x7fdb6c15a010 2024-02-02T13:25:34.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb6c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c005730 con 0x7fdb6c15a010 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168457476 0 0) 0x7fdb4c005730 con 0x7fdb6c15a010 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 msgr2=0x7fdb6c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb6c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 >> v2:172.21.15.113:3300/0 conn(0x7fdb6c0b0700 msgr2=0x7fdb6c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 --2- 172.21.15.91:0/3059004922 >> v2:172.21.15.113:3300/0 conn(0x7fdb6c0b0700 0x7fdb6c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c0cd640 con 0x7fdb6c15a010 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fdb5c001670 con 0x7fdb6c15a010 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 >> v1:172.21.15.91:6789/0 conn(0x7fdb6c15a010 legacy=0x7fdb6c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb4c005b80 0x7fdb4c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb4c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c006f10 0x7fdb4c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0b10d0 con 0x7fdb4c005b80 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0ce930 con 0x7fdb4c006f10 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00ff90 con 0x7fdb6c0ce130 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.849+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb4c005b80 0x7fdb4c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb6a800640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c006f10 0x7fdb4c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb4c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb69e00640 1 -- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 msgr2=0x7fdb4c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb4c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb69e00640 1 -- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c006f10 msgr2=0x7fdb4c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c006f10 0x7fdb4c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb69e00640 1 -- 172.21.15.91:0/3059004922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c005730 con 0x7fdb4c005b80 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb6a800640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c006f10 0x7fdb4c007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:34.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.850+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb6c0ce130 0x7fdb4c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:34.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.851+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb4c005b80 0x7fdb4c005fa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c110 tx=0x7fdb4c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480b4b849b2114c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.851+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c005ad0 con 0x7fdb4c005b80 2024-02-02T13:25:34.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.851+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c005c50 con 0x7fdb4c005b80 2024-02-02T13:25:34.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.851+0000 7fdb69400640 1 -- 172.21.15.91:0/3059004922 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c02c830 con 0x7fdb4c005b80 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 -- 172.21.15.91:0/3059004922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb4c005b80 msgr2=0x7fdb4c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 --2- 172.21.15.91:0/3059004922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb4c005b80 0x7fdb4c005fa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c110 tx=0x7fdb4c007a00 comp rx=0 tx=0).stop 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb6a800640 1 -- 172.21.15.91:0/3059004922 reap_dead start 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 -- 172.21.15.91:0/3059004922 shutdown_connections 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 -- 172.21.15.91:0/3059004922 >> 172.21.15.91:0/3059004922 conn(0x7fdb6c0cec80 msgr2=0x7fdb6c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:34.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 -- 172.21.15.91:0/3059004922 shutdown_connections 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.852+0000 7fdb71000640 1 -- 172.21.15.91:0/3059004922 wait complete. 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.853+0000 7fdb71000640 1 Processor -- start 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.854+0000 7fdb71000640 1 -- start start 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb4c005b80 0x7fdb6c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb6c0ae6e0 0x7fdb6c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0c54e0 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0b2a30 con 0x7fdb6c0ae6e0 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb71000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0af0b0 con 0x7fdb4c005b80 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51708/0 (socket says 172.21.15.91:51708) 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 -- 172.21.15.91:0/3520069109 learned_addr learned my addr 172.21.15.91:0/3520069109 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 -- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb4c005b80 msgr2=0x7fdb6c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb4c005b80 0x7fdb6c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 -- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb6c0ae6e0 msgr2=0x7fdb6c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb6c0ae6e0 0x7fdb6c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 -- 172.21.15.91:0/3520069109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c0c5200 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.855+0000 7fdb63e00640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e810 tx=0x7fdb5400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.856+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400c670 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.856+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5400c7d0 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.856+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400cb40 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.856+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c0b23e0 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.856+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c0b3400 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.857+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c066cf0 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.858+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdb5400cca0 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.859+0000 7fdb62000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 0x7fdb4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:34.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.859+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdb54088770 con 0x7fdb6c0c4d00 2024-02-02T13:25:34.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.859+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 0x7fdb4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:34.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.860+0000 7fdb69e00640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 0x7fdb4406c8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c034000 tx=0x7fdb5c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:34.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:34.863+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdb54053cd0 con 0x7fdb6c0c4d00 2024-02-02T13:25:35.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.089+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb6c0b2b70 con 0x7fdb6c0c4d00 2024-02-02T13:25:35.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.090+0000 7fdb62000640 1 -- 172.21.15.91:0/3520069109 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fdb540572f0 con 0x7fdb6c0c4d00 2024-02-02T13:25:35.092 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:35.092 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:25:35.093 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 msgr2=0x7fdb4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 0x7fdb4406c8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c034000 tx=0x7fdb5c02e000 comp rx=0 tx=0).stop 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 msgr2=0x7fdb6c0c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e810 tx=0x7fdb5400ecd0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 shutdown_connections 2024-02-02T13:25:35.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb4406a430 0x7fdb4406c8f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb6c0ae6e0 0x7fdb6c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb6c0c4d00 0x7fdb6c0c83f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 --2- 172.21.15.91:0/3520069109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb4c005b80 0x7fdb6c0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.093+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 >> 172.21.15.91:0/3520069109 conn(0x7fdb6c0cec80 msgr2=0x7fdb6c0afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.094+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 shutdown_connections 2024-02-02T13:25:35.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.094+0000 7fdb71000640 1 -- 172.21.15.91:0/3520069109 wait complete. 2024-02-02T13:25:35.114 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:35.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.322+0000 7fdc56c00640 1 Processor -- start 2024-02-02T13:25:35.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.322+0000 7fdc56c00640 1 -- start start 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.323+0000 7fdc56c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc5015d3a0 0x7fdc5006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc56c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc500ce3b0 con 0x7fdc500add80 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc56c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc500ce8e0 con 0x7fdc500c9450 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc56c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc500adbd0 con 0x7fdc5015d3a0 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc54e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdc500add80 0x7fdc500706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46818/0 (socket says 172.21.15.91:46818) 2024-02-02T13:25:35.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc54e00640 1 -- 172.21.15.91:0/4176615230 learned_addr learned my addr 172.21.15.91:0/4176615230 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc55800640 1 --2- 172.21.15.91:0/4176615230 >> v2:172.21.15.113:3300/0 conn(0x7fdc5015d3a0 0x7fdc5006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280937508 0 0) 0x7fdc500ce3b0 con 0x7fdc500add80 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.324+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc30003520 con 0x7fdc500add80 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 455128203 0 0) 0x7fdc30003520 con 0x7fdc500add80 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 >> v2:172.21.15.113:3300/0 conn(0x7fdc5015d3a0 msgr2=0x7fdc5006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:35.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 --2- 172.21.15.91:0/4176615230 >> v2:172.21.15.113:3300/0 conn(0x7fdc5015d3a0 0x7fdc5006b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 >> v1:172.21.15.113:6789/0 conn(0x7fdc500c9450 legacy=0x7fdc500c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50070db0 con 0x7fdc500add80 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fdc38001670 con 0x7fdc500add80 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 >> v1:172.21.15.91:6789/0 conn(0x7fdc500add80 legacy=0x7fdc500706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc30003970 0x7fdc30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 0x7fdc30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc300052a0 0x7fdc3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500adbd0 con 0x7fdc300047c0 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000c170 con 0x7fdc30003970 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000c330 con 0x7fdc300052a0 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.325+0000 7fdc54e00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 0x7fdc30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc54e00640 1 -- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc300052a0 msgr2=0x7fdc3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc56200640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc300052a0 0x7fdc3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc55800640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc30003970 0x7fdc30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc54e00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc300052a0 0x7fdc3000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc54e00640 1 -- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc30003970 msgr2=0x7fdc30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc54e00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc30003970 0x7fdc30003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc54e00640 1 -- 172.21.15.91:0/4176615230 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30003520 con 0x7fdc300047c0 2024-02-02T13:25:35.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.326+0000 7fdc56200640 1 -- 172.21.15.91:0/4176615230 reap_dead start 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc54e00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 0x7fdc30004c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc50071cb0 tx=0x7fdc300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6431a55e5e214f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38005ad0 con 0x7fdc300047c0 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc38005c30 con 0x7fdc300047c0 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc4fe00640 1 -- 172.21.15.91:0/4176615230 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3802ca50 con 0x7fdc300047c0 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc56c00640 1 -- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 msgr2=0x7fdc30004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.327+0000 7fdc56c00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 0x7fdc30004c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc50071cb0 tx=0x7fdc300057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.328+0000 7fdc56c00640 1 -- 172.21.15.91:0/4176615230 shutdown_connections 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.328+0000 7fdc56c00640 1 --2- 172.21.15.91:0/4176615230 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc300047c0 0x7fdc30004c40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.328+0000 7fdc56c00640 1 -- 172.21.15.91:0/4176615230 >> 172.21.15.91:0/4176615230 conn(0x7fdc500cea90 msgr2=0x7fdc500ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.328+0000 7fdc56c00640 1 -- 172.21.15.91:0/4176615230 shutdown_connections 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.328+0000 7fdc56c00640 1 -- 172.21.15.91:0/4176615230 wait complete. 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.329+0000 7fdc56c00640 1 Processor -- start 2024-02-02T13:25:35.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 -- start start 2024-02-02T13:25:35.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 0x7fdc500c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 0x7fdc500c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500b1b10 con 0x7fdc500c7ea0 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500b20d0 con 0x7fdc300047c0 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.330+0000 7fdc56c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500b44e0 con 0x7fdc500b4060 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51726/0 (socket says 172.21.15.91:51726) 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 -- 172.21.15.91:0/3277943781 learned_addr learned my addr 172.21.15.91:0/3277943781 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc55800640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 0x7fdc500c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc54e00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 0x7fdc500c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 -- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 msgr2=0x7fdc500c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 0x7fdc500c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 -- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 msgr2=0x7fdc500c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 0x7fdc500c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 -- 172.21.15.91:0/3277943781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5014c800 con 0x7fdc500c7ea0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc54e00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 0x7fdc500c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc55800640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 0x7fdc500c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.331+0000 7fdc56200640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ca970 tx=0x7fdc4800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.332+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48003d80 con 0x7fdc500c7ea0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.332+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480050f0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.332+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48005460 con 0x7fdc500c7ea0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.332+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5014cae0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.332+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc500c8350 con 0x7fdc500c7ea0 2024-02-02T13:25:35.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.333+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5004dff0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.334+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc480055c0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.335+0000 7fdc4d600640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 0x7fdc2806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.335+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdc480880e0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.335+0000 7fdc55800640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 0x7fdc2806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.336+0000 7fdc55800640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 0x7fdc2806c920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc500c3770 tx=0x7fdc40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.339+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc48053790 con 0x7fdc500c7ea0 2024-02-02T13:25:35.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.556+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc500c8ec0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.561+0000 7fdc4d600640 1 -- 172.21.15.91:0/3277943781 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fdc480015b0 con 0x7fdc500c7ea0 2024-02-02T13:25:35.562 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:35.563 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1911539136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1911539136},{"type":"v1","addr":"172.21.15.113:6835","nonce":1911539136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/487874859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":487874859},{"type":"v1","addr":"172.21.15.91:6836","nonce":487874859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1815710263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1815710263},{"type":"v1","addr":"172.21.15.91:6837","nonce":1815710263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:24:36.734786+0000","modified":"2024-02-02T13:24:40.234617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/1768842703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1768842703},{"type":"v1","addr":"172.21.15.113:6834","nonce":1768842703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T13:25:35.563 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 8 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 msgr2=0x7fdc2806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 0x7fdc2806c920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc500c3770 tx=0x7fdc40008040 comp rx=0 tx=0).stop 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 msgr2=0x7fdc500b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ca970 tx=0x7fdc4800d9f0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 shutdown_connections 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2806a460 0x7fdc2806c920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc500b4060 0x7fdc500c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc500c7ea0 0x7fdc500b2560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 --2- 172.21.15.91:0/3277943781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc300047c0 0x7fdc500c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 >> 172.21.15.91:0/3277943781 conn(0x7fdc500cea90 msgr2=0x7fdc5014dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:35.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 shutdown_connections 2024-02-02T13:25:35.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.563+0000 7fdc56c00640 1 -- 172.21.15.91:0/3277943781 wait complete. 2024-02-02T13:25:35.582 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:35.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.778+0000 7fc8f6e00640 1 Processor -- start 2024-02-02T13:25:35.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.778+0000 7fc8f6e00640 1 -- start start 2024-02-02T13:25:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f00ce130 0x7fc8f00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f6e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc8f00b0700 0x7fc8f00b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f6e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f00cc7c0 con 0x7fc8f015a010 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f6e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f00b10b0 con 0x7fc8f00b0700 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f00ce930 con 0x7fc8f00ce130 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8efe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc8f015a010 0x7fc8f00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46834/0 (socket says 172.21.15.91:46834) 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8efe00640 1 -- 172.21.15.91:0/147950522 learned_addr learned my addr 172.21.15.91:0/147950522 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f4e00640 1 --2- 172.21.15.91:0/147950522 >> v2:172.21.15.91:3300/0 conn(0x7fc8f00b0700 0x7fc8f00b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8ef400640 1 --2- 172.21.15.91:0/147950522 >> v2:172.21.15.113:3300/0 conn(0x7fc8f00ce130 0x7fc8f00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f4e00640 1 -- 172.21.15.91:0/147950522 >> v2:172.21.15.113:3300/0 conn(0x7fc8f00ce130 msgr2=0x7fc8f00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f4e00640 1 --2- 172.21.15.91:0/147950522 >> v2:172.21.15.113:3300/0 conn(0x7fc8f00ce130 0x7fc8f00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f4e00640 1 -- 172.21.15.91:0/147950522 >> v1:172.21.15.91:6789/0 conn(0x7fc8f015a010 legacy=0x7fc8f00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8f4e00640 1 -- 172.21.15.91:0/147950522 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00cd640 con 0x7fc8f00b0700 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.780+0000 7fc8eea00640 1 -- 172.21.15.91:0/147950522 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646532662 0 0) 0x7fc8f00cc7c0 con 0x7fc8f015a010 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.781+0000 7fc8f4e00640 1 --2- 172.21.15.91:0/147950522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00b0700 0x7fc8f00b0b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00b1870 tx=0x7fc8e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d713814026cacb40 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8eea00640 1 -- 172.21.15.91:0/147950522 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e802cc80 con 0x7fc8f00b0700 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8eea00640 1 -- 172.21.15.91:0/147950522 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e802cde0 con 0x7fc8f00b0700 2024-02-02T13:25:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8eea00640 1 -- 172.21.15.91:0/147950522 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8034660 con 0x7fc8f00b0700 2024-02-02T13:25:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 -- 172.21.15.91:0/147950522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00b0700 msgr2=0x7fc8f00b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/147950522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00b0700 0x7fc8f00b0b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00b1870 tx=0x7fc8e802bed0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 -- 172.21.15.91:0/147950522 shutdown_connections 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/147950522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00b0700 0x7fc8f00b0b70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/147950522 >> v2:172.21.15.113:3300/0 conn(0x7fc8f00ce130 0x7fc8f00c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 -- 172.21.15.91:0/147950522 >> 172.21.15.91:0/147950522 conn(0x7fc8f00cec80 msgr2=0x7fc8f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.782+0000 7fc8f6e00640 1 -- 172.21.15.91:0/147950522 shutdown_connections 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.783+0000 7fc8f6e00640 1 -- 172.21.15.91:0/147950522 wait complete. 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.784+0000 7fc8f6e00640 1 Processor -- start 2024-02-02T13:25:35.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.784+0000 7fc8f6e00640 1 -- start start 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f00b0700 0x7fc8f00c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8f015a010 0x7fc8f0145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00b1a30 con 0x7fc8f00ce130 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00b41d0 con 0x7fc8f00b0700 2024-02-02T13:25:35.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8f6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00c8510 con 0x7fc8f015a010 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51752/0 (socket says 172.21.15.91:51752) 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.785+0000 7fc8ef400640 1 -- 172.21.15.91:0/46952946 learned_addr learned my addr 172.21.15.91:0/46952946 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8efe00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f00b0700 0x7fc8f00c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8f4e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8f015a010 0x7fc8f0145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 -- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8f015a010 msgr2=0x7fc8f0145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8f015a010 0x7fc8f0145f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 -- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f00b0700 msgr2=0x7fc8f00c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f00b0700 0x7fc8f00c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 -- 172.21.15.91:0/46952946 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e8009a70 con 0x7fc8f00ce130 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8ef400640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7f0 tx=0x7fc8d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8004220 con 0x7fc8f00ce130 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d80043a0 con 0x7fc8f00ce130 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f00c87f0 con 0x7fc8f00ce130 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8007860 con 0x7fc8f00ce130 2024-02-02T13:25:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.786+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f00c8ad0 con 0x7fc8f00ce130 2024-02-02T13:25:35.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.788+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f00b1ee0 con 0x7fc8f00ce130 2024-02-02T13:25:35.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.788+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc8d8007bd0 con 0x7fc8f00ce130 2024-02-02T13:25:35.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.789+0000 7fc8e7e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 0x7fc8c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:35.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.789+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc8d8088970 con 0x7fc8f00ce130 2024-02-02T13:25:35.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.789+0000 7fc8efe00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 0x7fc8c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:35.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.790+0000 7fc8efe00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 0x7fc8c806c8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000f60 tx=0x7fc8e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:35.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.794+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8f00b1ee0 con 0x7fc8f00ce130 2024-02-02T13:25:35.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.985+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8f00b1ee0 con 0x7fc8f00ce130 2024-02-02T13:25:35.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.987+0000 7fc8e7e00640 1 -- 172.21.15.91:0/46952946 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fc8f00b1ee0 con 0x7fc8f00ce130 2024-02-02T13:25:35.989 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:35.989 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":22,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:25.481699+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:24:34.438237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T13:24:35.744074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.989+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 msgr2=0x7fc8c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.989+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 0x7fc8c806c8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000f60 tx=0x7fc8e0009040 comp rx=0 tx=0).stop 2024-02-02T13:25:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.989+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 msgr2=0x7fc8f0070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.989+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7f0 tx=0x7fc8d800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 shutdown_connections 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8c806a430 0x7fc8c806c8f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8f015a010 0x7fc8f0145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f00ce130 0x7fc8f0070af0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 --2- 172.21.15.91:0/46952946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f00b0700 0x7fc8f00c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 >> 172.21.15.91:0/46952946 conn(0x7fc8f00cec80 msgr2=0x7fc8f006fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 shutdown_connections 2024-02-02T13:25:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:35.990+0000 7fc8f6e00640 1 -- 172.21.15.91:0/46952946 wait complete. 2024-02-02T13:25:36.010 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:25:36.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.219+0000 7f7952200640 1 Processor -- start 2024-02-02T13:25:36.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.219+0000 7f7952200640 1 -- start start 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f794c1609b0 0x7f794c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c0ce530 0x7f794c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f794c0707b0 0x7f794c0d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c070c40 con 0x7f794c0707b0 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c160d90 con 0x7f794c1609b0 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f7952200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c070fe0 con 0x7f794c0ce530 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f794be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f794c0707b0 0x7f794c0d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f794be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f794c0707b0 0x7f794c0d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51764/0 (socket says 172.21.15.91:51764) 2024-02-02T13:25:36.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.221+0000 7f794be00640 1 -- 172.21.15.91:0/1944435828 learned_addr learned my addr 172.21.15.91:0/1944435828 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794aa00640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c0ce530 0x7f794c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794b400640 1 --2- 172.21.15.91:0/1944435828 >> v2:172.21.15.113:3300/0 conn(0x7f794c1609b0 0x7f794c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794be00640 1 -- 172.21.15.91:0/1944435828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c0ce530 msgr2=0x7f794c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794be00640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c0ce530 0x7f794c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794be00640 1 -- 172.21.15.91:0/1944435828 >> v2:172.21.15.113:3300/0 conn(0x7f794c1609b0 msgr2=0x7f794c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:36.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794be00640 1 --2- 172.21.15.91:0/1944435828 >> v2:172.21.15.113:3300/0 conn(0x7f794c1609b0 0x7f794c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.222+0000 7f794be00640 1 -- 172.21.15.91:0/1944435828 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0d2940 con 0x7f794c0707b0 2024-02-02T13:25:36.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.223+0000 7f794be00640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0707b0 0x7f794c0d0bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f794c0d4120 tx=0x7f794402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ea68b347589732 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.223+0000 7f794a000640 1 -- 172.21.15.91:0/1944435828 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944031750 con 0x7f794c0707b0 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.223+0000 7f794a000640 1 -- 172.21.15.91:0/1944435828 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79440318b0 con 0x7f794c0707b0 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.223+0000 7f794a000640 1 -- 172.21.15.91:0/1944435828 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944031c40 con 0x7f794c0707b0 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 -- 172.21.15.91:0/1944435828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0707b0 msgr2=0x7f794c0d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0707b0 0x7f794c0d0bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f794c0d4120 tx=0x7f794402b7f0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 -- 172.21.15.91:0/1944435828 shutdown_connections 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0707b0 0x7f794c0d0bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 --2- 172.21.15.91:0/1944435828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c0ce530 0x7f794c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 --2- 172.21.15.91:0/1944435828 >> v2:172.21.15.113:3300/0 conn(0x7f794c1609b0 0x7f794c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 -- 172.21.15.91:0/1944435828 >> 172.21.15.91:0/1944435828 conn(0x7f794c0d3020 msgr2=0x7f794c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 -- 172.21.15.91:0/1944435828 shutdown_connections 2024-02-02T13:25:36.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.224+0000 7f7952200640 1 -- 172.21.15.91:0/1944435828 wait complete. 2024-02-02T13:25:36.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.226+0000 7f7952200640 1 Processor -- start 2024-02-02T13:25:36.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.226+0000 7f7952200640 1 -- start start 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f794c0707b0 0x7f794c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 0x7f794c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0cb750 con 0x7f794c0ce530 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0caad0 con 0x7f794c0707b0 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f7952200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0cadb0 con 0x7f794c1609b0 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f794aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f794aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51776/0 (socket says 172.21.15.91:51776) 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f794aa00640 1 -- 172.21.15.91:0/1740540059 learned_addr learned my addr 172.21.15.91:0/1740540059 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.227+0000 7f794b400640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f794c0707b0 0x7f794c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794be00640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 0x7f794c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 -- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 msgr2=0x7f794c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 0x7f794c15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 -- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f794c0707b0 msgr2=0x7f794c0cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f794c0707b0 0x7f794c0cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 -- 172.21.15.91:0/1740540059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440094d0 con 0x7f794c0ce530 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794be00640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 0x7f794c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f794aa00640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f793c007e30 tx=0x7f793c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c014620 con 0x7f794c0ce530 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c00ee90 con 0x7f794c0ce530 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0149c0 con 0x7f794c0ce530 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c0cb090 con 0x7f794c0ce530 2024-02-02T13:25:36.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.228+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c0cc0b0 con 0x7f794c0ce530 2024-02-02T13:25:36.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.230+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c0cb890 con 0x7f794c0ce530 2024-02-02T13:25:36.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.230+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f793c014b20 con 0x7f794c0ce530 2024-02-02T13:25:36.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.231+0000 7f7943400640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 0x7f792406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.231+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f793c089380 con 0x7f794c0ce530 2024-02-02T13:25:36.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.231+0000 7f794b400640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 0x7f792406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.232+0000 7f794b400640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 0x7f792406c920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7934006fd0 tx=0x7f7934008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.236+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f794c0cb890 con 0x7f794c0ce530 2024-02-02T13:25:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.463+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f794c0cb890 con 0x7f794c0ce530 2024-02-02T13:25:36.466 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:25:36.465+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:25:36.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.480+0000 7f7943400640 1 -- 172.21.15.91:0/1740540059 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f794c0cb890 con 0x7f794c0ce530 2024-02-02T13:25:36.481 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:25:36.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 msgr2=0x7f792406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 0x7f792406c920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7934006fd0 tx=0x7f7934008040 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 msgr2=0x7f794c0ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f793c007e30 tx=0x7f793c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 shutdown_connections 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f792406a460 0x7f792406c920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f794c1609b0 0x7f794c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f794c0ce530 0x7f794c0ca0f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 --2- 172.21.15.91:0/1740540059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f794c0707b0 0x7f794c0cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 >> 172.21.15.91:0/1740540059 conn(0x7f794c0d3020 msgr2=0x7f794c0c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:36.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 shutdown_connections 2024-02-02T13:25:36.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.483+0000 7f7952200640 1 -- 172.21.15.91:0/1740540059 wait complete. 2024-02-02T13:25:36.502 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:25:36.516 INFO:tasks.ceph.mds.d.smithi113.stderr: -7410> 2024-02-02T13:24:42.598+0000 7f0378c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:25:36.604 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:25:36.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.705+0000 7f29b6200640 1 Processor -- start 2024-02-02T13:25:36.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.705+0000 7f29b6200640 1 -- start start 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b6200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f29b015e110 0x7f29b00cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b6200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00ad5c0 con 0x7f29b00acfe0 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b6200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00cae90 con 0x7f29b00cc600 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b6200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00ad8a0 con 0x7f29b015e110 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b4e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f29b015e110 0x7f29b00cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b5800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f29b00acfe0 0x7f29b00c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46836/0 (socket says 172.21.15.91:46836) 2024-02-02T13:25:36.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.707+0000 7f29b5800640 1 -- 172.21.15.91:0/44583602 learned_addr learned my addr 172.21.15.91:0/44583602 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:36.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.708+0000 7f29b4e00640 1 -- 172.21.15.91:0/44583602 >> v1:172.21.15.113:6789/0 conn(0x7f29b00cc600 legacy=0x7f29b00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:36.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.708+0000 7f29b4e00640 1 -- 172.21.15.91:0/44583602 >> v1:172.21.15.91:6789/0 conn(0x7f29b00acfe0 legacy=0x7f29b00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.708+0000 7f29b4e00640 1 -- 172.21.15.91:0/44583602 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00cb170 con 0x7f29b015e110 2024-02-02T13:25:36.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.709+0000 7f29b4e00640 1 --2- 172.21.15.91:0/44583602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00cb970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009d90 tx=0x7f29a002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605b71e0a5d09fa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.709+0000 7f29af400640 1 -- 172.21.15.91:0/44583602 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a002cbd0 con 0x7f29b015e110 2024-02-02T13:25:36.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.709+0000 7f29af400640 1 -- 172.21.15.91:0/44583602 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a002cd30 con 0x7f29b015e110 2024-02-02T13:25:36.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.709+0000 7f29af400640 1 -- 172.21.15.91:0/44583602 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a002cea0 con 0x7f29b015e110 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 -- 172.21.15.91:0/44583602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 msgr2=0x7f29b00cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 --2- 172.21.15.91:0/44583602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00cb970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009d90 tx=0x7f29a002be20 comp rx=0 tx=0).stop 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 -- 172.21.15.91:0/44583602 shutdown_connections 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 --2- 172.21.15.91:0/44583602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00cb970 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 -- 172.21.15.91:0/44583602 >> 172.21.15.91:0/44583602 conn(0x7f29b00b0560 msgr2=0x7f29b00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 -- 172.21.15.91:0/44583602 shutdown_connections 2024-02-02T13:25:36.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.710+0000 7f29b6200640 1 -- 172.21.15.91:0/44583602 wait complete. 2024-02-02T13:25:36.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.712+0000 7f29b6200640 1 Processor -- start 2024-02-02T13:25:36.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.712+0000 7f29b6200640 1 -- start start 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00acfe0 0x7f29b00c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b00cc600 0x7f29b0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015adc0 con 0x7f29b015e110 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00af530 con 0x7f29b00cc600 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b6200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00af810 con 0x7f29b00acfe0 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51794/0 (socket says 172.21.15.91:51794) 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 -- 172.21.15.91:0/950507367 learned_addr learned my addr 172.21.15.91:0/950507367 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:36.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 -- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00acfe0 msgr2=0x7f29b00c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00acfe0 0x7f29b00c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 -- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b00cc600 msgr2=0x7f29b0155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:36.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b00cc600 0x7f29b0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:36.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.713+0000 7f29b5800640 1 -- 172.21.15.91:0/950507367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009a70 con 0x7f29b015e110 2024-02-02T13:25:36.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29b5800640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c420 tx=0x7f29a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005ce0 con 0x7f29b015e110 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8005e40 con 0x7f29b015e110 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80072d0 con 0x7f29b015e110 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b00afaf0 con 0x7f29b015e110 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.714+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b015a4d0 con 0x7f29b015e110 2024-02-02T13:25:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.715+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b004dff0 con 0x7f29b015e110 2024-02-02T13:25:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.716+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f29a8007430 con 0x7f29b015e110 2024-02-02T13:25:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.717+0000 7f29acc00640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 0x7f298806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.717+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f29a8087e40 con 0x7f29b015e110 2024-02-02T13:25:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.717+0000 7f29b4e00640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 0x7f298806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:36.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.718+0000 7f29b4e00640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 0x7f298806c920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29a0036000 tx=0x7f29a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:36.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.721+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f29a8053400 con 0x7f29b015e110 2024-02-02T13:25:36.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:36.935+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f29b015eae0 con 0x7f29b015e110 2024-02-02T13:25:37.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.588+0000 7f29acc00640 1 -- 172.21.15.91:0/950507367 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29a8056a20 con 0x7f29b015e110 2024-02-02T13:25:37.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 msgr2=0x7f298806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 0x7f298806c920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29a0036000 tx=0x7f29a002e000 comp rx=0 tx=0).stop 2024-02-02T13:25:37.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 msgr2=0x7f29b00704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c420 tx=0x7f29a800c8e0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 shutdown_connections 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298806a460 0x7f298806c920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b015e110 0x7f29b00704e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b00cc600 0x7f29b0155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 --2- 172.21.15.91:0/950507367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00acfe0 0x7f29b00c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.591+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 >> 172.21.15.91:0/950507367 conn(0x7f29b00b0560 msgr2=0x7f29b0073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.592+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 shutdown_connections 2024-02-02T13:25:37.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.592+0000 7f29b6200640 1 -- 172.21.15.91:0/950507367 wait complete. 2024-02-02T13:25:37.615 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:25:37.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.828+0000 7f366c000640 1 Processor -- start 2024-02-02T13:25:37.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.829+0000 7f366c000640 1 -- start start 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.830+0000 7f366c000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3664151f90 0x7f366415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.830+0000 7f366c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 0x7f36640b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.830+0000 7f366c000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3664070f80 con 0x7f36640b0c40 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.830+0000 7f366c000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640b10d0 con 0x7f3664151f90 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.830+0000 7f366c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3664152370 con 0x7f366406f4f0 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3664151f90 0x7f366415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3664151f90 0x7f366415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51798/0 (socket says 172.21.15.91:51798) 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 -- 172.21.15.91:0/3124409480 learned_addr learned my addr 172.21.15.91:0/3124409480 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f366a000640 1 --1- 172.21.15.91:0/3124409480 >> v1:172.21.15.91:6789/0 conn(0x7f36640b0c40 0x7f36640b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46848/0 (socket says 172.21.15.91:46848) 2024-02-02T13:25:37.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 -- 172.21.15.91:0/3124409480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 msgr2=0x7f36640b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3668c00640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 0x7f36640b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 0x7f36640b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 -- 172.21.15.91:0/3124409480 >> v1:172.21.15.91:6789/0 conn(0x7f36640b0c40 legacy=0x7f36640b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3669600640 1 -- 172.21.15.91:0/3124409480 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640cb010 con 0x7f3664151f90 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3663e00640 1 -- 172.21.15.91:0/3124409480 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123387798 0 0) 0x7f3664070f80 con 0x7f36640b0c40 2024-02-02T13:25:37.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.831+0000 7f3668c00640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 0x7f36640b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:37.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.832+0000 7f3669600640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3664151f90 0x7f366415f5a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3654009870 tx=0x7f365402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b1a3fbf4471cb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f3663e00640 1 -- 172.21.15.91:0/3124409480 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654030840 con 0x7f3664151f90 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f3663e00640 1 -- 172.21.15.91:0/3124409480 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540309a0 con 0x7f3664151f90 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f3663e00640 1 -- 172.21.15.91:0/3124409480 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654030d30 con 0x7f3664151f90 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 -- 172.21.15.91:0/3124409480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3664151f90 msgr2=0x7f366415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3664151f90 0x7f366415f5a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3654009870 tx=0x7f365402b980 comp rx=0 tx=0).stop 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 -- 172.21.15.91:0/3124409480 shutdown_connections 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366406f4f0 0x7f36640b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 --2- 172.21.15.91:0/3124409480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3664151f90 0x7f366415f5a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 -- 172.21.15.91:0/3124409480 >> 172.21.15.91:0/3124409480 conn(0x7f36640cec80 msgr2=0x7f36640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.833+0000 7f366c000640 1 -- 172.21.15.91:0/3124409480 shutdown_connections 2024-02-02T13:25:37.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.834+0000 7f366c000640 1 -- 172.21.15.91:0/3124409480 wait complete. 2024-02-02T13:25:37.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.835+0000 7f366c000640 1 Processor -- start 2024-02-02T13:25:37.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.835+0000 7f366c000640 1 -- start start 2024-02-02T13:25:37.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36640b0c40 0x7f36641f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3664151f90 0x7f36640cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640c84a0 con 0x7f366406f4f0 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641f2560 con 0x7f36640b0c40 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f366c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640c7820 con 0x7f3664151f90 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f3669600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f3669600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51806/0 (socket says 172.21.15.91:51806) 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.836+0000 7f3669600640 1 -- 172.21.15.91:0/1535302781 learned_addr learned my addr 172.21.15.91:0/1535302781 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 -- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3664151f90 msgr2=0x7f36640cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3664151f90 0x7f36640cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 -- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36640b0c40 msgr2=0x7f36641f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36640b0c40 0x7f36641f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:37.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 -- 172.21.15.91:0/1535302781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540094d0 con 0x7f366406f4f0 2024-02-02T13:25:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3669600640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3654000c00 tx=0x7f3654004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654030810 con 0x7f366406f4f0 2024-02-02T13:25:37.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654043400 con 0x7f366406f4f0 2024-02-02T13:25:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540437d0 con 0x7f366406f4f0 2024-02-02T13:25:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.837+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640c7b00 con 0x7f366406f4f0 2024-02-02T13:25:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.838+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640c80c0 con 0x7f366406f4f0 2024-02-02T13:25:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.838+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36640c85e0 con 0x7f366406f4f0 2024-02-02T13:25:37.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.840+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3654043930 con 0x7f366406f4f0 2024-02-02T13:25:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.840+0000 7f3661600640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 0x7f363c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.841+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f365403e070 con 0x7f366406f4f0 2024-02-02T13:25:37.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.841+0000 7f3668c00640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 0x7f363c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:37.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.841+0000 7f3668c00640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 0x7f363c06c940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3664158060 tx=0x7f364c008330 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:37.845+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f36640c85e0 con 0x7f366406f4f0 2024-02-02T13:25:38.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.033+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36640c85e0 con 0x7f366406f4f0 2024-02-02T13:25:38.603 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:25:38.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.601+0000 7f3661600640 1 -- 172.21.15.91:0/1535302781 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f36640c85e0 con 0x7f366406f4f0 2024-02-02T13:25:38.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.605+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 msgr2=0x7f363c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.605+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 0x7f363c06c940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3664158060 tx=0x7f364c008330 comp rx=0 tx=0).stop 2024-02-02T13:25:38.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.605+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 msgr2=0x7f36640c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.605+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3654000c00 tx=0x7f3654004730 comp rx=0 tx=0).stop 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.605+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 shutdown_connections 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f363c06a480 0x7f363c06c940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3664151f90 0x7f36640cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36640b0c40 0x7f36641f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 --2- 172.21.15.91:0/1535302781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f366406f4f0 0x7f36640c7190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 >> 172.21.15.91:0/1535302781 conn(0x7f36640cec80 msgr2=0x7f36640cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 shutdown_connections 2024-02-02T13:25:38.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.606+0000 7f366c000640 1 -- 172.21.15.91:0/1535302781 wait complete. 2024-02-02T13:25:38.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:25:38.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.828+0000 7f8468400640 1 Processor -- start 2024-02-02T13:25:38.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.829+0000 7f8468400640 1 -- start start 2024-02-02T13:25:38.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8468400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f846015a010 0x7f84600c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8468400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f84600ce130 0x7f84600ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8468400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84600cc7c0 con 0x7f84600cd470 2024-02-02T13:25:38.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8468400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84600ce930 con 0x7f84600ce130 2024-02-02T13:25:38.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8468400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846006c890 con 0x7f846015a010 2024-02-02T13:25:38.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.830+0000 7f8465000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f84600ce130 0x7f84600ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:38.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8466400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f84600cd470 0x7f84600b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46860/0 (socket says 172.21.15.91:46860) 2024-02-02T13:25:38.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8466400640 1 -- 172.21.15.91:0/1023374254 learned_addr learned my addr 172.21.15.91:0/1023374254 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:38.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8465a00640 1 --2- 172.21.15.91:0/1023374254 >> v2:172.21.15.113:3300/0 conn(0x7f846015a010 0x7f84600c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8465000640 1 -- 172.21.15.91:0/1023374254 >> v2:172.21.15.113:3300/0 conn(0x7f846015a010 msgr2=0x7f84600c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8465000640 1 --2- 172.21.15.91:0/1023374254 >> v2:172.21.15.113:3300/0 conn(0x7f846015a010 0x7f84600c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8465000640 1 -- 172.21.15.91:0/1023374254 >> v1:172.21.15.91:6789/0 conn(0x7f84600cd470 legacy=0x7f84600b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f8465000640 1 -- 172.21.15.91:0/1023374254 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846006cb70 con 0x7f84600ce130 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.831+0000 7f845fe00640 1 -- 172.21.15.91:0/1023374254 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620180013 0 0) 0x7f84600cc7c0 con 0x7f84600cd470 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.832+0000 7f8465000640 1 --2- 172.21.15.91:0/1023374254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84600ce130 0x7f84600ccf30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84600cfaa0 tx=0x7f844802da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aec9f641dfeb02 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:38.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.833+0000 7f845fe00640 1 -- 172.21.15.91:0/1023374254 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448031870 con 0x7f84600ce130 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.833+0000 7f845fe00640 1 -- 172.21.15.91:0/1023374254 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84480319d0 con 0x7f84600ce130 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.833+0000 7f845fe00640 1 -- 172.21.15.91:0/1023374254 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448031d60 con 0x7f84600ce130 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.833+0000 7f8468400640 1 -- 172.21.15.91:0/1023374254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84600ce130 msgr2=0x7f84600ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.833+0000 7f8468400640 1 --2- 172.21.15.91:0/1023374254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84600ce130 0x7f84600ccf30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84600cfaa0 tx=0x7f844802da50 comp rx=0 tx=0).stop 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 -- 172.21.15.91:0/1023374254 shutdown_connections 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 --2- 172.21.15.91:0/1023374254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84600ce130 0x7f84600ccf30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 --2- 172.21.15.91:0/1023374254 >> v2:172.21.15.113:3300/0 conn(0x7f846015a010 0x7f84600c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 -- 172.21.15.91:0/1023374254 >> 172.21.15.91:0/1023374254 conn(0x7f84600cec80 msgr2=0x7f84600cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 -- 172.21.15.91:0/1023374254 shutdown_connections 2024-02-02T13:25:38.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.834+0000 7f8468400640 1 -- 172.21.15.91:0/1023374254 wait complete. 2024-02-02T13:25:38.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.835+0000 7f8468400640 1 Processor -- start 2024-02-02T13:25:38.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.835+0000 7f8468400640 1 -- start start 2024-02-02T13:25:38.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84600ce130 0x7f84600c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8460158240 0x7f84600b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84600c8f40 con 0x7f846015a010 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84600b2e00 con 0x7f8460158240 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.836+0000 7f8468400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84601586f0 con 0x7f84600ce130 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51816/0 (socket says 172.21.15.91:51816) 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 -- 172.21.15.91:0/3501546224 learned_addr learned my addr 172.21.15.91:0/3501546224 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 -- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84600ce130 msgr2=0x7f84600c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84600ce130 0x7f84600c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 -- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8460158240 msgr2=0x7f84600b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8460158240 0x7f84600b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 -- 172.21.15.91:0/3501546224 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448009400 con 0x7f846015a010 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.837+0000 7f8465000640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8448009890 tx=0x7f84480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.838+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844803d040 con 0x7f846015a010 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.838+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8448039bc0 con 0x7f846015a010 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.838+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448004030 con 0x7f846015a010 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.838+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84600c8980 con 0x7f846015a010 2024-02-02T13:25:38.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.838+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84600aca70 con 0x7f846015a010 2024-02-02T13:25:38.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.839+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84600ac8d0 con 0x7f846015a010 2024-02-02T13:25:38.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.840+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8448004190 con 0x7f846015a010 2024-02-02T13:25:38.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.841+0000 7f845d600640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 0x7f843806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:38.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.841+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f84480ab4d0 con 0x7f846015a010 2024-02-02T13:25:38.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.841+0000 7f8465a00640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 0x7f843806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:38.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.842+0000 7f8465a00640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 0x7f843806c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8450006fd0 tx=0x7f8450008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:38.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:38.845+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f84600ac8d0 con 0x7f846015a010 2024-02-02T13:25:39.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.035+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84600ac8d0 con 0x7f846015a010 2024-02-02T13:25:39.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.601+0000 7f845d600640 1 -- 172.21.15.91:0/3501546224 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f84600ac8d0 con 0x7f846015a010 2024-02-02T13:25:39.603 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 msgr2=0x7f843806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 0x7f843806c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8450006fd0 tx=0x7f8450008040 comp rx=0 tx=0).stop 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 msgr2=0x7f84600b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8448009890 tx=0x7f84480046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 shutdown_connections 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f843806a430 0x7f843806c8f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8460158240 0x7f84600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f846015a010 0x7f84600b28c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 --2- 172.21.15.91:0/3501546224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84600ce130 0x7f84600c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 >> 172.21.15.91:0/3501546224 conn(0x7f84600cec80 msgr2=0x7f84600b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:39.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 shutdown_connections 2024-02-02T13:25:39.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.605+0000 7f8468400640 1 -- 172.21.15.91:0/3501546224 wait complete. 2024-02-02T13:25:39.627 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:25:39.628 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:25:39.628 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:25:39.628 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:25:39.628 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:25:39.628 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:25:39.628 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:25:39.629 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:25:39.629 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:25:39.629 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:25:39.629 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:25:39.629 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:25:39.629 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:25:39.629 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:25:39.629 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:25:39.630 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:25:39.646 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:25:39.646 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:25:39.648 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:25:39.648 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:25:39.649 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:25:39.649 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:25:39.656 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:25:39.656 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:25:39.695 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:25:39.695 DEBUG:teuthology.parallel:result is None 2024-02-02T13:25:39.700 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:25:39.700 DEBUG:teuthology.parallel:result is None 2024-02-02T13:25:39.703 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:25:39.703 DEBUG:teuthology.parallel:result is None 2024-02-02T13:25:39.709 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:25:39.710 DEBUG:teuthology.parallel:result is None 2024-02-02T13:25:39.710 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:25:39.850 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:25:39.850 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:25:39.858 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:25:39.858 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:25:39.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.899+0000 7f6605000640 1 Processor -- start 2024-02-02T13:25:39.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.899+0000 7f6605000640 1 -- start start 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.900+0000 7f6605000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 0x7f66000cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.900+0000 7f6605000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f66000b8bb0 0x7f66000cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.900+0000 7f6605000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000cde20 con 0x7f660015d0d0 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.900+0000 7f6605000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66000d2a20 con 0x7f66000b8750 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.900+0000 7f6605000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660015d4b0 con 0x7f66000b8bb0 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 0x7f66000cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 0x7f66000cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39654/0 (socket says 172.21.15.91:39654) 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65f7e00640 1 -- 172.21.15.91:0/4106071252 learned_addr learned my addr 172.21.15.91:0/4106071252 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ffe00640 1 --2- 172.21.15.91:0/4106071252 >> v2:172.21.15.113:3300/0 conn(0x7f66000b8bb0 0x7f66000cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ff400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f660015d0d0 0x7f66000cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:35780/0 (socket says 172.21.15.91:35780) 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ffe00640 1 -- 172.21.15.91:0/4106071252 >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 msgr2=0x7f66000cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ffe00640 1 --2- 172.21.15.91:0/4106071252 >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 0x7f66000cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ffe00640 1 -- 172.21.15.91:0/4106071252 >> v1:172.21.15.113:6789/0 conn(0x7f660015d0d0 legacy=0x7f66000cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.901+0000 7f65ffe00640 1 -- 172.21.15.91:0/4106071252 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000d23f0 con 0x7f66000b8bb0 2024-02-02T13:25:39.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.902+0000 7f65ffe00640 1 --2- 172.21.15.91:0/4106071252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66000b8bb0 0x7f66000cfd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f660006a070 tx=0x7f65f802bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116f99c1a5f0c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:39.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f65fea00640 1 -- 172.21.15.91:0/4106071252 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f802ca30 con 0x7f66000b8bb0 2024-02-02T13:25:39.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f65fea00640 1 -- 172.21.15.91:0/4106071252 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f802cb90 con 0x7f66000b8bb0 2024-02-02T13:25:39.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f65fea00640 1 -- 172.21.15.91:0/4106071252 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8034440 con 0x7f66000b8bb0 2024-02-02T13:25:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 -- 172.21.15.91:0/4106071252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66000b8bb0 msgr2=0x7f66000cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 --2- 172.21.15.91:0/4106071252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66000b8bb0 0x7f66000cfd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f660006a070 tx=0x7f65f802bd00 comp rx=0 tx=0).stop 2024-02-02T13:25:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 -- 172.21.15.91:0/4106071252 shutdown_connections 2024-02-02T13:25:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 --2- 172.21.15.91:0/4106071252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66000b8bb0 0x7f66000cfd50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 --2- 172.21.15.91:0/4106071252 >> v2:172.21.15.91:3300/0 conn(0x7f66000b8750 0x7f66000cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 -- 172.21.15.91:0/4106071252 >> 172.21.15.91:0/4106071252 conn(0x7f6600065910 msgr2=0x7f6600065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 -- 172.21.15.91:0/4106071252 shutdown_connections 2024-02-02T13:25:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.903+0000 7f6605000640 1 -- 172.21.15.91:0/4106071252 wait complete. 2024-02-02T13:25:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.905+0000 7f6605000640 1 Processor -- start 2024-02-02T13:25:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.905+0000 7f6605000640 1 -- start start 2024-02-02T13:25:39.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66000b8750 0x7f66000cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 0x7f66000c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660015eeb0 con 0x7f660015f340 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c6fe0 con 0x7f660015d0d0 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f6605000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c6d00 con 0x7f66000b8750 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f65ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f65ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39658/0 (socket says 172.21.15.91:39658) 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f65ffe00640 1 -- 172.21.15.91:0/2793451417 learned_addr learned my addr 172.21.15.91:0/2793451417 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f65f7e00640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 0x7f66000c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.906+0000 7f65ff400640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66000b8750 0x7f66000cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 -- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66000b8750 msgr2=0x7f66000cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66000b8750 0x7f66000cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 -- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 msgr2=0x7f66000c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:39.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 0x7f66000c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:39.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 -- 172.21.15.91:0/2793451417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8009a70 con 0x7f660015f340 2024-02-02T13:25:39.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65f7e00640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 0x7f66000c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:25:39.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65ffe00640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e80 tx=0x7f65f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:39.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8040650 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80407b0 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.907+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8040b60 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.908+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660015e860 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.908+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000c9c20 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.909+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660004dff0 con 0x7f660015f340 2024-02-02T13:25:39.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.910+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f65f8040cc0 con 0x7f660015f340 2024-02-02T13:25:39.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.911+0000 7f65f7400640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 0x7f65d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:39.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.911+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65f80ab2b0 con 0x7f660015f340 2024-02-02T13:25:39.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.911+0000 7f65ff400640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 0x7f65d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:39.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.912+0000 7f65ff400640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 0x7f65d806c8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65f0000f60 tx=0x7f65f0007e00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:39.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:39.915+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f65f8076c30 con 0x7f660015f340 2024-02-02T13:25:40.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.111+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f660015eff0 con 0x7f660015f340 2024-02-02T13:25:40.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.899+0000 7f65f7400640 1 -- 172.21.15.91:0/2793451417 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f65f8009450 con 0x7f660015f340 2024-02-02T13:25:40.901 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:25:40.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 msgr2=0x7f65d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 0x7f65d806c8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65f0000f60 tx=0x7f65f0007e00 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 msgr2=0x7f66000cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e80 tx=0x7f65f8004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 shutdown_connections 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65d806a430 0x7f65d806c8f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f660015f340 0x7f66000cae60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f660015d0d0 0x7f66000c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 --2- 172.21.15.91:0/2793451417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66000b8750 0x7f66000cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 >> 172.21.15.91:0/2793451417 conn(0x7f6600065910 msgr2=0x7f66000b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 shutdown_connections 2024-02-02T13:25:40.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:40.903+0000 7f6605000640 1 -- 172.21.15.91:0/2793451417 wait complete. 2024-02-02T13:25:40.923 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:25:41.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.125+0000 7facaea00640 1 Processor -- start 2024-02-02T13:25:41.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.125+0000 7facaea00640 1 -- start start 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.126+0000 7facaea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 0x7faca806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faca80cb580 0x7faca80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca806d910 con 0x7faca815ad10 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca806d730 con 0x7faca80cb580 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca80cbf50 con 0x7faca815b0f0 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faca80cb580 0x7faca80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faca80cb580 0x7faca80cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39670/0 (socket says 172.21.15.91:39670) 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 -- 172.21.15.91:0/3534063487 learned_addr learned my addr 172.21.15.91:0/3534063487 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7faca7400640 1 --2- 172.21.15.91:0/3534063487 >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 0x7faca806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 -- 172.21.15.91:0/3534063487 >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 msgr2=0x7faca806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 --2- 172.21.15.91:0/3534063487 >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 0x7faca806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 -- 172.21.15.91:0/3534063487 >> v1:172.21.15.113:6789/0 conn(0x7faca815ad10 legacy=0x7faca806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7facaca00640 1 -- 172.21.15.91:0/3534063487 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80cc850 con 0x7faca80cb580 2024-02-02T13:25:41.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.127+0000 7faca7400640 1 --2- 172.21.15.91:0/3534063487 >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 0x7faca806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:41.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7facaca00640 1 --2- 172.21.15.91:0/3534063487 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca80cba10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faca0009e50 tx=0x7faca002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad98b321520852 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7faca6a00640 1 -- 172.21.15.91:0/3534063487 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca002c9f0 con 0x7faca80cb580 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7faca6a00640 1 -- 172.21.15.91:0/3534063487 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca002cb50 con 0x7faca80cb580 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7faca6a00640 1 -- 172.21.15.91:0/3534063487 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00343f0 con 0x7faca80cb580 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7facaea00640 1 -- 172.21.15.91:0/3534063487 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 msgr2=0x7faca80cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7facaea00640 1 --2- 172.21.15.91:0/3534063487 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca80cba10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faca0009e50 tx=0x7faca002be50 comp rx=0 tx=0).stop 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.129+0000 7facaea00640 1 -- 172.21.15.91:0/3534063487 shutdown_connections 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.130+0000 7facaea00640 1 --2- 172.21.15.91:0/3534063487 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca80cba10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.130+0000 7facaea00640 1 --2- 172.21.15.91:0/3534063487 >> v2:172.21.15.113:3300/0 conn(0x7faca815b0f0 0x7faca806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.130+0000 7facaea00640 1 -- 172.21.15.91:0/3534063487 >> 172.21.15.91:0/3534063487 conn(0x7faca80b0700 msgr2=0x7faca80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.130+0000 7facaea00640 1 -- 172.21.15.91:0/3534063487 shutdown_connections 2024-02-02T13:25:41.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.130+0000 7facaea00640 1 -- 172.21.15.91:0/3534063487 wait complete. 2024-02-02T13:25:41.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.131+0000 7facaea00640 1 Processor -- start 2024-02-02T13:25:41.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.131+0000 7facaea00640 1 -- start start 2024-02-02T13:25:41.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faca815ad10 0x7faca80c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faca815b0f0 0x7faca80c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca814bcc0 con 0x7faca80cb580 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca814b7c0 con 0x7faca815ad10 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.132+0000 7facaea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca80afa80 con 0x7faca815b0f0 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39680/0 (socket says 172.21.15.91:39680) 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 -- 172.21.15.91:0/1471184688 learned_addr learned my addr 172.21.15.91:0/1471184688 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 -- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faca815b0f0 msgr2=0x7faca80c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faca815b0f0 0x7faca80c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 -- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faca815ad10 msgr2=0x7faca80c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faca815ad10 0x7faca80c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 -- 172.21.15.91:0/1471184688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009b30 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.133+0000 7faca7e00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c440 tx=0x7fac9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.134+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98005480 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.134+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac980035f0 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.134+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980039c0 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.134+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca80afd60 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.134+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca80b0320 con 0x7faca80cb580 2024-02-02T13:25:41.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.135+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8066cf0 con 0x7faca80cb580 2024-02-02T13:25:41.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.136+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fac98003b20 con 0x7faca80cb580 2024-02-02T13:25:41.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.137+0000 7fac9fe00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 0x7fac8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.137+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fac98087680 con 0x7faca80cb580 2024-02-02T13:25:41.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.137+0000 7faca7400640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 0x7fac8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.137+0000 7faca7400640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 0x7fac8006c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac90006fd0 tx=0x7fac90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.141+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fac98052fc0 con 0x7faca80cb580 2024-02-02T13:25:41.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.396+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faca80c5060 con 0x7faca80cb580 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.397+0000 7fac9fe00640 1 -- 172.21.15.91:0/1471184688 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fac980565e0 con 0x7faca80cb580 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:25:41.399 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:25:41.401 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.402 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.403 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:25:41.404 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.406 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-mds", 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxltXALDxAAcmTAuUXdLQOt7u63huIbJQ==", 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-mds" 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-mgr", 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxlxXsLDxAAl5NPsWKLM0jE2+I7GLvTGg==", 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-mgr" 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.412 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-osd", 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxlPIQLDxAAYQCCbtObS6Q8fOzzQJ9seg==", 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-osd" 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-rbd", 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxltowLDxAAor7ZONd2boADzZXMX0S2OQ==", 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-rbd" 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.413 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxlrZULDxAAAUaO6fjzXiQwKeEKRuCtxw==", 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.bootstrap-rgw", 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQAD7bxlTp4LDxAAZpns73li/QF4gyK9m3I24w==", 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile bootstrap-rgw" 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:25:41.414 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:41.415 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:25:41.416 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:25:41.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 msgr2=0x7fac8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 0x7fac8006c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac90006fd0 tx=0x7fac90008040 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 msgr2=0x7faca814b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c440 tx=0x7fac9800c900 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 shutdown_connections 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac8006a430 0x7fac8006c8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faca815b0f0 0x7faca80c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faca815ad10 0x7faca80c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 --2- 172.21.15.91:0/1471184688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faca80cb580 0x7faca814b280 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 >> 172.21.15.91:0/1471184688 conn(0x7faca80b0700 msgr2=0x7faca815d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 shutdown_connections 2024-02-02T13:25:41.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.400+0000 7facaea00640 1 -- 172.21.15.91:0/1471184688 wait complete. 2024-02-02T13:25:41.418 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-02-02T13:25:41.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.619+0000 7fb46f000640 1 Processor -- start 2024-02-02T13:25:41.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.619+0000 7fb46f000640 1 -- start start 2024-02-02T13:25:41.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.620+0000 7fb46f000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb46815d3a0 0x7fb4680ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.620+0000 7fb46f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4680ac960 0x7fb4680acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.620+0000 7fb46f000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680c94a0 con 0x7fb4680cb580 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.620+0000 7fb46f000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680ad6e0 con 0x7fb46815d3a0 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.620+0000 7fb46f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680c96f0 con 0x7fb4680ac960 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb46815d3a0 0x7fb4680ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb46815d3a0 0x7fb4680ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39692/0 (socket says 172.21.15.91:39692) 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 -- 172.21.15.91:0/1309304757 learned_addr learned my addr 172.21.15.91:0/1309304757 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 -- 172.21.15.91:0/1309304757 >> v2:172.21.15.113:3300/0 conn(0x7fb4680ac960 msgr2=0x7fb4680acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 --2- 172.21.15.91:0/1309304757 >> v2:172.21.15.113:3300/0 conn(0x7fb4680ac960 0x7fb4680acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 -- 172.21.15.91:0/1309304757 >> v1:172.21.15.113:6789/0 conn(0x7fb4680cb580 legacy=0x7fb4680cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:41.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.621+0000 7fb467400640 1 -- 172.21.15.91:0/1309304757 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680c99d0 con 0x7fb46815d3a0 2024-02-02T13:25:41.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.622+0000 7fb467400640 1 --2- 172.21.15.91:0/1309304757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb4680ca1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb468071a30 tx=0x7fb45002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35dbdcf513dd840 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.622+0000 7fb466a00640 1 -- 172.21.15.91:0/1309304757 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450031870 con 0x7fb46815d3a0 2024-02-02T13:25:41.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.622+0000 7fb466a00640 1 -- 172.21.15.91:0/1309304757 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4500319d0 con 0x7fb46815d3a0 2024-02-02T13:25:41.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.622+0000 7fb466a00640 1 -- 172.21.15.91:0/1309304757 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450031d60 con 0x7fb46815d3a0 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 -- 172.21.15.91:0/1309304757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 msgr2=0x7fb4680ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 --2- 172.21.15.91:0/1309304757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb4680ca1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb468071a30 tx=0x7fb45002b910 comp rx=0 tx=0).stop 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 -- 172.21.15.91:0/1309304757 shutdown_connections 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 --2- 172.21.15.91:0/1309304757 >> v2:172.21.15.113:3300/0 conn(0x7fb4680ac960 0x7fb4680acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 --2- 172.21.15.91:0/1309304757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb4680ca1d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 -- 172.21.15.91:0/1309304757 >> 172.21.15.91:0/1309304757 conn(0x7fb4680b0700 msgr2=0x7fb4680b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:41.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 -- 172.21.15.91:0/1309304757 shutdown_connections 2024-02-02T13:25:41.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.623+0000 7fb46f000640 1 -- 172.21.15.91:0/1309304757 wait complete. 2024-02-02T13:25:41.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.625+0000 7fb46f000640 1 Processor -- start 2024-02-02T13:25:41.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.625+0000 7fb46f000640 1 -- start start 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4680ac960 0x7fb46814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 0x7fb46815f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b1c60 con 0x7fb46815d3a0 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b1f40 con 0x7fb4680ac960 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46f000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b2220 con 0x7fb4680cb580 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.626+0000 7fb46d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39700/0 (socket says 172.21.15.91:39700) 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 -- 172.21.15.91:0/2070231774 learned_addr learned my addr 172.21.15.91:0/2070231774 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:41.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb467e00640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4680ac960 0x7fb46814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb467400640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 0x7fb46815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 -- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 msgr2=0x7fb46815f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 0x7fb46815f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 -- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4680ac960 msgr2=0x7fb46814f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4680ac960 0x7fb46814f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 -- 172.21.15.91:0/2070231774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500094d0 con 0x7fb46815d3a0 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb467400640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 0x7fb46815f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:41.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.627+0000 7fb46d000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb460009fd0 tx=0x7fb46000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.630+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460011830 con 0x7fb46815d3a0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.630+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4680ad4b0 con 0x7fb46815d3a0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.631+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4680b27e0 con 0x7fb46815d3a0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.631+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460011990 con 0x7fb46815d3a0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.631+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460011d00 con 0x7fb46815d3a0 2024-02-02T13:25:41.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.632+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4680b2a20 con 0x7fb46815d3a0 2024-02-02T13:25:41.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.633+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb46001b460 con 0x7fb46815d3a0 2024-02-02T13:25:41.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.634+0000 7fb45fe00640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 0x7fb44006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:41.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.634+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb460088400 con 0x7fb46815d3a0 2024-02-02T13:25:41.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.634+0000 7fb467e00640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 0x7fb44006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:41.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.634+0000 7fb467e00640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 0x7fb44006c920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb458001db0 tx=0x7fb458009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:41.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.638+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb4680b2a20 con 0x7fb46815d3a0 2024-02-02T13:25:41.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.891+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fb4680b2a20 con 0x7fb46815d3a0 2024-02-02T13:25:41.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.900+0000 7fb45fe00640 1 -- 172.21.15.91:0/2070231774 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb4680b2a20 con 0x7fb46815d3a0 2024-02-02T13:25:41.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 msgr2=0x7fb44006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 0x7fb44006c920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb458001db0 tx=0x7fb458009040 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 msgr2=0x7fb468159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb460009fd0 tx=0x7fb46000b8e0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 shutdown_connections 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.903+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb44006a460 0x7fb44006c920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4680cb580 0x7fb46815f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb46815d3a0 0x7fb468159340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 --2- 172.21.15.91:0/2070231774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4680ac960 0x7fb46814f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 >> 172.21.15.91:0/2070231774 conn(0x7fb4680b0700 msgr2=0x7fb4680cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 shutdown_connections 2024-02-02T13:25:41.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:41.904+0000 7fb46f000640 1 -- 172.21.15.91:0/2070231774 wait complete. 2024-02-02T13:25:41.923 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-02-02T13:25:42.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.125+0000 7f4ff9600640 1 Processor -- start 2024-02-02T13:25:42.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.125+0000 7f4ff9600640 1 -- start start 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ff4070270 0x7f4ff40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff9600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff4160d90 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff4070bf0 con 0x7f4ff41609b0 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff4070fe0 con 0x7f4ff4070270 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff3400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4ff40b6bd0 0x7f4ff40b7020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37340/0 (socket says 172.21.15.91:37340) 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.127+0000 7f4ff3400640 1 -- 172.21.15.91:0/1495755232 learned_addr learned my addr 172.21.15.91:0/1495755232 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:42.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff2000640 1 --2- 172.21.15.91:0/1495755232 >> v2:172.21.15.113:3300/0 conn(0x7f4ff4070270 0x7f4ff40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378371834 0 0) 0x7f4fec001110 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd4003520 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2695514001 0 0) 0x7f4fd4003520 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 >> v2:172.21.15.113:3300/0 conn(0x7f4ff4070270 msgr2=0x7f4ff40706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 --2- 172.21.15.91:0/1495755232 >> v2:172.21.15.113:3300/0 conn(0x7f4ff4070270 0x7f4ff40706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 >> v1:172.21.15.113:6789/0 conn(0x7f4ff41609b0 legacy=0x7f4ff40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40d2940 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f4fec001800 con 0x7f4ff40b6bd0 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 >> v1:172.21.15.91:6789/0 conn(0x7f4ff40b6bd0 legacy=0x7f4ff40b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 0x7f4fd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd40052a0 0x7f4fd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4070fe0 con 0x7f4fd40047c0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd400c1a0 con 0x7f4fd40039c0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.128+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd400c360 con 0x7f4fd40052a0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 0x7f4fd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2a00640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 -- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd40052a0 msgr2=0x7f4fd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd40052a0 0x7f4fd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 -- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40039c0 msgr2=0x7f4fd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff2000640 1 -- 172.21.15.91:0/1495755232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4003520 con 0x7f4fd40047c0 2024-02-02T13:25:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.129+0000 7f4ff3400640 1 -- 172.21.15.91:0/1495755232 reap_dead start 2024-02-02T13:25:42.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.130+0000 7f4ff2000640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 0x7f4fd4004c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fc0 tx=0x7f4fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bc11ad196c10c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.130+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02cc20 con 0x7f4fd40047c0 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.130+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc00f040 con 0x7f4fd40047c0 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.130+0000 7f4ff1600640 1 -- 172.21.15.91:0/1495755232 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc034440 con 0x7f4fd40047c0 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 -- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 msgr2=0x7f4fd4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 0x7f4fd4004c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fc0 tx=0x7f4fd40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 -- 172.21.15.91:0/1495755232 shutdown_connections 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 --2- 172.21.15.91:0/1495755232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fd40047c0 0x7f4fd4004c40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 -- 172.21.15.91:0/1495755232 >> 172.21.15.91:0/1495755232 conn(0x7f4ff40d3020 msgr2=0x7f4ff40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 -- 172.21.15.91:0/1495755232 shutdown_connections 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.131+0000 7f4ff9600640 1 -- 172.21.15.91:0/1495755232 wait complete. 2024-02-02T13:25:42.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.133+0000 7f4ff9600640 1 Processor -- start 2024-02-02T13:25:42.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.133+0000 7f4ff9600640 1 -- start start 2024-02-02T13:25:42.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40047c0 0x7f4ff40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ff415a390 0x7f4ff415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4159660 con 0x7f4ff415f400 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40cc7c0 con 0x7f4fd40047c0 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415f960 con 0x7f4ff415a390 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39718/0 (socket says 172.21.15.91:39718) 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 -- 172.21.15.91:0/3101424774 learned_addr learned my addr 172.21.15.91:0/3101424774 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 -- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ff415a390 msgr2=0x7f4ff415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ff415a390 0x7f4ff415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 -- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40047c0 msgr2=0x7f4ff40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40047c0 0x7f4ff40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.134+0000 7f4ff3400640 1 -- 172.21.15.91:0/3101424774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff41598b0 con 0x7f4ff415f400 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4ff3400640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d3d70 tx=0x7f4fec00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0074e0 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec007660 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec004a20 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff4159b90 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.135+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff40b14e0 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.136+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff415a0b0 con 0x7f4ff415f400 2024-02-02T13:25:42.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.137+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4fec004b80 con 0x7f4ff415f400 2024-02-02T13:25:42.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.138+0000 7f4feaa00640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 0x7f4fcc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.138+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4fec087690 con 0x7f4ff415f400 2024-02-02T13:25:42.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.138+0000 7f4ff2a00640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 0x7f4fcc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.139+0000 7f4ff2a00640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 0x7f4fcc06c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40683b0 tx=0x7f4fe4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.142+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4ff415a0b0 con 0x7f4ff415f400 2024-02-02T13:25:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.401+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4ff4160d90 con 0x7f4ff415f400 2024-02-02T13:25:42.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.409+0000 7f4feaa00640 1 -- 172.21.15.91:0/3101424774 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4ff4160d90 con 0x7f4ff415f400 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 msgr2=0x7f4fcc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 0x7f4fcc06c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40683b0 tx=0x7f4fe4009040 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 msgr2=0x7f4ff41e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d3d70 tx=0x7f4fec00c870 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 shutdown_connections 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4fcc06a430 0x7f4fcc06c8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ff415a390 0x7f4ff415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ff415f400 0x7f4ff41e7270 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 --2- 172.21.15.91:0/3101424774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd40047c0 0x7f4ff40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 >> 172.21.15.91:0/3101424774 conn(0x7f4ff40d3020 msgr2=0x7f4ff40d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 shutdown_connections 2024-02-02T13:25:42.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.412+0000 7f4ff9600640 1 -- 172.21.15.91:0/3101424774 wait complete. 2024-02-02T13:25:42.430 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-02-02T13:25:42.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.601+0000 7fd046600640 1 Processor -- start 2024-02-02T13:25:42.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.601+0000 7fd046600640 1 -- start start 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.602+0000 7fd046600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 0x7fd0400cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.602+0000 7fd046600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd040070980 0x7fd0400b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.602+0000 7fd046600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd040160d90 con 0x7fd0400ce530 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.602+0000 7fd046600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd040070fe0 con 0x7fd040070980 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.602+0000 7fd046600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0400d2940 con 0x7fd0401609b0 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd040070980 0x7fd0400b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd0400ce530 0x7fd040070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37352/0 (socket says 172.21.15.91:37352) 2024-02-02T13:25:42.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03ea00640 1 -- 172.21.15.91:0/548969804 learned_addr learned my addr 172.21.15.91:0/548969804 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:42.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03f400640 1 --2- 172.21.15.91:0/548969804 >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 0x7fd0400cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03fe00640 1 -- 172.21.15.91:0/548969804 >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 msgr2=0x7fd0400cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:42.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03fe00640 1 --2- 172.21.15.91:0/548969804 >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 0x7fd0400cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03fe00640 1 -- 172.21.15.91:0/548969804 >> v1:172.21.15.91:6789/0 conn(0x7fd0400ce530 legacy=0x7fd040070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03fe00640 1 -- 172.21.15.91:0/548969804 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400d2c20 con 0x7fd040070980 2024-02-02T13:25:42.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03e000640 1 -- 172.21.15.91:0/548969804 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464934839 0 0) 0x7fd028001110 con 0x7fd0400ce530 2024-02-02T13:25:42.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.603+0000 7fd03f400640 1 --2- 172.21.15.91:0/548969804 >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 0x7fd0400cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:42.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.604+0000 7fd03fe00640 1 --2- 172.21.15.91:0/548969804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd040070980 0x7fd0400b6bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d4120 tx=0x7fd03802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1c443af5164b9e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.605+0000 7fd03e000640 1 -- 172.21.15.91:0/548969804 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03802cb60 con 0x7fd040070980 2024-02-02T13:25:42.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.605+0000 7fd03e000640 1 -- 172.21.15.91:0/548969804 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03802ccc0 con 0x7fd040070980 2024-02-02T13:25:42.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.605+0000 7fd03e000640 1 -- 172.21.15.91:0/548969804 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038034660 con 0x7fd040070980 2024-02-02T13:25:42.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.605+0000 7fd046600640 1 -- 172.21.15.91:0/548969804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd040070980 msgr2=0x7fd0400b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.605+0000 7fd046600640 1 --2- 172.21.15.91:0/548969804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd040070980 0x7fd0400b6bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d4120 tx=0x7fd03802bdb0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 -- 172.21.15.91:0/548969804 shutdown_connections 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 --2- 172.21.15.91:0/548969804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd040070980 0x7fd0400b6bd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 --2- 172.21.15.91:0/548969804 >> v2:172.21.15.113:3300/0 conn(0x7fd0401609b0 0x7fd0400cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 -- 172.21.15.91:0/548969804 >> 172.21.15.91:0/548969804 conn(0x7fd0400d3020 msgr2=0x7fd0400d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 -- 172.21.15.91:0/548969804 shutdown_connections 2024-02-02T13:25:42.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.606+0000 7fd046600640 1 -- 172.21.15.91:0/548969804 wait complete. 2024-02-02T13:25:42.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.607+0000 7fd046600640 1 Processor -- start 2024-02-02T13:25:42.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.607+0000 7fd046600640 1 -- start start 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd040070980 0x7fd0400ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0401609b0 0x7fd040143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040156020 con 0x7fd0400ce530 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040156300 con 0x7fd0401609b0 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.608+0000 7fd046600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401565e0 con 0x7fd040070980 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39742/0 (socket says 172.21.15.91:39742) 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 -- 172.21.15.91:0/1726351492 learned_addr learned my addr 172.21.15.91:0/1726351492 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:42.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 -- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd040070980 msgr2=0x7fd0400ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd040070980 0x7fd0400ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 -- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0401609b0 msgr2=0x7fd040143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0401609b0 0x7fd040143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 -- 172.21.15.91:0/1726351492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009a70 con 0x7fd0400ce530 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd03ea00640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c780 tx=0x7fd02800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280072d0 con 0x7fd0400ce530 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028007430 con 0x7fd0400ce530 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028001d10 con 0x7fd0400ce530 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.609+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0401568c0 con 0x7fd0400ce530 2024-02-02T13:25:42.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.610+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd040155c40 con 0x7fd0400ce530 2024-02-02T13:25:42.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.610+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd040156ae0 con 0x7fd0400ce530 2024-02-02T13:25:42.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.611+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd0280047e0 con 0x7fd0400ce530 2024-02-02T13:25:42.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.612+0000 7fd037400640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 0x7fd01806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:42.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.612+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd028087630 con 0x7fd0400ce530 2024-02-02T13:25:42.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.613+0000 7fd03f400640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 0x7fd01806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:42.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.613+0000 7fd03f400640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 0x7fd01806c8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd030006fd0 tx=0x7fd030008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:42.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.619+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd028052f80 con 0x7fd0400ce530 2024-02-02T13:25:42.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.869+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd040160d90 con 0x7fd0400ce530 2024-02-02T13:25:42.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.877+0000 7fd037400640 1 -- 172.21.15.91:0/1726351492 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd040160d90 con 0x7fd0400ce530 2024-02-02T13:25:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.879+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 msgr2=0x7fd01806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.879+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 0x7fd01806c8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd030006fd0 tx=0x7fd030008040 comp rx=0 tx=0).stop 2024-02-02T13:25:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.879+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 msgr2=0x7fd040142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.879+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c780 tx=0x7fd02800cc40 comp rx=0 tx=0).stop 2024-02-02T13:25:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 shutdown_connections 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd01806a430 0x7fd01806c8f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0401609b0 0x7fd040143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0400ce530 0x7fd040142c40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 --2- 172.21.15.91:0/1726351492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd040070980 0x7fd0400ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 >> 172.21.15.91:0/1726351492 conn(0x7fd0400d3020 msgr2=0x7fd04015fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 shutdown_connections 2024-02-02T13:25:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:42.880+0000 7fd046600640 1 -- 172.21.15.91:0/1726351492 wait complete. 2024-02-02T13:25:42.899 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-02-02T13:25:43.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.096+0000 7f6e46000640 1 Processor -- start 2024-02-02T13:25:43.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.096+0000 7f6e46000640 1 -- start start 2024-02-02T13:25:43.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e46000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 0x7f6e4006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e46000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e400cb580 0x7f6e400cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e46000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4006d910 con 0x7f6e4015ad10 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e46000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4006d730 con 0x7f6e400cb580 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e46000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400cbf50 con 0x7f6e4015b0f0 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e3fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e400cb580 0x7f6e400cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e3fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e400cb580 0x7f6e400cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39756/0 (socket says 172.21.15.91:39756) 2024-02-02T13:25:43.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.097+0000 7f6e3fe00640 1 -- 172.21.15.91:0/2489804555 learned_addr learned my addr 172.21.15.91:0/2489804555 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e36800640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 0x7f6e4006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e3fe00640 1 -- 172.21.15.91:0/2489804555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 msgr2=0x7f6e4006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e3fe00640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 0x7f6e4006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e3fe00640 1 -- 172.21.15.91:0/2489804555 >> v1:172.21.15.113:6789/0 conn(0x7f6e4015ad10 legacy=0x7f6e4006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e3fe00640 1 -- 172.21.15.91:0/2489804555 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400cc850 con 0x7f6e400cb580 2024-02-02T13:25:43.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.098+0000 7f6e36800640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 0x7f6e4006b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:43.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e3fe00640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e400cb580 0x7f6e400cba10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009e50 tx=0x7f6e3802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410357dd810afb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e3ea00640 1 -- 172.21.15.91:0/2489804555 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3802ca80 con 0x7f6e400cb580 2024-02-02T13:25:43.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e3ea00640 1 -- 172.21.15.91:0/2489804555 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3802cbe0 con 0x7f6e400cb580 2024-02-02T13:25:43.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e3ea00640 1 -- 172.21.15.91:0/2489804555 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380343f0 con 0x7f6e400cb580 2024-02-02T13:25:43.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e46000640 1 -- 172.21.15.91:0/2489804555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e400cb580 msgr2=0x7f6e400cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.099+0000 7f6e46000640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e400cb580 0x7f6e400cba10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009e50 tx=0x7f6e3802bee0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 -- 172.21.15.91:0/2489804555 shutdown_connections 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e400cb580 0x7f6e400cba10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 --2- 172.21.15.91:0/2489804555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e4015b0f0 0x7f6e4006b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 -- 172.21.15.91:0/2489804555 >> 172.21.15.91:0/2489804555 conn(0x7f6e400b0700 msgr2=0x7f6e400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 -- 172.21.15.91:0/2489804555 shutdown_connections 2024-02-02T13:25:43.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.100+0000 7f6e46000640 1 -- 172.21.15.91:0/2489804555 wait complete. 2024-02-02T13:25:43.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.101+0000 7f6e46000640 1 Processor -- start 2024-02-02T13:25:43.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.101+0000 7f6e46000640 1 -- start start 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e400cb580 0x7f6e400c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4015ad10 0x7f6e400af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400af7f0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400c6260 con 0x7f6e4015ad10 2024-02-02T13:25:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.102+0000 7f6e46000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400c6540 con 0x7f6e400cb580 2024-02-02T13:25:43.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39762/0 (socket says 172.21.15.91:39762) 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 -- 172.21.15.91:0/1538930516 learned_addr learned my addr 172.21.15.91:0/1538930516 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3f400640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e400cb580 0x7f6e400c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e36800640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4015ad10 0x7f6e400af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 -- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e400cb580 msgr2=0x7f6e400c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e400cb580 0x7f6e400c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 -- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4015ad10 msgr2=0x7f6e400af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4015ad10 0x7f6e400af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.103+0000 7f6e3fe00640 1 -- 172.21.15.91:0/1538930516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009b30 con 0x7f6e4015b0f0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e3fe00640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036040 tx=0x7f6e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3803e450 con 0x7f6e4015b0f0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3803e5d0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3803e980 con 0x7f6e4015b0f0 2024-02-02T13:25:43.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400c6820 con 0x7f6e4015b0f0 2024-02-02T13:25:43.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.104+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e400c6de0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.105+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4006ddf0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.106+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6e3802cd50 con 0x7f6e4015b0f0 2024-02-02T13:25:43.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.107+0000 7f6e37e00640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 0x7f6e1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.107+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6e380ab190 con 0x7f6e4015b0f0 2024-02-02T13:25:43.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.108+0000 7f6e3f400640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 0x7f6e1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.108+0000 7f6e3f400640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 0x7f6e1806c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e30004f60 tx=0x7f6e30007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.112+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6e38076ac0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.347+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f6e40066cf0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.356+0000 7f6e37e00640 1 -- 172.21.15.91:0/1538930516 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6e3807a0e0 con 0x7f6e4015b0f0 2024-02-02T13:25:43.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 msgr2=0x7f6e1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 0x7f6e1806c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e30004f60 tx=0x7f6e30007e30 comp rx=0 tx=0).stop 2024-02-02T13:25:43.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 msgr2=0x7f6e40070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036040 tx=0x7f6e38004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 shutdown_connections 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e1806a430 0x7f6e1806c8f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e4015b0f0 0x7f6e40070780 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4015ad10 0x7f6e400af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 --2- 172.21.15.91:0/1538930516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e400cb580 0x7f6e400c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 >> 172.21.15.91:0/1538930516 conn(0x7f6e400b0700 msgr2=0x7f6e4015d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 shutdown_connections 2024-02-02T13:25:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.359+0000 7f6e46000640 1 -- 172.21.15.91:0/1538930516 wait complete. 2024-02-02T13:25:43.379 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-02-02T13:25:43.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.576+0000 7fabe4000640 1 Processor -- start 2024-02-02T13:25:43.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.576+0000 7fabe4000640 1 -- start start 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 0x7fabdc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fabdc1610b0 0x7fabdc161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0cea30 con 0x7fabdc1610b0 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0cded0 con 0x7fabdc0ce070 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe4000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0d12c0 con 0x7fabdc15d650 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe2000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fabdc1610b0 0x7fabdc161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe2000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fabdc1610b0 0x7fabdc161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39778/0 (socket says 172.21.15.91:39778) 2024-02-02T13:25:43.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.578+0000 7fabe2000640 1 -- 172.21.15.91:0/4275336884 learned_addr learned my addr 172.21.15.91:0/4275336884 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe0c00640 1 --2- 172.21.15.91:0/4275336884 >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 0x7fabdc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe2000640 1 -- 172.21.15.91:0/4275336884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 msgr2=0x7fabdc0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe1600640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe2000640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe2000640 1 -- 172.21.15.91:0/4275336884 >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 msgr2=0x7fabdc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe2000640 1 --2- 172.21.15.91:0/4275336884 >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 0x7fabdc0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe2000640 1 -- 172.21.15.91:0/4275336884 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc071410 con 0x7fabdc1610b0 2024-02-02T13:25:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.579+0000 7fabe0c00640 1 --2- 172.21.15.91:0/4275336884 >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 0x7fabdc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.580+0000 7fabe2000640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc1610b0 0x7fabdc161540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b5c20 tx=0x7fabd402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b70de61d0a12ce server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.580+0000 7fabdbe00640 1 -- 172.21.15.91:0/4275336884 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4031860 con 0x7fabdc1610b0 2024-02-02T13:25:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.580+0000 7fabdbe00640 1 -- 172.21.15.91:0/4275336884 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40319c0 con 0x7fabdc1610b0 2024-02-02T13:25:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.580+0000 7fabdbe00640 1 -- 172.21.15.91:0/4275336884 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4031d50 con 0x7fabdc1610b0 2024-02-02T13:25:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 -- 172.21.15.91:0/4275336884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc1610b0 msgr2=0x7fabdc161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc1610b0 0x7fabdc161540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b5c20 tx=0x7fabd402b900 comp rx=0 tx=0).stop 2024-02-02T13:25:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 -- 172.21.15.91:0/4275336884 shutdown_connections 2024-02-02T13:25:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc1610b0 0x7fabdc161540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 --2- 172.21.15.91:0/4275336884 >> v2:172.21.15.113:3300/0 conn(0x7fabdc0ce070 0x7fabdc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 --2- 172.21.15.91:0/4275336884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 -- 172.21.15.91:0/4275336884 >> 172.21.15.91:0/4275336884 conn(0x7fabdc0d30a0 msgr2=0x7fabdc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 -- 172.21.15.91:0/4275336884 shutdown_connections 2024-02-02T13:25:43.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.581+0000 7fabe4000640 1 -- 172.21.15.91:0/4275336884 wait complete. 2024-02-02T13:25:43.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.583+0000 7fabe4000640 1 Processor -- start 2024-02-02T13:25:43.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.583+0000 7fabe4000640 1 -- start start 2024-02-02T13:25:43.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc1f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabdc1610b0 0x7fabdc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1f4140 con 0x7fabdc0ce070 2024-02-02T13:25:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0b3570 con 0x7fabdc1610b0 2024-02-02T13:25:43.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe4000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc160b80 con 0x7fabdc15d650 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39784/0 (socket says 172.21.15.91:39784) 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.584+0000 7fabe1600640 1 -- 172.21.15.91:0/3135999225 learned_addr learned my addr 172.21.15.91:0/3135999225 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 -- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 msgr2=0x7fabdc1f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc1f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 -- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabdc1610b0 msgr2=0x7fabdc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabdc1610b0 0x7fabdc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 -- 172.21.15.91:0/3135999225 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40094d0 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.585+0000 7fabe1600640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00ccd0 tx=0x7fabcc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.586+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003990 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.586+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc003af0 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.586+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc004020 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.586+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc1f0b10 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.586+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc0b2c40 con 0x7fabdc0ce070 2024-02-02T13:25:43.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.587+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc071a30 con 0x7fabdc0ce070 2024-02-02T13:25:43.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.588+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fabcc004180 con 0x7fabdc0ce070 2024-02-02T13:25:43.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.589+0000 7fabd9600640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 0x7fabb406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:43.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.589+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabcc088940 con 0x7fabdc0ce070 2024-02-02T13:25:43.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.589+0000 7fabe0c00640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 0x7fabb406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:43.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.589+0000 7fabe0c00640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 0x7fabb406c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabc4006b10 tx=0x7fabc4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:43.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.593+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fabcc054310 con 0x7fabdc0ce070 2024-02-02T13:25:43.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.851+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fabdc066cf0 con 0x7fabdc0ce070 2024-02-02T13:25:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.855+0000 7fabd9600640 1 -- 172.21.15.91:0/3135999225 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fabcc01d070 con 0x7fabdc0ce070 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 msgr2=0x7fabb406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 0x7fabb406c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabc4006b10 tx=0x7fabc4008040 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 msgr2=0x7fabdc1f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00ccd0 tx=0x7fabcc00a5c0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 shutdown_connections 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fabb406a430 0x7fabb406c8f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabdc15d650 0x7fabdc1f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabdc1610b0 0x7fabdc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 --2- 172.21.15.91:0/3135999225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabdc0ce070 0x7fabdc1f3c00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 >> 172.21.15.91:0/3135999225 conn(0x7fabdc0d30a0 msgr2=0x7fabdc0c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 shutdown_connections 2024-02-02T13:25:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:43.858+0000 7fabe4000640 1 -- 172.21.15.91:0/3135999225 wait complete. 2024-02-02T13:25:43.878 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-02-02T13:25:44.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.078+0000 7f0b0fe00640 1 Processor -- start 2024-02-02T13:25:44.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.079+0000 7f0b0fe00640 1 -- start start 2024-02-02T13:25:44.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.080+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.080+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 0x7f0b080b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.080+0000 7f0b0fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b08160d90 con 0x7f0b081609b0 2024-02-02T13:25:44.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.080+0000 7f0b0fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b08070bf0 con 0x7f0b080b6bd0 2024-02-02T13:25:44.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.080+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08070fe0 con 0x7f0b08070270 2024-02-02T13:25:44.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 0x7f0b080b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 0x7f0b080b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:42290/0 (socket says 172.21.15.91:42290) 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0de00640 1 -- 172.21.15.91:0/2510776818 learned_addr learned my addr 172.21.15.91:0/2510776818 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2510776818 >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 msgr2=0x7f0b080b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0ca00640 1 --2- 172.21.15.91:0/2510776818 >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 0x7f0b080b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2510776818 >> v1:172.21.15.113:6789/0 conn(0x7f0b081609b0 legacy=0x7f0b080cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.081+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2510776818 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080d2940 con 0x7f0b08070270 2024-02-02T13:25:44.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.083+0000 7f0b0ca00640 1 --2- 172.21.15.91:0/2510776818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0af00097a0 tx=0x7f0af002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5aed5e07bff828a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.083+0000 7f0b07e00640 1 -- 172.21.15.91:0/2510776818 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af002f050 con 0x7f0b08070270 2024-02-02T13:25:44.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.083+0000 7f0b07e00640 1 -- 172.21.15.91:0/2510776818 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0031700 con 0x7f0b08070270 2024-02-02T13:25:44.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.083+0000 7f0b07e00640 1 -- 172.21.15.91:0/2510776818 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0031a90 con 0x7f0b08070270 2024-02-02T13:25:44.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2510776818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 msgr2=0x7f0b080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2510776818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0af00097a0 tx=0x7f0af002b7a0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2510776818 shutdown_connections 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2510776818 >> v2:172.21.15.113:3300/0 conn(0x7f0b080b6bd0 0x7f0b080b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2510776818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080706b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2510776818 >> 172.21.15.91:0/2510776818 conn(0x7f0b080d3020 msgr2=0x7f0b080d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2510776818 shutdown_connections 2024-02-02T13:25:44.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.084+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2510776818 wait complete. 2024-02-02T13:25:44.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.085+0000 7f0b0fe00640 1 Processor -- start 2024-02-02T13:25:44.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 -- start start 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b081609b0 0x7f0b080b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0814a1f0 con 0x7f0b080b6bd0 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080b18e0 con 0x7f0b081609b0 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.086+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080c94e0 con 0x7f0b08070270 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39786/0 (socket says 172.21.15.91:39786) 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2605008826 learned_addr learned my addr 172.21.15.91:0/2605008826 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:44.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0de00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b081609b0 0x7f0b080b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0d400640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 msgr2=0x7f0b080b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b081609b0 msgr2=0x7f0b080b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b081609b0 0x7f0b080b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0ca00640 1 -- 172.21.15.91:0/2605008826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af0009400 con 0x7f0b080b6bd0 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.087+0000 7f0b0d400640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:44.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b0ca00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0af00097a0 tx=0x7f0af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0033b30 con 0x7f0b080b6bd0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0033c90 con 0x7f0b080b6bd0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af003d470 con 0x7f0b080b6bd0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b080cab20 con 0x7f0b080b6bd0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.088+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b08149850 con 0x7f0b080b6bd0 2024-02-02T13:25:44.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.089+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b08066cf0 con 0x7f0b080b6bd0 2024-02-02T13:25:44.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.090+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0af003a040 con 0x7f0b080b6bd0 2024-02-02T13:25:44.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.091+0000 7f0b05600640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 0x7f0ae006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.091+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0af00aa420 con 0x7f0b080b6bd0 2024-02-02T13:25:44.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.091+0000 7f0b0d400640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 0x7f0ae006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.092+0000 7f0b0d400640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 0x7f0ae006c8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0af8000f60 tx=0x7f0af8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.095+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0af0075d50 con 0x7f0b080b6bd0 2024-02-02T13:25:44.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.333+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f0b080b89e0 con 0x7f0b080b6bd0 2024-02-02T13:25:44.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.341+0000 7f0b05600640 1 -- 172.21.15.91:0/2605008826 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0af00380a0 con 0x7f0b080b6bd0 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 msgr2=0x7f0ae006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 0x7f0ae006c8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0af8000f60 tx=0x7f0af8009040 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 msgr2=0x7f0b080ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0af00097a0 tx=0x7f0af00046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 shutdown_connections 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ae006a430 0x7f0ae006c8f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b081609b0 0x7f0b080b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b080b6bd0 0x7f0b080ca5e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 --2- 172.21.15.91:0/2605008826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08070270 0x7f0b080b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 >> 172.21.15.91:0/2605008826 conn(0x7f0b080d3020 msgr2=0x7f0b0815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 shutdown_connections 2024-02-02T13:25:44.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.343+0000 7f0b0fe00640 1 -- 172.21.15.91:0/2605008826 wait complete. 2024-02-02T13:25:44.363 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:44.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.558+0000 7f255f400640 1 Processor -- start 2024-02-02T13:25:44.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.558+0000 7f255f400640 1 -- start start 2024-02-02T13:25:44.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.559+0000 7f255f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f255815b2a0 0x7f25580cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.559+0000 7f255f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f255806f3f0 0x7f25580ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.559+0000 7f255f400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255806c1c0 con 0x7f255815aec0 2024-02-02T13:25:44.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.559+0000 7f255f400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255806b5f0 con 0x7f255806f3f0 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.559+0000 7f255f400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255806cd60 con 0x7f255815b2a0 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f255806f3f0 0x7f25580ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f255806f3f0 0x7f25580ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39802/0 (socket says 172.21.15.91:39802) 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 -- 172.21.15.91:0/2306222788 learned_addr learned my addr 172.21.15.91:0/2306222788 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 -- 172.21.15.91:0/2306222788 >> v2:172.21.15.113:3300/0 conn(0x7f255815b2a0 msgr2=0x7f25580cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 --2- 172.21.15.91:0/2306222788 >> v2:172.21.15.113:3300/0 conn(0x7f255815b2a0 0x7f25580cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 -- 172.21.15.91:0/2306222788 >> v1:172.21.15.113:6789/0 conn(0x7f255815aec0 legacy=0x7f255806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.560+0000 7f255ea00640 1 -- 172.21.15.91:0/2306222788 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255806b890 con 0x7f255806f3f0 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.561+0000 7f255ea00640 1 --2- 172.21.15.91:0/2306222788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255806f3f0 0x7f25580ca620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f255806ee00 tx=0x7f255002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a773cb8aac6556 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.561+0000 7f255cc00640 1 -- 172.21.15.91:0/2306222788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255002cb20 con 0x7f255806f3f0 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.561+0000 7f255cc00640 1 -- 172.21.15.91:0/2306222788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255002cc80 con 0x7f255806f3f0 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.561+0000 7f255cc00640 1 -- 172.21.15.91:0/2306222788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25500343f0 con 0x7f255806f3f0 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 -- 172.21.15.91:0/2306222788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255806f3f0 msgr2=0x7f25580ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 --2- 172.21.15.91:0/2306222788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255806f3f0 0x7f25580ca620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f255806ee00 tx=0x7f255002bf80 comp rx=0 tx=0).stop 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 -- 172.21.15.91:0/2306222788 shutdown_connections 2024-02-02T13:25:44.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 --2- 172.21.15.91:0/2306222788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255806f3f0 0x7f25580ca620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 --2- 172.21.15.91:0/2306222788 >> v2:172.21.15.113:3300/0 conn(0x7f255815b2a0 0x7f25580cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 -- 172.21.15.91:0/2306222788 >> 172.21.15.91:0/2306222788 conn(0x7f25580cea90 msgr2=0x7f25580ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 -- 172.21.15.91:0/2306222788 shutdown_connections 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.562+0000 7f255f400640 1 -- 172.21.15.91:0/2306222788 wait complete. 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.564+0000 7f255f400640 1 Processor -- start 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.564+0000 7f255f400640 1 -- start start 2024-02-02T13:25:44.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f255806f3f0 0x7f25580c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255815aec0 0x7f25580c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255815b830 con 0x7f255815b2a0 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255815c3b0 con 0x7f255815aec0 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25580c91e0 con 0x7f255806f3f0 2024-02-02T13:25:44.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39806/0 (socket says 172.21.15.91:39806) 2024-02-02T13:25:44.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.565+0000 7f255ea00640 1 -- 172.21.15.91:0/2193545238 learned_addr learned my addr 172.21.15.91:0/2193545238 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:44.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255e000640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f255806f3f0 0x7f25580c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255d600640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255815aec0 0x7f25580c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 -- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f255806f3f0 msgr2=0x7f25580c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f255806f3f0 0x7f25580c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 -- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255815aec0 msgr2=0x7f25580c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255815aec0 0x7f25580c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 -- 172.21.15.91:0/2193545238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550009b30 con 0x7f255815b2a0 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255ea00640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2550009f40 tx=0x7f2550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550009510 con 0x7f255815b2a0 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255002c920 con 0x7f255815b2a0 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255003e670 con 0x7f255815b2a0 2024-02-02T13:25:44.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.566+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255815ba80 con 0x7f255815b2a0 2024-02-02T13:25:44.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.567+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255815c040 con 0x7f255815b2a0 2024-02-02T13:25:44.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.568+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255815c540 con 0x7f255815b2a0 2024-02-02T13:25:44.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.568+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f255003e7d0 con 0x7f255815b2a0 2024-02-02T13:25:44.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.569+0000 7f2556000640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 0x7f253006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:44.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.569+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25500aafb0 con 0x7f255815b2a0 2024-02-02T13:25:44.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.569+0000 7f255e000640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 0x7f253006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:44.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.570+0000 7f255e000640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 0x7f253006c920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25480015d0 tx=0x7f2548009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:44.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.574+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2550076850 con 0x7f255815b2a0 2024-02-02T13:25:44.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.763+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f255804dff0 con 0x7f255815b2a0 2024-02-02T13:25:44.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.765+0000 7f2556000640 1 -- 172.21.15.91:0/2193545238 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f2550079e70 con 0x7f255815b2a0 2024-02-02T13:25:44.767 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:44.768 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":30,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:40.888605+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 msgr2=0x7f253006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 0x7f253006c920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25480015d0 tx=0x7f2548009040 comp rx=0 tx=0).stop 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 msgr2=0x7f255815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2550009f40 tx=0x7f2550004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 shutdown_connections 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f253006a460 0x7f253006c920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f255815b2a0 0x7f255815de00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255815aec0 0x7f25580c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 --2- 172.21.15.91:0/2193545238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f255806f3f0 0x7f25580c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:44.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 >> 172.21.15.91:0/2193545238 conn(0x7f25580cea90 msgr2=0x7f2558070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:44.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 shutdown_connections 2024-02-02T13:25:44.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.768+0000 7f255f400640 1 -- 172.21.15.91:0/2193545238 wait complete. 2024-02-02T13:25:44.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:25:44.787 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:25:44.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.997+0000 7f2f53800640 1 Processor -- start 2024-02-02T13:25:44.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.997+0000 7f2f53800640 1 -- start start 2024-02-02T13:25:45.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f53800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f4c0b0e10 0x7f2f4c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f53800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c070f80 con 0x7f2f4c151f90 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f53800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c06c600 con 0x7f2f4c15f5a0 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f53800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c15fa30 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f50e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2f4c151f90 0x7f2f4c06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37356/0 (socket says 172.21.15.91:37356) 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f50e00640 1 -- 172.21.15.91:0/101996553 learned_addr learned my addr 172.21.15.91:0/101996553 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:45.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f51800640 1 --2- 172.21.15.91:0/101996553 >> v2:172.21.15.91:3300/0 conn(0x7f2f4c0b0e10 0x7f2f4c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:44.999+0000 7f2f4b400640 1 -- 172.21.15.91:0/101996553 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158137602 0 0) 0x7f2f4c070f80 con 0x7f2f4c151f90 2024-02-02T13:25:45.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.000+0000 7f2f4b400640 1 -- 172.21.15.91:0/101996553 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c003520 con 0x7f2f4c151f90 2024-02-02T13:25:45.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.000+0000 7f2f51800640 1 -- 172.21.15.91:0/101996553 >> v1:172.21.15.113:6789/0 conn(0x7f2f4c15f5a0 legacy=0x7f2f4c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.000+0000 7f2f51800640 1 -- 172.21.15.91:0/101996553 >> v1:172.21.15.91:6789/0 conn(0x7f2f4c151f90 legacy=0x7f2f4c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.000+0000 7f2f51800640 1 -- 172.21.15.91:0/101996553 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c152370 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.001+0000 7f2f51800640 1 --2- 172.21.15.91:0/101996553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c0b3af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009d90 tx=0x7f2f4402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25db9e68538bac31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.001+0000 7f2f4b400640 1 -- 172.21.15.91:0/101996553 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4402cc60 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.001+0000 7f2f4b400640 1 -- 172.21.15.91:0/101996553 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4402cdc0 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.001+0000 7f2f4b400640 1 -- 172.21.15.91:0/101996553 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44034660 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 -- 172.21.15.91:0/101996553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0b0e10 msgr2=0x7f2f4c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 --2- 172.21.15.91:0/101996553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c0b3af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009d90 tx=0x7f2f4402beb0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 -- 172.21.15.91:0/101996553 shutdown_connections 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 --2- 172.21.15.91:0/101996553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c0b3af0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 -- 172.21.15.91:0/101996553 >> 172.21.15.91:0/101996553 conn(0x7f2f4c0cec80 msgr2=0x7f2f4c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 -- 172.21.15.91:0/101996553 shutdown_connections 2024-02-02T13:25:45.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.002+0000 7f2f53800640 1 -- 172.21.15.91:0/101996553 wait complete. 2024-02-02T13:25:45.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.004+0000 7f2f53800640 1 Processor -- start 2024-02-02T13:25:45.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.004+0000 7f2f53800640 1 -- start start 2024-02-02T13:25:45.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c14ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 0x7f2f4c15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0c4630 con 0x7f2f4c151f90 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c14cfc0 con 0x7f2f4c0b0e10 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f53800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b25d0 con 0x7f2f4c15f5a0 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39822/0 (socket says 172.21.15.91:39822) 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 -- 172.21.15.91:0/1104496308 learned_addr learned my addr 172.21.15.91:0/1104496308 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f50e00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c14ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f51800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 0x7f2f4c15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 -- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 msgr2=0x7f2f4c15aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 0x7f2f4c15aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 -- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 msgr2=0x7f2f4c14ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c14ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f4be00640 1 -- 172.21.15.91:0/1104496308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009a70 con 0x7f2f4c151f90 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.005+0000 7f2f50e00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c14ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f51800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 0x7f2f4c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f4be00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c460 tx=0x7f2f3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012830 con 0x7f2f4c151f90 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f340129d0 con 0x7f2f4c151f90 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012d80 con 0x7f2f4c151f90 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c0c3d00 con 0x7f2f4c151f90 2024-02-02T13:25:45.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.006+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c0c42c0 con 0x7f2f4c151f90 2024-02-02T13:25:45.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.007+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c066cf0 con 0x7f2f4c151f90 2024-02-02T13:25:45.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.008+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f3401b400 con 0x7f2f4c151f90 2024-02-02T13:25:45.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.009+0000 7f2f48c00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 0x7f2f2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.009+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2f34088890 con 0x7f2f4c151f90 2024-02-02T13:25:45.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.009+0000 7f2f50e00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 0x7f2f2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.010+0000 7f2f50e00640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 0x7f2f2406c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0011e0 tx=0x7f2f3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.014+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f340541c0 con 0x7f2f4c151f90 2024-02-02T13:25:45.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.200+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f4c0cc200 con 0x7f2f4c151f90 2024-02-02T13:25:45.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.201+0000 7f2f48c00640 1 -- 172.21.15.91:0/1104496308 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f340577e0 con 0x7f2f4c151f90 2024-02-02T13:25:45.203 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 msgr2=0x7f2f2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 0x7f2f2406c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0011e0 tx=0x7f2f3c009040 comp rx=0 tx=0).stop 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 msgr2=0x7f2f4c15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c460 tx=0x7f2f3400c920 comp rx=0 tx=0).stop 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 shutdown_connections 2024-02-02T13:25:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f2406a430 0x7f2f2406c8f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f4c15f5a0 0x7f2f4c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c151f90 0x7f2f4c15a510 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 --2- 172.21.15.91:0/1104496308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c0b0e10 0x7f2f4c14ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 >> 172.21.15.91:0/1104496308 conn(0x7f2f4c0cec80 msgr2=0x7f2f4c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 shutdown_connections 2024-02-02T13:25:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.204+0000 7f2f53800640 1 -- 172.21.15.91:0/1104496308 wait complete. 2024-02-02T13:25:45.225 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:25:45.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.432+0000 7f04c0000640 1 Processor -- start 2024-02-02T13:25:45.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.432+0000 7f04c0000640 1 -- start start 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.433+0000 7f04c0000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f04b80ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.433+0000 7f04c0000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80c94a0 con 0x7f04b80cb580 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04c0000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80ad6e0 con 0x7f04b815d3a0 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04c0000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80c96f0 con 0x7f04b80ca1d0 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04bd600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f04b80cb580 0x7f04b80cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37360/0 (socket says 172.21.15.91:37360) 2024-02-02T13:25:45.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04bd600640 1 -- 172.21.15.91:0/2315602425 learned_addr learned my addr 172.21.15.91:0/2315602425 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:45.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704702431 0 0) 0x7f04b80c94a0 con 0x7f04b80cb580 2024-02-02T13:25:45.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04be000640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f04b80ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.434+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0498003520 con 0x7f04b80cb580 2024-02-02T13:25:45.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864018336 0 0) 0x7f0498003520 con 0x7f04b80cb580 2024-02-02T13:25:45.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 msgr2=0x7f04b80ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f04b80ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 >> v1:172.21.15.113:6789/0 conn(0x7f04b815d3a0 legacy=0x7f04b80ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80c99d0 con 0x7f04b80cb580 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f04a80019f0 con 0x7f04b80cb580 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 >> v1:172.21.15.91:6789/0 conn(0x7f04b80cb580 legacy=0x7f04b80cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f0498004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f049800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80c96f0 con 0x7f04980047c0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498005510 con 0x7f04980039a0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049800bec0 con 0x7f04b80ca1d0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f0498004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04be000640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f049800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04bd600640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.435+0000 7f04bcc00640 1 -- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 msgr2=0x7f049800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.436+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80ca1d0 0x7f049800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.436+0000 7f04bcc00640 1 -- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04980039a0 msgr2=0x7f0498003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.436+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.436+0000 7f04bcc00640 1 -- 172.21.15.91:0/2315602425 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498003520 con 0x7f04980047c0 2024-02-02T13:25:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.436+0000 7f04bd600640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f0498004c40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04a0001750 tx=0x7f04980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5959d87a4dc0a529 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00052d0 con 0x7f04980047c0 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a002e050 con 0x7f04980047c0 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04b7e00640 1 -- 172.21.15.91:0/2315602425 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00035f0 con 0x7f04980047c0 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04c0000640 1 -- 172.21.15.91:0/2315602425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 msgr2=0x7f0498004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04c0000640 1 --2- 172.21.15.91:0/2315602425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f0498004c40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04a0001750 tx=0x7f04980057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.437+0000 7f04be000640 1 -- 172.21.15.91:0/2315602425 reap_dead start 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.438+0000 7f04c0000640 1 -- 172.21.15.91:0/2315602425 shutdown_connections 2024-02-02T13:25:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.438+0000 7f04c0000640 1 -- 172.21.15.91:0/2315602425 >> 172.21.15.91:0/2315602425 conn(0x7f04b80b0700 msgr2=0x7f04b80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.438+0000 7f04c0000640 1 -- 172.21.15.91:0/2315602425 shutdown_connections 2024-02-02T13:25:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.438+0000 7f04c0000640 1 -- 172.21.15.91:0/2315602425 wait complete. 2024-02-02T13:25:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.439+0000 7f04c0000640 1 Processor -- start 2024-02-02T13:25:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.439+0000 7f04c0000640 1 -- start start 2024-02-02T13:25:45.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b80c66c0 0x7f04b81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80b3c50 0x7f04b80b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80b4620 con 0x7f04980047c0 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80c6ff0 con 0x7f04b80c66c0 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.440+0000 7f04c0000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80b2e30 con 0x7f04b80b3c50 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39838/0 (socket says 172.21.15.91:39838) 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 -- 172.21.15.91:0/2327110845 learned_addr learned my addr 172.21.15.91:0/2327110845 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 -- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80b3c50 msgr2=0x7f04b80b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80b3c50 0x7f04b80b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 -- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b80c66c0 msgr2=0x7f04b81587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b80c66c0 0x7f04b81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 -- 172.21.15.91:0/2327110845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80b3110 con 0x7f04980047c0 2024-02-02T13:25:45.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04bd600640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04a800dfc0 tx=0x7f04a800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.441+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8004d60 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.442+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a80070d0 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.442+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b80b33f0 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.442+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8007480 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.442+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b815e5b0 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.443+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b8066cf0 con 0x7f04980047c0 2024-02-02T13:25:45.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.444+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f04a80075e0 con 0x7f04980047c0 2024-02-02T13:25:45.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.445+0000 7f04b5600640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 0x7f049006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.445+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04a8088a60 con 0x7f04980047c0 2024-02-02T13:25:45.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.446+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 0x7f049006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.446+0000 7f04bcc00640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 0x7f049006c920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04a0033000 tx=0x7f04a002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.450+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f04a8054430 con 0x7f04980047c0 2024-02-02T13:25:45.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.639+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f04b8070410 con 0x7f04980047c0 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.642+0000 7f04b5600640 1 -- 172.21.15.91:0/2327110845 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f04a8019090 con 0x7f04980047c0 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:25:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:25:45.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 msgr2=0x7f049006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 0x7f049006c920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04a0033000 tx=0x7f04a002f000 comp rx=0 tx=0).stop 2024-02-02T13:25:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 msgr2=0x7f04b80b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04a800dfc0 tx=0x7f04a800b560 comp rx=0 tx=0).stop 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 shutdown_connections 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049006a460 0x7f049006c920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04b80b3c50 0x7f04b80b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b80c66c0 0x7f04b81587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 --2- 172.21.15.91:0/2327110845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04980047c0 0x7f04b80b27e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.645+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 >> 172.21.15.91:0/2327110845 conn(0x7f04b80b0700 msgr2=0x7f04b806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:45.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.646+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 shutdown_connections 2024-02-02T13:25:45.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.646+0000 7f04c0000640 1 -- 172.21.15.91:0/2327110845 wait complete. 2024-02-02T13:25:45.667 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:25:45.667 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:25:45.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.871+0000 7f3759800640 1 Processor -- start 2024-02-02T13:25:45.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.871+0000 7f3759800640 1 -- start start 2024-02-02T13:25:45.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3759800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3754151f90 0x7f375415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3759800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3754070f80 con 0x7f375406f4f0 2024-02-02T13:25:45.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3759800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375406c600 con 0x7f37540b0e10 2024-02-02T13:25:45.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3759800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3754152370 con 0x7f3754151f90 2024-02-02T13:25:45.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3752000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f375406f4f0 0x7f37540b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37376/0 (socket says 172.21.15.91:37376) 2024-02-02T13:25:45.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3752000640 1 -- 172.21.15.91:0/2976291761 learned_addr learned my addr 172.21.15.91:0/2976291761 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:45.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3752a00640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3754151f90 0x7f375415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.873+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142092987 0 0) 0x7f3754070f80 con 0x7f375406f4f0 2024-02-02T13:25:45.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3734003520 con 0x7f375406f4f0 2024-02-02T13:25:45.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2294553783 0 0) 0x7f3734003520 con 0x7f375406f4f0 2024-02-02T13:25:45.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3754151f90 msgr2=0x7f375415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3754151f90 0x7f375415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 >> v1:172.21.15.113:6789/0 conn(0x7f37540b0e10 legacy=0x7f37540b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37540cb010 con 0x7f375406f4f0 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f373c001670 con 0x7f375406f4f0 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 >> v1:172.21.15.91:6789/0 conn(0x7f375406f4f0 legacy=0x7f37540b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37340039a0 0x7f3734003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340047c0 0x7f3734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3734005180 0x7f373400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3754152370 con 0x7f37340039a0 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373400c080 con 0x7f3734005180 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373400c240 con 0x7f37340047c0 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.874+0000 7f3752a00640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37340039a0 0x7f3734003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752000640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340047c0 0x7f3734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3753400640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3734005180 0x7f373400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752a00640 1 -- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340047c0 msgr2=0x7f3734004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752a00640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340047c0 0x7f3734004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752a00640 1 -- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3734005180 msgr2=0x7f373400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:45.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752a00640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3734005180 0x7f373400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.875+0000 7f3752a00640 1 -- 172.21.15.91:0/2976291761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734003520 con 0x7f37340039a0 2024-02-02T13:25:45.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.876+0000 7f3752a00640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37340039a0 0x7f3734003dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3754066830 tx=0x7f37340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683882e40b52350e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744034610 con 0x7f37340039a0 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374402e070 con 0x7f37340039a0 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3751600640 1 -- 172.21.15.91:0/2976291761 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744034990 con 0x7f37340039a0 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3759800640 1 -- 172.21.15.91:0/2976291761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37340039a0 msgr2=0x7f3734003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3759800640 1 --2- 172.21.15.91:0/2976291761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37340039a0 0x7f3734003dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3754066830 tx=0x7f37340057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3753400640 1 -- 172.21.15.91:0/2976291761 reap_dead start 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3759800640 1 -- 172.21.15.91:0/2976291761 shutdown_connections 2024-02-02T13:25:45.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.877+0000 7f3759800640 1 -- 172.21.15.91:0/2976291761 >> 172.21.15.91:0/2976291761 conn(0x7f37540cec80 msgr2=0x7f37540cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.878+0000 7f3759800640 1 -- 172.21.15.91:0/2976291761 shutdown_connections 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.878+0000 7f3759800640 1 -- 172.21.15.91:0/2976291761 wait complete. 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.879+0000 7f3759800640 1 Processor -- start 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.880+0000 7f3759800640 1 -- start start 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.880+0000 7f3759800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340039a0 0x7f37540b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.880+0000 7f3759800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37541f2e40 0x7f37540ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.880+0000 7f3759800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3759800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375414fc20 con 0x7f37540c65f0 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3759800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37540c6fb0 con 0x7f37541f2e40 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3759800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37540c74e0 con 0x7f37340039a0 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39850/0 (socket says 172.21.15.91:39850) 2024-02-02T13:25:45.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 -- 172.21.15.91:0/941356592 learned_addr learned my addr 172.21.15.91:0/941356592 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 -- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340039a0 msgr2=0x7f37540b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340039a0 0x7f37540b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 -- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37541f2e40 msgr2=0x7f37540ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37541f2e40 0x7f37540ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 -- 172.21.15.91:0/941356592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37540ce7a0 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.881+0000 7f3753400640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f374c007fd0 tx=0x7f374c00e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.882+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c003b80 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.882+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c003ce0 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.882+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c005310 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.882+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37541f33b0 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.882+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37541f3970 con 0x7f37540c65f0 2024-02-02T13:25:45.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.883+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37540b46d0 con 0x7f37540c65f0 2024-02-02T13:25:45.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.884+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f374c005470 con 0x7f37540c65f0 2024-02-02T13:25:45.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.885+0000 7f374aa00640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 0x7f372c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:45.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.885+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f374c088330 con 0x7f37540c65f0 2024-02-02T13:25:45.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.885+0000 7f3752a00640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 0x7f372c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:45.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.886+0000 7f3752a00640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 0x7f372c06c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37540cc770 tx=0x7f3744032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:45.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:45.891+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f374c053b40 con 0x7f37540c65f0 2024-02-02T13:25:46.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.082+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3754066cf0 con 0x7f37540c65f0 2024-02-02T13:25:46.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.656+0000 7f374aa00640 1 -- 172.21.15.91:0/941356592 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f374c057160 con 0x7f37540c65f0 2024-02-02T13:25:46.659 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:25:46.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 msgr2=0x7f372c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 0x7f372c06c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37540cc770 tx=0x7f3744032000 comp rx=0 tx=0).stop 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 msgr2=0x7f37540c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f374c007fd0 tx=0x7f374c00e9e0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 shutdown_connections 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f372c06a430 0x7f372c06c8f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37540c65f0 0x7f37540c6a70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37541f2e40 0x7f37540ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 --2- 172.21.15.91:0/941356592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37340039a0 0x7f37540b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.661+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 >> 172.21.15.91:0/941356592 conn(0x7f37540cec80 msgr2=0x7f375414feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:46.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.662+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 shutdown_connections 2024-02-02T13:25:46.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.662+0000 7f3759800640 1 -- 172.21.15.91:0/941356592 wait complete. 2024-02-02T13:25:46.683 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:25:46.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.900+0000 7f6921e00640 1 Processor -- start 2024-02-02T13:25:46.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.900+0000 7f6921e00640 1 -- start start 2024-02-02T13:25:46.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.901+0000 7f6921e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f691c151f90 0x7f691c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6921e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 0x7f691c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6921e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c070f80 con 0x7f691c06f4f0 2024-02-02T13:25:46.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6921e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c152370 con 0x7f691c151f90 2024-02-02T13:25:46.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6921e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0cb010 con 0x7f691c0b0e10 2024-02-02T13:25:46.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6913e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f691c06f4f0 0x7f691c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37386/0 (socket says 172.21.15.91:37386) 2024-02-02T13:25:46.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f6913e00640 1 -- 172.21.15.91:0/1164791864 learned_addr learned my addr 172.21.15.91:0/1164791864 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f691aa00640 1 --2- 172.21.15.91:0/1164791864 >> v2:172.21.15.91:3300/0 conn(0x7f691c151f90 0x7f691c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f691b400640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 0x7f691c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.902+0000 7f691a000640 1 -- 172.21.15.91:0/1164791864 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466273699 0 0) 0x7f691c070f80 con 0x7f691c06f4f0 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691a000640 1 -- 172.21.15.91:0/1164791864 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc003520 con 0x7f691c06f4f0 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691aa00640 1 -- 172.21.15.91:0/1164791864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 msgr2=0x7f691c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691aa00640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 0x7f691c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691aa00640 1 -- 172.21.15.91:0/1164791864 >> v1:172.21.15.91:6789/0 conn(0x7f691c06f4f0 legacy=0x7f691c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691aa00640 1 -- 172.21.15.91:0/1164791864 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0cb2f0 con 0x7f691c151f90 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.903+0000 7f691b400640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 0x7f691c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:25:46.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.904+0000 7f691aa00640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c15f5a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f690c0097a0 tx=0x7f690c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721054f054315c44 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:46.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f691a000640 1 -- 172.21.15.91:0/1164791864 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c02d040 con 0x7f691c151f90 2024-02-02T13:25:46.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f691a000640 1 -- 172.21.15.91:0/1164791864 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c031810 con 0x7f691c151f90 2024-02-02T13:25:46.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f691a000640 1 -- 172.21.15.91:0/1164791864 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c031ba0 con 0x7f691c151f90 2024-02-02T13:25:46.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 -- 172.21.15.91:0/1164791864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 msgr2=0x7f691c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c15f5a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f690c0097a0 tx=0x7f690c02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 -- 172.21.15.91:0/1164791864 shutdown_connections 2024-02-02T13:25:46.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c0b0e10 0x7f691c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 --2- 172.21.15.91:0/1164791864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c15f5a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 -- 172.21.15.91:0/1164791864 >> 172.21.15.91:0/1164791864 conn(0x7f691c0cec80 msgr2=0x7f691c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:46.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 -- 172.21.15.91:0/1164791864 shutdown_connections 2024-02-02T13:25:46.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.905+0000 7f6921e00640 1 -- 172.21.15.91:0/1164791864 wait complete. 2024-02-02T13:25:46.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.907+0000 7f6921e00640 1 Processor -- start 2024-02-02T13:25:46.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.907+0000 7f6921e00640 1 -- start start 2024-02-02T13:25:46.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c06f4f0 0x7f691c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f691c0b0e10 0x7f691c1f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0c6d90 con 0x7f691c151f90 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c158710 con 0x7f691c0b0e10 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.908+0000 7f6921e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0c2a70 con 0x7f691c06f4f0 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39862/0 (socket says 172.21.15.91:39862) 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 -- 172.21.15.91:0/3173801840 learned_addr learned my addr 172.21.15.91:0/3173801840 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 -- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c06f4f0 msgr2=0x7f691c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c06f4f0 0x7f691c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 -- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f691c0b0e10 msgr2=0x7f691c1f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f691c0b0e10 0x7f691c1f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 -- 172.21.15.91:0/3173801840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009400 con 0x7f691c151f90 2024-02-02T13:25:46.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f691b400640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6914007e40 tx=0x7f691400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.909+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914015600 con 0x7f691c151f90 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.910+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691400f040 con 0x7f691c151f90 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.910+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140159a0 con 0x7f691c151f90 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.910+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c1f26e0 con 0x7f691c151f90 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.910+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c0c64f0 con 0x7f691c151f90 2024-02-02T13:25:46.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.911+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c066cf0 con 0x7f691c151f90 2024-02-02T13:25:46.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.913+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f691401f440 con 0x7f691c151f90 2024-02-02T13:25:46.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.914+0000 7f6913400640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 0x7f68f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:46.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.914+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6914088f50 con 0x7f691c151f90 2024-02-02T13:25:46.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.914+0000 7f691aa00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 0x7f68f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:46.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.915+0000 7f691aa00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 0x7f68f406c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f690c036000 tx=0x7f690c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:46.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:46.918+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6914054700 con 0x7f691c151f90 2024-02-02T13:25:47.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.114+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f691c06c7b0 con 0x7f691c151f90 2024-02-02T13:25:47.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.672+0000 7f6913400640 1 -- 172.21.15.91:0/3173801840 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f691401a0d0 con 0x7f691c151f90 2024-02-02T13:25:47.675 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.678+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 msgr2=0x7f68f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.678+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 0x7f68f406c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f690c036000 tx=0x7f690c030000 comp rx=0 tx=0).stop 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.678+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 msgr2=0x7f691c0af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.678+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6914007e40 tx=0x7f691400d4a0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 shutdown_connections 2024-02-02T13:25:47.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68f406a430 0x7f68f406c8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c151f90 0x7f691c0af310 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f691c0b0e10 0x7f691c1f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 --2- 172.21.15.91:0/3173801840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f691c06f4f0 0x7f691c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.679+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 >> 172.21.15.91:0/3173801840 conn(0x7f691c0cec80 msgr2=0x7f691c0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.681+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 shutdown_connections 2024-02-02T13:25:47.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.681+0000 7f6921e00640 1 -- 172.21.15.91:0/3173801840 wait complete. 2024-02-02T13:25:47.710 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:25:47.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.898+0000 7fa478400640 1 Processor -- start 2024-02-02T13:25:47.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.898+0000 7fa478400640 1 -- start start 2024-02-02T13:25:47.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.899+0000 7fa478400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa47015f5a0 0x7fa4700b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.899+0000 7fa478400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa478400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa470070f80 con 0x7fa470151f90 2024-02-02T13:25:47.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa478400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47015fa30 con 0x7fa47015f5a0 2024-02-02T13:25:47.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa478400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700b10b0 con 0x7fa4700b0c40 2024-02-02T13:25:47.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa47015f5a0 0x7fa4700b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa47015f5a0 0x7fa4700b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39876/0 (socket says 172.21.15.91:39876) 2024-02-02T13:25:47.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 -- 172.21.15.91:0/263967789 learned_addr learned my addr 172.21.15.91:0/263967789 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa476400640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 -- 172.21.15.91:0/263967789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 msgr2=0x7fa4700b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 -- 172.21.15.91:0/263967789 >> v1:172.21.15.113:6789/0 conn(0x7fa470151f90 legacy=0x7fa47006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.900+0000 7fa475000640 1 -- 172.21.15.91:0/263967789 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470152370 con 0x7fa47015f5a0 2024-02-02T13:25:47.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.902+0000 7fa475000640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa4700b0700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa458009870 tx=0x7fa45802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f8b42354139a85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:47.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.902+0000 7fa46fe00640 1 -- 172.21.15.91:0/263967789 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458031640 con 0x7fa47015f5a0 2024-02-02T13:25:47.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.902+0000 7fa46fe00640 1 -- 172.21.15.91:0/263967789 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580317a0 con 0x7fa47015f5a0 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.902+0000 7fa46fe00640 1 -- 172.21.15.91:0/263967789 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458031b30 con 0x7fa47015f5a0 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 -- 172.21.15.91:0/263967789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 msgr2=0x7fa4700b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa4700b0700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa458009870 tx=0x7fa45802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 -- 172.21.15.91:0/263967789 shutdown_connections 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 --2- 172.21.15.91:0/263967789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa4700b0700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 -- 172.21.15.91:0/263967789 >> 172.21.15.91:0/263967789 conn(0x7fa4700cec80 msgr2=0x7fa4700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 -- 172.21.15.91:0/263967789 shutdown_connections 2024-02-02T13:25:47.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.903+0000 7fa478400640 1 -- 172.21.15.91:0/263967789 wait complete. 2024-02-02T13:25:47.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.905+0000 7fa478400640 1 Processor -- start 2024-02-02T13:25:47.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.905+0000 7fa478400640 1 -- start start 2024-02-02T13:25:47.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470151f90 0x7fa4700c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700c8490 con 0x7fa47015f5a0 2024-02-02T13:25:47.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700c8770 con 0x7fa470151f90 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.906+0000 7fa478400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700ad380 con 0x7fa4700b0c40 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39886/0 (socket says 172.21.15.91:39886) 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 -- 172.21.15.91:0/3834019189 learned_addr learned my addr 172.21.15.91:0/3834019189 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa475a00640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa475000640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470151f90 0x7fa4700c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 -- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 msgr2=0x7fa4700adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 -- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470151f90 msgr2=0x7fa4700c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470151f90 0x7fa4700c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 -- 172.21.15.91:0/3834019189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580094d0 con 0x7fa47015f5a0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.907+0000 7fa476400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa468007cf0 tx=0x7fa46800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.908+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468014420 con 0x7fa47015f5a0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.908+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4680145a0 con 0x7fa47015f5a0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.908+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468014950 con 0x7fa47015f5a0 2024-02-02T13:25:47.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.908+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4700ad660 con 0x7fa47015f5a0 2024-02-02T13:25:47.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.908+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4700ae0c0 con 0x7fa47015f5a0 2024-02-02T13:25:47.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.909+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4700ae2e0 con 0x7fa47015f5a0 2024-02-02T13:25:47.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.910+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa468014ab0 con 0x7fa47015f5a0 2024-02-02T13:25:47.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.911+0000 7fa46d600640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 0x7fa44806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:47.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.912+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa468089150 con 0x7fa47015f5a0 2024-02-02T13:25:47.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.912+0000 7fa475a00640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 0x7fa44806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:47.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.912+0000 7fa475a00640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 0x7fa44806c8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa460006fd0 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:47.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:47.917+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa4680547a0 con 0x7fa47015f5a0 2024-02-02T13:25:48.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.144+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa4700ad850 con 0x7fa47015f5a0 2024-02-02T13:25:48.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.161+0000 7fa46d600640 1 -- 172.21.15.91:0/3834019189 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa468057dc0 con 0x7fa47015f5a0 2024-02-02T13:25:48.168 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 4 and data pool 5 2024-02-02T13:25:48.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 msgr2=0x7fa44806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 0x7fa44806c8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa460006fd0 tx=0x7fa460008040 comp rx=0 tx=0).stop 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 msgr2=0x7fa47014a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa468007cf0 tx=0x7fa46800e4b0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 shutdown_connections 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa44806a430 0x7fa44806c8f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa47015f5a0 0x7fa47014a1f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470151f90 0x7fa4700c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 --2- 172.21.15.91:0/3834019189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4700b0c40 0x7fa4700adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.170+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 >> 172.21.15.91:0/3834019189 conn(0x7fa4700cec80 msgr2=0x7fa47006fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:48.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.171+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 shutdown_connections 2024-02-02T13:25:48.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.172+0000 7fa478400640 1 -- 172.21.15.91:0/3834019189 wait complete. 2024-02-02T13:25:48.195 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:48.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.372+0000 7fd797000640 1 Processor -- start 2024-02-02T13:25:48.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.373+0000 7fd797000640 1 -- start start 2024-02-02T13:25:48.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd797000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7900b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd797000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7900b1d70 0x7fd7900711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd797000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900ce210 con 0x7fd790160830 2024-02-02T13:25:48.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd797000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7900ce4f0 con 0x7fd7900b1d70 2024-02-02T13:25:48.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd797000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ce7d0 con 0x7fd790070980 2024-02-02T13:25:48.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd78fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd790160830 0x7fd790070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37390/0 (socket says 172.21.15.91:37390) 2024-02-02T13:25:48.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd78fe00640 1 -- 172.21.15.91:0/1838254127 learned_addr learned my addr 172.21.15.91:0/1838254127 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd78f400640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7900b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016560501 0 0) 0x7fd7900ce210 con 0x7fd790160830 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd795000640 1 --2- 172.21.15.91:0/1838254127 >> v2:172.21.15.113:3300/0 conn(0x7fd7900b1d70 0x7fd7900711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.374+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd770003520 con 0x7fd790160830 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572264445 0 0) 0x7fd770003520 con 0x7fd790160830 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 msgr2=0x7fd7900b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7900b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 >> v2:172.21.15.113:3300/0 conn(0x7fd7900b1d70 msgr2=0x7fd7900711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 --2- 172.21.15.91:0/1838254127 >> v2:172.21.15.113:3300/0 conn(0x7fd7900b1d70 0x7fd7900711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900d17b0 con 0x7fd790160830 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd780001670 con 0x7fd790160830 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 >> v1:172.21.15.91:6789/0 conn(0x7fd790160830 legacy=0x7fd790070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770003970 0x7fd770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd770004d00 0x7fd770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ce4f0 con 0x7fd770004d00 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ce7d0 con 0x7fd770003970 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77000ba30 con 0x7fd790070980 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd795000640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd770004d00 0x7fd770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.375+0000 7fd78fe00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770003970 0x7fd770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd78f400640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd795000640 1 -- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 msgr2=0x7fd7700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd795000640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd795000640 1 -- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770003970 msgr2=0x7fd770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd795000640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770003970 0x7fd770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd795000640 1 -- 172.21.15.91:0/1838254127 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770003520 con 0x7fd770004d00 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd78fe00640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770003970 0x7fd770003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:48.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.376+0000 7fd78f400640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd790070980 0x7fd7700047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd795000640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd770004d00 0x7fd770005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd788007360 tx=0x7fd7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0611ae2ad034f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78802ccf0 con 0x7fd770004d00 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78802ce50 con 0x7fd770004d00 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd78ea00640 1 -- 172.21.15.91:0/1838254127 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788034660 con 0x7fd770004d00 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd797000640 1 -- 172.21.15.91:0/1838254127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd770004d00 msgr2=0x7fd770005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd797000640 1 --2- 172.21.15.91:0/1838254127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd770004d00 0x7fd770005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd788007360 tx=0x7fd7700057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.377+0000 7fd795000640 1 -- 172.21.15.91:0/1838254127 reap_dead start 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.378+0000 7fd797000640 1 -- 172.21.15.91:0/1838254127 shutdown_connections 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.378+0000 7fd797000640 1 -- 172.21.15.91:0/1838254127 >> 172.21.15.91:0/1838254127 conn(0x7fd7900d3030 msgr2=0x7fd7900d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.378+0000 7fd797000640 1 -- 172.21.15.91:0/1838254127 shutdown_connections 2024-02-02T13:25:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.378+0000 7fd797000640 1 -- 172.21.15.91:0/1838254127 wait complete. 2024-02-02T13:25:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.379+0000 7fd797000640 1 Processor -- start 2024-02-02T13:25:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.379+0000 7fd797000640 1 -- start start 2024-02-02T13:25:48.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770004d00 0x7fd7901514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7900cbb70 0x7fd7900b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900aba20 con 0x7fd79015dc10 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900aa440 con 0x7fd770004d00 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.380+0000 7fd797000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900aa720 con 0x7fd7900cbb70 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39906/0 (socket says 172.21.15.91:39906) 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 -- 172.21.15.91:0/3428513455 learned_addr learned my addr 172.21.15.91:0/3428513455 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 -- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7900cbb70 msgr2=0x7fd7900b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7900cbb70 0x7fd7900b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 -- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770004d00 msgr2=0x7fd7901514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770004d00 0x7fd7901514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 -- 172.21.15.91:0/3428513455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900aaa00 con 0x7fd79015dc10 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.381+0000 7fd78f400640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7780108b0 tx=0x7fd778010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.382+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77800eae0 con 0x7fd79015dc10 2024-02-02T13:25:48.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.382+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77800ec40 con 0x7fd79015dc10 2024-02-02T13:25:48.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.382+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77801c400 con 0x7fd79015dc10 2024-02-02T13:25:48.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.382+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7900aace0 con 0x7fd79015dc10 2024-02-02T13:25:48.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.382+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7900cbff0 con 0x7fd79015dc10 2024-02-02T13:25:48.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.383+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd790066cf0 con 0x7fd79015dc10 2024-02-02T13:25:48.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.384+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd77801c560 con 0x7fd79015dc10 2024-02-02T13:25:48.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.385+0000 7fd787e00640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 0x7fd76806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.385+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd778088eb0 con 0x7fd79015dc10 2024-02-02T13:25:48.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.385+0000 7fd78fe00640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 0x7fd76806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.385+0000 7fd78fe00640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 0x7fd76806c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7800017a0 tx=0x7fd780009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.389+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd778054540 con 0x7fd79015dc10 2024-02-02T13:25:48.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.578+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7900d3f40 con 0x7fd79015dc10 2024-02-02T13:25:48.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.579+0000 7fd787e00640 1 -- 172.21.15.91:0/3428513455 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16114 (secure 0 0 0) 0x7fd778057b60 con 0x7fd79015dc10 2024-02-02T13:25:48.581 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:48.581 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":33,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:48.145960+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:48.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 msgr2=0x7fd76806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 0x7fd76806c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7800017a0 tx=0x7fd780009040 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 msgr2=0x7fd7900ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7780108b0 tx=0x7fd778010d70 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 shutdown_connections 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd76806a430 0x7fd76806c8f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7900cbb70 0x7fd7900b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd79015dc10 0x7fd7900ab4e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 --2- 172.21.15.91:0/3428513455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd770004d00 0x7fd7901514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 >> 172.21.15.91:0/3428513455 conn(0x7fd7900d3030 msgr2=0x7fd7900b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 shutdown_connections 2024-02-02T13:25:48.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.582+0000 7fd797000640 1 -- 172.21.15.91:0/3428513455 wait complete. 2024-02-02T13:25:48.602 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:48.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.810+0000 7f98a2400640 1 Processor -- start 2024-02-02T13:25:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.810+0000 7f98a2400640 1 -- start start 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a2400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f989c15d2c0 0x7f989c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a2400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0ad2e0 con 0x7f989c06c820 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a2400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0ad5c0 con 0x7f989c15d2c0 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c06b4f0 con 0x7f989c0ccd70 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f989c15d2c0 0x7f989c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f989c15d2c0 0x7f989c06bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39922/0 (socket says 172.21.15.91:39922) 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 -- 172.21.15.91:0/3117186231 learned_addr learned my addr 172.21.15.91:0/3117186231 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:48.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 -- 172.21.15.91:0/3117186231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 msgr2=0x7f989c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 --2- 172.21.15.91:0/3117186231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 -- 172.21.15.91:0/3117186231 >> v1:172.21.15.113:6789/0 conn(0x7f989c06c820 legacy=0x7f989c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.812+0000 7f98a1000640 1 -- 172.21.15.91:0/3117186231 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c06b7d0 con 0x7f989c15d2c0 2024-02-02T13:25:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.813+0000 7f98a1000640 1 --2- 172.21.15.91:0/3117186231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c06bfd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f988c009870 tx=0x7f988c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48f29afd3ba97cf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.814+0000 7f989be00640 1 -- 172.21.15.91:0/3117186231 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c031870 con 0x7f989c15d2c0 2024-02-02T13:25:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.814+0000 7f989be00640 1 -- 172.21.15.91:0/3117186231 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c0319d0 con 0x7f989c15d2c0 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.814+0000 7f989be00640 1 -- 172.21.15.91:0/3117186231 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c031d60 con 0x7f989c15d2c0 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 -- 172.21.15.91:0/3117186231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 msgr2=0x7f989c06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 --2- 172.21.15.91:0/3117186231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c06bfd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f988c009870 tx=0x7f988c02b910 comp rx=0 tx=0).stop 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 -- 172.21.15.91:0/3117186231 shutdown_connections 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 --2- 172.21.15.91:0/3117186231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 --2- 172.21.15.91:0/3117186231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c06bfd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 -- 172.21.15.91:0/3117186231 >> 172.21.15.91:0/3117186231 conn(0x7f989c0cea90 msgr2=0x7f989c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 -- 172.21.15.91:0/3117186231 shutdown_connections 2024-02-02T13:25:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.815+0000 7f98a2400640 1 -- 172.21.15.91:0/3117186231 wait complete. 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.816+0000 7f98a2400640 1 Processor -- start 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 -- start start 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f989c06c820 0x7f989c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0c8c80 con 0x7f989c15d2c0 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c151450 con 0x7f989c06c820 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.817+0000 7f98a2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0c7be0 con 0x7f989c0ccd70 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39938/0 (socket says 172.21.15.91:39938) 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 -- 172.21.15.91:0/2469306206 learned_addr learned my addr 172.21.15.91:0/2469306206 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1000640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f989c06c820 0x7f989c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 -- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 msgr2=0x7f989c0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 -- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f989c06c820 msgr2=0x7f989c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f989c06c820 0x7f989c150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 -- 172.21.15.91:0/2469306206 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0094d0 con 0x7f989c15d2c0 2024-02-02T13:25:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.818+0000 7f98a1a00640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f989400b9e0 tx=0x7f989400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.819+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989400ca20 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.819+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989400cba0 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.819+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c0c7ec0 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.819+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894014440 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.819+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c0c8480 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.820+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c0adfe0 con 0x7f989c15d2c0 2024-02-02T13:25:48.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.821+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f989401d3b0 con 0x7f989c15d2c0 2024-02-02T13:25:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.822+0000 7f9899600640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 0x7f987406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.822+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98940886a0 con 0x7f989c15d2c0 2024-02-02T13:25:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.822+0000 7f98a1000640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 0x7f987406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.823+0000 7f98a1000640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 0x7f987406c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f988c036000 tx=0x7f988c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:48.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:48.826+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9894053d20 con 0x7f989c15d2c0 2024-02-02T13:25:49.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.001+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f989c06c490 con 0x7f989c15d2c0 2024-02-02T13:25:49.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.003+0000 7f9899600640 1 -- 172.21.15.91:0/2469306206 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f9894019090 con 0x7f989c15d2c0 2024-02-02T13:25:49.005 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:49.005 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":33,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:48.145960+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:49.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.005+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 msgr2=0x7f987406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.005+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 0x7f987406c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f988c036000 tx=0x7f988c03c040 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.005+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 msgr2=0x7f989c0b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.005+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f989400b9e0 tx=0x7f989400bea0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 shutdown_connections 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f987406a430 0x7f987406c8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f989c0ccd70 0x7f989c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f989c15d2c0 0x7f989c0b3850 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 --2- 172.21.15.91:0/2469306206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f989c06c820 0x7f989c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 >> 172.21.15.91:0/2469306206 conn(0x7f989c0cea90 msgr2=0x7f989c0cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 shutdown_connections 2024-02-02T13:25:49.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.006+0000 7f98a2400640 1 -- 172.21.15.91:0/2469306206 wait complete. 2024-02-02T13:25:49.026 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:25:49.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.229+0000 7fb434400640 1 Processor -- start 2024-02-02T13:25:49.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.229+0000 7fb434400640 1 -- start start 2024-02-02T13:25:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.230+0000 7fb434400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c15d430 0x7fb42c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.230+0000 7fb434400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb42c0ac8d0 0x7fb42c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.230+0000 7fb434400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0c9260 con 0x7fb42c0adfe0 2024-02-02T13:25:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.230+0000 7fb434400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42c15dc40 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.230+0000 7fb434400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c161460 con 0x7fb42c15d430 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.231+0000 7fb431a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c15d430 0x7fb42c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.231+0000 7fb432400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb42c0ac8d0 0x7fb42c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.231+0000 7fb432400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb42c0ac8d0 0x7fb42c0acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:42430/0 (socket says 172.21.15.91:42430) 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.231+0000 7fb432400640 1 -- 172.21.15.91:0/3542277357 learned_addr learned my addr 172.21.15.91:0/3542277357 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.232+0000 7fb432400640 1 -- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c15d430 msgr2=0x7fb42c160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:49.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.232+0000 7fb432400640 1 --2- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c15d430 0x7fb42c160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.232+0000 7fb432400640 1 -- 172.21.15.91:0/3542277357 >> v1:172.21.15.113:6789/0 conn(0x7fb42c0adfe0 legacy=0x7fb42c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.232+0000 7fb432400640 1 -- 172.21.15.91:0/3542277357 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c161700 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.233+0000 7fb432400640 1 --2- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c0ac8d0 0x7fb42c0acd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb424009e50 tx=0x7fb42402bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fe5ab46ff3ff9e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb42be00640 1 -- 172.21.15.91:0/3542277357 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42402cb90 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb42be00640 1 -- 172.21.15.91:0/3542277357 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42402ccf0 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb42be00640 1 -- 172.21.15.91:0/3542277357 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424034610 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 -- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c0ac8d0 msgr2=0x7fb42c0acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 --2- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c0ac8d0 0x7fb42c0acd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb424009e50 tx=0x7fb42402bff0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 -- 172.21.15.91:0/3542277357 shutdown_connections 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 --2- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c0ac8d0 0x7fb42c0acd60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 --2- 172.21.15.91:0/3542277357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c15d430 0x7fb42c160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.234+0000 7fb434400640 1 -- 172.21.15.91:0/3542277357 >> 172.21.15.91:0/3542277357 conn(0x7fb42c0ceb90 msgr2=0x7fb42c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.235+0000 7fb434400640 1 -- 172.21.15.91:0/3542277357 shutdown_connections 2024-02-02T13:25:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.235+0000 7fb434400640 1 -- 172.21.15.91:0/3542277357 wait complete. 2024-02-02T13:25:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.236+0000 7fb434400640 1 Processor -- start 2024-02-02T13:25:49.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.236+0000 7fb434400640 1 -- start start 2024-02-02T13:25:49.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c0ac8d0 0x7fb42c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c15d430 0x7fb42c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1f72f0 con 0x7fb42c0adfe0 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c15aa80 con 0x7fb42c15d430 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.237+0000 7fb434400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c0c8930 con 0x7fb42c0ac8d0 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39942/0 (socket says 172.21.15.91:39942) 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 -- 172.21.15.91:0/3429855220 learned_addr learned my addr 172.21.15.91:0/3429855220 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431a00640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c0ac8d0 0x7fb42c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 -- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c0ac8d0 msgr2=0x7fb42c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c0ac8d0 0x7fb42c15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb432400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c15d430 0x7fb42c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 -- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c15d430 msgr2=0x7fb42c1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c15d430 0x7fb42c1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 -- 172.21.15.91:0/3429855220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424009b30 con 0x7fb42c0adfe0 2024-02-02T13:25:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.238+0000 7fb431000640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c400 tx=0x7fb41400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:49.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.239+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140052d0 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.239+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414005450 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.239+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414007260 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.239+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c1f66e0 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.239+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c1f6ca0 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.240+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c15e760 con 0x7fb42c0adfe0 2024-02-02T13:25:49.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.241+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb4140073c0 con 0x7fb42c0adfe0 2024-02-02T13:25:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.241+0000 7fb429600640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 0x7fb40406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.242+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb414087d80 con 0x7fb42c0adfe0 2024-02-02T13:25:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.242+0000 7fb431a00640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 0x7fb40406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:49.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.242+0000 7fb431a00640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 0x7fb40406c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a10 tx=0x7fb41c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:49.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.246+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb414053360 con 0x7fb42c0adfe0 2024-02-02T13:25:49.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:49.477+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb42c066cf0 con 0x7fb42c0adfe0 2024-02-02T13:25:50.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.270+0000 7fb429600640 1 -- 172.21.15.91:0/3429855220 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb414056980 con 0x7fb42c0adfe0 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 msgr2=0x7fb40406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 0x7fb40406c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a10 tx=0x7fb41c008040 comp rx=0 tx=0).stop 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 msgr2=0x7fb42c0c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c400 tx=0x7fb41400c8c0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 shutdown_connections 2024-02-02T13:25:50.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb40406a430 0x7fb40406c8f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c15d430 0x7fb42c1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb42c0adfe0 0x7fb42c0c83f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 --2- 172.21.15.91:0/3429855220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb42c0ac8d0 0x7fb42c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 >> 172.21.15.91:0/3429855220 conn(0x7fb42c0ceb90 msgr2=0x7fb42c0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 shutdown_connections 2024-02-02T13:25:50.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.274+0000 7fb434400640 1 -- 172.21.15.91:0/3429855220 wait complete. 2024-02-02T13:25:50.297 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:25:50.297 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:25:50.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.507+0000 7fc1b6400640 1 Processor -- start 2024-02-02T13:25:50.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.507+0000 7fc1b6400640 1 -- start start 2024-02-02T13:25:50.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b6400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc1b006cce0 0x7fc1b015db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:50.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b6400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b00ca740 con 0x7fc1b0159c50 2024-02-02T13:25:50.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b6400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b015a030 con 0x7fc1b00cbe00 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b6400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b015d710 con 0x7fc1b006cce0 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b5000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc1b0159c50 0x7fc1b006fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52966/0 (socket says 172.21.15.91:52966) 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b5000640 1 -- 172.21.15.91:0/2954210704 learned_addr learned my addr 172.21.15.91:0/2954210704 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b5a00640 1 --2- 172.21.15.91:0/2954210704 >> v2:172.21.15.91:3300/0 conn(0x7fc1b006cce0 0x7fc1b015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1af400640 1 -- 172.21.15.91:0/2954210704 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841694078 0 0) 0x7fc1a0001110 con 0x7fc1b0159c50 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1af400640 1 -- 172.21.15.91:0/2954210704 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc190005730 con 0x7fc1b0159c50 2024-02-02T13:25:50.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b5a00640 1 -- 172.21.15.91:0/2954210704 >> v1:172.21.15.113:6789/0 conn(0x7fc1b00cbe00 legacy=0x7fc1b006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:50.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.509+0000 7fc1b5a00640 1 -- 172.21.15.91:0/2954210704 >> v1:172.21.15.91:6789/0 conn(0x7fc1b0159c50 legacy=0x7fc1b006fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.510+0000 7fc1b5a00640 1 -- 172.21.15.91:0/2954210704 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0070310 con 0x7fc1b006cce0 2024-02-02T13:25:50.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1b5a00640 1 --2- 172.21.15.91:0/2954210704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b015db30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009d90 tx=0x7fc1a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918924bbbf73332c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:50.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1af400640 1 -- 172.21.15.91:0/2954210704 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a802cc60 con 0x7fc1b006cce0 2024-02-02T13:25:50.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1af400640 1 -- 172.21.15.91:0/2954210704 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a802cdc0 con 0x7fc1b006cce0 2024-02-02T13:25:50.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1af400640 1 -- 172.21.15.91:0/2954210704 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8034660 con 0x7fc1b006cce0 2024-02-02T13:25:50.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1b6400640 1 -- 172.21.15.91:0/2954210704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 msgr2=0x7fc1b015db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.511+0000 7fc1b6400640 1 --2- 172.21.15.91:0/2954210704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b015db30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009d90 tx=0x7fc1a802beb0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.512+0000 7fc1b6400640 1 -- 172.21.15.91:0/2954210704 shutdown_connections 2024-02-02T13:25:50.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.512+0000 7fc1b6400640 1 --2- 172.21.15.91:0/2954210704 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b015db30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.512+0000 7fc1b6400640 1 -- 172.21.15.91:0/2954210704 >> 172.21.15.91:0/2954210704 conn(0x7fc1b00b0560 msgr2=0x7fc1b00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:50.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.512+0000 7fc1b6400640 1 -- 172.21.15.91:0/2954210704 shutdown_connections 2024-02-02T13:25:50.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.512+0000 7fc1b6400640 1 -- 172.21.15.91:0/2954210704 wait complete. 2024-02-02T13:25:50.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.513+0000 7fc1b6400640 1 Processor -- start 2024-02-02T13:25:50.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.513+0000 7fc1b6400640 1 -- start start 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b00cbe00 0x7fc1b00cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b0159c50 0x7fc1b00cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00b18a0 con 0x7fc1b006cce0 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c7be0 con 0x7fc1b0159c50 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.514+0000 7fc1b6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b015b300 con 0x7fc1b00cbe00 2024-02-02T13:25:50.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:50.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56474/0 (socket says 172.21.15.91:56474) 2024-02-02T13:25:50.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 -- 172.21.15.91:0/583863690 learned_addr learned my addr 172.21.15.91:0/583863690 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1afe00640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b00cbe00 0x7fc1b00cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5a00640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b0159c50 0x7fc1b00cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 -- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b00cbe00 msgr2=0x7fc1b00cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b00cbe00 0x7fc1b00cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 -- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b0159c50 msgr2=0x7fc1b00cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b0159c50 0x7fc1b00cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 -- 172.21.15.91:0/583863690 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8009a70 con 0x7fc1b006cce0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.515+0000 7fc1b5000640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c420 tx=0x7fc1a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.516+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0005ce0 con 0x7fc1b006cce0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.516+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0005e40 con 0x7fc1b006cce0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.516+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b015b5e0 con 0x7fc1b006cce0 2024-02-02T13:25:50.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.516+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00072f0 con 0x7fc1b006cce0 2024-02-02T13:25:50.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.516+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b01efb60 con 0x7fc1b006cce0 2024-02-02T13:25:50.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.517+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b00cc290 con 0x7fc1b006cce0 2024-02-02T13:25:50.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.518+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc1a0007450 con 0x7fc1b006cce0 2024-02-02T13:25:50.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.519+0000 7fc1acc00640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 0x7fc18806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:50.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.519+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1a0087db0 con 0x7fc1b006cce0 2024-02-02T13:25:50.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.519+0000 7fc1afe00640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 0x7fc18806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:50.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.520+0000 7fc1afe00640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 0x7fc18806c8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1b01ef5d0 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:50.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.523+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc1b00cc290 con 0x7fc1b006cce0 2024-02-02T13:25:50.639 INFO:tasks.ceph.mds.b.smithi113.stderr:2024-02-02T13:25:50.638+0000 7fb4dc200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:25:50.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:50.756+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc1b015bb00 con 0x7fc1b006cce0 2024-02-02T13:25:51.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.682+0000 7fc1acc00640 1 -- 172.21.15.91:0/583863690 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc1b015bb00 con 0x7fc1b006cce0 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 msgr2=0x7fc18806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 0x7fc18806c8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1b01ef5d0 tx=0x7fc198009040 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 msgr2=0x7fc1b01ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c420 tx=0x7fc1a000c8e0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 shutdown_connections 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18806a430 0x7fc18806c8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b00cbe00 0x7fc1b00cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b0159c50 0x7fc1b00cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 --2- 172.21.15.91:0/583863690 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b006cce0 0x7fc1b01ef620 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 >> 172.21.15.91:0/583863690 conn(0x7fc1b00b0560 msgr2=0x7fc1b00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:51.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 shutdown_connections 2024-02-02T13:25:51.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.686+0000 7fc1b6400640 1 -- 172.21.15.91:0/583863690 wait complete. 2024-02-02T13:25:51.714 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:25:51.714 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:51.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.922+0000 7fede9a00640 1 Processor -- start 2024-02-02T13:25:51.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.922+0000 7fede9a00640 1 -- start start 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.923+0000 7fede9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fede415f5a0 0x7fede40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede9a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40b1070 con 0x7fede4151f90 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede9a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede406c600 con 0x7fede40b0c40 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede9a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede415fa30 con 0x7fede415f5a0 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede2a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fede4151f90 0x7fede406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52972/0 (socket says 172.21.15.91:52972) 2024-02-02T13:25:51.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede2a00640 1 -- 172.21.15.91:0/576494022 learned_addr learned my addr 172.21.15.91:0/576494022 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:51.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7feddbe00640 1 --2- 172.21.15.91:0/576494022 >> v2:172.21.15.113:3300/0 conn(0x7fede415f5a0 0x7fede40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670971124 0 0) 0x7fede40b1070 con 0x7fede4151f90 2024-02-02T13:25:51.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.924+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc4003520 con 0x7fede4151f90 2024-02-02T13:25:51.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1741478883 0 0) 0x7fedc4003520 con 0x7fede4151f90 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 >> v2:172.21.15.113:3300/0 conn(0x7fede415f5a0 msgr2=0x7fede40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 --2- 172.21.15.91:0/576494022 >> v2:172.21.15.113:3300/0 conn(0x7fede415f5a0 0x7fede40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 >> v1:172.21.15.113:6789/0 conn(0x7fede40b0c40 legacy=0x7fede40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4070f80 con 0x7fede4151f90 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fedd4001670 con 0x7fede4151f90 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 >> v1:172.21.15.91:6789/0 conn(0x7fede4151f90 legacy=0x7fede406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc40039c0 0x7fedc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 0x7fedc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede415fa30 con 0x7fedc4005180 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc400c080 con 0x7fedc40047c0 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc400c240 con 0x7fedc40039c0 2024-02-02T13:25:51.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.925+0000 7fede3400640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 0x7fedc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede2a00640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc40039c0 0x7fedc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede3400640 1 -- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc40039c0 msgr2=0x7fedc4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede3400640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc40039c0 0x7fedc4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede3400640 1 -- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc40047c0 msgr2=0x7fedc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede3400640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede3400640 1 -- 172.21.15.91:0/576494022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4003520 con 0x7fedc4005180 2024-02-02T13:25:51.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.926+0000 7fede2a00640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc40039c0 0x7fedc4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.927+0000 7fede3400640 1 -- 172.21.15.91:0/576494022 reap_dead start 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.927+0000 7fede3400640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 0x7fedc400b980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feddc001e50 tx=0x7fedc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21b337ee739a398 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.927+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc003b70 con 0x7fedc4005180 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.927+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc003d10 con 0x7fedc4005180 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.927+0000 7fede2000640 1 -- 172.21.15.91:0/576494022 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc005310 con 0x7fedc4005180 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 -- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 msgr2=0x7fedc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 0x7fedc400b980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feddc001e50 tx=0x7fedc40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 -- 172.21.15.91:0/576494022 shutdown_connections 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 --2- 172.21.15.91:0/576494022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc4005180 0x7fedc400b980 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 -- 172.21.15.91:0/576494022 >> 172.21.15.91:0/576494022 conn(0x7fede40cec80 msgr2=0x7fede40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 -- 172.21.15.91:0/576494022 shutdown_connections 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.928+0000 7fede9a00640 1 -- 172.21.15.91:0/576494022 wait complete. 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.930+0000 7fede9a00640 1 Processor -- start 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.930+0000 7fede9a00640 1 -- start start 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 0x7fede414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 0x7fede40cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede40ad510 con 0x7fede40ad9a0 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede40ac970 con 0x7fedc4005180 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede40acbe0 con 0x7fede40afaf0 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56494/0 (socket says 172.21.15.91:56494) 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 -- 172.21.15.91:0/4182403331 learned_addr learned my addr 172.21.15.91:0/4182403331 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede2a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 0x7fede414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7feddbe00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 0x7fede40cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 -- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 msgr2=0x7fede40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 0x7fede40cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 -- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 msgr2=0x7fede414f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 0x7fede414f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.931+0000 7fede3400640 1 -- 172.21.15.91:0/4182403331 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40acec0 con 0x7fede40ad9a0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7fede2a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 0x7fede414f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7feddbe00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 0x7fede40cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7fede3400640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7feddc02c060 tx=0x7feddc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc006d50 con 0x7fede40ad9a0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc003b80 con 0x7fede40ad9a0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede40ad1a0 con 0x7fede40ad9a0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0396c0 con 0x7fede40ad9a0 2024-02-02T13:25:51.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.932+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede40cdb30 con 0x7fede40ad9a0 2024-02-02T13:25:51.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.933+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede40ad6c0 con 0x7fede40ad9a0 2024-02-02T13:25:51.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.934+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feddc039820 con 0x7fede40ad9a0 2024-02-02T13:25:51.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.935+0000 7feddaa00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 0x7fedbc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:51.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.935+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feddc0aa770 con 0x7fede40ad9a0 2024-02-02T13:25:51.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.935+0000 7fede2a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 0x7fedbc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:51.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.936+0000 7fede2a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 0x7fedbc06c920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:51.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:51.940+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fede40ad6c0 con 0x7fede40ad9a0 2024-02-02T13:25:52.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.166+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fede4066cf0 con 0x7fede40ad9a0 2024-02-02T13:25:52.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.167+0000 7feddaa00640 1 -- 172.21.15.91:0/4182403331 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7feddc048090 con 0x7fede40ad9a0 2024-02-02T13:25:52.169 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:52.169 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:25:52.169 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 msgr2=0x7fedbc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 0x7fedbc06c920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd4009040 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 msgr2=0x7fede40ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7feddc02c060 tx=0x7feddc0014a0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 shutdown_connections 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedbc06a460 0x7fedbc06c920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fede40afaf0 0x7fede40cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede40ad9a0 0x7fede40ade20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 --2- 172.21.15.91:0/4182403331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc4005180 0x7fede414f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 >> 172.21.15.91:0/4182403331 conn(0x7fede40cec80 msgr2=0x7fede415a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 shutdown_connections 2024-02-02T13:25:52.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.170+0000 7fede9a00640 1 -- 172.21.15.91:0/4182403331 wait complete. 2024-02-02T13:25:52.189 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:25:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.391+0000 7fd91ec00640 1 Processor -- start 2024-02-02T13:25:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.391+0000 7fd91ec00640 1 -- start start 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91ec00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd9180b0e10 0x7fd9180b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91ec00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd918070f80 con 0x7fd91815f5a0 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91ec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91806c600 con 0x7fd918151f90 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91ec00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91815fa30 con 0x7fd9180b0e10 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd9180b0e10 0x7fd9180b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd917400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd91815f5a0 0x7fd9180b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52988/0 (socket says 172.21.15.91:52988) 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd917400640 1 -- 172.21.15.91:0/293216140 learned_addr learned my addr 172.21.15.91:0/293216140 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91cc00640 1 -- 172.21.15.91:0/293216140 >> v1:172.21.15.113:6789/0 conn(0x7fd918151f90 legacy=0x7fd91806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91cc00640 1 -- 172.21.15.91:0/293216140 >> v1:172.21.15.91:6789/0 conn(0x7fd91815f5a0 legacy=0x7fd9180b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.393+0000 7fd91cc00640 1 -- 172.21.15.91:0/293216140 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918152370 con 0x7fd9180b0e10 2024-02-02T13:25:52.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.394+0000 7fd91cc00640 1 --2- 172.21.15.91:0/293216140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9180b3af0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575f8d9027113d7e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.394+0000 7fd916a00640 1 -- 172.21.15.91:0/293216140 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91002ccf0 con 0x7fd9180b0e10 2024-02-02T13:25:52.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.394+0000 7fd916a00640 1 -- 172.21.15.91:0/293216140 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91002ce50 con 0x7fd9180b0e10 2024-02-02T13:25:52.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.394+0000 7fd916a00640 1 -- 172.21.15.91:0/293216140 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910034660 con 0x7fd9180b0e10 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 -- 172.21.15.91:0/293216140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 msgr2=0x7fd9180b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 --2- 172.21.15.91:0/293216140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9180b3af0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91002bf40 comp rx=0 tx=0).stop 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 -- 172.21.15.91:0/293216140 shutdown_connections 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 --2- 172.21.15.91:0/293216140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9180b3af0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 -- 172.21.15.91:0/293216140 >> 172.21.15.91:0/293216140 conn(0x7fd9180cec80 msgr2=0x7fd9180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 -- 172.21.15.91:0/293216140 shutdown_connections 2024-02-02T13:25:52.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.395+0000 7fd91ec00640 1 -- 172.21.15.91:0/293216140 wait complete. 2024-02-02T13:25:52.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.397+0000 7fd91ec00640 1 Processor -- start 2024-02-02T13:25:52.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.397+0000 7fd91ec00640 1 -- start start 2024-02-02T13:25:52.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd918151f90 0x7fd9180cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91815f5a0 0x7fd9180cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ca6b0 con 0x7fd9180b0e10 2024-02-02T13:25:52.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180c7bf0 con 0x7fd91815f5a0 2024-02-02T13:25:52.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9181528c0 con 0x7fd918151f90 2024-02-02T13:25:52.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56514/0 (socket says 172.21.15.91:56514) 2024-02-02T13:25:52.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 -- 172.21.15.91:0/294282729 learned_addr learned my addr 172.21.15.91:0/294282729 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 -- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd918151f90 msgr2=0x7fd9180cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917400640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd918151f90 0x7fd9180cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd91cc00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91815f5a0 0x7fd9180cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd918151f90 0x7fd9180cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 -- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91815f5a0 msgr2=0x7fd9180cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91815f5a0 0x7fd9180cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.398+0000 7fd917e00640 1 -- 172.21.15.91:0/294282729 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009a70 con 0x7fd9180b0e10 2024-02-02T13:25:52.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd917e00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c400 tx=0x7fd90800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080052d0 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908005430 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908003830 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9180c7350 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.399+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918152b60 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.400+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918066cf0 con 0x7fd9180b0e10 2024-02-02T13:25:52.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.401+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd908003990 con 0x7fd9180b0e10 2024-02-02T13:25:52.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.402+0000 7fd90fe00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 0x7fd8f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.402+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd908087e80 con 0x7fd9180b0e10 2024-02-02T13:25:52.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.403+0000 7fd917400640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 0x7fd8f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.403+0000 7fd917400640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 0x7fd8f006c8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd918158470 tx=0x7fd900009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.408+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd908053470 con 0x7fd9180b0e10 2024-02-02T13:25:52.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.596+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91806c7b0 con 0x7fd9180b0e10 2024-02-02T13:25:52.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.598+0000 7fd90fe00640 1 -- 172.21.15.91:0/294282729 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fd908056a90 con 0x7fd9180b0e10 2024-02-02T13:25:52.600 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:52.600 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":34,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:25:49.151690+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:25:52.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.600+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 msgr2=0x7fd8f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.600+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 0x7fd8f006c8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd918158470 tx=0x7fd900009040 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.600+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 msgr2=0x7fd9181f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.600+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c400 tx=0x7fd90800c8c0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 shutdown_connections 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd8f006a430 0x7fd8f006c8f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd918151f90 0x7fd9180cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91815f5a0 0x7fd9180cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 --2- 172.21.15.91:0/294282729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9180b0e10 0x7fd9181f21e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 >> 172.21.15.91:0/294282729 conn(0x7fd9180cec80 msgr2=0x7fd9180b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 shutdown_connections 2024-02-02T13:25:52.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.601+0000 7fd91ec00640 1 -- 172.21.15.91:0/294282729 wait complete. 2024-02-02T13:25:52.621 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:25:52.621 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:25:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.819+0000 7fb6cbe00640 1 Processor -- start 2024-02-02T13:25:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.819+0000 7fb6cbe00640 1 -- start start 2024-02-02T13:25:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.820+0000 7fb6cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 0x7fb6c40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.820+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.820+0000 7fb6cbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c40cc7c0 con 0x7fb6c415a010 2024-02-02T13:25:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.820+0000 7fb6cbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c40b10d0 con 0x7fb6c40ce130 2024-02-02T13:25:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.820+0000 7fb6cbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40ce930 con 0x7fb6c40b0700 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.821+0000 7fb6c9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.821+0000 7fb6c8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 0x7fb6c40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.821+0000 7fb6c9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:41686/0 (socket says 172.21.15.91:41686) 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.821+0000 7fb6c9e00640 1 -- 172.21.15.91:0/648127610 learned_addr learned my addr 172.21.15.91:0/648127610 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.822+0000 7fb6c9e00640 1 -- 172.21.15.91:0/648127610 >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 msgr2=0x7fb6c40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:52.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.822+0000 7fb6c9e00640 1 --2- 172.21.15.91:0/648127610 >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 0x7fb6c40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.822+0000 7fb6c9e00640 1 -- 172.21.15.91:0/648127610 >> v1:172.21.15.113:6789/0 conn(0x7fb6c415a010 legacy=0x7fb6c40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.822+0000 7fb6c9e00640 1 -- 172.21.15.91:0/648127610 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40cd640 con 0x7fb6c40b0700 2024-02-02T13:25:52.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.822+0000 7fb6c8a00640 1 --2- 172.21.15.91:0/648127610 >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 0x7fb6c40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:25:52.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.823+0000 7fb6c9e00640 1 --2- 172.21.15.91:0/648127610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009e50 tx=0x7fb6bc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9043c3e460d6ed13 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6c3e00640 1 -- 172.21.15.91:0/648127610 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc02cb00 con 0x7fb6c40b0700 2024-02-02T13:25:52.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6c3e00640 1 -- 172.21.15.91:0/648127610 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc02cc60 con 0x7fb6c40b0700 2024-02-02T13:25:52.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6c3e00640 1 -- 172.21.15.91:0/648127610 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0343f0 con 0x7fb6c40b0700 2024-02-02T13:25:52.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6cbe00640 1 -- 172.21.15.91:0/648127610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 msgr2=0x7fb6c40b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/648127610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009e50 tx=0x7fb6bc02bf60 comp rx=0 tx=0).stop 2024-02-02T13:25:52.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.824+0000 7fb6cbe00640 1 -- 172.21.15.91:0/648127610 shutdown_connections 2024-02-02T13:25:52.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.825+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/648127610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40b0b90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.825+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/648127610 >> v2:172.21.15.113:3300/0 conn(0x7fb6c40ce130 0x7fb6c40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.825+0000 7fb6cbe00640 1 -- 172.21.15.91:0/648127610 >> 172.21.15.91:0/648127610 conn(0x7fb6c40cec80 msgr2=0x7fb6c40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:52.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.825+0000 7fb6cbe00640 1 -- 172.21.15.91:0/648127610 shutdown_connections 2024-02-02T13:25:52.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.825+0000 7fb6cbe00640 1 -- 172.21.15.91:0/648127610 wait complete. 2024-02-02T13:25:52.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.826+0000 7fb6cbe00640 1 Processor -- start 2024-02-02T13:25:52.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.826+0000 7fb6cbe00640 1 -- start start 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 0x7fb6c415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40adc10 con 0x7fb6c415a010 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40adef0 con 0x7fb6c40ce130 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.827+0000 7fb6cbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40ae1d0 con 0x7fb6c40b0700 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56522/0 (socket says 172.21.15.91:56522) 2024-02-02T13:25:52.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 -- 172.21.15.91:0/3912589621 learned_addr learned my addr 172.21.15.91:0/3912589621 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c8a00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 0x7fb6c415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9400640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 -- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 msgr2=0x7fb6c40ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 -- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 msgr2=0x7fb6c415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 0x7fb6c415af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 -- 172.21.15.91:0/3912589621 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac009400 con 0x7fb6c415a010 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c8a00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 0x7fb6c415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.828+0000 7fb6c9e00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009e50 tx=0x7fb6bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.829+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc009510 con 0x7fb6c415a010 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.829+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc03e450 con 0x7fb6c415a010 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.829+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc03e800 con 0x7fb6c415a010 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.829+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc009b30 con 0x7fb6c415a010 2024-02-02T13:25:52.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.829+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c40b1bf0 con 0x7fb6c415a010 2024-02-02T13:25:52.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.830+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c4066cf0 con 0x7fb6c415a010 2024-02-02T13:25:52.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.831+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6bc03e960 con 0x7fb6c415a010 2024-02-02T13:25:52.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.832+0000 7fb6c1600640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 0x7fb69c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:52.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.832+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc0ab600 con 0x7fb6c415a010 2024-02-02T13:25:52.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.832+0000 7fb6c9400640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 0x7fb69c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:52.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.833+0000 7fb6c9400640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 0x7fb69c06c8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4000f60 tx=0x7fb6b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:52.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:52.837+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb6bc076b50 con 0x7fb6c415a010 2024-02-02T13:25:53.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.022+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6c406cb70 con 0x7fb69c06a430 2024-02-02T13:25:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.049+0000 7fb6c1600640 1 -- 172.21.15.91:0/3912589621 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282015 (secure 0 0 0) 0x7fb6b40050d0 con 0x7fb69c06a430 2024-02-02T13:25:53.053 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:53.055 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:25:53.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.055+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 msgr2=0x7fb69c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.055+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 0x7fb69c06c8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4000f60 tx=0x7fb6b4009040 comp rx=0 tx=0).stop 2024-02-02T13:25:53.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.055+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 msgr2=0x7fb6c415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.055+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009e50 tx=0x7fb6bc004a40 comp rx=0 tx=0).stop 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 shutdown_connections 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb69c06a430 0x7fb69c06c8f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6c415a010 0x7fb6c415b4c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c40ce130 0x7fb6c415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 --2- 172.21.15.91:0/3912589621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c40b0700 0x7fb6c40ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 >> 172.21.15.91:0/3912589621 conn(0x7fb6c40cec80 msgr2=0x7fb6c415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 shutdown_connections 2024-02-02T13:25:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.056+0000 7fb6cbe00640 1 -- 172.21.15.91:0/3912589621 wait complete. 2024-02-02T13:25:53.076 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-02-02T13:25:52.258102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560160,"statfs":{"total":1600000000,"available":1597604736,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.605731"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489225+0000","last_change":"2024-02-02T13:25:48.181916+0000","last_active":"2024-02-02T13:25:49.489225+0000","last_peered":"2024-02-02T13:25:49.489225+0000","last_clean":"2024-02-02T13:25:49.489225+0000","last_became_active":"2024-02-02T13:25:48.181645+0000","last_became_peered":"2024-02-02T13:25:48.181645+0000","last_unstale":"2024-02-02T13:25:49.489225+0000","last_undegraded":"2024-02-02T13:25:49.489225+0000","last_fullsized":"2024-02-02T13:25:49.489225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164925+0000","last_change":"2024-02-02T13:25:48.159561+0000","last_active":"2024-02-02T13:25:49.164925+0000","last_peered":"2024-02-02T13:25:49.164925+0000","last_clean":"2024-02-02T13:25:49.164925+0000","last_became_active":"2024-02-02T13:25:48.159272+0000","last_became_peered":"2024-02-02T13:25:48.159272+0000","last_unstale":"2024-02-02T13:25:49.164925+0000","last_undegraded":"2024-02-02T13:25:49.164925+0000","last_fullsized":"2024-02-02T13:25:49.164925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165768+0000","last_change":"2024-02-02T13:25:48.177400+0000","last_active":"2024-02-02T13:25:49.165768+0000","last_peered":"2024-02-02T13:25:49.165768+0000","last_clean":"2024-02-02T13:25:49.165768+0000","last_became_active":"2024-02-02T13:25:48.172120+0000","last_became_peered":"2024-02-02T13:25:48.172120+0000","last_unstale":"2024-02-02T13:25:49.165768+0000","last_undegraded":"2024-02-02T13:25:49.165768+0000","last_fullsized":"2024-02-02T13:25:49.165768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488875+0000","last_change":"2024-02-02T13:25:48.179174+0000","last_active":"2024-02-02T13:25:49.488875+0000","last_peered":"2024-02-02T13:25:49.488875+0000","last_clean":"2024-02-02T13:25:49.488875+0000","last_became_active":"2024-02-02T13:25:48.178011+0000","last_became_peered":"2024-02-02T13:25:48.178011+0000","last_unstale":"2024-02-02T13:25:49.488875+0000","last_undegraded":"2024-02-02T13:25:49.488875+0000","last_fullsized":"2024-02-02T13:25:49.488875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116401+0000","last_change":"2024-02-02T13:25:48.165370+0000","last_active":"2024-02-02T13:25:50.116401+0000","last_peered":"2024-02-02T13:25:50.116401+0000","last_clean":"2024-02-02T13:25:50.116401+0000","last_became_active":"2024-02-02T13:25:48.165112+0000","last_became_peered":"2024-02-02T13:25:48.165112+0000","last_unstale":"2024-02-02T13:25:50.116401+0000","last_undegraded":"2024-02-02T13:25:50.116401+0000","last_fullsized":"2024-02-02T13:25:50.116401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:38:09.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164447+0000","last_change":"2024-02-02T13:25:48.170191+0000","last_active":"2024-02-02T13:25:49.164447+0000","last_peered":"2024-02-02T13:25:49.164447+0000","last_clean":"2024-02-02T13:25:49.164447+0000","last_became_active":"2024-02-02T13:25:48.169981+0000","last_became_peered":"2024-02-02T13:25:48.169981+0000","last_unstale":"2024-02-02T13:25:49.164447+0000","last_undegraded":"2024-02-02T13:25:49.164447+0000","last_fullsized":"2024-02-02T13:25:49.164447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:16.860852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166057+0000","last_change":"2024-02-02T13:25:48.183691+0000","last_active":"2024-02-02T13:25:49.166057+0000","last_peered":"2024-02-02T13:25:49.166057+0000","last_clean":"2024-02-02T13:25:49.166057+0000","last_became_active":"2024-02-02T13:25:48.173110+0000","last_became_peered":"2024-02-02T13:25:48.173110+0000","last_unstale":"2024-02-02T13:25:49.166057+0000","last_undegraded":"2024-02-02T13:25:49.166057+0000","last_fullsized":"2024-02-02T13:25:49.166057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166203+0000","last_change":"2024-02-02T13:25:48.179465+0000","last_active":"2024-02-02T13:25:49.166203+0000","last_peered":"2024-02-02T13:25:49.166203+0000","last_clean":"2024-02-02T13:25:49.166203+0000","last_became_active":"2024-02-02T13:25:48.178608+0000","last_became_peered":"2024-02-02T13:25:48.178608+0000","last_unstale":"2024-02-02T13:25:49.166203+0000","last_undegraded":"2024-02-02T13:25:49.166203+0000","last_fullsized":"2024-02-02T13:25:49.166203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117359+0000","last_change":"2024-02-02T13:25:48.181182+0000","last_active":"2024-02-02T13:25:50.117359+0000","last_peered":"2024-02-02T13:25:50.117359+0000","last_clean":"2024-02-02T13:25:50.117359+0000","last_became_active":"2024-02-02T13:25:48.180854+0000","last_became_peered":"2024-02-02T13:25:48.180854+0000","last_unstale":"2024-02-02T13:25:50.117359+0000","last_undegraded":"2024-02-02T13:25:50.117359+0000","last_fullsized":"2024-02-02T13:25:50.117359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488497+0000","last_change":"2024-02-02T13:25:48.174383+0000","last_active":"2024-02-02T13:25:49.488497+0000","last_peered":"2024-02-02T13:25:49.488497+0000","last_clean":"2024-02-02T13:25:49.488497+0000","last_became_active":"2024-02-02T13:25:48.174168+0000","last_became_peered":"2024-02-02T13:25:48.174168+0000","last_unstale":"2024-02-02T13:25:49.488497+0000","last_undegraded":"2024-02-02T13:25:49.488497+0000","last_fullsized":"2024-02-02T13:25:49.488497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:59:32.385556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.167871+0000","last_change":"2024-02-02T13:25:48.171650+0000","last_active":"2024-02-02T13:25:49.167871+0000","last_peered":"2024-02-02T13:25:49.167871+0000","last_clean":"2024-02-02T13:25:49.167871+0000","last_became_active":"2024-02-02T13:25:48.171411+0000","last_became_peered":"2024-02-02T13:25:48.171411+0000","last_unstale":"2024-02-02T13:25:49.167871+0000","last_undegraded":"2024-02-02T13:25:49.167871+0000","last_fullsized":"2024-02-02T13:25:49.167871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165813+0000","last_change":"2024-02-02T13:25:48.178242+0000","last_active":"2024-02-02T13:25:49.165813+0000","last_peered":"2024-02-02T13:25:49.165813+0000","last_clean":"2024-02-02T13:25:49.165813+0000","last_became_active":"2024-02-02T13:25:48.174688+0000","last_became_peered":"2024-02-02T13:25:48.174688+0000","last_unstale":"2024-02-02T13:25:49.165813+0000","last_undegraded":"2024-02-02T13:25:49.165813+0000","last_fullsized":"2024-02-02T13:25:49.165813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116286+0000","last_change":"2024-02-02T13:25:48.173222+0000","last_active":"2024-02-02T13:25:50.116286+0000","last_peered":"2024-02-02T13:25:50.116286+0000","last_clean":"2024-02-02T13:25:50.116286+0000","last_became_active":"2024-02-02T13:25:48.172662+0000","last_became_peered":"2024-02-02T13:25:48.172662+0000","last_unstale":"2024-02-02T13:25:50.116286+0000","last_undegraded":"2024-02-02T13:25:50.116286+0000","last_fullsized":"2024-02-02T13:25:50.116286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:58.893964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164815+0000","last_change":"2024-02-02T13:25:48.159361+0000","last_active":"2024-02-02T13:25:49.164815+0000","last_peered":"2024-02-02T13:25:49.164815+0000","last_clean":"2024-02-02T13:25:49.164815+0000","last_became_active":"2024-02-02T13:25:48.159107+0000","last_became_peered":"2024-02-02T13:25:48.159107+0000","last_unstale":"2024-02-02T13:25:49.164815+0000","last_undegraded":"2024-02-02T13:25:49.164815+0000","last_fullsized":"2024-02-02T13:25:49.164815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165812+0000","last_change":"2024-02-02T13:25:48.175571+0000","last_active":"2024-02-02T13:25:49.165812+0000","last_peered":"2024-02-02T13:25:49.165812+0000","last_clean":"2024-02-02T13:25:49.165812+0000","last_became_active":"2024-02-02T13:25:48.175286+0000","last_became_peered":"2024-02-02T13:25:48.175286+0000","last_unstale":"2024-02-02T13:25:49.165812+0000","last_undegraded":"2024-02-02T13:25:49.165812+0000","last_fullsized":"2024-02-02T13:25:49.165812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.168095+0000","last_change":"2024-02-02T13:25:48.177514+0000","last_active":"2024-02-02T13:25:49.168095+0000","last_peered":"2024-02-02T13:25:49.168095+0000","last_clean":"2024-02-02T13:25:49.168095+0000","last_became_active":"2024-02-02T13:25:48.173010+0000","last_became_peered":"2024-02-02T13:25:48.173010+0000","last_unstale":"2024-02-02T13:25:49.168095+0000","last_undegraded":"2024-02-02T13:25:49.168095+0000","last_fullsized":"2024-02-02T13:25:49.168095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:47.393543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166194+0000","last_change":"2024-02-02T13:25:48.184313+0000","last_active":"2024-02-02T13:25:49.166194+0000","last_peered":"2024-02-02T13:25:49.166194+0000","last_clean":"2024-02-02T13:25:49.166194+0000","last_became_active":"2024-02-02T13:25:48.182831+0000","last_became_peered":"2024-02-02T13:25:48.182831+0000","last_unstale":"2024-02-02T13:25:49.166194+0000","last_undegraded":"2024-02-02T13:25:49.166194+0000","last_fullsized":"2024-02-02T13:25:49.166194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164615+0000","last_change":"2024-02-02T13:25:48.168369+0000","last_active":"2024-02-02T13:25:49.164615+0000","last_peered":"2024-02-02T13:25:49.164615+0000","last_clean":"2024-02-02T13:25:49.164615+0000","last_became_active":"2024-02-02T13:25:48.168111+0000","last_became_peered":"2024-02-02T13:25:48.168111+0000","last_unstale":"2024-02-02T13:25:49.164615+0000","last_undegraded":"2024-02-02T13:25:49.164615+0000","last_fullsized":"2024-02-02T13:25:49.164615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:43.375840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164167+0000","last_change":"2024-02-02T13:25:48.169491+0000","last_active":"2024-02-02T13:25:49.164167+0000","last_peered":"2024-02-02T13:25:49.164167+0000","last_clean":"2024-02-02T13:25:49.164167+0000","last_became_active":"2024-02-02T13:25:48.169155+0000","last_became_peered":"2024-02-02T13:25:48.169155+0000","last_unstale":"2024-02-02T13:25:49.164167+0000","last_undegraded":"2024-02-02T13:25:49.164167+0000","last_fullsized":"2024-02-02T13:25:49.164167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116629+0000","last_change":"2024-02-02T13:25:48.166670+0000","last_active":"2024-02-02T13:25:50.116629+0000","last_peered":"2024-02-02T13:25:50.116629+0000","last_clean":"2024-02-02T13:25:50.116629+0000","last_became_active":"2024-02-02T13:25:48.166395+0000","last_became_peered":"2024-02-02T13:25:48.166395+0000","last_unstale":"2024-02-02T13:25:50.116629+0000","last_undegraded":"2024-02-02T13:25:50.116629+0000","last_fullsized":"2024-02-02T13:25:50.116629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:57:38.246646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116810+0000","last_change":"2024-02-02T13:25:48.183087+0000","last_active":"2024-02-02T13:25:50.116810+0000","last_peered":"2024-02-02T13:25:50.116810+0000","last_clean":"2024-02-02T13:25:50.116810+0000","last_became_active":"2024-02-02T13:25:48.182503+0000","last_became_peered":"2024-02-02T13:25:48.182503+0000","last_unstale":"2024-02-02T13:25:50.116810+0000","last_undegraded":"2024-02-02T13:25:50.116810+0000","last_fullsized":"2024-02-02T13:25:50.116810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:00.369366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117753+0000","last_change":"2024-02-02T13:25:48.177292+0000","last_active":"2024-02-02T13:25:50.117753+0000","last_peered":"2024-02-02T13:25:50.117753+0000","last_clean":"2024-02-02T13:25:50.117753+0000","last_became_active":"2024-02-02T13:25:48.176979+0000","last_became_peered":"2024-02-02T13:25:48.176979+0000","last_unstale":"2024-02-02T13:25:50.117753+0000","last_undegraded":"2024-02-02T13:25:50.117753+0000","last_fullsized":"2024-02-02T13:25:50.117753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:39.214130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860870+0000","last_change":"2024-02-02T13:25:48.167660+0000","last_active":"2024-02-02T13:25:49.860870+0000","last_peered":"2024-02-02T13:25:49.860870+0000","last_clean":"2024-02-02T13:25:49.860870+0000","last_became_active":"2024-02-02T13:25:48.167403+0000","last_became_peered":"2024-02-02T13:25:48.167403+0000","last_unstale":"2024-02-02T13:25:49.860870+0000","last_undegraded":"2024-02-02T13:25:49.860870+0000","last_fullsized":"2024-02-02T13:25:49.860870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165252+0000","last_change":"2024-02-02T13:25:48.174865+0000","last_active":"2024-02-02T13:25:49.165252+0000","last_peered":"2024-02-02T13:25:49.165252+0000","last_clean":"2024-02-02T13:25:49.165252+0000","last_became_active":"2024-02-02T13:25:48.174397+0000","last_became_peered":"2024-02-02T13:25:48.174397+0000","last_unstale":"2024-02-02T13:25:49.165252+0000","last_undegraded":"2024-02-02T13:25:49.165252+0000","last_fullsized":"2024-02-02T13:25:49.165252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:47.393543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488587+0000","last_change":"2024-02-02T13:25:48.171501+0000","last_active":"2024-02-02T13:25:49.488587+0000","last_peered":"2024-02-02T13:25:49.488587+0000","last_clean":"2024-02-02T13:25:49.488587+0000","last_became_active":"2024-02-02T13:25:48.171282+0000","last_became_peered":"2024-02-02T13:25:48.171282+0000","last_unstale":"2024-02-02T13:25:49.488587+0000","last_undegraded":"2024-02-02T13:25:49.488587+0000","last_fullsized":"2024-02-02T13:25:49.488587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:06:57.101739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164552+0000","last_change":"2024-02-02T13:25:48.171403+0000","last_active":"2024-02-02T13:25:49.164552+0000","last_peered":"2024-02-02T13:25:49.164552+0000","last_clean":"2024-02-02T13:25:49.164552+0000","last_became_active":"2024-02-02T13:25:48.171150+0000","last_became_peered":"2024-02-02T13:25:48.171150+0000","last_unstale":"2024-02-02T13:25:49.164552+0000","last_undegraded":"2024-02-02T13:25:49.164552+0000","last_fullsized":"2024-02-02T13:25:49.164552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:44:49.165370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488987+0000","last_change":"2024-02-02T13:25:48.177213+0000","last_active":"2024-02-02T13:25:49.488987+0000","last_peered":"2024-02-02T13:25:49.488987+0000","last_clean":"2024-02-02T13:25:49.488987+0000","last_became_active":"2024-02-02T13:25:48.176871+0000","last_became_peered":"2024-02-02T13:25:48.176871+0000","last_unstale":"2024-02-02T13:25:49.488987+0000","last_undegraded":"2024-02-02T13:25:49.488987+0000","last_fullsized":"2024-02-02T13:25:49.488987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165359+0000","last_change":"2024-02-02T13:25:48.177231+0000","last_active":"2024-02-02T13:25:49.165359+0000","last_peered":"2024-02-02T13:25:49.165359+0000","last_clean":"2024-02-02T13:25:49.165359+0000","last_became_active":"2024-02-02T13:25:48.171552+0000","last_became_peered":"2024-02-02T13:25:48.171552+0000","last_unstale":"2024-02-02T13:25:49.165359+0000","last_undegraded":"2024-02-02T13:25:49.165359+0000","last_fullsized":"2024-02-02T13:25:49.165359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117622+0000","last_change":"2024-02-02T13:25:48.176140+0000","last_active":"2024-02-02T13:25:50.117622+0000","last_peered":"2024-02-02T13:25:50.117622+0000","last_clean":"2024-02-02T13:25:50.117622+0000","last_became_active":"2024-02-02T13:25:48.175915+0000","last_became_peered":"2024-02-02T13:25:48.175915+0000","last_unstale":"2024-02-02T13:25:50.117622+0000","last_undegraded":"2024-02-02T13:25:50.117622+0000","last_fullsized":"2024-02-02T13:25:50.117622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166585+0000","last_change":"2024-02-02T13:25:48.179543+0000","last_active":"2024-02-02T13:25:49.166585+0000","last_peered":"2024-02-02T13:25:49.166585+0000","last_clean":"2024-02-02T13:25:49.166585+0000","last_became_active":"2024-02-02T13:25:48.177450+0000","last_became_peered":"2024-02-02T13:25:48.177450+0000","last_unstale":"2024-02-02T13:25:49.166585+0000","last_undegraded":"2024-02-02T13:25:49.166585+0000","last_fullsized":"2024-02-02T13:25:49.166585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489398+0000","last_change":"2024-02-02T13:25:48.174355+0000","last_active":"2024-02-02T13:25:49.489398+0000","last_peered":"2024-02-02T13:25:49.489398+0000","last_clean":"2024-02-02T13:25:49.489398+0000","last_became_active":"2024-02-02T13:25:48.174079+0000","last_became_peered":"2024-02-02T13:25:48.174079+0000","last_unstale":"2024-02-02T13:25:49.489398+0000","last_undegraded":"2024-02-02T13:25:49.489398+0000","last_fullsized":"2024-02-02T13:25:49.489398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166324+0000","last_change":"2024-02-02T13:25:48.184490+0000","last_active":"2024-02-02T13:25:49.166324+0000","last_peered":"2024-02-02T13:25:49.166324+0000","last_clean":"2024-02-02T13:25:49.166324+0000","last_became_active":"2024-02-02T13:25:48.184042+0000","last_became_peered":"2024-02-02T13:25:48.184042+0000","last_unstale":"2024-02-02T13:25:49.166324+0000","last_undegraded":"2024-02-02T13:25:49.166324+0000","last_fullsized":"2024-02-02T13:25:49.166324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860671+0000","last_change":"2024-02-02T13:25:48.170495+0000","last_active":"2024-02-02T13:25:49.860671+0000","last_peered":"2024-02-02T13:25:49.860671+0000","last_clean":"2024-02-02T13:25:49.860671+0000","last_became_active":"2024-02-02T13:25:48.170286+0000","last_became_peered":"2024-02-02T13:25:48.170286+0000","last_unstale":"2024-02-02T13:25:49.860671+0000","last_undegraded":"2024-02-02T13:25:49.860671+0000","last_fullsized":"2024-02-02T13:25:49.860671+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489206+0000","last_change":"2024-02-02T13:25:47.692263+0000","last_active":"2024-02-02T13:25:49.489206+0000","last_peered":"2024-02-02T13:25:49.489206+0000","last_clean":"2024-02-02T13:25:49.489206+0000","last_became_active":"2024-02-02T13:25:47.691984+0000","last_became_peered":"2024-02-02T13:25:47.691984+0000","last_unstale":"2024-02-02T13:25:49.489206+0000","last_undegraded":"2024-02-02T13:25:49.489206+0000","last_fullsized":"2024-02-02T13:25:49.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:45.912896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488931+0000","last_change":"2024-02-02T13:25:48.176543+0000","last_active":"2024-02-02T13:25:49.488931+0000","last_peered":"2024-02-02T13:25:49.488931+0000","last_clean":"2024-02-02T13:25:49.488931+0000","last_became_active":"2024-02-02T13:25:48.176305+0000","last_became_peered":"2024-02-02T13:25:48.176305+0000","last_unstale":"2024-02-02T13:25:49.488931+0000","last_undegraded":"2024-02-02T13:25:49.488931+0000","last_fullsized":"2024-02-02T13:25:49.488931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860469+0000","last_change":"2024-02-02T13:25:47.692249+0000","last_active":"2024-02-02T13:25:49.860469+0000","last_peered":"2024-02-02T13:25:49.860469+0000","last_clean":"2024-02-02T13:25:49.860469+0000","last_became_active":"2024-02-02T13:25:47.692020+0000","last_became_peered":"2024-02-02T13:25:47.692020+0000","last_unstale":"2024-02-02T13:25:49.860469+0000","last_undegraded":"2024-02-02T13:25:49.860469+0000","last_fullsized":"2024-02-02T13:25:49.860469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116986+0000","last_change":"2024-02-02T13:25:48.183243+0000","last_active":"2024-02-02T13:25:50.116986+0000","last_peered":"2024-02-02T13:25:50.116986+0000","last_clean":"2024-02-02T13:25:50.116986+0000","last_became_active":"2024-02-02T13:25:48.182845+0000","last_became_peered":"2024-02-02T13:25:48.182845+0000","last_unstale":"2024-02-02T13:25:50.116986+0000","last_undegraded":"2024-02-02T13:25:50.116986+0000","last_fullsized":"2024-02-02T13:25:50.116986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164991+0000","last_change":"2024-02-02T13:25:47.689523+0000","last_active":"2024-02-02T13:25:49.164991+0000","last_peered":"2024-02-02T13:25:49.164991+0000","last_clean":"2024-02-02T13:25:49.164991+0000","last_became_active":"2024-02-02T13:25:47.689309+0000","last_became_peered":"2024-02-02T13:25:47.689309+0000","last_unstale":"2024-02-02T13:25:49.164991+0000","last_undegraded":"2024-02-02T13:25:49.164991+0000","last_fullsized":"2024-02-02T13:25:49.164991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:35.117418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488666+0000","last_change":"2024-02-02T13:25:48.181603+0000","last_active":"2024-02-02T13:25:49.488666+0000","last_peered":"2024-02-02T13:25:49.488666+0000","last_clean":"2024-02-02T13:25:49.488666+0000","last_became_active":"2024-02-02T13:25:48.181395+0000","last_became_peered":"2024-02-02T13:25:48.181395+0000","last_unstale":"2024-02-02T13:25:49.488666+0000","last_undegraded":"2024-02-02T13:25:49.488666+0000","last_fullsized":"2024-02-02T13:25:49.488666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:07.573173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116603+0000","last_change":"2024-02-02T13:25:47.685276+0000","last_active":"2024-02-02T13:25:50.116603+0000","last_peered":"2024-02-02T13:25:50.116603+0000","last_clean":"2024-02-02T13:25:50.116603+0000","last_became_active":"2024-02-02T13:25:47.685017+0000","last_became_peered":"2024-02-02T13:25:47.685017+0000","last_unstale":"2024-02-02T13:25:50.116603+0000","last_undegraded":"2024-02-02T13:25:50.116603+0000","last_fullsized":"2024-02-02T13:25:50.116603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:09:26.842297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166640+0000","last_change":"2024-02-02T13:25:48.179602+0000","last_active":"2024-02-02T13:25:49.166640+0000","last_peered":"2024-02-02T13:25:49.166640+0000","last_clean":"2024-02-02T13:25:49.166640+0000","last_became_active":"2024-02-02T13:25:48.178947+0000","last_became_peered":"2024-02-02T13:25:48.178947+0000","last_unstale":"2024-02-02T13:25:49.166640+0000","last_undegraded":"2024-02-02T13:25:49.166640+0000","last_fullsized":"2024-02-02T13:25:49.166640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:06:57.101739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117533+0000","last_change":"2024-02-02T13:25:47.693332+0000","last_active":"2024-02-02T13:25:50.117533+0000","last_peered":"2024-02-02T13:25:50.117533+0000","last_clean":"2024-02-02T13:25:50.117533+0000","last_became_active":"2024-02-02T13:25:47.691310+0000","last_became_peered":"2024-02-02T13:25:47.691310+0000","last_unstale":"2024-02-02T13:25:50.117533+0000","last_undegraded":"2024-02-02T13:25:50.117533+0000","last_fullsized":"2024-02-02T13:25:50.117533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:32.393560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165932+0000","last_change":"2024-02-02T13:25:48.171986+0000","last_active":"2024-02-02T13:25:49.165932+0000","last_peered":"2024-02-02T13:25:49.165932+0000","last_clean":"2024-02-02T13:25:49.165932+0000","last_became_active":"2024-02-02T13:25:48.171244+0000","last_became_peered":"2024-02-02T13:25:48.171244+0000","last_unstale":"2024-02-02T13:25:49.165932+0000","last_undegraded":"2024-02-02T13:25:49.165932+0000","last_fullsized":"2024-02-02T13:25:49.165932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117482+0000","last_change":"2024-02-02T13:25:47.693296+0000","last_active":"2024-02-02T13:25:50.117482+0000","last_peered":"2024-02-02T13:25:50.117482+0000","last_clean":"2024-02-02T13:25:50.117482+0000","last_became_active":"2024-02-02T13:25:47.689967+0000","last_became_peered":"2024-02-02T13:25:47.689967+0000","last_unstale":"2024-02-02T13:25:50.117482+0000","last_undegraded":"2024-02-02T13:25:50.117482+0000","last_fullsized":"2024-02-02T13:25:50.117482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:10.600882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116824+0000","last_change":"2024-02-02T13:25:48.181146+0000","last_active":"2024-02-02T13:25:50.116824+0000","last_peered":"2024-02-02T13:25:50.116824+0000","last_clean":"2024-02-02T13:25:50.116824+0000","last_became_active":"2024-02-02T13:25:48.180832+0000","last_became_peered":"2024-02-02T13:25:48.180832+0000","last_unstale":"2024-02-02T13:25:50.116824+0000","last_undegraded":"2024-02-02T13:25:50.116824+0000","last_fullsized":"2024-02-02T13:25:50.116824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:53:14.026351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165008+0000","last_change":"2024-02-02T13:25:47.686436+0000","last_active":"2024-02-02T13:25:49.165008+0000","last_peered":"2024-02-02T13:25:49.165008+0000","last_clean":"2024-02-02T13:25:49.165008+0000","last_became_active":"2024-02-02T13:25:47.686177+0000","last_became_peered":"2024-02-02T13:25:47.686177+0000","last_unstale":"2024-02-02T13:25:49.165008+0000","last_undegraded":"2024-02-02T13:25:49.165008+0000","last_fullsized":"2024-02-02T13:25:49.165008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:13.511597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489106+0000","last_change":"2024-02-02T13:25:48.179380+0000","last_active":"2024-02-02T13:25:49.489106+0000","last_peered":"2024-02-02T13:25:49.489106+0000","last_clean":"2024-02-02T13:25:49.489106+0000","last_became_active":"2024-02-02T13:25:48.178692+0000","last_became_peered":"2024-02-02T13:25:48.178692+0000","last_unstale":"2024-02-02T13:25:49.489106+0000","last_undegraded":"2024-02-02T13:25:49.489106+0000","last_fullsized":"2024-02-02T13:25:49.489106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117229+0000","last_change":"2024-02-02T13:25:47.697074+0000","last_active":"2024-02-02T13:25:50.117229+0000","last_peered":"2024-02-02T13:25:50.117229+0000","last_clean":"2024-02-02T13:25:50.117229+0000","last_became_active":"2024-02-02T13:25:47.695643+0000","last_became_peered":"2024-02-02T13:25:47.695643+0000","last_unstale":"2024-02-02T13:25:50.117229+0000","last_undegraded":"2024-02-02T13:25:50.117229+0000","last_fullsized":"2024-02-02T13:25:50.117229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:25.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166174+0000","last_change":"2024-02-02T13:25:48.178094+0000","last_active":"2024-02-02T13:25:49.166174+0000","last_peered":"2024-02-02T13:25:49.166174+0000","last_clean":"2024-02-02T13:25:49.166174+0000","last_became_active":"2024-02-02T13:25:48.174276+0000","last_became_peered":"2024-02-02T13:25:48.174276+0000","last_unstale":"2024-02-02T13:25:49.166174+0000","last_undegraded":"2024-02-02T13:25:49.166174+0000","last_fullsized":"2024-02-02T13:25:49.166174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164738+0000","last_change":"2024-02-02T13:25:47.693903+0000","last_active":"2024-02-02T13:25:49.164738+0000","last_peered":"2024-02-02T13:25:49.164738+0000","last_clean":"2024-02-02T13:25:49.164738+0000","last_became_active":"2024-02-02T13:25:47.693585+0000","last_became_peered":"2024-02-02T13:25:47.693585+0000","last_unstale":"2024-02-02T13:25:49.164738+0000","last_undegraded":"2024-02-02T13:25:49.164738+0000","last_fullsized":"2024-02-02T13:25:49.164738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:18:30.346290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164650+0000","last_change":"2024-02-02T13:25:48.169686+0000","last_active":"2024-02-02T13:25:49.164650+0000","last_peered":"2024-02-02T13:25:49.164650+0000","last_clean":"2024-02-02T13:25:49.164650+0000","last_became_active":"2024-02-02T13:25:48.168577+0000","last_became_peered":"2024-02-02T13:25:48.168577+0000","last_unstale":"2024-02-02T13:25:49.164650+0000","last_undegraded":"2024-02-02T13:25:49.164650+0000","last_fullsized":"2024-02-02T13:25:49.164650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860614+0000","last_change":"2024-02-02T13:25:47.689244+0000","last_active":"2024-02-02T13:25:49.860614+0000","last_peered":"2024-02-02T13:25:49.860614+0000","last_clean":"2024-02-02T13:25:49.860614+0000","last_became_active":"2024-02-02T13:25:47.688869+0000","last_became_peered":"2024-02-02T13:25:47.688869+0000","last_unstale":"2024-02-02T13:25:49.860614+0000","last_undegraded":"2024-02-02T13:25:49.860614+0000","last_fullsized":"2024-02-02T13:25:49.860614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165464+0000","last_change":"2024-02-02T13:25:48.171367+0000","last_active":"2024-02-02T13:25:49.165464+0000","last_peered":"2024-02-02T13:25:49.165464+0000","last_clean":"2024-02-02T13:25:49.165464+0000","last_became_active":"2024-02-02T13:25:48.171165+0000","last_became_peered":"2024-02-02T13:25:48.171165+0000","last_unstale":"2024-02-02T13:25:49.165464+0000","last_undegraded":"2024-02-02T13:25:49.165464+0000","last_fullsized":"2024-02-02T13:25:49.165464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:40:49.822973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165896+0000","last_change":"2024-02-02T13:25:47.678863+0000","last_active":"2024-02-02T13:25:49.165896+0000","last_peered":"2024-02-02T13:25:49.165896+0000","last_clean":"2024-02-02T13:25:49.165896+0000","last_became_active":"2024-02-02T13:25:47.677772+0000","last_became_peered":"2024-02-02T13:25:47.677772+0000","last_unstale":"2024-02-02T13:25:49.165896+0000","last_undegraded":"2024-02-02T13:25:49.165896+0000","last_fullsized":"2024-02-02T13:25:49.165896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:57:37.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165624+0000","last_change":"2024-02-02T13:25:48.177550+0000","last_active":"2024-02-02T13:25:49.165624+0000","last_peered":"2024-02-02T13:25:49.165624+0000","last_clean":"2024-02-02T13:25:49.165624+0000","last_became_active":"2024-02-02T13:25:48.172620+0000","last_became_peered":"2024-02-02T13:25:48.172620+0000","last_unstale":"2024-02-02T13:25:49.165624+0000","last_undegraded":"2024-02-02T13:25:49.165624+0000","last_fullsized":"2024-02-02T13:25:49.165624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165253+0000","last_change":"2024-02-02T13:25:47.689295+0000","last_active":"2024-02-02T13:25:49.165253+0000","last_peered":"2024-02-02T13:25:49.165253+0000","last_clean":"2024-02-02T13:25:49.165253+0000","last_became_active":"2024-02-02T13:25:47.688731+0000","last_became_peered":"2024-02-02T13:25:47.688731+0000","last_unstale":"2024-02-02T13:25:49.165253+0000","last_undegraded":"2024-02-02T13:25:49.165253+0000","last_fullsized":"2024-02-02T13:25:49.165253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:32.327008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488497+0000","last_change":"2024-02-02T13:25:48.175241+0000","last_active":"2024-02-02T13:25:49.488497+0000","last_peered":"2024-02-02T13:25:49.488497+0000","last_clean":"2024-02-02T13:25:49.488497+0000","last_became_active":"2024-02-02T13:25:48.174719+0000","last_became_peered":"2024-02-02T13:25:48.174719+0000","last_unstale":"2024-02-02T13:25:49.488497+0000","last_undegraded":"2024-02-02T13:25:49.488497+0000","last_fullsized":"2024-02-02T13:25:49.488497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:53:14.026351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117360+0000","last_change":"2024-02-02T13:25:47.696876+0000","last_active":"2024-02-02T13:25:50.117360+0000","last_peered":"2024-02-02T13:25:50.117360+0000","last_clean":"2024-02-02T13:25:50.117360+0000","last_became_active":"2024-02-02T13:25:47.695315+0000","last_became_peered":"2024-02-02T13:25:47.695315+0000","last_unstale":"2024-02-02T13:25:50.117360+0000","last_undegraded":"2024-02-02T13:25:50.117360+0000","last_fullsized":"2024-02-02T13:25:50.117360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:18:26.426340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164596+0000","last_change":"2024-02-02T13:25:48.165452+0000","last_active":"2024-02-02T13:25:49.164596+0000","last_peered":"2024-02-02T13:25:49.164596+0000","last_clean":"2024-02-02T13:25:49.164596+0000","last_became_active":"2024-02-02T13:25:48.164530+0000","last_became_peered":"2024-02-02T13:25:48.164530+0000","last_unstale":"2024-02-02T13:25:49.164596+0000","last_undegraded":"2024-02-02T13:25:49.164596+0000","last_fullsized":"2024-02-02T13:25:49.164596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488640+0000","last_change":"2024-02-02T13:25:47.692403+0000","last_active":"2024-02-02T13:25:49.488640+0000","last_peered":"2024-02-02T13:25:49.488640+0000","last_clean":"2024-02-02T13:25:49.488640+0000","last_became_active":"2024-02-02T13:25:47.692124+0000","last_became_peered":"2024-02-02T13:25:47.692124+0000","last_unstale":"2024-02-02T13:25:49.488640+0000","last_undegraded":"2024-02-02T13:25:49.488640+0000","last_fullsized":"2024-02-02T13:25:49.488640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:35.117418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166866+0000","last_change":"2024-02-02T13:25:48.179726+0000","last_active":"2024-02-02T13:25:49.166866+0000","last_peered":"2024-02-02T13:25:49.166866+0000","last_clean":"2024-02-02T13:25:49.166866+0000","last_became_active":"2024-02-02T13:25:48.179235+0000","last_became_peered":"2024-02-02T13:25:48.179235+0000","last_unstale":"2024-02-02T13:25:49.166866+0000","last_undegraded":"2024-02-02T13:25:49.166866+0000","last_fullsized":"2024-02-02T13:25:49.166866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165410+0000","last_change":"2024-02-02T13:25:47.692716+0000","last_active":"2024-02-02T13:25:49.165410+0000","last_peered":"2024-02-02T13:25:49.165410+0000","last_clean":"2024-02-02T13:25:49.165410+0000","last_became_active":"2024-02-02T13:25:47.692524+0000","last_became_peered":"2024-02-02T13:25:47.692524+0000","last_unstale":"2024-02-02T13:25:49.165410+0000","last_undegraded":"2024-02-02T13:25:49.165410+0000","last_fullsized":"2024-02-02T13:25:49.165410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489311+0000","last_change":"2024-02-02T13:25:48.176715+0000","last_active":"2024-02-02T13:25:49.489311+0000","last_peered":"2024-02-02T13:25:49.489311+0000","last_clean":"2024-02-02T13:25:49.489311+0000","last_became_active":"2024-02-02T13:25:48.176282+0000","last_became_peered":"2024-02-02T13:25:48.176282+0000","last_unstale":"2024-02-02T13:25:49.489311+0000","last_undegraded":"2024-02-02T13:25:49.489311+0000","last_fullsized":"2024-02-02T13:25:49.489311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:39.214130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164848+0000","last_change":"2024-02-02T13:25:47.691906+0000","last_active":"2024-02-02T13:25:49.164848+0000","last_peered":"2024-02-02T13:25:49.164848+0000","last_clean":"2024-02-02T13:25:49.164848+0000","last_became_active":"2024-02-02T13:25:47.691398+0000","last_became_peered":"2024-02-02T13:25:47.691398+0000","last_unstale":"2024-02-02T13:25:49.164848+0000","last_undegraded":"2024-02-02T13:25:49.164848+0000","last_fullsized":"2024-02-02T13:25:49.164848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117096+0000","last_change":"2024-02-02T13:25:48.182985+0000","last_active":"2024-02-02T13:25:50.117096+0000","last_peered":"2024-02-02T13:25:50.117096+0000","last_clean":"2024-02-02T13:25:50.117096+0000","last_became_active":"2024-02-02T13:25:48.182751+0000","last_became_peered":"2024-02-02T13:25:48.182751+0000","last_unstale":"2024-02-02T13:25:50.117096+0000","last_undegraded":"2024-02-02T13:25:50.117096+0000","last_fullsized":"2024-02-02T13:25:50.117096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:18:31.358697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489385+0000","last_change":"2024-02-02T13:25:47.686600+0000","last_active":"2024-02-02T13:25:49.489385+0000","last_peered":"2024-02-02T13:25:49.489385+0000","last_clean":"2024-02-02T13:25:49.489385+0000","last_became_active":"2024-02-02T13:25:47.686361+0000","last_became_peered":"2024-02-02T13:25:47.686361+0000","last_unstale":"2024-02-02T13:25:49.489385+0000","last_undegraded":"2024-02-02T13:25:49.489385+0000","last_fullsized":"2024-02-02T13:25:49.489385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165389+0000","last_change":"2024-02-02T13:25:48.177369+0000","last_active":"2024-02-02T13:25:49.165389+0000","last_peered":"2024-02-02T13:25:49.165389+0000","last_clean":"2024-02-02T13:25:49.165389+0000","last_became_active":"2024-02-02T13:25:48.177132+0000","last_became_peered":"2024-02-02T13:25:48.177132+0000","last_unstale":"2024-02-02T13:25:49.165389+0000","last_undegraded":"2024-02-02T13:25:49.165389+0000","last_fullsized":"2024-02-02T13:25:49.165389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117529+0000","last_change":"2024-02-02T13:25:47.696259+0000","last_active":"2024-02-02T13:25:50.117529+0000","last_peered":"2024-02-02T13:25:50.117529+0000","last_clean":"2024-02-02T13:25:50.117529+0000","last_became_active":"2024-02-02T13:25:47.694928+0000","last_became_peered":"2024-02-02T13:25:47.694928+0000","last_unstale":"2024-02-02T13:25:50.117529+0000","last_undegraded":"2024-02-02T13:25:50.117529+0000","last_fullsized":"2024-02-02T13:25:50.117529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:17.518801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488525+0000","last_change":"2024-02-02T13:25:48.177080+0000","last_active":"2024-02-02T13:25:49.488525+0000","last_peered":"2024-02-02T13:25:49.488525+0000","last_clean":"2024-02-02T13:25:49.488525+0000","last_became_active":"2024-02-02T13:25:48.176641+0000","last_became_peered":"2024-02-02T13:25:48.176641+0000","last_unstale":"2024-02-02T13:25:49.488525+0000","last_undegraded":"2024-02-02T13:25:49.488525+0000","last_fullsized":"2024-02-02T13:25:49.488525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117122+0000","last_change":"2024-02-02T13:25:47.687195+0000","last_active":"2024-02-02T13:25:50.117122+0000","last_peered":"2024-02-02T13:25:50.117122+0000","last_clean":"2024-02-02T13:25:50.117122+0000","last_became_active":"2024-02-02T13:25:47.686807+0000","last_became_peered":"2024-02-02T13:25:47.686807+0000","last_unstale":"2024-02-02T13:25:50.117122+0000","last_undegraded":"2024-02-02T13:25:50.117122+0000","last_fullsized":"2024-02-02T13:25:50.117122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:09:26.842297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116342+0000","last_change":"2024-02-02T13:25:48.173270+0000","last_active":"2024-02-02T13:25:50.116342+0000","last_peered":"2024-02-02T13:25:50.116342+0000","last_clean":"2024-02-02T13:25:50.116342+0000","last_became_active":"2024-02-02T13:25:48.172673+0000","last_became_peered":"2024-02-02T13:25:48.172673+0000","last_unstale":"2024-02-02T13:25:50.116342+0000","last_undegraded":"2024-02-02T13:25:50.116342+0000","last_fullsized":"2024-02-02T13:25:50.116342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:01.804626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117940+0000","last_change":"2024-02-02T13:25:47.694212+0000","last_active":"2024-02-02T13:25:50.117940+0000","last_peered":"2024-02-02T13:25:50.117940+0000","last_clean":"2024-02-02T13:25:50.117940+0000","last_became_active":"2024-02-02T13:25:47.692002+0000","last_became_peered":"2024-02-02T13:25:47.692002+0000","last_unstale":"2024-02-02T13:25:50.117940+0000","last_undegraded":"2024-02-02T13:25:50.117940+0000","last_fullsized":"2024-02-02T13:25:50.117940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:39:29.715158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116159+0000","last_change":"2024-02-02T13:25:48.175089+0000","last_active":"2024-02-02T13:25:50.116159+0000","last_peered":"2024-02-02T13:25:50.116159+0000","last_clean":"2024-02-02T13:25:50.116159+0000","last_became_active":"2024-02-02T13:25:48.174785+0000","last_became_peered":"2024-02-02T13:25:48.174785+0000","last_unstale":"2024-02-02T13:25:50.116159+0000","last_undegraded":"2024-02-02T13:25:50.116159+0000","last_fullsized":"2024-02-02T13:25:50.116159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:36.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.861127+0000","last_change":"2024-02-02T13:25:47.691955+0000","last_active":"2024-02-02T13:25:49.861127+0000","last_peered":"2024-02-02T13:25:49.861127+0000","last_clean":"2024-02-02T13:25:49.861127+0000","last_became_active":"2024-02-02T13:25:47.690900+0000","last_became_peered":"2024-02-02T13:25:47.690900+0000","last_unstale":"2024-02-02T13:25:49.861127+0000","last_undegraded":"2024-02-02T13:25:49.861127+0000","last_fullsized":"2024-02-02T13:25:49.861127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116589+0000","last_change":"2024-02-02T13:25:48.161097+0000","last_active":"2024-02-02T13:25:50.116589+0000","last_peered":"2024-02-02T13:25:50.116589+0000","last_clean":"2024-02-02T13:25:50.116589+0000","last_became_active":"2024-02-02T13:25:48.160874+0000","last_became_peered":"2024-02-02T13:25:48.160874+0000","last_unstale":"2024-02-02T13:25:50.116589+0000","last_undegraded":"2024-02-02T13:25:50.116589+0000","last_fullsized":"2024-02-02T13:25:50.116589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:14.082563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488949+0000","last_change":"2024-02-02T13:25:47.694662+0000","last_active":"2024-02-02T13:25:49.488949+0000","last_peered":"2024-02-02T13:25:49.488949+0000","last_clean":"2024-02-02T13:25:49.488949+0000","last_became_active":"2024-02-02T13:25:47.693811+0000","last_became_peered":"2024-02-02T13:25:47.693811+0000","last_unstale":"2024-02-02T13:25:49.488949+0000","last_undegraded":"2024-02-02T13:25:49.488949+0000","last_fullsized":"2024-02-02T13:25:49.488949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116481+0000","last_change":"2024-02-02T13:25:48.173396+0000","last_active":"2024-02-02T13:25:50.116481+0000","last_peered":"2024-02-02T13:25:50.116481+0000","last_clean":"2024-02-02T13:25:50.116481+0000","last_became_active":"2024-02-02T13:25:48.173032+0000","last_became_peered":"2024-02-02T13:25:48.173032+0000","last_unstale":"2024-02-02T13:25:50.116481+0000","last_undegraded":"2024-02-02T13:25:50.116481+0000","last_fullsized":"2024-02-02T13:25:50.116481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:54.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117649+0000","last_change":"2024-02-02T13:25:47.695963+0000","last_active":"2024-02-02T13:25:50.117649+0000","last_peered":"2024-02-02T13:25:50.117649+0000","last_clean":"2024-02-02T13:25:50.117649+0000","last_became_active":"2024-02-02T13:25:47.694554+0000","last_became_peered":"2024-02-02T13:25:47.694554+0000","last_unstale":"2024-02-02T13:25:50.117649+0000","last_undegraded":"2024-02-02T13:25:50.117649+0000","last_fullsized":"2024-02-02T13:25:50.117649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:33:06.884874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166845+0000","last_change":"2024-02-02T13:25:48.180200+0000","last_active":"2024-02-02T13:25:49.166845+0000","last_peered":"2024-02-02T13:25:49.166845+0000","last_clean":"2024-02-02T13:25:49.166845+0000","last_became_active":"2024-02-02T13:25:48.177746+0000","last_became_peered":"2024-02-02T13:25:48.177746+0000","last_unstale":"2024-02-02T13:25:49.166845+0000","last_undegraded":"2024-02-02T13:25:49.166845+0000","last_fullsized":"2024-02-02T13:25:49.166845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165691+0000","last_change":"2024-02-02T13:25:47.680998+0000","last_active":"2024-02-02T13:25:49.165691+0000","last_peered":"2024-02-02T13:25:49.165691+0000","last_clean":"2024-02-02T13:25:49.165691+0000","last_became_active":"2024-02-02T13:25:47.680388+0000","last_became_peered":"2024-02-02T13:25:47.680388+0000","last_unstale":"2024-02-02T13:25:49.165691+0000","last_undegraded":"2024-02-02T13:25:49.165691+0000","last_fullsized":"2024-02-02T13:25:49.165691+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116800+0000","last_change":"2024-02-02T13:25:48.172937+0000","last_active":"2024-02-02T13:25:50.116800+0000","last_peered":"2024-02-02T13:25:50.116800+0000","last_clean":"2024-02-02T13:25:50.116800+0000","last_became_active":"2024-02-02T13:25:48.168179+0000","last_became_peered":"2024-02-02T13:25:48.168179+0000","last_unstale":"2024-02-02T13:25:50.116800+0000","last_undegraded":"2024-02-02T13:25:50.116800+0000","last_fullsized":"2024-02-02T13:25:50.116800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:46.925303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488754+0000","last_change":"2024-02-02T13:25:47.680391+0000","last_active":"2024-02-02T13:25:49.488754+0000","last_peered":"2024-02-02T13:25:49.488754+0000","last_clean":"2024-02-02T13:25:49.488754+0000","last_became_active":"2024-02-02T13:25:47.680155+0000","last_became_peered":"2024-02-02T13:25:47.680155+0000","last_unstale":"2024-02-02T13:25:49.488754+0000","last_undegraded":"2024-02-02T13:25:49.488754+0000","last_fullsized":"2024-02-02T13:25:49.488754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166055+0000","last_change":"2024-02-02T13:25:48.174963+0000","last_active":"2024-02-02T13:25:49.166055+0000","last_peered":"2024-02-02T13:25:49.166055+0000","last_clean":"2024-02-02T13:25:49.166055+0000","last_became_active":"2024-02-02T13:25:48.173205+0000","last_became_peered":"2024-02-02T13:25:48.173205+0000","last_unstale":"2024-02-02T13:25:49.166055+0000","last_undegraded":"2024-02-02T13:25:49.166055+0000","last_fullsized":"2024-02-02T13:25:49.166055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.863472+0000","last_change":"2024-02-02T13:25:47.691797+0000","last_active":"2024-02-02T13:25:49.863472+0000","last_peered":"2024-02-02T13:25:49.863472+0000","last_clean":"2024-02-02T13:25:49.863472+0000","last_became_active":"2024-02-02T13:25:47.690587+0000","last_became_peered":"2024-02-02T13:25:47.690587+0000","last_unstale":"2024-02-02T13:25:49.863472+0000","last_undegraded":"2024-02-02T13:25:49.863472+0000","last_fullsized":"2024-02-02T13:25:49.863472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:45.912896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860305+0000","last_change":"2024-02-02T13:25:48.169133+0000","last_active":"2024-02-02T13:25:49.860305+0000","last_peered":"2024-02-02T13:25:49.860305+0000","last_clean":"2024-02-02T13:25:49.860305+0000","last_became_active":"2024-02-02T13:25:48.168905+0000","last_became_peered":"2024-02-02T13:25:48.168905+0000","last_unstale":"2024-02-02T13:25:49.860305+0000","last_undegraded":"2024-02-02T13:25:49.860305+0000","last_fullsized":"2024-02-02T13:25:49.860305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488779+0000","last_change":"2024-02-02T13:25:47.684170+0000","last_active":"2024-02-02T13:25:49.488779+0000","last_peered":"2024-02-02T13:25:49.488779+0000","last_clean":"2024-02-02T13:25:49.488779+0000","last_became_active":"2024-02-02T13:25:47.683723+0000","last_became_peered":"2024-02-02T13:25:47.683723+0000","last_unstale":"2024-02-02T13:25:49.488779+0000","last_undegraded":"2024-02-02T13:25:49.488779+0000","last_fullsized":"2024-02-02T13:25:49.488779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489489+0000","last_change":"2024-02-02T13:25:48.176549+0000","last_active":"2024-02-02T13:25:49.489489+0000","last_peered":"2024-02-02T13:25:49.489489+0000","last_clean":"2024-02-02T13:25:49.489489+0000","last_became_active":"2024-02-02T13:25:48.175881+0000","last_became_peered":"2024-02-02T13:25:48.175881+0000","last_unstale":"2024-02-02T13:25:49.489489+0000","last_undegraded":"2024-02-02T13:25:49.489489+0000","last_fullsized":"2024-02-02T13:25:49.489489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860803+0000","last_change":"2024-02-02T13:25:47.694821+0000","last_active":"2024-02-02T13:25:49.860803+0000","last_peered":"2024-02-02T13:25:49.860803+0000","last_clean":"2024-02-02T13:25:49.860803+0000","last_became_active":"2024-02-02T13:25:47.694494+0000","last_became_peered":"2024-02-02T13:25:47.694494+0000","last_unstale":"2024-02-02T13:25:49.860803+0000","last_undegraded":"2024-02-02T13:25:49.860803+0000","last_fullsized":"2024-02-02T13:25:49.860803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:38:08.332585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860535+0000","last_change":"2024-02-02T13:25:48.172635+0000","last_active":"2024-02-02T13:25:49.860535+0000","last_peered":"2024-02-02T13:25:49.860535+0000","last_clean":"2024-02-02T13:25:49.860535+0000","last_became_active":"2024-02-02T13:25:48.172399+0000","last_became_peered":"2024-02-02T13:25:48.172399+0000","last_unstale":"2024-02-02T13:25:49.860535+0000","last_undegraded":"2024-02-02T13:25:49.860535+0000","last_fullsized":"2024-02-02T13:25:49.860535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489209+0000","last_change":"2024-02-02T13:25:47.681284+0000","last_active":"2024-02-02T13:25:49.489209+0000","last_peered":"2024-02-02T13:25:49.489209+0000","last_clean":"2024-02-02T13:25:49.489209+0000","last_became_active":"2024-02-02T13:25:47.681059+0000","last_became_peered":"2024-02-02T13:25:47.681059+0000","last_unstale":"2024-02-02T13:25:49.489209+0000","last_undegraded":"2024-02-02T13:25:49.489209+0000","last_fullsized":"2024-02-02T13:25:49.489209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164426+0000","last_change":"2024-02-02T13:25:48.169371+0000","last_active":"2024-02-02T13:25:49.164426+0000","last_peered":"2024-02-02T13:25:49.164426+0000","last_clean":"2024-02-02T13:25:49.164426+0000","last_became_active":"2024-02-02T13:25:48.168848+0000","last_became_peered":"2024-02-02T13:25:48.168848+0000","last_unstale":"2024-02-02T13:25:49.164426+0000","last_undegraded":"2024-02-02T13:25:49.164426+0000","last_fullsized":"2024-02-02T13:25:49.164426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117879+0000","last_change":"2024-02-02T13:25:47.693995+0000","last_active":"2024-02-02T13:25:50.117879+0000","last_peered":"2024-02-02T13:25:50.117879+0000","last_clean":"2024-02-02T13:25:50.117879+0000","last_became_active":"2024-02-02T13:25:47.693730+0000","last_became_peered":"2024-02-02T13:25:47.693730+0000","last_unstale":"2024-02-02T13:25:50.117879+0000","last_undegraded":"2024-02-02T13:25:50.117879+0000","last_fullsized":"2024-02-02T13:25:50.117879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:42:28.175905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117228+0000","last_change":"2024-02-02T13:25:48.178674+0000","last_active":"2024-02-02T13:25:50.117228+0000","last_peered":"2024-02-02T13:25:50.117228+0000","last_clean":"2024-02-02T13:25:50.117228+0000","last_became_active":"2024-02-02T13:25:48.178422+0000","last_became_peered":"2024-02-02T13:25:48.178422+0000","last_unstale":"2024-02-02T13:25:50.117228+0000","last_undegraded":"2024-02-02T13:25:50.117228+0000","last_fullsized":"2024-02-02T13:25:50.117228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165582+0000","last_change":"2024-02-02T13:25:47.693863+0000","last_active":"2024-02-02T13:25:49.165582+0000","last_peered":"2024-02-02T13:25:49.165582+0000","last_clean":"2024-02-02T13:25:49.165582+0000","last_became_active":"2024-02-02T13:25:47.693607+0000","last_became_peered":"2024-02-02T13:25:47.693607+0000","last_unstale":"2024-02-02T13:25:49.165582+0000","last_undegraded":"2024-02-02T13:25:49.165582+0000","last_fullsized":"2024-02-02T13:25:49.165582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:53.778705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.167167+0000","last_change":"2024-02-02T13:25:48.180580+0000","last_active":"2024-02-02T13:25:49.167167+0000","last_peered":"2024-02-02T13:25:49.167167+0000","last_clean":"2024-02-02T13:25:49.167167+0000","last_became_active":"2024-02-02T13:25:48.179944+0000","last_became_peered":"2024-02-02T13:25:48.179944+0000","last_unstale":"2024-02-02T13:25:49.167167+0000","last_undegraded":"2024-02-02T13:25:49.167167+0000","last_fullsized":"2024-02-02T13:25:49.167167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:07.573173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489239+0000","last_change":"2024-02-02T13:25:47.697550+0000","last_active":"2024-02-02T13:25:49.489239+0000","last_peered":"2024-02-02T13:25:49.489239+0000","last_clean":"2024-02-02T13:25:49.489239+0000","last_became_active":"2024-02-02T13:25:47.697173+0000","last_became_peered":"2024-02-02T13:25:47.697173+0000","last_unstale":"2024-02-02T13:25:49.489239+0000","last_undegraded":"2024-02-02T13:25:49.489239+0000","last_fullsized":"2024-02-02T13:25:49.489239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-02-02T13:25:48.165125+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:25:48.165125+0000","last_peered":"2024-02-02T13:25:48.165125+0000","last_clean":"2024-02-02T13:25:48.165125+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:25:48.165125+0000","last_undegraded":"2024-02-02T13:25:48.165125+0000","last_fullsized":"2024-02-02T13:25:48.165125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257807,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.024},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3819999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.98999999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.5549999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.155},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.266}]},{"osd":5,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.639},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":3.6840000000000002,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.331}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.579}]},{"osd":2,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.357},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.6180000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.26},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.47}]},{"osd":5,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.6579999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.5229999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.357},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.139}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.385}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.046}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.751},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.23}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.111}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.1659999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.204}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.01}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.419},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0469999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.365}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.577},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.526}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195162,"statfs":{"total":200000000,"available":199846679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.302},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.123},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1779999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.585,"5min":3.585,"15min":3.585},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.355},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.554}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.5009999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":1.454},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.411}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.88}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.1830000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.226}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.5329999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.585}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.038}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.268},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.383}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.532},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.4119999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.601},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.512}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.452}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.8070000000000004,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":1.006},{"interface":"front","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":10.781000000000001,"5min":10.781000000000001,"15min":10.781000000000001},"last":1.0609999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.8410000000000002,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.395}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":10.651,"5min":10.651,"15min":10.651},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.7869999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.589}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.238},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":10.708,"5min":10.708,"15min":10.708},"last":1.48}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":10.74,"5min":10.74,"15min":10.74},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.528}]}]}],"pool_statfs":[]}} 2024-02-02T13:25:53.081 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:25:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.330+0000 7f8f12c00640 1 Processor -- start 2024-02-02T13:25:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.330+0000 7f8f12c00640 1 -- start start 2024-02-02T13:25:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f12c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f0c15a010 0x7f8f0c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f12c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f0c0b0700 0x7f8f0c0b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f12c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0cc7c0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f12c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b10b0 con 0x7f8f0c15a010 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f12c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0ce930 con 0x7f8f0c0b0700 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f0c15a010 0x7f8f0c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f0c15a010 0x7f8f0c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56532/0 (socket says 172.21.15.91:56532) 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 -- 172.21.15.91:0/1670020154 learned_addr learned my addr 172.21.15.91:0/1670020154 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 -- 172.21.15.91:0/1670020154 >> v2:172.21.15.113:3300/0 conn(0x7f8f0c0b0700 msgr2=0x7f8f0c0b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 --2- 172.21.15.91:0/1670020154 >> v2:172.21.15.113:3300/0 conn(0x7f8f0c0b0700 0x7f8f0c0b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 -- 172.21.15.91:0/1670020154 >> v1:172.21.15.113:6789/0 conn(0x7f8f0c0ce130 legacy=0x7f8f0c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.332+0000 7f8f0be00640 1 -- 172.21.15.91:0/1670020154 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0cd640 con 0x7f8f0c15a010 2024-02-02T13:25:53.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.333+0000 7f8f0be00640 1 --2- 172.21.15.91:0/1670020154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c15a010 0x7f8f0c0c9670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cec10 tx=0x7f8efc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bb25640e3e5324 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f0b400640 1 -- 172.21.15.91:0/1670020154 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc031870 con 0x7f8f0c15a010 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f0b400640 1 -- 172.21.15.91:0/1670020154 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0319d0 con 0x7f8f0c15a010 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f0b400640 1 -- 172.21.15.91:0/1670020154 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc031d60 con 0x7f8f0c15a010 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f12c00640 1 -- 172.21.15.91:0/1670020154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c15a010 msgr2=0x7f8f0c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f12c00640 1 --2- 172.21.15.91:0/1670020154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c15a010 0x7f8f0c0c9670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cec10 tx=0x7f8efc02b910 comp rx=0 tx=0).stop 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.334+0000 7f8f12c00640 1 -- 172.21.15.91:0/1670020154 shutdown_connections 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.335+0000 7f8f12c00640 1 --2- 172.21.15.91:0/1670020154 >> v2:172.21.15.113:3300/0 conn(0x7f8f0c0b0700 0x7f8f0c0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.335+0000 7f8f12c00640 1 --2- 172.21.15.91:0/1670020154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c15a010 0x7f8f0c0c9670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.335+0000 7f8f12c00640 1 -- 172.21.15.91:0/1670020154 >> 172.21.15.91:0/1670020154 conn(0x7f8f0c0cec80 msgr2=0x7f8f0c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:53.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.335+0000 7f8f12c00640 1 -- 172.21.15.91:0/1670020154 shutdown_connections 2024-02-02T13:25:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.335+0000 7f8f12c00640 1 -- 172.21.15.91:0/1670020154 wait complete. 2024-02-02T13:25:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.336+0000 7f8f12c00640 1 Processor -- start 2024-02-02T13:25:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.336+0000 7f8f12c00640 1 -- start start 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f0c0b0700 0x7f8f0c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f0c15a010 0x7f8f0c0ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b2990 con 0x7f8f0c0ce130 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c15df10 con 0x7f8f0c0b0700 2024-02-02T13:25:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.337+0000 7f8f12c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c15e6c0 con 0x7f8f0c15a010 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56538/0 (socket says 172.21.15.91:56538) 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 -- 172.21.15.91:0/2151213989 learned_addr learned my addr 172.21.15.91:0/2151213989 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f10c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f0c15a010 0x7f8f0c0ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f0be00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f0c0b0700 0x7f8f0c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 -- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f0c15a010 msgr2=0x7f8f0c0ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f0c15a010 0x7f8f0c0ac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 -- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f0c0b0700 msgr2=0x7f8f0c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f0c0b0700 0x7f8f0c15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 -- 172.21.15.91:0/2151213989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0094d0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.338+0000 7f8f03e00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400c410 tx=0x7f8ef400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.339+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40035f0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.339+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef4003750 con 0x7f8f0c0ce130 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.339+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4003b40 con 0x7f8f0c0ce130 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.339+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0b1d10 con 0x7f8f0c0ce130 2024-02-02T13:25:53.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.339+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c0b22d0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.340+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c066cf0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.341+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ef4003ca0 con 0x7f8f0c0ce130 2024-02-02T13:25:53.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.341+0000 7f8f08c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 0x7f8ee406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.342+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ef4087b50 con 0x7f8f0c0ce130 2024-02-02T13:25:53.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.342+0000 7f8f0be00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 0x7f8ee406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.342+0000 7f8f0be00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 0x7f8ee406c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8efc036000 tx=0x7f8efc03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.346+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ef4053140 con 0x7f8f0c0ce130 2024-02-02T13:25:53.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.533+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f0c06cb70 con 0x7f8ee406a430 2024-02-02T13:25:53.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.555+0000 7f8f08c00640 1 -- 172.21.15.91:0/2151213989 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282015 (secure 0 0 0) 0x7f8efc031d10 con 0x7f8ee406a430 2024-02-02T13:25:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:53.561 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 msgr2=0x7f8ee406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 0x7f8ee406c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8efc036000 tx=0x7f8efc03c040 comp rx=0 tx=0).stop 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 msgr2=0x7f8f0c15cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400c410 tx=0x7f8ef400c8d0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 shutdown_connections 2024-02-02T13:25:53.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ee406a430 0x7f8ee406c8f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f0c15a010 0x7f8f0c0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f0c0ce130 0x7f8f0c15cd20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 --2- 172.21.15.91:0/2151213989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f0c0b0700 0x7f8f0c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 >> 172.21.15.91:0/2151213989 conn(0x7f8f0c0cec80 msgr2=0x7f8f0c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 shutdown_connections 2024-02-02T13:25:53.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.561+0000 7f8f12c00640 1 -- 172.21.15.91:0/2151213989 wait complete. 2024-02-02T13:25:53.580 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-02-02T13:25:52.258102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560160,"statfs":{"total":1600000000,"available":1597604736,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.605731"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489225+0000","last_change":"2024-02-02T13:25:48.181916+0000","last_active":"2024-02-02T13:25:49.489225+0000","last_peered":"2024-02-02T13:25:49.489225+0000","last_clean":"2024-02-02T13:25:49.489225+0000","last_became_active":"2024-02-02T13:25:48.181645+0000","last_became_peered":"2024-02-02T13:25:48.181645+0000","last_unstale":"2024-02-02T13:25:49.489225+0000","last_undegraded":"2024-02-02T13:25:49.489225+0000","last_fullsized":"2024-02-02T13:25:49.489225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164925+0000","last_change":"2024-02-02T13:25:48.159561+0000","last_active":"2024-02-02T13:25:49.164925+0000","last_peered":"2024-02-02T13:25:49.164925+0000","last_clean":"2024-02-02T13:25:49.164925+0000","last_became_active":"2024-02-02T13:25:48.159272+0000","last_became_peered":"2024-02-02T13:25:48.159272+0000","last_unstale":"2024-02-02T13:25:49.164925+0000","last_undegraded":"2024-02-02T13:25:49.164925+0000","last_fullsized":"2024-02-02T13:25:49.164925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165768+0000","last_change":"2024-02-02T13:25:48.177400+0000","last_active":"2024-02-02T13:25:49.165768+0000","last_peered":"2024-02-02T13:25:49.165768+0000","last_clean":"2024-02-02T13:25:49.165768+0000","last_became_active":"2024-02-02T13:25:48.172120+0000","last_became_peered":"2024-02-02T13:25:48.172120+0000","last_unstale":"2024-02-02T13:25:49.165768+0000","last_undegraded":"2024-02-02T13:25:49.165768+0000","last_fullsized":"2024-02-02T13:25:49.165768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488875+0000","last_change":"2024-02-02T13:25:48.179174+0000","last_active":"2024-02-02T13:25:49.488875+0000","last_peered":"2024-02-02T13:25:49.488875+0000","last_clean":"2024-02-02T13:25:49.488875+0000","last_became_active":"2024-02-02T13:25:48.178011+0000","last_became_peered":"2024-02-02T13:25:48.178011+0000","last_unstale":"2024-02-02T13:25:49.488875+0000","last_undegraded":"2024-02-02T13:25:49.488875+0000","last_fullsized":"2024-02-02T13:25:49.488875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116401+0000","last_change":"2024-02-02T13:25:48.165370+0000","last_active":"2024-02-02T13:25:50.116401+0000","last_peered":"2024-02-02T13:25:50.116401+0000","last_clean":"2024-02-02T13:25:50.116401+0000","last_became_active":"2024-02-02T13:25:48.165112+0000","last_became_peered":"2024-02-02T13:25:48.165112+0000","last_unstale":"2024-02-02T13:25:50.116401+0000","last_undegraded":"2024-02-02T13:25:50.116401+0000","last_fullsized":"2024-02-02T13:25:50.116401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:38:09.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164447+0000","last_change":"2024-02-02T13:25:48.170191+0000","last_active":"2024-02-02T13:25:49.164447+0000","last_peered":"2024-02-02T13:25:49.164447+0000","last_clean":"2024-02-02T13:25:49.164447+0000","last_became_active":"2024-02-02T13:25:48.169981+0000","last_became_peered":"2024-02-02T13:25:48.169981+0000","last_unstale":"2024-02-02T13:25:49.164447+0000","last_undegraded":"2024-02-02T13:25:49.164447+0000","last_fullsized":"2024-02-02T13:25:49.164447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:16.860852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166057+0000","last_change":"2024-02-02T13:25:48.183691+0000","last_active":"2024-02-02T13:25:49.166057+0000","last_peered":"2024-02-02T13:25:49.166057+0000","last_clean":"2024-02-02T13:25:49.166057+0000","last_became_active":"2024-02-02T13:25:48.173110+0000","last_became_peered":"2024-02-02T13:25:48.173110+0000","last_unstale":"2024-02-02T13:25:49.166057+0000","last_undegraded":"2024-02-02T13:25:49.166057+0000","last_fullsized":"2024-02-02T13:25:49.166057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166203+0000","last_change":"2024-02-02T13:25:48.179465+0000","last_active":"2024-02-02T13:25:49.166203+0000","last_peered":"2024-02-02T13:25:49.166203+0000","last_clean":"2024-02-02T13:25:49.166203+0000","last_became_active":"2024-02-02T13:25:48.178608+0000","last_became_peered":"2024-02-02T13:25:48.178608+0000","last_unstale":"2024-02-02T13:25:49.166203+0000","last_undegraded":"2024-02-02T13:25:49.166203+0000","last_fullsized":"2024-02-02T13:25:49.166203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:31.534284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117359+0000","last_change":"2024-02-02T13:25:48.181182+0000","last_active":"2024-02-02T13:25:50.117359+0000","last_peered":"2024-02-02T13:25:50.117359+0000","last_clean":"2024-02-02T13:25:50.117359+0000","last_became_active":"2024-02-02T13:25:48.180854+0000","last_became_peered":"2024-02-02T13:25:48.180854+0000","last_unstale":"2024-02-02T13:25:50.117359+0000","last_undegraded":"2024-02-02T13:25:50.117359+0000","last_fullsized":"2024-02-02T13:25:50.117359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488497+0000","last_change":"2024-02-02T13:25:48.174383+0000","last_active":"2024-02-02T13:25:49.488497+0000","last_peered":"2024-02-02T13:25:49.488497+0000","last_clean":"2024-02-02T13:25:49.488497+0000","last_became_active":"2024-02-02T13:25:48.174168+0000","last_became_peered":"2024-02-02T13:25:48.174168+0000","last_unstale":"2024-02-02T13:25:49.488497+0000","last_undegraded":"2024-02-02T13:25:49.488497+0000","last_fullsized":"2024-02-02T13:25:49.488497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:59:32.385556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.167871+0000","last_change":"2024-02-02T13:25:48.171650+0000","last_active":"2024-02-02T13:25:49.167871+0000","last_peered":"2024-02-02T13:25:49.167871+0000","last_clean":"2024-02-02T13:25:49.167871+0000","last_became_active":"2024-02-02T13:25:48.171411+0000","last_became_peered":"2024-02-02T13:25:48.171411+0000","last_unstale":"2024-02-02T13:25:49.167871+0000","last_undegraded":"2024-02-02T13:25:49.167871+0000","last_fullsized":"2024-02-02T13:25:49.167871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165813+0000","last_change":"2024-02-02T13:25:48.178242+0000","last_active":"2024-02-02T13:25:49.165813+0000","last_peered":"2024-02-02T13:25:49.165813+0000","last_clean":"2024-02-02T13:25:49.165813+0000","last_became_active":"2024-02-02T13:25:48.174688+0000","last_became_peered":"2024-02-02T13:25:48.174688+0000","last_unstale":"2024-02-02T13:25:49.165813+0000","last_undegraded":"2024-02-02T13:25:49.165813+0000","last_fullsized":"2024-02-02T13:25:49.165813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116286+0000","last_change":"2024-02-02T13:25:48.173222+0000","last_active":"2024-02-02T13:25:50.116286+0000","last_peered":"2024-02-02T13:25:50.116286+0000","last_clean":"2024-02-02T13:25:50.116286+0000","last_became_active":"2024-02-02T13:25:48.172662+0000","last_became_peered":"2024-02-02T13:25:48.172662+0000","last_unstale":"2024-02-02T13:25:50.116286+0000","last_undegraded":"2024-02-02T13:25:50.116286+0000","last_fullsized":"2024-02-02T13:25:50.116286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:58.893964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164815+0000","last_change":"2024-02-02T13:25:48.159361+0000","last_active":"2024-02-02T13:25:49.164815+0000","last_peered":"2024-02-02T13:25:49.164815+0000","last_clean":"2024-02-02T13:25:49.164815+0000","last_became_active":"2024-02-02T13:25:48.159107+0000","last_became_peered":"2024-02-02T13:25:48.159107+0000","last_unstale":"2024-02-02T13:25:49.164815+0000","last_undegraded":"2024-02-02T13:25:49.164815+0000","last_fullsized":"2024-02-02T13:25:49.164815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165812+0000","last_change":"2024-02-02T13:25:48.175571+0000","last_active":"2024-02-02T13:25:49.165812+0000","last_peered":"2024-02-02T13:25:49.165812+0000","last_clean":"2024-02-02T13:25:49.165812+0000","last_became_active":"2024-02-02T13:25:48.175286+0000","last_became_peered":"2024-02-02T13:25:48.175286+0000","last_unstale":"2024-02-02T13:25:49.165812+0000","last_undegraded":"2024-02-02T13:25:49.165812+0000","last_fullsized":"2024-02-02T13:25:49.165812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.168095+0000","last_change":"2024-02-02T13:25:48.177514+0000","last_active":"2024-02-02T13:25:49.168095+0000","last_peered":"2024-02-02T13:25:49.168095+0000","last_clean":"2024-02-02T13:25:49.168095+0000","last_became_active":"2024-02-02T13:25:48.173010+0000","last_became_peered":"2024-02-02T13:25:48.173010+0000","last_unstale":"2024-02-02T13:25:49.168095+0000","last_undegraded":"2024-02-02T13:25:49.168095+0000","last_fullsized":"2024-02-02T13:25:49.168095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:47.393543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166194+0000","last_change":"2024-02-02T13:25:48.184313+0000","last_active":"2024-02-02T13:25:49.166194+0000","last_peered":"2024-02-02T13:25:49.166194+0000","last_clean":"2024-02-02T13:25:49.166194+0000","last_became_active":"2024-02-02T13:25:48.182831+0000","last_became_peered":"2024-02-02T13:25:48.182831+0000","last_unstale":"2024-02-02T13:25:49.166194+0000","last_undegraded":"2024-02-02T13:25:49.166194+0000","last_fullsized":"2024-02-02T13:25:49.166194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164615+0000","last_change":"2024-02-02T13:25:48.168369+0000","last_active":"2024-02-02T13:25:49.164615+0000","last_peered":"2024-02-02T13:25:49.164615+0000","last_clean":"2024-02-02T13:25:49.164615+0000","last_became_active":"2024-02-02T13:25:48.168111+0000","last_became_peered":"2024-02-02T13:25:48.168111+0000","last_unstale":"2024-02-02T13:25:49.164615+0000","last_undegraded":"2024-02-02T13:25:49.164615+0000","last_fullsized":"2024-02-02T13:25:49.164615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:43.375840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164167+0000","last_change":"2024-02-02T13:25:48.169491+0000","last_active":"2024-02-02T13:25:49.164167+0000","last_peered":"2024-02-02T13:25:49.164167+0000","last_clean":"2024-02-02T13:25:49.164167+0000","last_became_active":"2024-02-02T13:25:48.169155+0000","last_became_peered":"2024-02-02T13:25:48.169155+0000","last_unstale":"2024-02-02T13:25:49.164167+0000","last_undegraded":"2024-02-02T13:25:49.164167+0000","last_fullsized":"2024-02-02T13:25:49.164167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:22.735006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116629+0000","last_change":"2024-02-02T13:25:48.166670+0000","last_active":"2024-02-02T13:25:50.116629+0000","last_peered":"2024-02-02T13:25:50.116629+0000","last_clean":"2024-02-02T13:25:50.116629+0000","last_became_active":"2024-02-02T13:25:48.166395+0000","last_became_peered":"2024-02-02T13:25:48.166395+0000","last_unstale":"2024-02-02T13:25:50.116629+0000","last_undegraded":"2024-02-02T13:25:50.116629+0000","last_fullsized":"2024-02-02T13:25:50.116629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:57:38.246646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116810+0000","last_change":"2024-02-02T13:25:48.183087+0000","last_active":"2024-02-02T13:25:50.116810+0000","last_peered":"2024-02-02T13:25:50.116810+0000","last_clean":"2024-02-02T13:25:50.116810+0000","last_became_active":"2024-02-02T13:25:48.182503+0000","last_became_peered":"2024-02-02T13:25:48.182503+0000","last_unstale":"2024-02-02T13:25:50.116810+0000","last_undegraded":"2024-02-02T13:25:50.116810+0000","last_fullsized":"2024-02-02T13:25:50.116810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:00.369366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117753+0000","last_change":"2024-02-02T13:25:48.177292+0000","last_active":"2024-02-02T13:25:50.117753+0000","last_peered":"2024-02-02T13:25:50.117753+0000","last_clean":"2024-02-02T13:25:50.117753+0000","last_became_active":"2024-02-02T13:25:48.176979+0000","last_became_peered":"2024-02-02T13:25:48.176979+0000","last_unstale":"2024-02-02T13:25:50.117753+0000","last_undegraded":"2024-02-02T13:25:50.117753+0000","last_fullsized":"2024-02-02T13:25:50.117753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:39.214130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860870+0000","last_change":"2024-02-02T13:25:48.167660+0000","last_active":"2024-02-02T13:25:49.860870+0000","last_peered":"2024-02-02T13:25:49.860870+0000","last_clean":"2024-02-02T13:25:49.860870+0000","last_became_active":"2024-02-02T13:25:48.167403+0000","last_became_peered":"2024-02-02T13:25:48.167403+0000","last_unstale":"2024-02-02T13:25:49.860870+0000","last_undegraded":"2024-02-02T13:25:49.860870+0000","last_fullsized":"2024-02-02T13:25:49.860870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165252+0000","last_change":"2024-02-02T13:25:48.174865+0000","last_active":"2024-02-02T13:25:49.165252+0000","last_peered":"2024-02-02T13:25:49.165252+0000","last_clean":"2024-02-02T13:25:49.165252+0000","last_became_active":"2024-02-02T13:25:48.174397+0000","last_became_peered":"2024-02-02T13:25:48.174397+0000","last_unstale":"2024-02-02T13:25:49.165252+0000","last_undegraded":"2024-02-02T13:25:49.165252+0000","last_fullsized":"2024-02-02T13:25:49.165252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:47.393543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488587+0000","last_change":"2024-02-02T13:25:48.171501+0000","last_active":"2024-02-02T13:25:49.488587+0000","last_peered":"2024-02-02T13:25:49.488587+0000","last_clean":"2024-02-02T13:25:49.488587+0000","last_became_active":"2024-02-02T13:25:48.171282+0000","last_became_peered":"2024-02-02T13:25:48.171282+0000","last_unstale":"2024-02-02T13:25:49.488587+0000","last_undegraded":"2024-02-02T13:25:49.488587+0000","last_fullsized":"2024-02-02T13:25:49.488587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:06:57.101739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164552+0000","last_change":"2024-02-02T13:25:48.171403+0000","last_active":"2024-02-02T13:25:49.164552+0000","last_peered":"2024-02-02T13:25:49.164552+0000","last_clean":"2024-02-02T13:25:49.164552+0000","last_became_active":"2024-02-02T13:25:48.171150+0000","last_became_peered":"2024-02-02T13:25:48.171150+0000","last_unstale":"2024-02-02T13:25:49.164552+0000","last_undegraded":"2024-02-02T13:25:49.164552+0000","last_fullsized":"2024-02-02T13:25:49.164552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:44:49.165370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488987+0000","last_change":"2024-02-02T13:25:48.177213+0000","last_active":"2024-02-02T13:25:49.488987+0000","last_peered":"2024-02-02T13:25:49.488987+0000","last_clean":"2024-02-02T13:25:49.488987+0000","last_became_active":"2024-02-02T13:25:48.176871+0000","last_became_peered":"2024-02-02T13:25:48.176871+0000","last_unstale":"2024-02-02T13:25:49.488987+0000","last_undegraded":"2024-02-02T13:25:49.488987+0000","last_fullsized":"2024-02-02T13:25:49.488987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165359+0000","last_change":"2024-02-02T13:25:48.177231+0000","last_active":"2024-02-02T13:25:49.165359+0000","last_peered":"2024-02-02T13:25:49.165359+0000","last_clean":"2024-02-02T13:25:49.165359+0000","last_became_active":"2024-02-02T13:25:48.171552+0000","last_became_peered":"2024-02-02T13:25:48.171552+0000","last_unstale":"2024-02-02T13:25:49.165359+0000","last_undegraded":"2024-02-02T13:25:49.165359+0000","last_fullsized":"2024-02-02T13:25:49.165359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117622+0000","last_change":"2024-02-02T13:25:48.176140+0000","last_active":"2024-02-02T13:25:50.117622+0000","last_peered":"2024-02-02T13:25:50.117622+0000","last_clean":"2024-02-02T13:25:50.117622+0000","last_became_active":"2024-02-02T13:25:48.175915+0000","last_became_peered":"2024-02-02T13:25:48.175915+0000","last_unstale":"2024-02-02T13:25:50.117622+0000","last_undegraded":"2024-02-02T13:25:50.117622+0000","last_fullsized":"2024-02-02T13:25:50.117622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166585+0000","last_change":"2024-02-02T13:25:48.179543+0000","last_active":"2024-02-02T13:25:49.166585+0000","last_peered":"2024-02-02T13:25:49.166585+0000","last_clean":"2024-02-02T13:25:49.166585+0000","last_became_active":"2024-02-02T13:25:48.177450+0000","last_became_peered":"2024-02-02T13:25:48.177450+0000","last_unstale":"2024-02-02T13:25:49.166585+0000","last_undegraded":"2024-02-02T13:25:49.166585+0000","last_fullsized":"2024-02-02T13:25:49.166585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489398+0000","last_change":"2024-02-02T13:25:48.174355+0000","last_active":"2024-02-02T13:25:49.489398+0000","last_peered":"2024-02-02T13:25:49.489398+0000","last_clean":"2024-02-02T13:25:49.489398+0000","last_became_active":"2024-02-02T13:25:48.174079+0000","last_became_peered":"2024-02-02T13:25:48.174079+0000","last_unstale":"2024-02-02T13:25:49.489398+0000","last_undegraded":"2024-02-02T13:25:49.489398+0000","last_fullsized":"2024-02-02T13:25:49.489398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:54.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166324+0000","last_change":"2024-02-02T13:25:48.184490+0000","last_active":"2024-02-02T13:25:49.166324+0000","last_peered":"2024-02-02T13:25:49.166324+0000","last_clean":"2024-02-02T13:25:49.166324+0000","last_became_active":"2024-02-02T13:25:48.184042+0000","last_became_peered":"2024-02-02T13:25:48.184042+0000","last_unstale":"2024-02-02T13:25:49.166324+0000","last_undegraded":"2024-02-02T13:25:49.166324+0000","last_fullsized":"2024-02-02T13:25:49.166324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:18.139345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860671+0000","last_change":"2024-02-02T13:25:48.170495+0000","last_active":"2024-02-02T13:25:49.860671+0000","last_peered":"2024-02-02T13:25:49.860671+0000","last_clean":"2024-02-02T13:25:49.860671+0000","last_became_active":"2024-02-02T13:25:48.170286+0000","last_became_peered":"2024-02-02T13:25:48.170286+0000","last_unstale":"2024-02-02T13:25:49.860671+0000","last_undegraded":"2024-02-02T13:25:49.860671+0000","last_fullsized":"2024-02-02T13:25:49.860671+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489206+0000","last_change":"2024-02-02T13:25:47.692263+0000","last_active":"2024-02-02T13:25:49.489206+0000","last_peered":"2024-02-02T13:25:49.489206+0000","last_clean":"2024-02-02T13:25:49.489206+0000","last_became_active":"2024-02-02T13:25:47.691984+0000","last_became_peered":"2024-02-02T13:25:47.691984+0000","last_unstale":"2024-02-02T13:25:49.489206+0000","last_undegraded":"2024-02-02T13:25:49.489206+0000","last_fullsized":"2024-02-02T13:25:49.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:45.912896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488931+0000","last_change":"2024-02-02T13:25:48.176543+0000","last_active":"2024-02-02T13:25:49.488931+0000","last_peered":"2024-02-02T13:25:49.488931+0000","last_clean":"2024-02-02T13:25:49.488931+0000","last_became_active":"2024-02-02T13:25:48.176305+0000","last_became_peered":"2024-02-02T13:25:48.176305+0000","last_unstale":"2024-02-02T13:25:49.488931+0000","last_undegraded":"2024-02-02T13:25:49.488931+0000","last_fullsized":"2024-02-02T13:25:49.488931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:33.339415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860469+0000","last_change":"2024-02-02T13:25:47.692249+0000","last_active":"2024-02-02T13:25:49.860469+0000","last_peered":"2024-02-02T13:25:49.860469+0000","last_clean":"2024-02-02T13:25:49.860469+0000","last_became_active":"2024-02-02T13:25:47.692020+0000","last_became_peered":"2024-02-02T13:25:47.692020+0000","last_unstale":"2024-02-02T13:25:49.860469+0000","last_undegraded":"2024-02-02T13:25:49.860469+0000","last_fullsized":"2024-02-02T13:25:49.860469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116986+0000","last_change":"2024-02-02T13:25:48.183243+0000","last_active":"2024-02-02T13:25:50.116986+0000","last_peered":"2024-02-02T13:25:50.116986+0000","last_clean":"2024-02-02T13:25:50.116986+0000","last_became_active":"2024-02-02T13:25:48.182845+0000","last_became_peered":"2024-02-02T13:25:48.182845+0000","last_unstale":"2024-02-02T13:25:50.116986+0000","last_undegraded":"2024-02-02T13:25:50.116986+0000","last_fullsized":"2024-02-02T13:25:50.116986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164991+0000","last_change":"2024-02-02T13:25:47.689523+0000","last_active":"2024-02-02T13:25:49.164991+0000","last_peered":"2024-02-02T13:25:49.164991+0000","last_clean":"2024-02-02T13:25:49.164991+0000","last_became_active":"2024-02-02T13:25:47.689309+0000","last_became_peered":"2024-02-02T13:25:47.689309+0000","last_unstale":"2024-02-02T13:25:49.164991+0000","last_undegraded":"2024-02-02T13:25:49.164991+0000","last_fullsized":"2024-02-02T13:25:49.164991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:35.117418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488666+0000","last_change":"2024-02-02T13:25:48.181603+0000","last_active":"2024-02-02T13:25:49.488666+0000","last_peered":"2024-02-02T13:25:49.488666+0000","last_clean":"2024-02-02T13:25:49.488666+0000","last_became_active":"2024-02-02T13:25:48.181395+0000","last_became_peered":"2024-02-02T13:25:48.181395+0000","last_unstale":"2024-02-02T13:25:49.488666+0000","last_undegraded":"2024-02-02T13:25:49.488666+0000","last_fullsized":"2024-02-02T13:25:49.488666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:07.573173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116603+0000","last_change":"2024-02-02T13:25:47.685276+0000","last_active":"2024-02-02T13:25:50.116603+0000","last_peered":"2024-02-02T13:25:50.116603+0000","last_clean":"2024-02-02T13:25:50.116603+0000","last_became_active":"2024-02-02T13:25:47.685017+0000","last_became_peered":"2024-02-02T13:25:47.685017+0000","last_unstale":"2024-02-02T13:25:50.116603+0000","last_undegraded":"2024-02-02T13:25:50.116603+0000","last_fullsized":"2024-02-02T13:25:50.116603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:09:26.842297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166640+0000","last_change":"2024-02-02T13:25:48.179602+0000","last_active":"2024-02-02T13:25:49.166640+0000","last_peered":"2024-02-02T13:25:49.166640+0000","last_clean":"2024-02-02T13:25:49.166640+0000","last_became_active":"2024-02-02T13:25:48.178947+0000","last_became_peered":"2024-02-02T13:25:48.178947+0000","last_unstale":"2024-02-02T13:25:49.166640+0000","last_undegraded":"2024-02-02T13:25:49.166640+0000","last_fullsized":"2024-02-02T13:25:49.166640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:06:57.101739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117533+0000","last_change":"2024-02-02T13:25:47.693332+0000","last_active":"2024-02-02T13:25:50.117533+0000","last_peered":"2024-02-02T13:25:50.117533+0000","last_clean":"2024-02-02T13:25:50.117533+0000","last_became_active":"2024-02-02T13:25:47.691310+0000","last_became_peered":"2024-02-02T13:25:47.691310+0000","last_unstale":"2024-02-02T13:25:50.117533+0000","last_undegraded":"2024-02-02T13:25:50.117533+0000","last_fullsized":"2024-02-02T13:25:50.117533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:32.393560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165932+0000","last_change":"2024-02-02T13:25:48.171986+0000","last_active":"2024-02-02T13:25:49.165932+0000","last_peered":"2024-02-02T13:25:49.165932+0000","last_clean":"2024-02-02T13:25:49.165932+0000","last_became_active":"2024-02-02T13:25:48.171244+0000","last_became_peered":"2024-02-02T13:25:48.171244+0000","last_unstale":"2024-02-02T13:25:49.165932+0000","last_undegraded":"2024-02-02T13:25:49.165932+0000","last_fullsized":"2024-02-02T13:25:49.165932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117482+0000","last_change":"2024-02-02T13:25:47.693296+0000","last_active":"2024-02-02T13:25:50.117482+0000","last_peered":"2024-02-02T13:25:50.117482+0000","last_clean":"2024-02-02T13:25:50.117482+0000","last_became_active":"2024-02-02T13:25:47.689967+0000","last_became_peered":"2024-02-02T13:25:47.689967+0000","last_unstale":"2024-02-02T13:25:50.117482+0000","last_undegraded":"2024-02-02T13:25:50.117482+0000","last_fullsized":"2024-02-02T13:25:50.117482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:10.600882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116824+0000","last_change":"2024-02-02T13:25:48.181146+0000","last_active":"2024-02-02T13:25:50.116824+0000","last_peered":"2024-02-02T13:25:50.116824+0000","last_clean":"2024-02-02T13:25:50.116824+0000","last_became_active":"2024-02-02T13:25:48.180832+0000","last_became_peered":"2024-02-02T13:25:48.180832+0000","last_unstale":"2024-02-02T13:25:50.116824+0000","last_undegraded":"2024-02-02T13:25:50.116824+0000","last_fullsized":"2024-02-02T13:25:50.116824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:53:14.026351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165008+0000","last_change":"2024-02-02T13:25:47.686436+0000","last_active":"2024-02-02T13:25:49.165008+0000","last_peered":"2024-02-02T13:25:49.165008+0000","last_clean":"2024-02-02T13:25:49.165008+0000","last_became_active":"2024-02-02T13:25:47.686177+0000","last_became_peered":"2024-02-02T13:25:47.686177+0000","last_unstale":"2024-02-02T13:25:49.165008+0000","last_undegraded":"2024-02-02T13:25:49.165008+0000","last_fullsized":"2024-02-02T13:25:49.165008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:13.511597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489106+0000","last_change":"2024-02-02T13:25:48.179380+0000","last_active":"2024-02-02T13:25:49.489106+0000","last_peered":"2024-02-02T13:25:49.489106+0000","last_clean":"2024-02-02T13:25:49.489106+0000","last_became_active":"2024-02-02T13:25:48.178692+0000","last_became_peered":"2024-02-02T13:25:48.178692+0000","last_unstale":"2024-02-02T13:25:49.489106+0000","last_undegraded":"2024-02-02T13:25:49.489106+0000","last_fullsized":"2024-02-02T13:25:49.489106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117229+0000","last_change":"2024-02-02T13:25:47.697074+0000","last_active":"2024-02-02T13:25:50.117229+0000","last_peered":"2024-02-02T13:25:50.117229+0000","last_clean":"2024-02-02T13:25:50.117229+0000","last_became_active":"2024-02-02T13:25:47.695643+0000","last_became_peered":"2024-02-02T13:25:47.695643+0000","last_unstale":"2024-02-02T13:25:50.117229+0000","last_undegraded":"2024-02-02T13:25:50.117229+0000","last_fullsized":"2024-02-02T13:25:50.117229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:25.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166174+0000","last_change":"2024-02-02T13:25:48.178094+0000","last_active":"2024-02-02T13:25:49.166174+0000","last_peered":"2024-02-02T13:25:49.166174+0000","last_clean":"2024-02-02T13:25:49.166174+0000","last_became_active":"2024-02-02T13:25:48.174276+0000","last_became_peered":"2024-02-02T13:25:48.174276+0000","last_unstale":"2024-02-02T13:25:49.166174+0000","last_undegraded":"2024-02-02T13:25:49.166174+0000","last_fullsized":"2024-02-02T13:25:49.166174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:02.413335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164738+0000","last_change":"2024-02-02T13:25:47.693903+0000","last_active":"2024-02-02T13:25:49.164738+0000","last_peered":"2024-02-02T13:25:49.164738+0000","last_clean":"2024-02-02T13:25:49.164738+0000","last_became_active":"2024-02-02T13:25:47.693585+0000","last_became_peered":"2024-02-02T13:25:47.693585+0000","last_unstale":"2024-02-02T13:25:49.164738+0000","last_undegraded":"2024-02-02T13:25:49.164738+0000","last_fullsized":"2024-02-02T13:25:49.164738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:18:30.346290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164650+0000","last_change":"2024-02-02T13:25:48.169686+0000","last_active":"2024-02-02T13:25:49.164650+0000","last_peered":"2024-02-02T13:25:49.164650+0000","last_clean":"2024-02-02T13:25:49.164650+0000","last_became_active":"2024-02-02T13:25:48.168577+0000","last_became_peered":"2024-02-02T13:25:48.168577+0000","last_unstale":"2024-02-02T13:25:49.164650+0000","last_undegraded":"2024-02-02T13:25:49.164650+0000","last_fullsized":"2024-02-02T13:25:49.164650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860614+0000","last_change":"2024-02-02T13:25:47.689244+0000","last_active":"2024-02-02T13:25:49.860614+0000","last_peered":"2024-02-02T13:25:49.860614+0000","last_clean":"2024-02-02T13:25:49.860614+0000","last_became_active":"2024-02-02T13:25:47.688869+0000","last_became_peered":"2024-02-02T13:25:47.688869+0000","last_unstale":"2024-02-02T13:25:49.860614+0000","last_undegraded":"2024-02-02T13:25:49.860614+0000","last_fullsized":"2024-02-02T13:25:49.860614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165464+0000","last_change":"2024-02-02T13:25:48.171367+0000","last_active":"2024-02-02T13:25:49.165464+0000","last_peered":"2024-02-02T13:25:49.165464+0000","last_clean":"2024-02-02T13:25:49.165464+0000","last_became_active":"2024-02-02T13:25:48.171165+0000","last_became_peered":"2024-02-02T13:25:48.171165+0000","last_unstale":"2024-02-02T13:25:49.165464+0000","last_undegraded":"2024-02-02T13:25:49.165464+0000","last_fullsized":"2024-02-02T13:25:49.165464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:40:49.822973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165896+0000","last_change":"2024-02-02T13:25:47.678863+0000","last_active":"2024-02-02T13:25:49.165896+0000","last_peered":"2024-02-02T13:25:49.165896+0000","last_clean":"2024-02-02T13:25:49.165896+0000","last_became_active":"2024-02-02T13:25:47.677772+0000","last_became_peered":"2024-02-02T13:25:47.677772+0000","last_unstale":"2024-02-02T13:25:49.165896+0000","last_undegraded":"2024-02-02T13:25:49.165896+0000","last_fullsized":"2024-02-02T13:25:49.165896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:57:37.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165624+0000","last_change":"2024-02-02T13:25:48.177550+0000","last_active":"2024-02-02T13:25:49.165624+0000","last_peered":"2024-02-02T13:25:49.165624+0000","last_clean":"2024-02-02T13:25:49.165624+0000","last_became_active":"2024-02-02T13:25:48.172620+0000","last_became_peered":"2024-02-02T13:25:48.172620+0000","last_unstale":"2024-02-02T13:25:49.165624+0000","last_undegraded":"2024-02-02T13:25:49.165624+0000","last_fullsized":"2024-02-02T13:25:49.165624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165253+0000","last_change":"2024-02-02T13:25:47.689295+0000","last_active":"2024-02-02T13:25:49.165253+0000","last_peered":"2024-02-02T13:25:49.165253+0000","last_clean":"2024-02-02T13:25:49.165253+0000","last_became_active":"2024-02-02T13:25:47.688731+0000","last_became_peered":"2024-02-02T13:25:47.688731+0000","last_unstale":"2024-02-02T13:25:49.165253+0000","last_undegraded":"2024-02-02T13:25:49.165253+0000","last_fullsized":"2024-02-02T13:25:49.165253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:32.327008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488497+0000","last_change":"2024-02-02T13:25:48.175241+0000","last_active":"2024-02-02T13:25:49.488497+0000","last_peered":"2024-02-02T13:25:49.488497+0000","last_clean":"2024-02-02T13:25:49.488497+0000","last_became_active":"2024-02-02T13:25:48.174719+0000","last_became_peered":"2024-02-02T13:25:48.174719+0000","last_unstale":"2024-02-02T13:25:49.488497+0000","last_undegraded":"2024-02-02T13:25:49.488497+0000","last_fullsized":"2024-02-02T13:25:49.488497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:53:14.026351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117360+0000","last_change":"2024-02-02T13:25:47.696876+0000","last_active":"2024-02-02T13:25:50.117360+0000","last_peered":"2024-02-02T13:25:50.117360+0000","last_clean":"2024-02-02T13:25:50.117360+0000","last_became_active":"2024-02-02T13:25:47.695315+0000","last_became_peered":"2024-02-02T13:25:47.695315+0000","last_unstale":"2024-02-02T13:25:50.117360+0000","last_undegraded":"2024-02-02T13:25:50.117360+0000","last_fullsized":"2024-02-02T13:25:50.117360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:18:26.426340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164596+0000","last_change":"2024-02-02T13:25:48.165452+0000","last_active":"2024-02-02T13:25:49.164596+0000","last_peered":"2024-02-02T13:25:49.164596+0000","last_clean":"2024-02-02T13:25:49.164596+0000","last_became_active":"2024-02-02T13:25:48.164530+0000","last_became_peered":"2024-02-02T13:25:48.164530+0000","last_unstale":"2024-02-02T13:25:49.164596+0000","last_undegraded":"2024-02-02T13:25:49.164596+0000","last_fullsized":"2024-02-02T13:25:49.164596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488640+0000","last_change":"2024-02-02T13:25:47.692403+0000","last_active":"2024-02-02T13:25:49.488640+0000","last_peered":"2024-02-02T13:25:49.488640+0000","last_clean":"2024-02-02T13:25:49.488640+0000","last_became_active":"2024-02-02T13:25:47.692124+0000","last_became_peered":"2024-02-02T13:25:47.692124+0000","last_unstale":"2024-02-02T13:25:49.488640+0000","last_undegraded":"2024-02-02T13:25:49.488640+0000","last_fullsized":"2024-02-02T13:25:49.488640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:35.117418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166866+0000","last_change":"2024-02-02T13:25:48.179726+0000","last_active":"2024-02-02T13:25:49.166866+0000","last_peered":"2024-02-02T13:25:49.166866+0000","last_clean":"2024-02-02T13:25:49.166866+0000","last_became_active":"2024-02-02T13:25:48.179235+0000","last_became_peered":"2024-02-02T13:25:48.179235+0000","last_unstale":"2024-02-02T13:25:49.166866+0000","last_undegraded":"2024-02-02T13:25:49.166866+0000","last_fullsized":"2024-02-02T13:25:49.166866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:27.370667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165410+0000","last_change":"2024-02-02T13:25:47.692716+0000","last_active":"2024-02-02T13:25:49.165410+0000","last_peered":"2024-02-02T13:25:49.165410+0000","last_clean":"2024-02-02T13:25:49.165410+0000","last_became_active":"2024-02-02T13:25:47.692524+0000","last_became_peered":"2024-02-02T13:25:47.692524+0000","last_unstale":"2024-02-02T13:25:49.165410+0000","last_undegraded":"2024-02-02T13:25:49.165410+0000","last_fullsized":"2024-02-02T13:25:49.165410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489311+0000","last_change":"2024-02-02T13:25:48.176715+0000","last_active":"2024-02-02T13:25:49.489311+0000","last_peered":"2024-02-02T13:25:49.489311+0000","last_clean":"2024-02-02T13:25:49.489311+0000","last_became_active":"2024-02-02T13:25:48.176282+0000","last_became_peered":"2024-02-02T13:25:48.176282+0000","last_unstale":"2024-02-02T13:25:49.489311+0000","last_undegraded":"2024-02-02T13:25:49.489311+0000","last_fullsized":"2024-02-02T13:25:49.489311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:39.214130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164848+0000","last_change":"2024-02-02T13:25:47.691906+0000","last_active":"2024-02-02T13:25:49.164848+0000","last_peered":"2024-02-02T13:25:49.164848+0000","last_clean":"2024-02-02T13:25:49.164848+0000","last_became_active":"2024-02-02T13:25:47.691398+0000","last_became_peered":"2024-02-02T13:25:47.691398+0000","last_unstale":"2024-02-02T13:25:49.164848+0000","last_undegraded":"2024-02-02T13:25:49.164848+0000","last_fullsized":"2024-02-02T13:25:49.164848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117096+0000","last_change":"2024-02-02T13:25:48.182985+0000","last_active":"2024-02-02T13:25:50.117096+0000","last_peered":"2024-02-02T13:25:50.117096+0000","last_clean":"2024-02-02T13:25:50.117096+0000","last_became_active":"2024-02-02T13:25:48.182751+0000","last_became_peered":"2024-02-02T13:25:48.182751+0000","last_unstale":"2024-02-02T13:25:50.117096+0000","last_undegraded":"2024-02-02T13:25:50.117096+0000","last_fullsized":"2024-02-02T13:25:50.117096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:18:31.358697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489385+0000","last_change":"2024-02-02T13:25:47.686600+0000","last_active":"2024-02-02T13:25:49.489385+0000","last_peered":"2024-02-02T13:25:49.489385+0000","last_clean":"2024-02-02T13:25:49.489385+0000","last_became_active":"2024-02-02T13:25:47.686361+0000","last_became_peered":"2024-02-02T13:25:47.686361+0000","last_unstale":"2024-02-02T13:25:49.489385+0000","last_undegraded":"2024-02-02T13:25:49.489385+0000","last_fullsized":"2024-02-02T13:25:49.489385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165389+0000","last_change":"2024-02-02T13:25:48.177369+0000","last_active":"2024-02-02T13:25:49.165389+0000","last_peered":"2024-02-02T13:25:49.165389+0000","last_clean":"2024-02-02T13:25:49.165389+0000","last_became_active":"2024-02-02T13:25:48.177132+0000","last_became_peered":"2024-02-02T13:25:48.177132+0000","last_unstale":"2024-02-02T13:25:49.165389+0000","last_undegraded":"2024-02-02T13:25:49.165389+0000","last_fullsized":"2024-02-02T13:25:49.165389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:15:17.587727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117529+0000","last_change":"2024-02-02T13:25:47.696259+0000","last_active":"2024-02-02T13:25:50.117529+0000","last_peered":"2024-02-02T13:25:50.117529+0000","last_clean":"2024-02-02T13:25:50.117529+0000","last_became_active":"2024-02-02T13:25:47.694928+0000","last_became_peered":"2024-02-02T13:25:47.694928+0000","last_unstale":"2024-02-02T13:25:50.117529+0000","last_undegraded":"2024-02-02T13:25:50.117529+0000","last_fullsized":"2024-02-02T13:25:50.117529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:17.518801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488525+0000","last_change":"2024-02-02T13:25:48.177080+0000","last_active":"2024-02-02T13:25:49.488525+0000","last_peered":"2024-02-02T13:25:49.488525+0000","last_clean":"2024-02-02T13:25:49.488525+0000","last_became_active":"2024-02-02T13:25:48.176641+0000","last_became_peered":"2024-02-02T13:25:48.176641+0000","last_unstale":"2024-02-02T13:25:49.488525+0000","last_undegraded":"2024-02-02T13:25:49.488525+0000","last_fullsized":"2024-02-02T13:25:49.488525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117122+0000","last_change":"2024-02-02T13:25:47.687195+0000","last_active":"2024-02-02T13:25:50.117122+0000","last_peered":"2024-02-02T13:25:50.117122+0000","last_clean":"2024-02-02T13:25:50.117122+0000","last_became_active":"2024-02-02T13:25:47.686807+0000","last_became_peered":"2024-02-02T13:25:47.686807+0000","last_unstale":"2024-02-02T13:25:50.117122+0000","last_undegraded":"2024-02-02T13:25:50.117122+0000","last_fullsized":"2024-02-02T13:25:50.117122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:09:26.842297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116342+0000","last_change":"2024-02-02T13:25:48.173270+0000","last_active":"2024-02-02T13:25:50.116342+0000","last_peered":"2024-02-02T13:25:50.116342+0000","last_clean":"2024-02-02T13:25:50.116342+0000","last_became_active":"2024-02-02T13:25:48.172673+0000","last_became_peered":"2024-02-02T13:25:48.172673+0000","last_unstale":"2024-02-02T13:25:50.116342+0000","last_undegraded":"2024-02-02T13:25:50.116342+0000","last_fullsized":"2024-02-02T13:25:50.116342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:01.804626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117940+0000","last_change":"2024-02-02T13:25:47.694212+0000","last_active":"2024-02-02T13:25:50.117940+0000","last_peered":"2024-02-02T13:25:50.117940+0000","last_clean":"2024-02-02T13:25:50.117940+0000","last_became_active":"2024-02-02T13:25:47.692002+0000","last_became_peered":"2024-02-02T13:25:47.692002+0000","last_unstale":"2024-02-02T13:25:50.117940+0000","last_undegraded":"2024-02-02T13:25:50.117940+0000","last_fullsized":"2024-02-02T13:25:50.117940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:39:29.715158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116159+0000","last_change":"2024-02-02T13:25:48.175089+0000","last_active":"2024-02-02T13:25:50.116159+0000","last_peered":"2024-02-02T13:25:50.116159+0000","last_clean":"2024-02-02T13:25:50.116159+0000","last_became_active":"2024-02-02T13:25:48.174785+0000","last_became_peered":"2024-02-02T13:25:48.174785+0000","last_unstale":"2024-02-02T13:25:50.116159+0000","last_undegraded":"2024-02-02T13:25:50.116159+0000","last_fullsized":"2024-02-02T13:25:50.116159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:36.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.861127+0000","last_change":"2024-02-02T13:25:47.691955+0000","last_active":"2024-02-02T13:25:49.861127+0000","last_peered":"2024-02-02T13:25:49.861127+0000","last_clean":"2024-02-02T13:25:49.861127+0000","last_became_active":"2024-02-02T13:25:47.690900+0000","last_became_peered":"2024-02-02T13:25:47.690900+0000","last_unstale":"2024-02-02T13:25:49.861127+0000","last_undegraded":"2024-02-02T13:25:49.861127+0000","last_fullsized":"2024-02-02T13:25:49.861127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116589+0000","last_change":"2024-02-02T13:25:48.161097+0000","last_active":"2024-02-02T13:25:50.116589+0000","last_peered":"2024-02-02T13:25:50.116589+0000","last_clean":"2024-02-02T13:25:50.116589+0000","last_became_active":"2024-02-02T13:25:48.160874+0000","last_became_peered":"2024-02-02T13:25:48.160874+0000","last_unstale":"2024-02-02T13:25:50.116589+0000","last_undegraded":"2024-02-02T13:25:50.116589+0000","last_fullsized":"2024-02-02T13:25:50.116589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:14.082563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488949+0000","last_change":"2024-02-02T13:25:47.694662+0000","last_active":"2024-02-02T13:25:49.488949+0000","last_peered":"2024-02-02T13:25:49.488949+0000","last_clean":"2024-02-02T13:25:49.488949+0000","last_became_active":"2024-02-02T13:25:47.693811+0000","last_became_peered":"2024-02-02T13:25:47.693811+0000","last_unstale":"2024-02-02T13:25:49.488949+0000","last_undegraded":"2024-02-02T13:25:49.488949+0000","last_fullsized":"2024-02-02T13:25:49.488949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116481+0000","last_change":"2024-02-02T13:25:48.173396+0000","last_active":"2024-02-02T13:25:50.116481+0000","last_peered":"2024-02-02T13:25:50.116481+0000","last_clean":"2024-02-02T13:25:50.116481+0000","last_became_active":"2024-02-02T13:25:48.173032+0000","last_became_peered":"2024-02-02T13:25:48.173032+0000","last_unstale":"2024-02-02T13:25:50.116481+0000","last_undegraded":"2024-02-02T13:25:50.116481+0000","last_fullsized":"2024-02-02T13:25:50.116481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:54.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117649+0000","last_change":"2024-02-02T13:25:47.695963+0000","last_active":"2024-02-02T13:25:50.117649+0000","last_peered":"2024-02-02T13:25:50.117649+0000","last_clean":"2024-02-02T13:25:50.117649+0000","last_became_active":"2024-02-02T13:25:47.694554+0000","last_became_peered":"2024-02-02T13:25:47.694554+0000","last_unstale":"2024-02-02T13:25:50.117649+0000","last_undegraded":"2024-02-02T13:25:50.117649+0000","last_fullsized":"2024-02-02T13:25:50.117649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:33:06.884874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166845+0000","last_change":"2024-02-02T13:25:48.180200+0000","last_active":"2024-02-02T13:25:49.166845+0000","last_peered":"2024-02-02T13:25:49.166845+0000","last_clean":"2024-02-02T13:25:49.166845+0000","last_became_active":"2024-02-02T13:25:48.177746+0000","last_became_peered":"2024-02-02T13:25:48.177746+0000","last_unstale":"2024-02-02T13:25:49.166845+0000","last_undegraded":"2024-02-02T13:25:49.166845+0000","last_fullsized":"2024-02-02T13:25:49.166845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165691+0000","last_change":"2024-02-02T13:25:47.680998+0000","last_active":"2024-02-02T13:25:49.165691+0000","last_peered":"2024-02-02T13:25:49.165691+0000","last_clean":"2024-02-02T13:25:49.165691+0000","last_became_active":"2024-02-02T13:25:47.680388+0000","last_became_peered":"2024-02-02T13:25:47.680388+0000","last_unstale":"2024-02-02T13:25:49.165691+0000","last_undegraded":"2024-02-02T13:25:49.165691+0000","last_fullsized":"2024-02-02T13:25:49.165691+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:00.792218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.116800+0000","last_change":"2024-02-02T13:25:48.172937+0000","last_active":"2024-02-02T13:25:50.116800+0000","last_peered":"2024-02-02T13:25:50.116800+0000","last_clean":"2024-02-02T13:25:50.116800+0000","last_became_active":"2024-02-02T13:25:48.168179+0000","last_became_peered":"2024-02-02T13:25:48.168179+0000","last_unstale":"2024-02-02T13:25:50.116800+0000","last_undegraded":"2024-02-02T13:25:50.116800+0000","last_fullsized":"2024-02-02T13:25:50.116800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:46.925303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488754+0000","last_change":"2024-02-02T13:25:47.680391+0000","last_active":"2024-02-02T13:25:49.488754+0000","last_peered":"2024-02-02T13:25:49.488754+0000","last_clean":"2024-02-02T13:25:49.488754+0000","last_became_active":"2024-02-02T13:25:47.680155+0000","last_became_peered":"2024-02-02T13:25:47.680155+0000","last_unstale":"2024-02-02T13:25:49.488754+0000","last_undegraded":"2024-02-02T13:25:49.488754+0000","last_fullsized":"2024-02-02T13:25:49.488754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:32:57.881556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.166055+0000","last_change":"2024-02-02T13:25:48.174963+0000","last_active":"2024-02-02T13:25:49.166055+0000","last_peered":"2024-02-02T13:25:49.166055+0000","last_clean":"2024-02-02T13:25:49.166055+0000","last_became_active":"2024-02-02T13:25:48.173205+0000","last_became_peered":"2024-02-02T13:25:48.173205+0000","last_unstale":"2024-02-02T13:25:49.166055+0000","last_undegraded":"2024-02-02T13:25:49.166055+0000","last_fullsized":"2024-02-02T13:25:49.166055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:10.693675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.863472+0000","last_change":"2024-02-02T13:25:47.691797+0000","last_active":"2024-02-02T13:25:49.863472+0000","last_peered":"2024-02-02T13:25:49.863472+0000","last_clean":"2024-02-02T13:25:49.863472+0000","last_became_active":"2024-02-02T13:25:47.690587+0000","last_became_peered":"2024-02-02T13:25:47.690587+0000","last_unstale":"2024-02-02T13:25:49.863472+0000","last_undegraded":"2024-02-02T13:25:49.863472+0000","last_fullsized":"2024-02-02T13:25:49.863472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:45.912896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860305+0000","last_change":"2024-02-02T13:25:48.169133+0000","last_active":"2024-02-02T13:25:49.860305+0000","last_peered":"2024-02-02T13:25:49.860305+0000","last_clean":"2024-02-02T13:25:49.860305+0000","last_became_active":"2024-02-02T13:25:48.168905+0000","last_became_peered":"2024-02-02T13:25:48.168905+0000","last_unstale":"2024-02-02T13:25:49.860305+0000","last_undegraded":"2024-02-02T13:25:49.860305+0000","last_fullsized":"2024-02-02T13:25:49.860305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.488779+0000","last_change":"2024-02-02T13:25:47.684170+0000","last_active":"2024-02-02T13:25:49.488779+0000","last_peered":"2024-02-02T13:25:49.488779+0000","last_clean":"2024-02-02T13:25:49.488779+0000","last_became_active":"2024-02-02T13:25:47.683723+0000","last_became_peered":"2024-02-02T13:25:47.683723+0000","last_unstale":"2024-02-02T13:25:49.488779+0000","last_undegraded":"2024-02-02T13:25:49.488779+0000","last_fullsized":"2024-02-02T13:25:49.488779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489489+0000","last_change":"2024-02-02T13:25:48.176549+0000","last_active":"2024-02-02T13:25:49.489489+0000","last_peered":"2024-02-02T13:25:49.489489+0000","last_clean":"2024-02-02T13:25:49.489489+0000","last_became_active":"2024-02-02T13:25:48.175881+0000","last_became_peered":"2024-02-02T13:25:48.175881+0000","last_unstale":"2024-02-02T13:25:49.489489+0000","last_undegraded":"2024-02-02T13:25:49.489489+0000","last_fullsized":"2024-02-02T13:25:49.489489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860803+0000","last_change":"2024-02-02T13:25:47.694821+0000","last_active":"2024-02-02T13:25:49.860803+0000","last_peered":"2024-02-02T13:25:49.860803+0000","last_clean":"2024-02-02T13:25:49.860803+0000","last_became_active":"2024-02-02T13:25:47.694494+0000","last_became_peered":"2024-02-02T13:25:47.694494+0000","last_unstale":"2024-02-02T13:25:49.860803+0000","last_undegraded":"2024-02-02T13:25:49.860803+0000","last_fullsized":"2024-02-02T13:25:49.860803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:38:08.332585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.860535+0000","last_change":"2024-02-02T13:25:48.172635+0000","last_active":"2024-02-02T13:25:49.860535+0000","last_peered":"2024-02-02T13:25:49.860535+0000","last_clean":"2024-02-02T13:25:49.860535+0000","last_became_active":"2024-02-02T13:25:48.172399+0000","last_became_peered":"2024-02-02T13:25:48.172399+0000","last_unstale":"2024-02-02T13:25:49.860535+0000","last_undegraded":"2024-02-02T13:25:49.860535+0000","last_fullsized":"2024-02-02T13:25:49.860535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:14.524004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489209+0000","last_change":"2024-02-02T13:25:47.681284+0000","last_active":"2024-02-02T13:25:49.489209+0000","last_peered":"2024-02-02T13:25:49.489209+0000","last_clean":"2024-02-02T13:25:49.489209+0000","last_became_active":"2024-02-02T13:25:47.681059+0000","last_became_peered":"2024-02-02T13:25:47.681059+0000","last_unstale":"2024-02-02T13:25:49.489209+0000","last_undegraded":"2024-02-02T13:25:49.489209+0000","last_fullsized":"2024-02-02T13:25:49.489209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:13.070155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.164426+0000","last_change":"2024-02-02T13:25:48.169371+0000","last_active":"2024-02-02T13:25:49.164426+0000","last_peered":"2024-02-02T13:25:49.164426+0000","last_clean":"2024-02-02T13:25:49.164426+0000","last_became_active":"2024-02-02T13:25:48.168848+0000","last_became_peered":"2024-02-02T13:25:48.168848+0000","last_unstale":"2024-02-02T13:25:49.164426+0000","last_undegraded":"2024-02-02T13:25:49.164426+0000","last_fullsized":"2024-02-02T13:25:49.164426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:27.180112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117879+0000","last_change":"2024-02-02T13:25:47.693995+0000","last_active":"2024-02-02T13:25:50.117879+0000","last_peered":"2024-02-02T13:25:50.117879+0000","last_clean":"2024-02-02T13:25:50.117879+0000","last_became_active":"2024-02-02T13:25:47.693730+0000","last_became_peered":"2024-02-02T13:25:47.693730+0000","last_unstale":"2024-02-02T13:25:50.117879+0000","last_undegraded":"2024-02-02T13:25:50.117879+0000","last_fullsized":"2024-02-02T13:25:50.117879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:42:28.175905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:50.117228+0000","last_change":"2024-02-02T13:25:48.178674+0000","last_active":"2024-02-02T13:25:50.117228+0000","last_peered":"2024-02-02T13:25:50.117228+0000","last_clean":"2024-02-02T13:25:50.117228+0000","last_became_active":"2024-02-02T13:25:48.178422+0000","last_became_peered":"2024-02-02T13:25:48.178422+0000","last_unstale":"2024-02-02T13:25:50.117228+0000","last_undegraded":"2024-02-02T13:25:50.117228+0000","last_fullsized":"2024-02-02T13:25:50.117228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:29.371994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.165582+0000","last_change":"2024-02-02T13:25:47.693863+0000","last_active":"2024-02-02T13:25:49.165582+0000","last_peered":"2024-02-02T13:25:49.165582+0000","last_clean":"2024-02-02T13:25:49.165582+0000","last_became_active":"2024-02-02T13:25:47.693607+0000","last_became_peered":"2024-02-02T13:25:47.693607+0000","last_unstale":"2024-02-02T13:25:49.165582+0000","last_undegraded":"2024-02-02T13:25:49.165582+0000","last_fullsized":"2024-02-02T13:25:49.165582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:53.778705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.167167+0000","last_change":"2024-02-02T13:25:48.180580+0000","last_active":"2024-02-02T13:25:49.167167+0000","last_peered":"2024-02-02T13:25:49.167167+0000","last_clean":"2024-02-02T13:25:49.167167+0000","last_became_active":"2024-02-02T13:25:48.179944+0000","last_became_peered":"2024-02-02T13:25:48.179944+0000","last_unstale":"2024-02-02T13:25:49.167167+0000","last_undegraded":"2024-02-02T13:25:49.167167+0000","last_fullsized":"2024-02-02T13:25:49.167167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:47.652178+0000","last_clean_scrub_stamp":"2024-02-02T13:25:47.652178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:26:07.573173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-02T13:25:49.489239+0000","last_change":"2024-02-02T13:25:47.697550+0000","last_active":"2024-02-02T13:25:49.489239+0000","last_peered":"2024-02-02T13:25:49.489239+0000","last_clean":"2024-02-02T13:25:49.489239+0000","last_became_active":"2024-02-02T13:25:47.697173+0000","last_became_peered":"2024-02-02T13:25:47.697173+0000","last_unstale":"2024-02-02T13:25:49.489239+0000","last_undegraded":"2024-02-02T13:25:49.489239+0000","last_fullsized":"2024-02-02T13:25:49.489239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:25:46.639771+0000","last_clean_scrub_stamp":"2024-02-02T13:25:46.639771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:05:53.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-02-02T13:25:48.165125+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:25:48.165125+0000","last_peered":"2024-02-02T13:25:48.165125+0000","last_clean":"2024-02-02T13:25:48.165125+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:25:48.165125+0000","last_undegraded":"2024-02-02T13:25:48.165125+0000","last_fullsized":"2024-02-02T13:25:48.165125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257807,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.024},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3819999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.98999999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.5549999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.155},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.266}]},{"osd":5,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.639},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":3.6840000000000002,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.331}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.579}]},{"osd":2,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.357},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.6180000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.26},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.47}]},{"osd":5,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.6579999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.5229999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.357},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.139}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.385}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.046}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.751},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.23}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.111}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.1659999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.204}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.01}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.419},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0469999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.365}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.577},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.526}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195162,"statfs":{"total":200000000,"available":199846679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.302},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.123},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1779999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.585,"5min":3.585,"15min":3.585},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.355},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.554}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.5009999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":1.454},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.411}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.88}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.1830000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.226}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.5329999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.585}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.038}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.268},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.383}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.532},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.4119999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.601},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.512}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.452}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.8070000000000004,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":1.006},{"interface":"front","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":10.781000000000001,"5min":10.781000000000001,"15min":10.781000000000001},"last":1.0609999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.8410000000000002,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.395}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":10.651,"5min":10.651,"15min":10.651},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.7869999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.589}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.238},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":10.708,"5min":10.708,"15min":10.708},"last":1.48}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":10.74,"5min":10.74,"15min":10.74},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.528}]}]}],"pool_statfs":[]}} 2024-02-02T13:25:53.589 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:25:53.589 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:25:53.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.837+0000 7f9c61c00640 1 Processor -- start 2024-02-02T13:25:53.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.838+0000 7f9c61c00640 1 -- start start 2024-02-02T13:25:53.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c5c15d0a0 0x7f9c5c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c5c06e080 0x7f9c5c0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c0b1500 con 0x7f9c5c0b0b60 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c161890 con 0x7f9c5c06e080 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c9cf0 con 0x7f9c5c15d0a0 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c5b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9c5c0b0b60 0x7f9c5c0c8250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53000/0 (socket says 172.21.15.91:53000) 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c5b400640 1 -- 172.21.15.91:0/3974395637 learned_addr learned my addr 172.21.15.91:0/3974395637 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.839+0000 7f9c61200640 1 --2- 172.21.15.91:0/3974395637 >> v2:172.21.15.91:3300/0 conn(0x7f9c5c06e080 0x7f9c5c0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c5be00640 1 --2- 172.21.15.91:0/3974395637 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c15d0a0 0x7f9c5c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c5aa00640 1 -- 172.21.15.91:0/3974395637 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337606837 0 0) 0x7f9c5c0b1500 con 0x7f9c5c0b0b60 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c5aa00640 1 -- 172.21.15.91:0/3974395637 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c3c003520 con 0x7f9c5c0b0b60 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c61200640 1 -- 172.21.15.91:0/3974395637 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c15d0a0 msgr2=0x7f9c5c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c61200640 1 --2- 172.21.15.91:0/3974395637 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c15d0a0 0x7f9c5c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c61200640 1 -- 172.21.15.91:0/3974395637 >> v1:172.21.15.91:6789/0 conn(0x7f9c5c0b0b60 legacy=0x7f9c5c0c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.840+0000 7f9c61200640 1 -- 172.21.15.91:0/3974395637 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c160c80 con 0x7f9c5c06e080 2024-02-02T13:25:53.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.841+0000 7f9c61200640 1 --2- 172.21.15.91:0/3974395637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c0b0fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0cc9f0 tx=0x7f9c5402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398c544971b2b813 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.841+0000 7f9c5aa00640 1 -- 172.21.15.91:0/3974395637 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5402cc70 con 0x7f9c5c06e080 2024-02-02T13:25:53.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.841+0000 7f9c5aa00640 1 -- 172.21.15.91:0/3974395637 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5402cdd0 con 0x7f9c5c06e080 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.841+0000 7f9c5aa00640 1 -- 172.21.15.91:0/3974395637 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54034660 con 0x7f9c5c06e080 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 -- 172.21.15.91:0/3974395637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 msgr2=0x7f9c5c0b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 --2- 172.21.15.91:0/3974395637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c0b0fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0cc9f0 tx=0x7f9c5402bec0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 -- 172.21.15.91:0/3974395637 shutdown_connections 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 --2- 172.21.15.91:0/3974395637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c0b0fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 --2- 172.21.15.91:0/3974395637 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c15d0a0 0x7f9c5c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 -- 172.21.15.91:0/3974395637 >> 172.21.15.91:0/3974395637 conn(0x7f9c5c0cef00 msgr2=0x7f9c5c0cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 -- 172.21.15.91:0/3974395637 shutdown_connections 2024-02-02T13:25:53.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.842+0000 7f9c61c00640 1 -- 172.21.15.91:0/3974395637 wait complete. 2024-02-02T13:25:53.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.844+0000 7f9c61c00640 1 Processor -- start 2024-02-02T13:25:53.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.844+0000 7f9c61c00640 1 -- start start 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b0b60 0x7f9c5c0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c15d0a0 0x7f9c5c0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1f36a0 con 0x7f9c5c06e080 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b3630 con 0x7f9c5c0b0b60 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1f3920 con 0x7f9c5c15d0a0 2024-02-02T13:25:53.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56548/0 (socket says 172.21.15.91:56548) 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 -- 172.21.15.91:0/1526571185 learned_addr learned my addr 172.21.15.91:0/1526571185 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5b400640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b0b60 0x7f9c5c0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 -- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c15d0a0 msgr2=0x7f9c5c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c61200640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c15d0a0 0x7f9c5c0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c15d0a0 0x7f9c5c0b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 -- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b0b60 msgr2=0x7f9c5c0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b0b60 0x7f9c5c0b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.845+0000 7f9c5be00640 1 -- 172.21.15.91:0/1526571185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54009a70 con 0x7f9c5c06e080 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c5be00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d5f0 tx=0x7f9c4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0125f0 con 0x7f9c5c06e080 2024-02-02T13:25:53.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c012750 con 0x7f9c5c06e080 2024-02-02T13:25:53.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c012b00 con 0x7f9c5c06e080 2024-02-02T13:25:53.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c1f3c00 con 0x7f9c5c06e080 2024-02-02T13:25:53.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.846+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c0b1ea0 con 0x7f9c5c06e080 2024-02-02T13:25:53.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.848+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c04dff0 con 0x7f9c5c06e080 2024-02-02T13:25:53.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.848+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c4c012c60 con 0x7f9c5c06e080 2024-02-02T13:25:53.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.849+0000 7f9c53e00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 0x7f9c3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:53.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.849+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c4c088d70 con 0x7f9c5c06e080 2024-02-02T13:25:53.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.850+0000 7f9c5b400640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 0x7f9c3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:53.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.850+0000 7f9c5b400640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 0x7f9c3406c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0af9b0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:53.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:53.854+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c4c0543f0 con 0x7f9c5c06e080 2024-02-02T13:25:54.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.110+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9c5c0b1760 con 0x7f9c5c06e080 2024-02-02T13:25:54.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.117+0000 7f9c53e00640 1 -- 172.21.15.91:0/1526571185 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9c4c057a10 con 0x7f9c5c06e080 2024-02-02T13:25:54.118 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 msgr2=0x7f9c3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 0x7f9c3406c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0af9b0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 msgr2=0x7f9c5c155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d5f0 tx=0x7f9c4c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 shutdown_connections 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c3406a430 0x7f9c3406c8f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c15d0a0 0x7f9c5c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b0b60 0x7f9c5c0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 --2- 172.21.15.91:0/1526571185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c5c06e080 0x7f9c5c155b70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 >> 172.21.15.91:0/1526571185 conn(0x7f9c5c0cef00 msgr2=0x7f9c5c0ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:54.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 shutdown_connections 2024-02-02T13:25:54.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.120+0000 7f9c61c00640 1 -- 172.21.15.91:0/1526571185 wait complete. 2024-02-02T13:25:54.139 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:54.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.343+0000 7f2080e00640 1 Processor -- start 2024-02-02T13:25:54.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.344+0000 7f2080e00640 1 -- start start 2024-02-02T13:25:54.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f207c0add80 0x7f207c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 0x7f207c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0adbd0 con 0x7f207c0add80 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0c59d0 con 0x7f207c0c9450 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2080e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0ce3b0 con 0x7f207c15d3a0 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2073e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f207c0add80 0x7f207c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2073e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f207c0add80 0x7f207c0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56564/0 (socket says 172.21.15.91:56564) 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.345+0000 7f2073e00640 1 -- 172.21.15.91:0/2599923839 learned_addr learned my addr 172.21.15.91:0/2599923839 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f207b400640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f207be00640 1 --2- 172.21.15.91:0/2599923839 >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 0x7f207c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f2073e00640 1 -- 172.21.15.91:0/2599923839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 msgr2=0x7f207c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f2073e00640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f2073e00640 1 -- 172.21.15.91:0/2599923839 >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 msgr2=0x7f207c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f2073e00640 1 --2- 172.21.15.91:0/2599923839 >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 0x7f207c0c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f2073e00640 1 -- 172.21.15.91:0/2599923839 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0ce8e0 con 0x7f207c0add80 2024-02-02T13:25:54.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.346+0000 7f207be00640 1 --2- 172.21.15.91:0/2599923839 >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 0x7f207c0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:54.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.347+0000 7f2073e00640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0add80 0x7f207c0706a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f207c0c24f0 tx=0x7f206402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35218f8788268d77 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.347+0000 7f207aa00640 1 -- 172.21.15.91:0/2599923839 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064031750 con 0x7f207c0add80 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.347+0000 7f207aa00640 1 -- 172.21.15.91:0/2599923839 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640318b0 con 0x7f207c0add80 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.347+0000 7f207aa00640 1 -- 172.21.15.91:0/2599923839 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064031c40 con 0x7f207c0add80 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 -- 172.21.15.91:0/2599923839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0add80 msgr2=0x7f207c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0add80 0x7f207c0706a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f207c0c24f0 tx=0x7f206402b7f0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 -- 172.21.15.91:0/2599923839 shutdown_connections 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 --2- 172.21.15.91:0/2599923839 >> v2:172.21.15.113:3300/0 conn(0x7f207c0c9450 0x7f207c0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0add80 0x7f207c0706a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 --2- 172.21.15.91:0/2599923839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 -- 172.21.15.91:0/2599923839 >> 172.21.15.91:0/2599923839 conn(0x7f207c0cea90 msgr2=0x7f207c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 -- 172.21.15.91:0/2599923839 shutdown_connections 2024-02-02T13:25:54.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.348+0000 7f2080e00640 1 -- 172.21.15.91:0/2599923839 wait complete. 2024-02-02T13:25:54.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.350+0000 7f2080e00640 1 Processor -- start 2024-02-02T13:25:54.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.350+0000 7f2080e00640 1 -- start start 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c0add80 0x7f207c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0ae7d0 con 0x7f207c0c9450 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0aea00 con 0x7f207c0add80 2024-02-02T13:25:54.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2080e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0aece0 con 0x7f207c15d3a0 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2073e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2073e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56576/0 (socket says 172.21.15.91:56576) 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f2073e00640 1 -- 172.21.15.91:0/980757423 learned_addr learned my addr 172.21.15.91:0/980757423 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f207b400640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c0add80 0x7f207c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.351+0000 7f207be00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 -- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 msgr2=0x7f207c151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 -- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c0add80 msgr2=0x7f207c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c0add80 0x7f207c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 -- 172.21.15.91:0/980757423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20640094d0 con 0x7f207c0c9450 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f207be00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:54.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f206402b830 tx=0x7f20640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064033c80 con 0x7f207c0c9450 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064033e00 con 0x7f207c0c9450 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206403c640 con 0x7f207c0c9450 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c0aefc0 con 0x7f207c0c9450 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.352+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c159ad0 con 0x7f207c0c9450 2024-02-02T13:25:54.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.353+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c04dff0 con 0x7f207c0c9450 2024-02-02T13:25:54.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.354+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f206403a040 con 0x7f207c0c9450 2024-02-02T13:25:54.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.355+0000 7f2073400640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 0x7f205406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:54.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.355+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20640aa6c0 con 0x7f207c0c9450 2024-02-02T13:25:54.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.355+0000 7f207b400640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 0x7f205406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:54.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.356+0000 7f207b400640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 0x7f205406c920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:54.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.359+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2064075ca0 con 0x7f207c0c9450 2024-02-02T13:25:54.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.588+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f207c0b4a10 con 0x7f207c0c9450 2024-02-02T13:25:54.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.589+0000 7f2073400640 1 -- 172.21.15.91:0/980757423 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f2064038020 con 0x7f207c0c9450 2024-02-02T13:25:54.590 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:25:54.590 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:25:54.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 msgr2=0x7f205406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 0x7f205406c920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).stop 2024-02-02T13:25:54.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 msgr2=0x7f207c0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:54.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f206402b830 tx=0x7f20640046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 shutdown_connections 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f205406a460 0x7f205406c920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f207c15d3a0 0x7f207c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f207c0c9450 0x7f207c0ae290 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 --2- 172.21.15.91:0/980757423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c0add80 0x7f207c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 >> 172.21.15.91:0/980757423 conn(0x7f207c0cea90 msgr2=0x7f207c0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 shutdown_connections 2024-02-02T13:25:54.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:54.592+0000 7f2080e00640 1 -- 172.21.15.91:0/980757423 wait complete. 2024-02-02T13:25:55.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:25:48.146857+0000', 'modified': '2024-02-02T13:25:51.676530+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5052': {'gid': 5052, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.113:6835/3241118880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6833', 'nonce': 3241118880}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 3241118880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:25:55.014 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:25:55.014 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-02T13:25:55.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.228+0000 7f8fb2800640 1 Processor -- start 2024-02-02T13:25:55.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.229+0000 7f8fb2800640 1 -- start start 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8fb2800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8fac164a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8fb2800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac0ce990 con 0x7f8fac071270 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8fb2800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac0cec70 con 0x7f8fac15a020 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8fb2800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac15a400 con 0x7f8fac0ce300 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8faaa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8fac071270 0x7f8fac0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53012/0 (socket says 172.21.15.91:53012) 2024-02-02T13:25:55.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.230+0000 7f8faaa00640 1 -- 172.21.15.91:0/2866320678 learned_addr learned my addr 172.21.15.91:0/2866320678 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8fabe00640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8fac164a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230330292 0 0) 0x7f8fac0ce990 con 0x7f8fac071270 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f8c003520 con 0x7f8fac071270 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 899171678 0 0) 0x7f8f8c003520 con 0x7f8fac071270 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 msgr2=0x7f8fac164a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8fac164a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 >> v1:172.21.15.113:6789/0 conn(0x7f8fac15a020 legacy=0x7f8fac070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0cef50 con 0x7f8fac071270 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8f94001670 con 0x7f8fac071270 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 >> v1:172.21.15.91:6789/0 conn(0x7f8fac071270 legacy=0x7f8fac0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f8c0039a0 0x7f8f8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8f8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c004d00 0x7f8f8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c00fe40 con 0x7f8f8c0039a0 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c00ff80 con 0x7f8f8c004d00 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.231+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c010140 con 0x7f8fac0ce300 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f8c0039a0 0x7f8f8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fabe00640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c004d00 0x7f8f8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8faaa00640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8f8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 -- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 msgr2=0x7f8f8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac0ce300 0x7f8f8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 -- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c004d00 msgr2=0x7f8f8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c004d00 0x7f8f8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.232+0000 7f8fab400640 1 -- 172.21.15.91:0/2866320678 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c003520 con 0x7f8f8c0039a0 2024-02-02T13:25:55.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8fab400640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f8c0039a0 0x7f8f8c003dc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c001720 tx=0x7f8f8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fab2227b9f3c04b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0052d0 con 0x7f8f8c0039a0 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c02e050 con 0x7f8f8c0039a0 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8faa000640 1 -- 172.21.15.91:0/2866320678 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0035f0 con 0x7f8f8c0039a0 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8fb2800640 1 -- 172.21.15.91:0/2866320678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f8c0039a0 msgr2=0x7f8f8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2866320678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f8c0039a0 0x7f8f8c003dc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c001720 tx=0x7f8f8c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8fabe00640 1 -- 172.21.15.91:0/2866320678 reap_dead start 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.234+0000 7f8fb2800640 1 -- 172.21.15.91:0/2866320678 shutdown_connections 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.235+0000 7f8fb2800640 1 -- 172.21.15.91:0/2866320678 >> 172.21.15.91:0/2866320678 conn(0x7f8fac0b18f0 msgr2=0x7f8fac0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.235+0000 7f8fb2800640 1 -- 172.21.15.91:0/2866320678 shutdown_connections 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.235+0000 7f8fb2800640 1 -- 172.21.15.91:0/2866320678 wait complete. 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.236+0000 7f8fb2800640 1 Processor -- start 2024-02-02T13:25:55.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.236+0000 7f8fb2800640 1 -- start start 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c0039a0 0x7f8fac1ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac1590d0 0x7f8fac1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1eef20 con 0x7f8fac159550 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0c9000 con 0x7f8f8c0039a0 2024-02-02T13:25:55.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.237+0000 7f8fb2800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1f29c0 con 0x7f8fac1590d0 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56586/0 (socket says 172.21.15.91:56586) 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 -- 172.21.15.91:0/2941265573 learned_addr learned my addr 172.21.15.91:0/2941265573 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 -- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac1590d0 msgr2=0x7f8fac1f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac1590d0 0x7f8fac1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 -- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c0039a0 msgr2=0x7f8fac1ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c0039a0 0x7f8fac1ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 -- 172.21.15.91:0/2941265573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac1f2ca0 con 0x7f8fac159550 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fabe00640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0cc7b0 tx=0x7f8fa40075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.238+0000 7f8fa3400640 1 -- 172.21.15.91:0/2941265573 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4012040 con 0x7f8fac159550 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.239+0000 7f8fa3400640 1 -- 172.21.15.91:0/2941265573 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa400be40 con 0x7f8fac159550 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.239+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1ee8d0 con 0x7f8fac159550 2024-02-02T13:25:55.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.239+0000 7f8fa3400640 1 -- 172.21.15.91:0/2941265573 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa400b310 con 0x7f8fac159550 2024-02-02T13:25:55.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.239+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac0c51d0 con 0x7f8fac159550 2024-02-02T13:25:55.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.240+0000 7f8fa3400640 1 -- 172.21.15.91:0/2941265573 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8fa4009040 con 0x7f8fac159550 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.240+0000 7f8fa3400640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 0x7f8f8406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.241+0000 7f8fa3400640 1 -- 172.21.15.91:0/2941265573 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa40886b0 con 0x7f8fac159550 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.241+0000 7f8fab400640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 0x7f8f8406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.241+0000 7f8fb562a740 1 Processor -- start 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.241+0000 7f8fb562a740 1 -- start start 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.241+0000 7f8fab400640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 0x7f8f8406c8f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c033000 tx=0x7f8f9c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd49cad0 0x55bfdd4c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd474650 con 0x55bfdd498ed0 2024-02-02T13:25:55.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd409000 con 0x55bfdd49cad0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.242+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd406790 con 0x55bfdd4c8f50 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56592/0 (socket says 172.21.15.91:56592) 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 -- 172.21.15.91:0/1296638310 learned_addr learned my addr 172.21.15.91:0/1296638310 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 -- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 msgr2=0x55bfdd4c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 -- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd49cad0 msgr2=0x55bfdd4c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd49cad0 0x55bfdd4c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 -- 172.21.15.91:0/1296638310 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfdd3c9710 con 0x55bfdd498ed0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8f940058a0 tx=0x7f8f94005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec9e2be5568dae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8f9be00640 1 -- 172.21.15.91:0/1296638310 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400e950 con 0x55bfdd498ed0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8f9be00640 1 -- 172.21.15.91:0/1296638310 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400ead0 con 0x55bfdd498ed0 2024-02-02T13:25:55.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.243+0000 7f8f9be00640 1 -- 172.21.15.91:0/1296638310 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400ee80 con 0x55bfdd498ed0 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 -- 172.21.15.91:0/1296638310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 msgr2=0x55bfdd4c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8f940058a0 tx=0x7f8f94005d60 comp rx=0 tx=0).stop 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 -- 172.21.15.91:0/1296638310 shutdown_connections 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd49cad0 0x55bfdd4c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd498ed0 0x55bfdd4c61c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1296638310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 -- 172.21.15.91:0/1296638310 >> 172.21.15.91:0/1296638310 conn(0x55bfdd49b1e0 msgr2=0x55bfdd4e3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 -- 172.21.15.91:0/1296638310 shutdown_connections 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.244+0000 7f8fb562a740 1 -- 172.21.15.91:0/1296638310 wait complete. 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.245+0000 7f8fb562a740 1 -- start start 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.245+0000 7f8fb562a740 1 Processor -- start 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd49cad0 0x55bfdd4c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd402080 con 0x55bfdd4c8f50 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd3a3850 con 0x55bfdd4e4600 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd3a34f0 con 0x55bfdd49cad0 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8fb562a740 10 client.0 _ll_register_callbacks cb 0x55bfdd3fb9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56596/0 (socket says 172.21.15.91:56596) 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.246+0000 7f8faaa00640 1 -- 172.21.15.91:0/1595971545 learned_addr learned my addr 172.21.15.91:0/1595971545 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 -- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd49cad0 msgr2=0x55bfdd4c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd49cad0 0x55bfdd4c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 -- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 msgr2=0x55bfdd4c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 -- 172.21.15.91:0/1595971545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9400f040 con 0x55bfdd4c8f50 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8faaa00640 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8f94013000 tx=0x7f8f94001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddd06f1fc74a7e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8f98c00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94014690 con 0x55bfdd4c8f50 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f940147f0 con 0x55bfdd4c8f50 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfdd42c5d0 con 0x55bfdd4c8f50 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f940149b0 con 0x55bfdd4c8f50 2024-02-02T13:25:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.247+0000 7f8fb562a740 10 client.5223 fetch_fsmap learned FSMap version 16 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bfdd400700 con 0x55bfdd4c8f50 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bfdd42c5d0 con 0x55bfdd4c8f50 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8f94014b10 con 0x55bfdd4c8f50 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 10 client.5223 fetch_fsmap finished waiting for FSMap version 16 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 10 client.5223 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 20 client.5223 populate_metadata read hostname 'smithi091' 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4cb560 0x55bfdd4cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 4 client.5223 mds_command: new command op to 5052 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 --> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfdd3f48d0 con 0x55bfdd4cb560 2024-02-02T13:25:55.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.248+0000 7f8fab400640 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4cb560 0x55bfdd4cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.249+0000 7f8f98c00640 10 client.5223 ms_handle_connect on v2:172.21.15.113:6833/3241118880 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.249+0000 7f8fab400640 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4cb560 0x55bfdd4cb940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4753cf128131a29c server_cookie=d3fa7b1f736c1f88 in_seq=0 out_seq=0 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.251+0000 7f8f98c00640 1 -- 172.21.15.91:0/1595971545 <== mds.0 v2:172.21.15.113:6833/3241118880 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bfdd3f48d0 con 0x55bfdd4cb560 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.251+0000 7f8f98c00640 10 client.5223 handle_command_reply: tid=0 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.251+0000 7f8fb562a740 1 client.5223 shutdown 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 msgr2=0x55bfdd4c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8f94013000 tx=0x7f8f94001110 comp rx=0 tx=0).stop 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 shutdown_connections 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4cb560 0x55bfdd4cb940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd4c8f50 0x55bfdd4c9330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 --2- 172.21.15.91:0/1595971545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd49cad0 0x55bfdd4c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8f98c00640 0 client.5223 ms_handle_reset on v2:172.21.15.113:6833/3241118880 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 >> 172.21.15.91:0/1595971545 conn(0x55bfdd498ed0 msgr2=0x55bfdd49d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 shutdown_connections 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 1 -- 172.21.15.91:0/1595971545 wait complete. 2024-02-02T13:25:55.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.252+0000 7f8fb562a740 20 client.5223 trim_cache size 0 max 16384 2024-02-02T13:25:55.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.267+0000 7f8fb562a740 1 Processor -- start 2024-02-02T13:25:55.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.267+0000 7f8fb562a740 1 -- start start 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd49ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd4ccda0 con 0x55bfdd49cad0 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd169f20 con 0x55bfdd4e4600 2024-02-02T13:25:55.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.268+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd4bdd40 con 0x55bfdd4c8f50 2024-02-02T13:25:55.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56606/0 (socket says 172.21.15.91:56606) 2024-02-02T13:25:55.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 -- 172.21.15.91:0/657719905 learned_addr learned my addr 172.21.15.91:0/657719905 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fabe00640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8faaa00640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd49ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 -- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 msgr2=0x55bfdd49ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd49ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 -- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 msgr2=0x55bfdd4bcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4bcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 -- 172.21.15.91:0/657719905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfdd436280 con 0x55bfdd49cad0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8faaa00640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd49ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fabe00640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.269+0000 7f8fab400640 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005040 tx=0x7f8f9c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea6422eb01ad2a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fa0c00640 1 -- 172.21.15.91:0/657719905 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c045040 con 0x55bfdd49cad0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fa0c00640 1 -- 172.21.15.91:0/657719905 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c04a3b0 con 0x55bfdd49cad0 2024-02-02T13:25:55.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fa0c00640 1 -- 172.21.15.91:0/657719905 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0067d0 con 0x55bfdd49cad0 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 -- 172.21.15.91:0/657719905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 msgr2=0x55bfdd4f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005040 tx=0x7f8f9c02d000 comp rx=0 tx=0).stop 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 -- 172.21.15.91:0/657719905 shutdown_connections 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd49ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd4bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 --2- 172.21.15.91:0/657719905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd4f66d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 -- 172.21.15.91:0/657719905 >> 172.21.15.91:0/657719905 conn(0x55bfdd498ed0 msgr2=0x55bfdd498030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 -- 172.21.15.91:0/657719905 shutdown_connections 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.270+0000 7f8fb562a740 1 -- 172.21.15.91:0/657719905 wait complete. 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.271+0000 7f8fb562a740 1 -- start start 2024-02-02T13:25:55.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.271+0000 7f8fb562a740 1 Processor -- start 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd53de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd4f5a40 con 0x55bfdd49cad0 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd54a590 con 0x55bfdd4e4600 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfdd49d4e0 con 0x55bfdd4c8f50 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fb562a740 10 client.0 _ll_register_callbacks cb 0x7f8f94017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fab400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fab400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56620/0 (socket says 172.21.15.91:56620) 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.272+0000 7f8fab400640 1 -- 172.21.15.91:0/2392314399 learned_addr learned my addr 172.21.15.91:0/2392314399 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 -- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 msgr2=0x55bfdd4f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 -- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 msgr2=0x55bfdd53de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd53de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 -- 172.21.15.91:0/2392314399 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c003520 con 0x55bfdd49cad0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fab400640 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02bcb0 tx=0x7f8f9c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf57015df4dce457 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8f98c00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c045070 con 0x55bfdd49cad0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f9c007510 con 0x55bfdd49cad0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f9400ead0 con 0x55bfdd49cad0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f9c0076d0 con 0x55bfdd49cad0 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fb562a740 10 client.5229 fetch_fsmap learned FSMap version 16 2024-02-02T13:25:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.273+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bfdd3a2a60 con 0x55bfdd49cad0 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400ead0 con 0x55bfdd49cad0 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8f9c0067d0 con 0x55bfdd49cad0 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 10 client.5229 fetch_fsmap finished waiting for FSMap version 16 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 10 client.5229 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 20 client.5229 populate_metadata read hostname 'smithi091' 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4f5e60 0x55bfdd4f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 4 client.5229 mds_command: new command op to 5052 tid=0[{"prefix": "status"}] 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 --> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bfdd3fb100 con 0x55bfdd4f5e60 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.274+0000 7f8faaa00640 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4f5e60 0x55bfdd4f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.275+0000 7f8faaa00640 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4f5e60 0x55bfdd4f6240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba58661c99d127b8 server_cookie=915272a4b281b1a8 in_seq=0 out_seq=0 2024-02-02T13:25:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.275+0000 7f8f98c00640 10 client.5229 ms_handle_connect on v2:172.21.15.113:6833/3241118880 2024-02-02T13:25:55.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8f98c00640 1 -- 172.21.15.91:0/2392314399 <== mds.0 v2:172.21.15.113:6833/3241118880 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55bfdd3fb100 con 0x55bfdd4f5e60 2024-02-02T13:25:55.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8f98c00640 10 client.5229 handle_command_reply: tid=0 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 client.5229 shutdown 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 msgr2=0x55bfdd498430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02bcb0 tx=0x7f8f9c001190 comp rx=0 tx=0).stop 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 shutdown_connections 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:6833/3241118880,v1:172.21.15.113:6835/3241118880] conn(0x55bfdd4f5e60 0x55bfdd4f6240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55bfdd4c8f50 0x55bfdd4f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bfdd4e4600 0x55bfdd53de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8f98c00640 0 client.5229 ms_handle_reset on v2:172.21.15.113:6833/3241118880 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 --2- 172.21.15.91:0/2392314399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bfdd49cad0 0x55bfdd498430 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 >> 172.21.15.91:0/2392314399 conn(0x55bfdd498ed0 msgr2=0x55bfdd4bebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.276+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 shutdown_connections 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb562a740 1 -- 172.21.15.91:0/2392314399 wait complete. 2024-02-02T13:25:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb562a740 20 client.5229 trim_cache size 0 max 16384 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5052, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 7.123415874, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 16, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 34, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 34, 2024-02-02T13:25:55.282 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 15.423227362 2024-02-02T13:25:55.283 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:25:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 msgr2=0x7f8f8406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 0x7f8f8406c8f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c033000 tx=0x7f8f9c02f000 comp rx=0 tx=0).stop 2024-02-02T13:25:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 msgr2=0x7f8fac1f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0cc7b0 tx=0x7f8fa40075c0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 shutdown_connections 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f8406a430 0x7f8f8406c8f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8fac1590d0 0x7f8fac1f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fac159550 0x7f8fac1f22f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 --2- 172.21.15.91:0/2941265573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8c0039a0 0x7f8fac1ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 >> 172.21.15.91:0/2941265573 conn(0x7f8fac0b18f0 msgr2=0x7f8fac0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 shutdown_connections 2024-02-02T13:25:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.277+0000 7f8fb2800640 1 -- 172.21.15.91:0/2941265573 wait complete. 2024-02-02T13:25:55.295 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:25:55.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.492+0000 7f7117e00640 1 Processor -- start 2024-02-02T13:25:55.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.492+0000 7f7117e00640 1 -- start start 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71180add80 0x7f71180706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180ce3b0 con 0x7f71180c9450 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180ce8e0 con 0x7f711815d3a0 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180adbd0 con 0x7f71180add80 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f710de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71180add80 0x7f71180706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f71180c9450 0x7f71180c98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53028/0 (socket says 172.21.15.91:53028) 2024-02-02T13:25:55.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.493+0000 7f7117400640 1 -- 172.21.15.91:0/2054424555 learned_addr learned my addr 172.21.15.91:0/2054424555 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:55.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.494+0000 7f710de00640 1 -- 172.21.15.91:0/2054424555 >> v1:172.21.15.113:6789/0 conn(0x7f711815d3a0 legacy=0x7f711806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.494+0000 7f710de00640 1 -- 172.21.15.91:0/2054424555 >> v1:172.21.15.91:6789/0 conn(0x7f71180c9450 legacy=0x7f71180c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.494+0000 7f710de00640 1 -- 172.21.15.91:0/2054424555 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711815d780 con 0x7f71180add80 2024-02-02T13:25:55.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.494+0000 7f7116000640 1 -- 172.21.15.91:0/2054424555 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638682231 0 0) 0x7f71180ce3b0 con 0x7f71180c9450 2024-02-02T13:25:55.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.495+0000 7f710de00640 1 --2- 172.21.15.91:0/2054424555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71180add80 0x7f71180706a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f71000097a0 tx=0x7f710002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ced69e359d16c2e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.495+0000 7f7116000640 1 -- 172.21.15.91:0/2054424555 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710002e070 con 0x7f71180add80 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.495+0000 7f7116000640 1 -- 172.21.15.91:0/2054424555 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71000315a0 con 0x7f71180add80 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.495+0000 7f7116000640 1 -- 172.21.15.91:0/2054424555 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100031930 con 0x7f71180add80 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 -- 172.21.15.91:0/2054424555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71180add80 msgr2=0x7f71180706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 --2- 172.21.15.91:0/2054424555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71180add80 0x7f71180706a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f71000097a0 tx=0x7f710002b840 comp rx=0 tx=0).stop 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 -- 172.21.15.91:0/2054424555 shutdown_connections 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 --2- 172.21.15.91:0/2054424555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71180add80 0x7f71180706a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 -- 172.21.15.91:0/2054424555 >> 172.21.15.91:0/2054424555 conn(0x7f71180cea90 msgr2=0x7f71180ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 -- 172.21.15.91:0/2054424555 shutdown_connections 2024-02-02T13:25:55.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.496+0000 7f7117e00640 1 -- 172.21.15.91:0/2054424555 wait complete. 2024-02-02T13:25:55.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.498+0000 7f7117e00640 1 Processor -- start 2024-02-02T13:25:55.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.498+0000 7f7117e00640 1 -- start start 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 0x7f71181f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71180c9450 0x7f71181f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f34a0 con 0x7f711815d3a0 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f7420 con 0x7f71180c9450 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f2ea0 con 0x7f71180add80 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56632/0 (socket says 172.21.15.91:56632) 2024-02-02T13:25:55.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117400640 1 -- 172.21.15.91:0/585255509 learned_addr learned my addr 172.21.15.91:0/585255509 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f710de00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71180c9450 0x7f71181f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7117400640 1 -- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 msgr2=0x7f71181f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.499+0000 7f7116a00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 0x7f71181f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117400640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 0x7f71181f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117400640 1 -- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71180c9450 msgr2=0x7f71181f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117400640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71180c9450 0x7f71181f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117400640 1 -- 172.21.15.91:0/585255509 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009400 con 0x7f711815d3a0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7116a00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 0x7f71181f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117400640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f711000c830 tx=0x7f711000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000ce20 con 0x7f711815d3a0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711000a640 con 0x7f711815d3a0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71181f3e70 con 0x7f711815d3a0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000a9f0 con 0x7f711815d3a0 2024-02-02T13:25:55.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.500+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711815b930 con 0x7f711815d3a0 2024-02-02T13:25:55.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.501+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71181f3650 con 0x7f711815d3a0 2024-02-02T13:25:55.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.502+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f711000ab50 con 0x7f711815d3a0 2024-02-02T13:25:55.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.503+0000 7f710f400640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 0x7f70f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.503+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7110087de0 con 0x7f711815d3a0 2024-02-02T13:25:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.503+0000 7f7116a00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 0x7f70f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.504+0000 7f7116a00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 0x7f70f006c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7108000f60 tx=0x7f7108009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:55.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.508+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f71181f3650 con 0x7f711815d3a0 2024-02-02T13:25:55.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.734+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f71181f3650 con 0x7f711815d3a0 2024-02-02T13:25:55.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.735+0000 7f710f400640 1 -- 172.21.15.91:0/585255509 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f71181f3650 con 0x7f711815d3a0 2024-02-02T13:25:55.737 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:25:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 msgr2=0x7f70f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 0x7f70f006c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7108000f60 tx=0x7f7108009040 comp rx=0 tx=0).stop 2024-02-02T13:25:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 msgr2=0x7f71180acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f711000c830 tx=0x7f711000ccf0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 shutdown_connections 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f70f006a430 0x7f70f006c8f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f711815d3a0 0x7f71180acb70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71180c9450 0x7f71181f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 --2- 172.21.15.91:0/585255509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71180add80 0x7f71181f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 >> 172.21.15.91:0/585255509 conn(0x7f71180cea90 msgr2=0x7f71180c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 shutdown_connections 2024-02-02T13:25:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:55.738+0000 7f7117e00640 1 -- 172.21.15.91:0/585255509 wait complete. 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:55.757 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:25:55.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:55.757 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:25:55.809 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:25:55.810 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:25:55.811 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:55.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:55.812 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:55.897 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:25:55.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:55.901 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:55.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:55.952 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:25:55.952 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:25:55.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:55.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:55.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:55.953 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:25:56.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:56.132 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:25:56.181 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:25:56.182 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:25:56.183 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:25:56.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:56.183 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:25:56.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:56.264 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:56.313 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:25:56.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:56.314 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:56.360 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:25:56.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:56.361 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:56.546 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:56.546 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:56.579 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:56.579 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:56.579 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:56.579 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:56.579 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:56.581 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:25:56.582 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:56.582 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:56.582 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:56.582 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:56.586 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:56.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:56.586 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:25:56.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:56.674 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:56.748 INFO:teuthology.orchestra.run.smithi091.stdout:client5241 2024-02-02T13:25:56.749 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:25:56.749 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:56.802 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:25:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.012+0000 7f3d92200640 1 Processor -- start 2024-02-02T13:25:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.012+0000 7f3d92200640 1 -- start start 2024-02-02T13:25:57.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d92200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d92200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d92200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c070f80 con 0x7f3d8c151f90 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d92200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c15fa30 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d92200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b10b0 con 0x7f3d8c0b0c40 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d83e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d83e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56634/0 (socket says 172.21.15.91:56634) 2024-02-02T13:25:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.014+0000 7f3d83e00640 1 -- 172.21.15.91:0/2922018692 learned_addr learned my addr 172.21.15.91:0/2922018692 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d83e00640 1 -- 172.21.15.91:0/2922018692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 msgr2=0x7f3d8c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d8be00640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d8aa00640 1 -- 172.21.15.91:0/2922018692 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901519779 0 0) 0x7f3d8c070f80 con 0x7f3d8c151f90 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d83e00640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d83e00640 1 -- 172.21.15.91:0/2922018692 >> v1:172.21.15.91:6789/0 conn(0x7f3d8c151f90 legacy=0x7f3d8c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d83e00640 1 -- 172.21.15.91:0/2922018692 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c152370 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.015+0000 7f3d8be00640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:57.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.016+0000 7f3d83e00640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009870 tx=0x7f3d7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edf94c455c2f637 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.016+0000 7f3d8aa00640 1 -- 172.21.15.91:0/2922018692 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74030840 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.016+0000 7f3d8aa00640 1 -- 172.21.15.91:0/2922018692 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d740309a0 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.016+0000 7f3d8aa00640 1 -- 172.21.15.91:0/2922018692 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74030d30 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 -- 172.21.15.91:0/2922018692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c15f5a0 msgr2=0x7f3d8c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009870 tx=0x7f3d7402b980 comp rx=0 tx=0).stop 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 -- 172.21.15.91:0/2922018692 shutdown_connections 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 --2- 172.21.15.91:0/2922018692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c0b0700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 -- 172.21.15.91:0/2922018692 >> 172.21.15.91:0/2922018692 conn(0x7f3d8c0cec80 msgr2=0x7f3d8c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 -- 172.21.15.91:0/2922018692 shutdown_connections 2024-02-02T13:25:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.017+0000 7f3d92200640 1 -- 172.21.15.91:0/2922018692 wait complete. 2024-02-02T13:25:57.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.019+0000 7f3d92200640 1 Processor -- start 2024-02-02T13:25:57.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.019+0000 7f3d92200640 1 -- start start 2024-02-02T13:25:57.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b1e60 con 0x7f3d8c151f90 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0ae210 con 0x7f3d8c15f5a0 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d92200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b46f0 con 0x7f3d8c0b0c40 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56650/0 (socket says 172.21.15.91:56650) 2024-02-02T13:25:57.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d83e00640 1 -- 172.21.15.91:0/1396068492 learned_addr learned my addr 172.21.15.91:0/1396068492 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:57.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d8be00640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.020+0000 7f3d8b400640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 -- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 msgr2=0x7f3d8c0ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 -- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 msgr2=0x7f3d8c157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 -- 172.21.15.91:0/1396068492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740094d0 con 0x7f3d8c151f90 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d8be00640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83e00640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3d740090b0 tx=0x7f3d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7403e070 con 0x7f3d8c151f90 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74039ba0 con 0x7f3d8c151f90 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c158410 con 0x7f3d8c151f90 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740423f0 con 0x7f3d8c151f90 2024-02-02T13:25:57.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.021+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c0adc50 con 0x7f3d8c151f90 2024-02-02T13:25:57.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.023+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c0c8f00 con 0x7f3d8c151f90 2024-02-02T13:25:57.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.023+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3d74042550 con 0x7f3d8c151f90 2024-02-02T13:25:57.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.024+0000 7f3d83400640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 0x7f3d6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:57.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.024+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d740aaff0 con 0x7f3d8c151f90 2024-02-02T13:25:57.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.025+0000 7f3d8b400640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 0x7f3d6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:57.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.025+0000 7f3d8b400640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 0x7f3d6406c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007fd0 tx=0x7f3d7c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:57.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.029+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3d740765a0 con 0x7f3d8c151f90 2024-02-02T13:25:57.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.257+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d8c0711d0 con 0x7f3d8c151f90 2024-02-02T13:25:57.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.258+0000 7f3d83400640 1 -- 172.21.15.91:0/1396068492 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3d74079bc0 con 0x7f3d8c151f90 2024-02-02T13:25:57.259 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:25:57.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.260+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 msgr2=0x7f3d6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.260+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 0x7f3d6406c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007fd0 tx=0x7f3d7c009040 comp rx=0 tx=0).stop 2024-02-02T13:25:57.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.260+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 msgr2=0x7f3d8c0b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:57.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.260+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3d740090b0 tx=0x7f3d740046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 shutdown_connections 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d6406a430 0x7f3d6406c8f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8c15f5a0 0x7f3d8c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d8c151f90 0x7f3d8c0b41b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 --2- 172.21.15.91:0/1396068492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d8c0b0c40 0x7f3d8c0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 >> 172.21.15.91:0/1396068492 conn(0x7f3d8c0cec80 msgr2=0x7f3d8c15a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 shutdown_connections 2024-02-02T13:25:57.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:57.261+0000 7f3d92200640 1 -- 172.21.15.91:0/1396068492 wait complete. 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:57.282 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:25:57.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:25:57.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:25:57.283 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:25:57.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.283 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:25:57.342 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:25:57.343 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42609sec preferred_lft 42609sec 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:25:57.344 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:25:57.345 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:25:57.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.346 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:25:57.436 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:25:57.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.441 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:25:57.496 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:25:57.496 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:25:57.496 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:25:57.497 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:25:57.497 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:25:57.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:25:57.498 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:25:57.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.681 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:25:57.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:25:57.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:25:57.732 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:25:57.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:25:57.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:57.827 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:57.877 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:25:57.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:25:57.877 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:57.926 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:25:57.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:57.926 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:58.063 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:58.063 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:25:58.097 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:25:58.097 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:25:58.097 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:25:58.097 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:25:58.097 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:25:58.101 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:25:58.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:58.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:25:58.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:58.180 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:58.231 INFO:teuthology.orchestra.run.smithi113.stdout:client5253 2024-02-02T13:25:58.232 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:25:58.232 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:25:58.274 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:25:58.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.489+0000 7f2e54600640 1 Processor -- start 2024-02-02T13:25:58.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.489+0000 7f2e54600640 1 -- start start 2024-02-02T13:25:58.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.490+0000 7f2e54600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e4c155430 0x7f2e4c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e54600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2e4c155810 0x7f2e4c0b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e54600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c071480 con 0x7f2e4c070fc0 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e54600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c071760 con 0x7f2e4c155810 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e54600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c15ebd0 con 0x7f2e4c155430 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e52600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2e4c070fc0 0x7f2e4c15b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53056/0 (socket says 172.21.15.91:53056) 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e52600640 1 -- 172.21.15.91:0/1049886797 learned_addr learned my addr 172.21.15.91:0/1049886797 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e51200640 1 --2- 172.21.15.91:0/1049886797 >> v2:172.21.15.91:3300/0 conn(0x7f2e4c155810 0x7f2e4c0b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e51c00640 1 --2- 172.21.15.91:0/1049886797 >> v2:172.21.15.113:3300/0 conn(0x7f2e4c155430 0x7f2e4c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e4be00640 1 -- 172.21.15.91:0/1049886797 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031050125 0 0) 0x7f2e4c071480 con 0x7f2e4c070fc0 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.491+0000 7f2e4be00640 1 -- 172.21.15.91:0/1049886797 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e2c003520 con 0x7f2e4c070fc0 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.492+0000 7f2e51200640 1 -- 172.21.15.91:0/1049886797 >> v2:172.21.15.113:3300/0 conn(0x7f2e4c155430 msgr2=0x7f2e4c15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.492+0000 7f2e51200640 1 --2- 172.21.15.91:0/1049886797 >> v2:172.21.15.113:3300/0 conn(0x7f2e4c155430 0x7f2e4c15f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.492+0000 7f2e51200640 1 -- 172.21.15.91:0/1049886797 >> v1:172.21.15.91:6789/0 conn(0x7f2e4c070fc0 legacy=0x7f2e4c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.492+0000 7f2e51200640 1 -- 172.21.15.91:0/1049886797 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c15eeb0 con 0x7f2e4c155810 2024-02-02T13:25:58.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.493+0000 7f2e51200640 1 --2- 172.21.15.91:0/1049886797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155810 0x7f2e4c0b0720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0b17d0 tx=0x7f2e3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8d76d278b2d3cf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:58.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.493+0000 7f2e4be00640 1 -- 172.21.15.91:0/1049886797 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3402e070 con 0x7f2e4c155810 2024-02-02T13:25:58.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.493+0000 7f2e4be00640 1 -- 172.21.15.91:0/1049886797 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e340315a0 con 0x7f2e4c155810 2024-02-02T13:25:58.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.493+0000 7f2e4be00640 1 -- 172.21.15.91:0/1049886797 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34031930 con 0x7f2e4c155810 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 -- 172.21.15.91:0/1049886797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155810 msgr2=0x7f2e4c0b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 --2- 172.21.15.91:0/1049886797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155810 0x7f2e4c0b0720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0b17d0 tx=0x7f2e3402b840 comp rx=0 tx=0).stop 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 -- 172.21.15.91:0/1049886797 shutdown_connections 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 --2- 172.21.15.91:0/1049886797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155810 0x7f2e4c0b0720 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 --2- 172.21.15.91:0/1049886797 >> v2:172.21.15.113:3300/0 conn(0x7f2e4c155430 0x7f2e4c15f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 -- 172.21.15.91:0/1049886797 >> 172.21.15.91:0/1049886797 conn(0x7f2e4c0ceca0 msgr2=0x7f2e4c0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 -- 172.21.15.91:0/1049886797 shutdown_connections 2024-02-02T13:25:58.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.494+0000 7f2e54600640 1 -- 172.21.15.91:0/1049886797 wait complete. 2024-02-02T13:25:58.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.496+0000 7f2e54600640 1 Processor -- start 2024-02-02T13:25:58.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.496+0000 7f2e54600640 1 -- start start 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 0x7f2e4c14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e4c155810 0x7f2e4c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c8bd0 con 0x7f2e4c155430 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0b30a0 con 0x7f2e4c070fc0 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e54600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c1454d0 con 0x7f2e4c155810 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56676/0 (socket says 172.21.15.91:56676) 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 -- 172.21.15.91:0/4207872281 learned_addr learned my addr 172.21.15.91:0/4207872281 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51c00640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 0x7f2e4c14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e52600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e4c155810 0x7f2e4c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 -- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e4c155810 msgr2=0x7f2e4c1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e4c155810 0x7f2e4c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 -- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 msgr2=0x7f2e4c14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 0x7f2e4c14ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.497+0000 7f2e51200640 1 -- 172.21.15.91:0/4207872281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34009400 con 0x7f2e4c155430 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e51c00640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 0x7f2e4c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:58.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e51200640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009890 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032840 con 0x7f2e4c155430 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34031f00 con 0x7f2e4c155430 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c1f3e20 con 0x7f2e4c155430 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032be0 con 0x7f2e4c155430 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.498+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c1f43e0 con 0x7f2e4c155430 2024-02-02T13:25:58.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.499+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c0b2e00 con 0x7f2e4c155430 2024-02-02T13:25:58.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.500+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2e34032d40 con 0x7f2e4c155430 2024-02-02T13:25:58.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.501+0000 7f2e49600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 0x7f2e2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:58.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.501+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e340aa880 con 0x7f2e4c155430 2024-02-02T13:25:58.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.501+0000 7f2e51c00640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 0x7f2e2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:58.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.502+0000 7f2e51c00640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 0x7f2e2406c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c006fd0 tx=0x7f2e3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:58.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.506+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2e4c0b2e00 con 0x7f2e4c155430 2024-02-02T13:25:58.655 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:25:58.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:25:58.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.732+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e4c0b2e00 con 0x7f2e4c155430 2024-02-02T13:25:58.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.733+0000 7f2e49600640 1 -- 172.21.15.91:0/4207872281 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f2e4c0b2e00 con 0x7f2e4c155430 2024-02-02T13:25:58.735 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:58.735 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:25:58.735 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 msgr2=0x7f2e2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 0x7f2e2406c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c006fd0 tx=0x7f2e3c008040 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 msgr2=0x7f2e4c144990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009890 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 shutdown_connections 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2e2406a430 0x7f2e2406c8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e4c155810 0x7f2e4c1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e4c155430 0x7f2e4c144990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 --2- 172.21.15.91:0/4207872281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c070fc0 0x7f2e4c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 >> 172.21.15.91:0/4207872281 conn(0x7f2e4c0ceca0 msgr2=0x7f2e4c06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 shutdown_connections 2024-02-02T13:25:58.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:58.735+0000 7f2e54600640 1 -- 172.21.15.91:0/4207872281 wait complete. 2024-02-02T13:25:58.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-02-02T13:25:58.961 INFO:teuthology.orchestra.run.smithi113.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5052,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.792838988,"mdsmap_epoch":16,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":19.092655150999999} 2024-02-02T13:25:58.962 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5052, "mdsmap_epoch": 16, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 10.792838988, "state": "up:active", "uptime": 19.092655151, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:25:58.962 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-02-02T13:25:59.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.175+0000 7f92fa800640 1 Processor -- start 2024-02-02T13:25:59.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.175+0000 7f92fa800640 1 -- start start 2024-02-02T13:25:59.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.176+0000 7f92fa800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92f40cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.176+0000 7f92fa800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92f406d1d0 0x7f92f40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.176+0000 7f92fa800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f40cf840 con 0x7f92f40cafd0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.176+0000 7f92fa800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f415d8a0 con 0x7f92f406d1d0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.176+0000 7f92fa800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4161910 con 0x7f92f415d4c0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f3e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f92f40cafd0 0x7f92f406bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53070/0 (socket says 172.21.15.91:53070) 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f3e00640 1 -- 172.21.15.91:0/4099135784 learned_addr learned my addr 172.21.15.91:0/4099135784 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f3400640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92f40cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92ea800640 1 --2- 172.21.15.91:0/4099135784 >> v2:172.21.15.113:3300/0 conn(0x7f92f406d1d0 0x7f92f40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422161707 0 0) 0x7f92f40cf840 con 0x7f92f40cafd0 2024-02-02T13:25:59.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d4003520 con 0x7f92f40cafd0 2024-02-02T13:25:59.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891503425 0 0) 0x7f92d4003520 con 0x7f92f40cafd0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 msgr2=0x7f92f40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92f40cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 >> v2:172.21.15.113:3300/0 conn(0x7f92f406d1d0 msgr2=0x7f92f40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 --2- 172.21.15.91:0/4099135784 >> v2:172.21.15.113:3300/0 conn(0x7f92f406d1d0 0x7f92f40cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4161e40 con 0x7f92f40cafd0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f92ec0019f0 con 0x7f92f40cafd0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 >> v1:172.21.15.91:6789/0 conn(0x7f92f40cafd0 legacy=0x7f92f406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.177+0000 7f92f2a00640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d4003970 0x7f92d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f2a00640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d40047c0 0x7f92d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f2a00640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f415d8a0 con 0x7f92d40047c0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4161910 con 0x7f92d4003970 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400b980 con 0x7f92f415d4c0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d40047c0 0x7f92d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92f3e00640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 -- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 msgr2=0x7f92d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f415d4c0 0x7f92d4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 -- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d4003970 msgr2=0x7f92d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d4003970 0x7f92d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.178+0000 7f92ea800640 1 -- 172.21.15.91:0/4099135784 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4003520 con 0x7f92d40047c0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.179+0000 7f92ea800640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d40047c0 0x7f92d4004c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009af0 tx=0x7f92d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc1c942ba054bc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.179+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02c710 con 0x7f92d40047c0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.179+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc02c870 con 0x7f92d40047c0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.179+0000 7f92f2a00640 1 -- 172.21.15.91:0/4099135784 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02cc00 con 0x7f92d40047c0 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 -- 172.21.15.91:0/4099135784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d40047c0 msgr2=0x7f92d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 --2- 172.21.15.91:0/4099135784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d40047c0 0x7f92d4004c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009af0 tx=0x7f92d40057f0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92f3e00640 1 -- 172.21.15.91:0/4099135784 reap_dead start 2024-02-02T13:25:59.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 -- 172.21.15.91:0/4099135784 shutdown_connections 2024-02-02T13:25:59.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 -- 172.21.15.91:0/4099135784 >> 172.21.15.91:0/4099135784 conn(0x7f92f40ced20 msgr2=0x7f92f40cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:59.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 -- 172.21.15.91:0/4099135784 shutdown_connections 2024-02-02T13:25:59.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.180+0000 7f92fa800640 1 -- 172.21.15.91:0/4099135784 wait complete. 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.182+0000 7f92fa800640 1 Processor -- start 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.182+0000 7f92fa800640 1 -- start start 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 0x7f92f4160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414cdf0 0x7f92f41f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4157cb0 con 0x7f92f415dae0 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40b1c40 con 0x7f92f414cdf0 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92fa800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f41606b0 con 0x7f92d40047c0 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56696/0 (socket says 172.21.15.91:56696) 2024-02-02T13:25:59.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 -- 172.21.15.91:0/3796941882 learned_addr learned my addr 172.21.15.91:0/3796941882 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92f3400640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 0x7f92f4160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 -- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 msgr2=0x7f92f4160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 0x7f92f4160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92f3e00640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414cdf0 0x7f92f41f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 -- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414cdf0 msgr2=0x7f92f41f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.183+0000 7f92ea800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414cdf0 0x7f92f41f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92ea800640 1 -- 172.21.15.91:0/3796941882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4157f50 con 0x7f92f415dae0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92f3400640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 0x7f92f4160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92ea800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92dc02c2b0 tx=0x7f92dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc003810 con 0x7f92f415dae0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc02de90 con 0x7f92f415dae0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f4158230 con 0x7f92f415dae0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc003bd0 con 0x7f92f415dae0 2024-02-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.184+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f415df60 con 0x7f92f415dae0 2024-02-02T13:25:59.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.185+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f406ca70 con 0x7f92f415dae0 2024-02-02T13:25:59.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.186+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92dc003d30 con 0x7f92f415dae0 2024-02-02T13:25:59.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.187+0000 7f92ebe00640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 0x7f92cc06c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:25:59.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.187+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92dc0aa990 con 0x7f92f415dae0 2024-02-02T13:25:59.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.187+0000 7f92f3400640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 0x7f92cc06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:25:59.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.188+0000 7f92f3400640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 0x7f92cc06c970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009950 tx=0x7f92e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:25:59.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.191+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f92dc075f60 con 0x7f92f415dae0 2024-02-02T13:25:59.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.451+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f92f41583c0 con 0x7f92f415dae0 2024-02-02T13:25:59.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.452+0000 7f92ebe00640 1 -- 172.21.15.91:0/3796941882 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f92dc079580 con 0x7f92f415dae0 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "stats": { 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "total_bytes": 1600000000, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "total_avail_bytes": 1597604736, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_bytes": 2395264, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_raw_bytes": 2395264, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_raw_ratio": 0.0014970400370657444, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "num_osds": 8, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "num_per_pool_osds": 0, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "num_per_pool_omap_osds": 0 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "stats_by_class": { 2024-02-02T13:25:59.454 INFO:teuthology.orchestra.run.smithi091.stdout: "hdd": { 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "total_bytes": 1600000000, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "total_avail_bytes": 1597604736, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_bytes": 2395264, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_raw_bytes": 2395264, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "total_used_raw_ratio": 0.0014970400370657444 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "name": ".mgr", 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 1, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "stats": { 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "stored": 590368, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "objects": 2, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "kb_used": 577, 2024-02-02T13:25:59.455 INFO:teuthology.orchestra.run.smithi091.stdout: "bytes_used": 590368, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "percent_used": 0.00052964512724429369, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "max_avail": 557028928 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "name": "cephfs_metadata", 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 4, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "stats": { 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "stored": 2362, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "objects": 22, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "kb_used": 3, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "bytes_used": 2362, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "percent_used": 2.1201726667641196e-06, 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: "max_avail": 557028928 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:59.456 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "name": "cephfs_data", 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "stats": { 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "stored": 0, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "objects": 0, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "kb_used": 0, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "bytes_used": 0, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "percent_used": 0, 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: "max_avail": 557028928 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:25:59.457 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:25:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 msgr2=0x7f92cc06c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 0x7f92cc06c970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009950 tx=0x7f92e4008040 comp rx=0 tx=0).stop 2024-02-02T13:25:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 msgr2=0x7f92f414c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:25:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92dc02c2b0 tx=0x7f92dc004730 comp rx=0 tx=0).stop 2024-02-02T13:25:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 shutdown_connections 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a4b0 0x7f92cc06c970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414cdf0 0x7f92f41f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f415dae0 0x7f92f414c8b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 --2- 172.21.15.91:0/3796941882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d40047c0 0x7f92f4160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 >> 172.21.15.91:0/3796941882 conn(0x7f92f40ced20 msgr2=0x7f92f40cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 shutdown_connections 2024-02-02T13:25:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:25:59.455+0000 7f92fa800640 1 -- 172.21.15.91:0/3796941882 wait complete. 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:25:59.476 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:25:59.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:59.794 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:25:59.898 INFO:teuthology.orchestra.run.smithi091.stdout:client5241 2024-02-02T13:25:59.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:25:59.899 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5241/osdmap') 2024-02-02T13:25:59.992 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 34 barrier 34 flags 0x588000 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:25:59.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:25:59.994 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:25:59.994 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:25:59.994 INFO:teuthology.orchestra.run.smithi091.stderr:795 bytes copied, 0.000140939 s, 5.6 MB/s 2024-02-02T13:25:59.997 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-02-02T13:26:00.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.207+0000 7f7087000640 1 Processor -- start 2024-02-02T13:26:00.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.207+0000 7f7087000640 1 -- start start 2024-02-02T13:26:00.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f7087000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f708015d870 0x7f70801644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:00.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f7087000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70800cd5e0 con 0x7f7080159340 2024-02-02T13:26:00.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f7087000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70800cd8c0 con 0x7f70800cd150 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f7087000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7080070630 con 0x7f708015d870 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f7085000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f708015d870 0x7f70801644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f707fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7080159340 0x7f708006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42484/0 (socket says 172.21.15.91:42484) 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.209+0000 7f707fe00640 1 -- 172.21.15.91:0/1798250778 learned_addr learned my addr 172.21.15.91:0/1798250778 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.210+0000 7f7085000640 1 -- 172.21.15.91:0/1798250778 >> v1:172.21.15.113:6789/0 conn(0x7f70800cd150 legacy=0x7f708015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.210+0000 7f7085000640 1 -- 172.21.15.91:0/1798250778 >> v1:172.21.15.91:6789/0 conn(0x7f7080159340 legacy=0x7f708006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.210+0000 7f7085000640 1 -- 172.21.15.91:0/1798250778 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70800cdba0 con 0x7f708015d870 2024-02-02T13:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.210+0000 7f707ea00640 1 -- 172.21.15.91:0/1798250778 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476575763 0 0) 0x7f70800cd5e0 con 0x7f7080159340 2024-02-02T13:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.211+0000 7f7085000640 1 --2- 172.21.15.91:0/1798250778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f708015d870 0x7f70801644f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7078009d90 tx=0x7f707802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f74c26efa00a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.211+0000 7f707ea00640 1 -- 172.21.15.91:0/1798250778 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707802cc60 con 0x7f708015d870 2024-02-02T13:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.211+0000 7f707ea00640 1 -- 172.21.15.91:0/1798250778 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707802cdc0 con 0x7f708015d870 2024-02-02T13:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.211+0000 7f707ea00640 1 -- 172.21.15.91:0/1798250778 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078034660 con 0x7f708015d870 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 -- 172.21.15.91:0/1798250778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f708015d870 msgr2=0x7f70801644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 --2- 172.21.15.91:0/1798250778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f708015d870 0x7f70801644f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7078009d90 tx=0x7f707802beb0 comp rx=0 tx=0).stop 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 -- 172.21.15.91:0/1798250778 shutdown_connections 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 --2- 172.21.15.91:0/1798250778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f708015d870 0x7f70801644f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 -- 172.21.15.91:0/1798250778 >> 172.21.15.91:0/1798250778 conn(0x7f70800b0700 msgr2=0x7f70800b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 -- 172.21.15.91:0/1798250778 shutdown_connections 2024-02-02T13:26:00.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.212+0000 7f7087000640 1 -- 172.21.15.91:0/1798250778 wait complete. 2024-02-02T13:26:00.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.214+0000 7f7087000640 1 Processor -- start 2024-02-02T13:26:00.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.214+0000 7f7087000640 1 -- start start 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708015d870 0x7f708015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70800b1790 0x7f708015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801553f0 con 0x7f7080159340 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c4120 con 0x7f70800b1790 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f7087000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c3b60 con 0x7f708015d870 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f707fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f707fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38872/0 (socket says 172.21.15.91:38872) 2024-02-02T13:26:00.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.215+0000 7f707fe00640 1 -- 172.21.15.91:0/2119455819 learned_addr learned my addr 172.21.15.91:0/2119455819 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:00.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707f400640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708015d870 0x7f708015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:00.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 -- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708015d870 msgr2=0x7f708015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:00.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f7085000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70800b1790 0x7f708015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708015d870 0x7f708015bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 -- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70800b1790 msgr2=0x7f708015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70800b1790 0x7f708015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 -- 172.21.15.91:0/2119455819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009a70 con 0x7f7080159340 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f707fe00640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f707000c4d0 tx=0x7f707000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70700074e0 con 0x7f7080159340 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070007660 con 0x7f7080159340 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070004a20 con 0x7f7080159340 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.216+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70800c3e40 con 0x7f7080159340 2024-02-02T13:26:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.217+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70800c7680 con 0x7f7080159340 2024-02-02T13:26:00.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.218+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70800c78a0 con 0x7f7080159340 2024-02-02T13:26:00.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.218+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7070004b80 con 0x7f7080159340 2024-02-02T13:26:00.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.220+0000 7f7077e00640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 0x7f705806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:00.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.220+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7070087a10 con 0x7f7080159340 2024-02-02T13:26:00.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.220+0000 7f707f400640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 0x7f705806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:00.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.221+0000 7f707f400640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 0x7f705806c920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70800ccf10 tx=0x7f7068008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:00.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.225+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7070053090 con 0x7f7080159340 2024-02-02T13:26:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:00.425+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f7080066cf0 con 0x7f7080159340 2024-02-02T13:26:01.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.296+0000 7f7077e00640 1 -- 172.21.15.91:0/2119455819 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7070018020 con 0x7f7080159340 2024-02-02T13:26:01.298 INFO:teuthology.orchestra.run.smithi091.stderr:pauserd,pausewr is set 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 msgr2=0x7f705806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 0x7f705806c920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70800ccf10 tx=0x7f7068008040 comp rx=0 tx=0).stop 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 msgr2=0x7f7080154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f707000c4d0 tx=0x7f707000c990 comp rx=0 tx=0).stop 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 shutdown_connections 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f705806a460 0x7f705806c920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708015d870 0x7f708015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70800b1790 0x7f708015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 --2- 172.21.15.91:0/2119455819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7080159340 0x7f7080154eb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 >> 172.21.15.91:0/2119455819 conn(0x7f70800b0700 msgr2=0x7f70800cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.300+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 shutdown_connections 2024-02-02T13:26:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.301+0000 7f7087000640 1 -- 172.21.15.91:0/2119455819 wait complete. 2024-02-02T13:26:01.322 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-02-02T13:26:01.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.531+0000 7fab9ca00640 1 Processor -- start 2024-02-02T13:26:01.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.531+0000 7fab9ca00640 1 -- start start 2024-02-02T13:26:01.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab9415ad10 0x7fab9406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9ca00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9406d910 con 0x7fab940cb580 2024-02-02T13:26:01.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab940cc850 con 0x7fab9415b0f0 2024-02-02T13:26:01.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9406d730 con 0x7fab9415ad10 2024-02-02T13:26:01.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9aa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fab940cb580 0x7fab940c26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42498/0 (socket says 172.21.15.91:42498) 2024-02-02T13:26:01.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.533+0000 7fab9aa00640 1 -- 172.21.15.91:0/2160796234 learned_addr learned my addr 172.21.15.91:0/2160796234 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab9a000640 1 --2- 172.21.15.91:0/2160796234 >> v2:172.21.15.113:3300/0 conn(0x7fab9415ad10 0x7fab9406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001608423 0 0) 0x7fab9406d910 con 0x7fab940cb580 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab74003520 con 0x7fab940cb580 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2338179857 0 0) 0x7fab74003520 con 0x7fab940cb580 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 >> v2:172.21.15.113:3300/0 conn(0x7fab9415ad10 msgr2=0x7fab9406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 --2- 172.21.15.91:0/2160796234 >> v2:172.21.15.113:3300/0 conn(0x7fab9415ad10 0x7fab9406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 >> v1:172.21.15.113:6789/0 conn(0x7fab9415b0f0 legacy=0x7fab9406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940ccb30 con 0x7fab940cb580 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fab84001670 con 0x7fab940cb580 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 >> v1:172.21.15.91:6789/0 conn(0x7fab940cb580 legacy=0x7fab940c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 0x7fab74003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab740047c0 0x7fab74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740051d0 0x7fab7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9406d730 con 0x7fab740039c0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400c120 con 0x7fab740047c0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.534+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400c2e0 con 0x7fab740051d0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 0x7fab74003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab99600640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab740047c0 0x7fab74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 -- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740051d0 msgr2=0x7fab7400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740051d0 0x7fab7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 -- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab740047c0 msgr2=0x7fab74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab740047c0 0x7fab74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9a000640 1 -- 172.21.15.91:0/2160796234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74003520 con 0x7fab740039c0 2024-02-02T13:26:01.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab99600640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab740047c0 0x7fab74004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:01.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.535+0000 7fab9aa00640 1 -- 172.21.15.91:0/2160796234 reap_dead start 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.536+0000 7fab9a000640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 0x7fab74003de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009a90 tx=0x7fab740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8af51a9b9a7965 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.536+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c02c680 con 0x7fab740039c0 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.536+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c02c7e0 con 0x7fab740039c0 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.536+0000 7fab98c00640 1 -- 172.21.15.91:0/2160796234 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c02cb70 con 0x7fab740039c0 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 -- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 msgr2=0x7fab74003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 0x7fab74003de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009a90 tx=0x7fab740057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 -- 172.21.15.91:0/2160796234 shutdown_connections 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 --2- 172.21.15.91:0/2160796234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab740039c0 0x7fab74003de0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 -- 172.21.15.91:0/2160796234 >> 172.21.15.91:0/2160796234 conn(0x7fab940b0700 msgr2=0x7fab940b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 -- 172.21.15.91:0/2160796234 shutdown_connections 2024-02-02T13:26:01.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.537+0000 7fab9ca00640 1 -- 172.21.15.91:0/2160796234 wait complete. 2024-02-02T13:26:01.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.539+0000 7fab9ca00640 1 Processor -- start 2024-02-02T13:26:01.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.539+0000 7fab9ca00640 1 -- start start 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740039c0 0x7fab94152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 0x7fab9414fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab940c8a60 con 0x7fab941549c0 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab940c5d70 con 0x7fab940c8250 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab940c6050 con 0x7fab740039c0 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38888/0 (socket says 172.21.15.91:38888) 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.540+0000 7fab9aa00640 1 -- 172.21.15.91:0/1589262106 learned_addr learned my addr 172.21.15.91:0/1589262106 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9a000640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740039c0 0x7fab94152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab99600640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 0x7fab9414fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 -- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740039c0 msgr2=0x7fab94152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740039c0 0x7fab94152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 -- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 msgr2=0x7fab9414fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 0x7fab9414fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940c6330 con 0x7fab941549c0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab99600640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 0x7fab9414fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9aa00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab940cc740 tx=0x7fab8400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840074e0 con 0x7fab941549c0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab84007640 con 0x7fab941549c0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84004a00 con 0x7fab941549c0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab940c6610 con 0x7fab941549c0 2024-02-02T13:26:01.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.541+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab940c86b0 con 0x7fab941549c0 2024-02-02T13:26:01.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.543+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab94066cf0 con 0x7fab941549c0 2024-02-02T13:26:01.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.543+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fab84004b60 con 0x7fab941549c0 2024-02-02T13:26:01.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.545+0000 7fab92000640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 0x7fab6c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:01.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.545+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab84088390 con 0x7fab941549c0 2024-02-02T13:26:01.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.545+0000 7fab9a000640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 0x7fab6c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:01.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.545+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=36+}) v3 -- 0x7fab6c071f90 con 0x7fab941549c0 2024-02-02T13:26:01.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.546+0000 7fab9a000640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 0x7fab6c06c920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab8c036000 tx=0x7fab8c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.549+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fab84019020 con 0x7fab941549c0 2024-02-02T13:26:01.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:01.753+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fab94071670 con 0x7fab941549c0 2024-02-02T13:26:02.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.303+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(36..36 src has 1..36) v4 ==== 288+0+0 (secure 0 0 0) 0x7fab8400e280 con 0x7fab941549c0 2024-02-02T13:26:02.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.303+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=37}) v3 -- 0x7fab6c072840 con 0x7fab941549c0 2024-02-02T13:26:02.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.308+0000 7fab92000640 1 -- 172.21.15.91:0/1589262106 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7fab84057400 con 0x7fab941549c0 2024-02-02T13:26:02.310 INFO:teuthology.orchestra.run.smithi091.stderr:pauserd,pausewr is unset 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 msgr2=0x7fab6c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 0x7fab6c06c920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab8c036000 tx=0x7fab8c030000 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 msgr2=0x7fab94150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab940cc740 tx=0x7fab8400c960 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 shutdown_connections 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6c06a460 0x7fab6c06c920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab941549c0 0x7fab94150a60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.312+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab940c8250 0x7fab9414fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.313+0000 7fab9ca00640 1 --2- 172.21.15.91:0/1589262106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab740039c0 0x7fab94152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.313+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 >> 172.21.15.91:0/1589262106 conn(0x7fab940b0700 msgr2=0x7fab9406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:02.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.313+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 shutdown_connections 2024-02-02T13:26:02.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.313+0000 7fab9ca00640 1 -- 172.21.15.91:0/1589262106 wait complete. 2024-02-02T13:26:02.337 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:02.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.541+0000 7f9cae200640 1 Processor -- start 2024-02-02T13:26:02.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.541+0000 7f9cae200640 1 -- start start 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.542+0000 7f9cae200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9ca815d330 0x7f9ca80b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.542+0000 7f9cae200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8161090 con 0x7f9ca80b2470 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.542+0000 7f9cae200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8161330 con 0x7f9ca80d3850 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.542+0000 7f9cae200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca8161cd0 con 0x7f9ca815d330 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9ca815d330 0x7f9ca80b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9ca815d330 0x7f9ca80b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38904/0 (socket says 172.21.15.91:38904) 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 -- 172.21.15.91:0/1620611398 learned_addr learned my addr 172.21.15.91:0/1620611398 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca6a00640 1 --1- 172.21.15.91:0/1620611398 >> v1:172.21.15.91:6789/0 conn(0x7f9ca80b2470 0x7f9ca80d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42512/0 (socket says 172.21.15.91:42512) 2024-02-02T13:26:02.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 -- 172.21.15.91:0/1620611398 >> v1:172.21.15.113:6789/0 conn(0x7f9ca80d3850 legacy=0x7f9ca80d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:02.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 -- 172.21.15.91:0/1620611398 >> v1:172.21.15.91:6789/0 conn(0x7f9ca80b2470 legacy=0x7f9ca80d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.543+0000 7f9ca7400640 1 -- 172.21.15.91:0/1620611398 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8161610 con 0x7f9ca815d330 2024-02-02T13:26:02.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.544+0000 7f9ca7400640 1 --2- 172.21.15.91:0/1620611398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815d330 0x7f9ca80b8b40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009850 tx=0x7f9c9802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f528e00748786647 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:02.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.544+0000 7f9ca6000640 1 -- 172.21.15.91:0/1620611398 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004230 con 0x7f9ca815d330 2024-02-02T13:26:02.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.544+0000 7f9ca6000640 1 -- 172.21.15.91:0/1620611398 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98004390 con 0x7f9ca815d330 2024-02-02T13:26:02.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.544+0000 7f9ca6000640 1 -- 172.21.15.91:0/1620611398 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98003810 con 0x7f9ca815d330 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 -- 172.21.15.91:0/1620611398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815d330 msgr2=0x7f9ca80b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 --2- 172.21.15.91:0/1620611398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815d330 0x7f9ca80b8b40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009850 tx=0x7f9c9802b860 comp rx=0 tx=0).stop 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 -- 172.21.15.91:0/1620611398 shutdown_connections 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 --2- 172.21.15.91:0/1620611398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca815d330 0x7f9ca80b8b40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 -- 172.21.15.91:0/1620611398 >> 172.21.15.91:0/1620611398 conn(0x7f9ca8070660 msgr2=0x7f9ca8070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 -- 172.21.15.91:0/1620611398 shutdown_connections 2024-02-02T13:26:02.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.545+0000 7f9cae200640 1 -- 172.21.15.91:0/1620611398 wait complete. 2024-02-02T13:26:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.547+0000 7f9cae200640 1 Processor -- start 2024-02-02T13:26:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.547+0000 7f9cae200640 1 -- start start 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca80b2470 0x7f9ca8158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca815d330 0x7f9ca80ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815e790 con 0x7f9ca80d3850 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815ea70 con 0x7f9ca80b2470 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9cae200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815ed50 con 0x7f9ca815d330 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38918/0 (socket says 172.21.15.91:38918) 2024-02-02T13:26:02.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.548+0000 7f9ca6a00640 1 -- 172.21.15.91:0/439489830 learned_addr learned my addr 172.21.15.91:0/439489830 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:02.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 -- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca815d330 msgr2=0x7f9ca80ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca815d330 0x7f9ca80ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 -- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca80b2470 msgr2=0x7f9ca8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca80b2470 0x7f9ca8158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 -- 172.21.15.91:0/439489830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c98009460 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9ca6a00640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c3e0 tx=0x7f9c9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900074e0 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90007640 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca815f030 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900049e0 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.549+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca815da20 con 0x7f9ca80d3850 2024-02-02T13:26:02.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.551+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca8067400 con 0x7f9ca80d3850 2024-02-02T13:26:02.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.551+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c90004b40 con 0x7f9ca80d3850 2024-02-02T13:26:02.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.552+0000 7f9c9f400640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 0x7f9c8006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:02.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.552+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c90088300 con 0x7f9ca80d3850 2024-02-02T13:26:02.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.552+0000 7f9ca7400640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 0x7f9c8006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:02.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.553+0000 7f9ca7400640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 0x7f9c8006c920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c98034000 tx=0x7f9c98030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:02.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.557+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c900538e0 con 0x7f9ca80d3850 2024-02-02T13:26:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.746+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ca80b76a0 con 0x7f9ca80d3850 2024-02-02T13:26:02.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.748+0000 7f9c9f400640 1 -- 172.21.15.91:0/439489830 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f9c90056f00 con 0x7f9ca80d3850 2024-02-02T13:26:02.750 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:02.750 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:02.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.750+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 msgr2=0x7f9c8006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.750+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 0x7f9c8006c920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c98034000 tx=0x7f9c98030000 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.750+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 msgr2=0x7f9ca80ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.750+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c3e0 tx=0x7f9c9000c8a0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 shutdown_connections 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9c8006a460 0x7f9c8006c920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca815d330 0x7f9ca80ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ca80d3850 0x7f9ca80ca280 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 --2- 172.21.15.91:0/439489830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca80b2470 0x7f9ca8158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 >> 172.21.15.91:0/439489830 conn(0x7f9ca8070660 msgr2=0x7f9ca80b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 shutdown_connections 2024-02-02T13:26:02.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:02.751+0000 7f9cae200640 1 -- 172.21.15.91:0/439489830 wait complete. 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:26:02.772 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:26:03.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:03.089 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:26:03.191 INFO:teuthology.orchestra.run.smithi091.stdout:client5241 2024-02-02T13:26:03.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:03.191 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5241/osdmap') 2024-02-02T13:26:03.285 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 34 barrier 34 flags 0x588000 2024-02-02T13:26:03.285 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:03.285 INFO:teuthology.orchestra.run.smithi091.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:26:03.286 INFO:teuthology.orchestra.run.smithi091.stderr:795 bytes copied, 0.000145454 s, 5.5 MB/s 2024-02-02T13:26:03.291 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:03.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.494+0000 7fe43a400640 1 Processor -- start 2024-02-02T13:26:03.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.495+0000 7fe43a400640 1 -- start start 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe43a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe43415d330 0x7fe4340b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe43a400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe434161090 con 0x7fe4340b2470 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe43a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe434161330 con 0x7fe4340d3850 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe43a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe434161cd0 con 0x7fe43415d330 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe433400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe43415d330 0x7fe4340b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe433400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe43415d330 0x7fe4340b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38926/0 (socket says 172.21.15.91:38926) 2024-02-02T13:26:03.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe432a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe4340b2470 0x7fe4340d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42516/0 (socket says 172.21.15.91:42516) 2024-02-02T13:26:03.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.496+0000 7fe433400640 1 -- 172.21.15.91:0/768275086 learned_addr learned my addr 172.21.15.91:0/768275086 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:03.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.497+0000 7fe433400640 1 -- 172.21.15.91:0/768275086 >> v1:172.21.15.113:6789/0 conn(0x7fe4340d3850 legacy=0x7fe4340d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:03.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.497+0000 7fe433400640 1 -- 172.21.15.91:0/768275086 >> v1:172.21.15.91:6789/0 conn(0x7fe4340b2470 legacy=0x7fe4340d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.497+0000 7fe433400640 1 -- 172.21.15.91:0/768275086 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe434161610 con 0x7fe43415d330 2024-02-02T13:26:03.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.498+0000 7fe433400640 1 --2- 172.21.15.91:0/768275086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe43415d330 0x7fe4340b8b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe424009850 tx=0x7fe42402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c45ee9695540e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:03.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.498+0000 7fe432000640 1 -- 172.21.15.91:0/768275086 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424004230 con 0x7fe43415d330 2024-02-02T13:26:03.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.498+0000 7fe432000640 1 -- 172.21.15.91:0/768275086 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe424004390 con 0x7fe43415d330 2024-02-02T13:26:03.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.498+0000 7fe432000640 1 -- 172.21.15.91:0/768275086 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424003810 con 0x7fe43415d330 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 -- 172.21.15.91:0/768275086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe43415d330 msgr2=0x7fe4340b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 --2- 172.21.15.91:0/768275086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe43415d330 0x7fe4340b8b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe424009850 tx=0x7fe42402b860 comp rx=0 tx=0).stop 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 -- 172.21.15.91:0/768275086 shutdown_connections 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 --2- 172.21.15.91:0/768275086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe43415d330 0x7fe4340b8b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 -- 172.21.15.91:0/768275086 >> 172.21.15.91:0/768275086 conn(0x7fe434070660 msgr2=0x7fe434070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 -- 172.21.15.91:0/768275086 shutdown_connections 2024-02-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.499+0000 7fe43a400640 1 -- 172.21.15.91:0/768275086 wait complete. 2024-02-02T13:26:03.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.501+0000 7fe43a400640 1 Processor -- start 2024-02-02T13:26:03.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.501+0000 7fe43a400640 1 -- start start 2024-02-02T13:26:03.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 0x7fe434158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:03.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:03.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43415d330 0x7fe4340ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:03.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43415e790 con 0x7fe4340d3850 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43415ea70 con 0x7fe4340b2470 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe43a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43415ed50 con 0x7fe43415d330 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe432a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe432a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38934/0 (socket says 172.21.15.91:38934) 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe432a00640 1 -- 172.21.15.91:0/2867736018 learned_addr learned my addr 172.21.15.91:0/2867736018 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:03.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe433400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 0x7fe434158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.502+0000 7fe433e00640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43415d330 0x7fe4340ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 -- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43415d330 msgr2=0x7fe4340ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43415d330 0x7fe4340ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 -- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 msgr2=0x7fe434158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 0x7fe434158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 -- 172.21.15.91:0/2867736018 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009460 con 0x7fe4340d3850 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe433400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 0x7fe434158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe432a00640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00c7a0 tx=0x7fe41c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00a640 con 0x7fe4340d3850 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c00a7a0 con 0x7fe4340d3850 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43415f030 con 0x7fe4340d3850 2024-02-02T13:26:03.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00ab10 con 0x7fe4340d3850 2024-02-02T13:26:03.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.503+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43415da20 con 0x7fe4340d3850 2024-02-02T13:26:03.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.504+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe434067400 con 0x7fe4340d3850 2024-02-02T13:26:03.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.505+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe41c00ac70 con 0x7fe4340d3850 2024-02-02T13:26:03.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.506+0000 7fe42b400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 0x7fe40c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.506+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe41c087fb0 con 0x7fe4340d3850 2024-02-02T13:26:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.506+0000 7fe433400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 0x7fe40c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.507+0000 7fe433400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 0x7fe40c06c920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe424034000 tx=0x7fe424008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.510+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe41c053640 con 0x7fe4340d3850 2024-02-02T13:26:03.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.738+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4340b76a0 con 0x7fe4340d3850 2024-02-02T13:26:03.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.739+0000 7fe42b400640 1 -- 172.21.15.91:0/2867736018 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fe41c017090 con 0x7fe4340d3850 2024-02-02T13:26:03.741 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:03.741 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:26:03.741 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:26:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.742+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 msgr2=0x7fe40c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.742+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 0x7fe40c06c920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe424034000 tx=0x7fe424008030 comp rx=0 tx=0).stop 2024-02-02T13:26:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.742+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 msgr2=0x7fe4340ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.742+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00c7a0 tx=0x7fe41c00cc60 comp rx=0 tx=0).stop 2024-02-02T13:26:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.742+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 shutdown_connections 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe40c06a460 0x7fe40c06c920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43415d330 0x7fe4340ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4340d3850 0x7fe4340ca280 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 --2- 172.21.15.91:0/2867736018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4340b2470 0x7fe434158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 >> 172.21.15.91:0/2867736018 conn(0x7fe434070660 msgr2=0x7fe4340b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 shutdown_connections 2024-02-02T13:26:03.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:03.743+0000 7fe43a400640 1 -- 172.21.15.91:0/2867736018 wait complete. 2024-02-02T13:26:03.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json osdmap barrier 36 2024-02-02T13:26:03.989 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:26:03.990 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:26:04.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:04.317 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:26:04.421 INFO:teuthology.orchestra.run.smithi091.stdout:client5241 2024-02-02T13:26:04.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:04.421 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5241/osdmap') 2024-02-02T13:26:04.514 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 36 barrier 36 flags 0x588000 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:26:04.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:26:04.516 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:26:04.516 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:26:04.516 INFO:teuthology.orchestra.run.smithi091.stderr:795 bytes copied, 8.3372e-05 s, 9.5 MB/s 2024-02-02T13:26:04.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:04.519 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:26:04.577 INFO:teuthology.orchestra.run.smithi091.stdout:client5241 2024-02-02T13:26:04.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:04.577 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5241/osdmap') 2024-02-02T13:26:04.665 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 36 barrier 36 flags 0x588000 2024-02-02T13:26:04.665 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.665 INFO:teuthology.orchestra.run.smithi091.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.665 INFO:teuthology.orchestra.run.smithi091.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:26:04.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:26:04.667 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:26:04.667 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:26:04.667 INFO:teuthology.orchestra.run.smithi091.stderr:795 bytes copied, 9.2331e-05 s, 8.6 MB/s 2024-02-02T13:26:04.669 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:26:04.669 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:26:04.669 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:26:04.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:26:04.893 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:26:04.893 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:04.893 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:04.951 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:04.952 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:05.004 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:05.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.217+0000 7fc603000640 1 Processor -- start 2024-02-02T13:26:05.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.217+0000 7fc603000640 1 -- start start 2024-02-02T13:26:05.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.218+0000 7fc603000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5fc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.218+0000 7fc603000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5fc0b0c40 0x7fc5fc0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.218+0000 7fc603000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc15fa00 con 0x7fc5fc151f90 2024-02-02T13:26:05.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.218+0000 7fc603000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc0b10d0 con 0x7fc5fc0b0c40 2024-02-02T13:26:05.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.218+0000 7fc603000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc070f80 con 0x7fc5fc15f5a0 2024-02-02T13:26:05.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc5fbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc5fc151f90 0x7fc5fc06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42528/0 (socket says 172.21.15.91:42528) 2024-02-02T13:26:05.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc5fbe00640 1 -- 172.21.15.91:0/3348678416 learned_addr learned my addr 172.21.15.91:0/3348678416 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:05.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714531858 0 0) 0x7fc5ec001110 con 0x7fc5fc151f90 2024-02-02T13:26:05.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc601000640 1 --2- 172.21.15.91:0/3348678416 >> v2:172.21.15.113:3300/0 conn(0x7fc5fc0b0c40 0x7fc5fc0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5dc003520 con 0x7fc5fc151f90 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.219+0000 7fc5fb400640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5fc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224300794 0 0) 0x7fc5dc003520 con 0x7fc5fc151f90 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 msgr2=0x7fc5fc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5fc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 >> v2:172.21.15.113:3300/0 conn(0x7fc5fc0b0c40 msgr2=0x7fc5fc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 --2- 172.21.15.91:0/3348678416 >> v2:172.21.15.113:3300/0 conn(0x7fc5fc0b0c40 0x7fc5fc0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc152370 con 0x7fc5fc151f90 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc5ec001800 con 0x7fc5fc151f90 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 >> v1:172.21.15.91:6789/0 conn(0x7fc5fc151f90 legacy=0x7fc5fc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc003970 0x7fc5dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5dc0047c0 0x7fc5dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0b10d0 con 0x7fc5dc0047c0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc070f80 con 0x7fc5dc003970 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc00b980 con 0x7fc5fc15f5a0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5fb400640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5dc0047c0 0x7fc5dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.220+0000 7fc5fbe00640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc003970 0x7fc5dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc601000640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc5fb400640 1 -- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 msgr2=0x7fc5dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc5fb400640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc5fb400640 1 -- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc003970 msgr2=0x7fc5dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc5fb400640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc003970 0x7fc5dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc5fb400640 1 -- 172.21.15.91:0/3348678416 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc003520 con 0x7fc5dc0047c0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.221+0000 7fc601000640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc15f5a0 0x7fc5dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc5fb400640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5dc0047c0 0x7fc5dc004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009ac0 tx=0x7fc5dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1940fb910d844678 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e402c6b0 con 0x7fc5dc0047c0 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e402c810 con 0x7fc5dc0047c0 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc5faa00640 1 -- 172.21.15.91:0/3348678416 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e402cba0 con 0x7fc5dc0047c0 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc603000640 1 -- 172.21.15.91:0/3348678416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5dc0047c0 msgr2=0x7fc5dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc603000640 1 --2- 172.21.15.91:0/3348678416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5dc0047c0 0x7fc5dc004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009ac0 tx=0x7fc5dc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.222+0000 7fc601000640 1 -- 172.21.15.91:0/3348678416 reap_dead start 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.223+0000 7fc603000640 1 -- 172.21.15.91:0/3348678416 shutdown_connections 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.223+0000 7fc603000640 1 -- 172.21.15.91:0/3348678416 >> 172.21.15.91:0/3348678416 conn(0x7fc5fc0cec80 msgr2=0x7fc5fc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.223+0000 7fc603000640 1 -- 172.21.15.91:0/3348678416 shutdown_connections 2024-02-02T13:26:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.223+0000 7fc603000640 1 -- 172.21.15.91:0/3348678416 wait complete. 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.224+0000 7fc603000640 1 Processor -- start 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 -- start start 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc0047c0 0x7fc5fc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc0b2c10 0x7fc5fc1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0b3710 con 0x7fc5fc0c5370 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0c2ed0 con 0x7fc5dc0047c0 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.225+0000 7fc603000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0b3150 con 0x7fc5fc0b2c10 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38948/0 (socket says 172.21.15.91:38948) 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 -- 172.21.15.91:0/391738697 learned_addr learned my addr 172.21.15.91:0/391738697 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc601000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc0b2c10 0x7fc5fc1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fbe00640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc0047c0 0x7fc5fc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 -- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc0b2c10 msgr2=0x7fc5fc1f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc0b2c10 0x7fc5fc1f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 -- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc0047c0 msgr2=0x7fc5fc0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc0047c0 0x7fc5fc0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 -- 172.21.15.91:0/391738697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0b3430 con 0x7fc5fc0c5370 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.226+0000 7fc5fb400640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4037040 tx=0x7fc5e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.227+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4003820 con 0x7fc5fc0c5370 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.227+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4003980 con 0x7fc5fc0c5370 2024-02-02T13:26:05.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.227+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4003d10 con 0x7fc5fc0c5370 2024-02-02T13:26:05.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.227+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc0c2b60 con 0x7fc5fc0c5370 2024-02-02T13:26:05.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.227+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc1f22e0 con 0x7fc5fc0c5370 2024-02-02T13:26:05.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.228+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc0c29c0 con 0x7fc5fc0c5370 2024-02-02T13:26:05.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.229+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc5e4034440 con 0x7fc5fc0c5370 2024-02-02T13:26:05.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.230+0000 7fc5f3e00640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 0x7fc5d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.230+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5e40aaa10 con 0x7fc5fc0c5370 2024-02-02T13:26:05.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.230+0000 7fc5fbe00640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 0x7fc5d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.231+0000 7fc5fbe00640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 0x7fc5d406c8f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001eb0 tx=0x7fc5ec000ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.234+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc5fc0c29c0 con 0x7fc5fc0c5370 2024-02-02T13:26:05.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.428+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5fc0c29c0 con 0x7fc5fc0c5370 2024-02-02T13:26:05.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.431+0000 7fc5f3e00640 1 -- 172.21.15.91:0/391738697 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fc5fc0c29c0 con 0x7fc5fc0c5370 2024-02-02T13:26:05.433 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:05.433 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 msgr2=0x7fc5d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 0x7fc5d406c8f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001eb0 tx=0x7fc5ec000ed0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 msgr2=0x7fc5fc1f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4037040 tx=0x7fc5e4004730 comp rx=0 tx=0).stop 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 shutdown_connections 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc5d406a430 0x7fc5d406c8f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5fc0b2c10 0x7fc5fc1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5fc0c5370 0x7fc5fc1f5b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.434+0000 7fc603000640 1 --2- 172.21.15.91:0/391738697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5dc0047c0 0x7fc5fc0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.435+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 >> 172.21.15.91:0/391738697 conn(0x7fc5fc0cec80 msgr2=0x7fc5fc15ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.435+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 shutdown_connections 2024-02-02T13:26:05.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.435+0000 7fc603000640 1 -- 172.21.15.91:0/391738697 wait complete. 2024-02-02T13:26:05.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:25:46.131921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:25:47.166639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:26:05.455 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:26:05.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.668+0000 7f965c200640 1 Processor -- start 2024-02-02T13:26:05.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.668+0000 7f965c200640 1 -- start start 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f965c200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f965415d420 0x7f96540d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f965c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f965c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540d19a0 con 0x7f96540d3130 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f965c200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540cd960 con 0x7f965415d420 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f965c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965415d800 con 0x7f96540b8b30 2024-02-02T13:26:05.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f9659800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f965415d420 0x7f96540d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f9659800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f965415d420 0x7f96540d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38954/0 (socket says 172.21.15.91:38954) 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.670+0000 7f9659800640 1 -- 172.21.15.91:0/55885819 learned_addr learned my addr 172.21.15.91:0/55885819 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.671+0000 7f9658e00640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.671+0000 7f9659800640 1 -- 172.21.15.91:0/55885819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 msgr2=0x7f96540d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.671+0000 7f9659800640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.671+0000 7f9659800640 1 -- 172.21.15.91:0/55885819 >> v1:172.21.15.113:6789/0 conn(0x7f96540d3130 legacy=0x7f96540b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:05.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.671+0000 7f9659800640 1 -- 172.21.15.91:0/55885819 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540d1c80 con 0x7f965415d420 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.672+0000 7f9659800640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f965415d420 0x7f96540d1250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f96540cdaa0 tx=0x7f964402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaca56c4e912389 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.672+0000 7f9653e00640 1 -- 172.21.15.91:0/55885819 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644031640 con 0x7f965415d420 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.672+0000 7f9653e00640 1 -- 172.21.15.91:0/55885819 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96440317a0 con 0x7f965415d420 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.672+0000 7f9653e00640 1 -- 172.21.15.91:0/55885819 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644031b30 con 0x7f965415d420 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 -- 172.21.15.91:0/55885819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f965415d420 msgr2=0x7f96540d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f965415d420 0x7f96540d1250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f96540cdaa0 tx=0x7f964402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 -- 172.21.15.91:0/55885819 shutdown_connections 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 --2- 172.21.15.91:0/55885819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f965415d420 0x7f96540d1250 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 -- 172.21.15.91:0/55885819 >> 172.21.15.91:0/55885819 conn(0x7f9654070690 msgr2=0x7f9654070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:05.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 -- 172.21.15.91:0/55885819 shutdown_connections 2024-02-02T13:26:05.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.673+0000 7f965c200640 1 -- 172.21.15.91:0/55885819 wait complete. 2024-02-02T13:26:05.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.675+0000 7f965c200640 1 Processor -- start 2024-02-02T13:26:05.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.675+0000 7f965c200640 1 -- start start 2024-02-02T13:26:05.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f965415d420 0x7f965414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540ca310 con 0x7f96540d3130 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540ca5f0 con 0x7f965415d420 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540ca8d0 con 0x7f96540b8b30 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f9658e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f9658e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38964/0 (socket says 172.21.15.91:38964) 2024-02-02T13:26:05.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f9658e00640 1 -- 172.21.15.91:0/3684169450 learned_addr learned my addr 172.21.15.91:0/3684169450 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:05.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f965a200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f965415d420 0x7f965414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.676+0000 7f9659800640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 -- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 msgr2=0x7f96540cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 -- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f965415d420 msgr2=0x7f965414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f965415d420 0x7f965414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 -- 172.21.15.91:0/3684169450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440094d0 con 0x7f96540d3130 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9658e00640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d800 tx=0x7f963c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c00b6c0 con 0x7f96540d3130 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c00b820 con 0x7f96540d3130 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540cabb0 con 0x7f96540d3130 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c00bbb0 con 0x7f96540d3130 2024-02-02T13:26:05.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.677+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540ccbc0 con 0x7f96540d3130 2024-02-02T13:26:05.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.678+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96540ccd80 con 0x7f96540d3130 2024-02-02T13:26:05.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.679+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f963c00bd10 con 0x7f96540d3130 2024-02-02T13:26:05.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.681+0000 7f9651600640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 0x7f962c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:05.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.681+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f963c088e30 con 0x7f96540d3130 2024-02-02T13:26:05.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.681+0000 7f9659800640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 0x7f962c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:05.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.682+0000 7f9659800640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 0x7f962c06c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9644036000 tx=0x7f964403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:05.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.685+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f963c054410 con 0x7f96540d3130 2024-02-02T13:26:05.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.872+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9654066cf0 con 0x7f96540d3130 2024-02-02T13:26:05.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.872+0000 7f9651600640 1 -- 172.21.15.91:0/3684169450 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f963c057a30 con 0x7f96540d3130 2024-02-02T13:26:05.874 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 msgr2=0x7f962c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 0x7f962c06c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9644036000 tx=0x7f964403c040 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 msgr2=0x7f96541f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d800 tx=0x7f963c00dcc0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 shutdown_connections 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f962c06a430 0x7f962c06c8f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f965415d420 0x7f965414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96540d3130 0x7f96541f6930 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 --2- 172.21.15.91:0/3684169450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96540b8b30 0x7f96540cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 >> 172.21.15.91:0/3684169450 conn(0x7f9654070690 msgr2=0x7f96540cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:05.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 shutdown_connections 2024-02-02T13:26:05.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:05.875+0000 7f965c200640 1 -- 172.21.15.91:0/3684169450 wait complete. 2024-02-02T13:26:05.896 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:26:06.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.092+0000 7fcc46e00640 1 Processor -- start 2024-02-02T13:26:06.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.092+0000 7fcc46e00640 1 -- start start 2024-02-02T13:26:06.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc46e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc4015d490 0x7fcc4006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc46e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcc400b0e10 0x7fcc400acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc46e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4015d870 con 0x7fcc400ac8d0 2024-02-02T13:26:06.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc46e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc400ad2a0 con 0x7fcc400b0e10 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc46e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4006bce0 con 0x7fcc4015d490 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc44e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcc400b0e10 0x7fcc400acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc37e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcc400ac8d0 0x7fcc400b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42540/0 (socket says 172.21.15.91:42540) 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.094+0000 7fcc37e00640 1 -- 172.21.15.91:0/746855653 learned_addr learned my addr 172.21.15.91:0/746855653 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc3fe00640 1 --2- 172.21.15.91:0/746855653 >> v2:172.21.15.113:3300/0 conn(0x7fcc4015d490 0x7fcc4006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc44e00640 1 -- 172.21.15.91:0/746855653 >> v2:172.21.15.113:3300/0 conn(0x7fcc4015d490 msgr2=0x7fcc4006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc44e00640 1 --2- 172.21.15.91:0/746855653 >> v2:172.21.15.113:3300/0 conn(0x7fcc4015d490 0x7fcc4006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc44e00640 1 -- 172.21.15.91:0/746855653 >> v1:172.21.15.91:6789/0 conn(0x7fcc400ac8d0 legacy=0x7fcc400b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc44e00640 1 -- 172.21.15.91:0/746855653 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40161220 con 0x7fcc400b0e10 2024-02-02T13:26:06.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.095+0000 7fcc3f400640 1 -- 172.21.15.91:0/746855653 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255375442 0 0) 0x7fcc4015d870 con 0x7fcc400ac8d0 2024-02-02T13:26:06.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.096+0000 7fcc44e00640 1 --2- 172.21.15.91:0/746855653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc400b0e10 0x7fcc400acd60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcc400ae6b0 tx=0x7fcc3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3afb8cc5c5ed61e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.096+0000 7fcc3f400640 1 -- 172.21.15.91:0/746855653 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3802cc80 con 0x7fcc400b0e10 2024-02-02T13:26:06.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.096+0000 7fcc3f400640 1 -- 172.21.15.91:0/746855653 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3802cde0 con 0x7fcc400b0e10 2024-02-02T13:26:06.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.096+0000 7fcc3f400640 1 -- 172.21.15.91:0/746855653 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38034660 con 0x7fcc400b0e10 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 -- 172.21.15.91:0/746855653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc400b0e10 msgr2=0x7fcc400acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 --2- 172.21.15.91:0/746855653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc400b0e10 0x7fcc400acd60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcc400ae6b0 tx=0x7fcc3802bed0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 -- 172.21.15.91:0/746855653 shutdown_connections 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 --2- 172.21.15.91:0/746855653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc400b0e10 0x7fcc400acd60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 --2- 172.21.15.91:0/746855653 >> v2:172.21.15.113:3300/0 conn(0x7fcc4015d490 0x7fcc4006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 -- 172.21.15.91:0/746855653 >> 172.21.15.91:0/746855653 conn(0x7fcc400cec80 msgr2=0x7fcc400cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 -- 172.21.15.91:0/746855653 shutdown_connections 2024-02-02T13:26:06.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.097+0000 7fcc46e00640 1 -- 172.21.15.91:0/746855653 wait complete. 2024-02-02T13:26:06.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.099+0000 7fcc46e00640 1 Processor -- start 2024-02-02T13:26:06.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.099+0000 7fcc46e00640 1 -- start start 2024-02-02T13:26:06.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc400ac8d0 0x7fcc400c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc400b0e10 0x7fcc401f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400c2c60 con 0x7fcc4015d490 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40159520 con 0x7fcc400b0e10 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc46e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40159800 con 0x7fcc400ac8d0 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.100+0000 7fcc44e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38992/0 (socket says 172.21.15.91:38992) 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 -- 172.21.15.91:0/3355747312 learned_addr learned my addr 172.21.15.91:0/3355747312 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 -- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc400ac8d0 msgr2=0x7fcc400c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc3fe00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc400ac8d0 0x7fcc400c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc400ac8d0 0x7fcc400c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 -- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc400b0e10 msgr2=0x7fcc401f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc37e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc400b0e10 0x7fcc401f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc400b0e10 0x7fcc401f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 -- 172.21.15.91:0/3355747312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009a70 con 0x7fcc4015d490 2024-02-02T13:26:06.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.101+0000 7fcc44e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009e80 tx=0x7fcc38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.102+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38040650 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.102+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc380407b0 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.102+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc40159ae0 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.102+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38040b60 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.102+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4015a0a0 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.103+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400ad770 con 0x7fcc4015d490 2024-02-02T13:26:06.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.104+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcc38040cc0 con 0x7fcc4015d490 2024-02-02T13:26:06.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.104+0000 7fcc3cc00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 0x7fcc1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.105+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc380ab6d0 con 0x7fcc4015d490 2024-02-02T13:26:06.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.105+0000 7fcc3fe00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 0x7fcc1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.105+0000 7fcc3fe00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 0x7fcc1806c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.109+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcc38076c20 con 0x7fcc4015d490 2024-02-02T13:26:06.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.296+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc40066cf0 con 0x7fcc4015d490 2024-02-02T13:26:06.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.297+0000 7fcc3cc00640 1 -- 172.21.15.91:0/3355747312 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc38009450 con 0x7fcc4015d490 2024-02-02T13:26:06.299 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:26:06.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.299+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 msgr2=0x7fcc1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.299+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 0x7fcc1806c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.299+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 msgr2=0x7fcc400b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.299+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009e80 tx=0x7fcc38004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 shutdown_connections 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc1806a430 0x7fcc1806c8f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc4015d490 0x7fcc400b29c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc400b0e10 0x7fcc401f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 --2- 172.21.15.91:0/3355747312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc400ac8d0 0x7fcc400c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 >> 172.21.15.91:0/3355747312 conn(0x7fcc400cec80 msgr2=0x7fcc401582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 shutdown_connections 2024-02-02T13:26:06.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.300+0000 7fcc46e00640 1 -- 172.21.15.91:0/3355747312 wait complete. 2024-02-02T13:26:06.321 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:26:06.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.523+0000 7f806d000640 1 Processor -- start 2024-02-02T13:26:06.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.523+0000 7f806d000640 1 -- start start 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.524+0000 7f806d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 0x7f806815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.524+0000 7f806d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f806806f4f0 0x7f80680b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.524+0000 7f806d000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8068070f80 con 0x7f80680b0c40 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.524+0000 7f806d000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680b10d0 con 0x7f806806f4f0 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.524+0000 7f806d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8068152370 con 0x7f8068151f90 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f806806f4f0 0x7f80680b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f806806f4f0 0x7f80680b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39002/0 (socket says 172.21.15.91:39002) 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 -- 172.21.15.91:0/7254199 learned_addr learned my addr 172.21.15.91:0/7254199 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f8066800640 1 --1- 172.21.15.91:0/7254199 >> v1:172.21.15.91:6789/0 conn(0x7f80680b0c40 0x7f80680b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42546/0 (socket says 172.21.15.91:42546) 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f8065e00640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 0x7f806815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 -- 172.21.15.91:0/7254199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 msgr2=0x7f806815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:06.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 0x7f806815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 -- 172.21.15.91:0/7254199 >> v1:172.21.15.91:6789/0 conn(0x7f80680b0c40 legacy=0x7f80680b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f805d200640 1 -- 172.21.15.91:0/7254199 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680cb010 con 0x7f806806f4f0 2024-02-02T13:26:06.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f8065e00640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 0x7f806815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:06.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.525+0000 7f8065400640 1 -- 172.21.15.91:0/7254199 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295339203 0 0) 0x7f8068070f80 con 0x7f80680b0c40 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.526+0000 7f805d200640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f806806f4f0 0x7f80680b0700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8050009870 tx=0x7f805002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a22e3fec8c37c4d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f8065400640 1 -- 172.21.15.91:0/7254199 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050030840 con 0x7f806806f4f0 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f8065400640 1 -- 172.21.15.91:0/7254199 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500309a0 con 0x7f806806f4f0 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f8065400640 1 -- 172.21.15.91:0/7254199 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050030d30 con 0x7f806806f4f0 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 -- 172.21.15.91:0/7254199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f806806f4f0 msgr2=0x7f80680b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f806806f4f0 0x7f80680b0700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8050009870 tx=0x7f805002b980 comp rx=0 tx=0).stop 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 -- 172.21.15.91:0/7254199 shutdown_connections 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f806806f4f0 0x7f80680b0700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 --2- 172.21.15.91:0/7254199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8068151f90 0x7f806815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 -- 172.21.15.91:0/7254199 >> 172.21.15.91:0/7254199 conn(0x7f80680cec80 msgr2=0x7f80680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:06.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 -- 172.21.15.91:0/7254199 shutdown_connections 2024-02-02T13:26:06.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.527+0000 7f806d000640 1 -- 172.21.15.91:0/7254199 wait complete. 2024-02-02T13:26:06.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.529+0000 7f806d000640 1 Processor -- start 2024-02-02T13:26:06.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.529+0000 7f806d000640 1 -- start start 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f806806f4f0 0x7f80681f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8068151f90 0x7f80680af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80681f6ce0 con 0x7f80680b0c40 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806815cc00 con 0x7f8068151f90 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f806d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80681f6720 con 0x7f806806f4f0 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f805d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f805d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39014/0 (socket says 172.21.15.91:39014) 2024-02-02T13:26:06.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.530+0000 7f805d200640 1 -- 172.21.15.91:0/1705504410 learned_addr learned my addr 172.21.15.91:0/1705504410 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:06.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 -- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f806806f4f0 msgr2=0x7f80681f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f806806f4f0 0x7f80681f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f8066800640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8068151f90 0x7f80680af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 -- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8068151f90 msgr2=0x7f80680af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8068151f90 0x7f80680af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 -- 172.21.15.91:0/1705504410 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80500094d0 con 0x7f80680b0c40 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805d200640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050030810 con 0x7f80680b0c40 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8050039d90 con 0x7f80680b0c40 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80681f6a00 con 0x7f80680b0c40 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.531+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050043670 con 0x7f80680b0c40 2024-02-02T13:26:06.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.532+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80680c8940 con 0x7f80680b0c40 2024-02-02T13:26:06.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.533+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806815c680 con 0x7f80680b0c40 2024-02-02T13:26:06.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.533+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f805002f390 con 0x7f80680b0c40 2024-02-02T13:26:06.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.534+0000 7f805ea00640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 0x7f804006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.534+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80500aac80 con 0x7f80680b0c40 2024-02-02T13:26:06.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.535+0000 7f8065e00640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 0x7f804006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.535+0000 7f8065e00640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 0x7f804006c920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.539+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f806815c680 con 0x7f80680b0c40 2024-02-02T13:26:06.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.726+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f806815cd40 con 0x7f80680b0c40 2024-02-02T13:26:06.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.727+0000 7f805ea00640 1 -- 172.21.15.91:0/1705504410 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f806815cd40 con 0x7f80680b0c40 2024-02-02T13:26:06.728 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:26:06.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 msgr2=0x7f804006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 0x7f804006c920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058008040 comp rx=0 tx=0).stop 2024-02-02T13:26:06.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 msgr2=0x7f80681f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050004730 comp rx=0 tx=0).stop 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 shutdown_connections 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f804006a460 0x7f804006c920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8068151f90 0x7f80680af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80680b0c40 0x7f80681f5cf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 --2- 172.21.15.91:0/1705504410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f806806f4f0 0x7f80681f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.729+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 >> 172.21.15.91:0/1705504410 conn(0x7f80680cec80 msgr2=0x7f80680ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.730+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 shutdown_connections 2024-02-02T13:26:06.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.730+0000 7f806d000640 1 -- 172.21.15.91:0/1705504410 wait complete. 2024-02-02T13:26:06.750 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:26:06.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.948+0000 7fc759200640 1 Processor -- start 2024-02-02T13:26:06.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.948+0000 7fc759200640 1 -- start start 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc75415d400 0x7fc754161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7540cc1c0 0x7fc7540cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7540b0f80 0x7fc7540b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540cd400 con 0x7fc7540cc1c0 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540ccb60 con 0x7fc7540b0f80 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc759200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540cf350 con 0x7fc75415d400 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc74be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7540cc1c0 0x7fc7540cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc74be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7540cc1c0 0x7fc7540cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39016/0 (socket says 172.21.15.91:39016) 2024-02-02T13:26:06.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.950+0000 7fc74be00640 1 -- 172.21.15.91:0/1789189891 learned_addr learned my addr 172.21.15.91:0/1789189891 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:06.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc751e00640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc75415d400 0x7fc754161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc752800640 1 --2- 172.21.15.91:0/1789189891 >> v2:172.21.15.113:3300/0 conn(0x7fc7540b0f80 0x7fc7540b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc74be00640 1 -- 172.21.15.91:0/1789189891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc75415d400 msgr2=0x7fc754161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:06.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc74be00640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc75415d400 0x7fc754161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc74be00640 1 -- 172.21.15.91:0/1789189891 >> v2:172.21.15.113:3300/0 conn(0x7fc7540b0f80 msgr2=0x7fc7540b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:06.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc74be00640 1 --2- 172.21.15.91:0/1789189891 >> v2:172.21.15.113:3300/0 conn(0x7fc7540b0f80 0x7fc7540b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.951+0000 7fc74be00640 1 -- 172.21.15.91:0/1789189891 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75406cdf0 con 0x7fc7540cc1c0 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.952+0000 7fc74be00640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7540cc1c0 0x7fc7540cc620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc754073490 tx=0x7fc73c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14937fa86cf676f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.952+0000 7fc751400640 1 -- 172.21.15.91:0/1789189891 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0317d0 con 0x7fc7540cc1c0 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc751400640 1 -- 172.21.15.91:0/1789189891 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c031930 con 0x7fc7540cc1c0 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc751400640 1 -- 172.21.15.91:0/1789189891 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c031cc0 con 0x7fc7540cc1c0 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 -- 172.21.15.91:0/1789189891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7540cc1c0 msgr2=0x7fc7540cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7540cc1c0 0x7fc7540cc620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc754073490 tx=0x7fc73c02b870 comp rx=0 tx=0).stop 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 -- 172.21.15.91:0/1789189891 shutdown_connections 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 --2- 172.21.15.91:0/1789189891 >> v2:172.21.15.113:3300/0 conn(0x7fc7540b0f80 0x7fc7540b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7540cc1c0 0x7fc7540cc620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 --2- 172.21.15.91:0/1789189891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc75415d400 0x7fc754161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 -- 172.21.15.91:0/1789189891 >> 172.21.15.91:0/1789189891 conn(0x7fc754069e30 msgr2=0x7fc75406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:06.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 -- 172.21.15.91:0/1789189891 shutdown_connections 2024-02-02T13:26:06.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.953+0000 7fc759200640 1 -- 172.21.15.91:0/1789189891 wait complete. 2024-02-02T13:26:06.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.955+0000 7fc759200640 1 Processor -- start 2024-02-02T13:26:06.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.956+0000 7fc759200640 1 -- start start 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7540b0f80 0x7fc7541467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 0x7fc7540b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415d870 con 0x7fc75415d400 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754146d30 con 0x7fc7540cc1c0 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc759200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541f2610 con 0x7fc7540b0f80 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc752800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc752800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39022/0 (socket says 172.21.15.91:39022) 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc752800640 1 -- 172.21.15.91:0/3796566681 learned_addr learned my addr 172.21.15.91:0/3796566681 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc751e00640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7540b0f80 0x7fc7541467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.957+0000 7fc74be00640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 0x7fc7540b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 -- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7540b0f80 msgr2=0x7fc7541467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7540b0f80 0x7fc7541467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 -- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 msgr2=0x7fc7540b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 0x7fc7540b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 -- 172.21.15.91:0/3796566681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0094d0 con 0x7fc75415d400 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc74be00640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 0x7fc7540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc752800640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007e60 tx=0x7fc74c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c014620 con 0x7fc75415d400 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c00ee90 con 0x7fc75415d400 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540c8a40 con 0x7fc75415d400 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0149e0 con 0x7fc75415d400 2024-02-02T13:26:06.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.958+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7540c9000 con 0x7fc75415d400 2024-02-02T13:26:06.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.960+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75415d9e0 con 0x7fc75415d400 2024-02-02T13:26:06.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.960+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc74c014b40 con 0x7fc75415d400 2024-02-02T13:26:06.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.962+0000 7fc74a000640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 0x7fc72c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:06.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.962+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc74c089260 con 0x7fc75415d400 2024-02-02T13:26:06.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.962+0000 7fc751e00640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 0x7fc72c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:06.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.962+0000 7fc751e00640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 0x7fc72c06c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc744006fd0 tx=0x7fc744008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:06.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:06.966+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc75415d9e0 con 0x7fc75415d400 2024-02-02T13:26:07.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.168+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc75415d9b0 con 0x7fc75415d400 2024-02-02T13:26:07.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.169+0000 7fc74a000640 1 -- 172.21.15.91:0/3796566681 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc74c054850 con 0x7fc75415d400 2024-02-02T13:26:07.171 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 msgr2=0x7fc72c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 0x7fc72c06c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc744006fd0 tx=0x7fc744008040 comp rx=0 tx=0).stop 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 msgr2=0x7fc7540b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007e60 tx=0x7fc74c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 shutdown_connections 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc72c06a430 0x7fc72c06c8f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc75415d400 0x7fc7540b1930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7540cc1c0 0x7fc7540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 --2- 172.21.15.91:0/3796566681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7540b0f80 0x7fc7541467f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 >> 172.21.15.91:0/3796566681 conn(0x7fc754069e30 msgr2=0x7fc75406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.172+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 shutdown_connections 2024-02-02T13:26:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.173+0000 7fc759200640 1 -- 172.21.15.91:0/3796566681 wait complete. 2024-02-02T13:26:07.192 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:26:07.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:07.192 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:26:07.334 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:26:07.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:07.334 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:26:07.374 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:07.374 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:07.414 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:07.415 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:07.503 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:07.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.714+0000 7f41d9600640 1 Processor -- start 2024-02-02T13:26:07.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.714+0000 7f41d9600640 1 -- start start 2024-02-02T13:26:07.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.715+0000 7f41d9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.715+0000 7f41d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f41d406f4f0 0x7f41d40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.715+0000 7f41d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d4070f80 con 0x7f41d40b0c40 2024-02-02T13:26:07.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.715+0000 7f41d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d40b10d0 con 0x7f41d406f4f0 2024-02-02T13:26:07.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.715+0000 7f41d9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4152370 con 0x7f41d4151f90 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.716+0000 7f41cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f41d406f4f0 0x7f41d40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.716+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.716+0000 7f41d3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f41d40b0c40 0x7f41d40b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:47370/0 (socket says 172.21.15.91:47370) 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.716+0000 7f41cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f41d406f4f0 0x7f41d40b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:51258/0 (socket says 172.21.15.91:51258) 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.716+0000 7f41d3400640 1 -- 172.21.15.91:0/3192065393 learned_addr learned my addr 172.21.15.91:0/3192065393 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.717+0000 7f41cbe00640 1 -- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 msgr2=0x7f41d415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:07.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.717+0000 7f41cbe00640 1 --2- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.717+0000 7f41cbe00640 1 -- 172.21.15.91:0/3192065393 >> v1:172.21.15.113:6789/0 conn(0x7f41d40b0c40 legacy=0x7f41d40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.717+0000 7f41cbe00640 1 -- 172.21.15.91:0/3192065393 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40cb010 con 0x7f41d406f4f0 2024-02-02T13:26:07.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.718+0000 7f41cbe00640 1 --2- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d406f4f0 0x7f41d40b0700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0097a0 tx=0x7f41bc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b469de0c5348624 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:07.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.719+0000 7f41d2000640 1 -- 172.21.15.91:0/3192065393 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc02d040 con 0x7f41d406f4f0 2024-02-02T13:26:07.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.719+0000 7f41d2000640 1 -- 172.21.15.91:0/3192065393 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc031810 con 0x7f41d406f4f0 2024-02-02T13:26:07.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.719+0000 7f41d2000640 1 -- 172.21.15.91:0/3192065393 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc031ba0 con 0x7f41d406f4f0 2024-02-02T13:26:07.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 -- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d406f4f0 msgr2=0x7f41d40b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 --2- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d406f4f0 0x7f41d40b0700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0097a0 tx=0x7f41bc02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 -- 172.21.15.91:0/3192065393 shutdown_connections 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 --2- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d406f4f0 0x7f41d40b0700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 --2- 172.21.15.91:0/3192065393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 -- 172.21.15.91:0/3192065393 >> 172.21.15.91:0/3192065393 conn(0x7f41d40cec80 msgr2=0x7f41d40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 -- 172.21.15.91:0/3192065393 shutdown_connections 2024-02-02T13:26:07.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.720+0000 7f41d9600640 1 -- 172.21.15.91:0/3192065393 wait complete. 2024-02-02T13:26:07.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.722+0000 7f41d9600640 1 Processor -- start 2024-02-02T13:26:07.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.722+0000 7f41d9600640 1 -- start start 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d40b0c40 0x7f41d4150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4151000 con 0x7f41d406f4f0 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41f5ec0 con 0x7f41d40b0c40 2024-02-02T13:26:07.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.723+0000 7f41d9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41f33e0 con 0x7f41d4151f90 2024-02-02T13:26:07.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:07.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39026/0 (socket says 172.21.15.91:39026) 2024-02-02T13:26:07.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 -- 172.21.15.91:0/726342516 learned_addr learned my addr 172.21.15.91:0/726342516 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 -- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 msgr2=0x7f41d40cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 -- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d40b0c40 msgr2=0x7f41d4150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d40b0c40 0x7f41d4150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.724+0000 7f41d2a00640 1 -- 172.21.15.91:0/726342516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009400 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41d2a00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f41c400e9b0 tx=0x7f41c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4009670 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c400c9a0 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400cd50 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d41f36c0 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.725+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d41f6440 con 0x7f41d406f4f0 2024-02-02T13:26:07.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.726+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d40cd850 con 0x7f41d406f4f0 2024-02-02T13:26:07.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.727+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f41c4013470 con 0x7f41d406f4f0 2024-02-02T13:26:07.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.728+0000 7f41caa00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:07.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.728+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41c4088ca0 con 0x7f41d406f4f0 2024-02-02T13:26:07.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.729+0000 7f41cbe00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:07.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.729+0000 7f41cbe00640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41bc036000 tx=0x7f41bc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:07.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.732+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f41c4054170 con 0x7f41d406f4f0 2024-02-02T13:26:07.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.926+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41d4066cf0 con 0x7f41d406f4f0 2024-02-02T13:26:07.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.929+0000 7f41caa00640 1 -- 172.21.15.91:0/726342516 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f41c4057790 con 0x7f41d406f4f0 2024-02-02T13:26:07.931 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:07.931 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 msgr2=0x7f41ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41bc036000 tx=0x7f41bc030000 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 msgr2=0x7f41d41f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f41c400e9b0 tx=0x7f41c400ee70 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 shutdown_connections 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d4151f90 0x7f41d40cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d40b0c40 0x7f41d4150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 --2- 172.21.15.91:0/726342516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d406f4f0 0x7f41d41f6960 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 >> 172.21.15.91:0/726342516 conn(0x7f41d40cec80 msgr2=0x7f41d40c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 shutdown_connections 2024-02-02T13:26:07.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:07.932+0000 7f41d9600640 1 -- 172.21.15.91:0/726342516 wait complete. 2024-02-02T13:26:07.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:25:46.131921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:25:47.166639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:26:07.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:26:08.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.156+0000 7f8b7bc00640 1 Processor -- start 2024-02-02T13:26:08.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.157+0000 7f8b7bc00640 1 -- start start 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b7bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740d0070 0x7f8b740b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b7bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b740b2260 0x7f8b740d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b7bc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b740cd930 con 0x7f8b740b6260 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b7bc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740b8f90 con 0x7f8b740b2260 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b7bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740d1680 con 0x7f8b740d0070 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b6be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b740b2260 0x7f8b740d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b6be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b740b2260 0x7f8b740d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39038/0 (socket says 172.21.15.91:39038) 2024-02-02T13:26:08.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.158+0000 7f8b6be00640 1 -- 172.21.15.91:0/2559390008 learned_addr learned my addr 172.21.15.91:0/2559390008 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:08.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.159+0000 7f8b79200640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740d0070 0x7f8b740b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.159+0000 7f8b6be00640 1 -- 172.21.15.91:0/2559390008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740d0070 msgr2=0x7f8b740b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:08.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.159+0000 7f8b6be00640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740d0070 0x7f8b740b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.159+0000 7f8b6be00640 1 -- 172.21.15.91:0/2559390008 >> v1:172.21.15.113:6789/0 conn(0x7f8b740b6260 legacy=0x7f8b740d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:08.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.159+0000 7f8b6be00640 1 -- 172.21.15.91:0/2559390008 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740d1960 con 0x7f8b740b2260 2024-02-02T13:26:08.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.160+0000 7f8b6be00640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740b2260 0x7f8b740d0450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009870 tx=0x7f8b5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999da6ca93ee3f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.160+0000 7f8b73e00640 1 -- 172.21.15.91:0/2559390008 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c031640 con 0x7f8b740b2260 2024-02-02T13:26:08.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.160+0000 7f8b73e00640 1 -- 172.21.15.91:0/2559390008 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c0317a0 con 0x7f8b740b2260 2024-02-02T13:26:08.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.160+0000 7f8b73e00640 1 -- 172.21.15.91:0/2559390008 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c031b30 con 0x7f8b740b2260 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2559390008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740b2260 msgr2=0x7f8b740d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740b2260 0x7f8b740d0450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009870 tx=0x7f8b5c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2559390008 shutdown_connections 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740b2260 0x7f8b740d0450 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2559390008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740d0070 0x7f8b740b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2559390008 >> 172.21.15.91:0/2559390008 conn(0x7f8b74070660 msgr2=0x7f8b74070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2559390008 shutdown_connections 2024-02-02T13:26:08.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.161+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2559390008 wait complete. 2024-02-02T13:26:08.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.163+0000 7f8b7bc00640 1 Processor -- start 2024-02-02T13:26:08.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.163+0000 7f8b7bc00640 1 -- start start 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740b2260 0x7f8b740b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b740b6260 0x7f8b740c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c7710 con 0x7f8b740d0070 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7415fb50 con 0x7f8b740b6260 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b7bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c6a90 con 0x7f8b740b2260 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b79c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b79c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39044/0 (socket says 172.21.15.91:39044) 2024-02-02T13:26:08.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.164+0000 7f8b79c00640 1 -- 172.21.15.91:0/2327779370 learned_addr learned my addr 172.21.15.91:0/2327779370 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:08.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79200640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740b2260 0x7f8b740b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 -- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740b2260 msgr2=0x7f8b740b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740b2260 0x7f8b740b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 -- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b740b6260 msgr2=0x7f8b740c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b740b6260 0x7f8b740c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 -- 172.21.15.91:0/2327779370 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0094d0 con 0x7f8b740d0070 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b79c00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c780 tx=0x7f8b6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c004b10 con 0x7f8b740d0070 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.165+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c004c70 con 0x7f8b740d0070 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.166+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00ce30 con 0x7f8b740d0070 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.166+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b740c6d70 con 0x7f8b740d0070 2024-02-02T13:26:08.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.166+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b740c7330 con 0x7f8b740d0070 2024-02-02T13:26:08.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.167+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b740c7850 con 0x7f8b740d0070 2024-02-02T13:26:08.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.168+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b6c00a850 con 0x7f8b740d0070 2024-02-02T13:26:08.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.168+0000 7f8b71600640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.169+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b6c088ae0 con 0x7f8b740d0070 2024-02-02T13:26:08.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.169+0000 7f8b79200640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.169+0000 7f8b79200640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b64006fd0 tx=0x7f8b64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.173+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b740c7850 con 0x7f8b740d0070 2024-02-02T13:26:08.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.364+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b740c7850 con 0x7f8b740d0070 2024-02-02T13:26:08.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.364+0000 7f8b71600640 1 -- 172.21.15.91:0/2327779370 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b740c7850 con 0x7f8b740d0070 2024-02-02T13:26:08.366 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 msgr2=0x7f8b4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b64006fd0 tx=0x7f8b64008040 comp rx=0 tx=0).stop 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 msgr2=0x7f8b741f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c780 tx=0x7f8b6c00cc40 comp rx=0 tx=0).stop 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 shutdown_connections 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b740d0070 0x7f8b741f5ea0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b740b6260 0x7f8b740c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 --2- 172.21.15.91:0/2327779370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b740b2260 0x7f8b740b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 >> 172.21.15.91:0/2327779370 conn(0x7f8b74070660 msgr2=0x7f8b7415c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:08.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 shutdown_connections 2024-02-02T13:26:08.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.367+0000 7f8b7bc00640 1 -- 172.21.15.91:0/2327779370 wait complete. 2024-02-02T13:26:08.388 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:26:08.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.588+0000 7f2339400640 1 Processor -- start 2024-02-02T13:26:08.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.588+0000 7f2339400640 1 -- start start 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2339400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f23340b0e10 0x7f233406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2339400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233415d870 con 0x7f23340acfe0 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2339400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233406bce0 con 0x7f233415d490 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2339400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2334161220 con 0x7f23340b0e10 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2331e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f23340acfe0 0x7f23340b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42556/0 (socket says 172.21.15.91:42556) 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2331e00640 1 -- 172.21.15.91:0/2967930967 learned_addr learned my addr 172.21.15.91:0/2967930967 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.590+0000 7f2333200640 1 --2- 172.21.15.91:0/2967930967 >> v2:172.21.15.91:3300/0 conn(0x7f23340b0e10 0x7f233406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.591+0000 7f2331400640 1 -- 172.21.15.91:0/2967930967 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387093496 0 0) 0x7f233415d870 con 0x7f23340acfe0 2024-02-02T13:26:08.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.591+0000 7f2331400640 1 -- 172.21.15.91:0/2967930967 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2314003520 con 0x7f23340acfe0 2024-02-02T13:26:08.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.591+0000 7f2333200640 1 -- 172.21.15.91:0/2967930967 >> v1:172.21.15.113:6789/0 conn(0x7f233415d490 legacy=0x7f23340ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:08.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.591+0000 7f2333200640 1 -- 172.21.15.91:0/2967930967 >> v1:172.21.15.91:6789/0 conn(0x7f23340acfe0 legacy=0x7f23340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.591+0000 7f2333200640 1 -- 172.21.15.91:0/2967930967 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340ad490 con 0x7f23340b0e10 2024-02-02T13:26:08.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.592+0000 7f2333200640 1 --2- 172.21.15.91:0/2967930967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340b0e10 0x7f233406c170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f232c009d90 tx=0x7f232c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69335cd2b15a4080 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.592+0000 7f2331400640 1 -- 172.21.15.91:0/2967930967 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c02cc60 con 0x7f23340b0e10 2024-02-02T13:26:08.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.592+0000 7f2331400640 1 -- 172.21.15.91:0/2967930967 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c02cdc0 con 0x7f23340b0e10 2024-02-02T13:26:08.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.592+0000 7f2331400640 1 -- 172.21.15.91:0/2967930967 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c034660 con 0x7f23340b0e10 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 -- 172.21.15.91:0/2967930967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340b0e10 msgr2=0x7f233406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 --2- 172.21.15.91:0/2967930967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340b0e10 0x7f233406c170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f232c009d90 tx=0x7f232c02beb0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 -- 172.21.15.91:0/2967930967 shutdown_connections 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 --2- 172.21.15.91:0/2967930967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340b0e10 0x7f233406c170 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 -- 172.21.15.91:0/2967930967 >> 172.21.15.91:0/2967930967 conn(0x7f23340cec80 msgr2=0x7f23340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 -- 172.21.15.91:0/2967930967 shutdown_connections 2024-02-02T13:26:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.593+0000 7f2339400640 1 -- 172.21.15.91:0/2967930967 wait complete. 2024-02-02T13:26:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.595+0000 7f2339400640 1 Processor -- start 2024-02-02T13:26:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.595+0000 7f2339400640 1 -- start start 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23340b0e10 0x7f23340c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 0x7f23340c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b18a0 con 0x7f23340acfe0 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2334156e00 con 0x7f233415d490 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2339400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340c5150 con 0x7f23340b0e10 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2332800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2332800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39062/0 (socket says 172.21.15.91:39062) 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2332800640 1 -- 172.21.15.91:0/2983385896 learned_addr learned my addr 172.21.15.91:0/2983385896 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:08.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2331e00640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23340b0e10 0x7f23340c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.596+0000 7f2333200640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 0x7f23340c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 -- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23340b0e10 msgr2=0x7f23340c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23340b0e10 0x7f23340c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 -- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 msgr2=0x7f23340c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 0x7f23340c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 -- 172.21.15.91:0/2983385896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c009a70 con 0x7f23340acfe0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2333200640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 0x7f23340c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2332800640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f232400c430 tx=0x7f232400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324011790 con 0x7f23340acfe0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23240118f0 con 0x7f23340acfe0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324011ce0 con 0x7f23340acfe0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23340c53e0 con 0x7f23340acfe0 2024-02-02T13:26:08.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.597+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23340b1af0 con 0x7f23340acfe0 2024-02-02T13:26:08.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.599+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233406a010 con 0x7f23340acfe0 2024-02-02T13:26:08.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.599+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f232401c460 con 0x7f23340acfe0 2024-02-02T13:26:08.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.600+0000 7f232aa00640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 0x7f230c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:08.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.600+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2324088800 con 0x7f23340acfe0 2024-02-02T13:26:08.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.601+0000 7f2331e00640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 0x7f230c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:08.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.601+0000 7f2331e00640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 0x7f230c06c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23340cf2c0 tx=0x7f231c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:08.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.605+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2324053e80 con 0x7f23340acfe0 2024-02-02T13:26:08.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.792+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2334066cf0 con 0x7f23340acfe0 2024-02-02T13:26:08.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.792+0000 7f232aa00640 1 -- 172.21.15.91:0/2983385896 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23240574a0 con 0x7f23340acfe0 2024-02-02T13:26:08.794 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:26:08.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 msgr2=0x7f230c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 0x7f230c06c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23340cf2c0 tx=0x7f231c009040 comp rx=0 tx=0).stop 2024-02-02T13:26:08.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 msgr2=0x7f23340aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:08.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f232400c430 tx=0x7f232400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 shutdown_connections 2024-02-02T13:26:08.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230c06a430 0x7f230c06c8f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23340b0e10 0x7f23340c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f233415d490 0x7f23340c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 --2- 172.21.15.91:0/2983385896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23340acfe0 0x7f23340aebf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:08.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 >> 172.21.15.91:0/2983385896 conn(0x7f23340cec80 msgr2=0x7f23340caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:08.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 shutdown_connections 2024-02-02T13:26:08.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:08.795+0000 7f2339400640 1 -- 172.21.15.91:0/2983385896 wait complete. 2024-02-02T13:26:08.815 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:26:09.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.026+0000 7fe8ab200640 1 Processor -- start 2024-02-02T13:26:09.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.026+0000 7fe8ab200640 1 -- start start 2024-02-02T13:26:09.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.027+0000 7fe8ab200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.027+0000 7fe8ab200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8ac0add80 0x7fe8ac0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.027+0000 7fe8ab200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0ce3b0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.027+0000 7fe8ab200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0adbd0 con 0x7fe8ac0add80 2024-02-02T13:26:09.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.027+0000 7fe8ab200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0c59d0 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.028+0000 7fe8a9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.028+0000 7fe8a9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:58762/0 (socket says 172.21.15.91:58762) 2024-02-02T13:26:09.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.028+0000 7fe8a9e00640 1 -- 172.21.15.91:0/3901080565 learned_addr learned my addr 172.21.15.91:0/3901080565 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:09.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.028+0000 7fe8a9400640 1 --2- 172.21.15.91:0/3901080565 >> v2:172.21.15.113:3300/0 conn(0x7fe8ac0add80 0x7fe8ac0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.029+0000 7fe8a9e00640 1 -- 172.21.15.91:0/3901080565 >> v2:172.21.15.113:3300/0 conn(0x7fe8ac0add80 msgr2=0x7fe8ac0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:09.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.029+0000 7fe8a9e00640 1 --2- 172.21.15.91:0/3901080565 >> v2:172.21.15.113:3300/0 conn(0x7fe8ac0add80 0x7fe8ac0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.029+0000 7fe8a9e00640 1 -- 172.21.15.91:0/3901080565 >> v1:172.21.15.113:6789/0 conn(0x7fe8ac0c9450 legacy=0x7fe8ac0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.029+0000 7fe8a9e00640 1 -- 172.21.15.91:0/3901080565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0ce8e0 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.030+0000 7fe8a9e00640 1 --2- 172.21.15.91:0/3901080565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009870 tx=0x7fe89c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491e47f3219376d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.031+0000 7fe8a8a00640 1 -- 172.21.15.91:0/3901080565 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0317c0 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.031+0000 7fe8a8a00640 1 -- 172.21.15.91:0/3901080565 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c031920 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.031+0000 7fe8a8a00640 1 -- 172.21.15.91:0/3901080565 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c031cb0 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.031+0000 7fe8ab200640 1 -- 172.21.15.91:0/3901080565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 msgr2=0x7fe8ac06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.031+0000 7fe8ab200640 1 --2- 172.21.15.91:0/3901080565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009870 tx=0x7fe89c02b860 comp rx=0 tx=0).stop 2024-02-02T13:26:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 -- 172.21.15.91:0/3901080565 shutdown_connections 2024-02-02T13:26:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 --2- 172.21.15.91:0/3901080565 >> v2:172.21.15.113:3300/0 conn(0x7fe8ac0add80 0x7fe8ac0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 --2- 172.21.15.91:0/3901080565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac06b450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 -- 172.21.15.91:0/3901080565 >> 172.21.15.91:0/3901080565 conn(0x7fe8ac0cea90 msgr2=0x7fe8ac0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:09.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 -- 172.21.15.91:0/3901080565 shutdown_connections 2024-02-02T13:26:09.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.032+0000 7fe8ab200640 1 -- 172.21.15.91:0/3901080565 wait complete. 2024-02-02T13:26:09.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.033+0000 7fe8ab200640 1 Processor -- start 2024-02-02T13:26:09.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 -- start start 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8ac0add80 0x7fe8ac1f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac0c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1f3c90 con 0x7fe8ac0c9450 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ae690 con 0x7fe8ac0add80 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.034+0000 7fe8ab200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ae970 con 0x7fe8ac15d3a0 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39074/0 (socket says 172.21.15.91:39074) 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 -- 172.21.15.91:0/2487523515 learned_addr learned my addr 172.21.15.91:0/2487523515 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:09.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9e00640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8ac0add80 0x7fe8ac1f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 -- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 msgr2=0x7fe8ac0c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac0c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 -- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8ac0add80 msgr2=0x7fe8ac1f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8ac0add80 0x7fe8ac1f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.035+0000 7fe8a9400640 1 -- 172.21.15.91:0/2487523515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0094d0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8a9400640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e840 tx=0x7fe89400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940094f0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89400c940 con 0x7fe8ac0c9450 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400ccd0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac0aec50 con 0x7fe8ac0c9450 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.036+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0af210 con 0x7fe8ac0c9450 2024-02-02T13:26:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.037+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac04dff0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.038+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe894013470 con 0x7fe8ac0c9450 2024-02-02T13:26:09.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.039+0000 7fe8a2000640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 0x7fe88406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.039+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe894088900 con 0x7fe8ac0c9450 2024-02-02T13:26:09.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.040+0000 7fe8a9e00640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 0x7fe88406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.041+0000 7fe8a9e00640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 0x7fe88406c920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe89c036000 tx=0x7fe89c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.043+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe894053f80 con 0x7fe8ac0c9450 2024-02-02T13:26:09.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.234+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8ac0b4a10 con 0x7fe8ac0c9450 2024-02-02T13:26:09.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.235+0000 7fe8a2000640 1 -- 172.21.15.91:0/2487523515 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8940170a0 con 0x7fe8ac0c9450 2024-02-02T13:26:09.237 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.237+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 msgr2=0x7fe88406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.237+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 0x7fe88406c920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe89c036000 tx=0x7fe89c03d040 comp rx=0 tx=0).stop 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.237+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 msgr2=0x7fe8ac1f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.237+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e840 tx=0x7fe89400ed00 comp rx=0 tx=0).stop 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 shutdown_connections 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe88406a460 0x7fe88406c920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8ac15d3a0 0x7fe8ac0c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8ac0c9450 0x7fe8ac1f5410 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 --2- 172.21.15.91:0/2487523515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8ac0add80 0x7fe8ac1f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 >> 172.21.15.91:0/2487523515 conn(0x7fe8ac0cea90 msgr2=0x7fe8ac0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 shutdown_connections 2024-02-02T13:26:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.238+0000 7fe8ab200640 1 -- 172.21.15.91:0/2487523515 wait complete. 2024-02-02T13:26:09.257 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:26:09.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.456+0000 7f2fcbc00640 1 Processor -- start 2024-02-02T13:26:09.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.456+0000 7f2fcbc00640 1 -- start start 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.457+0000 7f2fcbc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2fc40cbc90 0x7f2fc415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fcbc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2fc415e170 0x7f2fc406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fcbc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc40ca0c0 con 0x7f2fc4159de0 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fcbc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40ca380 con 0x7f2fc415e170 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fcbc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40ca660 con 0x7f2fc40cbc90 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2fc415e170 0x7f2fc406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2fc415e170 0x7f2fc406fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39090/0 (socket says 172.21.15.91:39090) 2024-02-02T13:26:09.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 -- 172.21.15.91:0/3573587424 learned_addr learned my addr 172.21.15.91:0/3573587424 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:09.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc3e00640 1 --2- 172.21.15.91:0/3573587424 >> v2:172.21.15.113:3300/0 conn(0x7f2fc40cbc90 0x7f2fc415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 -- 172.21.15.91:0/3573587424 >> v2:172.21.15.113:3300/0 conn(0x7f2fc40cbc90 msgr2=0x7f2fc415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:09.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 --2- 172.21.15.91:0/3573587424 >> v2:172.21.15.113:3300/0 conn(0x7f2fc40cbc90 0x7f2fc415dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 -- 172.21.15.91:0/3573587424 >> v1:172.21.15.113:6789/0 conn(0x7f2fc4159de0 legacy=0x7f2fc40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:09.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.458+0000 7f2fc9c00640 1 -- 172.21.15.91:0/3573587424 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40ca940 con 0x7f2fc415e170 2024-02-02T13:26:09.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.459+0000 7f2fc9c00640 1 --2- 172.21.15.91:0/3573587424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc415e170 0x7f2fc406fc90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40b1760 tx=0x7f2fbc02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb3a517396a24fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fc3400640 1 -- 172.21.15.91:0/3573587424 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc02ec90 con 0x7f2fc415e170 2024-02-02T13:26:09.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fc3400640 1 -- 172.21.15.91:0/3573587424 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc02edf0 con 0x7f2fc415e170 2024-02-02T13:26:09.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fc3400640 1 -- 172.21.15.91:0/3573587424 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc036610 con 0x7f2fc415e170 2024-02-02T13:26:09.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3573587424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc415e170 msgr2=0x7f2fc406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3573587424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc415e170 0x7f2fc406fc90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40b1760 tx=0x7f2fbc02e0f0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3573587424 shutdown_connections 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3573587424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc415e170 0x7f2fc406fc90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3573587424 >> v2:172.21.15.113:3300/0 conn(0x7f2fc40cbc90 0x7f2fc415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.460+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3573587424 >> 172.21.15.91:0/3573587424 conn(0x7f2fc40b0700 msgr2=0x7f2fc40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.461+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3573587424 shutdown_connections 2024-02-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.461+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3573587424 wait complete. 2024-02-02T13:26:09.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.462+0000 7f2fcbc00640 1 Processor -- start 2024-02-02T13:26:09.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.462+0000 7f2fcbc00640 1 -- start start 2024-02-02T13:26:09.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc4159de0 0x7f2fc414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc415e170 0x7f2fc40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40add10 con 0x7f2fc40cbc90 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40ace10 con 0x7f2fc415e170 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40ad090 con 0x7f2fc4159de0 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fc9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fc9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39104/0 (socket says 172.21.15.91:39104) 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.463+0000 7f2fc9200640 1 -- 172.21.15.91:0/3466900967 learned_addr learned my addr 172.21.15.91:0/3466900967 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 -- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc4159de0 msgr2=0x7f2fc414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc4159de0 0x7f2fc414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 -- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc415e170 msgr2=0x7f2fc40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc415e170 0x7f2fc40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 -- 172.21.15.91:0/3466900967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc009b30 con 0x7f2fc40cbc90 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc9200640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c410 tx=0x7f2fb400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40035f0 con 0x7f2fc40cbc90 2024-02-02T13:26:09.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4003750 con 0x7f2fc40cbc90 2024-02-02T13:26:09.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4003b00 con 0x7f2fc40cbc90 2024-02-02T13:26:09.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.464+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc40ad370 con 0x7f2fc40cbc90 2024-02-02T13:26:09.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.465+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc40ad930 con 0x7f2fc40cbc90 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.466+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc4066cf0 con 0x7f2fc40cbc90 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.466+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2fb4003c60 con 0x7f2fc40cbc90 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.467+0000 7f2fc0c00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 0x7f2f9c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.467+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fb4087cd0 con 0x7f2fc40cbc90 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.467+0000 7f2fc3e00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 0x7f2f9c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:09.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.468+0000 7f2fc3e00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 0x7f2f9c06c8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fc406d2c0 tx=0x7f2fac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:09.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.472+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2fb4053220 con 0x7f2fc40cbc90 2024-02-02T13:26:09.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.659+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2fc40c9a60 con 0x7f2fc40cbc90 2024-02-02T13:26:09.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.660+0000 7f2fc0c00640 1 -- 172.21.15.91:0/3466900967 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2fb4017080 con 0x7f2fc40cbc90 2024-02-02T13:26:09.662 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:26:09.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 msgr2=0x7f2f9c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 0x7f2f9c06c8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fc406d2c0 tx=0x7f2fac008040 comp rx=0 tx=0).stop 2024-02-02T13:26:09.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 msgr2=0x7f2fc415c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:09.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c410 tx=0x7f2fb400c8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 shutdown_connections 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f9c06a430 0x7f2f9c06c8f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc4159de0 0x7f2fc414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc415e170 0x7f2fc40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.662+0000 7f2fcbc00640 1 --2- 172.21.15.91:0/3466900967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fc40cbc90 0x7f2fc415c4b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.663+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 >> 172.21.15.91:0/3466900967 conn(0x7f2fc40b0700 msgr2=0x7f2fc40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.663+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 shutdown_connections 2024-02-02T13:26:09.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:09.663+0000 7f2fcbc00640 1 -- 172.21.15.91:0/3466900967 wait complete. 2024-02-02T13:26:09.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:26:09.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:09.683 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:26:09.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:26:09.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:09.785 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:26:09.836 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:09.836 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:09.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:26:09.892 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:09.892 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:09.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:26:09.950 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.163+0000 7f610b200640 1 Processor -- start 2024-02-02T13:26:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.164+0000 7f610b200640 1 -- start start 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f610b200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f610415a010 0x7f61040c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f610b200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61040cc7c0 con 0x7f61040ce130 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f610b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61040ce930 con 0x7f61040b0700 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f610b200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61040cd640 con 0x7f610415a010 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f6103e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f610415a010 0x7f61040c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f60fb200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f61040ce130 0x7f61040ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37382/0 (socket says 172.21.15.91:37382) 2024-02-02T13:26:10.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.165+0000 7f60fb200640 1 -- 172.21.15.91:0/4209121805 learned_addr learned my addr 172.21.15.91:0/4209121805 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:10.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.166+0000 7f6103e00640 1 -- 172.21.15.91:0/4209121805 >> v1:172.21.15.113:6789/0 conn(0x7f61040b0700 legacy=0x7f610406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:10.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.166+0000 7f6103e00640 1 -- 172.21.15.91:0/4209121805 >> v1:172.21.15.91:6789/0 conn(0x7f61040ce130 legacy=0x7f61040ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.166+0000 7f6103e00640 1 -- 172.21.15.91:0/4209121805 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610406c890 con 0x7f610415a010 2024-02-02T13:26:10.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.166+0000 7f6103400640 1 -- 172.21.15.91:0/4209121805 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383608461 0 0) 0x7f61040cc7c0 con 0x7f61040ce130 2024-02-02T13:26:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.167+0000 7f6103e00640 1 --2- 172.21.15.91:0/4209121805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f610415a010 0x7f61040c9670 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f60f40097a0 tx=0x7f60f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f70046b48fc0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.167+0000 7f6103400640 1 -- 172.21.15.91:0/4209121805 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4031830 con 0x7f610415a010 2024-02-02T13:26:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.167+0000 7f6103400640 1 -- 172.21.15.91:0/4209121805 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4031990 con 0x7f610415a010 2024-02-02T13:26:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.167+0000 7f6103400640 1 -- 172.21.15.91:0/4209121805 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4031d20 con 0x7f610415a010 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 -- 172.21.15.91:0/4209121805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f610415a010 msgr2=0x7f61040c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 --2- 172.21.15.91:0/4209121805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f610415a010 0x7f61040c9670 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f60f40097a0 tx=0x7f60f402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 -- 172.21.15.91:0/4209121805 shutdown_connections 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 --2- 172.21.15.91:0/4209121805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f610415a010 0x7f61040c9670 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 -- 172.21.15.91:0/4209121805 >> 172.21.15.91:0/4209121805 conn(0x7f61040cec80 msgr2=0x7f61040cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 -- 172.21.15.91:0/4209121805 shutdown_connections 2024-02-02T13:26:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.168+0000 7f610b200640 1 -- 172.21.15.91:0/4209121805 wait complete. 2024-02-02T13:26:10.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.170+0000 7f610b200640 1 Processor -- start 2024-02-02T13:26:10.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.170+0000 7f610b200640 1 -- start start 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61040ce130 0x7f6104152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610415a010 0x7f61040af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61040c63c0 con 0x7f61040b0700 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61040c66a0 con 0x7f610415a010 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f610b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61040c6980 con 0x7f61040ce130 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48972/0 (socket says 172.21.15.91:48972) 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 -- 172.21.15.91:0/2868882600 learned_addr learned my addr 172.21.15.91:0/2868882600 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 -- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61040ce130 msgr2=0x7f6104152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61040ce130 0x7f6104152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 -- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610415a010 msgr2=0x7f61040af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610415a010 0x7f61040af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.171+0000 7f6103e00640 1 -- 172.21.15.91:0/2868882600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f4009400 con 0x7f61040b0700 2024-02-02T13:26:10.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f6103e00640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f60f40097a0 tx=0x7f60f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f403aa60 con 0x7f61040b0700 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f403abc0 con 0x7f61040b0700 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61040c6c60 con 0x7f61040b0700 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f40443f0 con 0x7f61040b0700 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.172+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61040af8e0 con 0x7f61040b0700 2024-02-02T13:26:10.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.173+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61040c7180 con 0x7f61040b0700 2024-02-02T13:26:10.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.174+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f60f404d450 con 0x7f61040b0700 2024-02-02T13:26:10.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.175+0000 7f6100c00640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 0x7f60dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.175+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60f40aab20 con 0x7f61040b0700 2024-02-02T13:26:10.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.175+0000 7f60fb200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 0x7f60dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.176+0000 7f60fb200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 0x7f60dc06c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007fd0 tx=0x7f60ec007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.180+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f60f4076100 con 0x7f61040b0700 2024-02-02T13:26:10.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.396+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6104066cf0 con 0x7f61040b0700 2024-02-02T13:26:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.397+0000 7f6100c00640 1 -- 172.21.15.91:0/2868882600 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f60f4079720 con 0x7f61040b0700 2024-02-02T13:26:10.399 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:10.399 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:26:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:26:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.399+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 msgr2=0x7f60dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.399+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 0x7f60dc06c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007fd0 tx=0x7f60ec007920 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.399+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 msgr2=0x7f61040c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.399+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f60f40097a0 tx=0x7f60f40046c0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.399+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 shutdown_connections 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f60dc06a430 0x7f60dc06c8f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61040ce130 0x7f6104152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610415a010 0x7f61040af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 --2- 172.21.15.91:0/2868882600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61040b0700 0x7f61040c5cc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 >> 172.21.15.91:0/2868882600 conn(0x7f61040cec80 msgr2=0x7f6104073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 shutdown_connections 2024-02-02T13:26:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.400+0000 7f610b200640 1 -- 172.21.15.91:0/2868882600 wait complete. 2024-02-02T13:26:10.418 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:10.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.614+0000 7fe5e9800640 1 Processor -- start 2024-02-02T13:26:10.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.614+0000 7fe5e9800640 1 -- start start 2024-02-02T13:26:10.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 0x7fe5e40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e9800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe5e40b0700 0x7fe5e40b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e9800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40cc7c0 con 0x7fe5e40ce130 2024-02-02T13:26:10.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e9800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40b10b0 con 0x7fe5e40b0700 2024-02-02T13:26:10.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e9800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40ce930 con 0x7fe5e415a010 2024-02-02T13:26:10.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e2000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe5e40ce130 0x7fe5e40ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37396/0 (socket says 172.21.15.91:37396) 2024-02-02T13:26:10.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e2000640 1 -- 172.21.15.91:0/3200285208 learned_addr learned my addr 172.21.15.91:0/3200285208 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e3400640 1 --2- 172.21.15.91:0/3200285208 >> v2:172.21.15.91:3300/0 conn(0x7fe5e40b0700 0x7fe5e40b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e2a00640 1 --2- 172.21.15.91:0/3200285208 >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 0x7fe5e40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.616+0000 7fe5e1600640 1 -- 172.21.15.91:0/3200285208 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388120741 0 0) 0x7fe5e40cc7c0 con 0x7fe5e40ce130 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e1600640 1 -- 172.21.15.91:0/3200285208 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c4005730 con 0x7fe5e40ce130 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e3400640 1 -- 172.21.15.91:0/3200285208 >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 msgr2=0x7fe5e40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e3400640 1 --2- 172.21.15.91:0/3200285208 >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 0x7fe5e40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e3400640 1 -- 172.21.15.91:0/3200285208 >> v1:172.21.15.91:6789/0 conn(0x7fe5e40ce130 legacy=0x7fe5e40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e3400640 1 -- 172.21.15.91:0/3200285208 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40cd640 con 0x7fe5e40b0700 2024-02-02T13:26:10.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.617+0000 7fe5e2a00640 1 --2- 172.21.15.91:0/3200285208 >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 0x7fe5e40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:26:10.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.618+0000 7fe5e3400640 1 --2- 172.21.15.91:0/3200285208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40b0700 0x7fe5e40b0b70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40cfaa0 tx=0x7fe5dc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0d97265a48d145 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.618+0000 7fe5e1600640 1 -- 172.21.15.91:0/3200285208 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc02cc70 con 0x7fe5e40b0700 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.618+0000 7fe5e1600640 1 -- 172.21.15.91:0/3200285208 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc02cdd0 con 0x7fe5e40b0700 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.618+0000 7fe5e1600640 1 -- 172.21.15.91:0/3200285208 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc034660 con 0x7fe5e40b0700 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 -- 172.21.15.91:0/3200285208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40b0700 msgr2=0x7fe5e40b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 --2- 172.21.15.91:0/3200285208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40b0700 0x7fe5e40b0b70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40cfaa0 tx=0x7fe5dc02bec0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 -- 172.21.15.91:0/3200285208 shutdown_connections 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 --2- 172.21.15.91:0/3200285208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40b0700 0x7fe5e40b0b70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 --2- 172.21.15.91:0/3200285208 >> v2:172.21.15.113:3300/0 conn(0x7fe5e415a010 0x7fe5e40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 -- 172.21.15.91:0/3200285208 >> 172.21.15.91:0/3200285208 conn(0x7fe5e40cec80 msgr2=0x7fe5e40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 -- 172.21.15.91:0/3200285208 shutdown_connections 2024-02-02T13:26:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.619+0000 7fe5e9800640 1 -- 172.21.15.91:0/3200285208 wait complete. 2024-02-02T13:26:10.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.621+0000 7fe5e9800640 1 Processor -- start 2024-02-02T13:26:10.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.621+0000 7fe5e9800640 1 -- start start 2024-02-02T13:26:10.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5e40b0700 0x7fe5e4070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e415a010 0x7fe5e415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415d270 con 0x7fe5e40ce130 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415c660 con 0x7fe5e415a010 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e9800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415c940 con 0x7fe5e40b0700 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48986/0 (socket says 172.21.15.91:48986) 2024-02-02T13:26:10.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 -- 172.21.15.91:0/2519442846 learned_addr learned my addr 172.21.15.91:0/2519442846 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 -- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5e40b0700 msgr2=0x7fe5e4070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5e40b0700 0x7fe5e4070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 -- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e415a010 msgr2=0x7fe5e415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e415a010 0x7fe5e415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.622+0000 7fe5e2000640 1 -- 172.21.15.91:0/2519442846 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009a70 con 0x7fe5e40ce130 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5e2000640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c500 tx=0x7fe5cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc005ce0 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc005e40 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc007330 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e415cc20 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.623+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e415c2c0 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.624+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4066cf0 con 0x7fe5e40ce130 2024-02-02T13:26:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.625+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5cc007550 con 0x7fe5e40ce130 2024-02-02T13:26:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.626+0000 7fe5daa00640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.626+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5cc087ff0 con 0x7fe5e40ce130 2024-02-02T13:26:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.626+0000 7fe5e2a00640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:10.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.627+0000 7fe5e2a00640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4006e80 tx=0x7fe5d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:10.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.630+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe5cc0baca0 con 0x7fe5e40ce130 2024-02-02T13:26:10.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.819+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e40b4c00 con 0x7fe5e40ce130 2024-02-02T13:26:10.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.822+0000 7fe5daa00640 1 -- 172.21.15.91:0/2519442846 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fe5cc017090 con 0x7fe5e40ce130 2024-02-02T13:26:10.824 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:10.824 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:10.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 msgr2=0x7fe5bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4006e80 tx=0x7fe5d4008040 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 msgr2=0x7fe5e415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c500 tx=0x7fe5cc00c9c0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 shutdown_connections 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e415a010 0x7fe5e415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5e40ce130 0x7fe5e415a4d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 --2- 172.21.15.91:0/2519442846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5e40b0700 0x7fe5e4070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 >> 172.21.15.91:0/2519442846 conn(0x7fe5e40cec80 msgr2=0x7fe5e4070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 shutdown_connections 2024-02-02T13:26:10.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:10.825+0000 7fe5e9800640 1 -- 172.21.15.91:0/2519442846 wait complete. 2024-02-02T13:26:10.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:25:46.131921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:25:47.166639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:26:10.844 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:26:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.057+0000 7fa88fe00640 1 Processor -- start 2024-02-02T13:26:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.058+0000 7fa88fe00640 1 -- start start 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d2b0 0x7fa8880ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa88815d690 0x7fa8880cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8880ce560 0x7fa8880d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880d1e80 con 0x7fa88815d690 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880d2120 con 0x7fa8880ce560 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880d2400 con 0x7fa88815d2b0 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa88815d690 0x7fa8880cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa88815d690 0x7fa8880cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49002/0 (socket says 172.21.15.91:49002) 2024-02-02T13:26:11.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.059+0000 7fa88ca00640 1 -- 172.21.15.91:0/2575520741 learned_addr learned my addr 172.21.15.91:0/2575520741 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:11.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88d400640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d2b0 0x7fa8880ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88de00640 1 --2- 172.21.15.91:0/2575520741 >> v2:172.21.15.113:3300/0 conn(0x7fa8880ce560 0x7fa8880d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88ca00640 1 -- 172.21.15.91:0/2575520741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d2b0 msgr2=0x7fa8880ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88ca00640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d2b0 0x7fa8880ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88ca00640 1 -- 172.21.15.91:0/2575520741 >> v2:172.21.15.113:3300/0 conn(0x7fa8880ce560 msgr2=0x7fa8880d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88ca00640 1 --2- 172.21.15.91:0/2575520741 >> v2:172.21.15.113:3300/0 conn(0x7fa8880ce560 0x7fa8880d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.060+0000 7fa88ca00640 1 -- 172.21.15.91:0/2575520741 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880d26e0 con 0x7fa88815d690 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.061+0000 7fa88ca00640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa88815d690 0x7fa8880cfb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa8880d0c90 tx=0x7fa87002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a18f863525bcbc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.061+0000 7fa887e00640 1 -- 172.21.15.91:0/2575520741 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870030860 con 0x7fa88815d690 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.061+0000 7fa887e00640 1 -- 172.21.15.91:0/2575520741 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8700309c0 con 0x7fa88815d690 2024-02-02T13:26:11.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.061+0000 7fa887e00640 1 -- 172.21.15.91:0/2575520741 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870030d50 con 0x7fa88815d690 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 -- 172.21.15.91:0/2575520741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa88815d690 msgr2=0x7fa8880cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa88815d690 0x7fa8880cfb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa8880d0c90 tx=0x7fa87002b9a0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 -- 172.21.15.91:0/2575520741 shutdown_connections 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 --2- 172.21.15.91:0/2575520741 >> v2:172.21.15.113:3300/0 conn(0x7fa8880ce560 0x7fa8880d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa88815d690 0x7fa8880cfb40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 --2- 172.21.15.91:0/2575520741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d2b0 0x7fa8880ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 -- 172.21.15.91:0/2575520741 >> 172.21.15.91:0/2575520741 conn(0x7fa888065910 msgr2=0x7fa888065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:11.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 -- 172.21.15.91:0/2575520741 shutdown_connections 2024-02-02T13:26:11.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.062+0000 7fa88fe00640 1 -- 172.21.15.91:0/2575520741 wait complete. 2024-02-02T13:26:11.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.064+0000 7fa88fe00640 1 Processor -- start 2024-02-02T13:26:11.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.064+0000 7fa88fe00640 1 -- start start 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa88815d2b0 0x7fa8880b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d690 0x7fa8880b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888149d90 con 0x7fa8880ce560 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b3280 con 0x7fa88815d2b0 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b5a70 con 0x7fa88815d690 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49018/0 (socket says 172.21.15.91:49018) 2024-02-02T13:26:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.065+0000 7fa88d400640 1 -- 172.21.15.91:0/24990659 learned_addr learned my addr 172.21.15.91:0/24990659 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:11.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 -- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d690 msgr2=0x7fa8880b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d690 0x7fa8880b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 -- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa88815d2b0 msgr2=0x7fa8880b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa88815d2b0 0x7fa8880b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 -- 172.21.15.91:0/24990659 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700094d0 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88d400640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa87800d800 tx=0x7fa87800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800b6c0 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87800b820 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8880b4ef0 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800bb90 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.066+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8880b54b0 con 0x7fa8880ce560 2024-02-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.067+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa888066cf0 con 0x7fa8880ce560 2024-02-02T13:26:11.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.069+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa87800bcf0 con 0x7fa8880ce560 2024-02-02T13:26:11.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.070+0000 7fa885600640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 0x7fa86006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.070+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa878088e80 con 0x7fa8880ce560 2024-02-02T13:26:11.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.070+0000 7fa88ca00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 0x7fa86006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.070+0000 7fa88ca00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 0x7fa86006c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa870036000 tx=0x7fa870030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.074+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa878054460 con 0x7fa8880ce560 2024-02-02T13:26:11.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.266+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8880ce1c0 con 0x7fa8880ce560 2024-02-02T13:26:11.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.267+0000 7fa885600640 1 -- 172.21.15.91:0/24990659 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa878057a80 con 0x7fa8880ce560 2024-02-02T13:26:11.269 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:26:11.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 msgr2=0x7fa86006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 0x7fa86006c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa870036000 tx=0x7fa870030000 comp rx=0 tx=0).stop 2024-02-02T13:26:11.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 msgr2=0x7fa888149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa87800d800 tx=0x7fa87800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 shutdown_connections 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa86006a430 0x7fa86006c8f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa88815d690 0x7fa8880b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa88815d2b0 0x7fa8880b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 --2- 172.21.15.91:0/24990659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa8880ce560 0x7fa888149850 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 >> 172.21.15.91:0/24990659 conn(0x7fa888065910 msgr2=0x7fa88814d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 shutdown_connections 2024-02-02T13:26:11.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.270+0000 7fa88fe00640 1 -- 172.21.15.91:0/24990659 wait complete. 2024-02-02T13:26:11.289 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:26:11.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.492+0000 7effd1200640 1 Processor -- start 2024-02-02T13:26:11.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.492+0000 7effd1200640 1 -- start start 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.493+0000 7effd1200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effcc15aec0 0x7effcc0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.493+0000 7effd1200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7effcc06f3f0 0x7effcc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.493+0000 7effd1200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc06c1c0 con 0x7effcc15b2a0 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.493+0000 7effd1200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc06b5f0 con 0x7effcc06f3f0 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.493+0000 7effd1200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc06cd60 con 0x7effcc15aec0 2024-02-02T13:26:11.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7effcc06f3f0 0x7effcc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effc3e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7effcc15b2a0 0x7effcc06c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37410/0 (socket says 172.21.15.91:37410) 2024-02-02T13:26:11.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effc3e00640 1 -- 172.21.15.91:0/3994714524 learned_addr learned my addr 172.21.15.91:0/3994714524 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:11.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcb400640 1 --2- 172.21.15.91:0/3994714524 >> v2:172.21.15.113:3300/0 conn(0x7effcc15aec0 0x7effcc0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcbe00640 1 -- 172.21.15.91:0/3994714524 >> v2:172.21.15.113:3300/0 conn(0x7effcc15aec0 msgr2=0x7effcc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:11.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcbe00640 1 --2- 172.21.15.91:0/3994714524 >> v2:172.21.15.113:3300/0 conn(0x7effcc15aec0 0x7effcc0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcbe00640 1 -- 172.21.15.91:0/3994714524 >> v1:172.21.15.91:6789/0 conn(0x7effcc15b2a0 legacy=0x7effcc06c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcaa00640 1 -- 172.21.15.91:0/3994714524 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582196113 0 0) 0x7effcc06c1c0 con 0x7effcc15b2a0 2024-02-02T13:26:11.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.494+0000 7effcbe00640 1 -- 172.21.15.91:0/3994714524 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc06b890 con 0x7effcc06f3f0 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.495+0000 7effcbe00640 1 --2- 172.21.15.91:0/3994714524 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc06f3f0 0x7effcc0ca620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7effcc067800 tx=0x7effc402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a8d448dd9aa73 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effcaa00640 1 -- 172.21.15.91:0/3994714524 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc402cd00 con 0x7effcc06f3f0 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effcaa00640 1 -- 172.21.15.91:0/3994714524 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc402ce60 con 0x7effcc06f3f0 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effcaa00640 1 -- 172.21.15.91:0/3994714524 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4034660 con 0x7effcc06f3f0 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 -- 172.21.15.91:0/3994714524 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc06f3f0 msgr2=0x7effcc0ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 --2- 172.21.15.91:0/3994714524 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc06f3f0 0x7effcc0ca620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7effcc067800 tx=0x7effc402bf50 comp rx=0 tx=0).stop 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 -- 172.21.15.91:0/3994714524 shutdown_connections 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 --2- 172.21.15.91:0/3994714524 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc06f3f0 0x7effcc0ca620 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 --2- 172.21.15.91:0/3994714524 >> v2:172.21.15.113:3300/0 conn(0x7effcc15aec0 0x7effcc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 -- 172.21.15.91:0/3994714524 >> 172.21.15.91:0/3994714524 conn(0x7effcc0cea90 msgr2=0x7effcc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:11.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.496+0000 7effd1200640 1 -- 172.21.15.91:0/3994714524 shutdown_connections 2024-02-02T13:26:11.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.497+0000 7effd1200640 1 -- 172.21.15.91:0/3994714524 wait complete. 2024-02-02T13:26:11.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.498+0000 7effd1200640 1 Processor -- start 2024-02-02T13:26:11.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.498+0000 7effd1200640 1 -- start start 2024-02-02T13:26:11.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effcc06f3f0 0x7effcc1e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effcc15aec0 0x7effcc159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc159680 con 0x7effcc15b2a0 2024-02-02T13:26:11.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15b7d0 con 0x7effcc06f3f0 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effd1200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15bab0 con 0x7effcc15aec0 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.499+0000 7effcbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49036/0 (socket says 172.21.15.91:49036) 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 -- 172.21.15.91:0/3883813764 learned_addr learned my addr 172.21.15.91:0/3883813764 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effc3e00640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effcc15aec0 0x7effcc159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcb400640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effcc06f3f0 0x7effcc1e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 -- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effcc15aec0 msgr2=0x7effcc159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effcc15aec0 0x7effcc159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 -- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effcc06f3f0 msgr2=0x7effcc1e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effcc06f3f0 0x7effcc1e4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 -- 172.21.15.91:0/3883813764 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4009a70 con 0x7effcc15b2a0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.500+0000 7effcbe00640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7effc4009e80 tx=0x7effc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.501+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc403d600 con 0x7effcc15b2a0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.501+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc403d780 con 0x7effcc15b2a0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.501+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc15bd90 con 0x7effcc15b2a0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.501+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc403db30 con 0x7effcc15b2a0 2024-02-02T13:26:11.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.501+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc15c350 con 0x7effcc15b2a0 2024-02-02T13:26:11.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.502+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc15c540 con 0x7effcc15b2a0 2024-02-02T13:26:11.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.503+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7effc403b040 con 0x7effcc15b2a0 2024-02-02T13:26:11.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.504+0000 7effc3400640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 0x7effa406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.504+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effc40aae90 con 0x7effcc15b2a0 2024-02-02T13:26:11.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.504+0000 7effcb400640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 0x7effa406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.505+0000 7effcb400640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 0x7effa406c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.508+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7effc4076510 con 0x7effcc15b2a0 2024-02-02T13:26:11.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.699+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effcc04dff0 con 0x7effcc15b2a0 2024-02-02T13:26:11.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.700+0000 7effc3400640 1 -- 172.21.15.91:0/3883813764 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7effc4039020 con 0x7effcc15b2a0 2024-02-02T13:26:11.701 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.702+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 msgr2=0x7effa406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.702+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 0x7effa406c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.702+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 msgr2=0x7effcc0c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.702+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7effc4009e80 tx=0x7effc4004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 shutdown_connections 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effa406a430 0x7effa406c8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effcc15aec0 0x7effcc159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effcc15b2a0 0x7effcc0c9f70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 --2- 172.21.15.91:0/3883813764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effcc06f3f0 0x7effcc1e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 >> 172.21.15.91:0/3883813764 conn(0x7effcc0cea90 msgr2=0x7effcc0cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:11.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 shutdown_connections 2024-02-02T13:26:11.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.703+0000 7effd1200640 1 -- 172.21.15.91:0/3883813764 wait complete. 2024-02-02T13:26:11.722 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:26:11.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.917+0000 7f34f5200640 1 Processor -- start 2024-02-02T13:26:11.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.917+0000 7f34f5200640 1 -- start start 2024-02-02T13:26:11.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34f5200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f34f00ce1d0 0x7f34f00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34f5200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00b7d40 con 0x7f34f015cee0 2024-02-02T13:26:11.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34f5200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00b81d0 con 0x7f34f00b6990 2024-02-02T13:26:11.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34f5200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b7130 con 0x7f34f00ce1d0 2024-02-02T13:26:11.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34eea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f34f00ce1d0 0x7f34f00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34ef400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f34f015cee0 0x7f34f00cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37426/0 (socket says 172.21.15.91:37426) 2024-02-02T13:26:11.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.919+0000 7f34ef400640 1 -- 172.21.15.91:0/2180921378 learned_addr learned my addr 172.21.15.91:0/2180921378 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:11.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.920+0000 7f34eea00640 1 -- 172.21.15.91:0/2180921378 >> v1:172.21.15.113:6789/0 conn(0x7f34f00b6990 legacy=0x7f34f00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.920+0000 7f34eea00640 1 -- 172.21.15.91:0/2180921378 >> v1:172.21.15.91:6789/0 conn(0x7f34f015cee0 legacy=0x7f34f00cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.920+0000 7f34eea00640 1 -- 172.21.15.91:0/2180921378 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00b7410 con 0x7f34f00ce1d0 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.920+0000 7f34ee000640 1 -- 172.21.15.91:0/2180921378 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061592382 0 0) 0x7f34f00b7d40 con 0x7f34f015cee0 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.921+0000 7f34eea00640 1 --2- 172.21.15.91:0/2180921378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00ce1d0 0x7f34f00b8990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34d80097a0 tx=0x7f34d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b42807988f7ff4b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.921+0000 7f34ee000640 1 -- 172.21.15.91:0/2180921378 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8031830 con 0x7f34f00ce1d0 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.921+0000 7f34ee000640 1 -- 172.21.15.91:0/2180921378 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8031990 con 0x7f34f00ce1d0 2024-02-02T13:26:11.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.921+0000 7f34ee000640 1 -- 172.21.15.91:0/2180921378 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8031d20 con 0x7f34f00ce1d0 2024-02-02T13:26:11.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 -- 172.21.15.91:0/2180921378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00ce1d0 msgr2=0x7f34f00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 --2- 172.21.15.91:0/2180921378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00ce1d0 0x7f34f00b8990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34d80097a0 tx=0x7f34d802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 -- 172.21.15.91:0/2180921378 shutdown_connections 2024-02-02T13:26:11.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 --2- 172.21.15.91:0/2180921378 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00ce1d0 0x7f34f00b8990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 -- 172.21.15.91:0/2180921378 >> 172.21.15.91:0/2180921378 conn(0x7f34f0070690 msgr2=0x7f34f0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:11.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 -- 172.21.15.91:0/2180921378 shutdown_connections 2024-02-02T13:26:11.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.922+0000 7f34f5200640 1 -- 172.21.15.91:0/2180921378 wait complete. 2024-02-02T13:26:11.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.923+0000 7f34f5200640 1 Processor -- start 2024-02-02T13:26:11.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.924+0000 7f34f5200640 1 -- start start 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.924+0000 7f34f5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00b6990 0x7f34f0153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.924+0000 7f34f5200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34f00ce1d0 0x7f34f015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.924+0000 7f34f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34f5200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f015e7b0 con 0x7f34f015cee0 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34f5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f015ec40 con 0x7f34f00b6990 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34f5200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0154390 con 0x7f34f00ce1d0 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49060/0 (socket says 172.21.15.91:49060) 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 -- 172.21.15.91:0/513055070 learned_addr learned my addr 172.21.15.91:0/513055070 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34ef400640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00b6990 0x7f34f0153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 -- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34f00ce1d0 msgr2=0x7f34f015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34f00ce1d0 0x7f34f015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 -- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00b6990 msgr2=0x7f34f0153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:11.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00b6990 0x7f34f0153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:11.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.925+0000 7f34efe00640 1 -- 172.21.15.91:0/513055070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8009400 con 0x7f34f015cee0 2024-02-02T13:26:11.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34efe00640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34e800b970 tx=0x7f34e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800cb40 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800cca0 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8014680 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f015dba0 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.926+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f015e160 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.927+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f015e960 con 0x7f34f015cee0 2024-02-02T13:26:11.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.928+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f34e801d3b0 con 0x7f34f015cee0 2024-02-02T13:26:11.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.929+0000 7f34e7400640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 0x7f34c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:11.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.929+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34e8088860 con 0x7f34f015cee0 2024-02-02T13:26:11.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.929+0000 7f34ef400640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 0x7f34c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:11.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.930+0000 7f34ef400640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 0x7f34c806c8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34e0007fd0 tx=0x7f34e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:11.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:11.933+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f34f015e960 con 0x7f34f015cee0 2024-02-02T13:26:12.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.122+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34f015e960 con 0x7f34f015cee0 2024-02-02T13:26:12.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.123+0000 7f34e7400640 1 -- 172.21.15.91:0/513055070 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34f015e960 con 0x7f34f015cee0 2024-02-02T13:26:12.124 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 msgr2=0x7f34c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 0x7f34c806c8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34e0007fd0 tx=0x7f34e0009040 comp rx=0 tx=0).stop 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 msgr2=0x7f34f015f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34e800b970 tx=0x7f34e800be30 comp rx=0 tx=0).stop 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 shutdown_connections 2024-02-02T13:26:12.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.125+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34c806a430 0x7f34c806c8f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34f00ce1d0 0x7f34f015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f015cee0 0x7f34f015f760 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 --2- 172.21.15.91:0/513055070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00b6990 0x7f34f0153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 >> 172.21.15.91:0/513055070 conn(0x7f34f0070690 msgr2=0x7f34f00c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 shutdown_connections 2024-02-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.126+0000 7f34f5200640 1 -- 172.21.15.91:0/513055070 wait complete. 2024-02-02T13:26:12.146 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:12.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.342+0000 7fc787a00640 1 Processor -- start 2024-02-02T13:26:12.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.342+0000 7fc787a00640 1 -- start start 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc7800c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7800ce130 0x7fc7800ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7800cd470 0x7fc7800b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800cc7c0 con 0x7fc7800ce130 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800ce930 con 0x7fc7800cd470 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc787a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78006c890 con 0x7fc78015a010 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.343+0000 7fc777c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7800ce130 0x7fc7800ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc7800ce130 0x7fc7800ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49062/0 (socket says 172.21.15.91:49062) 2024-02-02T13:26:12.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 -- 172.21.15.91:0/2902853247 learned_addr learned my addr 172.21.15.91:0/2902853247 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:12.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 -- 172.21.15.91:0/2902853247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 msgr2=0x7fc7800c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:12.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 --2- 172.21.15.91:0/2902853247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc7800c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 -- 172.21.15.91:0/2902853247 >> v2:172.21.15.113:3300/0 conn(0x7fc7800cd470 msgr2=0x7fc7800b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:12.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 --2- 172.21.15.91:0/2902853247 >> v2:172.21.15.113:3300/0 conn(0x7fc7800cd470 0x7fc7800b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.344+0000 7fc777c00640 1 -- 172.21.15.91:0/2902853247 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78006cb70 con 0x7fc7800ce130 2024-02-02T13:26:12.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.345+0000 7fc777c00640 1 --2- 172.21.15.91:0/2902853247 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc7800ccf30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7800ce5c0 tx=0x7fc76802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f3d09eaa3ed023 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.345+0000 7fc77fe00640 1 -- 172.21.15.91:0/2902853247 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768031890 con 0x7fc7800ce130 2024-02-02T13:26:12.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.345+0000 7fc77fe00640 1 -- 172.21.15.91:0/2902853247 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7680319f0 con 0x7fc7800ce130 2024-02-02T13:26:12.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.345+0000 7fc77fe00640 1 -- 172.21.15.91:0/2902853247 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768031d80 con 0x7fc7800ce130 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 -- 172.21.15.91:0/2902853247 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 msgr2=0x7fc7800ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 --2- 172.21.15.91:0/2902853247 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc7800ccf30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7800ce5c0 tx=0x7fc76802b930 comp rx=0 tx=0).stop 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 -- 172.21.15.91:0/2902853247 shutdown_connections 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 --2- 172.21.15.91:0/2902853247 >> v2:172.21.15.113:3300/0 conn(0x7fc7800cd470 0x7fc7800b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 --2- 172.21.15.91:0/2902853247 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc7800ccf30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 --2- 172.21.15.91:0/2902853247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc7800c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 -- 172.21.15.91:0/2902853247 >> 172.21.15.91:0/2902853247 conn(0x7fc7800cec80 msgr2=0x7fc7800cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 -- 172.21.15.91:0/2902853247 shutdown_connections 2024-02-02T13:26:12.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.346+0000 7fc787a00640 1 -- 172.21.15.91:0/2902853247 wait complete. 2024-02-02T13:26:12.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.348+0000 7fc787a00640 1 Processor -- start 2024-02-02T13:26:12.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.348+0000 7fc787a00640 1 -- start start 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc78015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780155330 0x7fc78015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801565d0 con 0x7fc7800ce130 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800ae3f0 con 0x7fc780155330 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc787a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc780157c90 con 0x7fc78015a010 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc785000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc785000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49076/0 (socket says 172.21.15.91:49076) 2024-02-02T13:26:12.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.349+0000 7fc785000640 1 -- 172.21.15.91:0/3640191195 learned_addr learned my addr 172.21.15.91:0/3640191195 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:12.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780155330 0x7fc78015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 -- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 msgr2=0x7fc78015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc78015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 -- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780155330 msgr2=0x7fc78015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780155330 0x7fc78015c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 -- 172.21.15.91:0/3640191195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680094d0 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc785000640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc77000ba50 tx=0x7fc77000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000cbe0 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77000cd60 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7801f3840 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.350+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770014680 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.351+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780156c70 con 0x7fc7800ce130 2024-02-02T13:26:12.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.352+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780156ad0 con 0x7fc7800ce130 2024-02-02T13:26:12.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.353+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc77001d3b0 con 0x7fc7800ce130 2024-02-02T13:26:12.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.354+0000 7fc77d600640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 0x7fc75806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.354+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc770088880 con 0x7fc7800ce130 2024-02-02T13:26:12.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.354+0000 7fc777c00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 0x7fc75806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.355+0000 7fc777c00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 0x7fc75806c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc768036000 tx=0x7fc768030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.358+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc780156ad0 con 0x7fc7800ce130 2024-02-02T13:26:12.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.587+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc780156ad0 con 0x7fc7800ce130 2024-02-02T13:26:12.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.588+0000 7fc77d600640 1 -- 172.21.15.91:0/3640191195 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fc780156ad0 con 0x7fc7800ce130 2024-02-02T13:26:12.590 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:12.590 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:26:12.590 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 msgr2=0x7fc75806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 0x7fc75806c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc768036000 tx=0x7fc768030000 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 msgr2=0x7fc780157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc77000ba50 tx=0x7fc77000bf10 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 shutdown_connections 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc75806a430 0x7fc75806c8f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc78015a010 0x7fc78015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780155330 0x7fc78015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 --2- 172.21.15.91:0/3640191195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7800ce130 0x7fc780157750 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 >> 172.21.15.91:0/3640191195 conn(0x7fc7800cec80 msgr2=0x7fc78006be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:12.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 shutdown_connections 2024-02-02T13:26:12.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.591+0000 7fc787a00640 1 -- 172.21.15.91:0/3640191195 wait complete. 2024-02-02T13:26:12.610 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:12.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.816+0000 7f194fe00640 1 Processor -- start 2024-02-02T13:26:12.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.817+0000 7f194fe00640 1 -- start start 2024-02-02T13:26:12.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 0x7f19500b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 0x7f19500d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19500b7d40 con 0x7f195015cee0 2024-02-02T13:26:12.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19500b81d0 con 0x7f19500b6990 2024-02-02T13:26:12.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500b7130 con 0x7f19500ce1d0 2024-02-02T13:26:12.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 0x7f19500d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f195015cee0 0x7f19500cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:52878/0 (socket says 172.21.15.91:52878) 2024-02-02T13:26:12.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 0x7f19500d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:39798/0 (socket says 172.21.15.91:39798) 2024-02-02T13:26:12.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f194ea00640 1 -- 172.21.15.91:0/1025236345 learned_addr learned my addr 172.21.15.91:0/1025236345 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:12.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.818+0000 7f1947e00640 1 --2- 172.21.15.91:0/1025236345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 0x7f19500b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f194e000640 1 -- 172.21.15.91:0/1025236345 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042059602 0 0) 0x7f19500b7d40 con 0x7f195015cee0 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f194e000640 1 -- 172.21.15.91:0/1025236345 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1930003520 con 0x7f195015cee0 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f1947e00640 1 -- 172.21.15.91:0/1025236345 >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 msgr2=0x7f19500d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f1947e00640 1 --2- 172.21.15.91:0/1025236345 >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 0x7f19500d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f1947e00640 1 -- 172.21.15.91:0/1025236345 >> v1:172.21.15.113:6789/0 conn(0x7f195015cee0 legacy=0x7f19500cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.819+0000 7f1947e00640 1 -- 172.21.15.91:0/1025236345 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500b7410 con 0x7f19500ce1d0 2024-02-02T13:26:12.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.821+0000 7f1947e00640 1 --2- 172.21.15.91:0/1025236345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 0x7f19500b8990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19380097a0 tx=0x7f193802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc8472a0c506893 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.821+0000 7f194e000640 1 -- 172.21.15.91:0/1025236345 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938031830 con 0x7f19500ce1d0 2024-02-02T13:26:12.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.821+0000 7f194e000640 1 -- 172.21.15.91:0/1025236345 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938031990 con 0x7f19500ce1d0 2024-02-02T13:26:12.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.821+0000 7f194e000640 1 -- 172.21.15.91:0/1025236345 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938031d00 con 0x7f19500ce1d0 2024-02-02T13:26:12.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 -- 172.21.15.91:0/1025236345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 msgr2=0x7f19500b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 --2- 172.21.15.91:0/1025236345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 0x7f19500b8990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19380097a0 tx=0x7f193802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 -- 172.21.15.91:0/1025236345 shutdown_connections 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 --2- 172.21.15.91:0/1025236345 >> v2:172.21.15.113:3300/0 conn(0x7f19500b6990 0x7f19500d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 --2- 172.21.15.91:0/1025236345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19500ce1d0 0x7f19500b8990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 -- 172.21.15.91:0/1025236345 >> 172.21.15.91:0/1025236345 conn(0x7f1950070690 msgr2=0x7f1950070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 -- 172.21.15.91:0/1025236345 shutdown_connections 2024-02-02T13:26:12.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.822+0000 7f194fe00640 1 -- 172.21.15.91:0/1025236345 wait complete. 2024-02-02T13:26:12.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.824+0000 7f194fe00640 1 Processor -- start 2024-02-02T13:26:12.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.824+0000 7f194fe00640 1 -- start start 2024-02-02T13:26:12.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19500b6990 0x7f19500cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195015cee0 0x7f19500b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19501f2dd0 con 0x7f19500ce1d0 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195015e3f0 con 0x7f19500b6990 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500cd190 con 0x7f195015cee0 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49084/0 (socket says 172.21.15.91:49084) 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 -- 172.21.15.91:0/3623733480 learned_addr learned my addr 172.21.15.91:0/3623733480 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:12.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f194ea00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19500b6990 0x7f19500cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 -- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195015cee0 msgr2=0x7f19500b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195015cee0 0x7f19500b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 -- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19500b6990 msgr2=0x7f19500cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19500b6990 0x7f19500cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.825+0000 7f1947e00640 1 -- 172.21.15.91:0/3623733480 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938009400 con 0x7f19500ce1d0 2024-02-02T13:26:12.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f1947e00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f193802b910 tx=0x7f19380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938004250 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19380043b0 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938004520 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19501f2430 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.826+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19501f29f0 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.827+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195004dff0 con 0x7f19500ce1d0 2024-02-02T13:26:12.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.828+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1938003810 con 0x7f19500ce1d0 2024-02-02T13:26:12.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.829+0000 7f1947400640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 0x7f192806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:12.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.829+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19380a9ff0 con 0x7f19500ce1d0 2024-02-02T13:26:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.829+0000 7f194ea00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 0x7f192806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.830+0000 7f194ea00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 0x7f192806c920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19400011e0 tx=0x7f1940009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:12.833+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f19380dcca0 con 0x7f19500ce1d0 2024-02-02T13:26:13.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.026+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19500c6080 con 0x7f19500ce1d0 2024-02-02T13:26:13.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.029+0000 7f1947400640 1 -- 172.21.15.91:0/3623733480 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f1938035090 con 0x7f19500ce1d0 2024-02-02T13:26:13.030 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:13.031 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 msgr2=0x7f192806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 0x7f192806c920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19400011e0 tx=0x7f1940009040 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 msgr2=0x7f19500ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f193802b910 tx=0x7f19380046c0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 shutdown_connections 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.031+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f192806a460 0x7f192806c920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f195015cee0 0x7f19500b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19500ce1d0 0x7f19500ccba0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 --2- 172.21.15.91:0/3623733480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19500b6990 0x7f19500cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 >> 172.21.15.91:0/3623733480 conn(0x7f1950070690 msgr2=0x7f19500cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:13.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 shutdown_connections 2024-02-02T13:26:13.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.032+0000 7f194fe00640 1 -- 172.21.15.91:0/3623733480 wait complete. 2024-02-02T13:26:13.051 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:26:13.051 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:13.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.255+0000 7faa95600640 1 Processor -- start 2024-02-02T13:26:13.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.256+0000 7faa95600640 1 -- start start 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa95600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa9015cee0 0x7faa900b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa95600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa900b7d40 con 0x7faa900ce1d0 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa95600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa900b81d0 con 0x7faa900b6990 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa95600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa900b7130 con 0x7faa9015cee0 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa94c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faa900ce1d0 0x7faa900d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37436/0 (socket says 172.21.15.91:37436) 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa8fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa9015cee0 0x7faa900b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.257+0000 7faa94c00640 1 -- 172.21.15.91:0/3900869536 learned_addr learned my addr 172.21.15.91:0/3900869536 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.258+0000 7faa8ea00640 1 -- 172.21.15.91:0/3900869536 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308818063 0 0) 0x7faa900b7d40 con 0x7faa900ce1d0 2024-02-02T13:26:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.258+0000 7faa8ea00640 1 -- 172.21.15.91:0/3900869536 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa70003520 con 0x7faa900ce1d0 2024-02-02T13:26:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.258+0000 7faa8fe00640 1 -- 172.21.15.91:0/3900869536 >> v1:172.21.15.113:6789/0 conn(0x7faa900b6990 legacy=0x7faa900cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.258+0000 7faa8fe00640 1 -- 172.21.15.91:0/3900869536 >> v1:172.21.15.91:6789/0 conn(0x7faa900ce1d0 legacy=0x7faa900d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.258+0000 7faa8fe00640 1 -- 172.21.15.91:0/3900869536 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900b7410 con 0x7faa9015cee0 2024-02-02T13:26:13.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.259+0000 7faa8fe00640 1 --2- 172.21.15.91:0/3900869536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa9015cee0 0x7faa900b8990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa800097a0 tx=0x7faa8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf92cca6bf668af server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.259+0000 7faa8ea00640 1 -- 172.21.15.91:0/3900869536 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80031830 con 0x7faa9015cee0 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.259+0000 7faa8ea00640 1 -- 172.21.15.91:0/3900869536 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80031990 con 0x7faa9015cee0 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.259+0000 7faa8ea00640 1 -- 172.21.15.91:0/3900869536 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80031d20 con 0x7faa9015cee0 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 -- 172.21.15.91:0/3900869536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa9015cee0 msgr2=0x7faa900b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 --2- 172.21.15.91:0/3900869536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa9015cee0 0x7faa900b8990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa800097a0 tx=0x7faa8002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 -- 172.21.15.91:0/3900869536 shutdown_connections 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 --2- 172.21.15.91:0/3900869536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa9015cee0 0x7faa900b8990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 -- 172.21.15.91:0/3900869536 >> 172.21.15.91:0/3900869536 conn(0x7faa90070690 msgr2=0x7faa90070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:13.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 -- 172.21.15.91:0/3900869536 shutdown_connections 2024-02-02T13:26:13.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.260+0000 7faa95600640 1 -- 172.21.15.91:0/3900869536 wait complete. 2024-02-02T13:26:13.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.262+0000 7faa95600640 1 Processor -- start 2024-02-02T13:26:13.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.262+0000 7faa95600640 1 -- start start 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa900ce1d0 0x7faa900c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 0x7faa900ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900b4be0 con 0x7faa900b6990 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900b4ec0 con 0x7faa9015cee0 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa95600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900b51a0 con 0x7faa900ce1d0 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa8fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa8fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49098/0 (socket says 172.21.15.91:49098) 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa8fe00640 1 -- 172.21.15.91:0/3222051424 learned_addr learned my addr 172.21.15.91:0/3222051424 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:13.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.263+0000 7faa8f400640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa900ce1d0 0x7faa900c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa94c00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 0x7faa900ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 -- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa900ce1d0 msgr2=0x7faa900c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa900ce1d0 0x7faa900c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 -- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 msgr2=0x7faa900ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 0x7faa900ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 -- 172.21.15.91:0/3222051424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80009400 con 0x7faa900b6990 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa94c00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 0x7faa900ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa8fe00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8003a8f0 con 0x7faa900b6990 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8003aa50 con 0x7faa900b6990 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8003ae00 con 0x7faa900b6990 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9015ded0 con 0x7faa900b6990 2024-02-02T13:26:13.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.264+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901f2200 con 0x7faa900b6990 2024-02-02T13:26:13.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.266+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa900b6e00 con 0x7faa900b6990 2024-02-02T13:26:13.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.267+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faa8004d450 con 0x7faa900b6990 2024-02-02T13:26:13.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.268+0000 7faa87e00640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.268+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa800aa9a0 con 0x7faa900b6990 2024-02-02T13:26:13.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.268+0000 7faa8f400640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.269+0000 7faa8f400640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 0x7faa6806c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa901f5880 tx=0x7faa78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.272+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faa900b6e00 con 0x7faa900b6990 2024-02-02T13:26:13.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.495+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa900b6e00 con 0x7faa900b6990 2024-02-02T13:26:13.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.496+0000 7faa87e00640 1 -- 172.21.15.91:0/3222051424 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7faa900b6e00 con 0x7faa900b6990 2024-02-02T13:26:13.498 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:13.498 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:26:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:26:13.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.498+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 msgr2=0x7faa6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.498+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 0x7faa6806c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa901f5880 tx=0x7faa78009040 comp rx=0 tx=0).stop 2024-02-02T13:26:13.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.498+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 msgr2=0x7faa900b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.498+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa800046c0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 shutdown_connections 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa900ce1d0 0x7faa900c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9015cee0 0x7faa900ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 --2- 172.21.15.91:0/3222051424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa900b6990 0x7faa900b42b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 >> 172.21.15.91:0/3222051424 conn(0x7faa90070690 msgr2=0x7faa900c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 shutdown_connections 2024-02-02T13:26:13.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.499+0000 7faa95600640 1 -- 172.21.15.91:0/3222051424 wait complete. 2024-02-02T13:26:13.518 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:13.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.716+0000 7f5528600640 1 Processor -- start 2024-02-02T13:26:13.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.717+0000 7f5528600640 1 -- start start 2024-02-02T13:26:13.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5528600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f552015a010 0x7f55200c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5528600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55200cc7c0 con 0x7f55200b0700 2024-02-02T13:26:13.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5528600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55200ce930 con 0x7f55200ce130 2024-02-02T13:26:13.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5528600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55200cd640 con 0x7f552015a010 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5526600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f55200b0700 0x7f552006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37444/0 (socket says 172.21.15.91:37444) 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.718+0000 7f5526600640 1 -- 172.21.15.91:0/817170812 learned_addr learned my addr 172.21.15.91:0/817170812 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822715129 0 0) 0x7f55200cc7c0 con 0x7f55200b0700 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f5525c00640 1 --2- 172.21.15.91:0/817170812 >> v2:172.21.15.113:3300/0 conn(0x7f552015a010 0x7f55200c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5500003520 con 0x7f55200b0700 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652267730 0 0) 0x7f5500003520 con 0x7f55200b0700 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 >> v2:172.21.15.113:3300/0 conn(0x7f552015a010 msgr2=0x7f55200c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 --2- 172.21.15.91:0/817170812 >> v2:172.21.15.113:3300/0 conn(0x7f552015a010 0x7f55200c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 >> v1:172.21.15.113:6789/0 conn(0x7f55200ce130 legacy=0x7f55200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552006c890 con 0x7f55200b0700 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5518001670 con 0x7f55200b0700 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 >> v1:172.21.15.91:6789/0 conn(0x7f55200b0700 legacy=0x7f552006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5500003970 0x7f5500003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 0x7f5500004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5500005180 0x7f550000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200cd640 con 0x7f55000047c0 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550000e210 con 0x7f5500003970 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.719+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550000e3d0 con 0x7f5500005180 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 0x7f5500004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525c00640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5500003970 0x7f5500003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 -- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5500005180 msgr2=0x7f550000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5500005180 0x7f550000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 -- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5500003970 msgr2=0x7f5500003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5500003970 0x7f5500003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5525200640 1 -- 172.21.15.91:0/817170812 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500003520 con 0x7f55000047c0 2024-02-02T13:26:13.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.720+0000 7f5526600640 1 -- 172.21.15.91:0/817170812 reap_dead start 2024-02-02T13:26:13.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.721+0000 7f5525200640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 0x7f5500004c40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5508003db0 tx=0x7f55000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d25d97ecb540e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.721+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508032600 con 0x7f55000047c0 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.721+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550802e050 con 0x7f55000047c0 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.721+0000 7f551fe00640 1 -- 172.21.15.91:0/817170812 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55080329e0 con 0x7f55000047c0 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 -- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 msgr2=0x7f5500004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 0x7f5500004c40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5508003db0 tx=0x7f55000057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 -- 172.21.15.91:0/817170812 shutdown_connections 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 --2- 172.21.15.91:0/817170812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55000047c0 0x7f5500004c40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 -- 172.21.15.91:0/817170812 >> 172.21.15.91:0/817170812 conn(0x7f55200cec80 msgr2=0x7f55200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 -- 172.21.15.91:0/817170812 shutdown_connections 2024-02-02T13:26:13.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.722+0000 7f5528600640 1 -- 172.21.15.91:0/817170812 wait complete. 2024-02-02T13:26:13.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.724+0000 7f5528600640 1 Processor -- start 2024-02-02T13:26:13.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.724+0000 7f5528600640 1 -- start start 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55000047c0 0x7f55200b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55200c5cc0 0x7f55200c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200ae430 con 0x7f552015da40 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552015e400 con 0x7f55000047c0 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5528600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200c6680 con 0x7f55200c5cc0 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5525200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.725+0000 7f5525200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49122/0 (socket says 172.21.15.91:49122) 2024-02-02T13:26:13.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 -- 172.21.15.91:0/2386203123 learned_addr learned my addr 172.21.15.91:0/2386203123 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:13.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525c00640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55000047c0 0x7f55200b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 -- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55200c5cc0 msgr2=0x7f55200c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55200c5cc0 0x7f55200c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 -- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55000047c0 msgr2=0x7f55200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55000047c0 0x7f55200b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 -- 172.21.15.91:0/2386203123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200c6bb0 con 0x7f552015da40 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5525200640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5508035040 tx=0x7f55080036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550803c410 con 0x7f552015da40 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550803c570 con 0x7f552015da40 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.726+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55200c68d0 con 0x7f552015da40 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.727+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550803c900 con 0x7f552015da40 2024-02-02T13:26:13.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.727+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55200b2d20 con 0x7f552015da40 2024-02-02T13:26:13.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.728+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55200ae3c0 con 0x7f552015da40 2024-02-02T13:26:13.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.729+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5508039050 con 0x7f552015da40 2024-02-02T13:26:13.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.729+0000 7f551d600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 0x7f54f806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:13.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.730+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55080a9d80 con 0x7f552015da40 2024-02-02T13:26:13.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.730+0000 7f5525c00640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 0x7f54f806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:13.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.731+0000 7f5525c00640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 0x7f54f806c920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5510007380 tx=0x7f551000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:13.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.734+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f55080754a0 con 0x7f552015da40 2024-02-02T13:26:13.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.941+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5520066cf0 con 0x7f552015da40 2024-02-02T13:26:13.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.942+0000 7f551d600640 1 -- 172.21.15.91:0/2386203123 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f5508037090 con 0x7f552015da40 2024-02-02T13:26:13.944 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:13.944 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:25:48.146857+0000","modified":"2024-02-02T13:25:51.676530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5052":{"gid":5052,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/3241118880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3241118880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3241118880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-02T13:26:13.945 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 16 2024-02-02T13:26:13.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 msgr2=0x7f54f806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 0x7f54f806c920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5510007380 tx=0x7f551000b040 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 msgr2=0x7f552015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5508035040 tx=0x7f55080036f0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 shutdown_connections 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a460 0x7f54f806c920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55200c5cc0 0x7f55200c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552015da40 0x7f552015dec0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 --2- 172.21.15.91:0/2386203123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55000047c0 0x7f55200b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 >> 172.21.15.91:0/2386203123 conn(0x7f55200cec80 msgr2=0x7f55200ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 shutdown_connections 2024-02-02T13:26:13.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:13.945+0000 7f5528600640 1 -- 172.21.15.91:0/2386203123 wait complete. 2024-02-02T13:26:13.965 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:14.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.167+0000 7efd54e00640 1 Processor -- start 2024-02-02T13:26:14.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.168+0000 7efd54e00640 1 -- start start 2024-02-02T13:26:14.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd54e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd500ce130 0x7efd500c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd54e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500b0700 0x7efd500b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd54e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500cc7c0 con 0x7efd5015a010 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd54e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd500b10d0 con 0x7efd500ce130 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd54e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500ce930 con 0x7efd500b0700 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd45200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd500ce130 0x7efd500c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd45200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd500ce130 0x7efd500c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49132/0 (socket says 172.21.15.91:49132) 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.169+0000 7efd45200640 1 -- 172.21.15.91:0/623163904 learned_addr learned my addr 172.21.15.91:0/623163904 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.170+0000 7efd4e800640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500b0700 0x7efd500b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.170+0000 7efd45200640 1 -- 172.21.15.91:0/623163904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500b0700 msgr2=0x7efd500b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.170+0000 7efd45200640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500b0700 0x7efd500b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.170+0000 7efd45200640 1 -- 172.21.15.91:0/623163904 >> v1:172.21.15.113:6789/0 conn(0x7efd5015a010 legacy=0x7efd500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.170+0000 7efd45200640 1 -- 172.21.15.91:0/623163904 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500cd640 con 0x7efd500ce130 2024-02-02T13:26:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.171+0000 7efd45200640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500ce130 0x7efd500c9670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cf9708484dc9a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.171+0000 7efd4d400640 1 -- 172.21.15.91:0/623163904 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38031640 con 0x7efd500ce130 2024-02-02T13:26:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.171+0000 7efd4d400640 1 -- 172.21.15.91:0/623163904 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd380317a0 con 0x7efd500ce130 2024-02-02T13:26:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.171+0000 7efd4d400640 1 -- 172.21.15.91:0/623163904 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38031b30 con 0x7efd500ce130 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 -- 172.21.15.91:0/623163904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500ce130 msgr2=0x7efd500c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500ce130 0x7efd500c9670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 -- 172.21.15.91:0/623163904 shutdown_connections 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500b0700 0x7efd500b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 --2- 172.21.15.91:0/623163904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500ce130 0x7efd500c9670 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 -- 172.21.15.91:0/623163904 >> 172.21.15.91:0/623163904 conn(0x7efd500cec80 msgr2=0x7efd500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 -- 172.21.15.91:0/623163904 shutdown_connections 2024-02-02T13:26:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.172+0000 7efd54e00640 1 -- 172.21.15.91:0/623163904 wait complete. 2024-02-02T13:26:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.174+0000 7efd54e00640 1 Processor -- start 2024-02-02T13:26:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.174+0000 7efd54e00640 1 -- start start 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500ce130 0x7efd5014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd5015a010 0x7efd501507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd50150140 con 0x7efd500b0700 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b29f0 con 0x7efd5015a010 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd54e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500c8ba0 con 0x7efd500ce130 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd4de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd4de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49140/0 (socket says 172.21.15.91:49140) 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd4de00640 1 -- 172.21.15.91:0/2042308733 learned_addr learned my addr 172.21.15.91:0/2042308733 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd4e800640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd5015a010 0x7efd501507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.175+0000 7efd45200640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500ce130 0x7efd5014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 -- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500ce130 msgr2=0x7efd5014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500ce130 0x7efd5014af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 -- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd5015a010 msgr2=0x7efd501507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd5015a010 0x7efd501507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 -- 172.21.15.91:0/2042308733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380094d0 con 0x7efd500b0700 2024-02-02T13:26:14.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd4de00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd4000ba80 tx=0x7efd4000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000cba0 con 0x7efd500b0700 2024-02-02T13:26:14.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4000cd00 con 0x7efd500b0700 2024-02-02T13:26:14.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40003bb0 con 0x7efd500b0700 2024-02-02T13:26:14.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5014f8a0 con 0x7efd500b0700 2024-02-02T13:26:14.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.176+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5014fe60 con 0x7efd500b0700 2024-02-02T13:26:14.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.177+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd50066cf0 con 0x7efd500b0700 2024-02-02T13:26:14.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.178+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd40014440 con 0x7efd500b0700 2024-02-02T13:26:14.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.179+0000 7efd46a00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 0x7efd2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.179+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd40088710 con 0x7efd500b0700 2024-02-02T13:26:14.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.179+0000 7efd45200640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 0x7efd2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.180+0000 7efd45200640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 0x7efd2806c8f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efd38036000 tx=0x7efd3803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.184+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd40053cf0 con 0x7efd500b0700 2024-02-02T13:26:14.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.376+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd500b4c00 con 0x7efd500b0700 2024-02-02T13:26:14.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.379+0000 7efd46a00640 1 -- 172.21.15.91:0/2042308733 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7efd40057310 con 0x7efd500b0700 2024-02-02T13:26:14.381 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:14.381 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":36,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:02.291859+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:25:46.131921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-02T13:25:47.166639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:14.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.381+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 msgr2=0x7efd2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.381+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 0x7efd2806c8f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efd38036000 tx=0x7efd3803c040 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.381+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 msgr2=0x7efd500c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.381+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd4000ba80 tx=0x7efd4000bf40 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 shutdown_connections 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd2806a430 0x7efd2806c8f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd500ce130 0x7efd5014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd5015a010 0x7efd501507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 --2- 172.21.15.91:0/2042308733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd500b0700 0x7efd500c8660 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 >> 172.21.15.91:0/2042308733 conn(0x7efd500cec80 msgr2=0x7efd5015ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 shutdown_connections 2024-02-02T13:26:14.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.382+0000 7efd54e00640 1 -- 172.21.15.91:0/2042308733 wait complete. 2024-02-02T13:26:14.400 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:26:14.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.596+0000 7fe780000640 1 Processor -- start 2024-02-02T13:26:14.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.596+0000 7fe780000640 1 -- start start 2024-02-02T13:26:14.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe780000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe778160d00 0x7fe778071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe780000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe780000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780b8890 con 0x7fe7780b1c40 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe780000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe778161b10 con 0x7fe778160d00 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe780000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b8b30 con 0x7fe77815d3b0 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe77d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe778160d00 0x7fe778071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe77e000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe7780b1c40 0x7fe7781610e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37458/0 (socket says 172.21.15.91:37458) 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.598+0000 7fe77e000640 1 -- 172.21.15.91:0/3941153514 learned_addr learned my addr 172.21.15.91:0/3941153514 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77cc00640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77d600640 1 -- 172.21.15.91:0/3941153514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 msgr2=0x7fe7780d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77d600640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77d600640 1 -- 172.21.15.91:0/3941153514 >> v1:172.21.15.91:6789/0 conn(0x7fe7780b1c40 legacy=0x7fe7781610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77d600640 1 -- 172.21.15.91:0/3941153514 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780b8e10 con 0x7fe778160d00 2024-02-02T13:26:14.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe777e00640 1 -- 172.21.15.91:0/3941153514 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626117740 0 0) 0x7fe7780b8890 con 0x7fe7780b1c40 2024-02-02T13:26:14.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.599+0000 7fe77cc00640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.600+0000 7fe77d600640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe778160d00 0x7fe778071840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350906e6f324ed79 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.600+0000 7fe777e00640 1 -- 172.21.15.91:0/3941153514 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76802cc50 con 0x7fe778160d00 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.600+0000 7fe777e00640 1 -- 172.21.15.91:0/3941153514 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76802cdb0 con 0x7fe778160d00 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.600+0000 7fe777e00640 1 -- 172.21.15.91:0/3941153514 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768034660 con 0x7fe778160d00 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 -- 172.21.15.91:0/3941153514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe778160d00 msgr2=0x7fe778071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe778160d00 0x7fe778071840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802bea0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 -- 172.21.15.91:0/3941153514 shutdown_connections 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 --2- 172.21.15.91:0/3941153514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe778160d00 0x7fe778071840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 -- 172.21.15.91:0/3941153514 >> 172.21.15.91:0/3941153514 conn(0x7fe7780d3010 msgr2=0x7fe7780d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:14.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 -- 172.21.15.91:0/3941153514 shutdown_connections 2024-02-02T13:26:14.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.601+0000 7fe780000640 1 -- 172.21.15.91:0/3941153514 wait complete. 2024-02-02T13:26:14.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.603+0000 7fe780000640 1 Processor -- start 2024-02-02T13:26:14.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.603+0000 7fe780000640 1 -- start start 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe778160d00 0x7fe7780b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ca180 con 0x7fe7780b1c40 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ca460 con 0x7fe778160d00 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.604+0000 7fe780000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ca740 con 0x7fe77815d3b0 2024-02-02T13:26:14.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49152/0 (socket says 172.21.15.91:49152) 2024-02-02T13:26:14.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 -- 172.21.15.91:0/547279246 learned_addr learned my addr 172.21.15.91:0/547279246 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77cc00640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 -- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 msgr2=0x7fe7780c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77e000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe778160d00 0x7fe7780b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 -- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe778160d00 msgr2=0x7fe7780b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe778160d00 0x7fe7780b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.605+0000 7fe77d600640 1 -- 172.21.15.91:0/547279246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009a70 con 0x7fe7780b1c40 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe77d600640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe768037040 tx=0x7fe768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768040650 con 0x7fe7780b1c40 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680407b0 con 0x7fe7780b1c40 2024-02-02T13:26:14.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7780caa20 con 0x7fe7780b1c40 2024-02-02T13:26:14.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768040ba0 con 0x7fe7780b1c40 2024-02-02T13:26:14.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.606+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778151ab0 con 0x7fe7780b1c40 2024-02-02T13:26:14.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.607+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7780caf40 con 0x7fe7780b1c40 2024-02-02T13:26:14.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.608+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe768040d00 con 0x7fe7780b1c40 2024-02-02T13:26:14.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.609+0000 7fe775600640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 0x7fe75006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:14.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.609+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7680ab8b0 con 0x7fe7780b1c40 2024-02-02T13:26:14.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.609+0000 7fe77cc00640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 0x7fe75006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:14.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.610+0000 7fe77cc00640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 0x7fe75006c8f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7780cd250 tx=0x7fe760008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:14.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.614+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe7780caf40 con 0x7fe7780b1c40 2024-02-02T13:26:14.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.841+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7780caf40 con 0x7fe7780b1c40 2024-02-02T13:26:14.844 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:26:14.843+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:26:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.860+0000 7fe775600640 1 -- 172.21.15.91:0/547279246 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe768076f20 con 0x7fe7780b1c40 2024-02-02T13:26:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 msgr2=0x7fe75006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 0x7fe75006c8f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7780cd250 tx=0x7fe760008040 comp rx=0 tx=0).stop 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 msgr2=0x7fe778151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe768037040 tx=0x7fe768004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 shutdown_connections 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe75006a430 0x7fe75006c8f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe77815d3b0 0x7fe7780c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe778160d00 0x7fe7780b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 --2- 172.21.15.91:0/547279246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7780b1c40 0x7fe778151570 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:14.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 >> 172.21.15.91:0/547279246 conn(0x7fe7780d3010 msgr2=0x7fe7780b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:14.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 shutdown_connections 2024-02-02T13:26:14.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:14.863+0000 7fe780000640 1 -- 172.21.15.91:0/547279246 wait complete. 2024-02-02T13:26:14.881 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:26:14.889 INFO:tasks.ceph.mds.b.smithi113.stderr: -4961> 2024-02-02T13:25:50.638+0000 7fb4dc200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:26:14.957 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:26:15.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.081+0000 7f15a4c00640 1 Processor -- start 2024-02-02T13:26:15.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.081+0000 7f15a4c00640 1 -- start start 2024-02-02T13:26:15.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f15a4c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15a00cc8d0 0x7f15a015f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f15a4c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a015fc70 con 0x7f15a006cbb0 2024-02-02T13:26:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f15a4c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a0070450 con 0x7f15a015e640 2024-02-02T13:26:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f15a4c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00b50c0 con 0x7f15a00cc8d0 2024-02-02T13:26:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f159e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f15a006cbb0 0x7f15a00cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37468/0 (socket says 172.21.15.91:37468) 2024-02-02T13:26:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.083+0000 7f159e800640 1 -- 172.21.15.91:0/820311876 learned_addr learned my addr 172.21.15.91:0/820311876 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159d400640 1 --2- 172.21.15.91:0/820311876 >> v2:172.21.15.113:3300/0 conn(0x7f15a00cc8d0 0x7f15a015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269216015 0 0) 0x7f15a015fc70 con 0x7f15a006cbb0 2024-02-02T13:26:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1580003520 con 0x7f15a006cbb0 2024-02-02T13:26:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114772390 0 0) 0x7f1580003520 con 0x7f15a006cbb0 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 >> v2:172.21.15.113:3300/0 conn(0x7f15a00cc8d0 msgr2=0x7f15a015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 --2- 172.21.15.91:0/820311876 >> v2:172.21.15.113:3300/0 conn(0x7f15a00cc8d0 0x7f15a015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 >> v1:172.21.15.113:6789/0 conn(0x7f15a015e640 legacy=0x7f15a00cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00718c0 con 0x7f15a006cbb0 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1598001670 con 0x7f15a006cbb0 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 >> v1:172.21.15.91:6789/0 conn(0x7f15a006cbb0 legacy=0x7f15a00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580003970 0x7f1580003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15800047c0 0x7f1580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 0x7f158000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b50c0 con 0x7f1580005180 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158000c120 con 0x7f1580003970 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.084+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158000c2e0 con 0x7f15800047c0 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 0x7f158000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159de00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580003970 0x7f1580003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159d400640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15800047c0 0x7f1580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 -- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15800047c0 msgr2=0x7f1580004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15800047c0 0x7f1580004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 -- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580003970 msgr2=0x7f1580003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580003970 0x7f1580003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159e800640 1 -- 172.21.15.91:0/820311876 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1580003520 con 0x7f1580005180 2024-02-02T13:26:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.085+0000 7f159d400640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15800047c0 0x7f1580004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.086+0000 7f159e800640 1 -- 172.21.15.91:0/820311876 reap_dead start 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.086+0000 7f159e800640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 0x7f158000b980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f159800c240 tx=0x7f15800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530c693c5b71bcc6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.086+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598005ad0 con 0x7f1580005180 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.086+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598005c70 con 0x7f1580005180 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.086+0000 7f159ca00640 1 -- 172.21.15.91:0/820311876 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159802c8b0 con 0x7f1580005180 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 -- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 msgr2=0x7f158000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 0x7f158000b980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f159800c240 tx=0x7f15800057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 -- 172.21.15.91:0/820311876 shutdown_connections 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 --2- 172.21.15.91:0/820311876 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1580005180 0x7f158000b980 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 -- 172.21.15.91:0/820311876 >> 172.21.15.91:0/820311876 conn(0x7f15a0069e30 msgr2=0x7f15a006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 -- 172.21.15.91:0/820311876 shutdown_connections 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.087+0000 7f15a4c00640 1 -- 172.21.15.91:0/820311876 wait complete. 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.089+0000 7f15a4c00640 1 Processor -- start 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.089+0000 7f15a4c00640 1 -- start start 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580005180 0x7f15a00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a00ad880 0x7f15a01f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b2d20 con 0x7f15a01f2d60 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00cba50 con 0x7f1580005180 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f15a4c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01f3720 con 0x7f15a00ad880 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f159e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f159e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49166/0 (socket says 172.21.15.91:49166) 2024-02-02T13:26:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f159e800640 1 -- 172.21.15.91:0/1213208537 learned_addr learned my addr 172.21.15.91:0/1213208537 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.090+0000 7f159d400640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a00ad880 0x7f15a01f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159de00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580005180 0x7f15a00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 -- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a00ad880 msgr2=0x7f15a01f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a00ad880 0x7f15a01f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 -- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580005180 msgr2=0x7f15a00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580005180 0x7f15a00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 -- 172.21.15.91:0/1213208537 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00b1e50 con 0x7f15a01f2d60 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f159e800640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1598001930 tx=0x7f1598001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15980049e0 con 0x7f15a01f2d60 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598004b40 con 0x7f15a01f2d60 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a00addf0 con 0x7f15a01f2d60 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.091+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598000ed0 con 0x7f15a01f2d60 2024-02-02T13:26:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.092+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a00ca7b0 con 0x7f15a01f2d60 2024-02-02T13:26:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.092+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0067400 con 0x7f15a01f2d60 2024-02-02T13:26:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.094+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1598032070 con 0x7f15a01f2d60 2024-02-02T13:26:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.094+0000 7f1596000640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 0x7f157806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.094+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f159803c070 con 0x7f15a01f2d60 2024-02-02T13:26:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.095+0000 7f159de00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 0x7f157806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.095+0000 7f159de00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 0x7f157806c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1588005410 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:15.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.099+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1598074e00 con 0x7f15a01f2d60 2024-02-02T13:26:15.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.327+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f15a006c480 con 0x7f15a01f2d60 2024-02-02T13:26:15.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.863+0000 7f1596000640 1 -- 172.21.15.91:0/1213208537 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1598078420 con 0x7f15a01f2d60 2024-02-02T13:26:15.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 msgr2=0x7f157806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 0x7f157806c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1588005410 tx=0x7f1588009040 comp rx=0 tx=0).stop 2024-02-02T13:26:15.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 msgr2=0x7f15a01f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1598001930 tx=0x7f1598001a00 comp rx=0 tx=0).stop 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 shutdown_connections 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f157806a430 0x7f157806c8f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a00ad880 0x7f15a01f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.867+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a01f2d60 0x7f15a01f31e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.868+0000 7f15a4c00640 1 --2- 172.21.15.91:0/1213208537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1580005180 0x7f15a00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.868+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 >> 172.21.15.91:0/1213208537 conn(0x7f15a0069e30 msgr2=0x7f15a00c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.868+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 shutdown_connections 2024-02-02T13:26:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:15.868+0000 7f15a4c00640 1 -- 172.21.15.91:0/1213208537 wait complete. 2024-02-02T13:26:15.890 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:26:16.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.090+0000 7fbff7400640 1 Processor -- start 2024-02-02T13:26:16.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.090+0000 7fbff7400640 1 -- start start 2024-02-02T13:26:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbff015d2b0 0x7fbff00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff7400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00d1e80 con 0x7fbff00d0250 2024-02-02T13:26:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00d2120 con 0x7fbff015d690 2024-02-02T13:26:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff7400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00d2400 con 0x7fbff015d2b0 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff5400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbff00d0250 0x7fbff00b61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37472/0 (socket says 172.21.15.91:37472) 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff5400640 1 -- 172.21.15.91:0/1673361310 learned_addr learned my addr 172.21.15.91:0/1673361310 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.092+0000 7fbff4a00640 1 --2- 172.21.15.91:0/1673361310 >> v2:172.21.15.113:3300/0 conn(0x7fbff015d2b0 0x7fbff00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308127378 0 0) 0x7fbff00d1e80 con 0x7fbff00d0250 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd0003520 con 0x7fbff00d0250 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 265722422 0 0) 0x7fbfd0003520 con 0x7fbff00d0250 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 >> v2:172.21.15.113:3300/0 conn(0x7fbff015d2b0 msgr2=0x7fbff00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 --2- 172.21.15.91:0/1673361310 >> v2:172.21.15.113:3300/0 conn(0x7fbff015d2b0 0x7fbff00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 >> v1:172.21.15.113:6789/0 conn(0x7fbff015d690 legacy=0x7fbff00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00d26e0 con 0x7fbff00d0250 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fbfe80019f0 con 0x7fbff00d0250 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 >> v1:172.21.15.91:6789/0 conn(0x7fbff00d0250 legacy=0x7fbff00b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 0x7fbfd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00047c0 0x7fbfd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfd0005180 0x7fbfd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00d2400 con 0x7fbfd00039c0 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd000c060 con 0x7fbfd00047c0 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.093+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd000c220 con 0x7fbfd0005180 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 0x7fbfd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 -- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfd0005180 msgr2=0x7fbfd000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfd0005180 0x7fbfd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 -- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00047c0 msgr2=0x7fbfd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00047c0 0x7fbfd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff4a00640 1 -- 172.21.15.91:0/1673361310 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0003520 con 0x7fbfd00039c0 2024-02-02T13:26:16.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.094+0000 7fbff5400640 1 -- 172.21.15.91:0/1673361310 reap_dead start 2024-02-02T13:26:16.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.095+0000 7fbff4a00640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 0x7fbfd0003de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009ac0 tx=0x7fbfd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6175ba59ea655695 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:16.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.095+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe002c6b0 con 0x7fbfd00039c0 2024-02-02T13:26:16.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.095+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe002c810 con 0x7fbfd00039c0 2024-02-02T13:26:16.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.095+0000 7fbfef400640 1 -- 172.21.15.91:0/1673361310 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe002cba0 con 0x7fbfd00039c0 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 -- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 msgr2=0x7fbfd0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 0x7fbfd0003de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009ac0 tx=0x7fbfd00057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 -- 172.21.15.91:0/1673361310 shutdown_connections 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 --2- 172.21.15.91:0/1673361310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfd00039c0 0x7fbfd0003de0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 -- 172.21.15.91:0/1673361310 >> 172.21.15.91:0/1673361310 conn(0x7fbff0065910 msgr2=0x7fbff0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 -- 172.21.15.91:0/1673361310 shutdown_connections 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.096+0000 7fbff7400640 1 -- 172.21.15.91:0/1673361310 wait complete. 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.098+0000 7fbff7400640 1 Processor -- start 2024-02-02T13:26:16.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.098+0000 7fbff7400640 1 -- start start 2024-02-02T13:26:16.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00039c0 0x7fbff015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 0x7fbff014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b8270 con 0x7fbff00b78b0 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff01f71e0 con 0x7fbfd00039c0 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff7400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff01f6660 con 0x7fbff00b5400 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49182/0 (socket says 172.21.15.91:49182) 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 -- 172.21.15.91:0/2175886842 learned_addr learned my addr 172.21.15.91:0/2175886842 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff4a00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00039c0 0x7fbff015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 -- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 msgr2=0x7fbff014f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbfefe00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 0x7fbff014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 0x7fbff014f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 -- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00039c0 msgr2=0x7fbff015e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00039c0 0x7fbff015e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbff5400640 1 -- 172.21.15.91:0/2175886842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff01f6940 con 0x7fbff00b78b0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.099+0000 7fbfefe00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 0x7fbff014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbff5400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbff00b6110 tx=0x7fbfe800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800cea0 con 0x7fbff00b78b0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe800a610 con 0x7fbff00b78b0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff01f6c20 con 0x7fbff00b78b0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800a9c0 con 0x7fbff00b78b0 2024-02-02T13:26:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.100+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff014fb40 con 0x7fbff00b78b0 2024-02-02T13:26:16.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.101+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff0066cf0 con 0x7fbff00b78b0 2024-02-02T13:26:16.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.102+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbfe800ab20 con 0x7fbff00b78b0 2024-02-02T13:26:16.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.103+0000 7fbfecc00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 0x7fbfc806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:16.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.103+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbfe80883e0 con 0x7fbff00b78b0 2024-02-02T13:26:16.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.103+0000 7fbff4a00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 0x7fbfc806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:16.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.104+0000 7fbff4a00640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 0x7fbfc806c920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0036000 tx=0x7fbfe0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:16.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.108+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbfe8053a20 con 0x7fbff00b78b0 2024-02-02T13:26:16.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.287+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbff00cb640 con 0x7fbff00b78b0 2024-02-02T13:26:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.869+0000 7fbfecc00640 1 -- 172.21.15.91:0/2175886842 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbfe8057040 con 0x7fbff00b78b0 2024-02-02T13:26:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:26:16.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 msgr2=0x7fbfc806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 0x7fbfc806c920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0036000 tx=0x7fbfe0030000 comp rx=0 tx=0).stop 2024-02-02T13:26:16.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 msgr2=0x7fbff00b7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:16.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbff00b6110 tx=0x7fbfe800cc50 comp rx=0 tx=0).stop 2024-02-02T13:26:16.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 shutdown_connections 2024-02-02T13:26:16.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc806a460 0x7fbfc806c920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbff00b5400 0x7fbff014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff00b78b0 0x7fbff00b7d30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 --2- 172.21.15.91:0/2175886842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd00039c0 0x7fbff015e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:16.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.873+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 >> 172.21.15.91:0/2175886842 conn(0x7fbff0065910 msgr2=0x7fbff00d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:16.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.874+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 shutdown_connections 2024-02-02T13:26:16.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:16.874+0000 7fbff7400640 1 -- 172.21.15.91:0/2175886842 wait complete. 2024-02-02T13:26:16.896 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:26:17.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.104+0000 7f3e15400640 1 Processor -- start 2024-02-02T13:26:17.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.104+0000 7f3e15400640 1 -- start start 2024-02-02T13:26:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.105+0000 7f3e15400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e10151f90 0x7f3e1015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e15400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3e100b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e15400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e10070f80 con 0x7f3e1006f4f0 2024-02-02T13:26:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e15400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e10152370 con 0x7f3e10151f90 2024-02-02T13:26:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e15400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100cb010 con 0x7f3e100b0e10 2024-02-02T13:26:17.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e07e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3e1006f4f0 0x7f3e100b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37482/0 (socket says 172.21.15.91:37482) 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e07e00640 1 -- 172.21.15.91:0/206172725 learned_addr learned my addr 172.21.15.91:0/206172725 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e0e800640 1 --2- 172.21.15.91:0/206172725 >> v2:172.21.15.113:3300/0 conn(0x7f3e10151f90 0x7f3e1015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945124277 0 0) 0x7f3e10070f80 con 0x7f3e1006f4f0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df0003520 con 0x7f3e1006f4f0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.106+0000 7f3e0f200640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3e100b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 374674265 0 0) 0x7f3df0003520 con 0x7f3e1006f4f0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 msgr2=0x7f3e100b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3e100b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 >> v2:172.21.15.113:3300/0 conn(0x7f3e10151f90 msgr2=0x7f3e1015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 --2- 172.21.15.91:0/206172725 >> v2:172.21.15.113:3300/0 conn(0x7f3e10151f90 0x7f3e1015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100cb2f0 con 0x7f3e1006f4f0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3df8001670 con 0x7f3e1006f4f0 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 >> v1:172.21.15.91:6789/0 conn(0x7f3e1006f4f0 legacy=0x7f3e100b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3df0003970 0x7f3df0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3df0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3df0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10152370 con 0x7f3df00047c0 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100cb010 con 0x7f3df0003970 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df000ba00 con 0x7f3e100b0e10 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.107+0000 7f3e07e00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3df0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e0e800640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3df0003970 0x7f3df0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e0f200640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3df0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e07e00640 1 -- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 msgr2=0x7f3df0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e07e00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3df0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e07e00640 1 -- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3df0003970 msgr2=0x7f3df0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e07e00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3df0003970 0x7f3df0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e07e00640 1 -- 172.21.15.91:0/206172725 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0003520 con 0x7f3df00047c0 2024-02-02T13:26:17.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.108+0000 7f3e0f200640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100b0e10 0x7f3df0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:17.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.109+0000 7f3e07e00640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3df0004c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c160 tx=0x7f3df00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b573c596b44e24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:17.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.109+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005ad0 con 0x7f3df00047c0 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.109+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8005c30 con 0x7f3df00047c0 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.109+0000 7f3e0de00640 1 -- 172.21.15.91:0/206172725 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df802c830 con 0x7f3df00047c0 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 -- 172.21.15.91:0/206172725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 msgr2=0x7f3df0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 --2- 172.21.15.91:0/206172725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3df0004c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c160 tx=0x7f3df00057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e0f200640 1 -- 172.21.15.91:0/206172725 reap_dead start 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 -- 172.21.15.91:0/206172725 shutdown_connections 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 -- 172.21.15.91:0/206172725 >> 172.21.15.91:0/206172725 conn(0x7f3e100cec80 msgr2=0x7f3e100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 -- 172.21.15.91:0/206172725 shutdown_connections 2024-02-02T13:26:17.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.110+0000 7f3e15400640 1 -- 172.21.15.91:0/206172725 wait complete. 2024-02-02T13:26:17.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.112+0000 7f3e15400640 1 Processor -- start 2024-02-02T13:26:17.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.112+0000 7f3e15400640 1 -- start start 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e10145ec0 0x7f3e100c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100c7c60 0x7f3e100af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10145ba0 con 0x7f3df00047c0 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10155b40 con 0x7f3e10145ec0 2024-02-02T13:26:17.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e15400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c8100 con 0x7f3e100c7c60 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e0e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e0e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49196/0 (socket says 172.21.15.91:49196) 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.113+0000 7f3e0e800640 1 -- 172.21.15.91:0/1075900235 learned_addr learned my addr 172.21.15.91:0/1075900235 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 -- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100c7c60 msgr2=0x7f3e100af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0f200640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100c7c60 0x7f3e100af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e07e00640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e10145ec0 0x7f3e100c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100c7c60 0x7f3e100af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 -- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e10145ec0 msgr2=0x7f3e100c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e10145ec0 0x7f3e100c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 -- 172.21.15.91:0/1075900235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100c83e0 con 0x7f3df00047c0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e0e800640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e000116d0 tx=0x7f3e00011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000f9c0 con 0x7f3df00047c0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0000fb20 con 0x7f3df00047c0 2024-02-02T13:26:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.114+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000fe90 con 0x7f3df00047c0 2024-02-02T13:26:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.115+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e10155e00 con 0x7f3df00047c0 2024-02-02T13:26:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.115+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e100af6c0 con 0x7f3df00047c0 2024-02-02T13:26:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.116+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e10066cf0 con 0x7f3df00047c0 2024-02-02T13:26:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.117+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e00015460 con 0x7f3df00047c0 2024-02-02T13:26:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.118+0000 7f3e06a00640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 0x7f3de806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.118+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f3e00088dc0 con 0x7f3df00047c0 2024-02-02T13:26:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.118+0000 7f3e07e00640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 0x7f3de806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.119+0000 7f3e07e00640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 0x7f3de806c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3df8001050 tx=0x7f3df8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:17.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.122+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e00054500 con 0x7f3df00047c0 2024-02-02T13:26:17.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.312+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e100c24a0 con 0x7f3df00047c0 2024-02-02T13:26:17.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.884+0000 7f3e06a00640 1 -- 172.21.15.91:0/1075900235 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3e00057b20 con 0x7f3df00047c0 2024-02-02T13:26:17.886 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:26:17.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 msgr2=0x7f3de806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 0x7f3de806c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3df8001050 tx=0x7f3df8007000 comp rx=0 tx=0).stop 2024-02-02T13:26:17.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 msgr2=0x7f3e10145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:17.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e000116d0 tx=0x7f3e00011b90 comp rx=0 tx=0).stop 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 shutdown_connections 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3de806a430 0x7f3de806c8f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e100c7c60 0x7f3e100af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e10145ec0 0x7f3e100c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.888+0000 7f3e15400640 1 --2- 172.21.15.91:0/1075900235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3df00047c0 0x7f3e10145660 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.889+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 >> 172.21.15.91:0/1075900235 conn(0x7f3e100cec80 msgr2=0x7f3e1015edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.889+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 shutdown_connections 2024-02-02T13:26:17.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:17.889+0000 7f3e15400640 1 -- 172.21.15.91:0/1075900235 wait complete. 2024-02-02T13:26:17.911 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:26:18.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.121+0000 7fb6efe00640 1 Processor -- start 2024-02-02T13:26:18.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.121+0000 7fb6efe00640 1 -- start start 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.122+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6efe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6f00b1400 0x7fb6f00cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6efe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f00cd170 con 0x7fb6f006cf20 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6efe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f009cfc0 con 0x7fb6f00b1400 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6efe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f006d490 con 0x7fb6f01605f0 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6f00b1400 0x7fb6f00cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6f00b1400 0x7fb6f00cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49198/0 (socket says 172.21.15.91:49198) 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 -- 172.21.15.91:0/1423066413 learned_addr learned my addr 172.21.15.91:0/1423066413 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6eea00640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 -- 172.21.15.91:0/1423066413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 msgr2=0x7fb6f00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 -- 172.21.15.91:0/1423066413 >> v1:172.21.15.113:6789/0 conn(0x7fb6f006cf20 legacy=0x7fb6f00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:18.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.123+0000 7fb6ef400640 1 -- 172.21.15.91:0/1423066413 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f006d770 con 0x7fb6f00b1400 2024-02-02T13:26:18.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.125+0000 7fb6ef400640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00cd930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38db009f80a08a39 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.125+0000 7fb6ee000640 1 -- 172.21.15.91:0/1423066413 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e802c9e0 con 0x7fb6f00b1400 2024-02-02T13:26:18.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.125+0000 7fb6ee000640 1 -- 172.21.15.91:0/1423066413 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e802cb40 con 0x7fb6f00b1400 2024-02-02T13:26:18.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.125+0000 7fb6ee000640 1 -- 172.21.15.91:0/1423066413 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80343f0 con 0x7fb6f00b1400 2024-02-02T13:26:18.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 -- 172.21.15.91:0/1423066413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 msgr2=0x7fb6f00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00cd930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e802be40 comp rx=0 tx=0).stop 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 -- 172.21.15.91:0/1423066413 shutdown_connections 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00cd930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 --2- 172.21.15.91:0/1423066413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 -- 172.21.15.91:0/1423066413 >> 172.21.15.91:0/1423066413 conn(0x7fb6f00cbcf0 msgr2=0x7fb6f00cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 -- 172.21.15.91:0/1423066413 shutdown_connections 2024-02-02T13:26:18.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.126+0000 7fb6efe00640 1 -- 172.21.15.91:0/1423066413 wait complete. 2024-02-02T13:26:18.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.127+0000 7fb6efe00640 1 Processor -- start 2024-02-02T13:26:18.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.128+0000 7fb6efe00640 1 -- start start 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6f006cf20 0x7fb6f00afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f009bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00c3de0 con 0x7fb6f00b1400 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f009a820 con 0x7fb6f006cf20 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6efe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f015cf50 con 0x7fb6f01605f0 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49208/0 (socket says 172.21.15.91:49208) 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 -- 172.21.15.91:0/2074819875 learned_addr learned my addr 172.21.15.91:0/2074819875 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6eea00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6f006cf20 0x7fb6f00afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 -- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 msgr2=0x7fb6f009bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f009bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 -- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6f006cf20 msgr2=0x7fb6f00afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6f006cf20 0x7fb6f00afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.129+0000 7fb6e7e00640 1 -- 172.21.15.91:0/2074819875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009b30 con 0x7fb6f00b1400 2024-02-02T13:26:18.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6e7e00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b3f0 tx=0x7fb6d800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011830 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8011990 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011d40 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f00c3160 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.130+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f00c3720 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.131+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f00c3f20 con 0x7fb6f00b1400 2024-02-02T13:26:18.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.132+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6d801a400 con 0x7fb6f00b1400 2024-02-02T13:26:18.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.133+0000 7fb6e6a00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 0x7fb6c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.133+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6d8088480 con 0x7fb6f00b1400 2024-02-02T13:26:18.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.133+0000 7fb6eea00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 0x7fb6c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.134+0000 7fb6eea00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 0x7fb6c806c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000b9b0 tx=0x7fb6e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.137+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb6f00c3f20 con 0x7fb6f00b1400 2024-02-02T13:26:18.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.326+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb6f00c3f20 con 0x7fb6f00b1400 2024-02-02T13:26:18.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.335+0000 7fb6e6a00640 1 -- 172.21.15.91:0/2074819875 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6f00c3f20 con 0x7fb6f00b1400 2024-02-02T13:26:18.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 msgr2=0x7fb6c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 0x7fb6c806c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000b9b0 tx=0x7fb6e0008040 comp rx=0 tx=0).stop 2024-02-02T13:26:18.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 msgr2=0x7fb6f00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b3f0 tx=0x7fb6d800b8b0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 shutdown_connections 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb6c806a430 0x7fb6c806c8f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6f01605f0 0x7fb6f009bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6f00b1400 0x7fb6f00b0410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 --2- 172.21.15.91:0/2074819875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6f006cf20 0x7fb6f00afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.337+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 >> 172.21.15.91:0/2074819875 conn(0x7fb6f00cbcf0 msgr2=0x7fb6f006fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.338+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 shutdown_connections 2024-02-02T13:26:18.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.338+0000 7fb6efe00640 1 -- 172.21.15.91:0/2074819875 wait complete. 2024-02-02T13:26:18.358 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-02-02T13:26:18.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.567+0000 7f2761200640 1 Processor -- start 2024-02-02T13:26:18.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.567+0000 7f2761200640 1 -- start start 2024-02-02T13:26:18.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2761200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f275c151f90 0x7f275c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2761200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f275c0b0e10 0x7f275c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2761200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c070f80 con 0x7f275c06f4f0 2024-02-02T13:26:18.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2761200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c152370 con 0x7f275c0b0e10 2024-02-02T13:26:18.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2761200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c0cb010 con 0x7f275c151f90 2024-02-02T13:26:18.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2759400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f275c06f4f0 0x7f275c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37490/0 (socket says 172.21.15.91:37490) 2024-02-02T13:26:18.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2759400640 1 -- 172.21.15.91:0/1896356943 learned_addr learned my addr 172.21.15.91:0/1896356943 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:18.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f275a800640 1 --2- 172.21.15.91:0/1896356943 >> v2:172.21.15.91:3300/0 conn(0x7f275c0b0e10 0x7f275c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2759e00640 1 --2- 172.21.15.91:0/1896356943 >> v2:172.21.15.113:3300/0 conn(0x7f275c151f90 0x7f275c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.569+0000 7f2758a00640 1 -- 172.21.15.91:0/1896356943 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345580592 0 0) 0x7f275c070f80 con 0x7f275c06f4f0 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.570+0000 7f2758a00640 1 -- 172.21.15.91:0/1896356943 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f273c003520 con 0x7f275c06f4f0 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.570+0000 7f275a800640 1 -- 172.21.15.91:0/1896356943 >> v2:172.21.15.113:3300/0 conn(0x7f275c151f90 msgr2=0x7f275c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.570+0000 7f275a800640 1 --2- 172.21.15.91:0/1896356943 >> v2:172.21.15.113:3300/0 conn(0x7f275c151f90 0x7f275c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.570+0000 7f275a800640 1 -- 172.21.15.91:0/1896356943 >> v1:172.21.15.91:6789/0 conn(0x7f275c06f4f0 legacy=0x7f275c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.570+0000 7f275a800640 1 -- 172.21.15.91:0/1896356943 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0cb2f0 con 0x7f275c0b0e10 2024-02-02T13:26:18.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.571+0000 7f275a800640 1 --2- 172.21.15.91:0/1896356943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0b3af0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f275c0cc770 tx=0x7f275402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1894e7c74da67109 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.571+0000 7f2758a00640 1 -- 172.21.15.91:0/1896356943 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754031830 con 0x7f275c0b0e10 2024-02-02T13:26:18.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.571+0000 7f2758a00640 1 -- 172.21.15.91:0/1896356943 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2754031990 con 0x7f275c0b0e10 2024-02-02T13:26:18.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.571+0000 7f2758a00640 1 -- 172.21.15.91:0/1896356943 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754031d20 con 0x7f275c0b0e10 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 -- 172.21.15.91:0/1896356943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 msgr2=0x7f275c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 --2- 172.21.15.91:0/1896356943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0b3af0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f275c0cc770 tx=0x7f275402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 -- 172.21.15.91:0/1896356943 shutdown_connections 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 --2- 172.21.15.91:0/1896356943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0b3af0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 --2- 172.21.15.91:0/1896356943 >> v2:172.21.15.113:3300/0 conn(0x7f275c151f90 0x7f275c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 -- 172.21.15.91:0/1896356943 >> 172.21.15.91:0/1896356943 conn(0x7f275c0cec80 msgr2=0x7f275c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:18.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 -- 172.21.15.91:0/1896356943 shutdown_connections 2024-02-02T13:26:18.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.572+0000 7f2761200640 1 -- 172.21.15.91:0/1896356943 wait complete. 2024-02-02T13:26:18.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.574+0000 7f2761200640 1 Processor -- start 2024-02-02T13:26:18.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.574+0000 7f2761200640 1 -- start start 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f275c06f4f0 0x7f275c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f275c151f90 0x7f275c0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0c2720 con 0x7f275c0b0e10 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0c6b40 con 0x7f275c06f4f0 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2761200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0c2970 con 0x7f275c151f90 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.575+0000 7f2759400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49220/0 (socket says 172.21.15.91:49220) 2024-02-02T13:26:18.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 -- 172.21.15.91:0/765041142 learned_addr learned my addr 172.21.15.91:0/765041142 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759e00640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f275c06f4f0 0x7f275c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 -- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f275c151f90 msgr2=0x7f275c0ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f275c151f90 0x7f275c0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 -- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f275c06f4f0 msgr2=0x7f275c151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f275c06f4f0 0x7f275c151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 -- 172.21.15.91:0/765041142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754009400 con 0x7f275c0b0e10 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2759400640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f274400c390 tx=0x7f274400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744005ce0 con 0x7f275c0b0e10 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744005e40 con 0x7f275c0b0e10 2024-02-02T13:26:18.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c0c2c50 con 0x7f275c0b0e10 2024-02-02T13:26:18.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.576+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440072f0 con 0x7f275c0b0e10 2024-02-02T13:26:18.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.577+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c151680 con 0x7f275c0b0e10 2024-02-02T13:26:18.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.578+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0c3170 con 0x7f275c0b0e10 2024-02-02T13:26:18.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.578+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2744007450 con 0x7f275c0b0e10 2024-02-02T13:26:18.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.579+0000 7f2752000640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:18.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.579+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2744087c60 con 0x7f275c0b0e10 2024-02-02T13:26:18.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.580+0000 7f2759e00640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:18.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.580+0000 7f2759e00640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 0x7f273406c8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f275c158590 tx=0x7f274c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:18.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.584+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f275c0c3170 con 0x7f275c0b0e10 2024-02-02T13:26:18.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.842+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f275c0c3170 con 0x7f275c0b0e10 2024-02-02T13:26:18.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.891+0000 7f2752000640 1 -- 172.21.15.91:0/765041142 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v111) v1 ==== 130+0+0 (secure 0 0 0) 0x7f275c0c3170 con 0x7f275c0b0e10 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 msgr2=0x7f273406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 0x7f273406c8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f275c158590 tx=0x7f274c00b040 comp rx=0 tx=0).stop 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 msgr2=0x7f275c0ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f274400c390 tx=0x7f274400c850 comp rx=0 tx=0).stop 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 shutdown_connections 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f275c151f90 0x7f275c0ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f275c0b0e10 0x7f275c0ad920 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 --2- 172.21.15.91:0/765041142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f275c06f4f0 0x7f275c151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:18.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 >> 172.21.15.91:0/765041142 conn(0x7f275c0cec80 msgr2=0x7f275c15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:18.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 shutdown_connections 2024-02-02T13:26:18.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:18.894+0000 7f2761200640 1 -- 172.21.15.91:0/765041142 wait complete. 2024-02-02T13:26:18.913 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-02-02T13:26:18.914 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-02-02T13:26:18.914 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.116+0000 7f2543c00640 1 Processor -- start 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.117+0000 7f2543c00640 1 -- start start 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f253c15ea40 0x7f253c0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f253c0cdf30 0x7f253c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c15ee20 con 0x7f253c15ea40 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0d1db0 con 0x7f253c0cdf30 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2543c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0b2650 con 0x7f253c0cec60 2024-02-02T13:26:19.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2541200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f253c15ea40 0x7f253c0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:19.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2541200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f253c15ea40 0x7f253c0cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49236/0 (socket says 172.21.15.91:49236) 2024-02-02T13:26:19.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.118+0000 7f2541200640 1 -- 172.21.15.91:0/1461671935 learned_addr learned my addr 172.21.15.91:0/1461671935 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:19.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.119+0000 7f2541200640 1 -- 172.21.15.91:0/1461671935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 msgr2=0x7f253c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.119+0000 7f2541200640 1 --2- 172.21.15.91:0/1461671935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.119+0000 7f2541200640 1 -- 172.21.15.91:0/1461671935 >> v2:172.21.15.113:3300/0 conn(0x7f253c0cdf30 msgr2=0x7f253c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.119+0000 7f2541200640 1 --2- 172.21.15.91:0/1461671935 >> v2:172.21.15.113:3300/0 conn(0x7f253c0cdf30 0x7f253c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.119+0000 7f2541200640 1 -- 172.21.15.91:0/1461671935 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0d1ad0 con 0x7f253c15ea40 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.120+0000 7f2541200640 1 --2- 172.21.15.91:0/1461671935 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c15ea40 0x7f253c0cf460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f253c09ed70 tx=0x7f252c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19abf396698246e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.120+0000 7f253b400640 1 -- 172.21.15.91:0/1461671935 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0317e0 con 0x7f253c15ea40 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.120+0000 7f253b400640 1 -- 172.21.15.91:0/1461671935 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c031940 con 0x7f253c15ea40 2024-02-02T13:26:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.120+0000 7f253b400640 1 -- 172.21.15.91:0/1461671935 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c031cd0 con 0x7f253c15ea40 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 -- 172.21.15.91:0/1461671935 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c15ea40 msgr2=0x7f253c0cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 --2- 172.21.15.91:0/1461671935 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c15ea40 0x7f253c0cf460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f253c09ed70 tx=0x7f252c02b880 comp rx=0 tx=0).stop 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 -- 172.21.15.91:0/1461671935 shutdown_connections 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 --2- 172.21.15.91:0/1461671935 >> v2:172.21.15.113:3300/0 conn(0x7f253c0cdf30 0x7f253c0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 --2- 172.21.15.91:0/1461671935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 --2- 172.21.15.91:0/1461671935 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c15ea40 0x7f253c0cf460 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 -- 172.21.15.91:0/1461671935 >> 172.21.15.91:0/1461671935 conn(0x7f253c0d0e80 msgr2=0x7f253c0d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:19.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 -- 172.21.15.91:0/1461671935 shutdown_connections 2024-02-02T13:26:19.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.121+0000 7f2543c00640 1 -- 172.21.15.91:0/1461671935 wait complete. 2024-02-02T13:26:19.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.123+0000 7f2543c00640 1 Processor -- start 2024-02-02T13:26:19.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.123+0000 7f2543c00640 1 -- start start 2024-02-02T13:26:19.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f253c15ea40 0x7f253c0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c157f20 con 0x7f253c0cdf30 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0cac50 con 0x7f253c15ea40 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2543c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0caef0 con 0x7f253c0cec60 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49242/0 (socket says 172.21.15.91:49242) 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 -- 172.21.15.91:0/1604847402 learned_addr learned my addr 172.21.15.91:0/1604847402 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 -- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 msgr2=0x7f253c1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 -- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f253c15ea40 msgr2=0x7f253c0c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f253c15ea40 0x7f253c0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 -- 172.21.15.91:0/1604847402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0094d0 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.124+0000 7f2541200640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f252c02b8c0 tx=0x7f252c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.125+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c032940 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.125+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c032aa0 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.125+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c157bb0 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.125+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c032e50 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.125+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c1588f0 con 0x7f253c0cdf30 2024-02-02T13:26:19.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.126+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c157d40 con 0x7f253c0cdf30 2024-02-02T13:26:19.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.127+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f252c04e450 con 0x7f253c0cdf30 2024-02-02T13:26:19.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.128+0000 7f2538c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 0x7f251406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:19.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.128+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f252c0aa890 con 0x7f253c0cdf30 2024-02-02T13:26:19.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.128+0000 7f253be00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 0x7f251406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:19.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.128+0000 7f253be00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 0x7f251406c8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f253c151b10 tx=0x7f2524006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:19.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.132+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f252c076230 con 0x7f253c0cdf30 2024-02-02T13:26:19.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.373+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f253c0ccce0 con 0x7f253c0cdf30 2024-02-02T13:26:19.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.895+0000 7f2538c00640 1 -- 172.21.15.91:0/1604847402 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v112) v1 ==== 145+0+0 (secure 0 0 0) 0x7f252c079850 con 0x7f253c0cdf30 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 msgr2=0x7f251406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 0x7f251406c8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f253c151b10 tx=0x7f2524006bf0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 msgr2=0x7f253c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f252c02b8c0 tx=0x7f252c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 shutdown_connections 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f251406a430 0x7f251406c8f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f253c0cec60 0x7f253c1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f253c15ea40 0x7f253c0c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 --2- 172.21.15.91:0/1604847402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f253c0cdf30 0x7f253c1515d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 >> 172.21.15.91:0/1604847402 conn(0x7f253c0d0e80 msgr2=0x7f253c15d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 shutdown_connections 2024-02-02T13:26:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:19.899+0000 7f2543c00640 1 -- 172.21.15.91:0/1604847402 wait complete. 2024-02-02T13:26:19.921 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-02-02T13:26:20.140 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_objectstore":"memstore"} 2024-02-02T13:26:20.140 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-02-02T13:26:20.141 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:26:20.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.356+0000 7f4395200640 1 Processor -- start 2024-02-02T13:26:20.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.356+0000 7f4395200640 1 -- start start 2024-02-02T13:26:20.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.357+0000 7f4395200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43900ce130 0x7f43900c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.357+0000 7f4395200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.357+0000 7f4395200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43900cc7c0 con 0x7f439015a010 2024-02-02T13:26:20.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.357+0000 7f4395200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43900b10d0 con 0x7f43900ce130 2024-02-02T13:26:20.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.357+0000 7f4395200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43900ce930 con 0x7f43900b0700 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f4387e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43900ce130 0x7f43900c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f438e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f4387e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43900ce130 0x7f43900c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:42194/0 (socket says 172.21.15.91:42194) 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f4387e00640 1 -- 172.21.15.91:0/985854046 learned_addr learned my addr 172.21.15.91:0/985854046 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f438de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f439015a010 0x7f43900ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:38854/0 (socket says 172.21.15.91:38854) 2024-02-02T13:26:20.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.358+0000 7f438e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:34294/0 (socket says 172.21.15.91:34294) 2024-02-02T13:26:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.359+0000 7f4387e00640 1 -- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 msgr2=0x7f43900b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.359+0000 7f4387e00640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.359+0000 7f4387e00640 1 -- 172.21.15.91:0/985854046 >> v1:172.21.15.113:6789/0 conn(0x7f439015a010 legacy=0x7f43900ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.359+0000 7f4387e00640 1 -- 172.21.15.91:0/985854046 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43900cd640 con 0x7f43900ce130 2024-02-02T13:26:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.359+0000 7f438e800640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:26:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.361+0000 7f4387e00640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900ce130 0x7f43900c9670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4378009870 tx=0x7f437802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbd70c1497972d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:20.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.361+0000 7f438d400640 1 -- 172.21.15.91:0/985854046 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378030840 con 0x7f43900ce130 2024-02-02T13:26:20.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.361+0000 7f438d400640 1 -- 172.21.15.91:0/985854046 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43780309a0 con 0x7f43900ce130 2024-02-02T13:26:20.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.361+0000 7f438d400640 1 -- 172.21.15.91:0/985854046 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378030d30 con 0x7f43900ce130 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 -- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900ce130 msgr2=0x7f43900c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900ce130 0x7f43900c9670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4378009870 tx=0x7f437802b980 comp rx=0 tx=0).stop 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 -- 172.21.15.91:0/985854046 shutdown_connections 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43900b0700 0x7f43900b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 --2- 172.21.15.91:0/985854046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900ce130 0x7f43900c9670 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 -- 172.21.15.91:0/985854046 >> 172.21.15.91:0/985854046 conn(0x7f43900cec80 msgr2=0x7f43900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:20.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 -- 172.21.15.91:0/985854046 shutdown_connections 2024-02-02T13:26:20.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.362+0000 7f4395200640 1 -- 172.21.15.91:0/985854046 wait complete. 2024-02-02T13:26:20.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.364+0000 7f4395200640 1 Processor -- start 2024-02-02T13:26:20.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.364+0000 7f4395200640 1 -- start start 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 0x7f43900c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 0x7f43900aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439015bfb0 con 0x7f43900ce130 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4390155850 con 0x7f43900b0700 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4395200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439015b610 con 0x7f439015a010 2024-02-02T13:26:20.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55578/0 (socket says 172.21.15.91:55578) 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 -- 172.21.15.91:0/1093184910 learned_addr learned my addr 172.21.15.91:0/1093184910 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f438e800640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 0x7f43900aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f438de00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 0x7f43900c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 -- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 msgr2=0x7f43900aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 0x7f43900aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 -- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 msgr2=0x7f43900c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 0x7f43900c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f4387e00640 1 -- 172.21.15.91:0/1093184910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388009a70 con 0x7f43900ce130 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f438e800640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 0x7f43900aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.365+0000 7f438de00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 0x7f43900c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4387e00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f4378004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378039c90 con 0x7f43900ce130 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378039df0 con 0x7f43900ce130 2024-02-02T13:26:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378043640 con 0x7f43900ce130 2024-02-02T13:26:20.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780094d0 con 0x7f43900ce130 2024-02-02T13:26:20.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.366+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439015bbd0 con 0x7f43900ce130 2024-02-02T13:26:20.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.367+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439015c0f0 con 0x7f43900ce130 2024-02-02T13:26:20.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.368+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f43780437a0 con 0x7f43900ce130 2024-02-02T13:26:20.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.370+0000 7f4386000640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 0x7f436806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:20.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.370+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f437803e070 con 0x7f43900ce130 2024-02-02T13:26:20.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.370+0000 7f438de00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 0x7f436806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.371+0000 7f438de00640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 0x7f436806c8f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4380000f60 tx=0x7f4380009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:20.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.374+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f439015c0f0 con 0x7f43900ce130 2024-02-02T13:26:20.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.564+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f439015c0f0 con 0x7f43900ce130 2024-02-02T13:26:20.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.572+0000 7f4386000640 1 -- 172.21.15.91:0/1093184910 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f439015c0f0 con 0x7f43900ce130 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 msgr2=0x7f436806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 0x7f436806c8f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4380000f60 tx=0x7f4380009040 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 msgr2=0x7f43900ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4378000c00 tx=0x7f4378004730 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 shutdown_connections 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f436806a430 0x7f436806c8f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f439015a010 0x7f43900aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43900ce130 0x7f43900ae7c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 --2- 172.21.15.91:0/1093184910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43900b0700 0x7f43900c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 >> 172.21.15.91:0/1093184910 conn(0x7f43900cec80 msgr2=0x7f439006fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 shutdown_connections 2024-02-02T13:26:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:20.575+0000 7f4395200640 1 -- 172.21.15.91:0/1093184910 wait complete. 2024-02-02T13:26:20.596 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:26:20.596 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:26:20.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:26:20.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:26:20.822 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:20.822 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:20.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:26:20.875 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:20.875 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:20.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:26:20.934 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:20.934 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:20.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:26:20.990 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:20.990 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:21.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:26:21.046 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:26:21.046 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:21.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.259+0000 7f8a25000640 1 Processor -- start 2024-02-02T13:26:21.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.259+0000 7f8a25000640 1 -- start start 2024-02-02T13:26:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8a2015d490 0x7f8a2006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 0x7f8a200acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 0x7f8a200b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200ad2a0 con 0x7f8a2015d490 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200b10b0 con 0x7f8a200ac8d0 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a25000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2015d870 con 0x7f8a200b0700 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a1de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8a2015d490 0x7f8a2006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a1de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8a2015d490 0x7f8a2006c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55580/0 (socket says 172.21.15.91:55580) 2024-02-02T13:26:21.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a1de00640 1 -- 172.21.15.91:0/2961665802 learned_addr learned my addr 172.21.15.91:0/2961665802 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a17e00640 1 --2- 172.21.15.91:0/2961665802 >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 0x7f8a200acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.261+0000 7f8a1e800640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 0x7f8a200b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1de00640 1 -- 172.21.15.91:0/2961665802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 msgr2=0x7f8a200b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1de00640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 0x7f8a200b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1de00640 1 -- 172.21.15.91:0/2961665802 >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 msgr2=0x7f8a200acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1de00640 1 --2- 172.21.15.91:0/2961665802 >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 0x7f8a200acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1de00640 1 -- 172.21.15.91:0/2961665802 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2006bce0 con 0x7f8a2015d490 2024-02-02T13:26:21.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a17e00640 1 --2- 172.21.15.91:0/2961665802 >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 0x7f8a200acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.262+0000 7f8a1e800640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 0x7f8a200b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.263+0000 7f8a1de00640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a2006c170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a200b1870 tx=0x7f8a1002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c58dc7a2a83d03c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.263+0000 7f8a1d400640 1 -- 172.21.15.91:0/2961665802 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10031640 con 0x7f8a2015d490 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.263+0000 7f8a1d400640 1 -- 172.21.15.91:0/2961665802 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a100317a0 con 0x7f8a2015d490 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.263+0000 7f8a1d400640 1 -- 172.21.15.91:0/2961665802 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10031b30 con 0x7f8a2015d490 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 -- 172.21.15.91:0/2961665802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 msgr2=0x7f8a2006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a2006c170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a200b1870 tx=0x7f8a1002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 -- 172.21.15.91:0/2961665802 shutdown_connections 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200b0700 0x7f8a200b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 --2- 172.21.15.91:0/2961665802 >> v2:172.21.15.113:3300/0 conn(0x7f8a200ac8d0 0x7f8a200acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 --2- 172.21.15.91:0/2961665802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a2006c170 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 -- 172.21.15.91:0/2961665802 >> 172.21.15.91:0/2961665802 conn(0x7f8a200cec80 msgr2=0x7f8a200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 -- 172.21.15.91:0/2961665802 shutdown_connections 2024-02-02T13:26:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.264+0000 7f8a25000640 1 -- 172.21.15.91:0/2961665802 wait complete. 2024-02-02T13:26:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.266+0000 7f8a25000640 1 Processor -- start 2024-02-02T13:26:21.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.266+0000 7f8a25000640 1 -- start start 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200ac8d0 0x7f8a200b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a200b0700 0x7f8a200b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b2280 con 0x7f8a2015d490 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b30e0 con 0x7f8a200b0700 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.267+0000 7f8a25000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b24d0 con 0x7f8a200ac8d0 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55588/0 (socket says 172.21.15.91:55588) 2024-02-02T13:26:21.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 -- 172.21.15.91:0/1871177345 learned_addr learned my addr 172.21.15.91:0/1871177345 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a17e00640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a200b0700 0x7f8a200b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1de00640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200ac8d0 0x7f8a200b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 -- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200ac8d0 msgr2=0x7f8a200b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200ac8d0 0x7f8a200b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 -- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a200b0700 msgr2=0x7f8a200b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a200b0700 0x7f8a200b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 -- 172.21.15.91:0/1871177345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a100094d0 con 0x7f8a2015d490 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.268+0000 7f8a1e800640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800ccd0 tx=0x7f8a1800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.269+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800f680 con 0x7f8a2015d490 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.269+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1800f800 con 0x7f8a2015d490 2024-02-02T13:26:21.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.269+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800fbb0 con 0x7f8a2015d490 2024-02-02T13:26:21.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.269+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a200b27b0 con 0x7f8a2015d490 2024-02-02T13:26:21.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.269+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a200b2d70 con 0x7f8a2015d490 2024-02-02T13:26:21.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.270+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a20066cf0 con 0x7f8a2015d490 2024-02-02T13:26:21.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.271+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8a1800b040 con 0x7f8a2015d490 2024-02-02T13:26:21.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.272+0000 7f8a16000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 0x7f89f806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.272+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a18088fb0 con 0x7f8a2015d490 2024-02-02T13:26:21.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.272+0000 7f8a1de00640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 0x7f89f806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.273+0000 7f8a1de00640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 0x7f89f806c920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a10036000 tx=0x7f8a1003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.277+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8a18054800 con 0x7f8a2015d490 2024-02-02T13:26:21.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.510+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a200b3220 con 0x7f8a2015d490 2024-02-02T13:26:21.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.511+0000 7f8a16000640 1 -- 172.21.15.91:0/1871177345 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8a1801d020 con 0x7f8a2015d490 2024-02-02T13:26:21.512 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:21.512 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1749173445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1749173445},{"type":"v1","addr":"172.21.15.91:6836","nonce":1749173445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3136354831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3136354831},{"type":"v1","addr":"172.21.15.91:6837","nonce":3136354831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1185410421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1185410421},{"type":"v1","addr":"172.21.15.113:6835","nonce":1185410421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":4690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/108378410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":108378410},{"type":"v1","addr":"172.21.15.113:6834","nonce":108378410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-02-02T13:26:21.513 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 18 2024-02-02T13:26:21.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 msgr2=0x7f89f806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 0x7f89f806c920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a10036000 tx=0x7f8a1003c040 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 msgr2=0x7f8a201f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800ccd0 tx=0x7f8a1800a650 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 shutdown_connections 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f89f806a460 0x7f89f806c920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a2015d490 0x7f8a201f2470 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a200b0700 0x7f8a200b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 --2- 172.21.15.91:0/1871177345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a200ac8d0 0x7f8a200b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 >> 172.21.15.91:0/1871177345 conn(0x7f8a200cec80 msgr2=0x7f8a201579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 shutdown_connections 2024-02-02T13:26:21.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.514+0000 7f8a25000640 1 -- 172.21.15.91:0/1871177345 wait complete. 2024-02-02T13:26:21.535 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:26:21.535 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:26:21.535 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:26:21.535 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:26:21.536 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:26:21.536 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:26:21.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:26:21.536 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:26:21.536 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:26:21.536 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:26:21.536 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:26:21.536 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:26:21.537 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:26:21.537 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:26:21.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:26:21.537 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:26:21.554 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:26:21.554 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:26:21.555 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:26:21.555 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:26:21.558 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:26:21.558 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:26:21.562 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:26:21.562 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:26:21.602 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:26:21.602 DEBUG:teuthology.parallel:result is None 2024-02-02T13:26:21.608 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:26:21.608 DEBUG:teuthology.parallel:result is None 2024-02-02T13:26:21.609 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:26:21.609 DEBUG:teuthology.parallel:result is None 2024-02-02T13:26:21.615 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:26:21.615 DEBUG:teuthology.parallel:result is None 2024-02-02T13:26:21.615 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:26:21.753 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:26:21.757 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:26:21.776 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:26:21.777 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:26:21.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.800+0000 7fcbdbc00640 1 Processor -- start 2024-02-02T13:26:21.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.800+0000 7fcbdbc00640 1 -- start start 2024-02-02T13:26:21.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.801+0000 7fcbdbc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcbd4070270 0x7fcbd40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.801+0000 7fcbdbc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbd40b6bd0 0x7fcbd40b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.801+0000 7fcbdbc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd4160d90 con 0x7fcbd41609b0 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.801+0000 7fcbdbc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd4070bf0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.801+0000 7fcbdbc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd4070fe0 con 0x7fcbd4070270 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbd40b6bd0 0x7fcbd40b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbd40b6bd0 0x7fcbd40b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55622/0 (socket says 172.21.15.91:55622) 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 -- 172.21.15.91:0/3163480176 learned_addr learned my addr 172.21.15.91:0/3163480176 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd3e00640 1 --2- 172.21.15.91:0/3163480176 >> v2:172.21.15.113:3300/0 conn(0x7fcbd4070270 0x7fcbd40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 -- 172.21.15.91:0/3163480176 >> v2:172.21.15.113:3300/0 conn(0x7fcbd4070270 msgr2=0x7fcbd40706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 --2- 172.21.15.91:0/3163480176 >> v2:172.21.15.113:3300/0 conn(0x7fcbd4070270 0x7fcbd40706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 -- 172.21.15.91:0/3163480176 >> v1:172.21.15.113:6789/0 conn(0x7fcbd41609b0 legacy=0x7fcbd40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.802+0000 7fcbd9c00640 1 -- 172.21.15.91:0/3163480176 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40d2940 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.803+0000 7fcbd9c00640 1 --2- 172.21.15.91:0/3163480176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40b7020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4067830 tx=0x7fcbcc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361477bf1b3214fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.803+0000 7fcbd3400640 1 -- 172.21.15.91:0/3163480176 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc02c9f0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbd3400640 1 -- 172.21.15.91:0/3163480176 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc02cb50 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbd3400640 1 -- 172.21.15.91:0/3163480176 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0343f0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 -- 172.21.15.91:0/3163480176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 msgr2=0x7fcbd40b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/3163480176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40b7020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4067830 tx=0x7fcbcc02be50 comp rx=0 tx=0).stop 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 -- 172.21.15.91:0/3163480176 shutdown_connections 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/3163480176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40b7020 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/3163480176 >> v2:172.21.15.113:3300/0 conn(0x7fcbd4070270 0x7fcbd40706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 -- 172.21.15.91:0/3163480176 >> 172.21.15.91:0/3163480176 conn(0x7fcbd40d3020 msgr2=0x7fcbd40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:21.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 -- 172.21.15.91:0/3163480176 shutdown_connections 2024-02-02T13:26:21.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.804+0000 7fcbdbc00640 1 -- 172.21.15.91:0/3163480176 wait complete. 2024-02-02T13:26:21.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.806+0000 7fcbdbc00640 1 Processor -- start 2024-02-02T13:26:21.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.806+0000 7fcbdbc00640 1 -- start start 2024-02-02T13:26:21.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd4070270 0x7fcbd4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbd41609b0 0x7fcbd40d2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40ab270 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cd0b0 con 0x7fcbd4070270 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbdbc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cc2f0 con 0x7fcbd41609b0 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55630/0 (socket says 172.21.15.91:55630) 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.807+0000 7fcbd3e00640 1 -- 172.21.15.91:0/762019903 learned_addr learned my addr 172.21.15.91:0/762019903 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd9c00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbd41609b0 0x7fcbd40d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd9200640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd4070270 0x7fcbd4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 -- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbd41609b0 msgr2=0x7fcbd40d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbd41609b0 0x7fcbd40d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 -- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd4070270 msgr2=0x7fcbd4150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd4070270 0x7fcbd4150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:21.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 -- 172.21.15.91:0/762019903 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc009b30 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd3e00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e9b0 tx=0x7fcbbc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc009670 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc00c9a0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.808+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00cd50 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.809+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40cc5d0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.809+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40ccb90 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.810+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd4066cf0 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.811+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcbbc013470 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.812+0000 7fcbd0c00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 0x7fcbac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:21.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.812+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcbbc088870 con 0x7fcbd40b6bd0 2024-02-02T13:26:21.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.812+0000 7fcbd9200640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 0x7fcbac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:21.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.813+0000 7fcbd9200640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 0x7fcbac06c920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4000f60 tx=0x7fcbc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:21.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:21.816+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcbbc054170 con 0x7fcbd40b6bd0 2024-02-02T13:26:22.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.018+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcbd40b7fb0 con 0x7fcbd40b6bd0 2024-02-02T13:26:22.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.917+0000 7fcbd0c00640 1 -- 172.21.15.91:0/762019903 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcbbc057790 con 0x7fcbd40b6bd0 2024-02-02T13:26:22.919 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:26:22.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.921+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 msgr2=0x7fcbac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:22.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 0x7fcbac06c920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4000f60 tx=0x7fcbc4009040 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 msgr2=0x7fcbd40d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e9b0 tx=0x7fcbbc00ee70 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 shutdown_connections 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcbac06a460 0x7fcbac06c920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbd41609b0 0x7fcbd40d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd40b6bd0 0x7fcbd40d1b30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 --2- 172.21.15.91:0/762019903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd4070270 0x7fcbd4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 >> 172.21.15.91:0/762019903 conn(0x7fcbd40d3020 msgr2=0x7fcbd40cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 shutdown_connections 2024-02-02T13:26:22.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:22.922+0000 7fcbdbc00640 1 -- 172.21.15.91:0/762019903 wait complete. 2024-02-02T13:26:22.943 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:26:23.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.128+0000 7f37a4c00640 1 Processor -- start 2024-02-02T13:26:23.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.128+0000 7f37a4c00640 1 -- start start 2024-02-02T13:26:23.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.129+0000 7f37a4c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f37a015dfb0 0x7f37a00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.129+0000 7f37a4c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a006faa0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.129+0000 7f37a4c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00ca420 con 0x7f37a00c9a80 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.129+0000 7f37a4c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a015e800 con 0x7f37a015dfb0 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f37a00ce4d0 0x7f37a00707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42514/0 (socket says 172.21.15.91:42514) 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379f400640 1 -- 172.21.15.91:0/2334263073 learned_addr learned my addr 172.21.15.91:0/2334263073 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379fe00640 1 --2- 172.21.15.91:0/2334263073 >> v2:172.21.15.91:3300/0 conn(0x7f37a015dfb0 0x7f37a00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379ea00640 1 -- 172.21.15.91:0/2334263073 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672907203 0 0) 0x7f37a006faa0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379ea00640 1 -- 172.21.15.91:0/2334263073 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3780003520 con 0x7f37a00ce4d0 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379fe00640 1 -- 172.21.15.91:0/2334263073 >> v1:172.21.15.113:6789/0 conn(0x7f37a00c9a80 legacy=0x7f37a015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379fe00640 1 -- 172.21.15.91:0/2334263073 >> v1:172.21.15.91:6789/0 conn(0x7f37a00ce4d0 legacy=0x7f37a00707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.130+0000 7f379fe00640 1 -- 172.21.15.91:0/2334263073 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0070320 con 0x7f37a015dfb0 2024-02-02T13:26:23.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.131+0000 7f379fe00640 1 --2- 172.21.15.91:0/2334263073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a015dfb0 0x7f37a00c9ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37980097a0 tx=0x7f379802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7334c30f4d48665 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f379ea00640 1 -- 172.21.15.91:0/2334263073 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798031830 con 0x7f37a015dfb0 2024-02-02T13:26:23.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f379ea00640 1 -- 172.21.15.91:0/2334263073 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798031990 con 0x7f37a015dfb0 2024-02-02T13:26:23.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f379ea00640 1 -- 172.21.15.91:0/2334263073 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798031d20 con 0x7f37a015dfb0 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 -- 172.21.15.91:0/2334263073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a015dfb0 msgr2=0x7f37a00c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 --2- 172.21.15.91:0/2334263073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a015dfb0 0x7f37a00c9ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37980097a0 tx=0x7f379802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 -- 172.21.15.91:0/2334263073 shutdown_connections 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 --2- 172.21.15.91:0/2334263073 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a015dfb0 0x7f37a00c9ee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 -- 172.21.15.91:0/2334263073 >> 172.21.15.91:0/2334263073 conn(0x7f37a00cb4a0 msgr2=0x7f37a00cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 -- 172.21.15.91:0/2334263073 shutdown_connections 2024-02-02T13:26:23.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.132+0000 7f37a4c00640 1 -- 172.21.15.91:0/2334263073 wait complete. 2024-02-02T13:26:23.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.134+0000 7f37a4c00640 1 Processor -- start 2024-02-02T13:26:23.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.134+0000 7f37a4c00640 1 -- start start 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a00c9a80 0x7f37a0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a015dfb0 0x7f37a00ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0070ec0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00b3f00 con 0x7f37a015dfb0 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f37a4c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00b3070 con 0x7f37a00c9a80 2024-02-02T13:26:23.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f3796800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f3796800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55642/0 (socket says 172.21.15.91:55642) 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.135+0000 7f3796800640 1 -- 172.21.15.91:0/1779193285 learned_addr learned my addr 172.21.15.91:0/1779193285 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f379f400640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a00c9a80 0x7f37a0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 -- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a00c9a80 msgr2=0x7f37a0155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a00c9a80 0x7f37a0155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 -- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a015dfb0 msgr2=0x7f37a00ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a015dfb0 0x7f37a00ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 -- 172.21.15.91:0/1779193285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009400 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3796800640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f378800b420 tx=0x7f378800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011830 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788011990 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011d40 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.136+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a00b32f0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.137+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a00b38b0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.137+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00b40b0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.138+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f378801b460 con 0x7f37a00ce4d0 2024-02-02T13:26:23.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.139+0000 7f3797e00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 0x7f377806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.139+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3788088140 con 0x7f37a00ce4d0 2024-02-02T13:26:23.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.139+0000 7f379f400640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 0x7f377806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.140+0000 7f379f400640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 0x7f377806c8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3790005c00 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.144+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f37a00b40b0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.398+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f37a00b40b0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.399+0000 7f3797e00640 1 -- 172.21.15.91:0/1779193285 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f37a00b40b0 con 0x7f37a00ce4d0 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.401 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.402 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.403 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.404 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:26:23.405 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:26:23.406 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.407 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.408 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:26:23.410 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.411 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:26:23.412 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:23.414 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:23.414 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:23.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 msgr2=0x7f377806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 0x7f377806c8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3790005c00 tx=0x7f3790009040 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 msgr2=0x7f37a0156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f378800b420 tx=0x7f378800b8e0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 shutdown_connections 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f377806a430 0x7f377806c8f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a015dfb0 0x7f37a00ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37a00ce4d0 0x7f37a0156170 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 --2- 172.21.15.91:0/1779193285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a00c9a80 0x7f37a0155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.401+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 >> 172.21.15.91:0/1779193285 conn(0x7f37a00cb4a0 msgr2=0x7f37a006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.402+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 shutdown_connections 2024-02-02T13:26:23.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.402+0000 7f37a4c00640 1 -- 172.21.15.91:0/1779193285 wait complete. 2024-02-02T13:26:23.421 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:23.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.620+0000 7f6357e00640 1 Processor -- start 2024-02-02T13:26:23.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.620+0000 7f6357e00640 1 -- start start 2024-02-02T13:26:23.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63580cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6357e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635806c1c0 con 0x7f635815aec0 2024-02-02T13:26:23.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6357e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635806b5f0 con 0x7f635806f3f0 2024-02-02T13:26:23.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635806cd60 con 0x7f635815b2a0 2024-02-02T13:26:23.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6356a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f635815aec0 0x7f635806c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42528/0 (socket says 172.21.15.91:42528) 2024-02-02T13:26:23.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6356a00640 1 -- 172.21.15.91:0/898695798 learned_addr learned my addr 172.21.15.91:0/898695798 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965865308 0 0) 0x7f635806c1c0 con 0x7f635815aec0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.622+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6338003520 con 0x7f635815aec0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f634de00640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63580cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208537827 0 0) 0x7f6338003520 con 0x7f635815aec0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 msgr2=0x7f63580cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63580cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 >> v1:172.21.15.113:6789/0 conn(0x7f635806f3f0 legacy=0x7f63580ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635806b890 con 0x7f635815aec0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6348001670 con 0x7f635815aec0 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 >> v1:172.21.15.91:6789/0 conn(0x7f635815aec0 legacy=0x7f635806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6338004000 0x7f63380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6338004d00 0x7f6338005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635806cd60 con 0x7f6338004d00 2024-02-02T13:26:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bc00 con 0x7f6338004000 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bdc0 con 0x7f635815b2a0 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.623+0000 7f6357400640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6338004d00 0x7f6338005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f634de00640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6338004000 0x7f63380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6356a00640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6357400640 1 -- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 msgr2=0x7f63380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6357400640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f635815b2a0 0x7f63380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6357400640 1 -- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6338004000 msgr2=0x7f63380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6357400640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6338004000 0x7f63380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f6357400640 1 -- 172.21.15.91:0/898695798 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338003520 con 0x7f6338004d00 2024-02-02T13:26:23.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.624+0000 7f634de00640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6338004000 0x7f63380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:23.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6357400640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6338004d00 0x7f6338005180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f63500044b0 tx=0x7f63380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f2d8fd3d3ee78b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350030620 con 0x7f6338004d00 2024-02-02T13:26:23.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350030780 con 0x7f6338004d00 2024-02-02T13:26:23.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6356000640 1 -- 172.21.15.91:0/898695798 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350030b70 con 0x7f6338004d00 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6357e00640 1 -- 172.21.15.91:0/898695798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6338004d00 msgr2=0x7f6338005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.625+0000 7f6357e00640 1 --2- 172.21.15.91:0/898695798 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6338004d00 0x7f6338005180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f63500044b0 tx=0x7f63380057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.626+0000 7f6357400640 1 -- 172.21.15.91:0/898695798 reap_dead start 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.626+0000 7f6357e00640 1 -- 172.21.15.91:0/898695798 shutdown_connections 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.626+0000 7f6357e00640 1 -- 172.21.15.91:0/898695798 >> 172.21.15.91:0/898695798 conn(0x7f63580cea90 msgr2=0x7f63580ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.626+0000 7f6357e00640 1 -- 172.21.15.91:0/898695798 shutdown_connections 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.626+0000 7f6357e00640 1 -- 172.21.15.91:0/898695798 wait complete. 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.627+0000 7f6357e00640 1 Processor -- start 2024-02-02T13:26:23.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.627+0000 7f6357e00640 1 -- start start 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6338004d00 0x7f63580aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159960 0x7f63580ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580ab380 con 0x7f63580acbe0 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580a9da0 con 0x7f6358159960 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.628+0000 7f6357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63580aa080 con 0x7f6338004d00 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55654/0 (socket says 172.21.15.91:55654) 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 -- 172.21.15.91:0/3059386782 learned_addr learned my addr 172.21.15.91:0/3059386782 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 -- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6338004d00 msgr2=0x7f63580aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6338004d00 0x7f63580aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 -- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159960 msgr2=0x7f63580ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f634de00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159960 0x7f63580ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159960 0x7f63580ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 -- 172.21.15.91:0/3059386782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580aa360 con 0x7f63580acbe0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.629+0000 7f6357400640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f63500045d0 tx=0x7f6350003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.630+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635003f070 con 0x7f63580acbe0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.630+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350006e00 con 0x7f63580acbe0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.630+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63580aa640 con 0x7f63580acbe0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.630+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350030b00 con 0x7f63580acbe0 2024-02-02T13:26:23.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.630+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63580a9a50 con 0x7f63580acbe0 2024-02-02T13:26:23.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.631+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635804dff0 con 0x7f63580acbe0 2024-02-02T13:26:23.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.632+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6350030c60 con 0x7f63580acbe0 2024-02-02T13:26:23.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.633+0000 7f634f400640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 0x7f633006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:23.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.633+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63500a9740 con 0x7f63580acbe0 2024-02-02T13:26:23.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.633+0000 7f6356a00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 0x7f633006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:23.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.634+0000 7f6356a00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 0x7f633006c8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6348001be0 tx=0x7f6348000f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:23.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.637+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6350075090 con 0x7f63580acbe0 2024-02-02T13:26:23.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.826+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63580ae600 con 0x7f63580acbe0 2024-02-02T13:26:23.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.829+0000 7f634f400640 1 -- 172.21.15.91:0/3059386782 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f6350049090 con 0x7f63580acbe0 2024-02-02T13:26:23.831 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:23.831 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":44,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:22.910150+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:23.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.831+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 msgr2=0x7f633006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.831+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 0x7f633006c8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6348001be0 tx=0x7f6348000f10 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 msgr2=0x7f63581e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f63500045d0 tx=0x7f6350003a70 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 shutdown_connections 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f633006a430 0x7f633006c8f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f63580acbe0 0x7f63581e5a50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159960 0x7f63580ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 --2- 172.21.15.91:0/3059386782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6338004d00 0x7f63580aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 >> 172.21.15.91:0/3059386782 conn(0x7f63580cea90 msgr2=0x7f635815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 shutdown_connections 2024-02-02T13:26:23.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:23.832+0000 7f6357e00640 1 -- 172.21.15.91:0/3059386782 wait complete. 2024-02-02T13:26:23.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:26:23.852 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:26:24.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.060+0000 7fdec3800640 1 Processor -- start 2024-02-02T13:26:24.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.061+0000 7fdec3800640 1 -- start start 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdec3800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdebc15d5e0 0x7fdebc0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdec3800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc0d1280 con 0x7fdebc0ce070 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdec3800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc161ed0 con 0x7fdebc0d2720 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdec3800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc15d9c0 con 0x7fdebc15d5e0 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdebbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdebc0ce070 0x7fdebc1610b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42536/0 (socket says 172.21.15.91:42536) 2024-02-02T13:26:24.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.062+0000 7fdebbe00640 1 -- 172.21.15.91:0/1410243623 learned_addr learned my addr 172.21.15.91:0/1410243623 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1410243623 >> v2:172.21.15.113:3300/0 conn(0x7fdebc15d5e0 0x7fdebc0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665026133 0 0) 0x7fdebc0d1280 con 0x7fdebc0ce070 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9c003520 con 0x7fdebc0ce070 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242604484 0 0) 0x7fde9c003520 con 0x7fdebc0ce070 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 >> v2:172.21.15.113:3300/0 conn(0x7fdebc15d5e0 msgr2=0x7fdebc0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 --2- 172.21.15.91:0/1410243623 >> v2:172.21.15.113:3300/0 conn(0x7fdebc15d5e0 0x7fdebc0cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 >> v1:172.21.15.113:6789/0 conn(0x7fdebc0d2720 legacy=0x7fdebc1617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0d0c70 con 0x7fdebc0ce070 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fdeac0019f0 con 0x7fdebc0ce070 2024-02-02T13:26:24.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 >> v1:172.21.15.91:6789/0 conn(0x7fdebc0ce070 legacy=0x7fdebc1610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde9c0039c0 0x7fde9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c0047c0 0x7fde9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.063+0000 7fdebb400640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 0x7fde9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15d9c0 con 0x7fde9c005180 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c00c080 con 0x7fde9c0039c0 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c00c240 con 0x7fde9c0047c0 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 0x7fde9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdebbe00640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c0047c0 0x7fde9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde9c0039c0 0x7fde9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 -- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c0047c0 msgr2=0x7fde9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c0047c0 0x7fde9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 -- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde9c0039c0 msgr2=0x7fde9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde9c0039c0 0x7fde9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec1800640 1 -- 172.21.15.91:0/1410243623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c003520 con 0x7fde9c005180 2024-02-02T13:26:24.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.064+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde9c0039c0 0x7fde9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.065+0000 7fdec1800640 1 -- 172.21.15.91:0/1410243623 reap_dead start 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.065+0000 7fdec1800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 0x7fde9c00b980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4001720 tx=0x7fde9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0c66fc1249be76 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40052d0 con 0x7fde9c005180 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb402e050 con 0x7fde9c005180 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdebb400640 1 -- 172.21.15.91:0/1410243623 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40035f0 con 0x7fde9c005180 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 -- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 msgr2=0x7fde9c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 0x7fde9c00b980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4001720 tx=0x7fde9c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 -- 172.21.15.91:0/1410243623 shutdown_connections 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 --2- 172.21.15.91:0/1410243623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde9c005180 0x7fde9c00b980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 -- 172.21.15.91:0/1410243623 >> 172.21.15.91:0/1410243623 conn(0x7fdebc0d30a0 msgr2=0x7fdebc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:24.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 -- 172.21.15.91:0/1410243623 shutdown_connections 2024-02-02T13:26:24.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.066+0000 7fdec3800640 1 -- 172.21.15.91:0/1410243623 wait complete. 2024-02-02T13:26:24.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.068+0000 7fdec3800640 1 Processor -- start 2024-02-02T13:26:24.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.068+0000 7fdec3800640 1 -- start start 2024-02-02T13:26:24.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 0x7fdebc0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdebc0cc1f0 0x7fdebc1f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0cc6a0 con 0x7fdebc160b40 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0cb460 con 0x7fdebc0cc1f0 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdec3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15dba0 con 0x7fde9c005180 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.069+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55660/0 (socket says 172.21.15.91:55660) 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 -- 172.21.15.91:0/1185263791 learned_addr learned my addr 172.21.15.91:0/1185263791 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 0x7fdebc0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdec1800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdebc0cc1f0 0x7fdebc1f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 -- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 msgr2=0x7fdebc0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 0x7fdebc0cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 -- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdebc0cc1f0 msgr2=0x7fdebc1f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdebc0cc1f0 0x7fdebc1f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 -- 172.21.15.91:0/1185263791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc15de80 con 0x7fdebc160b40 2024-02-02T13:26:24.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 0x7fdebc0cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:24.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.070+0000 7fdebbe00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007a10 tx=0x7fdeac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.071+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0019d0 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.071+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00ee50 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.071+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac013660 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.071+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc15e160 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.071+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc0cb0c0 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.072+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc071a30 con 0x7fdebc160b40 2024-02-02T13:26:24.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.073+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdeac0137c0 con 0x7fdebc160b40 2024-02-02T13:26:24.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.073+0000 7fdeb8c00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 0x7fde9406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.074+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdeac088c50 con 0x7fdebc160b40 2024-02-02T13:26:24.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.074+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 0x7fde9406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.074+0000 7fdec0e00640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 0x7fde9406c920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009aa0 tx=0x7fdea4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.078+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdeac054580 con 0x7fdebc160b40 2024-02-02T13:26:24.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.253+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdebc15e680 con 0x7fdebc160b40 2024-02-02T13:26:24.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.254+0000 7fdeb8c00640 1 -- 172.21.15.91:0/1185263791 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdebc15e680 con 0x7fdebc160b40 2024-02-02T13:26:24.256 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:26:24.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 msgr2=0x7fde9406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 0x7fde9406c920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009aa0 tx=0x7fdea4008040 comp rx=0 tx=0).stop 2024-02-02T13:26:24.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 msgr2=0x7fdebc0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007a10 tx=0x7fdeac00e530 comp rx=0 tx=0).stop 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 shutdown_connections 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde9406a460 0x7fde9406c920 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdebc0cc1f0 0x7fdebc1f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdebc160b40 0x7fdebc0cbcb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 --2- 172.21.15.91:0/1185263791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde9c005180 0x7fdebc0cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 >> 172.21.15.91:0/1185263791 conn(0x7fdebc0d30a0 msgr2=0x7fdebc0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:24.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 shutdown_connections 2024-02-02T13:26:24.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.257+0000 7fdec3800640 1 -- 172.21.15.91:0/1185263791 wait complete. 2024-02-02T13:26:24.278 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:26:24.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.478+0000 7fe715400640 1 Processor -- start 2024-02-02T13:26:24.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.478+0000 7fe715400640 1 -- start start 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe715400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7100cbc90 0x7fe71015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe715400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100ca0c0 con 0x7fe710159de0 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe715400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100ca380 con 0x7fe71015e170 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe715400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100ca660 con 0x7fe7100cbc90 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe70e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe710159de0 0x7fe7100cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42538/0 (socket says 172.21.15.91:42538) 2024-02-02T13:26:24.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.479+0000 7fe70e800640 1 -- 172.21.15.91:0/1166666922 learned_addr learned my addr 172.21.15.91:0/1166666922 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489485068 0 0) 0x7fe7100ca0c0 con 0x7fe710159de0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> v2:172.21.15.113:3300/0 conn(0x7fe7100cbc90 0x7fe71015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f0003520 con 0x7fe710159de0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961159308 0 0) 0x7fe6f0003520 con 0x7fe710159de0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 >> v2:172.21.15.113:3300/0 conn(0x7fe7100cbc90 msgr2=0x7fe71015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 --2- 172.21.15.91:0/1166666922 >> v2:172.21.15.113:3300/0 conn(0x7fe7100cbc90 0x7fe71015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 >> v1:172.21.15.113:6789/0 conn(0x7fe71015e170 legacy=0x7fe71006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100ca940 con 0x7fe710159de0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe700001670 con 0x7fe710159de0 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> v2:172.21.15.113:3300/0 conn(0x7fe7100cbc90 0x7fe71015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 >> v1:172.21.15.91:6789/0 conn(0x7fe710159de0 legacy=0x7fe7100cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6f0003970 0x7fe6f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 0x7fe6f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.480+0000 7fe70de00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f0005180 0x7fe6f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100ca660 con 0x7fe6f00047c0 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f000c0d0 con 0x7fe6f0003970 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f000e4a0 con 0x7fe6f0005180 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 0x7fe6f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe70e800640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6f0003970 0x7fe6f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 -- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f0005180 msgr2=0x7fe6f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f0005180 0x7fe6f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 -- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6f0003970 msgr2=0x7fe6f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6f0003970 0x7fe6f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe705c00640 1 -- 172.21.15.91:0/1166666922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0003520 con 0x7fe6f00047c0 2024-02-02T13:26:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.481+0000 7fe70f200640 1 -- 172.21.15.91:0/1166666922 reap_dead start 2024-02-02T13:26:24.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.482+0000 7fe705c00640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 0x7fe6f0004c40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098a0 tx=0x7fe6f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4311f4331327fb23 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.482+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8004230 con 0x7fe6f00047c0 2024-02-02T13:26:24.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.482+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8004390 con 0x7fe6f00047c0 2024-02-02T13:26:24.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.482+0000 7fe70de00640 1 -- 172.21.15.91:0/1166666922 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003830 con 0x7fe6f00047c0 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 -- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 msgr2=0x7fe6f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 0x7fe6f0004c40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098a0 tx=0x7fe6f00057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 -- 172.21.15.91:0/1166666922 shutdown_connections 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 --2- 172.21.15.91:0/1166666922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6f00047c0 0x7fe6f0004c40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 -- 172.21.15.91:0/1166666922 >> 172.21.15.91:0/1166666922 conn(0x7fe7100b0700 msgr2=0x7fe7100b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 -- 172.21.15.91:0/1166666922 shutdown_connections 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.483+0000 7fe715400640 1 -- 172.21.15.91:0/1166666922 wait complete. 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.485+0000 7fe715400640 1 Processor -- start 2024-02-02T13:26:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.485+0000 7fe715400640 1 -- start start 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f00047c0 0x7fe7100c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe71014cc60 0x7fe71014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100c3a20 con 0x7fe71014ed80 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71014d620 con 0x7fe71014cc60 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe715400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71014f1e0 con 0x7fe6f00047c0 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe705c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe705c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55684/0 (socket says 172.21.15.91:55684) 2024-02-02T13:26:24.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.486+0000 7fe705c00640 1 -- 172.21.15.91:0/4205773220 learned_addr learned my addr 172.21.15.91:0/4205773220 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe70e800640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f00047c0 0x7fe7100c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe70f200640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe71014cc60 0x7fe71014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 -- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f00047c0 msgr2=0x7fe7100c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f00047c0 0x7fe7100c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 -- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe71014cc60 msgr2=0x7fe71014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe71014cc60 0x7fe71014d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 -- 172.21.15.91:0/4205773220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100c2da0 con 0x7fe71014ed80 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe705c00640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098a0 tx=0x7fe6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8030610 con 0x7fe71014ed80 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8030770 con 0x7fe71014ed80 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.487+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8030ae0 con 0x7fe71014ed80 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.488+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100c3080 con 0x7fe71014ed80 2024-02-02T13:26:24.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.488+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7100c3640 con 0x7fe71014ed80 2024-02-02T13:26:24.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.489+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7100c37d0 con 0x7fe71014ed80 2024-02-02T13:26:24.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.490+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe6f8030c40 con 0x7fe71014ed80 2024-02-02T13:26:24.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.490+0000 7fe707400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 0x7fe6e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.491+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe6f80aa280 con 0x7fe71014ed80 2024-02-02T13:26:24.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.491+0000 7fe70e800640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 0x7fe6e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.491+0000 7fe70e800640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 0x7fe6e806c920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe700001eb0 tx=0x7fe700001dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.495+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6f8075b40 con 0x7fe71014ed80 2024-02-02T13:26:24.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.685+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe7100b1e90 con 0x7fe71014ed80 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.686+0000 7fe707400640 1 -- 172.21.15.91:0/4205773220 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe6f8048090 con 0x7fe71014ed80 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:24.688 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.688+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 msgr2=0x7fe6e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.688+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 0x7fe6e806c920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe700001eb0 tx=0x7fe700001dc0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.688+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 msgr2=0x7fe710158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.688+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098a0 tx=0x7fe6f8004730 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 shutdown_connections 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe6e806a460 0x7fe6e806c920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe71014cc60 0x7fe71014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe71014ed80 0x7fe710158690 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 --2- 172.21.15.91:0/4205773220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6f00047c0 0x7fe7100c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 >> 172.21.15.91:0/4205773220 conn(0x7fe7100b0700 msgr2=0x7fe71006bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 shutdown_connections 2024-02-02T13:26:24.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.689+0000 7fe715400640 1 -- 172.21.15.91:0/4205773220 wait complete. 2024-02-02T13:26:24.708 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:26:24.708 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:26:24.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.922+0000 7fafd8800640 1 Processor -- start 2024-02-02T13:26:24.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.922+0000 7fafd8800640 1 -- start start 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.923+0000 7fafd8800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 0x7fafd00cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.923+0000 7fafd8800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafd00d2760 0x7fafd00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.923+0000 7fafd8800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd00d12c0 con 0x7fafd00ce070 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.923+0000 7fafd8800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00cea30 con 0x7fafd00d2760 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.923+0000 7fafd8800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00cded0 con 0x7fafd015d650 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd5400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fafd00ce070 0x7fafd01610b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42552/0 (socket says 172.21.15.91:42552) 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd5400640 1 -- 172.21.15.91:0/1609714124 learned_addr learned my addr 172.21.15.91:0/1609714124 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd5e00640 1 --2- 172.21.15.91:0/1609714124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 0x7fafd00cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd6800640 1 --2- 172.21.15.91:0/1609714124 >> v2:172.21.15.113:3300/0 conn(0x7fafd00d2760 0x7fafd00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd4a00640 1 -- 172.21.15.91:0/1609714124 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413955036 0 0) 0x7fafd00d12c0 con 0x7fafd00ce070 2024-02-02T13:26:24.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.924+0000 7fafd4a00640 1 -- 172.21.15.91:0/1609714124 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb0003520 con 0x7fafd00ce070 2024-02-02T13:26:24.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.925+0000 7fafd5e00640 1 -- 172.21.15.91:0/1609714124 >> v2:172.21.15.113:3300/0 conn(0x7fafd00d2760 msgr2=0x7fafd00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:24.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.925+0000 7fafd5e00640 1 --2- 172.21.15.91:0/1609714124 >> v2:172.21.15.113:3300/0 conn(0x7fafd00d2760 0x7fafd00ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.925+0000 7fafd5e00640 1 -- 172.21.15.91:0/1609714124 >> v1:172.21.15.91:6789/0 conn(0x7fafd00ce070 legacy=0x7fafd01610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.925+0000 7fafd5e00640 1 -- 172.21.15.91:0/1609714124 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0071410 con 0x7fafd015d650 2024-02-02T13:26:24.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.926+0000 7fafd5e00640 1 --2- 172.21.15.91:0/1609714124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 0x7fafd00cd990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb139ee0fe1322c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.929+0000 7fafd4a00640 1 -- 172.21.15.91:0/1609714124 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc002cc50 con 0x7fafd015d650 2024-02-02T13:26:24.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd4a00640 1 -- 172.21.15.91:0/1609714124 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc002cdb0 con 0x7fafd015d650 2024-02-02T13:26:24.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd4a00640 1 -- 172.21.15.91:0/1609714124 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0034660 con 0x7fafd015d650 2024-02-02T13:26:24.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 -- 172.21.15.91:0/1609714124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 msgr2=0x7fafd00cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 --2- 172.21.15.91:0/1609714124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 0x7fafd00cd990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc002bea0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 -- 172.21.15.91:0/1609714124 shutdown_connections 2024-02-02T13:26:24.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 --2- 172.21.15.91:0/1609714124 >> v2:172.21.15.113:3300/0 conn(0x7fafd00d2760 0x7fafd00ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 --2- 172.21.15.91:0/1609714124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd015d650 0x7fafd00cd990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.930+0000 7fafd8800640 1 -- 172.21.15.91:0/1609714124 >> 172.21.15.91:0/1609714124 conn(0x7fafd00d30a0 msgr2=0x7fafd00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:24.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.931+0000 7fafd8800640 1 -- 172.21.15.91:0/1609714124 shutdown_connections 2024-02-02T13:26:24.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.931+0000 7fafd8800640 1 -- 172.21.15.91:0/1609714124 wait complete. 2024-02-02T13:26:24.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.932+0000 7fafd8800640 1 Processor -- start 2024-02-02T13:26:24.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.932+0000 7fafd8800640 1 -- start start 2024-02-02T13:26:24.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd00d2760 0x7fafd00b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd015d650 0x7fafd00b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd014b090 con 0x7fafd00ce070 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b3e00 con 0x7fafd015d650 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.933+0000 7fafd8800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b40e0 con 0x7fafd00d2760 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55688/0 (socket says 172.21.15.91:55688) 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 -- 172.21.15.91:0/3808097827 learned_addr learned my addr 172.21.15.91:0/3808097827 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5400640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd00d2760 0x7fafd00b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd6800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd015d650 0x7fafd00b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 -- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd00d2760 msgr2=0x7fafd00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd00d2760 0x7fafd00b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 -- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd015d650 msgr2=0x7fafd00b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:24.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd015d650 0x7fafd00b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:24.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.934+0000 7fafd5e00640 1 -- 172.21.15.91:0/3808097827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009a70 con 0x7fafd00ce070 2024-02-02T13:26:24.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafd5e00640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0040650 con 0x7fafd00ce070 2024-02-02T13:26:24.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc00407b0 con 0x7fafd00ce070 2024-02-02T13:26:24.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0040b60 con 0x7fafd00ce070 2024-02-02T13:26:24.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd00ca170 con 0x7fafd00ce070 2024-02-02T13:26:24.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.935+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd00ca730 con 0x7fafd00ce070 2024-02-02T13:26:24.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.936+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd0071a30 con 0x7fafd00ce070 2024-02-02T13:26:24.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.937+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fafc0040cc0 con 0x7fafd00ce070 2024-02-02T13:26:24.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.938+0000 7fafce000640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 0x7fafa806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:24.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.938+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fafc00ab320 con 0x7fafd00ce070 2024-02-02T13:26:24.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.938+0000 7fafd5400640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 0x7fafa806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:24.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.939+0000 7fafd5400640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 0x7fafa806c8f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fafd00cd5a0 tx=0x7fafb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:24.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:24.943+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fafc0076d00 con 0x7fafd00ce070 2024-02-02T13:26:25.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.132+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fafd0066cf0 con 0x7fafd00ce070 2024-02-02T13:26:25.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.951+0000 7fafce000640 1 -- 172.21.15.91:0/3808097827 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7fafc0009450 con 0x7fafd00ce070 2024-02-02T13:26:25.954 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:26:25.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 msgr2=0x7fafa806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:25.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 0x7fafa806c8f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fafd00cd5a0 tx=0x7fafb8009040 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 msgr2=0x7fafd014d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009d90 tx=0x7fafc0004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 shutdown_connections 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fafa806a430 0x7fafa806c8f0 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fafd00cd5a0 tx=0x7fafb8009040 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafd00d2760 0x7fafd00b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd015d650 0x7fafd00b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 --2- 172.21.15.91:0/3808097827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd00ce070 0x7fafd014d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.958+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 >> 172.21.15.91:0/3808097827 conn(0x7fafd00d30a0 msgr2=0x7fafd00c82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.959+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 shutdown_connections 2024-02-02T13:26:25.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:25.959+0000 7fafd8800640 1 -- 172.21.15.91:0/3808097827 wait complete. 2024-02-02T13:26:25.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:26:26.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.191+0000 7f3ec4000640 1 Processor -- start 2024-02-02T13:26:26.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.191+0000 7f3ec4000640 1 -- start start 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec4000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ebc1597a0 0x7f3ebc06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec4000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc06b5f0 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec4000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc06c450 con 0x7f3ebc0b0770 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec4000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc159b80 con 0x7f3ebc1597a0 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec2000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3ebc0b0c00 0x7f3ebc0c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42558/0 (socket says 172.21.15.91:42558) 2024-02-02T13:26:26.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.193+0000 7f3ec2000640 1 -- 172.21.15.91:0/1974802748 learned_addr learned my addr 172.21.15.91:0/1974802748 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308118943 0 0) 0x7f3ebc06b5f0 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e9c005730 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ec1600640 1 --2- 172.21.15.91:0/1974802748 >> v2:172.21.15.113:3300/0 conn(0x7f3ebc1597a0 0x7f3ebc06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294930336 0 0) 0x7f3e9c005730 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 >> v2:172.21.15.113:3300/0 conn(0x7f3ebc1597a0 msgr2=0x7f3ebc06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 --2- 172.21.15.91:0/1974802748 >> v2:172.21.15.113:3300/0 conn(0x7f3ebc1597a0 0x7f3ebc06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 >> v1:172.21.15.113:6789/0 conn(0x7f3ebc0b0770 legacy=0x7f3ebc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc06b840 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3eb4001670 con 0x7f3ebc0b0c00 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 >> v1:172.21.15.91:6789/0 conn(0x7f3ebc0b0c00 legacy=0x7f3ebc0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c005b80 0x7f3e9c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3e9c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c007390 0x7f3e9c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc159b80 con 0x7f3e9c0069d0 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c00e230 con 0x7f3e9c005b80 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.194+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c00e3f0 con 0x7f3e9c007390 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3e9c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec1600640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c005b80 0x7f3e9c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 -- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c007390 msgr2=0x7f3e9c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c007390 0x7f3e9c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 -- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c005b80 msgr2=0x7f3e9c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c005b80 0x7f3e9c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec0c00640 1 -- 172.21.15.91:0/1974802748 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c005730 con 0x7f3e9c0069d0 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec1600640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c005b80 0x7f3e9c005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:26.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.195+0000 7f3ec2000640 1 -- 172.21.15.91:0/1974802748 reap_dead start 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.196+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3e9c006e50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4001e50 tx=0x7f3e9c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60053789ecef14fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.196+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4003b70 con 0x7f3e9c0069d0 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.196+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4003cd0 con 0x7f3e9c0069d0 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.196+0000 7f3ebbe00640 1 -- 172.21.15.91:0/1974802748 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4005310 con 0x7f3e9c0069d0 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 -- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 msgr2=0x7f3e9c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3e9c006e50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4001e50 tx=0x7f3e9c007a00 comp rx=0 tx=0).stop 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 -- 172.21.15.91:0/1974802748 shutdown_connections 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 --2- 172.21.15.91:0/1974802748 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3e9c006e50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 -- 172.21.15.91:0/1974802748 >> 172.21.15.91:0/1974802748 conn(0x7f3ebc0cec80 msgr2=0x7f3ebc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 -- 172.21.15.91:0/1974802748 shutdown_connections 2024-02-02T13:26:26.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.197+0000 7f3ec4000640 1 -- 172.21.15.91:0/1974802748 wait complete. 2024-02-02T13:26:26.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.199+0000 7f3ec4000640 1 Processor -- start 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.199+0000 7f3ec4000640 1 -- start start 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 0x7f3ebc06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ebc157f40 0x7f3ebc1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc158900 con 0x7f3e9c0069d0 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc14ed50 con 0x7f3ebc157f40 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec4000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc14eff0 con 0x7f3ebc14b420 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55714/0 (socket says 172.21.15.91:55714) 2024-02-02T13:26:26.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.200+0000 7f3ec1600640 1 -- 172.21.15.91:0/2877836779 learned_addr learned my addr 172.21.15.91:0/2877836779 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:26.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 0x7f3ebc06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec2000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ebc157f40 0x7f3ebc1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 -- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 msgr2=0x7f3ebc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 0x7f3ebc06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 -- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ebc157f40 msgr2=0x7f3ebc1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ebc157f40 0x7f3ebc1583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 -- 172.21.15.91:0/2877836779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc14f2d0 con 0x7f3e9c0069d0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 0x7f3ebc06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec1600640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0109d0 tx=0x7f3eac010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00ec20 con 0x7f3e9c0069d0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac00ed80 con 0x7f3e9c0069d0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac01c620 con 0x7f3e9c0069d0 2024-02-02T13:26:26.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc14f5b0 con 0x7f3e9c0069d0 2024-02-02T13:26:26.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.201+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc14b8a0 con 0x7f3e9c0069d0 2024-02-02T13:26:26.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.202+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc14f740 con 0x7f3e9c0069d0 2024-02-02T13:26:26.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.204+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3eac01c780 con 0x7f3e9c0069d0 2024-02-02T13:26:26.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.205+0000 7f3eb9600640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 0x7f3e9406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:26.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.205+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3eac088e80 con 0x7f3e9c0069d0 2024-02-02T13:26:26.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.205+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 0x7f3e9406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:26.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.206+0000 7f3ec0c00640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 0x7f3e9406c920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4034000 tx=0x7f3ea4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:26.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.210+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3eac054660 con 0x7f3e9c0069d0 2024-02-02T13:26:26.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.404+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ebc066cf0 con 0x7f3e9c0069d0 2024-02-02T13:26:26.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.970+0000 7f3eb9600640 1 -- 172.21.15.91:0/2877836779 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3eac057c80 con 0x7f3e9c0069d0 2024-02-02T13:26:26.974 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:26:26.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.979+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 msgr2=0x7f3e9406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.979+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 0x7f3e9406c920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4034000 tx=0x7f3ea4009030 comp rx=0 tx=0).stop 2024-02-02T13:26:26.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.979+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 msgr2=0x7f3ebc14c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:26.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.979+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0109d0 tx=0x7f3eac010e90 comp rx=0 tx=0).stop 2024-02-02T13:26:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 shutdown_connections 2024-02-02T13:26:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e9406a460 0x7f3e9406c920 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4034000 tx=0x7f3ea4009030 comp rx=0 tx=0).stop 2024-02-02T13:26:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ebc14b420 0x7f3ebc06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ebc157f40 0x7f3ebc1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 --2- 172.21.15.91:0/2877836779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0069d0 0x7f3ebc14c4b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.981+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 >> 172.21.15.91:0/2877836779 conn(0x7f3ebc0cec80 msgr2=0x7f3ebc14d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.988+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 shutdown_connections 2024-02-02T13:26:26.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:26.988+0000 7f3ec4000640 1 -- 172.21.15.91:0/2877836779 wait complete. 2024-02-02T13:26:27.012 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:26:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.194+0000 7f0ba4a00640 1 Processor -- start 2024-02-02T13:26:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.194+0000 7f0ba4a00640 1 -- start start 2024-02-02T13:26:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.195+0000 7f0ba4a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b9c0ce130 0x7f0b9c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba4a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 0x7f0b9c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba4a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c0cc7c0 con 0x7f0b9c15a010 2024-02-02T13:26:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba4a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c0b10d0 con 0x7f0b9c0ce130 2024-02-02T13:26:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba4a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0ce930 con 0x7f0b9c0b0700 2024-02-02T13:26:27.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba2000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0b9c15a010 0x7f0b9c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42566/0 (socket says 172.21.15.91:42566) 2024-02-02T13:26:27.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba2000640 1 -- 172.21.15.91:0/201040556 learned_addr learned my addr 172.21.15.91:0/201040556 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:27.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba1600640 1 --2- 172.21.15.91:0/201040556 >> v2:172.21.15.113:3300/0 conn(0x7f0b9c0ce130 0x7f0b9c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba2a00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 0x7f0b9c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749686614 0 0) 0x7f0b9c0cc7c0 con 0x7f0b9c15a010 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.196+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b7c003520 con 0x7f0b9c15a010 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1722285471 0 0) 0x7f0b7c003520 con 0x7f0b9c15a010 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 msgr2=0x7f0b9c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 0x7f0b9c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 >> v2:172.21.15.113:3300/0 conn(0x7f0b9c0ce130 msgr2=0x7f0b9c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 --2- 172.21.15.91:0/201040556 >> v2:172.21.15.113:3300/0 conn(0x7f0b9c0ce130 0x7f0b9c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0cd640 con 0x7f0b9c15a010 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0b940019f0 con 0x7f0b9c15a010 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 >> v1:172.21.15.91:6789/0 conn(0x7f0b9c15a010 legacy=0x7f0b9c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c003970 0x7f0b7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b7c0047c0 0x7f0b7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 0x7f0b7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0b10d0 con 0x7f0b7c0047c0 2024-02-02T13:26:27.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0ce930 con 0x7f0b7c003970 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c00b980 con 0x7f0b9c0b0700 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.197+0000 7f0ba1600640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b7c0047c0 0x7f0b7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba2000640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c003970 0x7f0b7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba1600640 1 -- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 msgr2=0x7f0b7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba1600640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b9c0b0700 0x7f0b7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba1600640 1 -- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c003970 msgr2=0x7f0b7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba1600640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c003970 0x7f0b7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.198+0000 7f0ba1600640 1 -- 172.21.15.91:0/201040556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c003520 con 0x7f0b7c0047c0 2024-02-02T13:26:27.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba1600640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b7c0047c0 0x7f0b7c004c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009ac0 tx=0x7f0b7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b1511e75c55e74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c02c710 con 0x7f0b7c0047c0 2024-02-02T13:26:27.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c02c870 con 0x7f0b7c0047c0 2024-02-02T13:26:27.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba0c00640 1 -- 172.21.15.91:0/201040556 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c02cc00 con 0x7f0b7c0047c0 2024-02-02T13:26:27.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba4a00640 1 -- 172.21.15.91:0/201040556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b7c0047c0 msgr2=0x7f0b7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/201040556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b7c0047c0 0x7f0b7c004c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009ac0 tx=0x7f0b7c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.199+0000 7f0ba2a00640 1 -- 172.21.15.91:0/201040556 reap_dead start 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.200+0000 7f0ba4a00640 1 -- 172.21.15.91:0/201040556 shutdown_connections 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.200+0000 7f0ba4a00640 1 -- 172.21.15.91:0/201040556 >> 172.21.15.91:0/201040556 conn(0x7f0b9c0cec80 msgr2=0x7f0b9c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.200+0000 7f0ba4a00640 1 -- 172.21.15.91:0/201040556 shutdown_connections 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.200+0000 7f0ba4a00640 1 -- 172.21.15.91:0/201040556 wait complete. 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.201+0000 7f0ba4a00640 1 Processor -- start 2024-02-02T13:26:27.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 -- start start 2024-02-02T13:26:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b7c0047c0 0x7f0b9c150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b9c0c2390 0x7f0b9c0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c3c90 con 0x7f0b9c0c4640 2024-02-02T13:26:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c5000 con 0x7f0b9c0c2390 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.202+0000 7f0ba4a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c3080 con 0x7f0b7c0047c0 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55732/0 (socket says 172.21.15.91:55732) 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 -- 172.21.15.91:0/623201308 learned_addr learned my addr 172.21.15.91:0/623201308 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 -- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b7c0047c0 msgr2=0x7f0b9c150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b7c0047c0 0x7f0b9c150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 -- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b9c0c2390 msgr2=0x7f0b9c0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba2a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b9c0c2390 0x7f0b9c0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b9c0c2390 0x7f0b9c0c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 -- 172.21.15.91:0/623201308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0c3360 con 0x7f0b9c0c4640 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.203+0000 7f0ba1600640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02c2b0 tx=0x7f0b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.204+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c003810 con 0x7f0b9c0c4640 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.204+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c02de90 con 0x7f0b9c0c4640 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.204+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c003b70 con 0x7f0b9c0c4640 2024-02-02T13:26:27.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.204+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c0c3640 con 0x7f0b9c0c4640 2024-02-02T13:26:27.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.204+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c150c50 con 0x7f0b9c0c4640 2024-02-02T13:26:27.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.205+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c066cf0 con 0x7f0b9c0c4640 2024-02-02T13:26:27.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.206+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0b8c003cd0 con 0x7f0b9c0c4640 2024-02-02T13:26:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.207+0000 7f0b9a000640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 0x7f0b7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.207+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0b8c0aa860 con 0x7f0b9c0c4640 2024-02-02T13:26:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.207+0000 7f0ba2000640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 0x7f0b7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.208+0000 7f0ba2000640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 0x7f0b7406c8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940012e0 tx=0x7f0b94001250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.212+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0b8c075e10 con 0x7f0b9c0c4640 2024-02-02T13:26:27.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.438+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0b9c0b4c00 con 0x7f0b9c0c4640 2024-02-02T13:26:27.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.448+0000 7f0b9a000640 1 -- 172.21.15.91:0/623201308 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0b8c0aa0c0 con 0x7f0b9c0c4640 2024-02-02T13:26:27.450 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 6 and data pool 7 2024-02-02T13:26:27.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.452+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 msgr2=0x7f0b7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.452+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 0x7f0b7406c8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940012e0 tx=0x7f0b94001250 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.452+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 msgr2=0x7f0b9c0c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.452+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02c2b0 tx=0x7f0b8c004730 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 shutdown_connections 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b7406a430 0x7f0b7406c8f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b9c0c2390 0x7f0b9c0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b9c0c4640 0x7f0b9c0c4ac0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 --2- 172.21.15.91:0/623201308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b7c0047c0 0x7f0b9c150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.453+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 >> 172.21.15.91:0/623201308 conn(0x7f0b9c0cec80 msgr2=0x7f0b9c0c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.454+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 shutdown_connections 2024-02-02T13:26:27.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.454+0000 7f0ba4a00640 1 -- 172.21.15.91:0/623201308 wait complete. 2024-02-02T13:26:27.483 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:27.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.656+0000 7f484e800640 1 Processor -- start 2024-02-02T13:26:27.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.656+0000 7f484e800640 1 -- start start 2024-02-02T13:26:27.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f484e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4848151f90 0x7f484815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f484e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48480b0e10 0x7f48480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f484e800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4848070f80 con 0x7f484806f4f0 2024-02-02T13:26:27.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f484e800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4848152370 con 0x7f48480b0e10 2024-02-02T13:26:27.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f484e800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48480cb010 con 0x7f4848151f90 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.657+0000 7f4847e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48480b0e10 0x7f48480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48480b0e10 0x7f48480b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55740/0 (socket says 172.21.15.91:55740) 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 -- 172.21.15.91:0/2256487258 learned_addr learned my addr 172.21.15.91:0/2256487258 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847400640 1 --2- 172.21.15.91:0/2256487258 >> v2:172.21.15.113:3300/0 conn(0x7f4848151f90 0x7f484815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 -- 172.21.15.91:0/2256487258 >> v2:172.21.15.113:3300/0 conn(0x7f4848151f90 msgr2=0x7f484815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 --2- 172.21.15.91:0/2256487258 >> v2:172.21.15.113:3300/0 conn(0x7f4848151f90 0x7f484815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 -- 172.21.15.91:0/2256487258 >> v1:172.21.15.113:6789/0 conn(0x7f484806f4f0 legacy=0x7f48480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:27.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.658+0000 7f4847e00640 1 -- 172.21.15.91:0/2256487258 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480cb2f0 con 0x7f48480b0e10 2024-02-02T13:26:27.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.659+0000 7f4847e00640 1 --2- 172.21.15.91:0/2256487258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48480b0e10 0x7f48480b3af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f48480cc770 tx=0x7f484002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e9c52753789c50 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.659+0000 7f4846a00640 1 -- 172.21.15.91:0/2256487258 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484002ca80 con 0x7f48480b0e10 2024-02-02T13:26:27.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.659+0000 7f4846a00640 1 -- 172.21.15.91:0/2256487258 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484002cbe0 con 0x7f48480b0e10 2024-02-02T13:26:27.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.659+0000 7f4846a00640 1 -- 172.21.15.91:0/2256487258 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48400343f0 con 0x7f48480b0e10 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 -- 172.21.15.91:0/2256487258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48480b0e10 msgr2=0x7f48480b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 --2- 172.21.15.91:0/2256487258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48480b0e10 0x7f48480b3af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f48480cc770 tx=0x7f484002bee0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 -- 172.21.15.91:0/2256487258 shutdown_connections 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 --2- 172.21.15.91:0/2256487258 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48480b0e10 0x7f48480b3af0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 --2- 172.21.15.91:0/2256487258 >> v2:172.21.15.113:3300/0 conn(0x7f4848151f90 0x7f484815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 -- 172.21.15.91:0/2256487258 >> 172.21.15.91:0/2256487258 conn(0x7f48480cec80 msgr2=0x7f48480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 -- 172.21.15.91:0/2256487258 shutdown_connections 2024-02-02T13:26:27.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.660+0000 7f484e800640 1 -- 172.21.15.91:0/2256487258 wait complete. 2024-02-02T13:26:27.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.662+0000 7f484e800640 1 Processor -- start 2024-02-02T13:26:27.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.662+0000 7f484e800640 1 -- start start 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48480b0e10 0x7f48480cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4848151f90 0x7f48481505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c6800 con 0x7f484806f4f0 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c6a30 con 0x7f4848151f90 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f484e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c6d10 con 0x7f48480b0e10 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55746/0 (socket says 172.21.15.91:55746) 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 -- 172.21.15.91:0/2359797689 learned_addr learned my addr 172.21.15.91:0/2359797689 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 -- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48480b0e10 msgr2=0x7f48480cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48480b0e10 0x7f48480cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 -- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4848151f90 msgr2=0x7f48481505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4848151f90 0x7f48481505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.663+0000 7f4847400640 1 -- 172.21.15.91:0/2359797689 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840009b30 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f4847400640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f483800c7f0 tx=0x7f483800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838004220 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48380043a0 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838007860 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480c6ff0 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.664+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48480c75b0 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.665+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4848066cf0 con 0x7f484806f4f0 2024-02-02T13:26:27.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.666+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4838007bd0 con 0x7f484806f4f0 2024-02-02T13:26:27.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.667+0000 7f483fe00640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 0x7f482006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:27.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.667+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48380888d0 con 0x7f484806f4f0 2024-02-02T13:26:27.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.667+0000 7f483e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 0x7f482006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:27.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.668+0000 7f483e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 0x7f482006c8f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48480ada40 tx=0x7f4830006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:27.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.671+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4838053eb0 con 0x7f484806f4f0 2024-02-02T13:26:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.849+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f484806a680 con 0x7f484806f4f0 2024-02-02T13:26:27.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.851+0000 7f483fe00640 1 -- 172.21.15.91:0/2359797689 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16058 (secure 0 0 0) 0x7f483801e020 con 0x7f484806f4f0 2024-02-02T13:26:27.853 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:27.853 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":47,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:27.439702+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:27.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.853+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 msgr2=0x7f482006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.853+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 0x7f482006c8f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48480ada40 tx=0x7f4830006ba0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.853+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 msgr2=0x7f48481f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:27.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.853+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f483800c7f0 tx=0x7f483800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 shutdown_connections 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f482006a430 0x7f482006c8f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48480b0e10 0x7f48480cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4848151f90 0x7f48481505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 --2- 172.21.15.91:0/2359797689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f484806f4f0 0x7f48481f2250 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 >> 172.21.15.91:0/2359797689 conn(0x7f48480cec80 msgr2=0x7f48480c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 shutdown_connections 2024-02-02T13:26:27.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:27.854+0000 7f484e800640 1 -- 172.21.15.91:0/2359797689 wait complete. 2024-02-02T13:26:27.874 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:28.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.079+0000 7f0212200640 1 Processor -- start 2024-02-02T13:26:28.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.079+0000 7f0212200640 1 -- start start 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0212200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f020c15b2a0 0x7f020c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0212200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f020c06f3f0 0x7f020c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0212200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c06c1c0 con 0x7f020c15aec0 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0212200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c06b5f0 con 0x7f020c15b2a0 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0212200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c06cd60 con 0x7f020c06f3f0 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.080+0000 7f0203c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f020c15b2a0 0x7f020c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f020c15b2a0 0x7f020c0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55750/0 (socket says 172.21.15.91:55750) 2024-02-02T13:26:28.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 -- 172.21.15.91:0/1507198115 learned_addr learned my addr 172.21.15.91:0/1507198115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:28.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0211800640 1 --2- 172.21.15.91:0/1507198115 >> v2:172.21.15.113:3300/0 conn(0x7f020c06f3f0 0x7f020c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 -- 172.21.15.91:0/1507198115 >> v2:172.21.15.113:3300/0 conn(0x7f020c06f3f0 msgr2=0x7f020c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:28.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 --2- 172.21.15.91:0/1507198115 >> v2:172.21.15.113:3300/0 conn(0x7f020c06f3f0 0x7f020c0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 -- 172.21.15.91:0/1507198115 >> v1:172.21.15.113:6789/0 conn(0x7f020c15aec0 legacy=0x7f020c06c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:28.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.081+0000 7f0203c00640 1 -- 172.21.15.91:0/1507198115 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c06b890 con 0x7f020c15b2a0 2024-02-02T13:26:28.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.082+0000 7f0203c00640 1 --2- 172.21.15.91:0/1507198115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15b2a0 0x7f020c0cf5b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b46d0 tx=0x7f01f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c01d5b20916cbd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.082+0000 7f020be00640 1 -- 172.21.15.91:0/1507198115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4030860 con 0x7f020c15b2a0 2024-02-02T13:26:28.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.082+0000 7f020be00640 1 -- 172.21.15.91:0/1507198115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f40309c0 con 0x7f020c15b2a0 2024-02-02T13:26:28.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.082+0000 7f020be00640 1 -- 172.21.15.91:0/1507198115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4030d50 con 0x7f020c15b2a0 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 -- 172.21.15.91:0/1507198115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15b2a0 msgr2=0x7f020c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 --2- 172.21.15.91:0/1507198115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15b2a0 0x7f020c0cf5b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b46d0 tx=0x7f01f402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 -- 172.21.15.91:0/1507198115 shutdown_connections 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 --2- 172.21.15.91:0/1507198115 >> v2:172.21.15.113:3300/0 conn(0x7f020c06f3f0 0x7f020c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 --2- 172.21.15.91:0/1507198115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15b2a0 0x7f020c0cf5b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 -- 172.21.15.91:0/1507198115 >> 172.21.15.91:0/1507198115 conn(0x7f020c0cea90 msgr2=0x7f020c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 -- 172.21.15.91:0/1507198115 shutdown_connections 2024-02-02T13:26:28.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.083+0000 7f0212200640 1 -- 172.21.15.91:0/1507198115 wait complete. 2024-02-02T13:26:28.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.085+0000 7f0212200640 1 Processor -- start 2024-02-02T13:26:28.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.086+0000 7f0212200640 1 -- start start 2024-02-02T13:26:28.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 0x7f020c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 0x7f020c0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0ad3f0 con 0x7f020c15aec0 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c150ec0 con 0x7f020c15b2a0 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0212200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0ac870 con 0x7f020c06f3f0 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55758/0 (socket says 172.21.15.91:55758) 2024-02-02T13:26:28.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 -- 172.21.15.91:0/2645717617 learned_addr learned my addr 172.21.15.91:0/2645717617 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0211800640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 0x7f020c0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0210e00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 0x7f020c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 -- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 msgr2=0x7f020c0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 0x7f020c0a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 -- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 msgr2=0x7f020c0b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.087+0000 7f0203c00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 0x7f020c0b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0203c00640 1 -- 172.21.15.91:0/2645717617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40094d0 con 0x7f020c15aec0 2024-02-02T13:26:28.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0211800640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 0x7f020c0b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0210e00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 0x7f020c0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0203c00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f01f402b9e0 tx=0x7f01f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4030830 con 0x7f020c15aec0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4039d90 con 0x7f020c15aec0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4043650 con 0x7f020c15aec0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0acb50 con 0x7f020c15aec0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.088+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0ad110 con 0x7f020c15aec0 2024-02-02T13:26:28.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.089+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c04dff0 con 0x7f020c15aec0 2024-02-02T13:26:28.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.090+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f01f404c3b0 con 0x7f020c15aec0 2024-02-02T13:26:28.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.092+0000 7f0209600640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 0x7f01e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.092+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01f40ab1d0 con 0x7f020c15aec0 2024-02-02T13:26:28.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.092+0000 7f0210e00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 0x7f01e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.093+0000 7f0210e00640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 0x7f01e406c8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.097+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f01f4076720 con 0x7f020c15aec0 2024-02-02T13:26:28.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.271+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020c06ebf0 con 0x7f020c15aec0 2024-02-02T13:26:28.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.274+0000 7f0209600640 1 -- 172.21.15.91:0/2645717617 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16058 (secure 0 0 0) 0x7f01f4079d40 con 0x7f020c15aec0 2024-02-02T13:26:28.275 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:28.276 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":47,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:27.439702+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:28.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 msgr2=0x7f01e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 0x7f01e406c8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2024-02-02T13:26:28.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 msgr2=0x7f020c1f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f01f402b9e0 tx=0x7f01f4004730 comp rx=0 tx=0).stop 2024-02-02T13:26:28.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 shutdown_connections 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f01e406a430 0x7f01e406c8f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15b2a0 0x7f020c0b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f020c15aec0 0x7f020c1f1f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 --2- 172.21.15.91:0/2645717617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f020c06f3f0 0x7f020c0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.276+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 >> 172.21.15.91:0/2645717617 conn(0x7f020c0cea90 msgr2=0x7f020c0c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.277+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 shutdown_connections 2024-02-02T13:26:28.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.277+0000 7f0212200640 1 -- 172.21.15.91:0/2645717617 wait complete. 2024-02-02T13:26:28.294 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:26:28.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.501+0000 7f4ca6c00640 1 Processor -- start 2024-02-02T13:26:28.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.501+0000 7f4ca6c00640 1 -- start start 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.502+0000 7f4ca6c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ca00b0da0 0x7f4ca00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.502+0000 7f4ca6c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca00b1e60 con 0x7f4ca00b1a00 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.502+0000 7f4ca6c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca015f1b0 con 0x7f4ca015b2f0 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.502+0000 7f4ca6c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca015fdc0 con 0x7f4ca00b0da0 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.502+0000 7f4c9f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4ca00b1a00 0x7f4ca00b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42576/0 (socket says 172.21.15.91:42576) 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9f400640 1 -- 172.21.15.91:0/2421375543 learned_addr learned my addr 172.21.15.91:0/2421375543 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:28.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4ca4c00640 1 --2- 172.21.15.91:0/2421375543 >> v2:172.21.15.113:3300/0 conn(0x7f4ca00b0da0 0x7f4ca00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875123187 0 0) 0x7f4ca00b1e60 con 0x7f4ca00b1a00 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c80003520 con 0x7f4ca00b1a00 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477296571 0 0) 0x7f4c80003520 con 0x7f4ca00b1a00 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 >> v2:172.21.15.113:3300/0 conn(0x7f4ca00b0da0 msgr2=0x7f4ca00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 --2- 172.21.15.91:0/2421375543 >> v2:172.21.15.113:3300/0 conn(0x7f4ca00b0da0 0x7f4ca00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 >> v1:172.21.15.113:6789/0 conn(0x7f4ca015b2f0 legacy=0x7f4ca00711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca015f490 con 0x7f4ca00b1a00 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f4c88001670 con 0x7f4ca00b1a00 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 >> v1:172.21.15.91:6789/0 conn(0x7f4ca00b1a00 legacy=0x7f4ca00b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80003970 0x7f4c80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800047c0 0x7f4c80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 0x7f4c8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca015fdc0 con 0x7f4c80005180 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8000c120 con 0x7f4c800047c0 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.503+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8000c2e0 con 0x7f4c80003970 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 0x7f4c8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4c9f400640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800047c0 0x7f4c80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 -- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80003970 msgr2=0x7f4c80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80003970 0x7f4c80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 -- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800047c0 msgr2=0x7f4c80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800047c0 0x7f4c80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.504+0000 7f4ca4c00640 1 -- 172.21.15.91:0/2421375543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80003520 con 0x7f4c80005180 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.505+0000 7f4ca4c00640 1 -- 172.21.15.91:0/2421375543 reap_dead start 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.505+0000 7f4ca4c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 0x7f4c8000b980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009f90 tx=0x7f4c800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b09888411dc230 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.506+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9802cc20 con 0x7f4c80005180 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.506+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9800f040 con 0x7f4c80005180 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.506+0000 7f4c9ea00640 1 -- 172.21.15.91:0/2421375543 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98003990 con 0x7f4c80005180 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 -- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 msgr2=0x7f4c8000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 0x7f4c8000b980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009f90 tx=0x7f4c800057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 -- 172.21.15.91:0/2421375543 shutdown_connections 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/2421375543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c80005180 0x7f4c8000b980 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 -- 172.21.15.91:0/2421375543 >> 172.21.15.91:0/2421375543 conn(0x7f4ca00ceb90 msgr2=0x7f4ca00cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 -- 172.21.15.91:0/2421375543 shutdown_connections 2024-02-02T13:26:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.507+0000 7f4ca6c00640 1 -- 172.21.15.91:0/2421375543 wait complete. 2024-02-02T13:26:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.509+0000 7f4ca6c00640 1 Processor -- start 2024-02-02T13:26:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.509+0000 7f4ca6c00640 1 -- start start 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80005180 0x7f4ca01f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca0146d90 0x7f4ca0147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00c5630 con 0x7f4ca00c4510 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01f1a50 con 0x7f4ca0146d90 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4ca6c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca0147760 con 0x7f4c80005180 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4c9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4c9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55774/0 (socket says 172.21.15.91:55774) 2024-02-02T13:26:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.510+0000 7f4c9f400640 1 -- 172.21.15.91:0/646966492 learned_addr learned my addr 172.21.15.91:0/646966492 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 -- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80005180 msgr2=0x7f4ca01f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9fe00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80005180 0x7f4ca01f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80005180 0x7f4ca01f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 -- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca0146d90 msgr2=0x7f4ca0147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca0146d90 0x7f4ca0147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 -- 172.21.15.91:0/646966492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca01f0900 con 0x7f4ca00c4510 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c9f400640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e780 tx=0x7f4c8800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.511+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800ee90 con 0x7f4ca00c4510 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.512+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800c8d0 con 0x7f4ca00c4510 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.512+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800cc80 con 0x7f4ca00c4510 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.512+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca00c49b0 con 0x7f4ca00c4510 2024-02-02T13:26:28.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.512+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca00c4f70 con 0x7f4ca00c4510 2024-02-02T13:26:28.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.513+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca00c5770 con 0x7f4ca00c4510 2024-02-02T13:26:28.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.514+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4c8801d460 con 0x7f4ca00c4510 2024-02-02T13:26:28.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.515+0000 7f4c97e00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 0x7f4c7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:28.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.515+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c88088870 con 0x7f4ca00c4510 2024-02-02T13:26:28.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.515+0000 7f4c9fe00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 0x7f4c7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:28.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.516+0000 7f4c9fe00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 0x7f4c7806c8f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c900015d0 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:28.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.519+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4ca00c5770 con 0x7f4ca00c4510 2024-02-02T13:26:28.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:28.744+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4ca00c5770 con 0x7f4ca00c4510 2024-02-02T13:26:28.776 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:26:28.784 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:26:29.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.462+0000 7f4c97e00640 1 -- 172.21.15.91:0/646966492 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4ca00c5770 con 0x7f4ca00c4510 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 msgr2=0x7f4c7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 0x7f4c7806c8f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c900015d0 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 msgr2=0x7f4ca0146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e780 tx=0x7f4c8800ec40 comp rx=0 tx=0).stop 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 shutdown_connections 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4c7806a430 0x7f4c7806c8f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca0146d90 0x7f4ca0147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4ca00c4510 0x7f4ca0146850 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 --2- 172.21.15.91:0/646966492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c80005180 0x7f4ca01f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.465+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 >> 172.21.15.91:0/646966492 conn(0x7f4ca00ceb90 msgr2=0x7f4ca015a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:29.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.466+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 shutdown_connections 2024-02-02T13:26:29.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.466+0000 7f4ca6c00640 1 -- 172.21.15.91:0/646966492 wait complete. 2024-02-02T13:26:29.488 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:26:29.488 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:26:29.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.701+0000 7f3e43200640 1 Processor -- start 2024-02-02T13:26:29.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.701+0000 7f3e43200640 1 -- start start 2024-02-02T13:26:29.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e43200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e3c15b2f0 0x7f3e3c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e43200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e3c0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e43200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c15f1b0 con 0x7f3e3c071650 2024-02-02T13:26:29.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e43200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c15fdc0 con 0x7f3e3c15b2f0 2024-02-02T13:26:29.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e43200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15b6d0 con 0x7f3e3c0711f0 2024-02-02T13:26:29.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e41200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3e3c071650 0x7f3e3c0c9480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:42590/0 (socket says 172.21.15.91:42590) 2024-02-02T13:26:29.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.703+0000 7f3e41200640 1 -- 172.21.15.91:0/1377140454 learned_addr learned my addr 172.21.15.91:0/1377140454 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3be00640 1 --2- 172.21.15.91:0/1377140454 >> v2:172.21.15.113:3300/0 conn(0x7f3e3c15b2f0 0x7f3e3c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e33200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e3c0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239677079 0 0) 0x7f3e3c15f1b0 con 0x7f3e3c071650 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e1c003520 con 0x7f3e3c071650 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270474975 0 0) 0x7f3e1c003520 con 0x7f3e3c071650 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 msgr2=0x7f3e3c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e3c0b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 >> v2:172.21.15.113:3300/0 conn(0x7f3e3c15b2f0 msgr2=0x7f3e3c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:29.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 --2- 172.21.15.91:0/1377140454 >> v2:172.21.15.113:3300/0 conn(0x7f3e3c15b2f0 0x7f3e3c0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c15f490 con 0x7f3e3c071650 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3e340019f0 con 0x7f3e3c071650 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 >> v1:172.21.15.91:6789/0 conn(0x7f3e3c071650 legacy=0x7f3e3c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e1c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c003f70 0x7f3e1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e1c004d00 0x7f3e1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15fdc0 con 0x7f3e1c004d00 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15b6d0 con 0x7f3e1c003f70 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.704+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c00ba60 con 0x7f3e3c0711f0 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e1c004d00 0x7f3e1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e33200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c003f70 0x7f3e1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e3be00640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e1c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 -- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 msgr2=0x7f3e1c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0711f0 0x7f3e1c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 -- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c003f70 msgr2=0x7f3e1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c003f70 0x7f3e1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.705+0000 7f3e41200640 1 -- 172.21.15.91:0/1377140454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c003520 con 0x7f3e1c004d00 2024-02-02T13:26:29.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.706+0000 7f3e41200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e1c004d00 0x7f3e1c005180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c071a50 tx=0x7f3e1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e2775ca08c8804 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:29.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034630 con 0x7f3e1c004d00 2024-02-02T13:26:29.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3402f070 con 0x7f3e1c004d00 2024-02-02T13:26:29.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e3b400640 1 -- 172.21.15.91:0/1377140454 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034a50 con 0x7f3e1c004d00 2024-02-02T13:26:29.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 -- 172.21.15.91:0/1377140454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e1c004d00 msgr2=0x7f3e1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 --2- 172.21.15.91:0/1377140454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e1c004d00 0x7f3e1c005180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c071a50 tx=0x7f3e1c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e41200640 1 -- 172.21.15.91:0/1377140454 reap_dead start 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 -- 172.21.15.91:0/1377140454 shutdown_connections 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 -- 172.21.15.91:0/1377140454 >> 172.21.15.91:0/1377140454 conn(0x7f3e3c0ceb90 msgr2=0x7f3e3c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 -- 172.21.15.91:0/1377140454 shutdown_connections 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.707+0000 7f3e43200640 1 -- 172.21.15.91:0/1377140454 wait complete. 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.709+0000 7f3e43200640 1 Processor -- start 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.709+0000 7f3e43200640 1 -- start start 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c004d00 0x7f3e3c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0c5bd0 0x7f3e3c0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c1f2ad0 con 0x7f3e3c157d80 2024-02-02T13:26:29.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c1578f0 con 0x7f3e1c004d00 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.710+0000 7f3e43200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c0c6c20 con 0x7f3e3c0c5bd0 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55800/0 (socket says 172.21.15.91:55800) 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 -- 172.21.15.91:0/2449904744 learned_addr learned my addr 172.21.15.91:0/2449904744 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 -- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0c5bd0 msgr2=0x7f3e3c0c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e3be00640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c004d00 0x7f3e3c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0c5bd0 0x7f3e3c0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 -- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c004d00 msgr2=0x7f3e3c156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c004d00 0x7f3e3c156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 -- 172.21.15.91:0/2449904744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c156fc0 con 0x7f3e3c157d80 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.711+0000 7f3e41200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3402ca70 tx=0x7f3e34001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.712+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034640 con 0x7f3e3c157d80 2024-02-02T13:26:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.712+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e340347a0 con 0x7f3e3c157d80 2024-02-02T13:26:29.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.712+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034b10 con 0x7f3e3c157d80 2024-02-02T13:26:29.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.712+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c1572a0 con 0x7f3e3c157d80 2024-02-02T13:26:29.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.712+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c0b08a0 con 0x7f3e3c157d80 2024-02-02T13:26:29.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.713+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c066cf0 con 0x7f3e3c157d80 2024-02-02T13:26:29.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.714+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e34031030 con 0x7f3e3c157d80 2024-02-02T13:26:29.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.715+0000 7f3e38c00640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 0x7f3e1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:29.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.715+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e340aad80 con 0x7f3e3c157d80 2024-02-02T13:26:29.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.715+0000 7f3e3be00640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 0x7f3e1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:29.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.716+0000 7f3e3be00640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 0x7f3e1406c8f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0099d0 tx=0x7f3e2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:29.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.720+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e340763f0 con 0x7f3e3c157d80 2024-02-02T13:26:29.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:29.951+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3e3c066cf0 con 0x7f3e3c157d80 2024-02-02T13:26:30.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.466+0000 7f3e38c00640 1 -- 172.21.15.91:0/2449904744 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3e34079a10 con 0x7f3e3c157d80 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 msgr2=0x7f3e1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 0x7f3e1406c8f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0099d0 tx=0x7f3e2c008040 comp rx=0 tx=0).stop 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 msgr2=0x7f3e3c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3402ca70 tx=0x7f3e34001cb0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 shutdown_connections 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e1406a430 0x7f3e1406c8f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3c0c5bd0 0x7f3e3c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e3c157d80 0x7f3e3c0cc6d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 --2- 172.21.15.91:0/2449904744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e1c004d00 0x7f3e3c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 >> 172.21.15.91:0/2449904744 conn(0x7f3e3c0ceb90 msgr2=0x7f3e3c06f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:30.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.469+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 shutdown_connections 2024-02-02T13:26:30.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.470+0000 7f3e43200640 1 -- 172.21.15.91:0/2449904744 wait complete. 2024-02-02T13:26:30.491 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:26:30.492 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:30.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.698+0000 7fd1a7200640 1 Processor -- start 2024-02-02T13:26:30.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.699+0000 7fd1a7200640 1 -- start start 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 0x7fd1a006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd1a7200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a015d870 con 0x7fd1a00acfe0 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd1a7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a006bce0 con 0x7fd1a015d490 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd1a7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0161220 con 0x7fd1a00b0e10 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd19f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd1a00acfe0 0x7fd1a00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34444/0 (socket says 172.21.15.91:34444) 2024-02-02T13:26:30.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.700+0000 7fd19f400640 1 -- 172.21.15.91:0/2296852349 learned_addr learned my addr 172.21.15.91:0/2296852349 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd1a5200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 0x7fd1a006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279146162 0 0) 0x7fd1a015d870 con 0x7fd1a00acfe0 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd180003520 con 0x7fd1a00acfe0 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802125720 0 0) 0x7fd180003520 con 0x7fd1a00acfe0 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 msgr2=0x7fd1a006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 0x7fd1a006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 >> v1:172.21.15.113:6789/0 conn(0x7fd1a015d490 legacy=0x7fd1a00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00ad490 con 0x7fd1a00acfe0 2024-02-02T13:26:30.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd188001670 con 0x7fd1a00acfe0 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 >> v1:172.21.15.91:6789/0 conn(0x7fd1a00acfe0 legacy=0x7fd1a00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1800039a0 0x7fd180003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 0x7fd1800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd180005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0161220 con 0x7fd180004d00 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18000ba90 con 0x7fd1800039a0 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.701+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18000bc50 con 0x7fd1a00b0e10 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd180005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd19fe00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1800039a0 0x7fd180003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 -- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 msgr2=0x7fd1800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00b0e10 0x7fd1800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 -- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1800039a0 msgr2=0x7fd180003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1800039a0 0x7fd180003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd1a5200640 1 -- 172.21.15.91:0/2296852349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180003520 con 0x7fd180004d00 2024-02-02T13:26:30.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.702+0000 7fd19fe00640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1800039a0 0x7fd180003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:30.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.703+0000 7fd1a5200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd180005180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00c91f0 tx=0x7fd1800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe42c06ca7c73d74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.703+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19802cbd0 con 0x7fd180004d00 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.703+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800f040 con 0x7fd180004d00 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.703+0000 7fd19ea00640 1 -- 172.21.15.91:0/2296852349 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198034440 con 0x7fd180004d00 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 -- 172.21.15.91:0/2296852349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 msgr2=0x7fd180005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 --2- 172.21.15.91:0/2296852349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd180005180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00c91f0 tx=0x7fd1800057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a5200640 1 -- 172.21.15.91:0/2296852349 reap_dead start 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 -- 172.21.15.91:0/2296852349 shutdown_connections 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 -- 172.21.15.91:0/2296852349 >> 172.21.15.91:0/2296852349 conn(0x7fd1a00cec80 msgr2=0x7fd1a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 -- 172.21.15.91:0/2296852349 shutdown_connections 2024-02-02T13:26:30.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.704+0000 7fd1a7200640 1 -- 172.21.15.91:0/2296852349 wait complete. 2024-02-02T13:26:30.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.706+0000 7fd1a7200640 1 Processor -- start 2024-02-02T13:26:30.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.706+0000 7fd1a7200640 1 -- start start 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00c5200 0x7fd1a01f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a01f6940 0x7fd1a00c1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c44c0 con 0x7fd180004d00 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f70c0 con 0x7fd1a01f6940 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd1a7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c4720 con 0x7fd1a00c5200 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47538/0 (socket says 172.21.15.91:47538) 2024-02-02T13:26:30.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.707+0000 7fd19fe00640 1 -- 172.21.15.91:0/1565242838 learned_addr learned my addr 172.21.15.91:0/1565242838 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:30.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 -- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00c5200 msgr2=0x7fd1a01f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00c5200 0x7fd1a01f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 -- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a01f6940 msgr2=0x7fd1a00c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd1a5200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a01f6940 0x7fd1a00c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a01f6940 0x7fd1a00c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 -- 172.21.15.91:0/1565242838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00c4a00 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd19fe00640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d7a0 tx=0x7fd19000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190017070 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190005070 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190005420 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.708+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a00c4ce0 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.709+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a00af1c0 con 0x7fd180004d00 2024-02-02T13:26:30.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.710+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a0066cf0 con 0x7fd180004d00 2024-02-02T13:26:30.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.710+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd190007030 con 0x7fd180004d00 2024-02-02T13:26:30.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.711+0000 7fd197e00640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 0x7fd17806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:30.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.711+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd190088230 con 0x7fd180004d00 2024-02-02T13:26:30.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.712+0000 7fd19f400640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 0x7fd17806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:30.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.712+0000 7fd19f400640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 0x7fd17806c920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1880017a0 tx=0x7fd188001590 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:30.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.716+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd1900537c0 con 0x7fd180004d00 2024-02-02T13:26:30.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.942+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a0160880 con 0x7fd180004d00 2024-02-02T13:26:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.943+0000 7fd197e00640 1 -- 172.21.15.91:0/1565242838 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fd190056de0 con 0x7fd180004d00 2024-02-02T13:26:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:30.945 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:26:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 msgr2=0x7fd17806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 0x7fd17806c920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1880017a0 tx=0x7fd188001590 comp rx=0 tx=0).stop 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 msgr2=0x7fd1a00aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d7a0 tx=0x7fd19000dc60 comp rx=0 tx=0).stop 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 shutdown_connections 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd17806a460 0x7fd17806c920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a00c5200 0x7fd1a01f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a01f6940 0x7fd1a00c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 --2- 172.21.15.91:0/1565242838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd180004d00 0x7fd1a00aec80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:30.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 >> 172.21.15.91:0/1565242838 conn(0x7fd1a00cec80 msgr2=0x7fd1a00b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:30.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.946+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 shutdown_connections 2024-02-02T13:26:30.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:30.947+0000 7fd1a7200640 1 -- 172.21.15.91:0/1565242838 wait complete. 2024-02-02T13:26:30.968 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:26:31.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.180+0000 7fcd76600640 1 Processor -- start 2024-02-02T13:26:31.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.180+0000 7fcd76600640 1 -- start start 2024-02-02T13:26:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd76600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd700b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd76600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd700b6990 0x7fcd700d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd76600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700b7d40 con 0x7fcd7015cee0 2024-02-02T13:26:31.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd76600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700b81d0 con 0x7fcd700b6990 2024-02-02T13:26:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd76600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b7130 con 0x7fcd700ce1d0 2024-02-02T13:26:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd75200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcd7015cee0 0x7fcd700cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34456/0 (socket says 172.21.15.91:34456) 2024-02-02T13:26:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd75200640 1 -- 172.21.15.91:0/2193907930 learned_addr learned my addr 172.21.15.91:0/2193907930 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd75c00640 1 --2- 172.21.15.91:0/2193907930 >> v2:172.21.15.91:3300/0 conn(0x7fcd700b6990 0x7fcd700d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd6fe00640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd700b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd6f400640 1 -- 172.21.15.91:0/2193907930 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180527712 0 0) 0x7fcd700b7d40 con 0x7fcd7015cee0 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.182+0000 7fcd6f400640 1 -- 172.21.15.91:0/2193907930 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd50003520 con 0x7fcd7015cee0 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.183+0000 7fcd75c00640 1 -- 172.21.15.91:0/2193907930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 msgr2=0x7fcd700b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.183+0000 7fcd75c00640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd700b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.183+0000 7fcd75c00640 1 -- 172.21.15.91:0/2193907930 >> v1:172.21.15.91:6789/0 conn(0x7fcd7015cee0 legacy=0x7fcd700cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.183+0000 7fcd75c00640 1 -- 172.21.15.91:0/2193907930 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700b7410 con 0x7fcd700b6990 2024-02-02T13:26:31.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.184+0000 7fcd75c00640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd700b6990 0x7fcd700d1130 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466cba72fed80d14 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.184+0000 7fcd6f400640 1 -- 172.21.15.91:0/2193907930 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6802cc50 con 0x7fcd700b6990 2024-02-02T13:26:31.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.184+0000 7fcd6f400640 1 -- 172.21.15.91:0/2193907930 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6802cdb0 con 0x7fcd700b6990 2024-02-02T13:26:31.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.184+0000 7fcd6f400640 1 -- 172.21.15.91:0/2193907930 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68034660 con 0x7fcd700b6990 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 -- 172.21.15.91:0/2193907930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd700b6990 msgr2=0x7fcd700d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd700b6990 0x7fcd700d1130 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802bea0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 -- 172.21.15.91:0/2193907930 shutdown_connections 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd700b6990 0x7fcd700d1130 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 --2- 172.21.15.91:0/2193907930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd700b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 -- 172.21.15.91:0/2193907930 >> 172.21.15.91:0/2193907930 conn(0x7fcd70070690 msgr2=0x7fcd70070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 -- 172.21.15.91:0/2193907930 shutdown_connections 2024-02-02T13:26:31.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.185+0000 7fcd76600640 1 -- 172.21.15.91:0/2193907930 wait complete. 2024-02-02T13:26:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.187+0000 7fcd76600640 1 Processor -- start 2024-02-02T13:26:31.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.187+0000 7fcd76600640 1 -- start start 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd700b6990 0x7fcd701f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd70158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70159a80 con 0x7fcd7015cee0 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70158e00 con 0x7fcd700b6990 2024-02-02T13:26:31.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd76600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701590e0 con 0x7fcd700ce1d0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd75c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd75c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47562/0 (socket says 172.21.15.91:47562) 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd75c00640 1 -- 172.21.15.91:0/3871221024 learned_addr learned my addr 172.21.15.91:0/3871221024 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd75200640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd700b6990 0x7fcd701f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.188+0000 7fcd6fe00640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd70158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 -- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 msgr2=0x7fcd70158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd70158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 -- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd700b6990 msgr2=0x7fcd701f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd700b6990 0x7fcd701f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 -- 172.21.15.91:0/3871221024 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009a70 con 0x7fcd7015cee0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd75c00640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68040650 con 0x7fcd7015cee0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd680407b0 con 0x7fcd7015cee0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68040b60 con 0x7fcd7015cee0 2024-02-02T13:26:31.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd701593c0 con 0x7fcd7015cee0 2024-02-02T13:26:31.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.189+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70158ac0 con 0x7fcd7015cee0 2024-02-02T13:26:31.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.190+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7004dff0 con 0x7fcd7015cee0 2024-02-02T13:26:31.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.191+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd68040cc0 con 0x7fcd7015cee0 2024-02-02T13:26:31.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.193+0000 7fcd6cc00640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.193+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd680ab660 con 0x7fcd7015cee0 2024-02-02T13:26:31.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.193+0000 7fcd75200640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.193+0000 7fcd75200640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 0x7fcd4806c8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd600011e0 tx=0x7fcd60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.197+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcd68076d70 con 0x7fcd7015cee0 2024-02-02T13:26:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.387+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd700b1e10 con 0x7fcd7015cee0 2024-02-02T13:26:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.390+0000 7fcd6cc00640 1 -- 172.21.15.91:0/3871221024 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16045 (secure 0 0 0) 0x7fcd68009450 con 0x7fcd7015cee0 2024-02-02T13:26:31.392 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:31.393 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:28.444632+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:26:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 msgr2=0x7fcd4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 0x7fcd4806c8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd600011e0 tx=0x7fcd60009040 comp rx=0 tx=0).stop 2024-02-02T13:26:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 msgr2=0x7fcd700caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd68004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 shutdown_connections 2024-02-02T13:26:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd700ce1d0 0x7fcd70158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd7015cee0 0x7fcd700caf00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 --2- 172.21.15.91:0/3871221024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd700b6990 0x7fcd701f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 >> 172.21.15.91:0/3871221024 conn(0x7fcd70070690 msgr2=0x7fcd700c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:31.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.393+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 shutdown_connections 2024-02-02T13:26:31.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.394+0000 7fcd76600640 1 -- 172.21.15.91:0/3871221024 wait complete. 2024-02-02T13:26:31.414 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:26:31.414 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:26:31.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.619+0000 7f24d5a00640 1 Processor -- start 2024-02-02T13:26:31.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.619+0000 7f24d5a00640 1 -- start start 2024-02-02T13:26:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24d5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24d015f5a0 0x7f24d00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 0x7f24d00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24d5a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d0070f80 con 0x7f24d0151f90 2024-02-02T13:26:31.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24d5a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d015fa30 con 0x7f24d015f5a0 2024-02-02T13:26:31.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24d5a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b10b0 con 0x7f24d00b0c40 2024-02-02T13:26:31.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24cea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f24d0151f90 0x7f24d006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34468/0 (socket says 172.21.15.91:34468) 2024-02-02T13:26:31.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24cea00640 1 -- 172.21.15.91:0/343855254 learned_addr learned my addr 172.21.15.91:0/343855254 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:31.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24cf400640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 0x7f24d00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017718512 0 0) 0x7f24d0070f80 con 0x7f24d0151f90 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24c5e00640 1 --2- 172.21.15.91:0/343855254 >> v2:172.21.15.113:3300/0 conn(0x7f24d015f5a0 0x7f24d00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b0003520 con 0x7f24d0151f90 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240013375 0 0) 0x7f24b0003520 con 0x7f24d0151f90 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 msgr2=0x7f24d00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 0x7f24d00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 >> v2:172.21.15.113:3300/0 conn(0x7f24d015f5a0 msgr2=0x7f24d00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 --2- 172.21.15.91:0/343855254 >> v2:172.21.15.113:3300/0 conn(0x7f24d015f5a0 0x7f24d00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0152370 con 0x7f24d0151f90 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f24c0001670 con 0x7f24d0151f90 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 >> v1:172.21.15.91:6789/0 conn(0x7f24d0151f90 legacy=0x7f24d006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b0003970 0x7f24b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 0x7f24b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d015fa30 con 0x7f24b00047c0 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b10b0 con 0x7f24b0003970 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b000b980 con 0x7f24d00b0c40 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.622+0000 7f24c5e00640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.623+0000 7f24c5e00640 1 -- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 msgr2=0x7f24b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.623+0000 7f24c5e00640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00b0c40 0x7f24b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.623+0000 7f24c5e00640 1 -- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b0003970 msgr2=0x7f24b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.623+0000 7f24c5e00640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b0003970 0x7f24b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.623+0000 7f24c5e00640 1 -- 172.21.15.91:0/343855254 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0003520 con 0x7f24b00047c0 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24c5e00640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24b0004c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ac0 tx=0x7f24b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76957ec1c715abf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b802c710 con 0x7f24b00047c0 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b802c870 con 0x7f24b00047c0 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24ce000640 1 -- 172.21.15.91:0/343855254 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b802cc00 con 0x7f24b00047c0 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24d5a00640 1 -- 172.21.15.91:0/343855254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 msgr2=0x7f24b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.624+0000 7f24d5a00640 1 --2- 172.21.15.91:0/343855254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24b0004c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ac0 tx=0x7f24b00057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.625+0000 7f24cf400640 1 -- 172.21.15.91:0/343855254 reap_dead start 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.625+0000 7f24d5a00640 1 -- 172.21.15.91:0/343855254 shutdown_connections 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.625+0000 7f24d5a00640 1 -- 172.21.15.91:0/343855254 >> 172.21.15.91:0/343855254 conn(0x7f24d00cec80 msgr2=0x7f24d00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.625+0000 7f24d5a00640 1 -- 172.21.15.91:0/343855254 shutdown_connections 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.625+0000 7f24d5a00640 1 -- 172.21.15.91:0/343855254 wait complete. 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.627+0000 7f24d5a00640 1 Processor -- start 2024-02-02T13:26:31.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.627+0000 7f24d5a00640 1 -- start start 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00c3400 0x7f24d01f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d00c4840 0x7f24d00c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01588d0 con 0x7f24b00047c0 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00c3a80 con 0x7f24d00c4840 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24d5a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00c3d60 con 0x7f24d00c3400 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47580/0 (socket says 172.21.15.91:47580) 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 -- 172.21.15.91:0/2548369651 learned_addr learned my addr 172.21.15.91:0/2548369651 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 -- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00c3400 msgr2=0x7f24d01f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00c3400 0x7f24d01f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cf400640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d00c4840 0x7f24d00c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 -- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d00c4840 msgr2=0x7f24d00c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d00c4840 0x7f24d00c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.628+0000 7f24cea00640 1 -- 172.21.15.91:0/2548369651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00c4040 con 0x7f24b00047c0 2024-02-02T13:26:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24cea00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c7a0 tx=0x7f24c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c00074e0 con 0x7f24b00047c0 2024-02-02T13:26:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0007660 con 0x7f24b00047c0 2024-02-02T13:26:31.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d00c4320 con 0x7f24b00047c0 2024-02-02T13:26:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004a20 con 0x7f24b00047c0 2024-02-02T13:26:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.629+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d00b18a0 con 0x7f24b00047c0 2024-02-02T13:26:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.630+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d0066cf0 con 0x7f24b00047c0 2024-02-02T13:26:31.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.631+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f24c0004b80 con 0x7f24b00047c0 2024-02-02T13:26:31.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.632+0000 7f24c7400640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 0x7f24a806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.632+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24c0088360 con 0x7f24b00047c0 2024-02-02T13:26:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.632+0000 7f24c5e00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 0x7f24a806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:31.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.633+0000 7f24c5e00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 0x7f24a806c940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24b8036000 tx=0x7f24b802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:31.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.636+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f24c0053930 con 0x7f24b00047c0 2024-02-02T13:26:31.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.825+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24d00c3bf0 con 0x7f24a806a480 2024-02-02T13:26:31.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.847+0000 7f24c7400640 1 -- 172.21.15.91:0/2548369651 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273360 (secure 0 0 0) 0x7f24d00c3bf0 con 0x7f24a806a480 2024-02-02T13:26:31.852 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:31.853 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:26:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.853+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 msgr2=0x7f24a806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.853+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 0x7f24a806c940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24b8036000 tx=0x7f24b802e000 comp rx=0 tx=0).stop 2024-02-02T13:26:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.853+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 msgr2=0x7f24d0158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.853+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c7a0 tx=0x7f24c000cc60 comp rx=0 tx=0).stop 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.853+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 shutdown_connections 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24a806a480 0x7f24a806c940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d00c3400 0x7f24d01f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d00c4840 0x7f24d00c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 --2- 172.21.15.91:0/2548369651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24b00047c0 0x7f24d0158210 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 >> 172.21.15.91:0/2548369651 conn(0x7f24d00cec80 msgr2=0x7f24d01590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 shutdown_connections 2024-02-02T13:26:31.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:31.854+0000 7f24d5a00640 1 -- 172.21.15.91:0/2548369651 wait complete. 2024-02-02T13:26:31.874 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-02-02T13:26:30.269712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2788,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559711,"statfs":{"total":1600000000,"available":1597144864,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.320460"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.957227+0000","last_change":"2024-02-02T13:26:25.957227+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.957227+0000","last_undegraded":"2024-02-02T13:26:25.957227+0000","last_fullsized":"2024-02-02T13:26:25.957227+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:17:17.895670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.956634+0000","last_change":"2024-02-02T13:26:25.956634+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.956634+0000","last_undegraded":"2024-02-02T13:26:25.956634+0000","last_fullsized":"2024-02-02T13:26:25.956634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:51.467309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.957175+0000","last_change":"2024-02-02T13:26:25.957175+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.957175+0000","last_undegraded":"2024-02-02T13:26:25.957175+0000","last_fullsized":"2024-02-02T13:26:25.957175+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:38.626831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.962548+0000","last_change":"2024-02-02T13:26:25.962548+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.962548+0000","last_undegraded":"2024-02-02T13:26:25.962548+0000","last_fullsized":"2024-02-02T13:26:25.962548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:51.430990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.955449+0000","last_change":"2024-02-02T13:26:25.955449+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.955449+0000","last_undegraded":"2024-02-02T13:26:25.955449+0000","last_fullsized":"2024-02-02T13:26:25.955449+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:07.667150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.955629+0000","last_change":"2024-02-02T13:26:25.955629+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.955629+0000","last_undegraded":"2024-02-02T13:26:25.955629+0000","last_fullsized":"2024-02-02T13:26:25.955629+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:23:19.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.953807+0000","last_change":"2024-02-02T13:26:25.953807+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.953807+0000","last_undegraded":"2024-02-02T13:26:25.953807+0000","last_fullsized":"2024-02-02T13:26:25.953807+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:10.295155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.960885+0000","last_change":"2024-02-02T13:26:25.960885+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.960885+0000","last_undegraded":"2024-02-02T13:26:25.960885+0000","last_fullsized":"2024-02-02T13:26:25.960885+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:23:19.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.957500+0000","last_change":"2024-02-02T13:26:25.957500+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.957500+0000","last_undegraded":"2024-02-02T13:26:25.957500+0000","last_fullsized":"2024-02-02T13:26:25.957500+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:09:00.652859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.958566+0000","last_change":"2024-02-02T13:26:25.958566+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.958566+0000","last_undegraded":"2024-02-02T13:26:25.958566+0000","last_fullsized":"2024-02-02T13:26:25.958566+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:54:30.859760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.961403+0000","last_change":"2024-02-02T13:26:25.961403+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.961403+0000","last_undegraded":"2024-02-02T13:26:25.961403+0000","last_fullsized":"2024-02-02T13:26:25.961403+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:00.980983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.952698+0000","last_change":"2024-02-02T13:26:25.952698+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.952698+0000","last_undegraded":"2024-02-02T13:26:25.952698+0000","last_fullsized":"2024-02-02T13:26:25.952698+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:11:13.242188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.954865+0000","last_change":"2024-02-02T13:26:25.954865+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.954865+0000","last_undegraded":"2024-02-02T13:26:25.954865+0000","last_fullsized":"2024-02-02T13:26:25.954865+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:26.385659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.961900+0000","last_change":"2024-02-02T13:26:25.961900+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.961900+0000","last_undegraded":"2024-02-02T13:26:25.961900+0000","last_fullsized":"2024-02-02T13:26:25.961900+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:11.576495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.955119+0000","last_change":"2024-02-02T13:26:25.955119+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.955119+0000","last_undegraded":"2024-02-02T13:26:25.955119+0000","last_fullsized":"2024-02-02T13:26:25.955119+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:47:01.775499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.958497+0000","last_change":"2024-02-02T13:26:25.958497+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.958497+0000","last_undegraded":"2024-02-02T13:26:25.958497+0000","last_fullsized":"2024-02-02T13:26:25.958497+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:20:27.178679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.951700+0000","last_change":"2024-02-02T13:26:25.951700+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.951700+0000","last_undegraded":"2024-02-02T13:26:25.951700+0000","last_fullsized":"2024-02-02T13:26:25.951700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:07.068518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.957168+0000","last_change":"2024-02-02T13:26:25.957168+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.957168+0000","last_undegraded":"2024-02-02T13:26:25.957168+0000","last_fullsized":"2024-02-02T13:26:25.957168+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:15:43.398407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.952722+0000","last_change":"2024-02-02T13:26:25.952722+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.952722+0000","last_undegraded":"2024-02-02T13:26:25.952722+0000","last_fullsized":"2024-02-02T13:26:25.952722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:19.308272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.954675+0000","last_change":"2024-02-02T13:26:25.954675+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.954675+0000","last_undegraded":"2024-02-02T13:26:25.954675+0000","last_fullsized":"2024-02-02T13:26:25.954675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:18.844454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.956206+0000","last_change":"2024-02-02T13:26:25.956206+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.956206+0000","last_undegraded":"2024-02-02T13:26:25.956206+0000","last_fullsized":"2024-02-02T13:26:25.956206+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:03.021475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.952519+0000","last_change":"2024-02-02T13:26:25.952519+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.952519+0000","last_undegraded":"2024-02-02T13:26:25.952519+0000","last_fullsized":"2024-02-02T13:26:25.952519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:00.980983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.960340+0000","last_change":"2024-02-02T13:26:25.960340+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.960340+0000","last_undegraded":"2024-02-02T13:26:25.960340+0000","last_fullsized":"2024-02-02T13:26:25.960340+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:59.426053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.961132+0000","last_change":"2024-02-02T13:26:25.961132+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.961132+0000","last_undegraded":"2024-02-02T13:26:25.961132+0000","last_fullsized":"2024-02-02T13:26:25.961132+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:31.534949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.955180+0000","last_change":"2024-02-02T13:26:25.955180+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.955180+0000","last_undegraded":"2024-02-02T13:26:25.955180+0000","last_fullsized":"2024-02-02T13:26:25.955180+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:51.430990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.960196+0000","last_change":"2024-02-02T13:26:25.960196+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.960196+0000","last_undegraded":"2024-02-02T13:26:25.960196+0000","last_fullsized":"2024-02-02T13:26:25.960196+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:59.094888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.956255+0000","last_change":"2024-02-02T13:26:25.956255+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.956255+0000","last_undegraded":"2024-02-02T13:26:25.956255+0000","last_fullsized":"2024-02-02T13:26:25.956255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:08.259274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.959398+0000","last_change":"2024-02-02T13:26:25.959398+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.959398+0000","last_undegraded":"2024-02-02T13:26:25.959398+0000","last_fullsized":"2024-02-02T13:26:25.959398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:10.295155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.957496+0000","last_change":"2024-02-02T13:26:25.957496+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.957496+0000","last_undegraded":"2024-02-02T13:26:25.957496+0000","last_fullsized":"2024-02-02T13:26:25.957496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:41.348138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.955519+0000","last_change":"2024-02-02T13:26:25.955519+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.955519+0000","last_undegraded":"2024-02-02T13:26:25.955519+0000","last_fullsized":"2024-02-02T13:26:25.955519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:12:58.587187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.958800+0000","last_change":"2024-02-02T13:26:25.958800+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.958800+0000","last_undegraded":"2024-02-02T13:26:25.958800+0000","last_fullsized":"2024-02-02T13:26:25.958800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:28:25.071113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:26:26.949422+0000","last_change":"2024-02-02T13:26:26.949422+0000","last_active":"2024-02-02T13:26:26.949422+0000","last_peered":"2024-02-02T13:26:26.949422+0000","last_clean":"2024-02-02T13:26:26.949422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:26.949422+0000","last_undegraded":"2024-02-02T13:26:26.949422+0000","last_fullsized":"2024-02-02T13:26:26.949422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-02-02T13:26:25.952499+0000","last_change":"2024-02-02T13:26:25.952499+0000","last_active":"2024-02-02T13:26:25.932683+0000","last_peered":"2024-02-02T13:26:25.932683+0000","last_clean":"2024-02-02T13:26:25.932683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:26:25.952499+0000","last_undegraded":"2024-02-02T13:26:25.952499+0000","last_fullsized":"2024-02-02T13:26:25.952499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:59.094888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":187,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-02-02T13:26:22.916429+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:26:22.916429+0000","last_peered":"2024-02-02T13:26:22.916429+0000","last_clean":"2024-02-02T13:26:22.916429+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:26:22.916429+0000","last_undegraded":"2024-02-02T13:26:22.916429+0000","last_fullsized":"2024-02-02T13:26:22.916429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194528,"statfs":{"total":200000000,"available":199196914,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.105},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.1739999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.339},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.2070000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.5309999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":1.226},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.6619999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.385},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.429}]},{"osd":6,"last update":"Fri Feb 2 13:25:26 2024","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":3.6840000000000002,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.571}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":206,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195105,"statfs":{"total":200000000,"available":199788301,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.77500000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.175},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.4079999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.4510000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.6040000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.7270000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.655}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793099,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.238},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.145}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0429999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.3}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.391},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.3320000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.4950000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":201,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195111,"statfs":{"total":200000000,"available":199794118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.365}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.216},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.5089999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.736},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6439999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.861}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.552},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.6639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793099,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.77300000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.111},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.585,"5min":3.585,"15min":3.585},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.202}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.6240000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":207,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195104,"statfs":{"total":200000000,"available":199787282,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.038},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.85699999999999998}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.012}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.878},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":206,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195105,"statfs":{"total":200000000,"available":199788301,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.94899999999999995}]},{"osd":4,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.2909999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.504}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194534,"statfs":{"total":200000000,"available":199203750,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.8070000000000004,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.157}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":10.781000000000001,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.751}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.8410000000000002,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":10.651,"5min":10.651,"15min":10.651},"last":1.304},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.7869999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.339}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.258},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":10.708,"5min":10.708,"15min":10.708},"last":1.3839999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":10.74,"5min":10.74,"15min":10.74},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.1180000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:26:31.879 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:26:31.922 INFO:tasks.ceph.mds.a.smithi091.stderr:2024-02-02T13:26:31.915+0000 7f8a34000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:26:32.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.124+0000 7f8f89a00640 1 Processor -- start 2024-02-02T13:26:32.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.124+0000 7f8f89a00640 1 -- start start 2024-02-02T13:26:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.125+0000 7f8f89a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f8406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f89a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8415d870 con 0x7f8f840acfe0 2024-02-02T13:26:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f89a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8406bce0 con 0x7f8f8415d490 2024-02-02T13:26:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f89a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84161220 con 0x7f8f840b0e10 2024-02-02T13:26:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f7be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8f840acfe0 0x7f8f840b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34470/0 (socket says 172.21.15.91:34470) 2024-02-02T13:26:32.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f7be00640 1 -- 172.21.15.91:0/923988369 learned_addr learned my addr 172.21.15.91:0/923988369 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:32.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f83400640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f8406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391361914 0 0) 0x7f8f8415d870 con 0x7f8f840acfe0 2024-02-02T13:26:32.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.126+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f64003520 con 0x7f8f840acfe0 2024-02-02T13:26:32.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3208838067 0 0) 0x7f8f64003520 con 0x7f8f840acfe0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 msgr2=0x7f8f8406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f8406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 >> v1:172.21.15.113:6789/0 conn(0x7f8f8415d490 legacy=0x7f8f840ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840ad490 con 0x7f8f840acfe0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8f6c001670 con 0x7f8f840acfe0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 >> v1:172.21.15.91:6789/0 conn(0x7f8f840acfe0 legacy=0x7f8f840b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f640039a0 0x7f8f64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84161220 con 0x7f8f640047c0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400ba90 con 0x7f8f640039a0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400bc50 con 0x7f8f840b0e10 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f7be00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.127+0000 7f8f82a00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f640039a0 0x7f8f64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f83400640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f7be00640 1 -- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 msgr2=0x7f8f64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f7be00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b0e10 0x7f8f64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f7be00640 1 -- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f640039a0 msgr2=0x7f8f64003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f7be00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f640039a0 0x7f8f64003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f7be00640 1 -- 172.21.15.91:0/923988369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64003520 con 0x7f8f640047c0 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.128+0000 7f8f82a00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f640039a0 0x7f8f64003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f7be00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f64004c40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001eb0 tx=0x7f8f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e911361cadadfaf1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c005ad0 con 0x7f8f640047c0 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c005c50 con 0x7f8f640047c0 2024-02-02T13:26:32.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f82000640 1 -- 172.21.15.91:0/923988369 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c02c7a0 con 0x7f8f640047c0 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f89a00640 1 -- 172.21.15.91:0/923988369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 msgr2=0x7f8f64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f89a00640 1 --2- 172.21.15.91:0/923988369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f64004c40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001eb0 tx=0x7f8f640057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.129+0000 7f8f83400640 1 -- 172.21.15.91:0/923988369 reap_dead start 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.130+0000 7f8f89a00640 1 -- 172.21.15.91:0/923988369 shutdown_connections 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.130+0000 7f8f89a00640 1 -- 172.21.15.91:0/923988369 >> 172.21.15.91:0/923988369 conn(0x7f8f840cec80 msgr2=0x7f8f840cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:32.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.130+0000 7f8f89a00640 1 -- 172.21.15.91:0/923988369 shutdown_connections 2024-02-02T13:26:32.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.130+0000 7f8f89a00640 1 -- 172.21.15.91:0/923988369 wait complete. 2024-02-02T13:26:32.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.132+0000 7f8f89a00640 1 Processor -- start 2024-02-02T13:26:32.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.132+0000 7f8f89a00640 1 -- start start 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b2250 0x7f8f8415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 0x7f8f8415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8415ec20 con 0x7f8f640047c0 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8415bd70 con 0x7f8f8415b3a0 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f89a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840b2740 con 0x7f8f840b2250 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f82a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f82a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47608/0 (socket says 172.21.15.91:47608) 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.133+0000 7f8f82a00640 1 -- 172.21.15.91:0/1469380776 learned_addr learned my addr 172.21.15.91:0/1469380776 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f83400640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 0x7f8f8415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f7be00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b2250 0x7f8f8415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 -- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b2250 msgr2=0x7f8f8415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b2250 0x7f8f8415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 -- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 msgr2=0x7f8f8415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 0x7f8f8415b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 -- 172.21.15.91:0/1469380776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840b2a20 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f83400640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 0x7f8f8415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f82a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400e480 tx=0x7f8f7400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740052e0 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74005440 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003830 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f840b2d00 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.134+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8415a230 con 0x7f8f640047c0 2024-02-02T13:26:32.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.136+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f84066cf0 con 0x7f8f640047c0 2024-02-02T13:26:32.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.136+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8f74003990 con 0x7f8f640047c0 2024-02-02T13:26:32.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.137+0000 7f8f7aa00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 0x7f8f5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.137+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f74088610 con 0x7f8f640047c0 2024-02-02T13:26:32.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.138+0000 7f8f7be00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 0x7f8f5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.138+0000 7f8f7be00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 0x7f8f5c06c8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c034000 tx=0x7f8f6c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.142+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8f74053c90 con 0x7f8f640047c0 2024-02-02T13:26:32.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.326+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f8406b690 con 0x7f8f5c06a430 2024-02-02T13:26:32.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.340+0000 7f8f7aa00640 1 -- 172.21.15.91:0/1469380776 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282155 (secure 0 0 0) 0x7f8f6c005dc0 con 0x7f8f5c06a430 2024-02-02T13:26:32.345 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:32.346 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 msgr2=0x7f8f5c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 0x7f8f5c06c8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c034000 tx=0x7f8f6c009030 comp rx=0 tx=0).stop 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 msgr2=0x7f8f84157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400e480 tx=0x7f8f7400e940 comp rx=0 tx=0).stop 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 shutdown_connections 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f5c06a430 0x7f8f5c06c8f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840b2250 0x7f8f8415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8415b3a0 0x7f8f8415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 --2- 172.21.15.91:0/1469380776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f640047c0 0x7f8f84157330 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 >> 172.21.15.91:0/1469380776 conn(0x7f8f840cec80 msgr2=0x7f8f84157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:32.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 shutdown_connections 2024-02-02T13:26:32.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.345+0000 7f8f89a00640 1 -- 172.21.15.91:0/1469380776 wait complete. 2024-02-02T13:26:32.367 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-02-02T13:26:32.270457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2966,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559533,"statfs":{"total":1600000000,"available":1596962136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510360+0000","last_change":"2024-02-02T13:26:27.460615+0000","last_active":"2024-02-02T13:26:28.510360+0000","last_peered":"2024-02-02T13:26:28.510360+0000","last_clean":"2024-02-02T13:26:28.510360+0000","last_became_active":"2024-02-02T13:26:27.459840+0000","last_became_peered":"2024-02-02T13:26:27.459840+0000","last_unstale":"2024-02-02T13:26:28.510360+0000","last_undegraded":"2024-02-02T13:26:28.510360+0000","last_fullsized":"2024-02-02T13:26:28.510360+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:47:26.540809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458926+0000","last_change":"2024-02-02T13:26:27.468756+0000","last_active":"2024-02-02T13:26:28.458926+0000","last_peered":"2024-02-02T13:26:28.458926+0000","last_clean":"2024-02-02T13:26:28.458926+0000","last_became_active":"2024-02-02T13:26:27.468538+0000","last_became_peered":"2024-02-02T13:26:27.468538+0000","last_unstale":"2024-02-02T13:26:28.458926+0000","last_undegraded":"2024-02-02T13:26:28.458926+0000","last_fullsized":"2024-02-02T13:26:28.458926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:15.188599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510002+0000","last_change":"2024-02-02T13:26:27.474683+0000","last_active":"2024-02-02T13:26:28.510002+0000","last_peered":"2024-02-02T13:26:28.510002+0000","last_clean":"2024-02-02T13:26:28.510002+0000","last_became_active":"2024-02-02T13:26:27.472924+0000","last_became_peered":"2024-02-02T13:26:27.472924+0000","last_unstale":"2024-02-02T13:26:28.510002+0000","last_undegraded":"2024-02-02T13:26:28.510002+0000","last_fullsized":"2024-02-02T13:26:28.510002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:01.997722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508505+0000","last_change":"2024-02-02T13:26:27.476431+0000","last_active":"2024-02-02T13:26:28.508505+0000","last_peered":"2024-02-02T13:26:28.508505+0000","last_clean":"2024-02-02T13:26:28.508505+0000","last_became_active":"2024-02-02T13:26:27.475942+0000","last_became_peered":"2024-02-02T13:26:27.475942+0000","last_unstale":"2024-02-02T13:26:28.508505+0000","last_undegraded":"2024-02-02T13:26:28.508505+0000","last_fullsized":"2024-02-02T13:26:28.508505+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:16:00.034179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.459539+0000","last_change":"2024-02-02T13:26:27.466759+0000","last_active":"2024-02-02T13:26:28.459539+0000","last_peered":"2024-02-02T13:26:28.459539+0000","last_clean":"2024-02-02T13:26:28.459539+0000","last_became_active":"2024-02-02T13:26:27.466549+0000","last_became_peered":"2024-02-02T13:26:27.466549+0000","last_unstale":"2024-02-02T13:26:28.459539+0000","last_undegraded":"2024-02-02T13:26:28.459539+0000","last_fullsized":"2024-02-02T13:26:28.459539+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:27.402398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457192+0000","last_change":"2024-02-02T13:26:27.463721+0000","last_active":"2024-02-02T13:26:28.457192+0000","last_peered":"2024-02-02T13:26:28.457192+0000","last_clean":"2024-02-02T13:26:28.457192+0000","last_became_active":"2024-02-02T13:26:27.463483+0000","last_became_peered":"2024-02-02T13:26:27.463483+0000","last_unstale":"2024-02-02T13:26:28.457192+0000","last_undegraded":"2024-02-02T13:26:28.457192+0000","last_fullsized":"2024-02-02T13:26:28.457192+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:20:53.316157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509770+0000","last_change":"2024-02-02T13:26:27.459326+0000","last_active":"2024-02-02T13:26:28.509770+0000","last_peered":"2024-02-02T13:26:28.509770+0000","last_clean":"2024-02-02T13:26:28.509770+0000","last_became_active":"2024-02-02T13:26:27.458693+0000","last_became_peered":"2024-02-02T13:26:27.458693+0000","last_unstale":"2024-02-02T13:26:28.509770+0000","last_undegraded":"2024-02-02T13:26:28.509770+0000","last_fullsized":"2024-02-02T13:26:28.509770+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:15.188599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508143+0000","last_change":"2024-02-02T13:26:27.475570+0000","last_active":"2024-02-02T13:26:28.508143+0000","last_peered":"2024-02-02T13:26:28.508143+0000","last_clean":"2024-02-02T13:26:28.508143+0000","last_became_active":"2024-02-02T13:26:27.475361+0000","last_became_peered":"2024-02-02T13:26:27.475361+0000","last_unstale":"2024-02-02T13:26:28.508143+0000","last_undegraded":"2024-02-02T13:26:28.508143+0000","last_fullsized":"2024-02-02T13:26:28.508143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:10.261522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508603+0000","last_change":"2024-02-02T13:26:27.463859+0000","last_active":"2024-02-02T13:26:28.508603+0000","last_peered":"2024-02-02T13:26:28.508603+0000","last_clean":"2024-02-02T13:26:28.508603+0000","last_became_active":"2024-02-02T13:26:27.463088+0000","last_became_peered":"2024-02-02T13:26:27.463088+0000","last_unstale":"2024-02-02T13:26:28.508603+0000","last_undegraded":"2024-02-02T13:26:28.508603+0000","last_fullsized":"2024-02-02T13:26:28.508603+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:11:15.188599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508423+0000","last_change":"2024-02-02T13:26:27.476353+0000","last_active":"2024-02-02T13:26:28.508423+0000","last_peered":"2024-02-02T13:26:28.508423+0000","last_clean":"2024-02-02T13:26:28.508423+0000","last_became_active":"2024-02-02T13:26:27.475926+0000","last_became_peered":"2024-02-02T13:26:27.475926+0000","last_unstale":"2024-02-02T13:26:28.508423+0000","last_undegraded":"2024-02-02T13:26:28.508423+0000","last_fullsized":"2024-02-02T13:26:28.508423+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:47:02.792238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508324+0000","last_change":"2024-02-02T13:26:27.462973+0000","last_active":"2024-02-02T13:26:28.508324+0000","last_peered":"2024-02-02T13:26:28.508324+0000","last_clean":"2024-02-02T13:26:28.508324+0000","last_became_active":"2024-02-02T13:26:27.462034+0000","last_became_peered":"2024-02-02T13:26:27.462034+0000","last_unstale":"2024-02-02T13:26:28.508324+0000","last_undegraded":"2024-02-02T13:26:28.508324+0000","last_fullsized":"2024-02-02T13:26:28.508324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:20.325011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.459175+0000","last_change":"2024-02-02T13:26:27.466705+0000","last_active":"2024-02-02T13:26:28.459175+0000","last_peered":"2024-02-02T13:26:28.459175+0000","last_clean":"2024-02-02T13:26:28.459175+0000","last_became_active":"2024-02-02T13:26:27.466335+0000","last_became_peered":"2024-02-02T13:26:27.466335+0000","last_unstale":"2024-02-02T13:26:28.459175+0000","last_undegraded":"2024-02-02T13:26:28.459175+0000","last_fullsized":"2024-02-02T13:26:28.459175+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:47:26.540809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458519+0000","last_change":"2024-02-02T13:26:27.465955+0000","last_active":"2024-02-02T13:26:28.458519+0000","last_peered":"2024-02-02T13:26:28.458519+0000","last_clean":"2024-02-02T13:26:28.458519+0000","last_became_active":"2024-02-02T13:26:27.465251+0000","last_became_peered":"2024-02-02T13:26:27.465251+0000","last_unstale":"2024-02-02T13:26:28.458519+0000","last_undegraded":"2024-02-02T13:26:28.458519+0000","last_fullsized":"2024-02-02T13:26:28.458519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:28:26.087853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508862+0000","last_change":"2024-02-02T13:26:27.463609+0000","last_active":"2024-02-02T13:26:28.508862+0000","last_peered":"2024-02-02T13:26:28.508862+0000","last_clean":"2024-02-02T13:26:28.508862+0000","last_became_active":"2024-02-02T13:26:27.462589+0000","last_became_peered":"2024-02-02T13:26:27.462589+0000","last_unstale":"2024-02-02T13:26:28.508862+0000","last_undegraded":"2024-02-02T13:26:28.508862+0000","last_fullsized":"2024-02-02T13:26:28.508862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:11.311894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457182+0000","last_change":"2024-02-02T13:26:27.471551+0000","last_active":"2024-02-02T13:26:28.457182+0000","last_peered":"2024-02-02T13:26:28.457182+0000","last_clean":"2024-02-02T13:26:28.457182+0000","last_became_active":"2024-02-02T13:26:27.471322+0000","last_became_peered":"2024-02-02T13:26:27.471322+0000","last_unstale":"2024-02-02T13:26:28.457182+0000","last_undegraded":"2024-02-02T13:26:28.457182+0000","last_fullsized":"2024-02-02T13:26:28.457182+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:28:01.655104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.512073+0000","last_change":"2024-02-02T13:26:27.473469+0000","last_active":"2024-02-02T13:26:28.512073+0000","last_peered":"2024-02-02T13:26:28.512073+0000","last_clean":"2024-02-02T13:26:28.512073+0000","last_became_active":"2024-02-02T13:26:27.472749+0000","last_became_peered":"2024-02-02T13:26:27.472749+0000","last_unstale":"2024-02-02T13:26:28.512073+0000","last_undegraded":"2024-02-02T13:26:28.512073+0000","last_fullsized":"2024-02-02T13:26:28.512073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:12.593235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510462+0000","last_change":"2024-02-02T13:26:27.476471+0000","last_active":"2024-02-02T13:26:28.510462+0000","last_peered":"2024-02-02T13:26:28.510462+0000","last_clean":"2024-02-02T13:26:28.510462+0000","last_became_active":"2024-02-02T13:26:27.473664+0000","last_became_peered":"2024-02-02T13:26:27.473664+0000","last_unstale":"2024-02-02T13:26:28.510462+0000","last_undegraded":"2024-02-02T13:26:28.510462+0000","last_fullsized":"2024-02-02T13:26:28.510462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:19.861194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510336+0000","last_change":"2024-02-02T13:26:27.477333+0000","last_active":"2024-02-02T13:26:28.510336+0000","last_peered":"2024-02-02T13:26:28.510336+0000","last_clean":"2024-02-02T13:26:28.510336+0000","last_became_active":"2024-02-02T13:26:27.474377+0000","last_became_peered":"2024-02-02T13:26:27.474377+0000","last_unstale":"2024-02-02T13:26:28.510336+0000","last_undegraded":"2024-02-02T13:26:28.510336+0000","last_fullsized":"2024-02-02T13:26:28.510336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:17:18.912409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.511168+0000","last_change":"2024-02-02T13:26:27.474231+0000","last_active":"2024-02-02T13:26:28.511168+0000","last_peered":"2024-02-02T13:26:28.511168+0000","last_clean":"2024-02-02T13:26:28.511168+0000","last_became_active":"2024-02-02T13:26:27.472060+0000","last_became_peered":"2024-02-02T13:26:27.472060+0000","last_unstale":"2024-02-02T13:26:28.511168+0000","last_undegraded":"2024-02-02T13:26:28.511168+0000","last_fullsized":"2024-02-02T13:26:28.511168+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:32.551689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458791+0000","last_change":"2024-02-02T13:26:27.471026+0000","last_active":"2024-02-02T13:26:28.458791+0000","last_peered":"2024-02-02T13:26:28.458791+0000","last_clean":"2024-02-02T13:26:28.458791+0000","last_became_active":"2024-02-02T13:26:27.470828+0000","last_became_peered":"2024-02-02T13:26:27.470828+0000","last_unstale":"2024-02-02T13:26:28.458791+0000","last_undegraded":"2024-02-02T13:26:28.458791+0000","last_fullsized":"2024-02-02T13:26:28.458791+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:09:01.669599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457325+0000","last_change":"2024-02-02T13:26:27.473486+0000","last_active":"2024-02-02T13:26:28.457325+0000","last_peered":"2024-02-02T13:26:28.457325+0000","last_clean":"2024-02-02T13:26:28.457325+0000","last_became_active":"2024-02-02T13:26:27.473263+0000","last_became_peered":"2024-02-02T13:26:27.473263+0000","last_unstale":"2024-02-02T13:26:28.457325+0000","last_undegraded":"2024-02-02T13:26:28.457325+0000","last_fullsized":"2024-02-02T13:26:28.457325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:07:25.970661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508377+0000","last_change":"2024-02-02T13:26:27.477106+0000","last_active":"2024-02-02T13:26:28.508377+0000","last_peered":"2024-02-02T13:26:28.508377+0000","last_clean":"2024-02-02T13:26:28.508377+0000","last_became_active":"2024-02-02T13:26:27.476707+0000","last_became_peered":"2024-02-02T13:26:27.476707+0000","last_unstale":"2024-02-02T13:26:28.508377+0000","last_undegraded":"2024-02-02T13:26:28.508377+0000","last_fullsized":"2024-02-02T13:26:28.508377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:47:26.540809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.511132+0000","last_change":"2024-02-02T13:26:27.477314+0000","last_active":"2024-02-02T13:26:28.511132+0000","last_peered":"2024-02-02T13:26:28.511132+0000","last_clean":"2024-02-02T13:26:28.511132+0000","last_became_active":"2024-02-02T13:26:27.476858+0000","last_became_peered":"2024-02-02T13:26:27.476858+0000","last_unstale":"2024-02-02T13:26:28.511132+0000","last_undegraded":"2024-02-02T13:26:28.511132+0000","last_fullsized":"2024-02-02T13:26:28.511132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:19:00.111627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510466+0000","last_change":"2024-02-02T13:26:27.476351+0000","last_active":"2024-02-02T13:26:28.510466+0000","last_peered":"2024-02-02T13:26:28.510466+0000","last_clean":"2024-02-02T13:26:28.510466+0000","last_became_active":"2024-02-02T13:26:27.475836+0000","last_became_peered":"2024-02-02T13:26:27.475836+0000","last_unstale":"2024-02-02T13:26:28.510466+0000","last_undegraded":"2024-02-02T13:26:28.510466+0000","last_fullsized":"2024-02-02T13:26:28.510466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:20.325011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.513307+0000","last_change":"2024-02-02T13:26:27.472738+0000","last_active":"2024-02-02T13:26:28.513307+0000","last_peered":"2024-02-02T13:26:28.513307+0000","last_clean":"2024-02-02T13:26:28.513307+0000","last_became_active":"2024-02-02T13:26:27.472329+0000","last_became_peered":"2024-02-02T13:26:27.472329+0000","last_unstale":"2024-02-02T13:26:28.513307+0000","last_undegraded":"2024-02-02T13:26:28.513307+0000","last_fullsized":"2024-02-02T13:26:28.513307+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:09.276014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458421+0000","last_change":"2024-02-02T13:26:27.467130+0000","last_active":"2024-02-02T13:26:28.458421+0000","last_peered":"2024-02-02T13:26:28.458421+0000","last_clean":"2024-02-02T13:26:28.458421+0000","last_became_active":"2024-02-02T13:26:27.466860+0000","last_became_peered":"2024-02-02T13:26:27.466860+0000","last_unstale":"2024-02-02T13:26:28.458421+0000","last_undegraded":"2024-02-02T13:26:28.458421+0000","last_fullsized":"2024-02-02T13:26:28.458421+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:15:44.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510716+0000","last_change":"2024-02-02T13:26:27.476610+0000","last_active":"2024-02-02T13:26:28.510716+0000","last_peered":"2024-02-02T13:26:28.510716+0000","last_clean":"2024-02-02T13:26:28.510716+0000","last_became_active":"2024-02-02T13:26:27.476176+0000","last_became_peered":"2024-02-02T13:26:27.476176+0000","last_unstale":"2024-02-02T13:26:28.510716+0000","last_undegraded":"2024-02-02T13:26:28.510716+0000","last_fullsized":"2024-02-02T13:26:28.510716+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:52.447730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458854+0000","last_change":"2024-02-02T13:26:27.467172+0000","last_active":"2024-02-02T13:26:28.458854+0000","last_peered":"2024-02-02T13:26:28.458854+0000","last_clean":"2024-02-02T13:26:28.458854+0000","last_became_active":"2024-02-02T13:26:27.466956+0000","last_became_peered":"2024-02-02T13:26:27.466956+0000","last_unstale":"2024-02-02T13:26:28.458854+0000","last_undegraded":"2024-02-02T13:26:28.458854+0000","last_fullsized":"2024-02-02T13:26:28.458854+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:04.038214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510713+0000","last_change":"2024-02-02T13:26:27.474597+0000","last_active":"2024-02-02T13:26:28.510713+0000","last_peered":"2024-02-02T13:26:28.510713+0000","last_clean":"2024-02-02T13:26:28.510713+0000","last_became_active":"2024-02-02T13:26:27.472919+0000","last_became_peered":"2024-02-02T13:26:27.472919+0000","last_unstale":"2024-02-02T13:26:28.510713+0000","last_undegraded":"2024-02-02T13:26:28.510713+0000","last_fullsized":"2024-02-02T13:26:28.510713+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:58:22.859449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509883+0000","last_change":"2024-02-02T13:26:27.477775+0000","last_active":"2024-02-02T13:26:28.509883+0000","last_peered":"2024-02-02T13:26:28.509883+0000","last_clean":"2024-02-02T13:26:28.509883+0000","last_became_active":"2024-02-02T13:26:27.477318+0000","last_became_peered":"2024-02-02T13:26:27.477318+0000","last_unstale":"2024-02-02T13:26:28.509883+0000","last_undegraded":"2024-02-02T13:26:28.509883+0000","last_fullsized":"2024-02-02T13:26:28.509883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:32.551689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457418+0000","last_change":"2024-02-02T13:26:27.455841+0000","last_active":"2024-02-02T13:26:28.457418+0000","last_peered":"2024-02-02T13:26:28.457418+0000","last_clean":"2024-02-02T13:26:28.457418+0000","last_became_active":"2024-02-02T13:26:27.455618+0000","last_became_peered":"2024-02-02T13:26:27.455618+0000","last_unstale":"2024-02-02T13:26:28.457418+0000","last_undegraded":"2024-02-02T13:26:28.457418+0000","last_fullsized":"2024-02-02T13:26:28.457418+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:56.639348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458788+0000","last_change":"2024-02-02T13:26:27.470074+0000","last_active":"2024-02-02T13:26:28.458788+0000","last_peered":"2024-02-02T13:26:28.458788+0000","last_clean":"2024-02-02T13:26:28.458788+0000","last_became_active":"2024-02-02T13:26:27.469816+0000","last_became_peered":"2024-02-02T13:26:27.469816+0000","last_unstale":"2024-02-02T13:26:28.458788+0000","last_undegraded":"2024-02-02T13:26:28.458788+0000","last_fullsized":"2024-02-02T13:26:28.458788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:23:20.023082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508451+0000","last_change":"2024-02-02T13:26:27.456469+0000","last_active":"2024-02-02T13:26:28.508451+0000","last_peered":"2024-02-02T13:26:28.508451+0000","last_clean":"2024-02-02T13:26:28.508451+0000","last_became_active":"2024-02-02T13:26:27.456243+0000","last_became_peered":"2024-02-02T13:26:27.456243+0000","last_unstale":"2024-02-02T13:26:28.508451+0000","last_undegraded":"2024-02-02T13:26:28.508451+0000","last_fullsized":"2024-02-02T13:26:28.508451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:01.997722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.474266+0000","last_change":"2024-02-02T13:26:26.985400+0000","last_active":"2024-02-02T13:26:27.474266+0000","last_peered":"2024-02-02T13:26:27.474266+0000","last_clean":"2024-02-02T13:26:27.474266+0000","last_became_active":"2024-02-02T13:26:26.984846+0000","last_became_peered":"2024-02-02T13:26:26.984846+0000","last_unstale":"2024-02-02T13:26:27.474266+0000","last_undegraded":"2024-02-02T13:26:27.474266+0000","last_fullsized":"2024-02-02T13:26:27.474266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:17:17.895670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.511553+0000","last_change":"2024-02-02T13:26:27.473184+0000","last_active":"2024-02-02T13:26:28.511553+0000","last_peered":"2024-02-02T13:26:28.511553+0000","last_clean":"2024-02-02T13:26:28.511553+0000","last_became_active":"2024-02-02T13:26:27.472786+0000","last_became_peered":"2024-02-02T13:26:27.472786+0000","last_unstale":"2024-02-02T13:26:28.511553+0000","last_undegraded":"2024-02-02T13:26:28.511553+0000","last_fullsized":"2024-02-02T13:26:28.511553+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:30.930142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.476035+0000","last_change":"2024-02-02T13:26:26.989866+0000","last_active":"2024-02-02T13:26:27.476035+0000","last_peered":"2024-02-02T13:26:27.476035+0000","last_clean":"2024-02-02T13:26:27.476035+0000","last_became_active":"2024-02-02T13:26:26.989497+0000","last_became_peered":"2024-02-02T13:26:26.989497+0000","last_unstale":"2024-02-02T13:26:27.476035+0000","last_undegraded":"2024-02-02T13:26:27.476035+0000","last_fullsized":"2024-02-02T13:26:27.476035+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:51.467309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.459139+0000","last_change":"2024-02-02T13:26:27.455129+0000","last_active":"2024-02-02T13:26:28.459139+0000","last_peered":"2024-02-02T13:26:28.459139+0000","last_clean":"2024-02-02T13:26:28.459139+0000","last_became_active":"2024-02-02T13:26:27.454376+0000","last_became_peered":"2024-02-02T13:26:27.454376+0000","last_unstale":"2024-02-02T13:26:28.459139+0000","last_undegraded":"2024-02-02T13:26:28.459139+0000","last_fullsized":"2024-02-02T13:26:28.459139+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:49:10.338105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.465862+0000","last_change":"2024-02-02T13:26:26.987675+0000","last_active":"2024-02-02T13:26:27.465862+0000","last_peered":"2024-02-02T13:26:27.465862+0000","last_clean":"2024-02-02T13:26:27.465862+0000","last_became_active":"2024-02-02T13:26:26.987189+0000","last_became_peered":"2024-02-02T13:26:26.987189+0000","last_unstale":"2024-02-02T13:26:27.465862+0000","last_undegraded":"2024-02-02T13:26:27.465862+0000","last_fullsized":"2024-02-02T13:26:27.465862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:38.626831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508049+0000","last_change":"2024-02-02T13:26:27.474295+0000","last_active":"2024-02-02T13:26:28.508049+0000","last_peered":"2024-02-02T13:26:28.508049+0000","last_clean":"2024-02-02T13:26:28.508049+0000","last_became_active":"2024-02-02T13:26:27.474007+0000","last_became_peered":"2024-02-02T13:26:27.474007+0000","last_unstale":"2024-02-02T13:26:28.508049+0000","last_undegraded":"2024-02-02T13:26:28.508049+0000","last_fullsized":"2024-02-02T13:26:28.508049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:11.311894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.479733+0000","last_change":"2024-02-02T13:26:26.987847+0000","last_active":"2024-02-02T13:26:27.479733+0000","last_peered":"2024-02-02T13:26:27.479733+0000","last_clean":"2024-02-02T13:26:27.479733+0000","last_became_active":"2024-02-02T13:26:26.985044+0000","last_became_peered":"2024-02-02T13:26:26.985044+0000","last_unstale":"2024-02-02T13:26:27.479733+0000","last_undegraded":"2024-02-02T13:26:27.479733+0000","last_fullsized":"2024-02-02T13:26:27.479733+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:51.430990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509767+0000","last_change":"2024-02-02T13:26:27.477734+0000","last_active":"2024-02-02T13:26:28.509767+0000","last_peered":"2024-02-02T13:26:28.509767+0000","last_clean":"2024-02-02T13:26:28.509767+0000","last_became_active":"2024-02-02T13:26:27.475379+0000","last_became_peered":"2024-02-02T13:26:27.475379+0000","last_unstale":"2024-02-02T13:26:28.509767+0000","last_undegraded":"2024-02-02T13:26:28.509767+0000","last_fullsized":"2024-02-02T13:26:28.509767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:12.593235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.474763+0000","last_change":"2024-02-02T13:26:26.991654+0000","last_active":"2024-02-02T13:26:27.474763+0000","last_peered":"2024-02-02T13:26:27.474763+0000","last_clean":"2024-02-02T13:26:27.474763+0000","last_became_active":"2024-02-02T13:26:26.991343+0000","last_became_peered":"2024-02-02T13:26:26.991343+0000","last_unstale":"2024-02-02T13:26:27.474763+0000","last_undegraded":"2024-02-02T13:26:27.474763+0000","last_fullsized":"2024-02-02T13:26:27.474763+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:07.667150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508463+0000","last_change":"2024-02-02T13:26:27.467614+0000","last_active":"2024-02-02T13:26:28.508463+0000","last_peered":"2024-02-02T13:26:28.508463+0000","last_clean":"2024-02-02T13:26:28.508463+0000","last_became_active":"2024-02-02T13:26:27.467394+0000","last_became_peered":"2024-02-02T13:26:27.467394+0000","last_unstale":"2024-02-02T13:26:28.508463+0000","last_undegraded":"2024-02-02T13:26:28.508463+0000","last_fullsized":"2024-02-02T13:26:28.508463+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:19.861194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.478669+0000","last_change":"2024-02-02T13:26:26.988252+0000","last_active":"2024-02-02T13:26:27.478669+0000","last_peered":"2024-02-02T13:26:27.478669+0000","last_clean":"2024-02-02T13:26:27.478669+0000","last_became_active":"2024-02-02T13:26:26.987105+0000","last_became_peered":"2024-02-02T13:26:26.987105+0000","last_unstale":"2024-02-02T13:26:27.478669+0000","last_undegraded":"2024-02-02T13:26:27.478669+0000","last_fullsized":"2024-02-02T13:26:27.478669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:23:19.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508506+0000","last_change":"2024-02-02T13:26:27.476166+0000","last_active":"2024-02-02T13:26:28.508506+0000","last_peered":"2024-02-02T13:26:28.508506+0000","last_clean":"2024-02-02T13:26:28.508506+0000","last_became_active":"2024-02-02T13:26:27.475621+0000","last_became_peered":"2024-02-02T13:26:27.475621+0000","last_unstale":"2024-02-02T13:26:28.508506+0000","last_undegraded":"2024-02-02T13:26:28.508506+0000","last_fullsized":"2024-02-02T13:26:28.508506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:17:18.912409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.448670+0000","last_change":"2024-02-02T13:26:26.989877+0000","last_active":"2024-02-02T13:26:27.448670+0000","last_peered":"2024-02-02T13:26:27.448670+0000","last_clean":"2024-02-02T13:26:27.448670+0000","last_became_active":"2024-02-02T13:26:26.989245+0000","last_became_peered":"2024-02-02T13:26:26.989245+0000","last_unstale":"2024-02-02T13:26:27.448670+0000","last_undegraded":"2024-02-02T13:26:27.448670+0000","last_fullsized":"2024-02-02T13:26:27.448670+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:10.295155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510947+0000","last_change":"2024-02-02T13:26:27.474394+0000","last_active":"2024-02-02T13:26:28.510947+0000","last_peered":"2024-02-02T13:26:28.510947+0000","last_clean":"2024-02-02T13:26:28.510947+0000","last_became_active":"2024-02-02T13:26:27.472515+0000","last_became_peered":"2024-02-02T13:26:27.472515+0000","last_unstale":"2024-02-02T13:26:28.510947+0000","last_undegraded":"2024-02-02T13:26:28.510947+0000","last_fullsized":"2024-02-02T13:26:28.510947+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:12:59.603927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.460526+0000","last_change":"2024-02-02T13:26:26.995755+0000","last_active":"2024-02-02T13:26:27.460526+0000","last_peered":"2024-02-02T13:26:27.460526+0000","last_clean":"2024-02-02T13:26:27.460526+0000","last_became_active":"2024-02-02T13:26:26.994875+0000","last_became_peered":"2024-02-02T13:26:26.994875+0000","last_unstale":"2024-02-02T13:26:27.460526+0000","last_undegraded":"2024-02-02T13:26:27.460526+0000","last_fullsized":"2024-02-02T13:26:27.460526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:23:19.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510935+0000","last_change":"2024-02-02T13:26:27.473117+0000","last_active":"2024-02-02T13:26:28.510935+0000","last_peered":"2024-02-02T13:26:28.510935+0000","last_clean":"2024-02-02T13:26:28.510935+0000","last_became_active":"2024-02-02T13:26:27.472814+0000","last_became_peered":"2024-02-02T13:26:27.472814+0000","last_unstale":"2024-02-02T13:26:28.510935+0000","last_undegraded":"2024-02-02T13:26:28.510935+0000","last_fullsized":"2024-02-02T13:26:28.510935+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:47:02.792238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.454284+0000","last_change":"2024-02-02T13:26:26.991578+0000","last_active":"2024-02-02T13:26:27.454284+0000","last_peered":"2024-02-02T13:26:27.454284+0000","last_clean":"2024-02-02T13:26:27.454284+0000","last_became_active":"2024-02-02T13:26:26.991270+0000","last_became_peered":"2024-02-02T13:26:26.991270+0000","last_unstale":"2024-02-02T13:26:27.454284+0000","last_undegraded":"2024-02-02T13:26:27.454284+0000","last_fullsized":"2024-02-02T13:26:27.454284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:09:00.652859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510606+0000","last_change":"2024-02-02T13:26:27.463124+0000","last_active":"2024-02-02T13:26:28.510606+0000","last_peered":"2024-02-02T13:26:28.510606+0000","last_clean":"2024-02-02T13:26:28.510606+0000","last_became_active":"2024-02-02T13:26:27.462384+0000","last_became_peered":"2024-02-02T13:26:27.462384+0000","last_unstale":"2024-02-02T13:26:28.510606+0000","last_undegraded":"2024-02-02T13:26:28.510606+0000","last_fullsized":"2024-02-02T13:26:28.510606+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:30.604034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.459160+0000","last_change":"2024-02-02T13:26:26.992488+0000","last_active":"2024-02-02T13:26:27.459160+0000","last_peered":"2024-02-02T13:26:27.459160+0000","last_clean":"2024-02-02T13:26:27.459160+0000","last_became_active":"2024-02-02T13:26:26.992147+0000","last_became_peered":"2024-02-02T13:26:26.992147+0000","last_unstale":"2024-02-02T13:26:27.459160+0000","last_undegraded":"2024-02-02T13:26:27.459160+0000","last_fullsized":"2024-02-02T13:26:27.459160+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:54:30.859760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457090+0000","last_change":"2024-02-02T13:26:27.471059+0000","last_active":"2024-02-02T13:26:28.457090+0000","last_peered":"2024-02-02T13:26:28.457090+0000","last_clean":"2024-02-02T13:26:28.457090+0000","last_became_active":"2024-02-02T13:26:27.470493+0000","last_became_peered":"2024-02-02T13:26:27.470493+0000","last_unstale":"2024-02-02T13:26:28.457090+0000","last_undegraded":"2024-02-02T13:26:28.457090+0000","last_fullsized":"2024-02-02T13:26:28.457090+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:21:23.477163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.479972+0000","last_change":"2024-02-02T13:26:26.989604+0000","last_active":"2024-02-02T13:26:27.479972+0000","last_peered":"2024-02-02T13:26:27.479972+0000","last_clean":"2024-02-02T13:26:27.479972+0000","last_became_active":"2024-02-02T13:26:26.989348+0000","last_became_peered":"2024-02-02T13:26:26.989348+0000","last_unstale":"2024-02-02T13:26:27.479972+0000","last_undegraded":"2024-02-02T13:26:27.479972+0000","last_fullsized":"2024-02-02T13:26:27.479972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:00.980983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509894+0000","last_change":"2024-02-02T13:26:27.457548+0000","last_active":"2024-02-02T13:26:28.509894+0000","last_peered":"2024-02-02T13:26:28.509894+0000","last_clean":"2024-02-02T13:26:28.509894+0000","last_became_active":"2024-02-02T13:26:27.456887+0000","last_became_peered":"2024-02-02T13:26:27.456887+0000","last_unstale":"2024-02-02T13:26:28.509894+0000","last_undegraded":"2024-02-02T13:26:28.509894+0000","last_fullsized":"2024-02-02T13:26:28.509894+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:16:00.034179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.478863+0000","last_change":"2024-02-02T13:26:26.989266+0000","last_active":"2024-02-02T13:26:27.478863+0000","last_peered":"2024-02-02T13:26:27.478863+0000","last_clean":"2024-02-02T13:26:27.478863+0000","last_became_active":"2024-02-02T13:26:26.988350+0000","last_became_peered":"2024-02-02T13:26:26.988350+0000","last_unstale":"2024-02-02T13:26:27.478863+0000","last_undegraded":"2024-02-02T13:26:27.478863+0000","last_fullsized":"2024-02-02T13:26:27.478863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:11:13.242188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.512275+0000","last_change":"2024-02-02T13:26:27.473226+0000","last_active":"2024-02-02T13:26:28.512275+0000","last_peered":"2024-02-02T13:26:28.512275+0000","last_clean":"2024-02-02T13:26:28.512275+0000","last_became_active":"2024-02-02T13:26:27.472425+0000","last_became_peered":"2024-02-02T13:26:27.472425+0000","last_unstale":"2024-02-02T13:26:28.512275+0000","last_undegraded":"2024-02-02T13:26:28.512275+0000","last_fullsized":"2024-02-02T13:26:28.512275+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:11.311894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.469466+0000","last_change":"2024-02-02T13:26:26.996793+0000","last_active":"2024-02-02T13:26:27.469466+0000","last_peered":"2024-02-02T13:26:27.469466+0000","last_clean":"2024-02-02T13:26:27.469466+0000","last_became_active":"2024-02-02T13:26:26.996537+0000","last_became_peered":"2024-02-02T13:26:26.996537+0000","last_unstale":"2024-02-02T13:26:27.469466+0000","last_undegraded":"2024-02-02T13:26:27.469466+0000","last_fullsized":"2024-02-02T13:26:27.469466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:26.385659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508554+0000","last_change":"2024-02-02T13:26:27.469843+0000","last_active":"2024-02-02T13:26:28.508554+0000","last_peered":"2024-02-02T13:26:28.508554+0000","last_clean":"2024-02-02T13:26:28.508554+0000","last_became_active":"2024-02-02T13:26:27.469641+0000","last_became_peered":"2024-02-02T13:26:27.469641+0000","last_unstale":"2024-02-02T13:26:28.508554+0000","last_undegraded":"2024-02-02T13:26:28.508554+0000","last_fullsized":"2024-02-02T13:26:28.508554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:04.038214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.456797+0000","last_change":"2024-02-02T13:26:26.991207+0000","last_active":"2024-02-02T13:26:27.456797+0000","last_peered":"2024-02-02T13:26:27.456797+0000","last_clean":"2024-02-02T13:26:27.456797+0000","last_became_active":"2024-02-02T13:26:26.990979+0000","last_became_peered":"2024-02-02T13:26:26.990979+0000","last_unstale":"2024-02-02T13:26:27.456797+0000","last_undegraded":"2024-02-02T13:26:27.456797+0000","last_fullsized":"2024-02-02T13:26:27.456797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:04:11.576495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457908+0000","last_change":"2024-02-02T13:26:27.467625+0000","last_active":"2024-02-02T13:26:28.457908+0000","last_peered":"2024-02-02T13:26:28.457908+0000","last_clean":"2024-02-02T13:26:28.457908+0000","last_became_active":"2024-02-02T13:26:27.467329+0000","last_became_peered":"2024-02-02T13:26:27.467329+0000","last_unstale":"2024-02-02T13:26:28.457908+0000","last_undegraded":"2024-02-02T13:26:28.457908+0000","last_fullsized":"2024-02-02T13:26:28.457908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:56:10.783015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.447084+0000","last_change":"2024-02-02T13:26:26.996126+0000","last_active":"2024-02-02T13:26:27.447084+0000","last_peered":"2024-02-02T13:26:27.447084+0000","last_clean":"2024-02-02T13:26:27.447084+0000","last_became_active":"2024-02-02T13:26:26.995855+0000","last_became_peered":"2024-02-02T13:26:26.995855+0000","last_unstale":"2024-02-02T13:26:27.447084+0000","last_undegraded":"2024-02-02T13:26:27.447084+0000","last_fullsized":"2024-02-02T13:26:27.447084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:47:01.775499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510033+0000","last_change":"2024-02-02T13:26:27.478147+0000","last_active":"2024-02-02T13:26:28.510033+0000","last_peered":"2024-02-02T13:26:28.510033+0000","last_clean":"2024-02-02T13:26:28.510033+0000","last_became_active":"2024-02-02T13:26:27.475267+0000","last_became_peered":"2024-02-02T13:26:27.475267+0000","last_unstale":"2024-02-02T13:26:28.510033+0000","last_undegraded":"2024-02-02T13:26:28.510033+0000","last_fullsized":"2024-02-02T13:26:28.510033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:03.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.466742+0000","last_change":"2024-02-02T13:26:26.993983+0000","last_active":"2024-02-02T13:26:27.466742+0000","last_peered":"2024-02-02T13:26:27.466742+0000","last_clean":"2024-02-02T13:26:27.466742+0000","last_became_active":"2024-02-02T13:26:26.993562+0000","last_became_peered":"2024-02-02T13:26:26.993562+0000","last_unstale":"2024-02-02T13:26:27.466742+0000","last_undegraded":"2024-02-02T13:26:27.466742+0000","last_fullsized":"2024-02-02T13:26:27.466742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:20:27.178679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458881+0000","last_change":"2024-02-02T13:26:27.469116+0000","last_active":"2024-02-02T13:26:28.458881+0000","last_peered":"2024-02-02T13:26:28.458881+0000","last_clean":"2024-02-02T13:26:28.458881+0000","last_became_active":"2024-02-02T13:26:27.468854+0000","last_became_peered":"2024-02-02T13:26:27.468854+0000","last_unstale":"2024-02-02T13:26:28.458881+0000","last_undegraded":"2024-02-02T13:26:28.458881+0000","last_fullsized":"2024-02-02T13:26:28.458881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:42.364877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.472783+0000","last_change":"2024-02-02T13:26:26.986551+0000","last_active":"2024-02-02T13:26:27.472783+0000","last_peered":"2024-02-02T13:26:27.472783+0000","last_clean":"2024-02-02T13:26:27.472783+0000","last_became_active":"2024-02-02T13:26:26.986135+0000","last_became_peered":"2024-02-02T13:26:26.986135+0000","last_unstale":"2024-02-02T13:26:27.472783+0000","last_undegraded":"2024-02-02T13:26:27.472783+0000","last_fullsized":"2024-02-02T13:26:27.472783+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:07.068518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508699+0000","last_change":"2024-02-02T13:26:27.463737+0000","last_active":"2024-02-02T13:26:28.508699+0000","last_peered":"2024-02-02T13:26:28.508699+0000","last_clean":"2024-02-02T13:26:28.508699+0000","last_became_active":"2024-02-02T13:26:27.462844+0000","last_became_peered":"2024-02-02T13:26:27.462844+0000","last_unstale":"2024-02-02T13:26:28.508699+0000","last_undegraded":"2024-02-02T13:26:28.508699+0000","last_fullsized":"2024-02-02T13:26:28.508699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:32.551689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.477561+0000","last_change":"2024-02-02T13:26:26.985146+0000","last_active":"2024-02-02T13:26:27.477561+0000","last_peered":"2024-02-02T13:26:27.477561+0000","last_clean":"2024-02-02T13:26:27.477561+0000","last_became_active":"2024-02-02T13:26:26.984419+0000","last_became_peered":"2024-02-02T13:26:26.984419+0000","last_unstale":"2024-02-02T13:26:27.477561+0000","last_undegraded":"2024-02-02T13:26:27.477561+0000","last_fullsized":"2024-02-02T13:26:27.477561+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:15:43.398407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.512879+0000","last_change":"2024-02-02T13:26:27.476359+0000","last_active":"2024-02-02T13:26:28.512879+0000","last_peered":"2024-02-02T13:26:28.512879+0000","last_clean":"2024-02-02T13:26:28.512879+0000","last_became_active":"2024-02-02T13:26:27.475681+0000","last_became_peered":"2024-02-02T13:26:27.475681+0000","last_unstale":"2024-02-02T13:26:28.512879+0000","last_undegraded":"2024-02-02T13:26:28.512879+0000","last_fullsized":"2024-02-02T13:26:28.512879+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:49.662679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.467501+0000","last_change":"2024-02-02T13:26:26.980212+0000","last_active":"2024-02-02T13:26:27.467501+0000","last_peered":"2024-02-02T13:26:27.467501+0000","last_clean":"2024-02-02T13:26:27.467501+0000","last_became_active":"2024-02-02T13:26:26.979745+0000","last_became_peered":"2024-02-02T13:26:26.979745+0000","last_unstale":"2024-02-02T13:26:27.467501+0000","last_undegraded":"2024-02-02T13:26:27.467501+0000","last_fullsized":"2024-02-02T13:26:27.467501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:19.308272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.512500+0000","last_change":"2024-02-02T13:26:27.473081+0000","last_active":"2024-02-02T13:26:28.512500+0000","last_peered":"2024-02-02T13:26:28.512500+0000","last_clean":"2024-02-02T13:26:28.512500+0000","last_became_active":"2024-02-02T13:26:27.472117+0000","last_became_peered":"2024-02-02T13:26:27.472117+0000","last_unstale":"2024-02-02T13:26:28.512500+0000","last_undegraded":"2024-02-02T13:26:28.512500+0000","last_fullsized":"2024-02-02T13:26:28.512500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:01.997722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.447174+0000","last_change":"2024-02-02T13:26:26.984525+0000","last_active":"2024-02-02T13:26:27.447174+0000","last_peered":"2024-02-02T13:26:27.447174+0000","last_clean":"2024-02-02T13:26:27.447174+0000","last_became_active":"2024-02-02T13:26:26.984282+0000","last_became_peered":"2024-02-02T13:26:26.984282+0000","last_unstale":"2024-02-02T13:26:27.447174+0000","last_undegraded":"2024-02-02T13:26:27.447174+0000","last_fullsized":"2024-02-02T13:26:27.447174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:18.844454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457204+0000","last_change":"2024-02-02T13:26:27.470903+0000","last_active":"2024-02-02T13:26:28.457204+0000","last_peered":"2024-02-02T13:26:28.457204+0000","last_clean":"2024-02-02T13:26:28.457204+0000","last_became_active":"2024-02-02T13:26:27.470093+0000","last_became_peered":"2024-02-02T13:26:27.470093+0000","last_unstale":"2024-02-02T13:26:28.457204+0000","last_undegraded":"2024-02-02T13:26:28.457204+0000","last_fullsized":"2024-02-02T13:26:28.457204+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:05.745097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.474787+0000","last_change":"2024-02-02T13:26:26.983811+0000","last_active":"2024-02-02T13:26:27.474787+0000","last_peered":"2024-02-02T13:26:27.474787+0000","last_clean":"2024-02-02T13:26:27.474787+0000","last_became_active":"2024-02-02T13:26:26.983366+0000","last_became_peered":"2024-02-02T13:26:26.983366+0000","last_unstale":"2024-02-02T13:26:27.474787+0000","last_undegraded":"2024-02-02T13:26:27.474787+0000","last_fullsized":"2024-02-02T13:26:27.474787+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:03.021475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.507789+0000","last_change":"2024-02-02T13:26:27.458456+0000","last_active":"2024-02-02T13:26:28.507789+0000","last_peered":"2024-02-02T13:26:28.507789+0000","last_clean":"2024-02-02T13:26:28.507789+0000","last_became_active":"2024-02-02T13:26:27.458252+0000","last_became_peered":"2024-02-02T13:26:27.458252+0000","last_unstale":"2024-02-02T13:26:28.507789+0000","last_undegraded":"2024-02-02T13:26:28.507789+0000","last_fullsized":"2024-02-02T13:26:28.507789+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:32.551689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.453666+0000","last_change":"2024-02-02T13:26:26.992691+0000","last_active":"2024-02-02T13:26:27.453666+0000","last_peered":"2024-02-02T13:26:27.453666+0000","last_clean":"2024-02-02T13:26:27.453666+0000","last_became_active":"2024-02-02T13:26:26.992414+0000","last_became_peered":"2024-02-02T13:26:26.992414+0000","last_unstale":"2024-02-02T13:26:27.453666+0000","last_undegraded":"2024-02-02T13:26:27.453666+0000","last_fullsized":"2024-02-02T13:26:27.453666+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:00.980983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458320+0000","last_change":"2024-02-02T13:26:27.476301+0000","last_active":"2024-02-02T13:26:28.458320+0000","last_peered":"2024-02-02T13:26:28.458320+0000","last_clean":"2024-02-02T13:26:28.458320+0000","last_became_active":"2024-02-02T13:26:27.476041+0000","last_became_peered":"2024-02-02T13:26:27.476041+0000","last_unstale":"2024-02-02T13:26:28.458320+0000","last_undegraded":"2024-02-02T13:26:28.458320+0000","last_fullsized":"2024-02-02T13:26:28.458320+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:52.484048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.460629+0000","last_change":"2024-02-02T13:26:26.995403+0000","last_active":"2024-02-02T13:26:27.460629+0000","last_peered":"2024-02-02T13:26:27.460629+0000","last_clean":"2024-02-02T13:26:27.460629+0000","last_became_active":"2024-02-02T13:26:26.994018+0000","last_became_peered":"2024-02-02T13:26:26.994018+0000","last_unstale":"2024-02-02T13:26:27.460629+0000","last_undegraded":"2024-02-02T13:26:27.460629+0000","last_fullsized":"2024-02-02T13:26:27.460629+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:59.426053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508621+0000","last_change":"2024-02-02T13:26:27.468545+0000","last_active":"2024-02-02T13:26:28.508621+0000","last_peered":"2024-02-02T13:26:28.508621+0000","last_clean":"2024-02-02T13:26:28.508621+0000","last_became_active":"2024-02-02T13:26:27.466356+0000","last_became_peered":"2024-02-02T13:26:27.466356+0000","last_unstale":"2024-02-02T13:26:28.508621+0000","last_undegraded":"2024-02-02T13:26:28.508621+0000","last_fullsized":"2024-02-02T13:26:28.508621+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:08.683890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.457043+0000","last_change":"2024-02-02T13:26:26.990726+0000","last_active":"2024-02-02T13:26:27.457043+0000","last_peered":"2024-02-02T13:26:27.457043+0000","last_clean":"2024-02-02T13:26:27.457043+0000","last_became_active":"2024-02-02T13:26:26.990491+0000","last_became_peered":"2024-02-02T13:26:26.990491+0000","last_unstale":"2024-02-02T13:26:27.457043+0000","last_undegraded":"2024-02-02T13:26:27.457043+0000","last_fullsized":"2024-02-02T13:26:27.457043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:31.534949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.513107+0000","last_change":"2024-02-02T13:26:27.476609+0000","last_active":"2024-02-02T13:26:28.513107+0000","last_peered":"2024-02-02T13:26:28.513107+0000","last_clean":"2024-02-02T13:26:28.513107+0000","last_became_active":"2024-02-02T13:26:27.476103+0000","last_became_peered":"2024-02-02T13:26:27.476103+0000","last_unstale":"2024-02-02T13:26:28.513107+0000","last_undegraded":"2024-02-02T13:26:28.513107+0000","last_fullsized":"2024-02-02T13:26:28.513107+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:39.643570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.447390+0000","last_change":"2024-02-02T13:26:26.982909+0000","last_active":"2024-02-02T13:26:27.447390+0000","last_peered":"2024-02-02T13:26:27.447390+0000","last_clean":"2024-02-02T13:26:27.447390+0000","last_became_active":"2024-02-02T13:26:26.982401+0000","last_became_peered":"2024-02-02T13:26:26.982401+0000","last_unstale":"2024-02-02T13:26:27.447390+0000","last_undegraded":"2024-02-02T13:26:27.447390+0000","last_fullsized":"2024-02-02T13:26:27.447390+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:51.430990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508714+0000","last_change":"2024-02-02T13:26:27.472803+0000","last_active":"2024-02-02T13:26:28.508714+0000","last_peered":"2024-02-02T13:26:28.508714+0000","last_clean":"2024-02-02T13:26:28.508714+0000","last_became_active":"2024-02-02T13:26:27.472591+0000","last_became_peered":"2024-02-02T13:26:27.472591+0000","last_unstale":"2024-02-02T13:26:28.508714+0000","last_undegraded":"2024-02-02T13:26:28.508714+0000","last_fullsized":"2024-02-02T13:26:28.508714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:49:10.338105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.453789+0000","last_change":"2024-02-02T13:26:26.992191+0000","last_active":"2024-02-02T13:26:27.453789+0000","last_peered":"2024-02-02T13:26:27.453789+0000","last_clean":"2024-02-02T13:26:27.453789+0000","last_became_active":"2024-02-02T13:26:26.991945+0000","last_became_peered":"2024-02-02T13:26:26.991945+0000","last_unstale":"2024-02-02T13:26:27.453789+0000","last_undegraded":"2024-02-02T13:26:27.453789+0000","last_fullsized":"2024-02-02T13:26:27.453789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:59.094888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509389+0000","last_change":"2024-02-02T13:26:27.474846+0000","last_active":"2024-02-02T13:26:28.509389+0000","last_peered":"2024-02-02T13:26:28.509389+0000","last_clean":"2024-02-02T13:26:28.509389+0000","last_became_active":"2024-02-02T13:26:27.474623+0000","last_became_peered":"2024-02-02T13:26:27.474623+0000","last_unstale":"2024-02-02T13:26:28.509389+0000","last_undegraded":"2024-02-02T13:26:28.509389+0000","last_fullsized":"2024-02-02T13:26:28.509389+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:49:10.338105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.451580+0000","last_change":"2024-02-02T13:26:26.989827+0000","last_active":"2024-02-02T13:26:27.451580+0000","last_peered":"2024-02-02T13:26:27.451580+0000","last_clean":"2024-02-02T13:26:27.451580+0000","last_became_active":"2024-02-02T13:26:26.989545+0000","last_became_peered":"2024-02-02T13:26:26.989545+0000","last_unstale":"2024-02-02T13:26:27.451580+0000","last_undegraded":"2024-02-02T13:26:27.451580+0000","last_fullsized":"2024-02-02T13:26:27.451580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:08.259274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.456998+0000","last_change":"2024-02-02T13:26:27.452957+0000","last_active":"2024-02-02T13:26:28.456998+0000","last_peered":"2024-02-02T13:26:28.456998+0000","last_clean":"2024-02-02T13:26:28.456998+0000","last_became_active":"2024-02-02T13:26:27.452672+0000","last_became_peered":"2024-02-02T13:26:27.452672+0000","last_unstale":"2024-02-02T13:26:28.456998+0000","last_undegraded":"2024-02-02T13:26:28.456998+0000","last_fullsized":"2024-02-02T13:26:28.456998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:06:23.418506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.457279+0000","last_change":"2024-02-02T13:26:26.993620+0000","last_active":"2024-02-02T13:26:27.457279+0000","last_peered":"2024-02-02T13:26:27.457279+0000","last_clean":"2024-02-02T13:26:27.457279+0000","last_became_active":"2024-02-02T13:26:26.993368+0000","last_became_peered":"2024-02-02T13:26:26.993368+0000","last_unstale":"2024-02-02T13:26:27.457279+0000","last_undegraded":"2024-02-02T13:26:27.457279+0000","last_fullsized":"2024-02-02T13:26:27.457279+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:10.295155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.457481+0000","last_change":"2024-02-02T13:26:27.468443+0000","last_active":"2024-02-02T13:26:28.457481+0000","last_peered":"2024-02-02T13:26:28.457481+0000","last_clean":"2024-02-02T13:26:28.457481+0000","last_became_active":"2024-02-02T13:26:27.468087+0000","last_became_peered":"2024-02-02T13:26:27.468087+0000","last_unstale":"2024-02-02T13:26:28.457481+0000","last_undegraded":"2024-02-02T13:26:28.457481+0000","last_fullsized":"2024-02-02T13:26:28.457481+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:07:29.951361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.465893+0000","last_change":"2024-02-02T13:26:26.986386+0000","last_active":"2024-02-02T13:26:27.465893+0000","last_peered":"2024-02-02T13:26:27.465893+0000","last_clean":"2024-02-02T13:26:27.465893+0000","last_became_active":"2024-02-02T13:26:26.986169+0000","last_became_peered":"2024-02-02T13:26:26.986169+0000","last_unstale":"2024-02-02T13:26:27.465893+0000","last_undegraded":"2024-02-02T13:26:27.465893+0000","last_fullsized":"2024-02-02T13:26:27.465893+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:41.348138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.458567+0000","last_change":"2024-02-02T13:26:27.455123+0000","last_active":"2024-02-02T13:26:28.458567+0000","last_peered":"2024-02-02T13:26:28.458567+0000","last_clean":"2024-02-02T13:26:28.458567+0000","last_became_active":"2024-02-02T13:26:27.454517+0000","last_became_peered":"2024-02-02T13:26:27.454517+0000","last_unstale":"2024-02-02T13:26:28.458567+0000","last_undegraded":"2024-02-02T13:26:28.458567+0000","last_fullsized":"2024-02-02T13:26:28.458567+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:10.261522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.451767+0000","last_change":"2024-02-02T13:26:26.990011+0000","last_active":"2024-02-02T13:26:27.451767+0000","last_peered":"2024-02-02T13:26:27.451767+0000","last_clean":"2024-02-02T13:26:27.451767+0000","last_became_active":"2024-02-02T13:26:26.988542+0000","last_became_peered":"2024-02-02T13:26:26.988542+0000","last_unstale":"2024-02-02T13:26:27.451767+0000","last_undegraded":"2024-02-02T13:26:27.451767+0000","last_fullsized":"2024-02-02T13:26:27.451767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:12:58.587187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.510206+0000","last_change":"2024-02-02T13:26:27.473784+0000","last_active":"2024-02-02T13:26:28.510206+0000","last_peered":"2024-02-02T13:26:28.510206+0000","last_clean":"2024-02-02T13:26:28.510206+0000","last_became_active":"2024-02-02T13:26:27.471617+0000","last_became_peered":"2024-02-02T13:26:27.471617+0000","last_unstale":"2024-02-02T13:26:28.510206+0000","last_undegraded":"2024-02-02T13:26:28.510206+0000","last_fullsized":"2024-02-02T13:26:28.510206+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:20.325011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.455183+0000","last_change":"2024-02-02T13:26:26.995064+0000","last_active":"2024-02-02T13:26:27.455183+0000","last_peered":"2024-02-02T13:26:27.455183+0000","last_clean":"2024-02-02T13:26:27.455183+0000","last_became_active":"2024-02-02T13:26:26.994287+0000","last_became_peered":"2024-02-02T13:26:26.994287+0000","last_unstale":"2024-02-02T13:26:27.455183+0000","last_undegraded":"2024-02-02T13:26:27.455183+0000","last_fullsized":"2024-02-02T13:26:27.455183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:28:25.071113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.508232+0000","last_change":"2024-02-02T13:26:27.475137+0000","last_active":"2024-02-02T13:26:28.508232+0000","last_peered":"2024-02-02T13:26:28.508232+0000","last_clean":"2024-02-02T13:26:28.508232+0000","last_became_active":"2024-02-02T13:26:27.474825+0000","last_became_peered":"2024-02-02T13:26:27.474825+0000","last_unstale":"2024-02-02T13:26:28.508232+0000","last_undegraded":"2024-02-02T13:26:28.508232+0000","last_fullsized":"2024-02-02T13:26:28.508232+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:26.949422+0000","last_clean_scrub_stamp":"2024-02-02T13:26:26.949422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:27.402398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-02-02T13:26:27.474001+0000","last_change":"2024-02-02T13:26:26.989545+0000","last_active":"2024-02-02T13:26:27.474001+0000","last_peered":"2024-02-02T13:26:27.474001+0000","last_clean":"2024-02-02T13:26:27.474001+0000","last_became_active":"2024-02-02T13:26:26.988901+0000","last_became_peered":"2024-02-02T13:26:26.988901+0000","last_unstale":"2024-02-02T13:26:27.474001+0000","last_undegraded":"2024-02-02T13:26:27.474001+0000","last_fullsized":"2024-02-02T13:26:27.474001+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:26:25.932683+0000","last_clean_scrub_stamp":"2024-02-02T13:26:25.932683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:59.094888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":191,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-02-02T13:26:28.509887+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:26:28.509887+0000","last_peered":"2024-02-02T13:26:28.509887+0000","last_clean":"2024-02-02T13:26:28.509887+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:26:28.509887+0000","last_undegraded":"2024-02-02T13:26:28.509887+0000","last_fullsized":"2024-02-02T13:26:28.509887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":803,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194509,"statfs":{"total":200000000,"available":199177482,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0580000000000001,"15min":1.052},"min":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5880000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.123,"15min":1.1299999999999999},"min":{"1min":0.76900000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5589999999999999,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.099}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.083},"min":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":1.893,"15min":1.893},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.105,"15min":1.099},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4099999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.208,"15min":1.2110000000000001},"min":{"1min":0.78100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.355},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.159,"15min":1.181},"min":{"1min":0.67500000000000004,"5min":0.623,"15min":0.623},"max":{"1min":1.615,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.76000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.327,"5min":1.819,"15min":1.819},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.661,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.155,"15min":1.159},"min":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.284},{"interface":"front","average":{"1min":1.244,"5min":1.302,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6619999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.347,"15min":1.343},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.6799999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.615},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7130000000000001,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.218}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5209999999999999,"15min":1.5409999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8049999999999999,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.407},{"interface":"front","average":{"1min":1.292,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.6299999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.3120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":226,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195086,"statfs":{"total":200000000,"available":199768287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.145}]},{"osd":1,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.01},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.3500000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.085},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.048}]},{"osd":3,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.226},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.2729999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.3859999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.212}]},{"osd":7,"last update":"Fri Feb 2 13:25:29 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.304},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.1890000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":226,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195085,"statfs":{"total":200000000,"available":199767870,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.022,"15min":1.018},"min":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0249999999999999,"15min":1.0329999999999999},"min":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2629999999999999,"15min":1.2969999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.4990000000000001,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.075,"5min":1.153,"15min":1.1659999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.444,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.333}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.026},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.171,"15min":1.169},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.96799999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1799999999999999,"15min":1.1830000000000001},"min":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.173},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.512,"5min":1.744,"15min":1.744},"last":1.127}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2330000000000001,"15min":1.2350000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2729999999999999,"15min":1.2789999999999999},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.002}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3029999999999999,"15min":1.2929999999999999},"min":{"1min":1.042,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.351,"15min":1.345},"min":{"1min":0.92200000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3799999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3979999999999999,"15min":1.397},"min":{"1min":0.91000000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3640000000000001,"15min":1.353},"min":{"1min":0.88600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.4239999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":227,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195085,"statfs":{"total":200000000,"available":199767395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.96099999999999997,"15min":0.94799999999999995},"min":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.036,"5min":1.0629999999999999,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.554,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.78700000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5,"5min":1.506,"15min":1.506},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.999,"15min":1},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.27,"5min":1.345,"15min":1.345},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.0660000000000001,"15min":1.0580000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4339999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0589999999999999,"15min":1.038},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.2110000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.161,"15min":1.145},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4490000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.292,"5min":1.2210000000000001,"15min":1.2090000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.5920000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.276}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.117,"15min":1.1040000000000001},"min":{"1min":0.98199999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.016},{"interface":"front","average":{"1min":1.335,"5min":1.2609999999999999,"15min":1.2490000000000001},"min":{"1min":1.008,"5min":0.751,"15min":0.751},"max":{"1min":1.6879999999999999,"5min":2.069,"15min":2.069},"last":1.1619999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.90700000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.325},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3640000000000001,"15min":1.361},"min":{"1min":1.123,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.123}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3580000000000001,"15min":1.3500000000000001},"min":{"1min":1.1659999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.6830000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.252},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.90100000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.675,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":227,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195085,"statfs":{"total":200000000,"available":199767302,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.585,"5min":3.585,"15min":3.585},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.153}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.546}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":1.369},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.2669999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":228,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195084,"statfs":{"total":200000000,"available":199766354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.755,"15min":0.77200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1120000000000001,"5min":1.29,"15min":1.29},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.38200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2549999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.87,"15min":0.88},"min":{"1min":0.50900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.2170000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.246,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.085}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.93899999999999995,"15min":0.94999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.1359999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1379999999999999,"15min":1.129},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.4039999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0589999999999999,"15min":1.0409999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.272},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.25},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.4550000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.337},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.173,"15min":1.1699999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.4299999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1639999999999999,"15min":1.1579999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.365}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195086,"statfs":{"total":200000000,"available":199768889,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.022,"15min":1.0649999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":1.391,"15min":1.4910000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5029999999999999,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.639,"15min":1.639},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.091,"15min":1.121},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3680000000000001,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.25,"15min":1.3080000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4830000000000001,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":1.119},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.2949999999999999,"15min":1.357},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.528,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.024}]},{"osd":4,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.335,"15min":1.377},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3799999999999999,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.083},{"interface":"front","average":{"1min":1.139,"5min":1.3580000000000001,"15min":1.3939999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.4930000000000001,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.268}]},{"osd":5,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.472,"15min":1.508},"min":{"1min":1.0609999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.532,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.298},{"interface":"front","average":{"1min":1.222,"5min":1.4470000000000001,"15min":1.4850000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.8959999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.3320000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.5880000000000001,"15min":1.653},"min":{"1min":0.85099999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.661,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.153},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4950000000000001,"15min":1.5429999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.806,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5609999999999999,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.496,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.6299999999999999,"15min":1.6890000000000001},"min":{"1min":0.97399999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5880000000000001,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.3560000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":802,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194510,"statfs":{"total":200000000,"available":199178557,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.8070000000000004,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.157}]},{"osd":2,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":10.781000000000001,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.751}]},{"osd":3,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.8410000000000002,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":10.651,"5min":10.651,"15min":10.651},"last":1.304},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":3.7869999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.339}]},{"osd":6,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.258},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":10.708,"5min":10.708,"15min":10.708},"last":1.3839999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:25:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":10.74,"5min":10.74,"15min":10.74},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.1180000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:26:32.373 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:26:32.374 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:26:32.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.627+0000 7ff918e00640 1 Processor -- start 2024-02-02T13:26:32.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.627+0000 7ff918e00640 1 -- start start 2024-02-02T13:26:32.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff918e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff914151f90 0x7ff91415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff918e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9140b0e10 0x7ff9140b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff918e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914070f80 con 0x7ff91406f4f0 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff918e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff914152370 con 0x7ff9140b0e10 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff918e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140cb010 con 0x7ff914151f90 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9140b0e10 0x7ff9140b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9140b0e10 0x7ff9140b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47614/0 (socket says 172.21.15.91:47614) 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 -- 172.21.15.91:0/1907502863 learned_addr learned my addr 172.21.15.91:0/1907502863 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:32.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff911e00640 1 --2- 172.21.15.91:0/1907502863 >> v2:172.21.15.113:3300/0 conn(0x7ff914151f90 0x7ff91415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 -- 172.21.15.91:0/1907502863 >> v2:172.21.15.113:3300/0 conn(0x7ff914151f90 msgr2=0x7ff91415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:32.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 --2- 172.21.15.91:0/1907502863 >> v2:172.21.15.113:3300/0 conn(0x7ff914151f90 0x7ff91415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 -- 172.21.15.91:0/1907502863 >> v1:172.21.15.113:6789/0 conn(0x7ff91406f4f0 legacy=0x7ff9140b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:32.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.629+0000 7ff912800640 1 -- 172.21.15.91:0/1907502863 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140cb2f0 con 0x7ff9140b0e10 2024-02-02T13:26:32.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff912800640 1 --2- 172.21.15.91:0/1907502863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff9140b3af0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff9140cc770 tx=0x7ff90c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9924d4d01d8e9e67 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff910a00640 1 -- 172.21.15.91:0/1907502863 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c02ca80 con 0x7ff9140b0e10 2024-02-02T13:26:32.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff910a00640 1 -- 172.21.15.91:0/1907502863 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c02cbe0 con 0x7ff9140b0e10 2024-02-02T13:26:32.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff910a00640 1 -- 172.21.15.91:0/1907502863 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0343f0 con 0x7ff9140b0e10 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff918e00640 1 -- 172.21.15.91:0/1907502863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 msgr2=0x7ff9140b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.631+0000 7ff918e00640 1 --2- 172.21.15.91:0/1907502863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff9140b3af0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff9140cc770 tx=0x7ff90c02bee0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 -- 172.21.15.91:0/1907502863 shutdown_connections 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 --2- 172.21.15.91:0/1907502863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff9140b3af0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 --2- 172.21.15.91:0/1907502863 >> v2:172.21.15.113:3300/0 conn(0x7ff914151f90 0x7ff91415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 -- 172.21.15.91:0/1907502863 >> 172.21.15.91:0/1907502863 conn(0x7ff9140cec80 msgr2=0x7ff9140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 -- 172.21.15.91:0/1907502863 shutdown_connections 2024-02-02T13:26:32.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.632+0000 7ff918e00640 1 -- 172.21.15.91:0/1907502863 wait complete. 2024-02-02T13:26:32.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.633+0000 7ff918e00640 1 Processor -- start 2024-02-02T13:26:32.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.633+0000 7ff918e00640 1 -- start start 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 0x7ff9141f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff914151f90 0x7ff914155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914156b80 con 0x7ff9140b0e10 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141565c0 con 0x7ff91406f4f0 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.634+0000 7ff918e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141568a0 con 0x7ff914151f90 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47616/0 (socket says 172.21.15.91:47616) 2024-02-02T13:26:32.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 -- 172.21.15.91:0/3899559483 learned_addr learned my addr 172.21.15.91:0/3899559483 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff912800640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff914151f90 0x7ff914155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 0x7ff9141f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 -- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff914151f90 msgr2=0x7ff914155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff914151f90 0x7ff914155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 -- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 msgr2=0x7ff9141f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 0x7ff9141f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 -- 172.21.15.91:0/3899559483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009b30 con 0x7ff9140b0e10 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 0x7ff9141f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.635+0000 7ff911400640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00b3e0 tx=0x7ff8fc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.636+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc011830 con 0x7ff9140b0e10 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.636+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc0119b0 con 0x7ff9140b0e10 2024-02-02T13:26:32.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.636+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc011d60 con 0x7ff9140b0e10 2024-02-02T13:26:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.636+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9140c6290 con 0x7ff9140b0e10 2024-02-02T13:26:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.636+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141573b0 con 0x7ff9140b0e10 2024-02-02T13:26:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.637+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9140c60f0 con 0x7ff9140b0e10 2024-02-02T13:26:32.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.638+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff8fc01a400 con 0x7ff9140b0e10 2024-02-02T13:26:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.639+0000 7ff90a000640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 0x7ff8ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.639+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8fc088ab0 con 0x7ff9140b0e10 2024-02-02T13:26:32.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.639+0000 7ff911e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 0x7ff8ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:32.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.640+0000 7ff911e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 0x7ff8ec06c8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff904006fd0 tx=0x7ff904008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:32.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.643+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff9140c60f0 con 0x7ff9140b0e10 2024-02-02T13:26:32.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.910+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff9140c67b0 con 0x7ff9140b0e10 2024-02-02T13:26:32.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.916+0000 7ff90a000640 1 -- 172.21.15.91:0/3899559483 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff9140c67b0 con 0x7ff9140b0e10 2024-02-02T13:26:32.917 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 msgr2=0x7ff8ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 0x7ff8ec06c8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff904006fd0 tx=0x7ff904008040 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 msgr2=0x7ff914150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00b3e0 tx=0x7ff8fc00b8a0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 shutdown_connections 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8ec06a430 0x7ff8ec06c8f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff914151f90 0x7ff914155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9140b0e10 0x7ff914150cf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 --2- 172.21.15.91:0/3899559483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff91406f4f0 0x7ff9141f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 >> 172.21.15.91:0/3899559483 conn(0x7ff9140cec80 msgr2=0x7ff9140adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 shutdown_connections 2024-02-02T13:26:32.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:32.920+0000 7ff918e00640 1 -- 172.21.15.91:0/3899559483 wait complete. 2024-02-02T13:26:32.939 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:33.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.153+0000 7f2570200640 1 Processor -- start 2024-02-02T13:26:33.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.153+0000 7f2570200640 1 -- start start 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f2570200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2568151f90 0x7f256815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f2570200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25680b0e10 0x7f25680b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f2570200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2568070f80 con 0x7f256806f4f0 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f2570200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2568152370 con 0x7f2568151f90 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f2570200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25680cb010 con 0x7f25680b0e10 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f256d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2568151f90 0x7f256815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f256d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2568151f90 0x7f256815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47618/0 (socket says 172.21.15.91:47618) 2024-02-02T13:26:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f256d800640 1 -- 172.21.15.91:0/3794525823 learned_addr learned my addr 172.21.15.91:0/3794525823 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.155+0000 7f256e200640 1 --2- 172.21.15.91:0/3794525823 >> v2:172.21.15.113:3300/0 conn(0x7f25680b0e10 0x7f25680b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.156+0000 7f256d800640 1 -- 172.21.15.91:0/3794525823 >> v2:172.21.15.113:3300/0 conn(0x7f25680b0e10 msgr2=0x7f25680b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.156+0000 7f256d800640 1 --2- 172.21.15.91:0/3794525823 >> v2:172.21.15.113:3300/0 conn(0x7f25680b0e10 0x7f25680b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.156+0000 7f256d800640 1 -- 172.21.15.91:0/3794525823 >> v1:172.21.15.113:6789/0 conn(0x7f256806f4f0 legacy=0x7f25680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.156+0000 7f256d800640 1 -- 172.21.15.91:0/3794525823 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680cb2f0 con 0x7f2568151f90 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.157+0000 7f256d800640 1 --2- 172.21.15.91:0/3794525823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f256815f5a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25680ca520 tx=0x7f255802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52955442494b89a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.157+0000 7f2567e00640 1 -- 172.21.15.91:0/3794525823 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558031870 con 0x7f2568151f90 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.157+0000 7f2567e00640 1 -- 172.21.15.91:0/3794525823 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580319d0 con 0x7f2568151f90 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.157+0000 7f2567e00640 1 -- 172.21.15.91:0/3794525823 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558031d60 con 0x7f2568151f90 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 -- 172.21.15.91:0/3794525823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 msgr2=0x7f256815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 --2- 172.21.15.91:0/3794525823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f256815f5a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25680ca520 tx=0x7f255802b910 comp rx=0 tx=0).stop 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 -- 172.21.15.91:0/3794525823 shutdown_connections 2024-02-02T13:26:33.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 --2- 172.21.15.91:0/3794525823 >> v2:172.21.15.113:3300/0 conn(0x7f25680b0e10 0x7f25680b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 --2- 172.21.15.91:0/3794525823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f256815f5a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 -- 172.21.15.91:0/3794525823 >> 172.21.15.91:0/3794525823 conn(0x7f25680cec80 msgr2=0x7f25680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:33.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 -- 172.21.15.91:0/3794525823 shutdown_connections 2024-02-02T13:26:33.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.158+0000 7f2570200640 1 -- 172.21.15.91:0/3794525823 wait complete. 2024-02-02T13:26:33.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.159+0000 7f2570200640 1 Processor -- start 2024-02-02T13:26:33.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.160+0000 7f2570200640 1 -- start start 2024-02-02T13:26:33.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.160+0000 7f2570200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f256806f4f0 0x7f25680c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.160+0000 7f2570200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25680b0e10 0x7f25680b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f2570200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f2570200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680b3e60 con 0x7f2568151f90 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f2570200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680c6db0 con 0x7f25680b0e10 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f2570200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25680c7090 con 0x7f256806f4f0 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47620/0 (socket says 172.21.15.91:47620) 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 -- 172.21.15.91:0/1714706975 learned_addr learned my addr 172.21.15.91:0/1714706975 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256ce00640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25680b0e10 0x7f25680b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256d800640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f256806f4f0 0x7f25680c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 -- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f256806f4f0 msgr2=0x7f25680c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f256806f4f0 0x7f25680c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 -- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25680b0e10 msgr2=0x7f25680b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25680b0e10 0x7f25680b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 -- 172.21.15.91:0/1714706975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25580094d0 con 0x7f2568151f90 2024-02-02T13:26:33.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.161+0000 7f256e200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2560007d80 tx=0x7f256000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.162+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560014420 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.162+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560014580 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.162+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25680c7370 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.162+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560014930 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.162+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25680c7930 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.163+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256806f960 con 0x7f2568151f90 2024-02-02T13:26:33.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.164+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2560014a90 con 0x7f2568151f90 2024-02-02T13:26:33.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.165+0000 7f2565600640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 0x7f254006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:33.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.165+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2560089200 con 0x7f2568151f90 2024-02-02T13:26:33.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.165+0000 7f256d800640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 0x7f254006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:33.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.166+0000 7f256d800640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 0x7f254006c8f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2558036000 tx=0x7f255803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:33.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.169+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f256806f960 con 0x7f2568151f90 2024-02-02T13:26:33.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.398+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256806f960 con 0x7f2568151f90 2024-02-02T13:26:33.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.399+0000 7f2565600640 1 -- 172.21.15.91:0/1714706975 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f256806f960 con 0x7f2568151f90 2024-02-02T13:26:33.400 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:33.400 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:26:33.400 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 msgr2=0x7f254006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 0x7f254006c8f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2558036000 tx=0x7f255803c040 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 msgr2=0x7f2568150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2560007d80 tx=0x7f256000e4b0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 shutdown_connections 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f254006a430 0x7f254006c8f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2568151f90 0x7f2568150450 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25680b0e10 0x7f25680b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 --2- 172.21.15.91:0/1714706975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f256806f4f0 0x7f25680c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 >> 172.21.15.91:0/1714706975 conn(0x7f25680cec80 msgr2=0x7f25680b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 shutdown_connections 2024-02-02T13:26:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:33.402+0000 7f2570200640 1 -- 172.21.15.91:0/1714706975 wait complete. 2024-02-02T13:26:33.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:26:27.440160+0000', 'modified': '2024-02-02T13:26:30.459190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5508': {'gid': 5508, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/681798855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6836', 'nonce': 681798855}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 681798855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:26:33.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:26:33.824 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-02T13:26:34.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.044+0000 7fdbdf400640 1 Processor -- start 2024-02-02T13:26:34.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.044+0000 7fdbdf400640 1 -- start start 2024-02-02T13:26:34.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdf400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdbd815ad10 0x7fdbd806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbd80cb580 0x7fdbd80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdf400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd806d910 con 0x7fdbd815b0f0 2024-02-02T13:26:34.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdf400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd806d730 con 0x7fdbd815ad10 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdf400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80cbf50 con 0x7fdbd80cb580 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdbd815ad10 0x7fdbd806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdbd815ad10 0x7fdbd806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47630/0 (socket says 172.21.15.91:47630) 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbdca00640 1 -- 172.21.15.91:0/2005797085 learned_addr learned my addr 172.21.15.91:0/2005797085 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.045+0000 7fdbcfc00640 1 --1- 172.21.15.91:0/2005797085 >> v1:172.21.15.91:6789/0 conn(0x7fdbd815b0f0 0x7fdbd806d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34482/0 (socket says 172.21.15.91:34482) 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.046+0000 7fdbdca00640 1 -- 172.21.15.91:0/2005797085 >> v2:172.21.15.113:3300/0 conn(0x7fdbd80cb580 msgr2=0x7fdbd80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.046+0000 7fdbdca00640 1 --2- 172.21.15.91:0/2005797085 >> v2:172.21.15.113:3300/0 conn(0x7fdbd80cb580 0x7fdbd80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.046+0000 7fdbdca00640 1 -- 172.21.15.91:0/2005797085 >> v1:172.21.15.91:6789/0 conn(0x7fdbd815b0f0 legacy=0x7fdbd806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.046+0000 7fdbdca00640 1 -- 172.21.15.91:0/2005797085 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80cc850 con 0x7fdbd815ad10 2024-02-02T13:26:34.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.046+0000 7fdbd7e00640 1 -- 172.21.15.91:0/2005797085 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562617270 0 0) 0x7fdbd806d910 con 0x7fdbd815b0f0 2024-02-02T13:26:34.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.047+0000 7fdbdca00640 1 --2- 172.21.15.91:0/2005797085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd806b7a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009870 tx=0x7fdbc802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3161854a2d661b57 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.047+0000 7fdbd7e00640 1 -- 172.21.15.91:0/2005797085 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8031860 con 0x7fdbd815ad10 2024-02-02T13:26:34.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.047+0000 7fdbd7e00640 1 -- 172.21.15.91:0/2005797085 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80319c0 con 0x7fdbd815ad10 2024-02-02T13:26:34.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.047+0000 7fdbd7e00640 1 -- 172.21.15.91:0/2005797085 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8031d50 con 0x7fdbd815ad10 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 -- 172.21.15.91:0/2005797085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 msgr2=0x7fdbd806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 --2- 172.21.15.91:0/2005797085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd806b7a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009870 tx=0x7fdbc802b900 comp rx=0 tx=0).stop 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 -- 172.21.15.91:0/2005797085 shutdown_connections 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 --2- 172.21.15.91:0/2005797085 >> v2:172.21.15.113:3300/0 conn(0x7fdbd80cb580 0x7fdbd80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 --2- 172.21.15.91:0/2005797085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd806b7a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 -- 172.21.15.91:0/2005797085 >> 172.21.15.91:0/2005797085 conn(0x7fdbd80b0700 msgr2=0x7fdbd80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 -- 172.21.15.91:0/2005797085 shutdown_connections 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.048+0000 7fdbdf400640 1 -- 172.21.15.91:0/2005797085 wait complete. 2024-02-02T13:26:34.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.050+0000 7fdbdf400640 1 Processor -- start 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.050+0000 7fdbdf400640 1 -- start start 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 0x7fdbd814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 0x7fdbd814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd814bd60 con 0x7fdbd815ad10 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd814c040 con 0x7fdbd815b0f0 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbdf400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd814c320 con 0x7fdbd80cb580 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbcfc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbcfc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47632/0 (socket says 172.21.15.91:47632) 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.051+0000 7fdbcfc00640 1 -- 172.21.15.91:0/816142999 learned_addr learned my addr 172.21.15.91:0/816142999 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdca00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 0x7fdbd814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdd400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 0x7fdbd814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 -- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 msgr2=0x7fdbd814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 0x7fdbd814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 -- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 msgr2=0x7fdbd814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 0x7fdbd814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 -- 172.21.15.91:0/816142999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80094d0 con 0x7fdbd815ad10 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdca00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 0x7fdbd814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdd400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 0x7fdbd814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c430 tx=0x7fdbc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbd5600640 1 -- 172.21.15.91:0/816142999 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00074e0 con 0x7fdbd815ad10 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbd5600640 1 -- 172.21.15.91:0/816142999 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0007660 con 0x7fdbd815ad10 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbd5600640 1 -- 172.21.15.91:0/816142999 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0004a20 con 0x7fdbd815ad10 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd814c600 con 0x7fdbd815ad10 2024-02-02T13:26:34.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.052+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd814b480 con 0x7fdbd815ad10 2024-02-02T13:26:34.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.055+0000 7fdbd5600640 1 -- 172.21.15.91:0/816142999 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdbc0004b80 con 0x7fdbd815ad10 2024-02-02T13:26:34.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.055+0000 7fdbd5600640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 0x7fdbb006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.056+0000 7fdbd5600640 1 -- 172.21.15.91:0/816142999 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc00888f0 con 0x7fdbd815ad10 2024-02-02T13:26:34.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.056+0000 7fdbdca00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 0x7fdbb006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.056+0000 7fdbe1fc1740 1 Processor -- start 2024-02-02T13:26:34.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.056+0000 7fdbe1fc1740 1 -- start start 2024-02-02T13:26:34.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.056+0000 7fdbdca00640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 0x7fdbb006c8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8036000 tx=0x7fdbc803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa8c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa872650 con 0x564aaa896ed0 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa807000 con 0x564aaa89aad0 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.057+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564aaa804790 con 0x564aaa8c6f50 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47646/0 (socket says 172.21.15.91:47646) 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 -- 172.21.15.91:0/4157216007 learned_addr learned my addr 172.21.15.91:0/4157216007 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbdca00640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 -- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 msgr2=0x564aaa8c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbdd400640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa8c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 -- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 msgr2=0x564aaa8c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa8c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 -- 172.21.15.91:0/4157216007 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aaa7c7710 con 0x564aaa896ed0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbdca00640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0057770 tx=0x7fdbc0004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3272157af6324ee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.058+0000 7fdbcd400640 1 -- 172.21.15.91:0/4157216007 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc006f100 con 0x564aaa896ed0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbcd400640 1 -- 172.21.15.91:0/4157216007 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc005b8a0 con 0x564aaa896ed0 2024-02-02T13:26:34.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbcd400640 1 -- 172.21.15.91:0/4157216007 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc005fae0 con 0x564aaa896ed0 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/4157216007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 msgr2=0x564aaa8c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0057770 tx=0x7fdbc0004f60 comp rx=0 tx=0).stop 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/4157216007 shutdown_connections 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa8c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa896ed0 0x564aaa8c41c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/4157216007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/4157216007 >> 172.21.15.91:0/4157216007 conn(0x564aaa8991e0 msgr2=0x564aaa8e1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/4157216007 shutdown_connections 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.059+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/4157216007 wait complete. 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.060+0000 7fdbe1fc1740 1 -- start start 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.060+0000 7fdbe1fc1740 1 Processor -- start 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa800080 con 0x564aaa89aad0 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa7a1850 con 0x564aaa8b3c10 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564aaa7a14f0 con 0x564aaa8c6f50 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.061+0000 7fdbe1fc1740 10 client.0 _ll_register_callbacks cb 0x564aaa7f99c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47650/0 (socket says 172.21.15.91:47650) 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 -- 172.21.15.91:0/1668030936 learned_addr learned my addr 172.21.15.91:0/1668030936 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdd400640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 -- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 msgr2=0x564aaa8c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 -- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 msgr2=0x564aaa8e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 -- 172.21.15.91:0/1668030936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd814b480 con 0x564aaa89aad0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbdca00640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de64c147d80dbbf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.062+0000 7fdbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc804a070 con 0x564aaa89aad0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc80458d0 con 0x564aaa89aad0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aaa82a5d0 con 0x564aaa89aad0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc8045a90 con 0x564aaa89aad0 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 10 client.5646 fetch_fsmap learned FSMap version 24 2024-02-02T13:26:34.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x564aaa7fe700 con 0x564aaa89aad0 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564aaa82a5d0 con 0x564aaa89aad0 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbc8045bf0 con 0x564aaa89aad0 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 10 client.5646 fetch_fsmap finished waiting for FSMap version 24 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 10 client.5646 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 20 client.5646 populate_metadata read hostname 'smithi091' 2024-02-02T13:26:34.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8c7890 0x564aaa8c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 4 client.5646 mds_command: new command op to 5508 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.063+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 --> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564aaa7f28d0 con 0x564aaa8c7890 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.064+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8c7890 0x564aaa8c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.064+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8c7890 0x564aaa8c7c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb41189e707706e server_cookie=f6da48e60e6a93d4 in_seq=0 out_seq=0 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.064+0000 7fdbc6000640 10 client.5646 ms_handle_connect on v2:172.21.15.91:6836/681798855 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbc6000640 1 -- 172.21.15.91:0/1668030936 <== mds.0 v2:172.21.15.91:6836/681798855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564aaa7f28d0 con 0x564aaa8c7890 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbc6000640 10 client.5646 handle_command_reply: tid=0 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 client.5646 shutdown 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 msgr2=0x564aaa8b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8004730 comp rx=0 tx=0).stop 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 shutdown_connections 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8c7890 0x564aaa8c7c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8c6f50 0x564aaa8c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1668030936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa89aad0 0x564aaa8b3470 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbc6000640 0 client.5646 ms_handle_reset on v2:172.21.15.91:6836/681798855 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 >> 172.21.15.91:0/1668030936 conn(0x564aaa896ed0 msgr2=0x564aaa89b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 shutdown_connections 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.065+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1668030936 wait complete. 2024-02-02T13:26:34.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.066+0000 7fdbe1fc1740 20 client.5646 trim_cache size 0 max 16384 2024-02-02T13:26:34.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.082+0000 7fdbe1fc1740 1 Processor -- start 2024-02-02T13:26:34.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.082+0000 7fdbe1fc1740 1 -- start start 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 0x564aaa8c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa567f20 con 0x564aaa8c6f50 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa897980 con 0x564aaa8b3c10 2024-02-02T13:26:34.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564aaa8c9880 con 0x564aaa89aad0 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47660/0 (socket says 172.21.15.91:47660) 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 -- 172.21.15.91:0/1107896067 learned_addr learned my addr 172.21.15.91:0/1107896067 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 -- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 msgr2=0x564aaa8c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdca00640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 0x564aaa8c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 0x564aaa8c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 -- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 msgr2=0x564aaa8c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdd400640 1 -- 172.21.15.91:0/1107896067 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aaa834280 con 0x564aaa8c6f50 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.083+0000 7fdbdca00640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 0x564aaa8c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbdd400640 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e50 tx=0x7fdbd0009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5595d9aeebb6d7f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbcde00640 1 -- 172.21.15.91:0/1107896067 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000f410 con 0x564aaa8c6f50 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbcde00640 1 -- 172.21.15.91:0/1107896067 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd000f570 con 0x564aaa8c6f50 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbcde00640 1 -- 172.21.15.91:0/1107896067 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000f920 con 0x564aaa8c6f50 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1107896067 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 msgr2=0x564aaa8d9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.084+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e50 tx=0x7fdbd0009f40 comp rx=0 tx=0).stop 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1107896067 shutdown_connections 2024-02-02T13:26:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8d9a90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa8b3c10 0x564aaa8c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/1107896067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa89aad0 0x564aaa8c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1107896067 >> 172.21.15.91:0/1107896067 conn(0x564aaa896ed0 msgr2=0x564aaa8caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1107896067 shutdown_connections 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/1107896067 wait complete. 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 -- start start 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.085+0000 7fdbe1fc1740 1 Processor -- start 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa950170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 0x564aaa8f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa9506b0 con 0x564aaa8c6f50 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aaa8f4490 con 0x564aaa89aad0 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564aaa8c9de0 con 0x564aaa8b3c10 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.086+0000 7fdbe1fc1740 10 client.0 _ll_register_callbacks cb 0x7fdbc80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47672/0 (socket says 172.21.15.91:47672) 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 -- 172.21.15.91:0/232125592 learned_addr learned my addr 172.21.15.91:0/232125592 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 -- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 msgr2=0x564aaa8f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 0x564aaa8f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdca00640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa950170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 0x564aaa8f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 -- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 msgr2=0x564aaa950170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa950170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 -- 172.21.15.91:0/232125592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd0009a80 con 0x564aaa8c6f50 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbcfc00640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 0x564aaa8f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.087+0000 7fdbdd400640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x564aaa8b43b0 tx=0x7fdbd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156292ff9fe8d08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0023470 con 0x564aaa8c6f50 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbd00235d0 con 0x564aaa8c6f50 2024-02-02T13:26:34.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbc005b8a0 con 0x564aaa8c6f50 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd0023790 con 0x564aaa8c6f50 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 10 client.5652 fetch_fsmap learned FSMap version 24 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x564aaa7a0a60 con 0x564aaa8c6f50 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc005b8a0 con 0x564aaa8c6f50 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbd001f040 con 0x564aaa8c6f50 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 10 client.5652 fetch_fsmap finished waiting for FSMap version 24 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 10 client.5652 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 20 client.5652 populate_metadata read hostname 'smithi091' 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8d9fe0 0x564aaa8f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 4 client.5652 mds_command: new command op to 5508 tid=0[{"prefix": "status"}] 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.088+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 --> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564aaa7f9100 con 0x564aaa8d9fe0 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbdca00640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8d9fe0 0x564aaa8f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbdca00640 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8d9fe0 0x564aaa8f09c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cef3a7c68154ee1 server_cookie=92e479b8e0573cb2 in_seq=0 out_seq=0 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbc6000640 10 client.5652 ms_handle_connect on v2:172.21.15.91:6836/681798855 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbc6000640 1 -- 172.21.15.91:0/232125592 <== mds.0 v2:172.21.15.91:6836/681798855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564aaa7f9100 con 0x564aaa8d9fe0 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbc6000640 10 client.5652 handle_command_reply: tid=0 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.089+0000 7fdbe1fc1740 1 client.5652 shutdown 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 msgr2=0x564aaa8f0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x564aaa8b43b0 tx=0x7fdbd0004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 shutdown_connections 2024-02-02T13:26:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:6836/681798855,v1:172.21.15.91:6837/681798855] conn(0x564aaa8d9fe0 0x564aaa8f09c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aaa8b3c10 0x564aaa8f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564aaa8c6f50 0x564aaa8f0480 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 --2- 172.21.15.91:0/232125592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aaa89aad0 0x564aaa950170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbc6000640 0 client.5652 ms_handle_reset on v2:172.21.15.91:6836/681798855 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 >> 172.21.15.91:0/232125592 conn(0x564aaa896ed0 msgr2=0x564aaa8c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 shutdown_connections 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 1 -- 172.21.15.91:0/232125592 wait complete. 2024-02-02T13:26:34.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.090+0000 7fdbe1fc1740 20 client.5652 trim_cache size 0 max 16384 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5508, 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:26:34.100 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.6439970759999998, 2024-02-02T13:26:34.101 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 24, 2024-02-02T13:26:34.101 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 48, 2024-02-02T13:26:34.101 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 48, 2024-02-02T13:26:34.101 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 12.327660396000001 2024-02-02T13:26:34.101 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:34.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 msgr2=0x7fdbb006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 0x7fdbb006c8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8036000 tx=0x7fdbc803c040 comp rx=0 tx=0).stop 2024-02-02T13:26:34.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 msgr2=0x7fdbd80af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c430 tx=0x7fdbc000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 shutdown_connections 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbb006a430 0x7fdbb006c8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd815b0f0 0x7fdbd814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbd815ad10 0x7fdbd80af0d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 --2- 172.21.15.91:0/816142999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbd80cb580 0x7fdbd814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 >> 172.21.15.91:0/816142999 conn(0x7fdbd80b0700 msgr2=0x7fdbd8073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 shutdown_connections 2024-02-02T13:26:34.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.091+0000 7fdbdf400640 1 -- 172.21.15.91:0/816142999 wait complete. 2024-02-02T13:26:34.109 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:26:34.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.315+0000 7f9470c00640 1 Processor -- start 2024-02-02T13:26:34.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.315+0000 7f9470c00640 1 -- start start 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 0x7f946c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 0x7f946c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f946c0cd470 0x7f946c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0cc7c0 con 0x7f946c0cd470 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0ce930 con 0x7f946c0ce130 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.316+0000 7f9470c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c06c890 con 0x7f946c15a010 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f946c0cd470 0x7f946c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f946c0cd470 0x7f946c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47684/0 (socket says 172.21.15.91:47684) 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 -- 172.21.15.91:0/2481101752 learned_addr learned my addr 172.21.15.91:0/2481101752 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 -- 172.21.15.91:0/2481101752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 msgr2=0x7f946c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f9469400640 1 --2- 172.21.15.91:0/2481101752 >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 0x7f946c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f9469e00640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 0x7f946c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 0x7f946c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 -- 172.21.15.91:0/2481101752 >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 msgr2=0x7f946c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 --2- 172.21.15.91:0/2481101752 >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 0x7f946c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f946a800640 1 -- 172.21.15.91:0/2481101752 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c06cb70 con 0x7f946c0cd470 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f9469e00640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 0x7f946c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.317+0000 7f9469400640 1 --2- 172.21.15.91:0/2481101752 >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 0x7f946c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.319+0000 7f946a800640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0cd470 0x7f946c0b0700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f946c06d7d0 tx=0x7f946402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583e1e41cb71e631 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.319+0000 7f9468a00640 1 -- 172.21.15.91:0/2481101752 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946402ec80 con 0x7f946c0cd470 2024-02-02T13:26:34.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.319+0000 7f9468a00640 1 -- 172.21.15.91:0/2481101752 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946402ede0 con 0x7f946c0cd470 2024-02-02T13:26:34.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.319+0000 7f9468a00640 1 -- 172.21.15.91:0/2481101752 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464036610 con 0x7f946c0cd470 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 -- 172.21.15.91:0/2481101752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0cd470 msgr2=0x7f946c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0cd470 0x7f946c0b0700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f946c06d7d0 tx=0x7f946402e0e0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 -- 172.21.15.91:0/2481101752 shutdown_connections 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0cd470 0x7f946c0b0700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 --2- 172.21.15.91:0/2481101752 >> v2:172.21.15.113:3300/0 conn(0x7f946c0ce130 0x7f946c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 --2- 172.21.15.91:0/2481101752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c15a010 0x7f946c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 -- 172.21.15.91:0/2481101752 >> 172.21.15.91:0/2481101752 conn(0x7f946c0cec80 msgr2=0x7f946c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 -- 172.21.15.91:0/2481101752 shutdown_connections 2024-02-02T13:26:34.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.320+0000 7f9470c00640 1 -- 172.21.15.91:0/2481101752 wait complete. 2024-02-02T13:26:34.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.322+0000 7f9470c00640 1 Processor -- start 2024-02-02T13:26:34.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.322+0000 7f9470c00640 1 -- start start 2024-02-02T13:26:34.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f946c15a010 0x7f946c0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c149450 0x7f946c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1498d0 con 0x7f946c0ce130 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c14ca90 con 0x7f946c15a010 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9470c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c8a80 con 0x7f946c149450 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9469e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9469e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47690/0 (socket says 172.21.15.91:47690) 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.323+0000 7f9469e00640 1 -- 172.21.15.91:0/1487758804 learned_addr learned my addr 172.21.15.91:0/1487758804 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f946a800640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c149450 0x7f946c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469400640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f946c15a010 0x7f946c0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 -- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c149450 msgr2=0x7f946c14c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c149450 0x7f946c14c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 -- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f946c15a010 msgr2=0x7f946c0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f946c15a010 0x7f946c0c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 -- 172.21.15.91:0/1487758804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009b30 con 0x7f946c0ce130 2024-02-02T13:26:34.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9469e00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c7f0 tx=0x7f945c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.324+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c007820 con 0x7f946c0ce130 2024-02-02T13:26:34.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.325+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c007980 con 0x7f946c0ce130 2024-02-02T13:26:34.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.325+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0b3170 con 0x7f946c0ce130 2024-02-02T13:26:34.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.325+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c007cf0 con 0x7f946c0ce130 2024-02-02T13:26:34.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.325+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0b3730 con 0x7f946c0ce130 2024-02-02T13:26:34.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.326+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c066cf0 con 0x7f946c0ce130 2024-02-02T13:26:34.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.327+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f945c0113f0 con 0x7f946c0ce130 2024-02-02T13:26:34.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.328+0000 7f9462000640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 0x7f944406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:34.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.328+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f945c088bc0 con 0x7f946c0ce130 2024-02-02T13:26:34.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.328+0000 7f9469400640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 0x7f944406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:34.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.329+0000 7f9469400640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 0x7f944406c8f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f946c157cd0 tx=0x7f9454008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:34.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.332+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f945c0541a0 con 0x7f946c0ce130 2024-02-02T13:26:34.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.565+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f946c0c61f0 con 0x7f946c0ce130 2024-02-02T13:26:34.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.566+0000 7f9462000640 1 -- 172.21.15.91:0/1487758804 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f945c01f070 con 0x7f946c0ce130 2024-02-02T13:26:34.567 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:26:34.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.568+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 msgr2=0x7f944406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.568+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 0x7f944406c8f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f946c157cd0 tx=0x7f9454008040 comp rx=0 tx=0).stop 2024-02-02T13:26:34.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.568+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 msgr2=0x7f946c149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.568+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c7f0 tx=0x7f945c00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 shutdown_connections 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f944406a430 0x7f944406c8f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946c149450 0x7f946c14c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f946c15a010 0x7f946c0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 --2- 172.21.15.91:0/1487758804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c0ce130 0x7f946c149b10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 >> 172.21.15.91:0/1487758804 conn(0x7f946c0cec80 msgr2=0x7f946c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 shutdown_connections 2024-02-02T13:26:34.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:34.569+0000 7f9470c00640 1 -- 172.21.15.91:0/1487758804 wait complete. 2024-02-02T13:26:34.588 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:26:34.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:26:34.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:26:34.589 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:26:34.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:34.589 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:34.638 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42531sec preferred_lft 42531sec 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:34.639 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:26:34.640 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:34.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:34.641 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:26:34.732 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:26:34.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:34.737 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:26:34.787 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:26:34.787 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:26:34.787 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:26:34.787 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:26:34.787 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:26:34.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:26:34.788 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:26:34.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:34.961 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:26:35.011 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:26:35.011 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:26:35.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:26:35.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:35.012 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:26:35.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:26:35.100 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:26:35.152 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:26:35.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:26:35.153 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:26:35.199 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:26:35.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:35.199 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:35.333 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:35.333 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:26:35.366 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:35.366 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:26:35.366 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:26:35.366 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:26:35.366 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:26:35.368 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:26:35.368 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:26:35.368 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:26:35.368 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:26:35.368 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:26:35.373 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:26:35.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:35.373 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:26:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:35.449 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:26:35.511 INFO:teuthology.orchestra.run.smithi091.stdout:client5661 2024-02-02T13:26:35.512 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:35.512 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:35.565 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:26:35.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.769+0000 7f5ca2c00640 1 Processor -- start 2024-02-02T13:26:35.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.769+0000 7f5ca2c00640 1 -- start start 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c06ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca2c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c160e60 con 0x7f5c9c15d430 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca2c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cbca0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cee70 con 0x7f5c9c15d050 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.771+0000 7f5ca0c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47696/0 (socket says 172.21.15.91:47696) 2024-02-02T13:26:35.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 -- 172.21.15.91:0/2014380610 learned_addr learned my addr 172.21.15.91:0/2014380610 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:35.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5c9be00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c06ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:35.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 -- 172.21.15.91:0/2014380610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 msgr2=0x7f5c9c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:35.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c06ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 -- 172.21.15.91:0/2014380610 >> v1:172.21.15.113:6789/0 conn(0x7f5c9c15d430 legacy=0x7f5c9c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:35.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.772+0000 7f5ca0c00640 1 -- 172.21.15.91:0/2014380610 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c1610b0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.773+0000 7f5ca0c00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009e80 tx=0x7f5c9402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985dea5a1a1433d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:35.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.773+0000 7f5c9b400640 1 -- 172.21.15.91:0/2014380610 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9402ca50 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.773+0000 7f5c9b400640 1 -- 172.21.15.91:0/2014380610 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9402cbb0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.773+0000 7f5c9b400640 1 -- 172.21.15.91:0/2014380610 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940343f0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 -- 172.21.15.91:0/2014380610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 msgr2=0x7f5c9c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009e80 tx=0x7f5c9402beb0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 -- 172.21.15.91:0/2014380610 shutdown_connections 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c0ce930 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/2014380610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c06ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 -- 172.21.15.91:0/2014380610 >> 172.21.15.91:0/2014380610 conn(0x7f5c9c0cbfc0 msgr2=0x7f5c9c0cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 -- 172.21.15.91:0/2014380610 shutdown_connections 2024-02-02T13:26:35.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.774+0000 7f5ca2c00640 1 -- 172.21.15.91:0/2014380610 wait complete. 2024-02-02T13:26:35.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.777+0000 7f5ca2c00640 1 Processor -- start 2024-02-02T13:26:35.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.777+0000 7f5ca2c00640 1 -- start start 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c9c15d430 0x7f5c9c158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c8cc0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c8fa0 con 0x7f5c9c15d430 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5ca2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c9280 con 0x7f5c9c15d050 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5c9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:35.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5c9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47698/0 (socket says 172.21.15.91:47698) 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.778+0000 7f5c9be00640 1 -- 172.21.15.91:0/1773608131 learned_addr learned my addr 172.21.15.91:0/1773608131 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5ca0c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c9c15d430 0x7f5c9c158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 -- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 msgr2=0x7f5c9c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 -- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c9c15d430 msgr2=0x7f5c9c158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c9c15d430 0x7f5c9c158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 -- 172.21.15.91:0/1773608131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c94009b30 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c9be00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c7f0 tx=0x7f5c8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004220 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0043a0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c0c4260 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.779+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c007860 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.780+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c15d920 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.781+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c066980 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.782+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5c8c007bd0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.783+0000 7f5c98c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 0x7f5c7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.783+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c8c088a40 con 0x7f5c9c0ce4c0 2024-02-02T13:26:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.783+0000 7f5c93200640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 0x7f5c7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:35.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.784+0000 7f5c93200640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 0x7f5c7406c8f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006fd0 tx=0x7f5c84006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:35.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:35.788+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5c8c053f90 con 0x7f5c9c0ce4c0 2024-02-02T13:26:36.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.016+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c9c0c40c0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:36.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.017+0000 7f5c98c00640 1 -- 172.21.15.91:0/1773608131 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5c9c0c40c0 con 0x7f5c9c0ce4c0 2024-02-02T13:26:36.019 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 msgr2=0x7f5c7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 0x7f5c7406c8f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006fd0 tx=0x7f5c84006bc0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 msgr2=0x7f5c9c14f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c7f0 tx=0x7f5c8c00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 shutdown_connections 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5c7406a430 0x7f5c7406c8f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c9c15d050 0x7f5c9c0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c9c15d430 0x7f5c9c158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 --2- 172.21.15.91:0/1773608131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c9c0ce4c0 0x7f5c9c14f470 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 >> 172.21.15.91:0/1773608131 conn(0x7f5c9c0cbfc0 msgr2=0x7f5c9c0c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 shutdown_connections 2024-02-02T13:26:36.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:36.020+0000 7f5ca2c00640 1 -- 172.21.15.91:0/1773608131 wait complete. 2024-02-02T13:26:36.039 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:26:36.040 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:26:36.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.040 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:26:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:26:36.099 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:26:36.100 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:26:36.101 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:26:36.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.102 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:26:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:26:36.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.189 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:26:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:26:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:26:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:26:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:26:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:26:36.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:26:36.242 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:26:36.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.423 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:26:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:26:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:26:36.476 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:26:36.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:26:36.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:26:36.570 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:26:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:26:36.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:26:36.625 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:26:36.680 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:26:36.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:26:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:26:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:26:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:26:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:26:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:26:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:26:36.854 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:26:36.854 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:26:36.854 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:26:36.854 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:26:36.858 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:26:36.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:26:36.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:36.943 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:26:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:client4697 2024-02-02T13:26:36.999 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:26:36.999 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:37.048 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:37.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.262+0000 7f8295200640 1 Processor -- start 2024-02-02T13:26:37.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.262+0000 7f8295200640 1 -- start start 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 0x7f829015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 0x7f82900b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82900b0c40 0x7f82900b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900b10d0 con 0x7f82900b0c40 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8290070f80 con 0x7f829006f4f0 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.263+0000 7f8295200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8290152370 con 0x7f8290151f90 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82900b0c40 0x7f82900b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82900b0c40 0x7f82900b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47710/0 (socket says 172.21.15.91:47710) 2024-02-02T13:26:37.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 -- 172.21.15.91:0/1706880387 learned_addr learned my addr 172.21.15.91:0/1706880387 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:37.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828de00640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 0x7f829015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f8285200640 1 --2- 172.21.15.91:0/1706880387 >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 0x7f82900b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 -- 172.21.15.91:0/1706880387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 msgr2=0x7f829015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 0x7f829015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 -- 172.21.15.91:0/1706880387 >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 msgr2=0x7f82900b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 --2- 172.21.15.91:0/1706880387 >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 0x7f82900b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828e800640 1 -- 172.21.15.91:0/1706880387 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900cb010 con 0x7f82900b0c40 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f828de00640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 0x7f829015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:37.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.264+0000 7f8285200640 1 --2- 172.21.15.91:0/1706880387 >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 0x7f82900b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:37.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.266+0000 7f828e800640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82900b0c40 0x7f82900b3af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f82900ceac0 tx=0x7f828802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9327bbade288a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:37.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.266+0000 7f828d400640 1 -- 172.21.15.91:0/1706880387 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828802ca80 con 0x7f82900b0c40 2024-02-02T13:26:37.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.266+0000 7f828d400640 1 -- 172.21.15.91:0/1706880387 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828802cbe0 con 0x7f82900b0c40 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.266+0000 7f828d400640 1 -- 172.21.15.91:0/1706880387 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880343f0 con 0x7f82900b0c40 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 -- 172.21.15.91:0/1706880387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82900b0c40 msgr2=0x7f82900b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82900b0c40 0x7f82900b3af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f82900ceac0 tx=0x7f828802bee0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 -- 172.21.15.91:0/1706880387 shutdown_connections 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82900b0c40 0x7f82900b3af0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 --2- 172.21.15.91:0/1706880387 >> v2:172.21.15.113:3300/0 conn(0x7f829006f4f0 0x7f82900b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 --2- 172.21.15.91:0/1706880387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8290151f90 0x7f829015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 -- 172.21.15.91:0/1706880387 >> 172.21.15.91:0/1706880387 conn(0x7f82900cec80 msgr2=0x7f82900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 -- 172.21.15.91:0/1706880387 shutdown_connections 2024-02-02T13:26:37.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.267+0000 7f8295200640 1 -- 172.21.15.91:0/1706880387 wait complete. 2024-02-02T13:26:37.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.269+0000 7f8295200640 1 Processor -- start 2024-02-02T13:26:37.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.269+0000 7f8295200640 1 -- start start 2024-02-02T13:26:37.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829006f4f0 0x7f82901f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82900b0c40 0x7f82900ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901535c0 con 0x7f8290151f90 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901f64b0 con 0x7f829006f4f0 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.270+0000 7f8295200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82901f66e0 con 0x7f82900b0c40 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47712/0 (socket says 172.21.15.91:47712) 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 -- 172.21.15.91:0/1075249440 learned_addr learned my addr 172.21.15.91:0/1075249440 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828de00640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829006f4f0 0x7f82901f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f8285200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82900b0c40 0x7f82900ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 -- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82900b0c40 msgr2=0x7f82900ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82900b0c40 0x7f82900ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 -- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829006f4f0 msgr2=0x7f82901f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829006f4f0 0x7f82901f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 -- 172.21.15.91:0/1075249440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009b30 con 0x7f8290151f90 2024-02-02T13:26:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.271+0000 7f828e800640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8288036040 tx=0x7f8288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.272+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288009510 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.272+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828803e450 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.272+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82901f69c0 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.272+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828803e800 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.272+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901537d0 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.273+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290066cf0 con 0x7f8290151f90 2024-02-02T13:26:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.274+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f828802cd50 con 0x7f8290151f90 2024-02-02T13:26:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.275+0000 7f8286a00640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 0x7f826806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:37.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.275+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82880ab500 con 0x7f8290151f90 2024-02-02T13:26:37.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.276+0000 7f828de00640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 0x7f826806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:37.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.276+0000 7f828de00640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 0x7f826806c8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82900b4750 tx=0x7f8280008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:37.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.280+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8288076a50 con 0x7f8290151f90 2024-02-02T13:26:37.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.508+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f829006c7b0 con 0x7f8290151f90 2024-02-02T13:26:37.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.509+0000 7f8286a00640 1 -- 172.21.15.91:0/1075249440 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f828803e960 con 0x7f8290151f90 2024-02-02T13:26:37.511 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:37.511 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:26:37.511 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 msgr2=0x7f826806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 0x7f826806c8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82900b4750 tx=0x7f8280008040 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 msgr2=0x7f8290153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8288036040 tx=0x7f8288004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 shutdown_connections 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f826806a430 0x7f826806c8f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82900b0c40 0x7f82900ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8290151f90 0x7f8290153cf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 --2- 172.21.15.91:0/1075249440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829006f4f0 0x7f82901f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:37.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 >> 172.21.15.91:0/1075249440 conn(0x7f82900cec80 msgr2=0x7f82900ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:37.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 shutdown_connections 2024-02-02T13:26:37.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:37.512+0000 7f8295200640 1 -- 172.21.15.91:0/1075249440 wait complete. 2024-02-02T13:26:37.535 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:26:37.760 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5508,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.297965232999999,"mdsmap_epoch":24,"osdmap_epoch":48,"osdmap_epoch_barrier":48,"uptime":15.981641103999999} 2024-02-02T13:26:37.760 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5508, "mdsmap_epoch": 24, "osdmap_epoch": 48, "osdmap_epoch_barrier": 48, "rank_uptime": 10.297965233, "state": "up:active", "uptime": 15.981641104, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:26:37.761 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:26:37.986 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:26:37.987 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:26:37.987 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-02-02T13:26:37.987 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:37.987 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:38.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:38.036 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-02-02T13:26:40.011 INFO:teuthology.orchestra.run.smithi091.stderr:265+0 records in 2024-02-02T13:26:40.011 INFO:teuthology.orchestra.run.smithi091.stderr:265+0 records out 2024-02-02T13:26:40.011 INFO:teuthology.orchestra.run.smithi091.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.87458 s, 148 MB/s 2024-02-02T13:26:40.012 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:40.012 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:40.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:40.059 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-02-02T13:26:42.202 INFO:teuthology.orchestra.run.smithi091.stderr:292+0 records in 2024-02-02T13:26:42.202 INFO:teuthology.orchestra.run.smithi091.stderr:292+0 records out 2024-02-02T13:26:42.202 INFO:teuthology.orchestra.run.smithi091.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.04775 s, 150 MB/s 2024-02-02T13:26:42.203 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-02-02T13:26:42.203 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:26:42.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.376+0000 7f2518e00640 1 Processor -- start 2024-02-02T13:26:42.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.376+0000 7f2518e00640 1 -- start start 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2518e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f2514160d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2518e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140ca850 con 0x7f25140adf60 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2518e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140cb3f0 con 0x7f25140cd7c0 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2518e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140cdd40 con 0x7f25140cd360 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2511e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f25140adf60 0x7f25140cace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:38040/0 (socket says 172.21.15.91:38040) 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.378+0000 7f2511e00640 1 -- 172.21.15.91:0/3117699238 learned_addr learned my addr 172.21.15.91:0/3117699238 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899934915 0 0) 0x7f25140ca850 con 0x7f25140adf60 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f4003520 con 0x7f25140adf60 2024-02-02T13:26:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2511400640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f2514160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 477940919 0 0) 0x7f24f4003520 con 0x7f25140adf60 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 msgr2=0x7f2514160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f2514160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 >> v1:172.21.15.113:6789/0 conn(0x7f25140cd7c0 legacy=0x7f25140ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140c9f20 con 0x7f25140adf60 2024-02-02T13:26:42.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f25040019f0 con 0x7f25140adf60 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 >> v1:172.21.15.91:6789/0 conn(0x7f25140adf60 legacy=0x7f25140cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039a0 0x7f24f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f24f4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140cdd40 con 0x7f24f40039a0 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f400bb40 con 0x7f24f40047c0 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f400bd00 con 0x7f25140cd360 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.379+0000 7f2511e00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039a0 0x7f24f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2512800640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f24f4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511400640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511e00640 1 -- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 msgr2=0x7f24f4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511e00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140cd360 0x7f24f4005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511e00640 1 -- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 msgr2=0x7f24f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511e00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511e00640 1 -- 172.21.15.91:0/3117699238 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4003520 con 0x7f24f40039a0 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.380+0000 7f2511400640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.381+0000 7f2511e00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039a0 0x7f24f4003da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f250400a830 tx=0x7f24f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c84bd498b564db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.381+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504034630 con 0x7f24f40039a0 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.381+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250402f070 con 0x7f24f40039a0 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.381+0000 7f2510a00640 1 -- 172.21.15.91:0/3117699238 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040349f0 con 0x7f24f40039a0 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 -- 172.21.15.91:0/3117699238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039a0 msgr2=0x7f24f4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 --2- 172.21.15.91:0/3117699238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039a0 0x7f24f4003da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f250400a830 tx=0x7f24f40057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2512800640 1 -- 172.21.15.91:0/3117699238 reap_dead start 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 -- 172.21.15.91:0/3117699238 shutdown_connections 2024-02-02T13:26:42.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 -- 172.21.15.91:0/3117699238 >> 172.21.15.91:0/3117699238 conn(0x7f25140ceb70 msgr2=0x7f25140cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:42.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 -- 172.21.15.91:0/3117699238 shutdown_connections 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.382+0000 7f2518e00640 1 -- 172.21.15.91:0/3117699238 wait complete. 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.384+0000 7f2518e00640 1 Processor -- start 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.384+0000 7f2518e00640 1 -- start start 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40039a0 0x7f25141542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140b2af0 0x7f25140b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c8170 con 0x7f25140c77d0 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b37e0 con 0x7f24f40039a0 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2518e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b3ac0 con 0x7f25140b2af0 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60022/0 (socket says 172.21.15.91:60022) 2024-02-02T13:26:42.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 -- 172.21.15.91:0/3656236186 learned_addr learned my addr 172.21.15.91:0/3656236186 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2512800640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140b2af0 0x7f25140b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40039a0 0x7f25141542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 -- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140b2af0 msgr2=0x7f25140b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140b2af0 0x7f25140b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 -- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40039a0 msgr2=0x7f25141542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40039a0 0x7f25141542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.385+0000 7f2511400640 1 -- 172.21.15.91:0/3656236186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140b3da0 con 0x7f25140c77d0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f2511400640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e7e0 tx=0x7f24fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00c6d0 con 0x7f25140c77d0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00c830 con 0x7f25140c77d0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00cbe0 con 0x7f25140c77d0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140b4080 con 0x7f25140c77d0 2024-02-02T13:26:42.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.386+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2514154830 con 0x7f25140c77d0 2024-02-02T13:26:42.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.387+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25141613d0 con 0x7f25140c77d0 2024-02-02T13:26:42.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.388+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f24fc00cd40 con 0x7f25140c77d0 2024-02-02T13:26:42.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.389+0000 7f250a000640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:42.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.389+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24fc088740 con 0x7f25140c77d0 2024-02-02T13:26:42.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.389+0000 7f2511e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:42.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.390+0000 7f2511e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2504034000 tx=0x7f250402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:42.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.393+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f24fc053c00 con 0x7f25140c77d0 2024-02-02T13:26:42.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.580+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2514066cf0 con 0x7f25140c77d0 2024-02-02T13:26:42.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.583+0000 7f250a000640 1 -- 172.21.15.91:0/3656236186 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+33480 (secure 0 0 0) 0x7f24fc057220 con 0x7f25140c77d0 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 48, 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:28.444632+0000", 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:26:42.585 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:26:42.586 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:42.587 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:26:42.588 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "16", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:42.589 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:42.590 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:42.591 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:26:42.592 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:26:42.593 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:26:42.594 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "47", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:42.595 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:42.596 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:42.597 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:42.598 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.599 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:42.600 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.601 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "48", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:42.602 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:42.603 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:42.604 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:42.605 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.606 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.607 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.608 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:42.609 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.610 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.611 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:26:42.612 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.613 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.614 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.615 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.616 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.617 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.618 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.619 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.620 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.621 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.622 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.624 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:26:42.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.626 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:26:42.627 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.628 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.629 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:26:42.630 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.631 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.632 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.633 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:26:42.634 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.635 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:26:42.636 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.637 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.638 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.639 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:26:42.641 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:42.642 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.643 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.645 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:26:42.646 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.647 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.648 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.649 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.650 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:42.652 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:26:42.653 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:42.654 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 msgr2=0x7f24ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2504034000 tx=0x7f250402f000 comp rx=0 tx=0).stop 2024-02-02T13:26:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 msgr2=0x7f25140c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e7e0 tx=0x7f24fc00eca0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 shutdown_connections 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140b2af0 0x7f25140b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25140c77d0 0x7f25140c7c30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 --2- 172.21.15.91:0/3656236186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40039a0 0x7f25141542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.585+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 >> 172.21.15.91:0/3656236186 conn(0x7f25140ceb70 msgr2=0x7f25140aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.586+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 shutdown_connections 2024-02-02T13:26:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:42.586+0000 7f2518e00640 1 -- 172.21.15.91:0/3656236186 wait complete. 2024-02-02T13:26:42.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:26:43.321 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:26:43.319+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-02-02T13:26:47.659 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:26:47.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.880+0000 7fbf17600640 1 Processor -- start 2024-02-02T13:26:47.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.880+0000 7fbf17600640 1 -- start start 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.881+0000 7fbf17600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf1015b0f0 0x7fbf1006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.881+0000 7fbf17600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1006d910 con 0x7fbf100cb580 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.881+0000 7fbf17600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf100cc850 con 0x7fbf1015ad10 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.881+0000 7fbf17600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1006d730 con 0x7fbf1015b0f0 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf15600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbf100cb580 0x7fbf100c26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:38054/0 (socket says 172.21.15.91:38054) 2024-02-02T13:26:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf15600640 1 -- 172.21.15.91:0/2807500080 learned_addr learned my addr 172.21.15.91:0/2807500080 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf07c00640 1 --2- 172.21.15.91:0/2807500080 >> v2:172.21.15.113:3300/0 conn(0x7fbf1015b0f0 0x7fbf1006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605948142 0 0) 0x7fbf1006d910 con 0x7fbf100cb580 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef0003520 con 0x7fbf100cb580 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1379259165 0 0) 0x7fbef0003520 con 0x7fbf100cb580 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 >> v2:172.21.15.113:3300/0 conn(0x7fbf1015b0f0 msgr2=0x7fbf1006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 --2- 172.21.15.91:0/2807500080 >> v2:172.21.15.113:3300/0 conn(0x7fbf1015b0f0 0x7fbf1006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 >> v1:172.21.15.113:6789/0 conn(0x7fbf1015ad10 legacy=0x7fbf1006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100ccb30 con 0x7fbf100cb580 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fbf08001670 con 0x7fbf100cb580 2024-02-02T13:26:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 >> v1:172.21.15.91:6789/0 conn(0x7fbf100cb580 legacy=0x7fbf100c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbef0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.882+0000 7fbf0fe00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbef00047c0 0x7fbef0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf0fe00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef00051d0 0x7fbef000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1006d730 con 0x7fbef00039a0 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef000c100 con 0x7fbef00051d0 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef000c2c0 con 0x7fbef00047c0 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbef0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 -- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbef00047c0 msgr2=0x7fbef0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbef00047c0 0x7fbef0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 -- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef00051d0 msgr2=0x7fbef000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef00051d0 0x7fbef000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf14c00640 1 -- 172.21.15.91:0/2807500080 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0003520 con 0x7fbef00039a0 2024-02-02T13:26:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.883+0000 7fbf15600640 1 -- 172.21.15.91:0/2807500080 reap_dead start 2024-02-02T13:26:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.884+0000 7fbf14c00640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbef0003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001e50 tx=0x7fbef00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb548e564b54676 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00003b70 con 0x7fbef00039a0 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00003cd0 con 0x7fbef00039a0 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf0fe00640 1 -- 172.21.15.91:0/2807500080 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00005310 con 0x7fbef00039a0 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 -- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 msgr2=0x7fbef0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbef0003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001e50 tx=0x7fbef00057f0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 -- 172.21.15.91:0/2807500080 shutdown_connections 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 --2- 172.21.15.91:0/2807500080 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbef0003dc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 -- 172.21.15.91:0/2807500080 >> 172.21.15.91:0/2807500080 conn(0x7fbf100b0700 msgr2=0x7fbf100b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 -- 172.21.15.91:0/2807500080 shutdown_connections 2024-02-02T13:26:47.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.885+0000 7fbf17600640 1 -- 172.21.15.91:0/2807500080 wait complete. 2024-02-02T13:26:47.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.887+0000 7fbf17600640 1 Processor -- start 2024-02-02T13:26:47.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.887+0000 7fbf17600640 1 -- start start 2024-02-02T13:26:47.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf10150a60 0x7fbf101f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf101f64d0 0x7fbf101f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c8c10 con 0x7fbef00039a0 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf101538b0 con 0x7fbf10150a60 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf17600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf101f3950 con 0x7fbf101f64d0 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60048/0 (socket says 172.21.15.91:60048) 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.888+0000 7fbf14c00640 1 -- 172.21.15.91:0/476192281 learned_addr learned my addr 172.21.15.91:0/476192281 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf15600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf101f64d0 0x7fbf101f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf07c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf10150a60 0x7fbf101f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 -- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf101f64d0 msgr2=0x7fbf101f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf101f64d0 0x7fbf101f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 -- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf10150a60 msgr2=0x7fbf101f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf10150a60 0x7fbf101f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:47.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10152810 con 0x7fbef00039a0 2024-02-02T13:26:47.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.889+0000 7fbf14c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf0002bf70 tx=0x7fbf000014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:47.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.890+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0003e070 con 0x7fbef00039a0 2024-02-02T13:26:47.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.890+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0002f440 con 0x7fbef00039a0 2024-02-02T13:26:47.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.890+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf10152af0 con 0x7fbef00039a0 2024-02-02T13:26:47.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.890+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0002f7b0 con 0x7fbef00039a0 2024-02-02T13:26:47.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.890+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf101530b0 con 0x7fbef00039a0 2024-02-02T13:26:47.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.891+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10066cf0 con 0x7fbef00039a0 2024-02-02T13:26:47.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.892+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbf0002f910 con 0x7fbef00039a0 2024-02-02T13:26:47.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.893+0000 7fbf0d600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 0x7fbee806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:47.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.893+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf000aa140 con 0x7fbef00039a0 2024-02-02T13:26:47.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.893+0000 7fbf07c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 0x7fbee806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:47.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.893+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=53}) v3 -- 0x7fbee8071f60 con 0x7fbef00039a0 2024-02-02T13:26:47.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.894+0000 7fbf07c00640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 0x7fbee806c8f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015e790 tx=0x7fbef8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:47.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:47.898+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbf00047060 con 0x7fbef00039a0 2024-02-02T13:26:48.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.088+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbf1015a3a0 con 0x7fbef00039a0 2024-02-02T13:26:48.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.090+0000 7fbf0d600640 1 -- 172.21.15.91:0/476192281 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+33623 (secure 0 0 0) 0x7fbf00047070 con 0x7fbef00039a0 2024-02-02T13:26:48.092 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 52, 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:47.358839+0000", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:26:48.093 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:26:48.094 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:48.095 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:48.096 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:48.097 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:48.098 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.099 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.100 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:26:48.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:48.102 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:48.103 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:48.105 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:48.105 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:48.106 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.107 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:26:48.108 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:48.109 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:26:48.123 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:48.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:48.125 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:48.126 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.127 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:48.128 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.129 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.130 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:26:48.132 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.133 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.134 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:26:48.136 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.138 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.139 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:26:48.143 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.146 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.148 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.149 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:26:48.151 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.154 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:26:48.157 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:48.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.159 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:26:48.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:26:48.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.164 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:26:48.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:26:48.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:48.169 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.170 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.172 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.173 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:48.174 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:26:48.175 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:26:48.176 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:48.177 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 msgr2=0x7fbee806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 0x7fbee806c8f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015e790 tx=0x7fbef8008040 comp rx=0 tx=0).stop 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 msgr2=0x7fbf101f3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf0002bf70 tx=0x7fbf000014a0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 shutdown_connections 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbee806a430 0x7fbee806c8f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf101f64d0 0x7fbf101f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf10150a60 0x7fbf101f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 --2- 172.21.15.91:0/476192281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbef00039a0 0x7fbf101f3410 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 >> 172.21.15.91:0/476192281 conn(0x7fbf100b0700 msgr2=0x7fbf1015dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:48.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 shutdown_connections 2024-02-02T13:26:48.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.093+0000 7fbf17600640 1 -- 172.21.15.91:0/476192281 wait complete. 2024-02-02T13:26:48.179 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-02-02T13:26:48.179 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:48.180 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:48.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:48.235 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-02-02T13:26:48.669 INFO:teuthology.orchestra.run.smithi091.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-02-02T13:26:48.669 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:26:48.669 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:26:48.670 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.372684 s, 141 MB/s 2024-02-02T13:26:48.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:26:48.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:26:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.869+0000 7faf61e00640 1 Processor -- start 2024-02-02T13:26:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.869+0000 7faf61e00640 1 -- start start 2024-02-02T13:26:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf61e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 0x7faf5c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf61e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf5c0cb580 0x7faf5c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf61e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c06d910 con 0x7faf5c15b0f0 2024-02-02T13:26:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf61e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c06d730 con 0x7faf5c0cb580 2024-02-02T13:26:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf61e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0cbf50 con 0x7faf5c15ad10 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf5c0cb580 0x7faf5c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf5c0cb580 0x7faf5c0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60062/0 (socket says 172.21.15.91:60062) 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 -- 172.21.15.91:0/2916468064 learned_addr learned my addr 172.21.15.91:0/2916468064 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 -- 172.21.15.91:0/2916468064 >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 msgr2=0x7faf5c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5aa00640 1 --2- 172.21.15.91:0/2916468064 >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 0x7faf5c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 --2- 172.21.15.91:0/2916468064 >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 0x7faf5c06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.871+0000 7faf5b400640 1 -- 172.21.15.91:0/2916468064 >> v1:172.21.15.113:6789/0 conn(0x7faf5c15b0f0 legacy=0x7faf5c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.872+0000 7faf5b400640 1 -- 172.21.15.91:0/2916468064 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0cc850 con 0x7faf5c0cb580 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.872+0000 7faf5aa00640 1 --2- 172.21.15.91:0/2916468064 >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 0x7faf5c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf5b400640 1 --2- 172.21.15.91:0/2916468064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0cba10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faf54009e50 tx=0x7faf5402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b725e8bd57394c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf59600640 1 -- 172.21.15.91:0/2916468064 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5402c9e0 con 0x7faf5c0cb580 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf59600640 1 -- 172.21.15.91:0/2916468064 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5402cb40 con 0x7faf5c0cb580 2024-02-02T13:26:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf59600640 1 -- 172.21.15.91:0/2916468064 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540343f0 con 0x7faf5c0cb580 2024-02-02T13:26:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf61e00640 1 -- 172.21.15.91:0/2916468064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 msgr2=0x7faf5c0cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf61e00640 1 --2- 172.21.15.91:0/2916468064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0cba10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faf54009e50 tx=0x7faf5402be40 comp rx=0 tx=0).stop 2024-02-02T13:26:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf61e00640 1 -- 172.21.15.91:0/2916468064 shutdown_connections 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf61e00640 1 --2- 172.21.15.91:0/2916468064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0cba10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.873+0000 7faf61e00640 1 --2- 172.21.15.91:0/2916468064 >> v2:172.21.15.113:3300/0 conn(0x7faf5c15ad10 0x7faf5c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.874+0000 7faf61e00640 1 -- 172.21.15.91:0/2916468064 >> 172.21.15.91:0/2916468064 conn(0x7faf5c0b0700 msgr2=0x7faf5c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.874+0000 7faf61e00640 1 -- 172.21.15.91:0/2916468064 shutdown_connections 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.874+0000 7faf61e00640 1 -- 172.21.15.91:0/2916468064 wait complete. 2024-02-02T13:26:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.875+0000 7faf61e00640 1 Processor -- start 2024-02-02T13:26:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.875+0000 7faf61e00640 1 -- start start 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 0x7faf5c0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c15b0f0 0x7faf5c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0c6180 con 0x7faf5c0cb580 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0cb260 con 0x7faf5c15b0f0 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.876+0000 7faf61e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0c6390 con 0x7faf5c15ad10 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60074/0 (socket says 172.21.15.91:60074) 2024-02-02T13:26:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 -- 172.21.15.91:0/1891949281 learned_addr learned my addr 172.21.15.91:0/1891949281 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:48.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5b400640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c15b0f0 0x7faf5c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5a000640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 0x7faf5c0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 -- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 msgr2=0x7faf5c0c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 0x7faf5c0c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 -- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c15b0f0 msgr2=0x7faf5c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c15b0f0 0x7faf5c0cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54009b30 con 0x7faf5c0cb580 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5a000640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 0x7faf5c0c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.877+0000 7faf5aa00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c7f0 tx=0x7faf4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.878+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c007a30 con 0x7faf5c0cb580 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.878+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c007b90 con 0x7faf5c0cb580 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.878+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ce40 con 0x7faf5c0cb580 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.878+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c0c6670 con 0x7faf5c0cb580 2024-02-02T13:26:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.878+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c0c6c30 con 0x7faf5c0cb580 2024-02-02T13:26:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.879+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c066cf0 con 0x7faf5c0cb580 2024-02-02T13:26:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.880+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf4c0113f0 con 0x7faf5c0cb580 2024-02-02T13:26:48.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.881+0000 7faf53400640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:48.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.881+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf4c088b10 con 0x7faf5c0cb580 2024-02-02T13:26:48.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.881+0000 7faf5a000640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:48.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.881+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=53}) v3 -- 0x7faf34071f60 con 0x7faf5c0cb580 2024-02-02T13:26:48.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.882+0000 7faf5a000640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf5c158900 tx=0x7faf44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:48.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:48.886+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf4c01f060 con 0x7faf5c0cb580 2024-02-02T13:26:49.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.083+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faf5c1e3a40 con 0x7faf5c0cb580 2024-02-02T13:26:49.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.086+0000 7faf53400640 1 -- 172.21.15.91:0/1891949281 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+33623 (secure 0 0 0) 0x7faf4c01f020 con 0x7faf5c0cb580 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 52, 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:47.358839+0000", 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:26:49.089 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:26:49.090 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:49.091 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:49.092 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:49.093 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:49.094 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:49.095 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:49.096 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:49.097 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:49.098 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:49.099 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:49.100 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:49.101 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:26:49.102 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:26:49.103 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:49.104 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:49.105 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:49.106 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:49.107 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.108 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:26:49.109 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.110 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.111 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:26:49.112 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.114 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.115 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.117 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.118 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.119 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.120 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.121 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:26:49.122 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:49.123 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.124 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.125 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.126 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.127 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:26:49.128 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:26:49.130 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.132 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:26:49.133 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.134 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.135 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.136 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.137 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:26:49.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:49.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:49.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:26:49.149 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:26:49.150 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.151 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:26:49.152 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.153 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.154 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:26:49.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:26:49.156 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:49.157 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 msgr2=0x7faf3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf5c158900 tx=0x7faf44008040 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 msgr2=0x7faf5c0c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c7f0 tx=0x7faf4c00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 shutdown_connections 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c15ad10 0x7faf5c0c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c15b0f0 0x7faf5c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 --2- 172.21.15.91:0/1891949281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0cb580 0x7faf5c0c7b20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.089+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 >> 172.21.15.91:0/1891949281 conn(0x7faf5c0b0700 msgr2=0x7faf5c159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.090+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 shutdown_connections 2024-02-02T13:26:49.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.090+0000 7faf61e00640 1 -- 172.21.15.91:0/1891949281 wait complete. 2024-02-02T13:26:49.160 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:26:49.283 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:26:49.282+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-02-02T13:26:49.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.372+0000 7f77eda00640 1 Processor -- start 2024-02-02T13:26:49.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.372+0000 7f77eda00640 1 -- start start 2024-02-02T13:26:49.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77eda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77e8151f90 0x7f77e815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77eda00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 0x7f77e80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77eda00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e8070f80 con 0x7f77e806f4f0 2024-02-02T13:26:49.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77eda00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e8152370 con 0x7f77e8151f90 2024-02-02T13:26:49.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77eda00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80cb010 con 0x7f77e80b0e10 2024-02-02T13:26:49.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77e6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77e8151f90 0x7f77e815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.374+0000 7f77e7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 0x7f77e80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77dfe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f77e806f4f0 0x7f77e80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:47372/0 (socket says 172.21.15.91:47372) 2024-02-02T13:26:49.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77dfe00640 1 -- 172.21.15.91:0/588832066 learned_addr learned my addr 172.21.15.91:0/588832066 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:49.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77e6000640 1 -- 172.21.15.91:0/588832066 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433364442 0 0) 0x7f77e8070f80 con 0x7f77e806f4f0 2024-02-02T13:26:49.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77e7400640 1 -- 172.21.15.91:0/588832066 >> v2:172.21.15.113:3300/0 conn(0x7f77e8151f90 msgr2=0x7f77e815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:49.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77e7400640 1 --2- 172.21.15.91:0/588832066 >> v2:172.21.15.113:3300/0 conn(0x7f77e8151f90 0x7f77e815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77e7400640 1 -- 172.21.15.91:0/588832066 >> v1:172.21.15.113:6789/0 conn(0x7f77e806f4f0 legacy=0x7f77e80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.375+0000 7f77e7400640 1 -- 172.21.15.91:0/588832066 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80cb2f0 con 0x7f77e80b0e10 2024-02-02T13:26:49.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.377+0000 7f77e7400640 1 --2- 172.21.15.91:0/588832066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 0x7f77e80b3af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c3019594305ae0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:49.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.377+0000 7f77e6000640 1 -- 172.21.15.91:0/588832066 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e002cc60 con 0x7f77e80b0e10 2024-02-02T13:26:49.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.377+0000 7f77e6000640 1 -- 172.21.15.91:0/588832066 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e002cdc0 con 0x7f77e80b0e10 2024-02-02T13:26:49.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.377+0000 7f77e6000640 1 -- 172.21.15.91:0/588832066 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0034660 con 0x7f77e80b0e10 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 -- 172.21.15.91:0/588832066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 msgr2=0x7f77e80b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 --2- 172.21.15.91:0/588832066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 0x7f77e80b3af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e002beb0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 -- 172.21.15.91:0/588832066 shutdown_connections 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 --2- 172.21.15.91:0/588832066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e80b0e10 0x7f77e80b3af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 --2- 172.21.15.91:0/588832066 >> v2:172.21.15.113:3300/0 conn(0x7f77e8151f90 0x7f77e815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 -- 172.21.15.91:0/588832066 >> 172.21.15.91:0/588832066 conn(0x7f77e80cec80 msgr2=0x7f77e80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 -- 172.21.15.91:0/588832066 shutdown_connections 2024-02-02T13:26:49.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.378+0000 7f77eda00640 1 -- 172.21.15.91:0/588832066 wait complete. 2024-02-02T13:26:49.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.380+0000 7f77eda00640 1 Processor -- start 2024-02-02T13:26:49.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.380+0000 7f77eda00640 1 -- start start 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e806f4f0 0x7f77e80c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e80b0e10 0x7f77e80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80b2c20 con 0x7f77e8151f90 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c36a0 con 0x7f77e80b0e10 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77eda00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80b2660 con 0x7f77e806f4f0 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77e7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77e7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60082/0 (socket says 172.21.15.91:60082) 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77e7400640 1 -- 172.21.15.91:0/2465846323 learned_addr learned my addr 172.21.15.91:0/2465846323 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.381+0000 7f77dfe00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e80b0e10 0x7f77e80b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e6a00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e806f4f0 0x7f77e80c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 -- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e806f4f0 msgr2=0x7f77e80c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e806f4f0 0x7f77e80c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 -- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e80b0e10 msgr2=0x7f77e80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e80b0e10 0x7f77e80b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009a70 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77e7400640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0040650 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e00407b0 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.382+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0040b60 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.383+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e80b2940 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.383+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e80c7190 con 0x7f77e8151f90 2024-02-02T13:26:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.384+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e80c6d10 con 0x7f77e8151f90 2024-02-02T13:26:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.387+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77e0040cc0 con 0x7f77e8151f90 2024-02-02T13:26:49.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.388+0000 7f77dea00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 0x7f77c006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:49.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.389+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e00ab870 con 0x7f77e8151f90 2024-02-02T13:26:49.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.389+0000 7f77e6a00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 0x7f77c006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:49.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.389+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=53}) v3 -- 0x7f77c0071f90 con 0x7f77e8151f90 2024-02-02T13:26:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.389+0000 7f77e6a00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 0x7f77c006c920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f77e80c2f50 tx=0x7f77d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.392+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77e0076e40 con 0x7f77e8151f90 2024-02-02T13:26:49.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.620+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e80c6d10 con 0x7f77e8151f90 2024-02-02T13:26:49.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.621+0000 7f77dea00640 1 -- 172.21.15.91:0/2465846323 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f77e80c6d10 con 0x7f77e8151f90 2024-02-02T13:26:49.622 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:49.622 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:26:49.623 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 msgr2=0x7f77c006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 0x7f77c006c920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f77e80c2f50 tx=0x7f77d8008040 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 msgr2=0x7f77e80b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e0004a40 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 shutdown_connections 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77c006a460 0x7f77c006c920 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e8151f90 0x7f77e80b37e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e80b0e10 0x7f77e80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 --2- 172.21.15.91:0/2465846323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e806f4f0 0x7f77e80c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 >> 172.21.15.91:0/2465846323 conn(0x7f77e80cec80 msgr2=0x7f77e815a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 shutdown_connections 2024-02-02T13:26:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:49.623+0000 7f77eda00640 1 -- 172.21.15.91:0/2465846323 wait complete. 2024-02-02T13:26:49.641 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:26:49.859 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5508,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.398001132000001,"mdsmap_epoch":24,"osdmap_epoch":52,"osdmap_epoch_barrier":52,"uptime":28.081676113} 2024-02-02T13:26:49.860 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5508, "mdsmap_epoch": 24, "osdmap_epoch": 52, "osdmap_epoch_barrier": 52, "rank_uptime": 22.398001132, "state": "up:active", "uptime": 28.081676113, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:26:49.860 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:26:49.860 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:26:49.860 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:26:49.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:49.910 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-02-02T13:26:50.008 INFO:teuthology.orchestra.run.smithi091.stderr:dd: failed to open 'small_file_1': Permission denied 2024-02-02T13:26:50.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:26:50.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:50.009 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-02-02T13:26:50.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:26:50.066 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-02-02T13:26:50.192 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:26:50.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.394+0000 7feb3fa00640 1 Processor -- start 2024-02-02T13:26:50.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.395+0000 7feb3fa00640 1 -- start start 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3fa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb380cbc90 0x7feb3806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3fa00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3806d910 con 0x7feb3815b0f0 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3fa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380cc850 con 0x7feb3815ad10 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3fa00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3806d730 con 0x7feb380cbc90 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3da00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb380cbc90 0x7feb3806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3da00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb380cbc90 0x7feb3806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37934/0 (socket says 172.21.15.91:37934) 2024-02-02T13:26:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.396+0000 7feb3da00640 1 -- 172.21.15.91:0/730285025 learned_addr learned my addr 172.21.15.91:0/730285025 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:50.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.397+0000 7feb3da00640 1 -- 172.21.15.91:0/730285025 >> v1:172.21.15.113:6789/0 conn(0x7feb3815ad10 legacy=0x7feb3806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:50.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.397+0000 7feb3da00640 1 -- 172.21.15.91:0/730285025 >> v1:172.21.15.91:6789/0 conn(0x7feb3815b0f0 legacy=0x7feb380cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.397+0000 7feb3da00640 1 -- 172.21.15.91:0/730285025 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380ccb30 con 0x7feb380cbc90 2024-02-02T13:26:50.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.398+0000 7feb3da00640 1 --2- 172.21.15.91:0/730285025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb3806b7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb30009e80 tx=0x7feb3002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b8811671648e68 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:50.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.398+0000 7feb37e00640 1 -- 172.21.15.91:0/730285025 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3002ca60 con 0x7feb380cbc90 2024-02-02T13:26:50.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.398+0000 7feb37e00640 1 -- 172.21.15.91:0/730285025 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3002cbc0 con 0x7feb380cbc90 2024-02-02T13:26:50.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.398+0000 7feb37e00640 1 -- 172.21.15.91:0/730285025 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300343f0 con 0x7feb380cbc90 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 -- 172.21.15.91:0/730285025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 msgr2=0x7feb3806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 --2- 172.21.15.91:0/730285025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb3806b7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb30009e80 tx=0x7feb3002bec0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 -- 172.21.15.91:0/730285025 shutdown_connections 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 --2- 172.21.15.91:0/730285025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb3806b7a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 -- 172.21.15.91:0/730285025 >> 172.21.15.91:0/730285025 conn(0x7feb380b0700 msgr2=0x7feb380b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 -- 172.21.15.91:0/730285025 shutdown_connections 2024-02-02T13:26:50.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.399+0000 7feb3fa00640 1 -- 172.21.15.91:0/730285025 wait complete. 2024-02-02T13:26:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.401+0000 7feb3fa00640 1 Processor -- start 2024-02-02T13:26:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.401+0000 7feb3fa00640 1 -- start start 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb3815ad10 0x7feb380c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3815b0f0 0x7feb380c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381e7530 con 0x7feb380cbc90 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381595f0 con 0x7feb3815b0f0 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb380cb030 con 0x7feb3815ad10 2024-02-02T13:26:50.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37946/0 (socket says 172.21.15.91:37946) 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3d000640 1 -- 172.21.15.91:0/3932700309 learned_addr learned my addr 172.21.15.91:0/3932700309 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3d000640 1 -- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb3815ad10 msgr2=0x7feb380c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.402+0000 7feb3d000640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb3815ad10 0x7feb380c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3d000640 1 -- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3815b0f0 msgr2=0x7feb380c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3da00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3815b0f0 0x7feb380c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3d000640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3815b0f0 0x7feb380c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3d000640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30009b30 con 0x7feb380cbc90 2024-02-02T13:26:50.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3d000640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c430 tx=0x7feb2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28011790 con 0x7feb380cbc90 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280118f0 con 0x7feb380cbc90 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28011ce0 con 0x7feb380cbc90 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb381f2a80 con 0x7feb380cbc90 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.403+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb381f3040 con 0x7feb380cbc90 2024-02-02T13:26:50.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.404+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38066cf0 con 0x7feb380cbc90 2024-02-02T13:26:50.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.405+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feb2801c460 con 0x7feb380cbc90 2024-02-02T13:26:50.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.406+0000 7feb35600640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 0x7feb1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:50.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.406+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb28088930 con 0x7feb380cbc90 2024-02-02T13:26:50.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.406+0000 7feb2fc00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 0x7feb1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:50.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.406+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=53}) v3 -- 0x7feb10071f60 con 0x7feb380cbc90 2024-02-02T13:26:50.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.407+0000 7feb2fc00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 0x7feb1006c8f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb20000f60 tx=0x7feb200079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:50.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.411+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feb28053e80 con 0x7feb380cbc90 2024-02-02T13:26:50.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.601+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7feb3815e620 con 0x7feb380cbc90 2024-02-02T13:26:50.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.603+0000 7feb35600640 1 -- 172.21.15.91:0/3932700309 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+33623 (secure 0 0 0) 0x7feb280574a0 con 0x7feb380cbc90 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 52, 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:47.358839+0000", 2024-02-02T13:26:50.606 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:26:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:50.608 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:26:50.609 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:50.610 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:50.611 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:50.612 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:26:50.613 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.614 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:50.615 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:50.616 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:50.617 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:50.618 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:50.619 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:26:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:26:50.622 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:50.623 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:50.624 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:50.625 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:26:50.626 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:50.627 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.628 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.629 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.630 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.631 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:26:50.632 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.633 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:26:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:26:50.635 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:26:50.636 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:26:50.637 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.639 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.640 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.641 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.642 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:50.643 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.644 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.645 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.646 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:26:50.647 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:26:50.648 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.649 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.650 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:26:50.651 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.652 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.653 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.654 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.655 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:26:50.656 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.657 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.659 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:26:50.660 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.661 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.662 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.664 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:50.666 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:26:50.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:26:50.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:26:50.669 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:50.670 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:50.671 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:50.672 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.673 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.674 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.675 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:26:50.676 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:50.677 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:26:50.678 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:26:50.679 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:26:50.680 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:26:50.680 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:50.680 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 msgr2=0x7feb1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 0x7feb1006c8f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb20000f60 tx=0x7feb200079b0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 msgr2=0x7feb381590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c430 tx=0x7feb2800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 shutdown_connections 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb1006a430 0x7feb1006c8f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb3815ad10 0x7feb380c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3815b0f0 0x7feb380c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 --2- 172.21.15.91:0/3932700309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb380cbc90 0x7feb381590b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.606+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 >> 172.21.15.91:0/3932700309 conn(0x7feb380b0700 msgr2=0x7feb3815df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.607+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 shutdown_connections 2024-02-02T13:26:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:50.607+0000 7feb3fa00640 1 -- 172.21.15.91:0/3932700309 wait complete. 2024-02-02T13:26:50.683 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:26:54.285 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:26:54.283+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-02-02T13:26:55.684 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:26:55.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.903+0000 7f41dc400640 1 Processor -- start 2024-02-02T13:26:55.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.903+0000 7f41dc400640 1 -- start start 2024-02-02T13:26:55.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41dc400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f41d40cbc90 0x7f41d406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:55.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41dc400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d406d910 con 0x7f41d415ad10 2024-02-02T13:26:55.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41dc400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40cc850 con 0x7f41d415b0f0 2024-02-02T13:26:55.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41dc400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d406d730 con 0x7f41d40cbc90 2024-02-02T13:26:55.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41d9a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f41d415ad10 0x7f41d406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34480/0 (socket says 172.21.15.91:34480) 2024-02-02T13:26:55.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41d9a00640 1 -- 172.21.15.91:0/416019325 learned_addr learned my addr 172.21.15.91:0/416019325 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:26:55.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.905+0000 7f41da400640 1 --2- 172.21.15.91:0/416019325 >> v2:172.21.15.91:3300/0 conn(0x7f41d40cbc90 0x7f41d406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:55.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.906+0000 7f41da400640 1 -- 172.21.15.91:0/416019325 >> v1:172.21.15.113:6789/0 conn(0x7f41d415b0f0 legacy=0x7f41d40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:26:55.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.906+0000 7f41da400640 1 -- 172.21.15.91:0/416019325 >> v1:172.21.15.91:6789/0 conn(0x7f41d415ad10 legacy=0x7f41d406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:55.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.906+0000 7f41da400640 1 -- 172.21.15.91:0/416019325 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40ccb30 con 0x7f41d40cbc90 2024-02-02T13:26:55.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.906+0000 7f41d3e00640 1 -- 172.21.15.91:0/416019325 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938963237 0 0) 0x7f41d406d910 con 0x7f41d415ad10 2024-02-02T13:26:55.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.907+0000 7f41da400640 1 --2- 172.21.15.91:0/416019325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d406b7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097d0 tx=0x7f41cc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22f223a4026ea70 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:55.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.907+0000 7f41d3e00640 1 -- 172.21.15.91:0/416019325 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc031780 con 0x7f41d40cbc90 2024-02-02T13:26:55.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.907+0000 7f41d3e00640 1 -- 172.21.15.91:0/416019325 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc0318e0 con 0x7f41d40cbc90 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.907+0000 7f41d3e00640 1 -- 172.21.15.91:0/416019325 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc031c70 con 0x7f41d40cbc90 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.907+0000 7f41dc400640 1 -- 172.21.15.91:0/416019325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 msgr2=0x7f41d406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 --2- 172.21.15.91:0/416019325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d406b7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097d0 tx=0x7f41cc02b820 comp rx=0 tx=0).stop 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 -- 172.21.15.91:0/416019325 shutdown_connections 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 --2- 172.21.15.91:0/416019325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d406b7a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:55.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 -- 172.21.15.91:0/416019325 >> 172.21.15.91:0/416019325 conn(0x7f41d40b0700 msgr2=0x7f41d40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:55.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 -- 172.21.15.91:0/416019325 shutdown_connections 2024-02-02T13:26:55.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.908+0000 7f41dc400640 1 -- 172.21.15.91:0/416019325 wait complete. 2024-02-02T13:26:55.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.909+0000 7f41dc400640 1 Processor -- start 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 -- start start 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d415ad10 0x7f41d415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d415b0f0 0x7f41d40ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40a99a0 con 0x7f41d40cbc90 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41576e0 con 0x7f41d415b0f0 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.910+0000 7f41dc400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40a8ab0 con 0x7f41d415ad10 2024-02-02T13:26:55.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:55.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37952/0 (socket says 172.21.15.91:37952) 2024-02-02T13:26:55.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 -- 172.21.15.91:0/3376603025 learned_addr learned my addr 172.21.15.91:0/3376603025 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:26:55.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 -- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d415ad10 msgr2=0x7f41d415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:55.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d415ad10 0x7f41d415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:55.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41da400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d415b0f0 0x7f41d40ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:55.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.911+0000 7f41d9a00640 1 -- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d415b0f0 msgr2=0x7f41d40ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:55.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d9a00640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d415b0f0 0x7f41d40ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d9a00640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009400 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d9a00640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c4d0 tx=0x7f41c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c40074e0 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c4007640 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004a20 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d40a8d90 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.912+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d40a9350 con 0x7f41d40cbc90 2024-02-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.913+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d4066cf0 con 0x7f41d40cbc90 2024-02-02T13:26:55.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.914+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f41c4004b80 con 0x7f41d40cbc90 2024-02-02T13:26:55.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.915+0000 7f41d1600640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:26:55.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.915+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41c4087b50 con 0x7f41d40cbc90 2024-02-02T13:26:55.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.915+0000 7f41d9000640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:26:55.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.915+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=55}) v3 -- 0x7f41ac071f60 con 0x7f41d40cbc90 2024-02-02T13:26:55.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.916+0000 7f41d9000640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41d40c3210 tx=0x7f41bc009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:26:55.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:55.919+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f41c4018020 con 0x7f41d40cbc90 2024-02-02T13:26:56.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.108+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f41d40cb790 con 0x7f41d40cbc90 2024-02-02T13:26:56.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.112+0000 7f41d1600640 1 -- 172.21.15.91:0/3376603025 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+33543 (secure 0 0 0) 0x7f41c4053130 con 0x7f41d40cbc90 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 54, 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:55.426156+0000", 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:26:56.115 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:26:56.116 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:26:56.117 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:56.118 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:56.119 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:56.120 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:56.121 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.122 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:26:56.123 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:26:56.124 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:56.125 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:56.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:26:56.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:56.128 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:56.130 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:56.131 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:26:56.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:56.133 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:26:56.134 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:26:56.135 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "49", 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:26:56.136 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:26:56.137 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:26:56.138 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:26:56.139 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:26:56.140 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.141 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.144 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.145 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.147 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.150 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.151 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.152 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.154 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.155 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.159 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.160 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.161 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.164 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:26:56.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:26:56.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.167 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:26:56.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.172 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.173 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.174 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.178 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.179 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.182 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:26:56.183 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.184 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.185 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:26:56.186 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.187 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.188 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:26:56.189 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.190 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:26:56.192 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:26:56.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:26:56.194 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.195 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.196 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.197 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.198 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.199 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:26:56.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:26:56.201 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:26:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:26:56.203 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.114+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 msgr2=0x7f41ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.114+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41d40c3210 tx=0x7f41bc009230 comp rx=0 tx=0).stop 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.114+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 msgr2=0x7f41d40a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.114+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c4d0 tx=0x7f41c400c990 comp rx=0 tx=0).stop 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 shutdown_connections 2024-02-02T13:26:56.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f41ac06a430 0x7f41ac06c8f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41d415ad10 0x7f41d415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41d415b0f0 0x7f41d40ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 --2- 172.21.15.91:0/3376603025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41d40cbc90 0x7f41d40a9e30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 >> 172.21.15.91:0/3376603025 conn(0x7f41d40b0700 msgr2=0x7f41d40c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 shutdown_connections 2024-02-02T13:26:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:26:56.115+0000 7f41dc400640 1 -- 172.21.15.91:0/3376603025 wait complete. 2024-02-02T13:26:56.207 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:26:58.940 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:26:58.963 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:27:01.208 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.425+0000 7f4a58600640 1 Processor -- start 2024-02-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.425+0000 7f4a58600640 1 -- start start 2024-02-02T13:27:01.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.426+0000 7f4a58600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a5015ad10 0x7f4a5006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.426+0000 7f4a58600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a5006d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.426+0000 7f4a58600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5006d910 con 0x7f4a500cb580 2024-02-02T13:27:01.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.426+0000 7f4a58600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500cd610 con 0x7f4a5015ad10 2024-02-02T13:27:01.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.426+0000 7f4a58600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500cc850 con 0x7f4a5015b0f0 2024-02-02T13:27:01.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a56600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4a500cb580 0x7f4a5006f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45332/0 (socket says 172.21.15.91:45332) 2024-02-02T13:27:01.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a56600640 1 -- 172.21.15.91:0/1698403801 learned_addr learned my addr 172.21.15.91:0/1698403801 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a55c00640 1 --2- 172.21.15.91:0/1698403801 >> v2:172.21.15.113:3300/0 conn(0x7f4a5015ad10 0x7f4a5006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a55200640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a5006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706112597 0 0) 0x7f4a5006d910 con 0x7f4a500cb580 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.427+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a30003520 con 0x7f4a500cb580 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822889333 0 0) 0x7f4a30003520 con 0x7f4a500cb580 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 msgr2=0x7f4a5006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a5006d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 >> v2:172.21.15.113:3300/0 conn(0x7f4a5015ad10 msgr2=0x7f4a5006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 --2- 172.21.15.91:0/1698403801 >> v2:172.21.15.113:3300/0 conn(0x7f4a5015ad10 0x7f4a5006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500ccb30 con 0x7f4a500cb580 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f4a480019f0 con 0x7f4a500cb580 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 >> v1:172.21.15.91:6789/0 conn(0x7f4a500cb580 legacy=0x7f4a5006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a30003970 0x7f4a30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500cd610 con 0x7f4a30004d00 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500cc850 con 0x7f4a30003970 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000ba30 con 0x7f4a5015b0f0 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a56600640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.428+0000 7f4a55200640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a55c00640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a30003970 0x7f4a30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a56600640 1 -- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 msgr2=0x7f4a300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a56600640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a5015b0f0 0x7f4a300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a56600640 1 -- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a30003970 msgr2=0x7f4a30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a56600640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a30003970 0x7f4a30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.429+0000 7f4a56600640 1 -- 172.21.15.91:0/1698403801 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30003520 con 0x7f4a30004d00 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.430+0000 7f4a56600640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a30005180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800a790 tx=0x7f4a300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2732a8a340f0d39f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.430+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48034630 con 0x7f4a30004d00 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.430+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4802f070 con 0x7f4a30004d00 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.430+0000 7f4a4fe00640 1 -- 172.21.15.91:0/1698403801 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480349f0 con 0x7f4a30004d00 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 -- 172.21.15.91:0/1698403801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 msgr2=0x7f4a30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 --2- 172.21.15.91:0/1698403801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a30005180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800a790 tx=0x7f4a300057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a56600640 1 -- 172.21.15.91:0/1698403801 reap_dead start 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 -- 172.21.15.91:0/1698403801 shutdown_connections 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 -- 172.21.15.91:0/1698403801 >> 172.21.15.91:0/1698403801 conn(0x7f4a500b0700 msgr2=0x7f4a500b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:01.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 -- 172.21.15.91:0/1698403801 shutdown_connections 2024-02-02T13:27:01.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.431+0000 7f4a58600640 1 -- 172.21.15.91:0/1698403801 wait complete. 2024-02-02T13:27:01.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.433+0000 7f4a58600640 1 Processor -- start 2024-02-02T13:27:01.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.433+0000 7f4a58600640 1 -- start start 2024-02-02T13:27:01.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a501f2210 0x7f4a501f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a500c91c0 0x7f4a500c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500c6680 con 0x7f4a30004d00 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501f5600 con 0x7f4a500c91c0 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a58600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501f2670 con 0x7f4a501f2210 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44982/0 (socket says 172.21.15.91:44982) 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 -- 172.21.15.91:0/1537213636 learned_addr learned my addr 172.21.15.91:0/1537213636 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a56600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a500c91c0 0x7f4a500c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55200640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a501f2210 0x7f4a501f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 -- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a501f2210 msgr2=0x7f4a501f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a501f2210 0x7f4a501f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 -- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a500c91c0 msgr2=0x7f4a500c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a500c91c0 0x7f4a500c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.434+0000 7f4a55c00640 1 -- 172.21.15.91:0/1537213636 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5015d190 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a55c00640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a400116d0 tx=0x7f4a40011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000f960 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000fac0 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000fe30 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a500c5ce0 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.435+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a500c62a0 con 0x7f4a30004d00 2024-02-02T13:27:01.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.436+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a500b0550 con 0x7f4a30004d00 2024-02-02T13:27:01.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.437+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4a40015460 con 0x7f4a30004d00 2024-02-02T13:27:01.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.438+0000 7f4a4d600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 0x7f4a2806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.438+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a40088db0 con 0x7f4a30004d00 2024-02-02T13:27:01.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.438+0000 7f4a55200640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 0x7f4a2806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.439+0000 7f4a55200640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 0x7f4a2806c920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a5015df30 tx=0x7f4a38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.443+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4a4008d070 con 0x7f4a30004d00 2024-02-02T13:27:01.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.634+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4a50066cf0 con 0x7f4a30004d00 2024-02-02T13:27:01.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.636+0000 7f4a4d600640 1 -- 172.21.15.91:0/1537213636 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+33480 (secure 0 0 0) 0x7f4a40057ed0 con 0x7f4a30004d00 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 56, 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:27:01.638 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:57.446022+0000", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:27:01.639 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:27:01.640 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:01.643 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:01.644 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:01.645 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:01.646 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:01.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:01.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:01.649 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:01.650 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:01.651 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:27:01.652 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:01.653 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:27:01.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:01.655 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:01.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:01.657 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:01.658 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:27:01.659 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.660 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.661 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.662 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:27:01.663 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.664 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.665 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.666 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:27:01.669 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.671 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.672 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.673 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.674 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.675 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.676 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:27:01.678 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.679 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.680 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.681 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.683 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.684 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:27:01.685 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.686 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:27:01.687 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.688 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.689 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.691 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.692 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.693 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.694 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.695 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.696 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:01.697 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:01.698 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.699 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:01.700 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:27:01.701 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.702 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.703 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:01.707 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:27:01.708 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:01.709 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 msgr2=0x7f4a2806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 0x7f4a2806c920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a5015df30 tx=0x7f4a38008040 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 msgr2=0x7f4a5015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a400116d0 tx=0x7f4a40011b90 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 shutdown_connections 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4a2806a460 0x7f4a2806c920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a501f2210 0x7f4a501f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a500c91c0 0x7f4a500c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 --2- 172.21.15.91:0/1537213636 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a30004d00 0x7f4a5015cb10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 >> 172.21.15.91:0/1537213636 conn(0x7f4a500b0700 msgr2=0x7f4a50159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:01.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 shutdown_connections 2024-02-02T13:27:01.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.639+0000 7f4a58600640 1 -- 172.21.15.91:0/1537213636 wait complete. 2024-02-02T13:27:01.713 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-02-02T13:27:01.713 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:27:01.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.920+0000 7f0b8a200640 1 Processor -- start 2024-02-02T13:27:01.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.920+0000 7f0b8a200640 1 -- start start 2024-02-02T13:27:01.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.921+0000 7f0b8a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b8415aef0 0x7f0b840b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.921+0000 7f0b8a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 0x7f0b84070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b8a200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b840b0540 con 0x7f0b840adf20 2024-02-02T13:27:01.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b8a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8415b2d0 con 0x7f0b8415aef0 2024-02-02T13:27:01.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b8a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840ca410 con 0x7f0b8406ce50 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b89800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0b840adf20 0x7f0b8406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45336/0 (socket says 172.21.15.91:45336) 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b89800640 1 -- 172.21.15.91:0/2498145212 learned_addr learned my addr 172.21.15.91:0/2498145212 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095507409 0 0) 0x7f0b840b0540 con 0x7f0b840adf20 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> v2:172.21.15.113:3300/0 conn(0x7f0b8415aef0 0x7f0b840b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b7bc00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 0x7f0b84070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.922+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b64003520 con 0x7f0b840adf20 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568025575 0 0) 0x7f0b64003520 con 0x7f0b840adf20 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 msgr2=0x7f0b84070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 0x7f0b84070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 >> v2:172.21.15.113:3300/0 conn(0x7f0b8415aef0 msgr2=0x7f0b840b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 --2- 172.21.15.91:0/2498145212 >> v2:172.21.15.113:3300/0 conn(0x7f0b8415aef0 0x7f0b840b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840b07e0 con 0x7f0b840adf20 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0b7c0019f0 con 0x7f0b840adf20 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> v2:172.21.15.113:3300/0 conn(0x7f0b8415aef0 0x7f0b840b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 >> v1:172.21.15.91:6789/0 conn(0x7f0b840adf20 legacy=0x7f0b8406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b640047c0 0x7f0b64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 0x7f0b64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8415b2d0 con 0x7f0b64003970 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840ca410 con 0x7f0b640047c0 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6400b9d0 con 0x7f0b8406ce50 2024-02-02T13:27:01.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.923+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.924+0000 7f0b88e00640 1 -- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 msgr2=0x7f0b64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.924+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406ce50 0x7f0b64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.924+0000 7f0b88e00640 1 -- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b640047c0 msgr2=0x7f0b64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:01.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.924+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b640047c0 0x7f0b64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.924+0000 7f0b88e00640 1 -- 172.21.15.91:0/2498145212 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b64003520 con 0x7f0b64003970 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b64003d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0b740098a0 tx=0x7f0b640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ed7ee6409759ab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74004230 con 0x7f0b64003970 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74004390 con 0x7f0b64003970 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b83e00640 1 -- 172.21.15.91:0/2498145212 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74003830 con 0x7f0b64003970 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b8a200640 1 -- 172.21.15.91:0/2498145212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 msgr2=0x7f0b64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2498145212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b64003d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0b740098a0 tx=0x7f0b640057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.925+0000 7f0b89800640 1 -- 172.21.15.91:0/2498145212 reap_dead start 2024-02-02T13:27:01.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.926+0000 7f0b8a200640 1 -- 172.21.15.91:0/2498145212 shutdown_connections 2024-02-02T13:27:01.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.926+0000 7f0b8a200640 1 -- 172.21.15.91:0/2498145212 >> 172.21.15.91:0/2498145212 conn(0x7f0b840cb4a0 msgr2=0x7f0b840cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:01.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.926+0000 7f0b8a200640 1 -- 172.21.15.91:0/2498145212 shutdown_connections 2024-02-02T13:27:01.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.926+0000 7f0b8a200640 1 -- 172.21.15.91:0/2498145212 wait complete. 2024-02-02T13:27:01.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.927+0000 7f0b8a200640 1 Processor -- start 2024-02-02T13:27:01.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.928+0000 7f0b8a200640 1 -- start start 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.928+0000 7f0b8a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.928+0000 7f0b8a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840c4f10 0x7f0b841e4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.928+0000 7f0b8a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840cc860 0x7f0b84155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b8a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8415d180 con 0x7f0b64003970 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b8a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841e4c60 con 0x7f0b840c4f10 2024-02-02T13:27:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b8a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840c5390 con 0x7f0b840cc860 2024-02-02T13:27:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45002/0 (socket says 172.21.15.91:45002) 2024-02-02T13:27:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 -- 172.21.15.91:0/2987661293 learned_addr learned my addr 172.21.15.91:0/2987661293 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 -- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840cc860 msgr2=0x7f0b84155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840cc860 0x7f0b84155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 -- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840c4f10 msgr2=0x7f0b841e4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840c4f10 0x7f0b841e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 -- 172.21.15.91:0/2987661293 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8415c500 con 0x7f0b64003970 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.929+0000 7f0b88e00640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0b740099c0 tx=0x7f0b74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.930+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74042600 con 0x7f0b64003970 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.930+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74042760 con 0x7f0b64003970 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.930+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74042b10 con 0x7f0b64003970 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.930+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8415c7e0 con 0x7f0b64003970 2024-02-02T13:27:01.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.930+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8415cda0 con 0x7f0b64003970 2024-02-02T13:27:01.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.931+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8404dff0 con 0x7f0b64003970 2024-02-02T13:27:01.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.932+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0b740373f0 con 0x7f0b64003970 2024-02-02T13:27:01.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.933+0000 7f0b81600640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 0x7f0b5c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:01.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.933+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b740aa570 con 0x7f0b64003970 2024-02-02T13:27:01.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.933+0000 7f0b7bc00640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 0x7f0b5c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:01.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.933+0000 7f0b7bc00640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 0x7f0b5c06c920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006a90 tx=0x7f0b6c006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:01.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:01.937+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0b74075b50 con 0x7f0b64003970 2024-02-02T13:27:02.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.130+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0b8406d570 con 0x7f0b64003970 2024-02-02T13:27:02.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.134+0000 7f0b81600640 1 -- 172.21.15.91:0/2987661293 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+33480 (secure 0 0 0) 0x7f0b74079170 con 0x7f0b64003970 2024-02-02T13:27:02.136 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:02.136 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:27:02.136 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 56, 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:26:57.446022+0000", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:27:02.137 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 7, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:27:02.138 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:02.139 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:02.140 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:02.141 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:02.142 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:02.143 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:27:02.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 6, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:25.185334+0000", 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:02.146 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:02.147 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:02.148 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:02.149 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:02.150 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:27:02.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:27:02.152 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 7, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:26:26.457324+0000", 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:27:02.153 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:27:02.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "56", 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:27:02.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:27:02.156 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:27:02.157 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:27:02.158 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:27:02.159 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.160 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.164 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:27:02.165 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.166 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:27:02.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.169 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.171 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.172 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.174 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.175 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.176 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.179 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.180 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:27:02.181 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.181 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.181 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.182 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.185 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.186 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:27:02.189 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.190 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.191 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:27:02.192 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.194 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.195 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.196 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.198 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.199 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.200 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.201 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:27:02.201 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.201 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.201 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.202 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.203 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.204 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.205 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 46, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.206 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.207 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.208 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:27:02.210 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.211 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.212 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:27:02.213 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.214 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.215 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:27:02.216 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:27:02.217 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:02.218 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:27:02.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 msgr2=0x7f0b5c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 0x7f0b5c06c920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006a90 tx=0x7f0b6c006a20 comp rx=0 tx=0).stop 2024-02-02T13:27:02.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 msgr2=0x7f0b8415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0b740099c0 tx=0x7f0b74004730 comp rx=0 tx=0).stop 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 shutdown_connections 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0b5c06a460 0x7f0b5c06c920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840cc860 0x7f0b84155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840c4f10 0x7f0b841e4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 --2- 172.21.15.91:0/2987661293 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b64003970 0x7f0b8415be20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 >> 172.21.15.91:0/2987661293 conn(0x7f0b840cb4a0 msgr2=0x7f0b840ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 shutdown_connections 2024-02-02T13:27:02.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.137+0000 7f0b8a200640 1 -- 172.21.15.91:0/2987661293 wait complete. 2024-02-02T13:27:02.222 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.436+0000 7fca00200640 1 Processor -- start 2024-02-02T13:27:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.436+0000 7fca00200640 1 -- start start 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.437+0000 7fca00200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9f8151f90 0x7fc9f815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.437+0000 7fca00200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 0x7fc9f80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.437+0000 7fca00200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f8070f80 con 0x7fc9f806f4f0 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.437+0000 7fca00200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f8152370 con 0x7fc9f8151f90 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.437+0000 7fca00200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80cb010 con 0x7fc9f80b0e10 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9fce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc9f806f4f0 0x7fc9f80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45342/0 (socket says 172.21.15.91:45342) 2024-02-02T13:27:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9fce00640 1 -- 172.21.15.91:0/3257343715 learned_addr learned my addr 172.21.15.91:0/3257343715 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9fd800640 1 --2- 172.21.15.91:0/3257343715 >> v2:172.21.15.113:3300/0 conn(0x7fc9f8151f90 0x7fc9f815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9fe200640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 0x7fc9f80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204514931 0 0) 0x7fc9f8070f80 con 0x7fc9f806f4f0 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d8003520 con 0x7fc9f806f4f0 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982662727 0 0) 0x7fc9d8003520 con 0x7fc9f806f4f0 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 msgr2=0x7fc9f80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 0x7fc9f80b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 >> v2:172.21.15.113:3300/0 conn(0x7fc9f8151f90 msgr2=0x7fc9f815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 --2- 172.21.15.91:0/3257343715 >> v2:172.21.15.113:3300/0 conn(0x7fc9f8151f90 0x7fc9f815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80cb2f0 con 0x7fc9f806f4f0 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc9e0001670 con 0x7fc9f806f4f0 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 >> v1:172.21.15.91:6789/0 conn(0x7fc9f806f4f0 legacy=0x7fc9f80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.438+0000 7fc9f7e00640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9f7e00640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 0x7fc9d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9f7e00640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d8004d00 0x7fc9d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8152370 con 0x7fc9d8003970 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80cb010 con 0x7fc9d8004d00 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d800b980 con 0x7fc9f80b0e10 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 -- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 msgr2=0x7fc9d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80b0e10 0x7fc9d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 -- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d8004d00 msgr2=0x7fc9d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d8004d00 0x7fc9d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.439+0000 7fc9fd800640 1 -- 172.21.15.91:0/3257343715 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8003520 con 0x7fc9d8003970 2024-02-02T13:27:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.440+0000 7fc9fd800640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9d8003d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009af0 tx=0x7fc9d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f60be501376070 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.440+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e802c710 con 0x7fc9d8003970 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.440+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e802c870 con 0x7fc9d8003970 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.440+0000 7fc9f7e00640 1 -- 172.21.15.91:0/3257343715 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e802cc00 con 0x7fc9d8003970 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 -- 172.21.15.91:0/3257343715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 msgr2=0x7fc9d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 --2- 172.21.15.91:0/3257343715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9d8003d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009af0 tx=0x7fc9d80057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fc9fe200640 1 -- 172.21.15.91:0/3257343715 reap_dead start 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 -- 172.21.15.91:0/3257343715 shutdown_connections 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 -- 172.21.15.91:0/3257343715 >> 172.21.15.91:0/3257343715 conn(0x7fc9f80cec80 msgr2=0x7fc9f80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 -- 172.21.15.91:0/3257343715 shutdown_connections 2024-02-02T13:27:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.441+0000 7fca00200640 1 -- 172.21.15.91:0/3257343715 wait complete. 2024-02-02T13:27:02.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.443+0000 7fca00200640 1 Processor -- start 2024-02-02T13:27:02.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.443+0000 7fca00200640 1 -- start start 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80c7dc0 0x7fc9f80c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f80c2720 0x7fc9f80c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f815a640 con 0x7fc9d8003970 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81498a0 con 0x7fc9f80c2720 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fca00200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8159720 con 0x7fc9f80c7dc0 2024-02-02T13:27:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fc9fd800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fc9fd800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45018/0 (socket says 172.21.15.91:45018) 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.444+0000 7fc9fd800640 1 -- 172.21.15.91:0/392680997 learned_addr learned my addr 172.21.15.91:0/392680997 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 -- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80c7dc0 msgr2=0x7fc9f80c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80c7dc0 0x7fc9f80c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 -- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f80c2720 msgr2=0x7fc9f80c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f80c2720 0x7fc9f80c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 -- 172.21.15.91:0/392680997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f81599c0 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9fd800640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8037040 tx=0x7fc9e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8003810 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e802de90 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8003b70 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.445+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8159ca0 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.446+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f815a260 con 0x7fc9d8003970 2024-02-02T13:27:02.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.447+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f80b1530 con 0x7fc9d8003970 2024-02-02T13:27:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.447+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc9e8003cd0 con 0x7fc9d8003970 2024-02-02T13:27:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.448+0000 7fc9f5600640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 0x7fc9d006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.448+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9e80aa670 con 0x7fc9d8003970 2024-02-02T13:27:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.449+0000 7fc9fce00640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 0x7fc9d006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.449+0000 7fc9fce00640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 0x7fc9d006c920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc9f815b290 tx=0x7fc9e0001550 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.453+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc9e8075c50 con 0x7fc9d8003970 2024-02-02T13:27:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.678+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9f8066cf0 con 0x7fc9d8003970 2024-02-02T13:27:02.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.680+0000 7fc9f5600640 1 -- 172.21.15.91:0/392680997 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fc9e8079270 con 0x7fc9d8003970 2024-02-02T13:27:02.682 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:02.682 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:27:02.682 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:27:02.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 msgr2=0x7fc9d006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 0x7fc9d006c920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc9f815b290 tx=0x7fc9e0001550 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 msgr2=0x7fc9f815aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8037040 tx=0x7fc9e8004730 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 shutdown_connections 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc9d006a460 0x7fc9d006c920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9f80c7dc0 0x7fc9f80c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f80c2720 0x7fc9f80c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 --2- 172.21.15.91:0/392680997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9d8003970 0x7fc9f815aa60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.682+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 >> 172.21.15.91:0/392680997 conn(0x7fc9f80cec80 msgr2=0x7fc9f80c1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.683+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 shutdown_connections 2024-02-02T13:27:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:02.683+0000 7fca00200640 1 -- 172.21.15.91:0/392680997 wait complete. 2024-02-02T13:27:02.702 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:27:02.929 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5508,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.467481300000003,"mdsmap_epoch":24,"osdmap_epoch":56,"osdmap_epoch_barrier":52,"uptime":41.151165382999999} 2024-02-02T13:27:02.930 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5508, "mdsmap_epoch": 24, "osdmap_epoch": 56, "osdmap_epoch_barrier": 52, "rank_uptime": 35.4674813, "state": "up:active", "uptime": 41.151165383, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:27:02.930 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:27:02.930 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:02.930 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:02.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:02.981 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-02-02T13:27:03.415 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:27:03.415 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:27:03.415 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.383194 s, 137 MB/s 2024-02-02T13:27:03.416 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:27:03.417 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:27:03.509 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:27:03.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:27:03.623 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:03.623 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:03.679 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:03.679 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:03.733 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:03.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.945+0000 7f24dd000640 1 Processor -- start 2024-02-02T13:27:03.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.945+0000 7f24dd000640 1 -- start start 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.946+0000 7f24dd000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f24d815f5a0 0x7f24d80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24dd000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d80b0c40 0x7f24d80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24dd000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d8070f80 con 0x7f24d8151f90 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24dd000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d815fa30 con 0x7f24d815f5a0 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24dd000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80b10b0 con 0x7f24d80b0c40 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f24d815f5a0 0x7f24d80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f24d815f5a0 0x7f24d80b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45024/0 (socket says 172.21.15.91:45024) 2024-02-02T13:27:03.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 -- 172.21.15.91:0/2853500533 learned_addr learned my addr 172.21.15.91:0/2853500533 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 -- 172.21.15.91:0/2853500533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d80b0c40 msgr2=0x7f24d80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 --2- 172.21.15.91:0/2853500533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d80b0c40 0x7f24d80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 -- 172.21.15.91:0/2853500533 >> v1:172.21.15.113:6789/0 conn(0x7f24d8151f90 legacy=0x7f24d806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.947+0000 7f24cfe00640 1 -- 172.21.15.91:0/2853500533 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8152370 con 0x7f24d815f5a0 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24cfe00640 1 --2- 172.21.15.91:0/2853500533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d815f5a0 0x7f24d80b0700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009870 tx=0x7f24c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18db1fc950d0fed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24d5400640 1 -- 172.21.15.91:0/2853500533 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0031640 con 0x7f24d815f5a0 2024-02-02T13:27:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24d5400640 1 -- 172.21.15.91:0/2853500533 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c00317a0 con 0x7f24d815f5a0 2024-02-02T13:27:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24d5400640 1 -- 172.21.15.91:0/2853500533 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0031b30 con 0x7f24d815f5a0 2024-02-02T13:27:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24dd000640 1 -- 172.21.15.91:0/2853500533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d815f5a0 msgr2=0x7f24d80b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.949+0000 7f24dd000640 1 --2- 172.21.15.91:0/2853500533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d815f5a0 0x7f24d80b0700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009870 tx=0x7f24c002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 -- 172.21.15.91:0/2853500533 shutdown_connections 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 --2- 172.21.15.91:0/2853500533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d80b0c40 0x7f24d80b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 --2- 172.21.15.91:0/2853500533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d815f5a0 0x7f24d80b0700 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 -- 172.21.15.91:0/2853500533 >> 172.21.15.91:0/2853500533 conn(0x7f24d80cec80 msgr2=0x7f24d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 -- 172.21.15.91:0/2853500533 shutdown_connections 2024-02-02T13:27:03.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.950+0000 7f24dd000640 1 -- 172.21.15.91:0/2853500533 wait complete. 2024-02-02T13:27:03.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.951+0000 7f24dd000640 1 Processor -- start 2024-02-02T13:27:03.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.952+0000 7f24dd000640 1 -- start start 2024-02-02T13:27:03.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.952+0000 7f24dd000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.952+0000 7f24dd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d8151f90 0x7f24d814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24dd000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d815f5a0 0x7f24d80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24dd000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80ad5a0 con 0x7f24d80b0c40 2024-02-02T13:27:03.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24dd000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80ad880 con 0x7f24d8151f90 2024-02-02T13:27:03.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24dd000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80c49d0 con 0x7f24d815f5a0 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45026/0 (socket says 172.21.15.91:45026) 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 -- 172.21.15.91:0/410581161 learned_addr learned my addr 172.21.15.91:0/410581161 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 -- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d815f5a0 msgr2=0x7f24d80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d815f5a0 0x7f24d80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 -- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d8151f90 msgr2=0x7f24d814b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d8151f90 0x7f24d814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:03.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.953+0000 7f24d5e00640 1 -- 172.21.15.91:0/410581161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00094d0 con 0x7f24d80b0c40 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24d5e00640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24c800b3f0 tx=0x7f24c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011830 con 0x7f24d80b0c40 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8011990 con 0x7f24d80b0c40 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d80c4cb0 con 0x7f24d80b0c40 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011d40 con 0x7f24d80b0c40 2024-02-02T13:27:03.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.954+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d80c2420 con 0x7f24d80b0c40 2024-02-02T13:27:03.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.955+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d80c51d0 con 0x7f24d80b0c40 2024-02-02T13:27:03.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.956+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f24c801a400 con 0x7f24d80b0c40 2024-02-02T13:27:03.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.957+0000 7f24ce000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 0x7f24b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:03.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.957+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24c80886d0 con 0x7f24d80b0c40 2024-02-02T13:27:03.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.957+0000 7f24cfe00640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 0x7f24b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:03.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.958+0000 7f24cfe00640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 0x7f24b006c8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24c0036000 tx=0x7f24c0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:03.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:03.961+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f24d80c51d0 con 0x7f24d80b0c40 2024-02-02T13:27:04.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.150+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24d80c51d0 con 0x7f24d80b0c40 2024-02-02T13:27:04.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.153+0000 7f24ce000640 1 -- 172.21.15.91:0/410581161 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f24c8053d50 con 0x7f24d80b0c40 2024-02-02T13:27:04.155 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:04.155 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:57.446022+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 msgr2=0x7f24b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 0x7f24b006c8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24c0036000 tx=0x7f24c0030000 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 msgr2=0x7f24d814aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24c800b3f0 tx=0x7f24c800b8b0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 shutdown_connections 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24b006a430 0x7f24b006c8f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24d815f5a0 0x7f24d80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24d8151f90 0x7f24d814b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 --2- 172.21.15.91:0/410581161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24d80b0c40 0x7f24d814aa60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 >> 172.21.15.91:0/410581161 conn(0x7f24d80cec80 msgr2=0x7f24d80ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 shutdown_connections 2024-02-02T13:27:04.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.156+0000 7f24dd000640 1 -- 172.21.15.91:0/410581161 wait complete. 2024-02-02T13:27:04.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:26:25.185334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:26:26.457324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:27:04.178 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:27:04.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.384+0000 7f3b6bc00640 1 Processor -- start 2024-02-02T13:27:04.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.385+0000 7f3b6bc00640 1 -- start start 2024-02-02T13:27:04.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.386+0000 7f3b6bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b640c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.386+0000 7f3b6bc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b640b0700 0x7f3b640b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.386+0000 7f3b6bc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640cc7c0 con 0x7f3b640ce130 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.386+0000 7f3b6bc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b640b10d0 con 0x7f3b640b0700 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.386+0000 7f3b6bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640ce930 con 0x7f3b6415a010 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b5bc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3b640ce130 0x7f3b640ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45344/0 (socket says 172.21.15.91:45344) 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b5bc00640 1 -- 172.21.15.91:0/775978803 learned_addr learned my addr 172.21.15.91:0/775978803 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b69200640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b640c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b69c00640 1 --2- 172.21.15.91:0/775978803 >> v2:172.21.15.113:3300/0 conn(0x7f3b640b0700 0x7f3b640b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101278749 0 0) 0x7f3b640cc7c0 con 0x7f3b640ce130 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.387+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b44003520 con 0x7f3b640ce130 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606014372 0 0) 0x7f3b44003520 con 0x7f3b640ce130 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 msgr2=0x7f3b640c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b640c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 >> v2:172.21.15.113:3300/0 conn(0x7f3b640b0700 msgr2=0x7f3b640b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 --2- 172.21.15.91:0/775978803 >> v2:172.21.15.113:3300/0 conn(0x7f3b640b0700 0x7f3b640b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640cd640 con 0x7f3b640ce130 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3b4c001670 con 0x7f3b640ce130 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 >> v1:172.21.15.91:6789/0 conn(0x7f3b640ce130 legacy=0x7f3b640ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b44003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b44003ef0 0x7f3b440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640b10d0 con 0x7f3b44004d00 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640ce930 con 0x7f3b44003ef0 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4400b980 con 0x7f3b6415a010 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b69c00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b69200640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b44003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b69c00640 1 -- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 msgr2=0x7f3b44003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.388+0000 7f3b69c00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b44003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.389+0000 7f3b5bc00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b44003ef0 0x7f3b440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.389+0000 7f3b69c00640 1 -- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b44003ef0 msgr2=0x7f3b440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.389+0000 7f3b69c00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b44003ef0 0x7f3b440047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.389+0000 7f3b69c00640 1 -- 172.21.15.91:0/775978803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44003520 con 0x7f3b44004d00 2024-02-02T13:27:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.389+0000 7f3b69200640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6415a010 0x7f3b44003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:04.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b69c00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b44005180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009fc0 tx=0x7f3b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42da07b9e4fb734a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c02cc40 con 0x7f3b44004d00 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00f040 con 0x7f3b44004d00 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b63e00640 1 -- 172.21.15.91:0/775978803 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c034440 con 0x7f3b44004d00 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b6bc00640 1 -- 172.21.15.91:0/775978803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 msgr2=0x7f3b44005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/775978803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b44005180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009fc0 tx=0x7f3b440057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.390+0000 7f3b69c00640 1 -- 172.21.15.91:0/775978803 reap_dead start 2024-02-02T13:27:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.391+0000 7f3b6bc00640 1 -- 172.21.15.91:0/775978803 shutdown_connections 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.391+0000 7f3b6bc00640 1 -- 172.21.15.91:0/775978803 >> 172.21.15.91:0/775978803 conn(0x7f3b640cec80 msgr2=0x7f3b640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.391+0000 7f3b6bc00640 1 -- 172.21.15.91:0/775978803 shutdown_connections 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.391+0000 7f3b6bc00640 1 -- 172.21.15.91:0/775978803 wait complete. 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.392+0000 7f3b6bc00640 1 Processor -- start 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.392+0000 7f3b6bc00640 1 -- start start 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b641508f0 0x7f3b64153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6414f500 0x7f3b640c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6414f0e0 con 0x7f3b44004d00 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64151390 con 0x7f3b641508f0 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.393+0000 7f3b6bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6414ea60 con 0x7f3b6414f500 2024-02-02T13:27:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45038/0 (socket says 172.21.15.91:45038) 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 -- 172.21.15.91:0/1935479768 learned_addr learned my addr 172.21.15.91:0/1935479768 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 -- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6414f500 msgr2=0x7f3b640c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6414f500 0x7f3b640c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 -- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b641508f0 msgr2=0x7f3b64153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b641508f0 0x7f3b64153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 -- 172.21.15.91:0/1935479768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b641510b0 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b69200640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3b540108b0 tx=0x7f3b54010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.394+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5400eae0 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.395+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5400ec40 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.395+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401c400 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.395+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6414ed00 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.395+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b64153ba0 con 0x7f3b44004d00 2024-02-02T13:27:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.396+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6414e8c0 con 0x7f3b44004d00 2024-02-02T13:27:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.397+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3b5401c560 con 0x7f3b44004d00 2024-02-02T13:27:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.398+0000 7f3b61600640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 0x7f3b3c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.398+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b54088e90 con 0x7f3b44004d00 2024-02-02T13:27:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.398+0000 7f3b5bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 0x7f3b3c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.399+0000 7f3b5bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 0x7f3b3c06c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c001eb0 tx=0x7f3b4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.403+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3b6414e8c0 con 0x7f3b44004d00 2024-02-02T13:27:04.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.590+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b6414e8c0 con 0x7f3b44004d00 2024-02-02T13:27:04.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.591+0000 7f3b61600640 1 -- 172.21.15.91:0/1935479768 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b6414e8c0 con 0x7f3b44004d00 2024-02-02T13:27:04.592 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:27:04.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.593+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 msgr2=0x7f3b3c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.593+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 0x7f3b3c06c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c001eb0 tx=0x7f3b4c009040 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.593+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 msgr2=0x7f3b64152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.593+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3b540108b0 tx=0x7f3b54010d70 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 shutdown_connections 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3b3c06a430 0x7f3b3c06c8f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b6414f500 0x7f3b640c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b641508f0 0x7f3b64153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 --2- 172.21.15.91:0/1935479768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b44004d00 0x7f3b64152750 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 >> 172.21.15.91:0/1935479768 conn(0x7f3b640cec80 msgr2=0x7f3b6415a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 shutdown_connections 2024-02-02T13:27:04.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.594+0000 7f3b6bc00640 1 -- 172.21.15.91:0/1935479768 wait complete. 2024-02-02T13:27:04.615 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:27:04.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.826+0000 7f8d9be00640 1 Processor -- start 2024-02-02T13:27:04.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.826+0000 7f8d9be00640 1 -- start start 2024-02-02T13:27:04.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d9c0addd0 0x7f8d9c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9be00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0ae1b0 con 0x7f8d9c0cd0e0 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9be00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c06b450 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c161b30 con 0x7f8d9c0addd0 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d93e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d93e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45042/0 (socket says 172.21.15.91:45042) 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d93e00640 1 -- 172.21.15.91:0/893668394 learned_addr learned my addr 172.21.15.91:0/893668394 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9b400640 1 --1- 172.21.15.91:0/893668394 >> v1:172.21.15.91:6789/0 conn(0x7f8d9c0cd0e0 0x7f8d9c1651b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45356/0 (socket says 172.21.15.91:45356) 2024-02-02T13:27:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.828+0000 7f8d9aa00640 1 --2- 172.21.15.91:0/893668394 >> v2:172.21.15.113:3300/0 conn(0x7f8d9c0addd0 0x7f8d9c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.829+0000 7f8d93e00640 1 -- 172.21.15.91:0/893668394 >> v2:172.21.15.113:3300/0 conn(0x7f8d9c0addd0 msgr2=0x7f8d9c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.829+0000 7f8d93e00640 1 --2- 172.21.15.91:0/893668394 >> v2:172.21.15.113:3300/0 conn(0x7f8d9c0addd0 0x7f8d9c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.829+0000 7f8d93e00640 1 -- 172.21.15.91:0/893668394 >> v1:172.21.15.91:6789/0 conn(0x7f8d9c0cd0e0 legacy=0x7f8d9c1651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.829+0000 7f8d93e00640 1 -- 172.21.15.91:0/893668394 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0c9430 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d93e00640 1 --2- 172.21.15.91:0/893668394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cb470 tx=0x7f8d8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda1b1150e5e006c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d9a000640 1 -- 172.21.15.91:0/893668394 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84030860 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d9a000640 1 -- 172.21.15.91:0/893668394 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d840309c0 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d9a000640 1 -- 172.21.15.91:0/893668394 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84030d50 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d9be00640 1 -- 172.21.15.91:0/893668394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 msgr2=0x7f8d9c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.830+0000 7f8d9be00640 1 --2- 172.21.15.91:0/893668394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cb470 tx=0x7f8d8402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 -- 172.21.15.91:0/893668394 shutdown_connections 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 --2- 172.21.15.91:0/893668394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0ccba0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 --2- 172.21.15.91:0/893668394 >> v2:172.21.15.113:3300/0 conn(0x7f8d9c0addd0 0x7f8d9c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 -- 172.21.15.91:0/893668394 >> 172.21.15.91:0/893668394 conn(0x7f8d9c0cea90 msgr2=0x7f8d9c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 -- 172.21.15.91:0/893668394 shutdown_connections 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.831+0000 7f8d9be00640 1 -- 172.21.15.91:0/893668394 wait complete. 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.832+0000 7f8d9be00640 1 Processor -- start 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.833+0000 7f8d9be00640 1 -- start start 2024-02-02T13:27:04.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.833+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c0addd0 0x7f8d9c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.833+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.833+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d9c0cd0e0 0x7f8d9c1f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d9be00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0c7e00 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d9be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1fa970 con 0x7f8d9c0addd0 2024-02-02T13:27:04.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d9be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1fac50 con 0x7f8d9c0cd0e0 2024-02-02T13:27:04.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45050/0 (socket says 172.21.15.91:45050) 2024-02-02T13:27:04.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 -- 172.21.15.91:0/2684904244 learned_addr learned my addr 172.21.15.91:0/2684904244 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:04.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d9aa00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c0addd0 0x7f8d9c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d9b400640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d9c0cd0e0 0x7f8d9c1f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 -- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d9c0cd0e0 msgr2=0x7f8d9c1f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d9c0cd0e0 0x7f8d9c1f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 -- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c0addd0 msgr2=0x7f8d9c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c0addd0 0x7f8d9c1637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.834+0000 7f8d93e00640 1 -- 172.21.15.91:0/2684904244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840094d0 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d93e00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8d8402b9e0 tx=0x7f8d84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84039d60 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d84043400 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d840437b0 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c0c7740 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.835+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c15a0b0 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.836+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c0c7f40 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.837+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8d84043910 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.838+0000 7f8d92a00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 0x7f8d7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:04.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.839+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d8403e070 con 0x7f8d9c0c1c70 2024-02-02T13:27:04.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.839+0000 7f8d9aa00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 0x7f8d7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:04.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.839+0000 7f8d9aa00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 0x7f8d7406c8f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:04.843+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8d9c0c7f40 con 0x7f8d9c0c1c70 2024-02-02T13:27:05.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.024+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d9c0c7f40 con 0x7f8d9c0c1c70 2024-02-02T13:27:05.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.025+0000 7f8d92a00640 1 -- 172.21.15.91:0/2684904244 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d9c0c7f40 con 0x7f8d9c0c1c70 2024-02-02T13:27:05.028 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:27:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 msgr2=0x7f8d7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 0x7f8d7406c8f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c008040 comp rx=0 tx=0).stop 2024-02-02T13:27:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 msgr2=0x7f8d9c0c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8d8402b9e0 tx=0x7f8d84004730 comp rx=0 tx=0).stop 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 shutdown_connections 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d7406a430 0x7f8d7406c8f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d9c0cd0e0 0x7f8d9c1f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d9c0c1c70 0x7f8d9c0c8220 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 --2- 172.21.15.91:0/2684904244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c0addd0 0x7f8d9c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 >> 172.21.15.91:0/2684904244 conn(0x7f8d9c0cea90 msgr2=0x7f8d9c165140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 shutdown_connections 2024-02-02T13:27:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.028+0000 7f8d9be00640 1 -- 172.21.15.91:0/2684904244 wait complete. 2024-02-02T13:27:05.046 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:27:05.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.237+0000 7fec8da00640 1 Processor -- start 2024-02-02T13:27:05.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.237+0000 7fec8da00640 1 -- start start 2024-02-02T13:27:05.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.239+0000 7fec8da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 0x7fec8806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.239+0000 7fec8da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.239+0000 7fec8da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8815d870 con 0x7fec880ac8d0 2024-02-02T13:27:05.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.239+0000 7fec8da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880ad2a0 con 0x7fec8815d490 2024-02-02T13:27:05.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.239+0000 7fec8da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806bce0 con 0x7fec880b0e10 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec86a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 0x7fec8806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec86a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 0x7fec8806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:52664/0 (socket says 172.21.15.91:52664) 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:47682/0 (socket says 172.21.15.91:47682) 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec86a00640 1 -- 172.21.15.91:0/3339874045 learned_addr learned my addr 172.21.15.91:0/3339874045 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 -- 172.21.15.91:0/3339874045 >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 msgr2=0x7fec8806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 --2- 172.21.15.91:0/3339874045 >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 0x7fec8806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 -- 172.21.15.91:0/3339874045 >> v1:172.21.15.113:6789/0 conn(0x7fec880ac8d0 legacy=0x7fec880b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.240+0000 7fec87400640 1 -- 172.21.15.91:0/3339874045 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88161220 con 0x7fec880b0e10 2024-02-02T13:27:05.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.242+0000 7fec87400640 1 --2- 172.21.15.91:0/3339874045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389f78fbf902aa39 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.242+0000 7fec86000640 1 -- 172.21.15.91:0/3339874045 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8002cc50 con 0x7fec880b0e10 2024-02-02T13:27:05.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.242+0000 7fec86000640 1 -- 172.21.15.91:0/3339874045 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8002cdb0 con 0x7fec880b0e10 2024-02-02T13:27:05.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.242+0000 7fec86000640 1 -- 172.21.15.91:0/3339874045 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80034660 con 0x7fec880b0e10 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 -- 172.21.15.91:0/3339874045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 msgr2=0x7fec880acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 --2- 172.21.15.91:0/3339874045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8002bea0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 -- 172.21.15.91:0/3339874045 shutdown_connections 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 --2- 172.21.15.91:0/3339874045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880b0e10 0x7fec880acd60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 --2- 172.21.15.91:0/3339874045 >> v2:172.21.15.113:3300/0 conn(0x7fec8815d490 0x7fec8806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 -- 172.21.15.91:0/3339874045 >> 172.21.15.91:0/3339874045 conn(0x7fec880cec80 msgr2=0x7fec880cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 -- 172.21.15.91:0/3339874045 shutdown_connections 2024-02-02T13:27:05.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.243+0000 7fec8da00640 1 -- 172.21.15.91:0/3339874045 wait complete. 2024-02-02T13:27:05.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.245+0000 7fec8da00640 1 Processor -- start 2024-02-02T13:27:05.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.245+0000 7fec8da00640 1 -- start start 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880b0e10 0x7fec8815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 0x7fec880c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8814e9c0 con 0x7fec880ac8d0 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec880c3ef0 con 0x7fec880b0e10 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec8da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8815df40 con 0x7fec8815d490 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec86a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec86a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45062/0 (socket says 172.21.15.91:45062) 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec86a00640 1 -- 172.21.15.91:0/3283446827 learned_addr learned my addr 172.21.15.91:0/3283446827 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:05.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec7fe00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880b0e10 0x7fec8815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.246+0000 7fec87400640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 0x7fec880c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 -- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 msgr2=0x7fec880c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 0x7fec880c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 -- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880b0e10 msgr2=0x7fec8815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880b0e10 0x7fec8815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 -- 172.21.15.91:0/3283446827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80009a70 con 0x7fec880ac8d0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec87400640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 0x7fec880c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec86a00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e8b0 tx=0x7fec7800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800c940 con 0x7fec880ac8d0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800caa0 con 0x7fec880ac8d0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800ce30 con 0x7fec880ac8d0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880b2540 con 0x7fec880ac8d0 2024-02-02T13:27:05.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.247+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880b2b00 con 0x7fec880ac8d0 2024-02-02T13:27:05.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.249+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec880b2360 con 0x7fec880ac8d0 2024-02-02T13:27:05.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.249+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fec78013470 con 0x7fec880ac8d0 2024-02-02T13:27:05.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.250+0000 7fec7ea00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 0x7fec6006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.250+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec78088ab0 con 0x7fec880ac8d0 2024-02-02T13:27:05.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.251+0000 7fec7fe00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 0x7fec6006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.251+0000 7fec7fe00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 0x7fec6006c8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec88152030 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.255+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fec780540a0 con 0x7fec880ac8d0 2024-02-02T13:27:05.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.430+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec88066cf0 con 0x7fec880ac8d0 2024-02-02T13:27:05.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.431+0000 7fec7ea00640 1 -- 172.21.15.91:0/3283446827 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec780170a0 con 0x7fec880ac8d0 2024-02-02T13:27:05.432 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 msgr2=0x7fec6006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 0x7fec6006c8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec88152030 tx=0x7fec70009040 comp rx=0 tx=0).stop 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 msgr2=0x7fec8815da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e8b0 tx=0x7fec7800ed70 comp rx=0 tx=0).stop 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 shutdown_connections 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fec6006a430 0x7fec6006c8f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec8815d490 0x7fec880c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880b0e10 0x7fec8815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 --2- 172.21.15.91:0/3283446827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec880ac8d0 0x7fec8815da00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 >> 172.21.15.91:0/3283446827 conn(0x7fec880cec80 msgr2=0x7fec88070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.433+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 shutdown_connections 2024-02-02T13:27:05.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.434+0000 7fec8da00640 1 -- 172.21.15.91:0/3283446827 wait complete. 2024-02-02T13:27:05.454 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:27:05.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.645+0000 7f0995c00640 1 Processor -- start 2024-02-02T13:27:05.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.646+0000 7f0995c00640 1 -- start start 2024-02-02T13:27:05.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0995c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09900cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0995c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0990070980 0x7f09900b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0995c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0990160d90 con 0x7f09900ce530 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0995c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0990070fe0 con 0x7f0990070980 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0995c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900d2940 con 0x7f09901609b0 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0985e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f09900ce530 0x7f0990070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45362/0 (socket says 172.21.15.91:45362) 2024-02-02T13:27:05.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.647+0000 7f0985e00640 1 -- 172.21.15.91:0/2631374189 learned_addr learned my addr 172.21.15.91:0/2631374189 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098ea00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09900cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500962405 0 0) 0x7f0978001110 con 0x7f09900ce530 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0970003520 con 0x7f09900ce530 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098f400640 1 --2- 172.21.15.91:0/2631374189 >> v2:172.21.15.113:3300/0 conn(0x7f0990070980 0x7f09900b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 389233247 0 0) 0x7f0970003520 con 0x7f09900ce530 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 msgr2=0x7f09900cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09900cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 >> v2:172.21.15.113:3300/0 conn(0x7f0990070980 msgr2=0x7f09900b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 --2- 172.21.15.91:0/2631374189 >> v2:172.21.15.113:3300/0 conn(0x7f0990070980 0x7f09900b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900d2c20 con 0x7f09900ce530 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0978001800 con 0x7f09900ce530 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098ea00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09900cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:05.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 >> v1:172.21.15.91:6789/0 conn(0x7f09900ce530 legacy=0x7f0990070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0970003970 0x7f0970003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09700047c0 0x7f0970004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09700051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0990070fe0 con 0x7f09700047c0 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900d2940 con 0x7f0970003970 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097000bb40 con 0x7f09901609b0 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.648+0000 7f0985e00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09700047c0 0x7f0970004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f098ea00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0970003970 0x7f0970003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f098f400640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09700051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f0985e00640 1 -- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 msgr2=0x7f09700051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f0985e00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09901609b0 0x7f09700051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f0985e00640 1 -- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0970003970 msgr2=0x7f0970003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f0985e00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0970003970 0x7f0970003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.649+0000 7f0985e00640 1 -- 172.21.15.91:0/2631374189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970003520 con 0x7f09700047c0 2024-02-02T13:27:05.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f0985e00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09700047c0 0x7f0970004c40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f09900ce910 tx=0x7f09700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db7bf9cbe2b2aee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978005ad0 con 0x7f09700047c0 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978005c50 con 0x7f09700047c0 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f098e000640 1 -- 172.21.15.91:0/2631374189 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097802c8b0 con 0x7f09700047c0 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f0995c00640 1 -- 172.21.15.91:0/2631374189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09700047c0 msgr2=0x7f0970004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.650+0000 7f0995c00640 1 --2- 172.21.15.91:0/2631374189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09700047c0 0x7f0970004c40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f09900ce910 tx=0x7f09700057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.651+0000 7f098f400640 1 -- 172.21.15.91:0/2631374189 reap_dead start 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.651+0000 7f0995c00640 1 -- 172.21.15.91:0/2631374189 shutdown_connections 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.651+0000 7f0995c00640 1 -- 172.21.15.91:0/2631374189 >> 172.21.15.91:0/2631374189 conn(0x7f09900d3020 msgr2=0x7f09900d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.651+0000 7f0995c00640 1 -- 172.21.15.91:0/2631374189 shutdown_connections 2024-02-02T13:27:05.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.651+0000 7f0995c00640 1 -- 172.21.15.91:0/2631374189 wait complete. 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.653+0000 7f0995c00640 1 Processor -- start 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.653+0000 7f0995c00640 1 -- start start 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09700047c0 0x7f09900aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09901f1f30 0x7f09901f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09901595a0 con 0x7f09900ccb60 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900aad00 con 0x7f09901f1f30 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0995c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09901f28f0 con 0x7f09700047c0 2024-02-02T13:27:05.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45076/0 (socket says 172.21.15.91:45076) 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 -- 172.21.15.91:0/2971749334 learned_addr learned my addr 172.21.15.91:0/2971749334 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 -- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09700047c0 msgr2=0x7f09900aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09700047c0 0x7f09900aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 -- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09901f1f30 msgr2=0x7f09901f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09901f1f30 0x7f09901f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.654+0000 7f0985e00640 1 -- 172.21.15.91:0/2971749334 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900aaa20 con 0x7f09900ccb60 2024-02-02T13:27:05.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0985e00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f097802c450 tx=0x7f0978001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978039070 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097802ccc0 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09900ca190 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097803c040 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.655+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09900ca750 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.656+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09900ca8e0 con 0x7f09900ccb60 2024-02-02T13:27:05.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.657+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0978032070 con 0x7f09900ccb60 2024-02-02T13:27:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.658+0000 7f0987400640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 0x7f096806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.658+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09780a9af0 con 0x7f09900ccb60 2024-02-02T13:27:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.658+0000 7f098ea00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 0x7f096806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.659+0000 7f098ea00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 0x7f096806c920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0980006f80 tx=0x7f0980006e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:05.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.662+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f09780750d0 con 0x7f09900ccb60 2024-02-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.857+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0990160d90 con 0x7f09900ccb60 2024-02-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.858+0000 7f0987400640 1 -- 172.21.15.91:0/2971749334 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0990160d90 con 0x7f09900ccb60 2024-02-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 msgr2=0x7f096806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 0x7f096806c920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0980006f80 tx=0x7f0980006e30 comp rx=0 tx=0).stop 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 msgr2=0x7f09900ab920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f097802c450 tx=0x7f0978001b90 comp rx=0 tx=0).stop 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 shutdown_connections 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f096806a460 0x7f096806c920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09901f1f30 0x7f09901f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09900ccb60 0x7f09900ab920 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 --2- 172.21.15.91:0/2971749334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09700047c0 0x7f09900aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:05.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 >> 172.21.15.91:0/2971749334 conn(0x7f09900d3020 msgr2=0x7f099015c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:05.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 shutdown_connections 2024-02-02T13:27:05.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:05.860+0000 7f0995c00640 1 -- 172.21.15.91:0/2971749334 wait complete. 2024-02-02T13:27:05.879 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:27:05.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:05.880 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:27:06.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:27:06.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:06.023 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:27:06.069 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:06.112 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:06.112 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:06.205 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:06.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.410+0000 7fee7b200640 1 Processor -- start 2024-02-02T13:27:06.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.410+0000 7fee7b200640 1 -- start start 2024-02-02T13:27:06.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.411+0000 7fee7b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee7c15d3a0 0x7fee7c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 0x7fee7c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7b200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fee7c0c9450 0x7fee7c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7b200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0adbd0 con 0x7fee7c0c9450 2024-02-02T13:27:06.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0c59d0 con 0x7fee7c15d3a0 2024-02-02T13:27:06.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0ce3b0 con 0x7fee7c0add80 2024-02-02T13:27:06.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fee7c0c9450 0x7fee7c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fee7c0c9450 0x7fee7c0c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45080/0 (socket says 172.21.15.91:45080) 2024-02-02T13:27:06.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 -- 172.21.15.91:0/1973013388 learned_addr learned my addr 172.21.15.91:0/1973013388 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:06.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee79e00640 1 --2- 172.21.15.91:0/1973013388 >> v2:172.21.15.113:3300/0 conn(0x7fee7c15d3a0 0x7fee7c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee73e00640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 0x7fee7c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 -- 172.21.15.91:0/1973013388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 msgr2=0x7fee7c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:06.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 0x7fee7c0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 -- 172.21.15.91:0/1973013388 >> v2:172.21.15.113:3300/0 conn(0x7fee7c15d3a0 msgr2=0x7fee7c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 --2- 172.21.15.91:0/1973013388 >> v2:172.21.15.113:3300/0 conn(0x7fee7c15d3a0 0x7fee7c06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee7a800640 1 -- 172.21.15.91:0/1973013388 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0ce8e0 con 0x7fee7c0c9450 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.412+0000 7fee73e00640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 0x7fee7c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.413+0000 7fee7a800640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0c9450 0x7fee7c0c98e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee7c06b360 tx=0x7fee7402dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a60a2ac556bb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.414+0000 7fee79400640 1 -- 172.21.15.91:0/1973013388 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7402eb60 con 0x7fee7c0c9450 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.414+0000 7fee79400640 1 -- 172.21.15.91:0/1973013388 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7402ecc0 con 0x7fee7c0c9450 2024-02-02T13:27:06.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.414+0000 7fee79400640 1 -- 172.21.15.91:0/1973013388 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74036610 con 0x7fee7c0c9450 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.414+0000 7fee7b200640 1 -- 172.21.15.91:0/1973013388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0c9450 msgr2=0x7fee7c0c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.414+0000 7fee7b200640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0c9450 0x7fee7c0c98e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee7c06b360 tx=0x7fee7402dfc0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 -- 172.21.15.91:0/1973013388 shutdown_connections 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0c9450 0x7fee7c0c98e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 --2- 172.21.15.91:0/1973013388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c0add80 0x7fee7c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 --2- 172.21.15.91:0/1973013388 >> v2:172.21.15.113:3300/0 conn(0x7fee7c15d3a0 0x7fee7c06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 -- 172.21.15.91:0/1973013388 >> 172.21.15.91:0/1973013388 conn(0x7fee7c0cea90 msgr2=0x7fee7c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 -- 172.21.15.91:0/1973013388 shutdown_connections 2024-02-02T13:27:06.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.415+0000 7fee7b200640 1 -- 172.21.15.91:0/1973013388 wait complete. 2024-02-02T13:27:06.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.416+0000 7fee7b200640 1 Processor -- start 2024-02-02T13:27:06.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.416+0000 7fee7b200640 1 -- start start 2024-02-02T13:27:06.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7c0c9450 0x7fee7c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c15d3a0 0x7fee7c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0c50f0 con 0x7fee7c0add80 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1f5ca0 con 0x7fee7c0c9450 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee7b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0ae910 con 0x7fee7c15d3a0 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee79e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.417+0000 7fee79e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45096/0 (socket says 172.21.15.91:45096) 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 -- 172.21.15.91:0/956954897 learned_addr learned my addr 172.21.15.91:0/956954897 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 -- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c15d3a0 msgr2=0x7fee7c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c15d3a0 0x7fee7c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 -- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7c0c9450 msgr2=0x7fee7c1f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7c0c9450 0x7fee7c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 -- 172.21.15.91:0/956954897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74009b30 con 0x7fee7c0add80 2024-02-02T13:27:06.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee79e00640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d5f0 tx=0x7fee6c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0125f0 con 0x7fee7c0add80 2024-02-02T13:27:06.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c012750 con 0x7fee7c0add80 2024-02-02T13:27:06.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c012b00 con 0x7fee7c0add80 2024-02-02T13:27:06.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.418+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c0ae350 con 0x7fee7c0add80 2024-02-02T13:27:06.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.419+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c1f5f40 con 0x7fee7c0add80 2024-02-02T13:27:06.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.419+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c1f6100 con 0x7fee7c0add80 2024-02-02T13:27:06.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.420+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fee6c012c60 con 0x7fee7c0add80 2024-02-02T13:27:06.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.421+0000 7fee72000640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 0x7fee5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.421+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee6c088da0 con 0x7fee7c0add80 2024-02-02T13:27:06.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.421+0000 7fee73e00640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 0x7fee5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.422+0000 7fee73e00640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 0x7fee5406c8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fee7c159610 tx=0x7fee64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.426+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fee6c054390 con 0x7fee7c0add80 2024-02-02T13:27:06.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.602+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee7c1f5b00 con 0x7fee7c0add80 2024-02-02T13:27:06.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.605+0000 7fee72000640 1 -- 172.21.15.91:0/956954897 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16045 (secure 0 0 0) 0x7fee7c1f5b00 con 0x7fee7c0add80 2024-02-02T13:27:06.607 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:06.607 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:57.446022+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:06.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.607+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 msgr2=0x7fee5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.607+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 0x7fee5406c8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fee7c159610 tx=0x7fee64008040 comp rx=0 tx=0).stop 2024-02-02T13:27:06.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.607+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 msgr2=0x7fee7c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.607+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d5f0 tx=0x7fee6c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 shutdown_connections 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee5406a430 0x7fee5406c8f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee7c15d3a0 0x7fee7c0b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7c0c9450 0x7fee7c1f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 --2- 172.21.15.91:0/956954897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee7c0add80 0x7fee7c0b1990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 >> 172.21.15.91:0/956954897 conn(0x7fee7c0cea90 msgr2=0x7fee7c159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 shutdown_connections 2024-02-02T13:27:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.608+0000 7fee7b200640 1 -- 172.21.15.91:0/956954897 wait complete. 2024-02-02T13:27:06.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:26:25.185334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:26:26.457324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:27:06.628 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:27:06.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.827+0000 7f7488800640 1 Processor -- start 2024-02-02T13:27:06.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.827+0000 7f7488800640 1 -- start start 2024-02-02T13:27:06.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.828+0000 7f7488800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7480151f90 0x7f748015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7488800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7480070f80 con 0x7f748006f4f0 2024-02-02T13:27:06.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7488800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748006c600 con 0x7f74800b0e10 2024-02-02T13:27:06.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7488800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7480152370 con 0x7f7480151f90 2024-02-02T13:27:06.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7485400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f748006f4f0 0x7f74800b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45372/0 (socket says 172.21.15.91:45372) 2024-02-02T13:27:06.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7485400640 1 -- 172.21.15.91:0/1874623266 learned_addr learned my addr 172.21.15.91:0/1874623266 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7485e00640 1 --2- 172.21.15.91:0/1874623266 >> v2:172.21.15.113:3300/0 conn(0x7f7480151f90 0x7f748015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014287507 0 0) 0x7f7480070f80 con 0x7f748006f4f0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.829+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7460003520 con 0x7f748006f4f0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764849836 0 0) 0x7f7460003520 con 0x7f748006f4f0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 >> v2:172.21.15.113:3300/0 conn(0x7f7480151f90 msgr2=0x7f748015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 --2- 172.21.15.91:0/1874623266 >> v2:172.21.15.113:3300/0 conn(0x7f7480151f90 0x7f748015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 >> v1:172.21.15.113:6789/0 conn(0x7f74800b0e10 legacy=0x7f74800b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800cb010 con 0x7f748006f4f0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f7468001670 con 0x7f748006f4f0 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 >> v1:172.21.15.91:6789/0 conn(0x7f748006f4f0 legacy=0x7f74800b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f7460003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74600047c0 0x7f7460004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7460005180 0x7f746000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480152370 con 0x7f74600039c0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746000c080 con 0x7f74600047c0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746000c240 con 0x7f7460005180 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.830+0000 7f7485e00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f7460003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485400640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74600047c0 0x7f7460004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485e00640 1 -- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7460005180 msgr2=0x7f746000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7486800640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7460005180 0x7f746000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485e00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7460005180 0x7f746000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485e00640 1 -- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74600047c0 msgr2=0x7f7460004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485e00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74600047c0 0x7f7460004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485e00640 1 -- 172.21.15.91:0/1874623266 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460003520 con 0x7f74600039c0 2024-02-02T13:27:06.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7485400640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74600047c0 0x7f7460004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7486800640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7460005180 0x7f746000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.831+0000 7f7486800640 1 -- 172.21.15.91:0/1874623266 reap_dead start 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.832+0000 7f7485e00640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f7460003de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7470009ac0 tx=0x7f74600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4075696236dee50b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.832+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747002c6b0 con 0x7f74600039c0 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.832+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747002c810 con 0x7f74600039c0 2024-02-02T13:27:06.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.832+0000 7f7484a00640 1 -- 172.21.15.91:0/1874623266 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747002cba0 con 0x7f74600039c0 2024-02-02T13:27:06.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 -- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 msgr2=0x7f7460003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f7460003de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7470009ac0 tx=0x7f74600057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 -- 172.21.15.91:0/1874623266 shutdown_connections 2024-02-02T13:27:06.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 --2- 172.21.15.91:0/1874623266 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f7460003de0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 -- 172.21.15.91:0/1874623266 >> 172.21.15.91:0/1874623266 conn(0x7f74800cec80 msgr2=0x7f74800cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 -- 172.21.15.91:0/1874623266 shutdown_connections 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.833+0000 7f7488800640 1 -- 172.21.15.91:0/1874623266 wait complete. 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.835+0000 7f7488800640 1 Processor -- start 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.835+0000 7f7488800640 1 -- start start 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 0x7f74801504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 0x7f74800c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c8a00 con 0x7f74600039c0 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c7380 con 0x7f7480150b70 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7488800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c6770 con 0x7f74800c8040 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7485e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7485e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45112/0 (socket says 172.21.15.91:45112) 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.836+0000 7f7485e00640 1 -- 172.21.15.91:0/2150978270 learned_addr learned my addr 172.21.15.91:0/2150978270 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485400640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 0x7f74801504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 -- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 msgr2=0x7f74800c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7486800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 0x7f74800c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 0x7f74800c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 -- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 msgr2=0x7f74801504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 0x7f74801504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 -- 172.21.15.91:0/2150978270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800c6a50 con 0x7f74600039c0 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7486800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 0x7f74800c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485400640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 0x7f74801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:06.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7485e00640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7470009ac0 tx=0x7f7470004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470003810 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470003970 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470003d00 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.837+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74800c6d30 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.838+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7480151350 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.838+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480066cf0 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.839+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7470040450 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.840+0000 7f747e000640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 0x7f745806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.840+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74700aaab0 con 0x7f74600039c0 2024-02-02T13:27:06.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.840+0000 7f7485400640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 0x7f745806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:06.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.841+0000 7f7485400640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 0x7f745806c8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74801f61b0 tx=0x7f7468009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:06.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:06.844+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f74700760a0 con 0x7f74600039c0 2024-02-02T13:27:07.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.037+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74800c9600 con 0x7f74600039c0 2024-02-02T13:27:07.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.038+0000 7f747e000640 1 -- 172.21.15.91:0/2150978270 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7470003e60 con 0x7f74600039c0 2024-02-02T13:27:07.040 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:27:07.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.040+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 msgr2=0x7f745806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.040+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 0x7f745806c8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74801f61b0 tx=0x7f7468009040 comp rx=0 tx=0).stop 2024-02-02T13:27:07.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.040+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 msgr2=0x7f74800c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.040+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7470009ac0 tx=0x7f7470004730 comp rx=0 tx=0).stop 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 shutdown_connections 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f745806a430 0x7f745806c8f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800c8040 0x7f74800c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7480150b70 0x7f74801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 --2- 172.21.15.91:0/2150978270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74600039c0 0x7f74800c5120 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 >> 172.21.15.91:0/2150978270 conn(0x7f74800cec80 msgr2=0x7f74800cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 shutdown_connections 2024-02-02T13:27:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.041+0000 7f7488800640 1 -- 172.21.15.91:0/2150978270 wait complete. 2024-02-02T13:27:07.059 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:27:07.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.263+0000 7f647ea00640 1 Processor -- start 2024-02-02T13:27:07.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.263+0000 7f647ea00640 1 -- start start 2024-02-02T13:27:07.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.264+0000 7f647ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f647815a010 0x7f64780c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.264+0000 7f647ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 0x7f64780ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f647ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64780cd470 0x7f64780b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f647ea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780cc7c0 con 0x7f647815a010 2024-02-02T13:27:07.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f647ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780ce930 con 0x7f64780cd470 2024-02-02T13:27:07.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f647ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647806c890 con 0x7f64780ce130 2024-02-02T13:27:07.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f647815a010 0x7f64780c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f647815a010 0x7f64780c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45118/0 (socket says 172.21.15.91:45118) 2024-02-02T13:27:07.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 -- 172.21.15.91:0/3642234068 learned_addr learned my addr 172.21.15.91:0/3642234068 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:07.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f647ca00640 1 --2- 172.21.15.91:0/3642234068 >> v2:172.21.15.113:3300/0 conn(0x7f64780cd470 0x7f64780b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477400640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 0x7f64780ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 -- 172.21.15.91:0/3642234068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 msgr2=0x7f64780ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:07.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 0x7f64780ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 -- 172.21.15.91:0/3642234068 >> v2:172.21.15.113:3300/0 conn(0x7f64780cd470 msgr2=0x7f64780b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 --2- 172.21.15.91:0/3642234068 >> v2:172.21.15.113:3300/0 conn(0x7f64780cd470 0x7f64780b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477e00640 1 -- 172.21.15.91:0/3642234068 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647806cb70 con 0x7f647815a010 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.265+0000 7f6477400640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 0x7f64780ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.266+0000 7f6477e00640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f647815a010 0x7f64780c9670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f64780ce5c0 tx=0x7f646802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d02084d6ce861c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.267+0000 7f6476a00640 1 -- 172.21.15.91:0/3642234068 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646802ee90 con 0x7f647815a010 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.267+0000 7f6476a00640 1 -- 172.21.15.91:0/3642234068 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6468036440 con 0x7f647815a010 2024-02-02T13:27:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.267+0000 7f6476a00640 1 -- 172.21.15.91:0/3642234068 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64680367d0 con 0x7f647815a010 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.267+0000 7f647ea00640 1 -- 172.21.15.91:0/3642234068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f647815a010 msgr2=0x7f64780c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.267+0000 7f647ea00640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f647815a010 0x7f64780c9670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f64780ce5c0 tx=0x7f646802e0e0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 -- 172.21.15.91:0/3642234068 shutdown_connections 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 --2- 172.21.15.91:0/3642234068 >> v2:172.21.15.113:3300/0 conn(0x7f64780cd470 0x7f64780b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64780ce130 0x7f64780ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 --2- 172.21.15.91:0/3642234068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f647815a010 0x7f64780c9670 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 -- 172.21.15.91:0/3642234068 >> 172.21.15.91:0/3642234068 conn(0x7f64780cec80 msgr2=0x7f64780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:07.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 -- 172.21.15.91:0/3642234068 shutdown_connections 2024-02-02T13:27:07.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.268+0000 7f647ea00640 1 -- 172.21.15.91:0/3642234068 wait complete. 2024-02-02T13:27:07.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.269+0000 7f647ea00640 1 Processor -- start 2024-02-02T13:27:07.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.269+0000 7f647ea00640 1 -- start start 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f647815a010 0x7f64780c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 0x7f64781f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647814fd80 con 0x7f64780ce130 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64780c60d0 con 0x7f647815a010 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.270+0000 7f647ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64781f3330 con 0x7f64781f6c10 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45128/0 (socket says 172.21.15.91:45128) 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 -- 172.21.15.91:0/893404709 learned_addr learned my addr 172.21.15.91:0/893404709 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f647ca00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 0x7f64781f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477400640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f647815a010 0x7f64780c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 -- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 msgr2=0x7f64781f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 0x7f64781f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 -- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f647815a010 msgr2=0x7f64780c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f647815a010 0x7f64780c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f6477e00640 1 -- 172.21.15.91:0/893404709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646800b770 con 0x7f64780ce130 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.271+0000 7f647ca00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 0x7f64781f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:07.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f6477e00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f646802e120 tx=0x7f64680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468034850 con 0x7f64780ce130 2024-02-02T13:27:07.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64680349b0 con 0x7f64780ce130 2024-02-02T13:27:07.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468034d60 con 0x7f64780ce130 2024-02-02T13:27:07.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64781f3610 con 0x7f64780ce130 2024-02-02T13:27:07.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.272+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780c5d60 con 0x7f64780ce130 2024-02-02T13:27:07.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.273+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64780c5ef0 con 0x7f64780ce130 2024-02-02T13:27:07.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.274+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f64680403e0 con 0x7f64780ce130 2024-02-02T13:27:07.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.275+0000 7f646fe00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 0x7f645006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.275+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64680aacc0 con 0x7f64780ce130 2024-02-02T13:27:07.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.275+0000 7f6477400640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 0x7f645006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.276+0000 7f6477400640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 0x7f645006c8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.280+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f64680b1080 con 0x7f64780ce130 2024-02-02T13:27:07.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.472+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6478066cf0 con 0x7f64780ce130 2024-02-02T13:27:07.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.473+0000 7f646fe00640 1 -- 172.21.15.91:0/893404709 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64680460a0 con 0x7f64780ce130 2024-02-02T13:27:07.475 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:27:07.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.475+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 msgr2=0x7f645006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.475+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 0x7f645006c8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-02-02T13:27:07.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.475+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 msgr2=0x7f647814f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.475+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f646802e120 tx=0x7f64680046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 shutdown_connections 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f645006a430 0x7f645006c8f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64781f6c10 0x7f64781f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f647815a010 0x7f64780c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 --2- 172.21.15.91:0/893404709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64780ce130 0x7f647814f840 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 >> 172.21.15.91:0/893404709 conn(0x7f64780cec80 msgr2=0x7f64780af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 shutdown_connections 2024-02-02T13:27:07.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.476+0000 7f647ea00640 1 -- 172.21.15.91:0/893404709 wait complete. 2024-02-02T13:27:07.496 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:27:07.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.694+0000 7f9acf200640 1 Processor -- start 2024-02-02T13:27:07.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.694+0000 7f9acf200640 1 -- start start 2024-02-02T13:27:07.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.695+0000 7f9acf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ac80b0700 0x7f9ac80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.695+0000 7f9acf200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80ccdd0 con 0x7f9ac80cd260 2024-02-02T13:27:07.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.695+0000 7f9acf200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80cc7c0 con 0x7f9ac8161660 2024-02-02T13:27:07.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.695+0000 7f9acf200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80b10d0 con 0x7f9ac80b0700 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9acd200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9ac80cd260 0x7f9ac80cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45380/0 (socket says 172.21.15.91:45380) 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9acd200640 1 -- 172.21.15.91:0/2610077019 learned_addr learned my addr 172.21.15.91:0/2610077019 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266210642 0 0) 0x7f9ac80ccdd0 con 0x7f9ac80cd260 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa8003520 con 0x7f9ac80cd260 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9abf200640 1 --2- 172.21.15.91:0/2610077019 >> v2:172.21.15.113:3300/0 conn(0x7f9ac80b0700 0x7f9ac80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862113127 0 0) 0x7f9aa8003520 con 0x7f9ac80cd260 2024-02-02T13:27:07.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 >> v2:172.21.15.113:3300/0 conn(0x7f9ac80b0700 msgr2=0x7f9ac80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 --2- 172.21.15.91:0/2610077019 >> v2:172.21.15.113:3300/0 conn(0x7f9ac80b0700 0x7f9ac80b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.696+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 >> v1:172.21.15.113:6789/0 conn(0x7f9ac8161660 legacy=0x7f9ac806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80c9570 con 0x7f9ac80cd260 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9ac00019f0 con 0x7f9ac80cd260 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 >> v1:172.21.15.91:6789/0 conn(0x7f9ac80cd260 legacy=0x7f9ac80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 0x7f9aa8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa80047c0 0x7f9aa8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aa80052a0 0x7f9aa800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b10d0 con 0x7f9aa8003970 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa800c170 con 0x7f9aa80047c0 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa800c330 con 0x7f9aa80052a0 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 0x7f9aa8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9abf200640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa80047c0 0x7f9aa8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 -- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aa80052a0 msgr2=0x7f9aa800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:07.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aa80052a0 0x7f9aa800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 -- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa80047c0 msgr2=0x7f9aa8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:07.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa80047c0 0x7f9aa8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.697+0000 7f9ac7e00640 1 -- 172.21.15.91:0/2610077019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8003520 con 0x7f9aa8003970 2024-02-02T13:27:07.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.698+0000 7f9acd200640 1 -- 172.21.15.91:0/2610077019 reap_dead start 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.698+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 0x7f9aa8003d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8001720 tx=0x7f9aa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581eec014bb8acd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80052d0 con 0x7f9aa8003970 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab802e050 con 0x7f9aa8003970 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9ac7400640 1 -- 172.21.15.91:0/2610077019 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8003830 con 0x7f9aa8003970 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 -- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 msgr2=0x7f9aa8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 0x7f9aa8003d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8001720 tx=0x7f9aa80057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 -- 172.21.15.91:0/2610077019 shutdown_connections 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 --2- 172.21.15.91:0/2610077019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aa8003970 0x7f9aa8003d90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 -- 172.21.15.91:0/2610077019 >> 172.21.15.91:0/2610077019 conn(0x7f9ac80cec80 msgr2=0x7f9ac80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:07.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 -- 172.21.15.91:0/2610077019 shutdown_connections 2024-02-02T13:27:07.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.699+0000 7f9acf200640 1 -- 172.21.15.91:0/2610077019 wait complete. 2024-02-02T13:27:07.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.701+0000 7f9acf200640 1 Processor -- start 2024-02-02T13:27:07.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.701+0000 7f9acf200640 1 -- start start 2024-02-02T13:27:07.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 0x7f9ac80c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ac815f5d0 0x7f9ac815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b1ed0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8161b50 con 0x7f9aa8003970 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9acf200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80c3f70 con 0x7f9ac815f5d0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.702+0000 7f9abf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45154/0 (socket says 172.21.15.91:45154) 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 -- 172.21.15.91:0/3449094123 learned_addr learned my addr 172.21.15.91:0/3449094123 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 0x7f9ac80c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 -- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ac815f5d0 msgr2=0x7f9ac815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ac815f5d0 0x7f9ac815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 -- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 msgr2=0x7f9ac80c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 0x7f9ac80c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 -- 172.21.15.91:0/3449094123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80b18a0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 0x7f9ac80c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9abf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e7e0 tx=0x7f9ab000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.703+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000c6d0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.704+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab000c830 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.704+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac80c3670 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.704+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000cba0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.704+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80c44a0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.705+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac815ffc0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.706+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9ab000cd00 con 0x7f9ac80c49c0 2024-02-02T13:27:07.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.706+0000 7f9ac4c00640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 0x7f9aa006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:07.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.707+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab0088660 con 0x7f9ac80c49c0 2024-02-02T13:27:07.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.707+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 0x7f9aa006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:07.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.707+0000 7f9ac7e00640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 0x7f9aa006c8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8033000 tx=0x7f9ab8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:07.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.710+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9ab0053c40 con 0x7f9ac80c49c0 2024-02-02T13:27:07.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.898+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ac8066cf0 con 0x7f9ac80c49c0 2024-02-02T13:27:07.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.899+0000 7f9ac4c00640 1 -- 172.21.15.91:0/3449094123 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ab0057260 con 0x7f9ac80c49c0 2024-02-02T13:27:07.900 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:27:07.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 msgr2=0x7f9aa006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 0x7f9aa006c8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8033000 tx=0x7f9ab8009030 comp rx=0 tx=0).stop 2024-02-02T13:27:07.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 msgr2=0x7f9ac80707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:07.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e7e0 tx=0x7f9ab000eca0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 shutdown_connections 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9aa006a430 0x7f9aa006c8f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ac815f5d0 0x7f9ac815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ac80c49c0 0x7f9ac80707b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 --2- 172.21.15.91:0/3449094123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aa8003970 0x7f9ac80c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.901+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 >> 172.21.15.91:0/3449094123 conn(0x7f9ac80cec80 msgr2=0x7f9ac815bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.902+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 shutdown_connections 2024-02-02T13:27:07.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:07.902+0000 7f9acf200640 1 -- 172.21.15.91:0/3449094123 wait complete. 2024-02-02T13:27:07.921 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:27:08.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.118+0000 7fb52c400640 1 Processor -- start 2024-02-02T13:27:08.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.118+0000 7fb52c400640 1 -- start start 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.119+0000 7fb52c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb52406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.119+0000 7fb52c400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52406d910 con 0x7fb52415ad10 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.119+0000 7fb52c400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5240cc850 con 0x7fb52415b0f0 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.119+0000 7fb52c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52406d730 con 0x7fb5240cbc90 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb529a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb52415ad10 0x7fb52406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45384/0 (socket says 172.21.15.91:45384) 2024-02-02T13:27:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb529a00640 1 -- 172.21.15.91:0/45467406 learned_addr learned my addr 172.21.15.91:0/45467406 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb52406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900405561 0 0) 0x7fb52406d910 con 0x7fb52415ad10 2024-02-02T13:27:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003520 con 0x7fb52415ad10 2024-02-02T13:27:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4200916083 0 0) 0x7fb504003520 con 0x7fb52415ad10 2024-02-02T13:27:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 msgr2=0x7fb52406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb52406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 >> v1:172.21.15.113:6789/0 conn(0x7fb52415b0f0 legacy=0x7fb5240cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240ccb30 con 0x7fb52415ad10 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb514001670 con 0x7fb52415ad10 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.120+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb52406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 >> v1:172.21.15.91:6789/0 conn(0x7fb52415ad10 legacy=0x7fb52406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb5040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504003fe0 0x7fb5040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb504005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52406d730 con 0x7fb504004d00 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400bbc0 con 0x7fb504003fe0 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400bd80 con 0x7fb5240cbc90 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb504005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb529a00640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb5040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb529000640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504003fe0 0x7fb5040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 -- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 msgr2=0x7fb5040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:08.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5240cbc90 0x7fb5040039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 -- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504003fe0 msgr2=0x7fb5040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:08.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504003fe0 0x7fb5040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.121+0000 7fb52a400640 1 -- 172.21.15.91:0/45467406 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504003520 con 0x7fb504004d00 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52a400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb504005180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0073e0 tx=0x7fb5040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e7396127359e58 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c02cd30 con 0x7fb504004d00 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c02ce90 con 0x7fb504004d00 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb523e00640 1 -- 172.21.15.91:0/45467406 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c034640 con 0x7fb504004d00 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52c400640 1 -- 172.21.15.91:0/45467406 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 msgr2=0x7fb504005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52c400640 1 --2- 172.21.15.91:0/45467406 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb504005180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0073e0 tx=0x7fb5040057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52a400640 1 -- 172.21.15.91:0/45467406 reap_dead start 2024-02-02T13:27:08.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52c400640 1 -- 172.21.15.91:0/45467406 shutdown_connections 2024-02-02T13:27:08.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.123+0000 7fb52c400640 1 -- 172.21.15.91:0/45467406 >> 172.21.15.91:0/45467406 conn(0x7fb5240b0700 msgr2=0x7fb5240b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:08.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.124+0000 7fb52c400640 1 -- 172.21.15.91:0/45467406 shutdown_connections 2024-02-02T13:27:08.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.124+0000 7fb52c400640 1 -- 172.21.15.91:0/45467406 wait complete. 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.125+0000 7fb52c400640 1 Processor -- start 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.125+0000 7fb52c400640 1 -- start start 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb524158d00 0x7fb524150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb524150c80 0x7fb5240af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240acbe0 con 0x7fb504004d00 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ace80 con 0x7fb524158d00 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb52c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ca400 con 0x7fb524150c80 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb529a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb529a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45172/0 (socket says 172.21.15.91:45172) 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.126+0000 7fb529a00640 1 -- 172.21.15.91:0/390790009 learned_addr learned my addr 172.21.15.91:0/390790009 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 -- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb524150c80 msgr2=0x7fb5240af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb524150c80 0x7fb5240af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 -- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb524158d00 msgr2=0x7fb524150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb524158d00 0x7fb524150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 -- 172.21.15.91:0/390790009 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240ca6e0 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb529a00640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d800 tx=0x7fb51400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140074e0 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514007640 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140049e0 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.127+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5240ca9c0 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.128+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5240af7d0 con 0x7fb504004d00 2024-02-02T13:27:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.128+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5240cabb0 con 0x7fb504004d00 2024-02-02T13:27:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.130+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb514004b40 con 0x7fb504004d00 2024-02-02T13:27:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.130+0000 7fb521600640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 0x7fb4fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.130+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb514088330 con 0x7fb504004d00 2024-02-02T13:27:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.131+0000 7fb529000640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 0x7fb4fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.131+0000 7fb529000640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 0x7fb4fc06c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5240cb3a0 tx=0x7fb50c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.135+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb5240cabb0 con 0x7fb504004d00 2024-02-02T13:27:08.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.337+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb5240cabb0 con 0x7fb504004d00 2024-02-02T13:27:08.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.337+0000 7fb521600640 1 -- 172.21.15.91:0/390790009 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb5240cabb0 con 0x7fb504004d00 2024-02-02T13:27:08.339 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:27:08.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.340+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 msgr2=0x7fb4fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.340+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 0x7fb4fc06c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5240cb3a0 tx=0x7fb50c009040 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.340+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 msgr2=0x7fb5240c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.340+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d800 tx=0x7fb51400dcc0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 shutdown_connections 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a430 0x7fb4fc06c8f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb524150c80 0x7fb5240af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb524158d00 0x7fb524150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 --2- 172.21.15.91:0/390790009 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb504004d00 0x7fb5240c9bd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 >> 172.21.15.91:0/390790009 conn(0x7fb5240b0700 msgr2=0x7fb52415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 shutdown_connections 2024-02-02T13:27:08.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.341+0000 7fb52c400640 1 -- 172.21.15.91:0/390790009 wait complete. 2024-02-02T13:27:08.359 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:27:08.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:08.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:27:08.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:27:08.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:08.465 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:27:08.518 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:08.518 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:08.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:27:08.573 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:08.573 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:08.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:27:08.628 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:08.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.841+0000 7fa15be00640 1 Processor -- start 2024-02-02T13:27:08.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.841+0000 7fa15be00640 1 -- start start 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.842+0000 7fa15be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa154151f90 0x7fa15415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.842+0000 7fa15be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa15be00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa154070f80 con 0x7fa1540b0c40 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa15be00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540b10d0 con 0x7fa154151f90 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa15be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154152370 con 0x7fa15406f4f0 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa1540b0c40 0x7fa1540b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45390/0 (socket says 172.21.15.91:45390) 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159e00640 1 -- 172.21.15.91:0/3640693606 learned_addr learned my addr 172.21.15.91:0/3640693606 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa154151f90 0x7fa15415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa158a00640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159400640 1 -- 172.21.15.91:0/3640693606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 msgr2=0x7fa1540b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159400640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159400640 1 -- 172.21.15.91:0/3640693606 >> v1:172.21.15.91:6789/0 conn(0x7fa1540b0c40 legacy=0x7fa1540b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa159400640 1 -- 172.21.15.91:0/3640693606 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540cb010 con 0x7fa154151f90 2024-02-02T13:27:08.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa153e00640 1 -- 172.21.15.91:0/3640693606 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25602448 0 0) 0x7fa154070f80 con 0x7fa1540b0c40 2024-02-02T13:27:08.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.843+0000 7fa158a00640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.844+0000 7fa159400640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa154151f90 0x7fa15415f5a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097a0 tx=0x7fa14402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7c440b30bffcf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa153e00640 1 -- 172.21.15.91:0/3640693606 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14402d040 con 0x7fa154151f90 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa153e00640 1 -- 172.21.15.91:0/3640693606 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144031810 con 0x7fa154151f90 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa153e00640 1 -- 172.21.15.91:0/3640693606 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144031ba0 con 0x7fa154151f90 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 -- 172.21.15.91:0/3640693606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa154151f90 msgr2=0x7fa15415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa154151f90 0x7fa15415f5a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097a0 tx=0x7fa14402b8b0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 -- 172.21.15.91:0/3640693606 shutdown_connections 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 --2- 172.21.15.91:0/3640693606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa154151f90 0x7fa15415f5a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 -- 172.21.15.91:0/3640693606 >> 172.21.15.91:0/3640693606 conn(0x7fa1540cec80 msgr2=0x7fa1540cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:08.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.845+0000 7fa15be00640 1 -- 172.21.15.91:0/3640693606 shutdown_connections 2024-02-02T13:27:08.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.846+0000 7fa15be00640 1 -- 172.21.15.91:0/3640693606 wait complete. 2024-02-02T13:27:08.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.847+0000 7fa15be00640 1 Processor -- start 2024-02-02T13:27:08.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.847+0000 7fa15be00640 1 -- start start 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa154151f90 0x7fa1540cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540c84a0 con 0x7fa1540b0c40 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541f2560 con 0x7fa154151f90 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa15be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540c7820 con 0x7fa15406f4f0 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa158a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa158a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45198/0 (socket says 172.21.15.91:45198) 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.848+0000 7fa158a00640 1 -- 172.21.15.91:0/2378903988 learned_addr learned my addr 172.21.15.91:0/2378903988 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 -- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 msgr2=0x7fa1540c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 -- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa154151f90 msgr2=0x7fa1540cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa154151f90 0x7fa1540cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 -- 172.21.15.91:0/2378903988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144009400 con 0x7fa1540b0c40 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa158a00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7f0 tx=0x7fa13c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c007820 con 0x7fa1540b0c40 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c007980 con 0x7fa1540b0c40 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540c7b00 con 0x7fa1540b0c40 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.849+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c007cf0 con 0x7fa1540b0c40 2024-02-02T13:27:08.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.850+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1540c80c0 con 0x7fa1540b0c40 2024-02-02T13:27:08.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.850+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa154066cf0 con 0x7fa1540b0c40 2024-02-02T13:27:08.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.851+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa13c0113f0 con 0x7fa1540b0c40 2024-02-02T13:27:08.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.852+0000 7fa151600640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 0x7fa12c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:08.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.852+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa13c088c50 con 0x7fa1540b0c40 2024-02-02T13:27:08.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.853+0000 7fa159400640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 0x7fa12c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:08.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.853+0000 7fa159400640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 0x7fa12c06c8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa144036000 tx=0x7fa144030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:08.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:08.857+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa13c054230 con 0x7fa1540b0c40 2024-02-02T13:27:09.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.072+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa15406c7b0 con 0x7fa1540b0c40 2024-02-02T13:27:09.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.074+0000 7fa151600640 1 -- 172.21.15.91:0/2378903988 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fa13c01f090 con 0x7fa1540b0c40 2024-02-02T13:27:09.076 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:09.076 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:27:09.076 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:27:09.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 msgr2=0x7fa12c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 0x7fa12c06c8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa144036000 tx=0x7fa144030000 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 msgr2=0x7fa1541f2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7f0 tx=0x7fa13c00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 shutdown_connections 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa12c06a430 0x7fa12c06c8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa154151f90 0x7fa1540cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1540b0c40 0x7fa1541f2020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 --2- 172.21.15.91:0/2378903988 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa15406f4f0 0x7fa1540c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.076+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 >> 172.21.15.91:0/2378903988 conn(0x7fa1540cec80 msgr2=0x7fa1540cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.077+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 shutdown_connections 2024-02-02T13:27:09.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.077+0000 7fa15be00640 1 -- 172.21.15.91:0/2378903988 wait complete. 2024-02-02T13:27:09.095 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:09.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.295+0000 7f154e800640 1 Processor -- start 2024-02-02T13:27:09.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.295+0000 7f154e800640 1 -- start start 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.296+0000 7f154e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1548151f90 0x7f154815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.296+0000 7f154e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15480b0e10 0x7f15480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.296+0000 7f154e800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1548070f80 con 0x7f154806f4f0 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f154e800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1548152370 con 0x7f15480b0e10 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f154e800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480cb010 con 0x7f1548151f90 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15480b0e10 0x7f15480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15480b0e10 0x7f15480b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45214/0 (socket says 172.21.15.91:45214) 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 -- 172.21.15.91:0/4125096922 learned_addr learned my addr 172.21.15.91:0/4125096922 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547400640 1 --2- 172.21.15.91:0/4125096922 >> v2:172.21.15.113:3300/0 conn(0x7f1548151f90 0x7f154815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 -- 172.21.15.91:0/4125096922 >> v2:172.21.15.113:3300/0 conn(0x7f1548151f90 msgr2=0x7f154815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:09.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 --2- 172.21.15.91:0/4125096922 >> v2:172.21.15.113:3300/0 conn(0x7f1548151f90 0x7f154815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 -- 172.21.15.91:0/4125096922 >> v1:172.21.15.113:6789/0 conn(0x7f154806f4f0 legacy=0x7f15480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:09.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.297+0000 7f1547e00640 1 -- 172.21.15.91:0/4125096922 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480cb2f0 con 0x7f15480b0e10 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.298+0000 7f1547e00640 1 --2- 172.21.15.91:0/4125096922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15480b0e10 0x7f15480b3af0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15480cc770 tx=0x7f154002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620cdc636c800a6f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f1546000640 1 -- 172.21.15.91:0/4125096922 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154002ca80 con 0x7f15480b0e10 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f1546000640 1 -- 172.21.15.91:0/4125096922 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154002cbe0 con 0x7f15480b0e10 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f1546000640 1 -- 172.21.15.91:0/4125096922 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400343f0 con 0x7f15480b0e10 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 -- 172.21.15.91:0/4125096922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15480b0e10 msgr2=0x7f15480b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 --2- 172.21.15.91:0/4125096922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15480b0e10 0x7f15480b3af0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15480cc770 tx=0x7f154002bee0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 -- 172.21.15.91:0/4125096922 shutdown_connections 2024-02-02T13:27:09.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 --2- 172.21.15.91:0/4125096922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15480b0e10 0x7f15480b3af0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 --2- 172.21.15.91:0/4125096922 >> v2:172.21.15.113:3300/0 conn(0x7f1548151f90 0x7f154815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 -- 172.21.15.91:0/4125096922 >> 172.21.15.91:0/4125096922 conn(0x7f15480cec80 msgr2=0x7f15480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 -- 172.21.15.91:0/4125096922 shutdown_connections 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.299+0000 7f154e800640 1 -- 172.21.15.91:0/4125096922 wait complete. 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.301+0000 7f154e800640 1 Processor -- start 2024-02-02T13:27:09.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.301+0000 7f154e800640 1 -- start start 2024-02-02T13:27:09.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15480b0e10 0x7f15480cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1548151f90 0x7f15481505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c6800 con 0x7f154806f4f0 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c6a30 con 0x7f15480b0e10 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f154e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c6d10 con 0x7f1548151f90 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f1547400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f1547400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45226/0 (socket says 172.21.15.91:45226) 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.302+0000 7f1547400640 1 -- 172.21.15.91:0/3364535395 learned_addr learned my addr 172.21.15.91:0/3364535395 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 -- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1548151f90 msgr2=0x7f15481505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1548151f90 0x7f15481505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1546a00640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15480b0e10 0x7f15480cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 -- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15480b0e10 msgr2=0x7f15480cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15480b0e10 0x7f15480cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 -- 172.21.15.91:0/3364535395 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540009b30 con 0x7f154806f4f0 2024-02-02T13:27:09.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f1547400640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f153800d800 tx=0x7f153800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800b6c0 con 0x7f154806f4f0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153800b820 con 0x7f154806f4f0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800bbb0 con 0x7f154806f4f0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.303+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480c6ff0 con 0x7f154806f4f0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.304+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15480c75b0 con 0x7f154806f4f0 2024-02-02T13:27:09.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.305+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15480c7740 con 0x7f154806f4f0 2024-02-02T13:27:09.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.306+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f153800bd10 con 0x7f154806f4f0 2024-02-02T13:27:09.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.306+0000 7f153f400640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 0x7f152006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.306+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1538088e50 con 0x7f154806f4f0 2024-02-02T13:27:09.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.307+0000 7f1546a00640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 0x7f152006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.307+0000 7f1546a00640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 0x7f152006c8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15480ada40 tx=0x7f1530008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.311+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1538054430 con 0x7f154806f4f0 2024-02-02T13:27:09.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.500+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1548066cf0 con 0x7f154806f4f0 2024-02-02T13:27:09.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.503+0000 7f153f400640 1 -- 172.21.15.91:0/3364535395 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f1538057a50 con 0x7f154806f4f0 2024-02-02T13:27:09.505 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:09.505 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:57.446022+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 msgr2=0x7f152006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 0x7f152006c8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15480ada40 tx=0x7f1530008040 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 msgr2=0x7f15481f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f153800d800 tx=0x7f153800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 shutdown_connections 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f152006a430 0x7f152006c8f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1548151f90 0x7f15481505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15480b0e10 0x7f15480cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 --2- 172.21.15.91:0/3364535395 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f154806f4f0 0x7f15481f2250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 >> 172.21.15.91:0/3364535395 conn(0x7f15480cec80 msgr2=0x7f15480c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 shutdown_connections 2024-02-02T13:27:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.505+0000 7f154e800640 1 -- 172.21.15.91:0/3364535395 wait complete. 2024-02-02T13:27:09.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:26:25.185334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:26:26.457324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:27:09.526 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:27:09.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.727+0000 7f50ece00640 1 Processor -- start 2024-02-02T13:27:09.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.727+0000 7f50ece00640 1 -- start start 2024-02-02T13:27:09.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50ece00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50e815aec0 0x7f50e80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50ece00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e806c1c0 con 0x7f50e806f3f0 2024-02-02T13:27:09.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50ece00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e806b5f0 con 0x7f50e815b2a0 2024-02-02T13:27:09.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50ece00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e806cd60 con 0x7f50e815aec0 2024-02-02T13:27:09.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50e7e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f50e806f3f0 0x7f50e80ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45392/0 (socket says 172.21.15.91:45392) 2024-02-02T13:27:09.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50e7e00640 1 -- 172.21.15.91:0/4104332093 learned_addr learned my addr 172.21.15.91:0/4104332093 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50e7400640 1 --2- 172.21.15.91:0/4104332093 >> v2:172.21.15.113:3300/0 conn(0x7f50e815aec0 0x7f50e80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822677362 0 0) 0x7f50e806c1c0 con 0x7f50e806f3f0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.729+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c8003520 con 0x7f50e806f3f0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301655134 0 0) 0x7f50c8003520 con 0x7f50e806f3f0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 >> v2:172.21.15.113:3300/0 conn(0x7f50e815aec0 msgr2=0x7f50e80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 --2- 172.21.15.91:0/4104332093 >> v2:172.21.15.113:3300/0 conn(0x7f50e815aec0 0x7f50e80cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 >> v1:172.21.15.113:6789/0 conn(0x7f50e815b2a0 legacy=0x7f50e806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e806b890 con 0x7f50e806f3f0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f50e00019f0 con 0x7f50e806f3f0 2024-02-02T13:27:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 >> v1:172.21.15.91:6789/0 conn(0x7f50e806f3f0 legacy=0x7f50e80ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80039c0 0x7f50c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 0x7f50c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50c8005180 0x7f50c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e806cd60 con 0x7f50c80047c0 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c800c060 con 0x7f50c80039c0 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c800c220 con 0x7f50c8005180 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.730+0000 7f50e6a00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 0x7f50c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e6a00640 1 -- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50c8005180 msgr2=0x7f50c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e6a00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50c8005180 0x7f50c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e6a00640 1 -- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80039c0 msgr2=0x7f50c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e6a00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80039c0 0x7f50c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e6a00640 1 -- 172.21.15.91:0/4104332093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8003520 con 0x7f50c80047c0 2024-02-02T13:27:09.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.731+0000 7f50e7e00640 1 -- 172.21.15.91:0/4104332093 reap_dead start 2024-02-02T13:27:09.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50e6a00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 0x7f50c8004c40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001750 tx=0x7f50c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be28929c3ac346fc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d00052d0 con 0x7f50c80047c0 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d002e050 con 0x7f50c80047c0 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50e6000640 1 -- 172.21.15.91:0/4104332093 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0003830 con 0x7f50c80047c0 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50ece00640 1 -- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 msgr2=0x7f50c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50ece00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 0x7f50c8004c40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001750 tx=0x7f50c80057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50ece00640 1 -- 172.21.15.91:0/4104332093 shutdown_connections 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50ece00640 1 --2- 172.21.15.91:0/4104332093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50c80047c0 0x7f50c8004c40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.732+0000 7f50ece00640 1 -- 172.21.15.91:0/4104332093 >> 172.21.15.91:0/4104332093 conn(0x7f50e80cea90 msgr2=0x7f50e80ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:09.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.733+0000 7f50ece00640 1 -- 172.21.15.91:0/4104332093 shutdown_connections 2024-02-02T13:27:09.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.733+0000 7f50ece00640 1 -- 172.21.15.91:0/4104332093 wait complete. 2024-02-02T13:27:09.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.734+0000 7f50ece00640 1 Processor -- start 2024-02-02T13:27:09.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.734+0000 7f50ece00640 1 -- start start 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80047c0 0x7f50e8157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50e80a9fe0 0x7f50e80ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c6680 con 0x7f50e80a9b60 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c9470 con 0x7f50c80047c0 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.735+0000 7f50ece00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c5ce0 con 0x7f50e80a9fe0 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45250/0 (socket says 172.21.15.91:45250) 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 -- 172.21.15.91:0/461755241 learned_addr learned my addr 172.21.15.91:0/461755241 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e7400640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80047c0 0x7f50e8157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e7e00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50e80a9fe0 0x7f50e80ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 -- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50e80a9fe0 msgr2=0x7f50e80ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50e80a9fe0 0x7f50e80ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 -- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80047c0 msgr2=0x7f50e8157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80047c0 0x7f50e8157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.736+0000 7f50e6a00640 1 -- 172.21.15.91:0/461755241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80c5fc0 con 0x7f50e80a9b60 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50e6a00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000c00 tx=0x7f50d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d00076c0 con 0x7f50e80a9b60 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d0007a30 con 0x7f50e80a9b60 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e80c62a0 con 0x7f50e80a9b60 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0006be0 con 0x7f50e80a9b60 2024-02-02T13:27:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.737+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e80c6aa0 con 0x7f50e80a9b60 2024-02-02T13:27:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.738+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e804dff0 con 0x7f50e80a9b60 2024-02-02T13:27:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.739+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f50d0006d40 con 0x7f50e80a9b60 2024-02-02T13:27:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.740+0000 7f50df400640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 0x7f50c006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.740+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d00aa4e0 con 0x7f50e80a9b60 2024-02-02T13:27:09.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.740+0000 7f50e7400640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 0x7f50c006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:09.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.741+0000 7f50e7400640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 0x7f50c006c920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f50e806d020 tx=0x7f50d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:09.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.744+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f50d0075ac0 con 0x7f50e80a9b60 2024-02-02T13:27:09.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.934+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50e80b46b0 con 0x7f50e80a9b60 2024-02-02T13:27:09.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.935+0000 7f50df400640 1 -- 172.21.15.91:0/461755241 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50d00790e0 con 0x7f50e80a9b60 2024-02-02T13:27:09.936 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 msgr2=0x7f50c006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 0x7f50c006c920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f50e806d020 tx=0x7f50d8008040 comp rx=0 tx=0).stop 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 msgr2=0x7f50e80c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000c00 tx=0x7f50d0001120 comp rx=0 tx=0).stop 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 shutdown_connections 2024-02-02T13:27:09.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c006a460 0x7f50c006c920 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50e80a9fe0 0x7f50e80ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50e80a9b60 0x7f50e80c9e60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 --2- 172.21.15.91:0/461755241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c80047c0 0x7f50e8157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.937+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 >> 172.21.15.91:0/461755241 conn(0x7f50e80cea90 msgr2=0x7f50e8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.938+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 shutdown_connections 2024-02-02T13:27:09.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:09.938+0000 7f50ece00640 1 -- 172.21.15.91:0/461755241 wait complete. 2024-02-02T13:27:09.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:27:10.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.158+0000 7fc49c400640 1 Processor -- start 2024-02-02T13:27:10.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.158+0000 7fc49c400640 1 -- start start 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.159+0000 7fc49c400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc49415d650 0x7fc4940ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.159+0000 7fc49c400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4940b0220 0x7fc49406a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.159+0000 7fc49c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc49c400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49406ab00 con 0x7fc4940b0220 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc49c400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940d1db0 con 0x7fc49415d650 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc49c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940d0690 con 0x7fc49406a650 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4940b0220 0x7fc49406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4940b0220 0x7fc49406a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53946/0 (socket says 172.21.15.91:53946) 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 -- 172.21.15.91:0/2279648046 learned_addr learned my addr 172.21.15.91:0/2279648046 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499a00640 1 --2- 172.21.15.91:0/2279648046 >> v2:172.21.15.113:3300/0 conn(0x7fc49415d650 0x7fc4940ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc49a400640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 -- 172.21.15.91:0/2279648046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 msgr2=0x7fc4940cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 -- 172.21.15.91:0/2279648046 >> v2:172.21.15.113:3300/0 conn(0x7fc49415d650 msgr2=0x7fc4940ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 --2- 172.21.15.91:0/2279648046 >> v2:172.21.15.113:3300/0 conn(0x7fc49415d650 0x7fc4940ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.160+0000 7fc499000640 1 -- 172.21.15.91:0/2279648046 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940d17f0 con 0x7fc4940b0220 2024-02-02T13:27:10.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.162+0000 7fc499000640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc49406a110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc49408b600 tx=0x7fc47c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd804b5ceda92f74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.162+0000 7fc493e00640 1 -- 172.21.15.91:0/2279648046 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c031870 con 0x7fc4940b0220 2024-02-02T13:27:10.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.162+0000 7fc493e00640 1 -- 172.21.15.91:0/2279648046 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c0319d0 con 0x7fc4940b0220 2024-02-02T13:27:10.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.162+0000 7fc493e00640 1 -- 172.21.15.91:0/2279648046 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c031d60 con 0x7fc4940b0220 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 -- 172.21.15.91:0/2279648046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 msgr2=0x7fc49406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc49406a110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc49408b600 tx=0x7fc47c02b910 comp rx=0 tx=0).stop 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 -- 172.21.15.91:0/2279648046 shutdown_connections 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 --2- 172.21.15.91:0/2279648046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc49406a110 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 --2- 172.21.15.91:0/2279648046 >> v2:172.21.15.113:3300/0 conn(0x7fc49415d650 0x7fc4940ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 -- 172.21.15.91:0/2279648046 >> 172.21.15.91:0/2279648046 conn(0x7fc49409a1a0 msgr2=0x7fc4940d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 -- 172.21.15.91:0/2279648046 shutdown_connections 2024-02-02T13:27:10.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.163+0000 7fc49c400640 1 -- 172.21.15.91:0/2279648046 wait complete. 2024-02-02T13:27:10.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.165+0000 7fc49c400640 1 Processor -- start 2024-02-02T13:27:10.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.165+0000 7fc49c400640 1 -- start start 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 0x7fc49415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b3e00 con 0x7fc4940b0220 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b3af0 con 0x7fc49415d650 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940ca730 con 0x7fc49406a650 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53956/0 (socket says 172.21.15.91:53956) 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 -- 172.21.15.91:0/212302177 learned_addr learned my addr 172.21.15.91:0/212302177 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499a00640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc49a400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 0x7fc49415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 -- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 msgr2=0x7fc4940c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 -- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 msgr2=0x7fc49415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.166+0000 7fc499000640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 0x7fc49415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc499000640 1 -- 172.21.15.91:0/212302177 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0094d0 con 0x7fc4940b0220 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc49a400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 0x7fc49415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc499000640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc47c02b950 tx=0x7fc47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c03ad30 con 0x7fc4940b0220 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c0443f0 con 0x7fc4940b0220 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0447a0 con 0x7fc4940b0220 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940ca990 con 0x7fc4940b0220 2024-02-02T13:27:10.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.167+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940caf50 con 0x7fc4940b0220 2024-02-02T13:27:10.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.169+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4940b3f40 con 0x7fc4940b0220 2024-02-02T13:27:10.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.169+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc47c04d450 con 0x7fc4940b0220 2024-02-02T13:27:10.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.170+0000 7fc491600640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 0x7fc46c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.170+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc47c02d040 con 0x7fc4940b0220 2024-02-02T13:27:10.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.171+0000 7fc499a00640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 0x7fc46c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.171+0000 7fc499a00640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 0x7fc46c06c8f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.175+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc4940b3f40 con 0x7fc4940b0220 2024-02-02T13:27:10.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.364+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4940b3f40 con 0x7fc4940b0220 2024-02-02T13:27:10.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.365+0000 7fc491600640 1 -- 172.21.15.91:0/212302177 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4940b3f40 con 0x7fc4940b0220 2024-02-02T13:27:10.366 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.367+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 msgr2=0x7fc46c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.367+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 0x7fc46c06c8f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484008040 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.367+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 msgr2=0x7fc4940ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.367+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc47c02b950 tx=0x7fc47c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 shutdown_connections 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc46c06a430 0x7fc46c06c8f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc49415d650 0x7fc49415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4940b0220 0x7fc4940ca1f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 --2- 172.21.15.91:0/212302177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc49406a650 0x7fc4940c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 >> 172.21.15.91:0/212302177 conn(0x7fc49409a1a0 msgr2=0x7fc49415b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 shutdown_connections 2024-02-02T13:27:10.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.368+0000 7fc49c400640 1 -- 172.21.15.91:0/212302177 wait complete. 2024-02-02T13:27:10.387 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:27:10.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.594+0000 7f9ab3e00640 1 Processor -- start 2024-02-02T13:27:10.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.594+0000 7f9ab3e00640 1 -- start start 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9aac0b8a50 0x7f9aac0d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab3e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac0cd930 con 0x7f9aac0b2260 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac0d1680 con 0x7f9aac0d0070 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aac0d1960 con 0x7f9aac0b8a50 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab0a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9aac0b2260 0x7f9aac0b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45512/0 (socket says 172.21.15.91:45512) 2024-02-02T13:27:10.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.596+0000 7f9ab0a00640 1 -- 172.21.15.91:0/3750611822 learned_addr learned my addr 172.21.15.91:0/3750611822 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/3750611822 >> v2:172.21.15.113:3300/0 conn(0x7f9aac0b8a50 0x7f9aac0d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958301709 0 0) 0x7f9a94001110 con 0x7f9aac0b2260 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a8c003520 con 0x7f9aac0b2260 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479399301 0 0) 0x7f9a8c003520 con 0x7f9aac0b2260 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 >> v2:172.21.15.113:3300/0 conn(0x7f9aac0b8a50 msgr2=0x7f9aac0d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 --2- 172.21.15.91:0/3750611822 >> v2:172.21.15.113:3300/0 conn(0x7f9aac0b8a50 0x7f9aac0d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 >> v1:172.21.15.113:6789/0 conn(0x7f9aac0d0070 legacy=0x7f9aac0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0d1c40 con 0x7f9aac0b2260 2024-02-02T13:27:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9a94001800 con 0x7f9aac0b2260 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 >> v1:172.21.15.91:6789/0 conn(0x7f9aac0b2260 legacy=0x7f9aac0b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c003970 0x7f9a8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a8c0047c0 0x7f9a8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 0x7f9a8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0d1960 con 0x7f9a8c005180 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c00c020 con 0x7f9a8c003970 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.597+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c00c1e0 con 0x7f9a8c0047c0 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 0x7f9a8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1400640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c003970 0x7f9a8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 -- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a8c0047c0 msgr2=0x7f9a8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a8c0047c0 0x7f9a8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 -- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c003970 msgr2=0x7f9a8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:10.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c003970 0x7f9a8c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1e00640 1 -- 172.21.15.91:0/3750611822 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c003520 con 0x7f9a8c005180 2024-02-02T13:27:10.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.598+0000 7f9ab1400640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c003970 0x7f9a8c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:10.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.599+0000 7f9ab1e00640 1 -- 172.21.15.91:0/3750611822 reap_dead start 2024-02-02T13:27:10.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.599+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 0x7f9a8c00b980 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009ac0 tx=0x7f9a8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f30eb7dace971 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.599+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004020 con 0x7f9a8c005180 2024-02-02T13:27:10.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.599+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4004180 con 0x7f9a8c005180 2024-02-02T13:27:10.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.599+0000 7f9aabe00640 1 -- 172.21.15.91:0/3750611822 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402c710 con 0x7f9a8c005180 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 -- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 msgr2=0x7f9a8c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 0x7f9a8c00b980 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009ac0 tx=0x7f9a8c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 -- 172.21.15.91:0/3750611822 shutdown_connections 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/3750611822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a8c005180 0x7f9a8c00b980 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 -- 172.21.15.91:0/3750611822 >> 172.21.15.91:0/3750611822 conn(0x7f9aac070660 msgr2=0x7f9aac070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 -- 172.21.15.91:0/3750611822 shutdown_connections 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.600+0000 7f9ab3e00640 1 -- 172.21.15.91:0/3750611822 wait complete. 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.602+0000 7f9ab3e00640 1 Processor -- start 2024-02-02T13:27:10.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.602+0000 7f9ab3e00640 1 -- start start 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c005180 0x7f9aac149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aac0cbea0 0x7f9aac1460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0c9b60 con 0x7f9aac15f850 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac160210 con 0x7f9a8c005180 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cc6d0 con 0x7f9aac0cbea0 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53972/0 (socket says 172.21.15.91:53972) 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.603+0000 7f9ab1e00640 1 -- 172.21.15.91:0/2110526947 learned_addr learned my addr 172.21.15.91:0/2110526947 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 -- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aac0cbea0 msgr2=0x7f9aac1460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aac0cbea0 0x7f9aac1460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 -- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c005180 msgr2=0x7f9aac149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c005180 0x7f9aac149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 -- 172.21.15.91:0/2110526947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0cc9b0 con 0x7f9aac15f850 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab1e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402c380 tx=0x7f9aa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa403a070 con 0x7f9aac15f850 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa40042f0 con 0x7f9aac15f850 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac0ccc90 con 0x7f9aac15f850 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003610 con 0x7f9aac15f850 2024-02-02T13:27:10.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.604+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac0cd250 con 0x7f9aac15f850 2024-02-02T13:27:10.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.605+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac066cf0 con 0x7f9aac15f850 2024-02-02T13:27:10.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.606+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9aa4003770 con 0x7f9aac15f850 2024-02-02T13:27:10.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.607+0000 7f9aa9600640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 0x7f9a8406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:10.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.607+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9aa40a9120 con 0x7f9aac15f850 2024-02-02T13:27:10.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.608+0000 7f9ab1400640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 0x7f9a8406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:10.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.608+0000 7f9ab1400640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 0x7f9a8406c8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0012e0 tx=0x7f9a9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:10.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.612+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9aa4074690 con 0x7f9aac15f850 2024-02-02T13:27:10.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.803+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9aac0d0660 con 0x7f9aac15f850 2024-02-02T13:27:10.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.804+0000 7f9aa9600640 1 -- 172.21.15.91:0/2110526947 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9aa40450a0 con 0x7f9aac15f850 2024-02-02T13:27:10.805 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 msgr2=0x7f9a8406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 0x7f9a8406c8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0012e0 tx=0x7f9a9c009040 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 msgr2=0x7f9aac15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402c380 tx=0x7f9aa4004730 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 shutdown_connections 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8406a430 0x7f9a8406c8f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9aac0cbea0 0x7f9aac1460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9aac15f850 0x7f9aac15fcd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 --2- 172.21.15.91:0/2110526947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a8c005180 0x7f9aac149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 >> 172.21.15.91:0/2110526947 conn(0x7f9aac070660 msgr2=0x7f9aac0b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:10.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 shutdown_connections 2024-02-02T13:27:10.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:10.806+0000 7f9ab3e00640 1 -- 172.21.15.91:0/2110526947 wait complete. 2024-02-02T13:27:10.827 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:11.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.033+0000 7f3517c00640 1 Processor -- start 2024-02-02T13:27:11.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.033+0000 7f3517c00640 1 -- start start 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.034+0000 7f3517c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f351015a010 0x7f35100c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.034+0000 7f3517c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 0x7f35100b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.034+0000 7f3517c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35100cc7c0 con 0x7f35100ce130 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.034+0000 7f3517c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35100b10d0 con 0x7f351015a010 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.034+0000 7f3517c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35100ce930 con 0x7f35100b0700 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f3507c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f35100ce130 0x7f35100ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45516/0 (socket says 172.21.15.91:45516) 2024-02-02T13:27:11.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f3507c00640 1 -- 172.21.15.91:0/836711014 learned_addr learned my addr 172.21.15.91:0/836711014 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:11.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f3515200640 1 --2- 172.21.15.91:0/836711014 >> v2:172.21.15.113:3300/0 conn(0x7f351015a010 0x7f35100c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f3515c00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 0x7f35100b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814613510 0 0) 0x7f35100cc7c0 con 0x7f35100ce130 2024-02-02T13:27:11.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f0003520 con 0x7f35100ce130 2024-02-02T13:27:11.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2162851120 0 0) 0x7f34f0003520 con 0x7f35100ce130 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 msgr2=0x7f35100b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 0x7f35100b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 >> v2:172.21.15.113:3300/0 conn(0x7f351015a010 msgr2=0x7f35100c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 --2- 172.21.15.91:0/836711014 >> v2:172.21.15.113:3300/0 conn(0x7f351015a010 0x7f35100c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.035+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35100cd640 con 0x7f35100ce130 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f34f8001670 con 0x7f35100ce130 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 >> v1:172.21.15.91:6789/0 conn(0x7f35100ce130 legacy=0x7f35100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f34f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 0x7f34f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100b10d0 con 0x7f34f0003970 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100ce930 con 0x7f34f00047c0 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f000db90 con 0x7f35100b0700 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f34f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3507c00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 -- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 msgr2=0x7f34f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35100b0700 0x7f34f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 -- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00047c0 msgr2=0x7f34f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3515200640 1 -- 172.21.15.91:0/836711014 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0003520 con 0x7f34f0003970 2024-02-02T13:27:11.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.036+0000 7f3507c00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.037+0000 7f3515200640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f34f0003d90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fc0 tx=0x7f34f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07908d432846f9c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350002cc60 con 0x7f34f0003970 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350000f040 con 0x7f34f0003970 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f350fe00640 1 -- 172.21.15.91:0/836711014 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500034440 con 0x7f34f0003970 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 -- 172.21.15.91:0/836711014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 msgr2=0x7f34f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 --2- 172.21.15.91:0/836711014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f34f0003d90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fc0 tx=0x7f34f00057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3515c00640 1 -- 172.21.15.91:0/836711014 reap_dead start 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 -- 172.21.15.91:0/836711014 shutdown_connections 2024-02-02T13:27:11.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 -- 172.21.15.91:0/836711014 >> 172.21.15.91:0/836711014 conn(0x7f35100cec80 msgr2=0x7f35100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 -- 172.21.15.91:0/836711014 shutdown_connections 2024-02-02T13:27:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.038+0000 7f3517c00640 1 -- 172.21.15.91:0/836711014 wait complete. 2024-02-02T13:27:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.040+0000 7f3517c00640 1 Processor -- start 2024-02-02T13:27:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.040+0000 7f3517c00640 1 -- start start 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35100c2da0 0x7f35101ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35101efb50 0x7f35100b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100b26e0 con 0x7f34f0003970 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510158ff0 con 0x7f35100c2da0 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3517c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35100b17f0 con 0x7f35101efb50 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.041+0000 7f3515200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53988/0 (socket says 172.21.15.91:53988) 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 -- 172.21.15.91:0/1110582032 learned_addr learned my addr 172.21.15.91:0/1110582032 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3507c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35100c2da0 0x7f35101ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 -- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35101efb50 msgr2=0x7f35100b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35101efb50 0x7f35100b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 -- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35100c2da0 msgr2=0x7f35101ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35100c2da0 0x7f35101ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 -- 172.21.15.91:0/1110582032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35100b1ad0 con 0x7f34f0003970 2024-02-02T13:27:11.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.042+0000 7f3515200640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f350000e230 tx=0x7f3500009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.043+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500034d50 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.043+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500040450 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.043+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000407e0 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.043+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35100b1db0 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.043+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35100b2370 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.044+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3510066cf0 con 0x7f34f0003970 2024-02-02T13:27:11.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.045+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3500040940 con 0x7f34f0003970 2024-02-02T13:27:11.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.045+0000 7f350d600640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 0x7f34e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.046+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35000ab310 con 0x7f34f0003970 2024-02-02T13:27:11.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.046+0000 7f3507c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 0x7f34e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.046+0000 7f3507c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 0x7f34e806c8f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000fd0 tx=0x7f34f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.050+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f35000767d0 con 0x7f34f0003970 2024-02-02T13:27:11.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.279+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35100b4c00 con 0x7f34f0003970 2024-02-02T13:27:11.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.280+0000 7f350d600640 1 -- 172.21.15.91:0/1110582032 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f3500079df0 con 0x7f34f0003970 2024-02-02T13:27:11.282 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:11.282 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:27:11.283 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:27:11.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 msgr2=0x7f34e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 0x7f34e806c8f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000fd0 tx=0x7f34f8009040 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 msgr2=0x7f3510158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f350000e230 tx=0x7f3500009750 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 shutdown_connections 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f34e806a430 0x7f34e806c8f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35101efb50 0x7f35100b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35100c2da0 0x7f35101ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 --2- 172.21.15.91:0/1110582032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f0003970 0x7f3510158ab0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 >> 172.21.15.91:0/1110582032 conn(0x7f35100cec80 msgr2=0x7f35100c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 shutdown_connections 2024-02-02T13:27:11.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.283+0000 7f3517c00640 1 -- 172.21.15.91:0/1110582032 wait complete. 2024-02-02T13:27:11.305 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:11.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.511+0000 7f6b3e000640 1 Processor -- start 2024-02-02T13:27:11.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.511+0000 7f6b3e000640 1 -- start start 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b3e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6b381597a0 0x7f6b3806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b3e000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3806b5f0 con 0x7f6b380c9240 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b3e000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b380ce030 con 0x7f6b381597a0 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b3e000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3806c450 con 0x7f6b380cd690 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b37400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6b381597a0 0x7f6b3806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b37e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6b380c9240 0x7f6b3806f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45528/0 (socket says 172.21.15.91:45528) 2024-02-02T13:27:11.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.512+0000 7f6b37e00640 1 -- 172.21.15.91:0/3317346538 learned_addr learned my addr 172.21.15.91:0/3317346538 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:11.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.513+0000 7f6b37400640 1 -- 172.21.15.91:0/3317346538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 msgr2=0x7f6b380cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.513+0000 7f6b37400640 1 --2- 172.21.15.91:0/3317346538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.513+0000 7f6b37400640 1 -- 172.21.15.91:0/3317346538 >> v1:172.21.15.91:6789/0 conn(0x7f6b380c9240 legacy=0x7f6b3806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.513+0000 7f6b37400640 1 -- 172.21.15.91:0/3317346538 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38159b80 con 0x7f6b381597a0 2024-02-02T13:27:11.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.513+0000 7f6b36a00640 1 -- 172.21.15.91:0/3317346538 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779269903 0 0) 0x7f6b3806b5f0 con 0x7f6b380c9240 2024-02-02T13:27:11.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.514+0000 7f6b37400640 1 --2- 172.21.15.91:0/3317346538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b381597a0 0x7f6b3806c8e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94798acb9d2aa52a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.514+0000 7f6b36a00640 1 -- 172.21.15.91:0/3317346538 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2802d040 con 0x7f6b381597a0 2024-02-02T13:27:11.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.514+0000 7f6b36a00640 1 -- 172.21.15.91:0/3317346538 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28031810 con 0x7f6b381597a0 2024-02-02T13:27:11.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.514+0000 7f6b36a00640 1 -- 172.21.15.91:0/3317346538 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28031ba0 con 0x7f6b381597a0 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 -- 172.21.15.91:0/3317346538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b381597a0 msgr2=0x7f6b3806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 --2- 172.21.15.91:0/3317346538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b381597a0 0x7f6b3806c8e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802b8b0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 -- 172.21.15.91:0/3317346538 shutdown_connections 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 --2- 172.21.15.91:0/3317346538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 --2- 172.21.15.91:0/3317346538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b381597a0 0x7f6b3806c8e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 -- 172.21.15.91:0/3317346538 >> 172.21.15.91:0/3317346538 conn(0x7f6b380cec80 msgr2=0x7f6b380cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 -- 172.21.15.91:0/3317346538 shutdown_connections 2024-02-02T13:27:11.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.515+0000 7f6b3e000640 1 -- 172.21.15.91:0/3317346538 wait complete. 2024-02-02T13:27:11.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.517+0000 7f6b3e000640 1 Processor -- start 2024-02-02T13:27:11.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.517+0000 7f6b3e000640 1 -- start start 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b381597a0 0x7f6b380c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3814d590 con 0x7f6b380c9240 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380c40e0 con 0x7f6b381597a0 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b3e000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380c3b30 con 0x7f6b380cd690 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b37400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b37400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54002/0 (socket says 172.21.15.91:54002) 2024-02-02T13:27:11.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.518+0000 7f6b37400640 1 -- 172.21.15.91:0/2982491407 learned_addr learned my addr 172.21.15.91:0/2982491407 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 -- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 msgr2=0x7f6b380c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 -- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b381597a0 msgr2=0x7f6b380c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37e00640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b381597a0 0x7f6b380c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b381597a0 0x7f6b380c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 -- 172.21.15.91:0/2982491407 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28009400 con 0x7f6b380c9240 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b37400640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2803ad60 con 0x7f6b380c9240 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b280443f0 con 0x7f6b380c9240 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280447a0 con 0x7f6b380c9240 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.519+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b380c3e00 con 0x7f6b380c9240 2024-02-02T13:27:11.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.520+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b380c60b0 con 0x7f6b380c9240 2024-02-02T13:27:11.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.521+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b38066cf0 con 0x7f6b380c9240 2024-02-02T13:27:11.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.522+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6b2804d450 con 0x7f6b380c9240 2024-02-02T13:27:11.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.523+0000 7f6b2fe00640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 0x7f6b1006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.523+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b280aac10 con 0x7f6b380c9240 2024-02-02T13:27:11.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.523+0000 7f6b2e800640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 0x7f6b1006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.524+0000 7f6b2e800640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 0x7f6b1006c920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b380c1320 tx=0x7f6b20006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.527+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6b28076290 con 0x7f6b380c9240 2024-02-02T13:27:11.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.716+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b380cf500 con 0x7f6b380c9240 2024-02-02T13:27:11.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.719+0000 7f6b2fe00640 1 -- 172.21.15.91:0/2982491407 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16045 (secure 0 0 0) 0x7f6b280798b0 con 0x7f6b380c9240 2024-02-02T13:27:11.720 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:11.721 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:57.446022+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:11.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 msgr2=0x7f6b1006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 0x7f6b1006c920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b380c1320 tx=0x7f6b20006c90 comp rx=0 tx=0).stop 2024-02-02T13:27:11.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 msgr2=0x7f6b3814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b280046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 shutdown_connections 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b1006a460 0x7f6b1006c920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b380cd690 0x7f6b380c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b381597a0 0x7f6b380c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 --2- 172.21.15.91:0/2982491407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b380c9240 0x7f6b3814d050 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 >> 172.21.15.91:0/2982491407 conn(0x7f6b380cec80 msgr2=0x7f6b380ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 shutdown_connections 2024-02-02T13:27:11.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.722+0000 7f6b3e000640 1 -- 172.21.15.91:0/2982491407 wait complete. 2024-02-02T13:27:11.741 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:27:11.742 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:11.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.940+0000 7f6909200640 1 Processor -- start 2024-02-02T13:27:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.940+0000 7f6909200640 1 -- start start 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f690415d3a0 0x7f690406b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 0x7f69040706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 0x7f69040c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040adbd0 con 0x7f690415d3a0 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040c59d0 con 0x7f69040add80 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6909200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040ce3b0 con 0x7f69040c9450 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.941+0000 7f6903400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f690415d3a0 0x7f690406b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f690415d3a0 0x7f690406b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54016/0 (socket says 172.21.15.91:54016) 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 -- 172.21.15.91:0/2054112489 learned_addr learned my addr 172.21.15.91:0/2054112489 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6902a00640 1 --2- 172.21.15.91:0/2054112489 >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 0x7f69040706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 -- 172.21.15.91:0/2054112489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 msgr2=0x7f69040c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903e00640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 0x7f69040c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 0x7f69040c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 -- 172.21.15.91:0/2054112489 >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 msgr2=0x7f69040706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 --2- 172.21.15.91:0/2054112489 >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 0x7f69040706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903400640 1 -- 172.21.15.91:0/2054112489 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040ce8e0 con 0x7f690415d3a0 2024-02-02T13:27:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6902a00640 1 --2- 172.21.15.91:0/2054112489 >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 0x7f69040706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.942+0000 7f6903e00640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 0x7f69040c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.943+0000 7f6903400640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f690415d3a0 0x7f690406b450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f69040cb4b0 tx=0x7f68f402dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8013801600309747 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.943+0000 7f6902000640 1 -- 172.21.15.91:0/2054112489 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f402ed70 con 0x7f690415d3a0 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6902000640 1 -- 172.21.15.91:0/2054112489 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4036440 con 0x7f690415d3a0 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6902000640 1 -- 172.21.15.91:0/2054112489 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40367d0 con 0x7f690415d3a0 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 -- 172.21.15.91:0/2054112489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f690415d3a0 msgr2=0x7f690406b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f690415d3a0 0x7f690406b450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f69040cb4b0 tx=0x7f68f402dfc0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 -- 172.21.15.91:0/2054112489 shutdown_connections 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69040c9450 0x7f69040c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 --2- 172.21.15.91:0/2054112489 >> v2:172.21.15.113:3300/0 conn(0x7f69040add80 0x7f69040706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 --2- 172.21.15.91:0/2054112489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f690415d3a0 0x7f690406b450 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 -- 172.21.15.91:0/2054112489 >> 172.21.15.91:0/2054112489 conn(0x7f69040cea90 msgr2=0x7f69040ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 -- 172.21.15.91:0/2054112489 shutdown_connections 2024-02-02T13:27:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.944+0000 7f6909200640 1 -- 172.21.15.91:0/2054112489 wait complete. 2024-02-02T13:27:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.946+0000 7f6909200640 1 Processor -- start 2024-02-02T13:27:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.946+0000 7f6909200640 1 -- start start 2024-02-02T13:27:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69040c9450 0x7f6904147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f690415d3a0 0x7f6904147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690414af10 con 0x7f69040add80 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904148280 con 0x7f69040c9450 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6909200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040c88a0 con 0x7f690415d3a0 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6903400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6903400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54032/0 (socket says 172.21.15.91:54032) 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.947+0000 7f6903400640 1 -- 172.21.15.91:0/3836370866 learned_addr learned my addr 172.21.15.91:0/3836370866 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6902a00640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69040c9450 0x7f6904147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 -- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f690415d3a0 msgr2=0x7f6904147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f690415d3a0 0x7f6904147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 -- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69040c9450 msgr2=0x7f6904147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69040c9450 0x7f6904147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 -- 172.21.15.91:0/3836370866 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f400b770 con 0x7f69040add80 2024-02-02T13:27:11.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6903400640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4034750 con 0x7f69040add80 2024-02-02T13:27:11.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f40348b0 con 0x7f69040add80 2024-02-02T13:27:11.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4034c60 con 0x7f69040add80 2024-02-02T13:27:11.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.948+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690414a8c0 con 0x7f69040add80 2024-02-02T13:27:11.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.949+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690414b3a0 con 0x7f69040add80 2024-02-02T13:27:11.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.949+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690404dff0 con 0x7f69040add80 2024-02-02T13:27:11.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.950+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f68f404a3b0 con 0x7f69040add80 2024-02-02T13:27:11.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.951+0000 7f68fb400640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 0x7f68dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:11.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.951+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68f40aaec0 con 0x7f69040add80 2024-02-02T13:27:11.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.951+0000 7f6902a00640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 0x7f68dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:11.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.952+0000 7f6902a00640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 0x7f68dc06c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f69040cfab0 tx=0x7f68ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:11.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:11.956+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f68f40764a0 con 0x7f69040add80 2024-02-02T13:27:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.181+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69040b4a10 con 0x7f69040add80 2024-02-02T13:27:12.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.183+0000 7f68fb400640 1 -- 172.21.15.91:0/3836370866 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f68f4046090 con 0x7f69040add80 2024-02-02T13:27:12.183 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:12.184 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:27:12.184 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:27:12.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 msgr2=0x7f68dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 0x7f68dc06c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f69040cfab0 tx=0x7f68ec008040 comp rx=0 tx=0).stop 2024-02-02T13:27:12.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 msgr2=0x7f69040708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f40046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 shutdown_connections 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f68dc06a430 0x7f68dc06c8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f690415d3a0 0x7f6904147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69040c9450 0x7f6904147800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 --2- 172.21.15.91:0/3836370866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69040add80 0x7f69040708a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.185+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 >> 172.21.15.91:0/3836370866 conn(0x7f69040cea90 msgr2=0x7f69040c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.186+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 shutdown_connections 2024-02-02T13:27:12.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.186+0000 7f6909200640 1 -- 172.21.15.91:0/3836370866 wait complete. 2024-02-02T13:27:12.205 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:12.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.406+0000 7fc56a400640 1 Processor -- start 2024-02-02T13:27:12.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.406+0000 7fc56a400640 1 -- start start 2024-02-02T13:27:12.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.407+0000 7fc56a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5640ce130 0x7fc5640c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.407+0000 7fc56a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5640b0700 0x7fc5640b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.407+0000 7fc56a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5640cc7c0 con 0x7fc56415a010 2024-02-02T13:27:12.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.407+0000 7fc56a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5640b10d0 con 0x7fc5640b0700 2024-02-02T13:27:12.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.407+0000 7fc56a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5640ce930 con 0x7fc5640ce130 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.408+0000 7fc563e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5640b0700 0x7fc5640b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.408+0000 7fc55a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5640ce130 0x7fc5640c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.408+0000 7fc563e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5640b0700 0x7fc5640b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:35022/0 (socket says 172.21.15.91:35022) 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.408+0000 7fc563e00640 1 -- 172.21.15.91:0/463050525 learned_addr learned my addr 172.21.15.91:0/463050525 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.408+0000 7fc563400640 1 --1- 172.21.15.91:0/463050525 >> v1:172.21.15.113:6789/0 conn(0x7fc56415a010 0x7fc5640ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:59426/0 (socket says 172.21.15.91:59426) 2024-02-02T13:27:12.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.409+0000 7fc563e00640 1 -- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5640ce130 msgr2=0x7fc5640c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:12.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.409+0000 7fc563e00640 1 --2- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5640ce130 0x7fc5640c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.409+0000 7fc563e00640 1 -- 172.21.15.91:0/463050525 >> v1:172.21.15.113:6789/0 conn(0x7fc56415a010 legacy=0x7fc5640ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.409+0000 7fc563e00640 1 -- 172.21.15.91:0/463050525 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5640cd640 con 0x7fc5640b0700 2024-02-02T13:27:12.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.410+0000 7fc563e00640 1 --2- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640b0700 0x7fc5640b0b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009e50 tx=0x7fc55c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ec75dc3eabac8c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc562a00640 1 -- 172.21.15.91:0/463050525 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c02cb00 con 0x7fc5640b0700 2024-02-02T13:27:12.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc562a00640 1 -- 172.21.15.91:0/463050525 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c02cc60 con 0x7fc5640b0700 2024-02-02T13:27:12.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc562a00640 1 -- 172.21.15.91:0/463050525 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0343f0 con 0x7fc5640b0700 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 -- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640b0700 msgr2=0x7fc5640b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 --2- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640b0700 0x7fc5640b0b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009e50 tx=0x7fc55c02bf60 comp rx=0 tx=0).stop 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 -- 172.21.15.91:0/463050525 shutdown_connections 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 --2- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640b0700 0x7fc5640b0b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 --2- 172.21.15.91:0/463050525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5640ce130 0x7fc5640c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 -- 172.21.15.91:0/463050525 >> 172.21.15.91:0/463050525 conn(0x7fc5640cec80 msgr2=0x7fc5640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 -- 172.21.15.91:0/463050525 shutdown_connections 2024-02-02T13:27:12.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.411+0000 7fc56a400640 1 -- 172.21.15.91:0/463050525 wait complete. 2024-02-02T13:27:12.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.413+0000 7fc56a400640 1 Processor -- start 2024-02-02T13:27:12.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.413+0000 7fc56a400640 1 -- start start 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 0x7fc5640c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc56415a010 0x7fc5640c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5641516f0 con 0x7fc5640b0700 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5640b2f30 con 0x7fc5640ce130 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc56a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5640b2320 con 0x7fc56415a010 2024-02-02T13:27:12.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc563400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc563400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54044/0 (socket says 172.21.15.91:54044) 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.414+0000 7fc563400640 1 -- 172.21.15.91:0/3391201107 learned_addr learned my addr 172.21.15.91:0/3391201107 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc55a800640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 0x7fc5640c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 -- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc56415a010 msgr2=0x7fc5640c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563e00640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc56415a010 0x7fc5640c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc56415a010 0x7fc5640c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 -- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 msgr2=0x7fc5640c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 0x7fc5640c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 -- 172.21.15.91:0/3391201107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c009b30 con 0x7fc5640b0700 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc55a800640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 0x7fc5640c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc563400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c3e0 tx=0x7fc55400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.415+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540074e0 con 0x7fc5640b0700 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.416+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc554007640 con 0x7fc5640b0700 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.416+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5640b2600 con 0x7fc5640b0700 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.416+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540049e0 con 0x7fc5640b0700 2024-02-02T13:27:12.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.416+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5640b2bc0 con 0x7fc5640b0700 2024-02-02T13:27:12.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.417+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5640b2d80 con 0x7fc5640b0700 2024-02-02T13:27:12.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.418+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc554004b40 con 0x7fc5640b0700 2024-02-02T13:27:12.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.419+0000 7fc55be00640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 0x7fc53c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.419+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5540882a0 con 0x7fc5640b0700 2024-02-02T13:27:12.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.420+0000 7fc55a800640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 0x7fc53c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.420+0000 7fc55a800640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 0x7fc53c06c8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc56415e8c0 tx=0x7fc54c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.424+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc554053920 con 0x7fc5640b0700 2024-02-02T13:27:12.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.649+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5640ccf30 con 0x7fc5640b0700 2024-02-02T13:27:12.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.650+0000 7fc55be00640 1 -- 172.21.15.91:0/3391201107 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fc554056f40 con 0x7fc5640b0700 2024-02-02T13:27:12.651 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:12.651 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:26:27.440160+0000","modified":"2024-02-02T13:26:30.459190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5508":{"gid":5508,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/681798855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":681798855},{"type":"v1","addr":"172.21.15.91:6837","nonce":681798855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-02T13:27:12.651 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 24 2024-02-02T13:27:12.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 msgr2=0x7fc53c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 0x7fc53c06c8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc56415e8c0 tx=0x7fc54c008040 comp rx=0 tx=0).stop 2024-02-02T13:27:12.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 msgr2=0x7fc5641511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c3e0 tx=0x7fc55400c8a0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 shutdown_connections 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc53c06a430 0x7fc53c06c8f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc56415a010 0x7fc5640c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5640ce130 0x7fc5640c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 --2- 172.21.15.91:0/3391201107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5640b0700 0x7fc5641511b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 >> 172.21.15.91:0/3391201107 conn(0x7fc5640cec80 msgr2=0x7fc5640ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 shutdown_connections 2024-02-02T13:27:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.653+0000 7fc56a400640 1 -- 172.21.15.91:0/3391201107 wait complete. 2024-02-02T13:27:12.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:12.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.869+0000 7fcbcf000640 1 Processor -- start 2024-02-02T13:27:12.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.869+0000 7fcbcf000640 1 -- start start 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.870+0000 7fcbcf000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.870+0000 7fcbcf000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbc80b0700 0x7fcbc80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.870+0000 7fcbcf000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80cc7c0 con 0x7fcbc80ce130 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.870+0000 7fcbcf000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80b10d0 con 0x7fcbc80b0700 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.870+0000 7fcbcf000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80ce930 con 0x7fcbc815a010 2024-02-02T13:27:12.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbc80b0700 0x7fcbc80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbc80b0700 0x7fcbc80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54054/0 (socket says 172.21.15.91:54054) 2024-02-02T13:27:12.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 -- 172.21.15.91:0/1037417522 learned_addr learned my addr 172.21.15.91:0/1037417522 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:12.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbc7e00640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 -- 172.21.15.91:0/1037417522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 msgr2=0x7fcbc80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:12.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 -- 172.21.15.91:0/1037417522 >> v1:172.21.15.113:6789/0 conn(0x7fcbc80ce130 legacy=0x7fcbc80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:12.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.871+0000 7fcbcd000640 1 -- 172.21.15.91:0/1037417522 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80cd640 con 0x7fcbc80b0700 2024-02-02T13:27:12.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.872+0000 7fcbcd000640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80b0b90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009e50 tx=0x7fcbc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e89c5e7541b1995 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbc6a00640 1 -- 172.21.15.91:0/1037417522 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc002ca70 con 0x7fcbc80b0700 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbc6a00640 1 -- 172.21.15.91:0/1037417522 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc002cbd0 con 0x7fcbc80b0700 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbc6a00640 1 -- 172.21.15.91:0/1037417522 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc00343f0 con 0x7fcbc80b0700 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 -- 172.21.15.91:0/1037417522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 msgr2=0x7fcbc80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80b0b90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009e50 tx=0x7fcbc002bed0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 -- 172.21.15.91:0/1037417522 shutdown_connections 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80b0b90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1037417522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 -- 172.21.15.91:0/1037417522 >> 172.21.15.91:0/1037417522 conn(0x7fcbc80cec80 msgr2=0x7fcbc80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:12.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 -- 172.21.15.91:0/1037417522 shutdown_connections 2024-02-02T13:27:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.873+0000 7fcbcf000640 1 -- 172.21.15.91:0/1037417522 wait complete. 2024-02-02T13:27:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.875+0000 7fcbcf000640 1 Processor -- start 2024-02-02T13:27:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.875+0000 7fcbcf000640 1 -- start start 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc80ce130 0x7fcbc80b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b3880 con 0x7fcbc80b0700 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b3b60 con 0x7fcbc80ce130 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbcf000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b3e40 con 0x7fcbc815a010 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54068/0 (socket says 172.21.15.91:54068) 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.876+0000 7fcbc7e00640 1 -- 172.21.15.91:0/1492000795 learned_addr learned my addr 172.21.15.91:0/1492000795 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:12.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 -- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 msgr2=0x7fcbc80b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 -- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc80ce130 msgr2=0x7fcbc80b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc80ce130 0x7fcbc80b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 -- 172.21.15.91:0/1492000795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0009b30 con 0x7fcbc80b0700 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbc7e00640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c7f0 tx=0x7fcbb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8004220 con 0x7fcbc80b0700 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb8004380 con 0x7fcbc80b0700 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc80b4120 con 0x7fcbc80b0700 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8007840 con 0x7fcbc80b0700 2024-02-02T13:27:12.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.877+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc80b4640 con 0x7fcbc80b0700 2024-02-02T13:27:12.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.879+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc8066cf0 con 0x7fcbc80b0700 2024-02-02T13:27:12.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.879+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcbb8007bb0 con 0x7fcbc80b0700 2024-02-02T13:27:12.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.881+0000 7fcbbfe00640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 0x7fcba006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:12.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.881+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbb80888c0 con 0x7fcbc80b0700 2024-02-02T13:27:12.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.881+0000 7fcbc7400640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 0x7fcba006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:12.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.882+0000 7fcbc7400640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 0x7fcba006c8f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ae2e0 tx=0x7fcbb00079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:12.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:12.885+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcbb8053e10 con 0x7fcbc80b0700 2024-02-02T13:27:13.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.074+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc80b39c0 con 0x7fcbc80b0700 2024-02-02T13:27:13.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.076+0000 7fcbbfe00640 1 -- 172.21.15.91:0/1492000795 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16045 (secure 0 0 0) 0x7fcbc80b39c0 con 0x7fcbc80b0700 2024-02-02T13:27:13.077 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:13.077 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:26:57.446022+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:26:25.185334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-02-02T13:26:26.457324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 msgr2=0x7fcba006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 0x7fcba006c8f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ae2e0 tx=0x7fcbb00079b0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 msgr2=0x7fcbc80c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c7f0 tx=0x7fcbb800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 shutdown_connections 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcba006a430 0x7fcba006c8f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbc815a010 0x7fcbc80b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc80ce130 0x7fcbc80b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 --2- 172.21.15.91:0/1492000795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbc80b0700 0x7fcbc80c4d30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 >> 172.21.15.91:0/1492000795 conn(0x7fcbc80cec80 msgr2=0x7fcbc80c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 shutdown_connections 2024-02-02T13:27:13.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.079+0000 7fcbcf000640 1 -- 172.21.15.91:0/1492000795 wait complete. 2024-02-02T13:27:13.097 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:27:13.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.304+0000 7f33f3000640 1 Processor -- start 2024-02-02T13:27:13.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.304+0000 7f33f3000640 1 -- start start 2024-02-02T13:27:13.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.305+0000 7f33f3000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f33ec15b0f0 0x7f33ec06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33f3000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33ec0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33f3000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec06d910 con 0x7f33ec15ad10 2024-02-02T13:27:13.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33f3000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec06d730 con 0x7f33ec15b0f0 2024-02-02T13:27:13.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33f3000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0cbf50 con 0x7f33ec0cb580 2024-02-02T13:27:13.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33ebe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f33ec15ad10 0x7f33ec06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45534/0 (socket says 172.21.15.91:45534) 2024-02-02T13:27:13.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33ebe00640 1 -- 172.21.15.91:0/288389463 learned_addr learned my addr 172.21.15.91:0/288389463 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33eb400640 1 --2- 172.21.15.91:0/288389463 >> v2:172.21.15.113:3300/0 conn(0x7f33ec15b0f0 0x7f33ec06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33f1000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33ec0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263523837 0 0) 0x7f33ec06d910 con 0x7f33ec15ad10 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.306+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33cc003520 con 0x7f33ec15ad10 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4044021708 0 0) 0x7f33cc003520 con 0x7f33ec15ad10 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 msgr2=0x7f33ec0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33ec0cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 >> v2:172.21.15.113:3300/0 conn(0x7f33ec15b0f0 msgr2=0x7f33ec06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 --2- 172.21.15.91:0/288389463 >> v2:172.21.15.113:3300/0 conn(0x7f33ec15b0f0 0x7f33ec06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0cc850 con 0x7f33ec15ad10 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f33dc001670 con 0x7f33ec15ad10 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 >> v1:172.21.15.91:6789/0 conn(0x7f33ec15ad10 legacy=0x7f33ec06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33cc003ef0 0x7f33cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec06d730 con 0x7f33cc004d00 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0cbf50 con 0x7f33cc003ef0 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc00b9d0 con 0x7f33ec0cb580 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.307+0000 7f33f1000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33ebe00640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33eb400640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33cc003ef0 0x7f33cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33f1000640 1 -- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 msgr2=0x7f33cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33f1000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0cb580 0x7f33cc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33f1000640 1 -- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33cc003ef0 msgr2=0x7f33cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33f1000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33cc003ef0 0x7f33cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.308+0000 7f33f1000640 1 -- 172.21.15.91:0/288389463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc003520 con 0x7f33cc004d00 2024-02-02T13:27:13.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.309+0000 7f33f1000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e170 tx=0x7f33cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf28e2d7ce04af server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.309+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e402cbf0 con 0x7f33cc004d00 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.309+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e400f040 con 0x7f33cc004d00 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.309+0000 7f33eaa00640 1 -- 172.21.15.91:0/288389463 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4034440 con 0x7f33cc004d00 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 -- 172.21.15.91:0/288389463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33cc004d00 msgr2=0x7f33cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 --2- 172.21.15.91:0/288389463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e170 tx=0x7f33cc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f1000640 1 -- 172.21.15.91:0/288389463 reap_dead start 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 -- 172.21.15.91:0/288389463 shutdown_connections 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 -- 172.21.15.91:0/288389463 >> 172.21.15.91:0/288389463 conn(0x7f33ec0b0700 msgr2=0x7f33ec0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 -- 172.21.15.91:0/288389463 shutdown_connections 2024-02-02T13:27:13.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.310+0000 7f33f3000640 1 -- 172.21.15.91:0/288389463 wait complete. 2024-02-02T13:27:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.312+0000 7f33f3000640 1 Processor -- start 2024-02-02T13:27:13.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.312+0000 7f33f3000640 1 -- start start 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 0x7f33ec0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0c5c40 0x7f33ec15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec14e770 con 0x7f33ec0c6600 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c8d40 con 0x7f33ec14e390 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33f3000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c6e10 con 0x7f33ec0c5c40 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54080/0 (socket says 172.21.15.91:54080) 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33eb400640 1 -- 172.21.15.91:0/2759298371 learned_addr learned my addr 172.21.15.91:0/2759298371 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.313+0000 7f33ebe00640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 0x7f33ec0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33f1000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0c5c40 0x7f33ec15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 -- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0c5c40 msgr2=0x7f33ec15c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0c5c40 0x7f33ec15c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 -- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 msgr2=0x7f33ec0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 0x7f33ec0c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 -- 172.21.15.91:0/2759298371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0c6b30 con 0x7f33ec0c6600 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33ebe00640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 0x7f33ec0c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33eb400640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011600 tx=0x7f33d4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400f920 con 0x7f33ec0c6600 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d400fa80 con 0x7f33ec0c6600 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec151750 con 0x7f33ec0c6600 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400fdf0 con 0x7f33ec0c6600 2024-02-02T13:27:13.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.314+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec151d10 con 0x7f33ec0c6600 2024-02-02T13:27:13.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.315+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec1515b0 con 0x7f33ec0c6600 2024-02-02T13:27:13.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.316+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f33d4015460 con 0x7f33ec0c6600 2024-02-02T13:27:13.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.317+0000 7f33e3e00640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 0x7f33c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.317+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d4088de0 con 0x7f33ec0c6600 2024-02-02T13:27:13.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.318+0000 7f33ebe00640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 0x7f33c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.318+0000 7f33ebe00640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 0x7f33c406c920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33dc001eb0 tx=0x7f33dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.322+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33ec1515b0 con 0x7f33ec0c6600 2024-02-02T13:27:13.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.535+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f33ec1515b0 con 0x7f33ec0c6600 2024-02-02T13:27:13.537 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:27:13.537+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:27:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.551+0000 7f33e3e00640 1 -- 172.21.15.91:0/2759298371 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f33ec1515b0 con 0x7f33ec0c6600 2024-02-02T13:27:13.551 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 msgr2=0x7f33c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 0x7f33c406c920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33dc001eb0 tx=0x7f33dc009040 comp rx=0 tx=0).stop 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 msgr2=0x7f33ec0c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011600 tx=0x7f33d4011ac0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 shutdown_connections 2024-02-02T13:27:13.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33c406a460 0x7f33c406c920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33ec0c5c40 0x7f33ec15c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33ec0c6600 0x7f33ec0c8800 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011600 tx=0x7f33d4011ac0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 --2- 172.21.15.91:0/2759298371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33ec14e390 0x7f33ec0c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 >> 172.21.15.91:0/2759298371 conn(0x7f33ec0b0700 msgr2=0x7f33ec14ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:13.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 shutdown_connections 2024-02-02T13:27:13.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.555+0000 7f33f3000640 1 -- 172.21.15.91:0/2759298371 wait complete. 2024-02-02T13:27:13.572 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:27:13.636 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:27:13.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.757+0000 7f7a60a00640 1 Processor -- start 2024-02-02T13:27:13.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.757+0000 7f7a60a00640 1 -- start start 2024-02-02T13:27:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a60a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7a5806d690 0x7f7a58150530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a60a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a58159b90 con 0x7f7a580c9c30 2024-02-02T13:27:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a60a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a58159e70 con 0x7f7a580b0690 2024-02-02T13:27:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a60a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5815a810 con 0x7f7a5806d690 2024-02-02T13:27:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a5ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7a580c9c30 0x7f7a58164140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45542/0 (socket says 172.21.15.91:45542) 2024-02-02T13:27:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.758+0000 7f7a5ea00640 1 -- 172.21.15.91:0/2104489161 learned_addr learned my addr 172.21.15.91:0/2104489161 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5e000640 1 --2- 172.21.15.91:0/2104489161 >> v2:172.21.15.113:3300/0 conn(0x7f7a5806d690 0x7f7a58150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786193854 0 0) 0x7f7a58159b90 con 0x7f7a580c9c30 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a38003520 con 0x7f7a580c9c30 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 898330150 0 0) 0x7f7a38003520 con 0x7f7a580c9c30 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 >> v2:172.21.15.113:3300/0 conn(0x7f7a5806d690 msgr2=0x7f7a58150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 --2- 172.21.15.91:0/2104489161 >> v2:172.21.15.113:3300/0 conn(0x7f7a5806d690 0x7f7a58150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 >> v1:172.21.15.113:6789/0 conn(0x7f7a580b0690 legacy=0x7f7a580c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5815a150 con 0x7f7a580c9c30 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f7a500019f0 con 0x7f7a580c9c30 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 >> v1:172.21.15.91:6789/0 conn(0x7f7a580c9c30 legacy=0x7f7a58164140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a38003970 0x7f7a38003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a380047c0 0x7f7a38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 0x7f7a3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5815a810 con 0x7f7a380052a0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3800c1f0 con 0x7f7a380047c0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.759+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3800c3b0 con 0x7f7a38003970 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 0x7f7a3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5e000640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a38003970 0x7f7a38003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5d600640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a380047c0 0x7f7a38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 -- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a38003970 msgr2=0x7f7a38003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a38003970 0x7f7a38003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 -- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a380047c0 msgr2=0x7f7a38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a380047c0 0x7f7a38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5ea00640 1 -- 172.21.15.91:0/2104489161 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38003520 con 0x7f7a380052a0 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.760+0000 7f7a5d600640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a380047c0 0x7f7a38004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.761+0000 7f7a5ea00640 1 -- 172.21.15.91:0/2104489161 reap_dead start 2024-02-02T13:27:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.761+0000 7f7a5ea00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 0x7f7a3800b980 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7a50002230 tx=0x7f7a380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db940f6a3cbef03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.761+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50034630 con 0x7f7a380052a0 2024-02-02T13:27:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.761+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5002f070 con 0x7f7a380052a0 2024-02-02T13:27:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.761+0000 7f7a5cc00640 1 -- 172.21.15.91:0/2104489161 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500349f0 con 0x7f7a380052a0 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 -- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 msgr2=0x7f7a3800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 0x7f7a3800b980 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7a50002230 tx=0x7f7a380057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 -- 172.21.15.91:0/2104489161 shutdown_connections 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 --2- 172.21.15.91:0/2104489161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a380052a0 0x7f7a3800b980 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 -- 172.21.15.91:0/2104489161 >> 172.21.15.91:0/2104489161 conn(0x7f7a580ceb90 msgr2=0x7f7a580cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 -- 172.21.15.91:0/2104489161 shutdown_connections 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.762+0000 7f7a60a00640 1 -- 172.21.15.91:0/2104489161 wait complete. 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.764+0000 7f7a60a00640 1 Processor -- start 2024-02-02T13:27:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.764+0000 7f7a60a00640 1 -- start start 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a380052a0 0x7f7a5814f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a5814cb30 0x7f7a5814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5814d4f0 con 0x7f7a58144b40 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5814e290 con 0x7f7a5814cb30 2024-02-02T13:27:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a60a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58144fa0 con 0x7f7a380052a0 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54130/0 (socket says 172.21.15.91:54130) 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 -- 172.21.15.91:0/3015688403 learned_addr learned my addr 172.21.15.91:0/3015688403 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5e000640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a380052a0 0x7f7a5814f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5ea00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a5814cb30 0x7f7a5814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 -- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a380052a0 msgr2=0x7f7a5814f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a380052a0 0x7f7a5814f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 -- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a5814cb30 msgr2=0x7f7a5814cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a5814cb30 0x7f7a5814cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.765+0000 7f7a5d600640 1 -- 172.21.15.91:0/3015688403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5815b320 con 0x7f7a58144b40 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a5d600640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d5c0 tx=0x7f7a4000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40007220 con 0x7f7a58144b40 2024-02-02T13:27:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a400073a0 con 0x7f7a58144b40 2024-02-02T13:27:13.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400067d0 con 0x7f7a58144b40 2024-02-02T13:27:13.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5814a2b0 con 0x7f7a58144b40 2024-02-02T13:27:13.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.766+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5814a870 con 0x7f7a58144b40 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.767+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a58073830 con 0x7f7a58144b40 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.768+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a40006930 con 0x7f7a58144b40 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.769+0000 7f7a56000640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 0x7f7a3006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.769+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7a40088780 con 0x7f7a58144b40 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.769+0000 7f7a5e000640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 0x7f7a3006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:13.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.770+0000 7f7a5e000640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 0x7f7a3006c920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006fd0 tx=0x7f7a48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:13.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.773+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a40053c80 con 0x7f7a58144b40 2024-02-02T13:27:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:13.992+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7a58066cf0 con 0x7f7a58144b40 2024-02-02T13:27:14.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.564+0000 7f7a56000640 1 -- 172.21.15.91:0/3015688403 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a4001f0a0 con 0x7f7a58144b40 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 msgr2=0x7f7a3006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 0x7f7a3006c920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006fd0 tx=0x7f7a48008040 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 msgr2=0x7f7a5815ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d5c0 tx=0x7f7a4000da80 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 shutdown_connections 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a3006a460 0x7f7a3006c920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a5814cb30 0x7f7a5814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a58144b40 0x7f7a5815ade0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 --2- 172.21.15.91:0/3015688403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a380052a0 0x7f7a5814f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 >> 172.21.15.91:0/3015688403 conn(0x7f7a580ceb90 msgr2=0x7f7a580cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 shutdown_connections 2024-02-02T13:27:14.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.568+0000 7f7a60a00640 1 -- 172.21.15.91:0/3015688403 wait complete. 2024-02-02T13:27:14.590 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:27:14.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.794+0000 7f3eea400640 1 Processor -- start 2024-02-02T13:27:14.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.794+0000 7f3eea400640 1 -- start start 2024-02-02T13:27:14.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3eea400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ee40b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3eea400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ee40708c0 0x7f3ee4161580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3eea400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee4071650 con 0x7f3ee40704e0 2024-02-02T13:27:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3eea400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee40b6070 con 0x7f3ee40708c0 2024-02-02T13:27:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3eea400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40d0cb0 con 0x7f3ee40b2300 2024-02-02T13:27:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3ee9000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3ee40704e0 0x7f3ee40d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45554/0 (socket says 172.21.15.91:45554) 2024-02-02T13:27:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3ee9000640 1 -- 172.21.15.91:0/2549002020 learned_addr learned my addr 172.21.15.91:0/2549002020 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.796+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894279644 0 0) 0x7f3ecc001110 con 0x7f3ee40704e0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ee40b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec4003520 con 0x7f3ee40704e0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee9a00640 1 --2- 172.21.15.91:0/2549002020 >> v2:172.21.15.113:3300/0 conn(0x7f3ee40708c0 0x7f3ee4161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473827360 0 0) 0x7f3ec4003520 con 0x7f3ee40704e0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 msgr2=0x7f3ee40b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ee40b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 >> v2:172.21.15.113:3300/0 conn(0x7f3ee40708c0 msgr2=0x7f3ee4161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 --2- 172.21.15.91:0/2549002020 >> v2:172.21.15.113:3300/0 conn(0x7f3ee40708c0 0x7f3ee4161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40cdfc0 con 0x7f3ee40704e0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3ecc001800 con 0x7f3ee40704e0 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 >> v1:172.21.15.91:6789/0 conn(0x7f3ee40704e0 legacy=0x7f3ee40d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec4003970 0x7f3ec4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ec40047c0 0x7f3ec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ec4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40d0cb0 con 0x7f3ec40047c0 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec400bb10 con 0x7f3ec4003970 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec400bcd0 con 0x7f3ee40b2300 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.797+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ec40047c0 0x7f3ec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee9000640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec4003970 0x7f3ec4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee9a00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ec4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee3e00640 1 -- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 msgr2=0x7f3ec4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40b2300 0x7f3ec4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee3e00640 1 -- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec4003970 msgr2=0x7f3ec4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec4003970 0x7f3ec4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.798+0000 7f3ee3e00640 1 -- 172.21.15.91:0/2549002020 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec4003520 con 0x7f3ec40047c0 2024-02-02T13:27:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.799+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ec40047c0 0x7f3ec4004c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007330 tx=0x7f3ec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9b5ce34f7f9e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.799+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed402cca0 con 0x7f3ec40047c0 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.799+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed402ce00 con 0x7f3ec40047c0 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.799+0000 7f3ee3400640 1 -- 172.21.15.91:0/2549002020 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4034660 con 0x7f3ec40047c0 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 -- 172.21.15.91:0/2549002020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ec40047c0 msgr2=0x7f3ec4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 --2- 172.21.15.91:0/2549002020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ec40047c0 0x7f3ec4004c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007330 tx=0x7f3ec40057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3ee9a00640 1 -- 172.21.15.91:0/2549002020 reap_dead start 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 -- 172.21.15.91:0/2549002020 shutdown_connections 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 -- 172.21.15.91:0/2549002020 >> 172.21.15.91:0/2549002020 conn(0x7f3ee40cfbf0 msgr2=0x7f3ee40d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 -- 172.21.15.91:0/2549002020 shutdown_connections 2024-02-02T13:27:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.800+0000 7f3eea400640 1 -- 172.21.15.91:0/2549002020 wait complete. 2024-02-02T13:27:14.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.802+0000 7f3eea400640 1 Processor -- start 2024-02-02T13:27:14.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.802+0000 7f3eea400640 1 -- start start 2024-02-02T13:27:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40047c0 0x7f3ee4158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee4159480 0x7f3ee4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4158ab0 con 0x7f3ee4151570 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4158180 con 0x7f3ec40047c0 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3eea400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4158460 con 0x7f3ee4159480 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54154/0 (socket says 172.21.15.91:54154) 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9a00640 1 -- 172.21.15.91:0/1921348450 learned_addr learned my addr 172.21.15.91:0/1921348450 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9000640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40047c0 0x7f3ee4158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee3e00640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee4159480 0x7f3ee4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9a00640 1 -- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee4159480 msgr2=0x7f3ee4153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.803+0000 7f3ee9a00640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee4159480 0x7f3ee4153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee9a00640 1 -- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40047c0 msgr2=0x7f3ee4158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee9a00640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40047c0 0x7f3ee4158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee9a00640 1 -- 172.21.15.91:0/1921348450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4158740 con 0x7f3ee4151570 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee9a00640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40b92e0 tx=0x7f3edc00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc004440 con 0x7f3ee4151570 2024-02-02T13:27:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc003610 con 0x7f3ee4151570 2024-02-02T13:27:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0039c0 con 0x7f3ee4151570 2024-02-02T13:27:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4151d60 con 0x7f3ee4151570 2024-02-02T13:27:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.804+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee41511a0 con 0x7f3ee4151570 2024-02-02T13:27:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.805+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee4151390 con 0x7f3ee4151570 2024-02-02T13:27:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.806+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3edc003b20 con 0x7f3ee4151570 2024-02-02T13:27:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.807+0000 7f3ee0c00640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 0x7f3ebc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.807+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3edc088720 con 0x7f3ee4151570 2024-02-02T13:27:14.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.807+0000 7f3ee9000640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 0x7f3ebc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:14.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.808+0000 7f3ee9000640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 0x7f3ebc06c920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007b10 tx=0x7f3ecc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:14.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:14.812+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3edc053c10 con 0x7f3ee4151570 2024-02-02T13:27:15.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.003+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ee41588d0 con 0x7f3ee4151570 2024-02-02T13:27:15.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.569+0000 7f3ee0c00640 1 -- 172.21.15.91:0/1921348450 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3edc057230 con 0x7f3ee4151570 2024-02-02T13:27:15.571 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 msgr2=0x7f3ebc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 0x7f3ebc06c920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007b10 tx=0x7f3ecc009040 comp rx=0 tx=0).stop 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 msgr2=0x7f3ee4153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40b92e0 tx=0x7f3edc00f8a0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 shutdown_connections 2024-02-02T13:27:15.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3ebc06a460 0x7f3ebc06c920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee4159480 0x7f3ee4153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ee4151570 0x7f3ee4153f80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 --2- 172.21.15.91:0/1921348450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40047c0 0x7f3ee4158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 >> 172.21.15.91:0/1921348450 conn(0x7f3ee40cfbf0 msgr2=0x7f3ee40b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 shutdown_connections 2024-02-02T13:27:15.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.573+0000 7f3eea400640 1 -- 172.21.15.91:0/1921348450 wait complete. 2024-02-02T13:27:15.594 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:27:15.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.797+0000 7f2257400640 1 Processor -- start 2024-02-02T13:27:15.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.798+0000 7f2257400640 1 -- start start 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.799+0000 7f2257400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f225015d330 0x7f22500b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.799+0000 7f2257400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 0x7f22500d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.799+0000 7f2257400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2250161090 con 0x7f22500d3680 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.799+0000 7f2257400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2250161cd0 con 0x7f225015d330 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.799+0000 7f2257400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22500d3af0 con 0x7f22500b2470 2024-02-02T13:27:15.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f2255400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f22500d3680 0x7f22500b4c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45564/0 (socket says 172.21.15.91:45564) 2024-02-02T13:27:15.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f2255400640 1 -- 172.21.15.91:0/629755882 learned_addr learned my addr 172.21.15.91:0/629755882 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:15.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f2254a00640 1 --2- 172.21.15.91:0/629755882 >> v2:172.21.15.113:3300/0 conn(0x7f225015d330 0x7f22500b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582320263 0 0) 0x7f2250161090 con 0x7f22500d3680 2024-02-02T13:27:15.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2230003520 con 0x7f22500d3680 2024-02-02T13:27:15.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f224fe00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 0x7f22500d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.800+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590745948 0 0) 0x7f2230003520 con 0x7f22500d3680 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 msgr2=0x7f22500d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 0x7f22500d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 >> v2:172.21.15.113:3300/0 conn(0x7f225015d330 msgr2=0x7f22500b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 --2- 172.21.15.91:0/629755882 >> v2:172.21.15.113:3300/0 conn(0x7f225015d330 0x7f22500b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250161330 con 0x7f22500d3680 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2248001670 con 0x7f22500d3680 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 >> v1:172.21.15.91:6789/0 conn(0x7f22500d3680 legacy=0x7f22500b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2230003970 0x7f2230003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22300047c0 0x7f2230004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 0x7f2230005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250161cd0 con 0x7f22300047c0 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500d3af0 con 0x7f2230003970 2024-02-02T13:27:15.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223000ba00 con 0x7f22500b2470 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22300047c0 0x7f2230004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f2254a00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2230003970 0x7f2230003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 -- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 msgr2=0x7f2230005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500b2470 0x7f2230005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 -- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2230003970 msgr2=0x7f2230003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2230003970 0x7f2230003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.801+0000 7f224fe00640 1 -- 172.21.15.91:0/629755882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2230003520 con 0x7f22300047c0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f224fe00640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22300047c0 0x7f2230004c40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2238009f90 tx=0x7f22300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e684054232a6d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223802cc20 con 0x7f22300047c0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223800f040 con 0x7f22300047c0 2024-02-02T13:27:15.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f224f400640 1 -- 172.21.15.91:0/629755882 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238034440 con 0x7f22300047c0 2024-02-02T13:27:15.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f2257400640 1 -- 172.21.15.91:0/629755882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22300047c0 msgr2=0x7f2230004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f2257400640 1 --2- 172.21.15.91:0/629755882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22300047c0 0x7f2230004c40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2238009f90 tx=0x7f22300057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f2255400640 1 -- 172.21.15.91:0/629755882 reap_dead start 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f2257400640 1 -- 172.21.15.91:0/629755882 shutdown_connections 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.803+0000 7f2257400640 1 -- 172.21.15.91:0/629755882 >> 172.21.15.91:0/629755882 conn(0x7f2250070660 msgr2=0x7f2250070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.804+0000 7f2257400640 1 -- 172.21.15.91:0/629755882 shutdown_connections 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.804+0000 7f2257400640 1 -- 172.21.15.91:0/629755882 wait complete. 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.805+0000 7f2257400640 1 Processor -- start 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.805+0000 7f2257400640 1 -- start start 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22300047c0 0x7f22500b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 0x7f22500b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250160540 con 0x7f22501600c0 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225015f020 con 0x7f22500b5080 2024-02-02T13:27:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.806+0000 7f2257400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225015f300 con 0x7f22300047c0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54160/0 (socket says 172.21.15.91:54160) 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 -- 172.21.15.91:0/2887937556 learned_addr learned my addr 172.21.15.91:0/2887937556 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f2255400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 0x7f22500b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 -- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22300047c0 msgr2=0x7f22500b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22300047c0 0x7f22500b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 -- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 msgr2=0x7f22500b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 0x7f22500b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 -- 172.21.15.91:0/2887937556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225015f5e0 con 0x7f22501600c0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f2255400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 0x7f22500b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.807+0000 7f224fe00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f223802c5c0 tx=0x7f2238009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.808+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238034e50 con 0x7f22501600c0 2024-02-02T13:27:15.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.808+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238040450 con 0x7f22501600c0 2024-02-02T13:27:15.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.808+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238040800 con 0x7f22501600c0 2024-02-02T13:27:15.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.808+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225015f8c0 con 0x7f22501600c0 2024-02-02T13:27:15.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.808+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22500b5a40 con 0x7f22501600c0 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.809+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250067400 con 0x7f22501600c0 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.810+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2238040960 con 0x7f22501600c0 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.811+0000 7f224cc00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 0x7f222806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.811+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f22380aaff0 con 0x7f22501600c0 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.811+0000 7f2254a00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 0x7f222806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:15.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.812+0000 7f2254a00640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 0x7f222806c920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2240009980 tx=0x7f2240008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:15.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:15.815+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22380ddca0 con 0x7f22501600c0 2024-02-02T13:27:16.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.005+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f225006a050 con 0x7f22501600c0 2024-02-02T13:27:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.579+0000 7f224cc00640 1 -- 172.21.15.91:0/2887937556 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7f22380766e0 con 0x7f22501600c0 2024-02-02T13:27:16.581 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:27:16.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 msgr2=0x7f222806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 0x7f222806c920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2240009980 tx=0x7f2240008040 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 msgr2=0x7f2250157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f223802c5c0 tx=0x7f2238009750 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 shutdown_connections 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a460 0x7f222806c920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22500b5080 0x7f22500b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22501600c0 0x7f2250157700 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 --2- 172.21.15.91:0/2887937556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22300047c0 0x7f22500b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 >> 172.21.15.91:0/2887937556 conn(0x7f2250070660 msgr2=0x7f22500d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 shutdown_connections 2024-02-02T13:27:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.583+0000 7f2257400640 1 -- 172.21.15.91:0/2887937556 wait complete. 2024-02-02T13:27:16.607 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:27:16.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.817+0000 7f1f05c00640 1 Processor -- start 2024-02-02T13:27:16.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.817+0000 7f1f05c00640 1 -- start start 2024-02-02T13:27:16.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1f05c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 0x7f1f000c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1f05c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f000b0da0 0x7f1f000ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1f05c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f000cca70 con 0x7f1f00070ea0 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1f05c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000cc770 con 0x7f1f000b0da0 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1f05c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000ca570 con 0x7f1f0015a1e0 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1efe000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1f00070ea0 0x7f1f000b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45574/0 (socket says 172.21.15.91:45574) 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1efe000640 1 -- 172.21.15.91:0/386374831 learned_addr learned my addr 172.21.15.91:0/386374831 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:16.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.818+0000 7f1eff400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f000b0da0 0x7f1f000ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1efea00640 1 --2- 172.21.15.91:0/386374831 >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 0x7f1f000c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1eff400640 1 -- 172.21.15.91:0/386374831 >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 msgr2=0x7f1f000c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1eff400640 1 --2- 172.21.15.91:0/386374831 >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 0x7f1f000c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1eff400640 1 -- 172.21.15.91:0/386374831 >> v1:172.21.15.91:6789/0 conn(0x7f1f00070ea0 legacy=0x7f1f000b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1eff400640 1 -- 172.21.15.91:0/386374831 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000ca850 con 0x7f1f000b0da0 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1efd600640 1 -- 172.21.15.91:0/386374831 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375978387 0 0) 0x7f1f000cca70 con 0x7f1f00070ea0 2024-02-02T13:27:16.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.819+0000 7f1efea00640 1 --2- 172.21.15.91:0/386374831 >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 0x7f1f000c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:16.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.820+0000 7f1eff400640 1 --2- 172.21.15.91:0/386374831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f000b0da0 0x7f1f000ccf00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009d90 tx=0x7f1ef802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1380a325f6abf0a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:16.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.820+0000 7f1efd600640 1 -- 172.21.15.91:0/386374831 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef802cc60 con 0x7f1f000b0da0 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.820+0000 7f1efd600640 1 -- 172.21.15.91:0/386374831 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef802cdc0 con 0x7f1f000b0da0 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.820+0000 7f1efd600640 1 -- 172.21.15.91:0/386374831 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8034660 con 0x7f1f000b0da0 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 -- 172.21.15.91:0/386374831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f000b0da0 msgr2=0x7f1f000ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 --2- 172.21.15.91:0/386374831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f000b0da0 0x7f1f000ccf00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009d90 tx=0x7f1ef802beb0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 -- 172.21.15.91:0/386374831 shutdown_connections 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 --2- 172.21.15.91:0/386374831 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f000b0da0 0x7f1f000ccf00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 --2- 172.21.15.91:0/386374831 >> v2:172.21.15.113:3300/0 conn(0x7f1f0015a1e0 0x7f1f000c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 -- 172.21.15.91:0/386374831 >> 172.21.15.91:0/386374831 conn(0x7f1f000ceb90 msgr2=0x7f1f000cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 -- 172.21.15.91:0/386374831 shutdown_connections 2024-02-02T13:27:16.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.821+0000 7f1f05c00640 1 -- 172.21.15.91:0/386374831 wait complete. 2024-02-02T13:27:16.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.823+0000 7f1f05c00640 1 Processor -- start 2024-02-02T13:27:16.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.823+0000 7f1f05c00640 1 -- start start 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f00070ea0 0x7f1f0015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f000b0da0 0x7f1f001589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0015be90 con 0x7f1f0015a1e0 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00158f10 con 0x7f1f00070ea0 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1f05c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00158540 con 0x7f1f000b0da0 2024-02-02T13:27:16.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54182/0 (socket says 172.21.15.91:54182) 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 -- 172.21.15.91:0/4005827119 learned_addr learned my addr 172.21.15.91:0/4005827119 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1efea00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f00070ea0 0x7f1f0015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 -- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f000b0da0 msgr2=0x7f1f001589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f000b0da0 0x7f1f001589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 -- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f00070ea0 msgr2=0x7f1f0015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f00070ea0 0x7f1f0015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.824+0000 7f1eff400640 1 -- 172.21.15.91:0/4005827119 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8009a70 con 0x7f1f0015a1e0 2024-02-02T13:27:16.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1eff400640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009eb0 tx=0x7f1ef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8040650 con 0x7f1f0015a1e0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef80407d0 con 0x7f1f0015a1e0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8040b80 con 0x7f1f0015a1e0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0014e880 con 0x7f1f0015a1e0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.825+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f00157c10 con 0x7f1f0015a1e0 2024-02-02T13:27:16.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.826+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00066cf0 con 0x7f1f0015a1e0 2024-02-02T13:27:16.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.827+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1ef8040ce0 con 0x7f1f0015a1e0 2024-02-02T13:27:16.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.828+0000 7f1ef6a00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 0x7f1ed806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:16.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.828+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1ef80ab530 con 0x7f1f0015a1e0 2024-02-02T13:27:16.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.828+0000 7f1efea00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 0x7f1ed806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:16.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.829+0000 7f1efea00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 0x7f1ed806c8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f0015ad00 tx=0x7f1ef0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:16.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:16.833+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1ef8076e00 con 0x7f1f0015a1e0 2024-02-02T13:27:17.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.023+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1f00158680 con 0x7f1f0015a1e0 2024-02-02T13:27:17.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.032+0000 7f1ef6a00640 1 -- 172.21.15.91:0/4005827119 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ef807a420 con 0x7f1f0015a1e0 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.034+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 msgr2=0x7f1ed806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.034+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 0x7f1ed806c8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f0015ad00 tx=0x7f1ef0008040 comp rx=0 tx=0).stop 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.034+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 msgr2=0x7f1f0014e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.034+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009eb0 tx=0x7f1ef8004a40 comp rx=0 tx=0).stop 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 shutdown_connections 2024-02-02T13:27:17.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ed806a430 0x7f1ed806c8f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f000b0da0 0x7f1f001589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0015a1e0 0x7f1f0014e340 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 --2- 172.21.15.91:0/4005827119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f00070ea0 0x7f1f0015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 >> 172.21.15.91:0/4005827119 conn(0x7f1f000ceb90 msgr2=0x7f1f000ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 shutdown_connections 2024-02-02T13:27:17.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.035+0000 7f1f05c00640 1 -- 172.21.15.91:0/4005827119 wait complete. 2024-02-02T13:27:17.055 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-02-02T13:27:17.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.256+0000 7f2cff000640 1 Processor -- start 2024-02-02T13:27:17.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.256+0000 7f2cff000640 1 -- start start 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.257+0000 7f2cff000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 0x7f2cf80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.257+0000 7f2cff000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf80b0540 con 0x7f2cf806ce50 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.257+0000 7f2cff000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf815b2d0 con 0x7f2cf80adf20 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.257+0000 7f2cff000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80ca410 con 0x7f2cf815aef0 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cfd200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2cf806ce50 0x7f2cf8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45586/0 (socket says 172.21.15.91:45586) 2024-02-02T13:27:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cfd200640 1 -- 172.21.15.91:0/2500647986 learned_addr learned my addr 172.21.15.91:0/2500647986 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 0x7f2cf80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575177286 0 0) 0x7f2cf80b0540 con 0x7f2cf806ce50 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd8003520 con 0x7f2cf806ce50 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574654935 0 0) 0x7f2cd8003520 con 0x7f2cf806ce50 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 msgr2=0x7f2cf80b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 0x7f2cf80b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 >> v1:172.21.15.113:6789/0 conn(0x7f2cf80adf20 legacy=0x7f2cf806b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80b07e0 con 0x7f2cf806ce50 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.258+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2ce0001670 con 0x7f2cf806ce50 2024-02-02T13:27:17.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 >> v1:172.21.15.91:6789/0 conn(0x7f2cf806ce50 legacy=0x7f2cf8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cd80039a0 0x7f2cd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80047c0 0x7f2cd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 0x7f2cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80ca410 con 0x7f2cd80039a0 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd800ba60 con 0x7f2cd80047c0 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd800bc20 con 0x7f2cf815aef0 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cd80039a0 0x7f2cd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 -- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 msgr2=0x7f2cd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf815aef0 0x7f2cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 -- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80047c0 msgr2=0x7f2cd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80047c0 0x7f2cd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.259+0000 7f2cfdc00640 1 -- 172.21.15.91:0/2500647986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8003520 con 0x7f2cd80039a0 2024-02-02T13:27:17.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.260+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cd80039a0 0x7f2cd8003dc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009a90 tx=0x7f2cd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c607eac34bdfd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.260+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce802c6b0 con 0x7f2cd80039a0 2024-02-02T13:27:17.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce802c810 con 0x7f2cd80039a0 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cf7e00640 1 -- 172.21.15.91:0/2500647986 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce802cba0 con 0x7f2cd80039a0 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 -- 172.21.15.91:0/2500647986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cd80039a0 msgr2=0x7f2cd8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 --2- 172.21.15.91:0/2500647986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cd80039a0 0x7f2cd8003dc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009a90 tx=0x7f2cd80057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cfe600640 1 -- 172.21.15.91:0/2500647986 reap_dead start 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 -- 172.21.15.91:0/2500647986 shutdown_connections 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 -- 172.21.15.91:0/2500647986 >> 172.21.15.91:0/2500647986 conn(0x7f2cf80cb4a0 msgr2=0x7f2cf80cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 -- 172.21.15.91:0/2500647986 shutdown_connections 2024-02-02T13:27:17.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.261+0000 7f2cff000640 1 -- 172.21.15.91:0/2500647986 wait complete. 2024-02-02T13:27:17.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.263+0000 7f2cff000640 1 Processor -- start 2024-02-02T13:27:17.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.263+0000 7f2cff000640 1 -- start start 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 0x7f2cf8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf81477d0 0x7f2cf8147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80cc5d0 con 0x7f2cf80cd300 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c87b0 con 0x7f2cd80039a0 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cff000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80cc820 con 0x7f2cf81477d0 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cfd200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cfd200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54206/0 (socket says 172.21.15.91:54206) 2024-02-02T13:27:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.264+0000 7f2cfd200640 1 -- 172.21.15.91:0/3439920308 learned_addr learned my addr 172.21.15.91:0/3439920308 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 0x7f2cf8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 -- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf81477d0 msgr2=0x7f2cf8147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfe600640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf81477d0 0x7f2cf8147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf81477d0 0x7f2cf8147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 -- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 msgr2=0x7f2cf8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 0x7f2cf8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 -- 172.21.15.91:0/3439920308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80ccb00 con 0x7f2cf80cd300 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 0x7f2cf8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cfd200640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c770 tx=0x7f2ce000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00074e0 con 0x7f2cf80cd300 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0007640 con 0x7f2cf80cd300 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004a20 con 0x7f2cf80cd300 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf80ccde0 con 0x7f2cf80cd300 2024-02-02T13:27:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.265+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf80c8360 con 0x7f2cf80cd300 2024-02-02T13:27:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.266+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf80c8580 con 0x7f2cf80cd300 2024-02-02T13:27:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.267+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2ce0004b80 con 0x7f2cf80cd300 2024-02-02T13:27:17.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.269+0000 7f2cf5600640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 0x7f2cd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.269+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ce0087a70 con 0x7f2cf80cd300 2024-02-02T13:27:17.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.269+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 0x7f2cd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.269+0000 7f2cfdc00640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 0x7f2cd006c8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8036000 tx=0x7f2ce8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.273+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2ce0053350 con 0x7f2cf80cd300 2024-02-02T13:27:17.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.545+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f2cf80ccfd0 con 0x7f2cf80cd300 2024-02-02T13:27:17.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.582+0000 7f2cf5600640 1 -- 172.21.15.91:0/3439920308 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v164) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2ce00180d0 con 0x7f2cf80cd300 2024-02-02T13:27:17.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.584+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 msgr2=0x7f2cd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.584+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 0x7f2cd006c8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8036000 tx=0x7f2ce8030000 comp rx=0 tx=0).stop 2024-02-02T13:27:17.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.584+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 msgr2=0x7f2cf8071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.584+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c770 tx=0x7f2ce000cc30 comp rx=0 tx=0).stop 2024-02-02T13:27:17.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 shutdown_connections 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2cd006a430 0x7f2cd006c8f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cf81477d0 0x7f2cf8147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cf80cd300 0x7f2cf8071070 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 --2- 172.21.15.91:0/3439920308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cd80039a0 0x7f2cf8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 >> 172.21.15.91:0/3439920308 conn(0x7f2cf80cb4a0 msgr2=0x7f2cf80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 shutdown_connections 2024-02-02T13:27:17.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.585+0000 7f2cff000640 1 -- 172.21.15.91:0/3439920308 wait complete. 2024-02-02T13:27:17.603 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-02-02T13:27:17.604 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-02-02T13:27:17.604 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-02-02T13:27:17.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.807+0000 7f7554200640 1 Processor -- start 2024-02-02T13:27:17.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.808+0000 7f7554200640 1 -- start start 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7554200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 0x7f754c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7554200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f754c06f4f0 0x7f754c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7554200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c070f80 con 0x7f754c0b0c40 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7554200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0b10d0 con 0x7f754c06f4f0 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7554200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c152370 con 0x7f754c151f90 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7552200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f754c0b0c40 0x7f754c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45598/0 (socket says 172.21.15.91:45598) 2024-02-02T13:27:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.809+0000 7f7552200640 1 -- 172.21.15.91:0/2436534210 learned_addr learned my addr 172.21.15.91:0/2436534210 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:17.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f7551800640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 0x7f754c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f7550e00640 1 --2- 172.21.15.91:0/2436534210 >> v2:172.21.15.113:3300/0 conn(0x7f754c06f4f0 0x7f754c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778698585 0 0) 0x7f754c070f80 con 0x7f754c0b0c40 2024-02-02T13:27:17.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c003520 con 0x7f754c0b0c40 2024-02-02T13:27:17.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739029507 0 0) 0x7f752c003520 con 0x7f754c0b0c40 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 msgr2=0x7f754c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 0x7f754c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 >> v2:172.21.15.113:3300/0 conn(0x7f754c06f4f0 msgr2=0x7f754c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 --2- 172.21.15.91:0/2436534210 >> v2:172.21.15.113:3300/0 conn(0x7f754c06f4f0 0x7f754c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0cb010 con 0x7f754c0b0c40 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f7544001670 con 0x7f754c0b0c40 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 >> v1:172.21.15.91:6789/0 conn(0x7f754c0b0c40 legacy=0x7f754c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.810+0000 7f754be00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f752c003970 0x7f752c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f754be00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f754be00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 0x7f752c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b10d0 con 0x7f752c0047c0 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c152370 con 0x7f752c003970 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00b980 con 0x7f754c151f90 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 -- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 msgr2=0x7f752c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c151f90 0x7f752c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7551800640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f752c003970 0x7f752c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 -- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f752c003970 msgr2=0x7f752c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:17.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f752c003970 0x7f752c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.811+0000 7f7550e00640 1 -- 172.21.15.91:0/2436534210 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c003520 con 0x7f752c0047c0 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.812+0000 7f7550e00640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7534009fc0 tx=0x7f752c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3d39747e1ba4fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.812+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753402cc40 con 0x7f752c0047c0 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.812+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753400f040 con 0x7f752c0047c0 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.812+0000 7f754be00640 1 -- 172.21.15.91:0/2436534210 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534034440 con 0x7f752c0047c0 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 -- 172.21.15.91:0/2436534210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 msgr2=0x7f752c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 --2- 172.21.15.91:0/2436534210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7534009fc0 tx=0x7f752c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7552200640 1 -- 172.21.15.91:0/2436534210 reap_dead start 2024-02-02T13:27:17.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 -- 172.21.15.91:0/2436534210 shutdown_connections 2024-02-02T13:27:17.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 -- 172.21.15.91:0/2436534210 >> 172.21.15.91:0/2436534210 conn(0x7f754c0cec80 msgr2=0x7f754c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:17.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 -- 172.21.15.91:0/2436534210 shutdown_connections 2024-02-02T13:27:17.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.813+0000 7f7554200640 1 -- 172.21.15.91:0/2436534210 wait complete. 2024-02-02T13:27:17.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.815+0000 7f7554200640 1 Processor -- start 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.815+0000 7f7554200640 1 -- start start 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c0c5cc0 0x7f754c150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f754c0c2720 0x7f754c0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c6db0 con 0x7f752c0047c0 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c6230 con 0x7f754c0c2720 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7554200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c6510 con 0x7f754c0c5cc0 2024-02-02T13:27:17.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7551800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7551800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54236/0 (socket says 172.21.15.91:54236) 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.816+0000 7f7551800640 1 -- 172.21.15.91:0/2690021390 learned_addr learned my addr 172.21.15.91:0/2690021390 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 -- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c0c5cc0 msgr2=0x7f754c150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c0c5cc0 0x7f754c150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 -- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f754c0c2720 msgr2=0x7f754c0c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f754c0c2720 0x7f754c0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 -- 172.21.15.91:0/2690021390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0c67f0 con 0x7f752c0047c0 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7551800640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f753c010860 tx=0x7f753c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c009610 con 0x7f752c0047c0 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c00eb60 con 0x7f752c0047c0 2024-02-02T13:27:17.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c01c400 con 0x7f752c0047c0 2024-02-02T13:27:17.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c0c6ad0 con 0x7f752c0047c0 2024-02-02T13:27:17.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.817+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c150cc0 con 0x7f752c0047c0 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.818+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c066cf0 con 0x7f752c0047c0 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.819+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f753c01c560 con 0x7f752c0047c0 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.820+0000 7f7549600640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 0x7f752406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.820+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f753c088a50 con 0x7f752c0047c0 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.820+0000 7f7550e00640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 0x7f752406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:17.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.821+0000 7f7550e00640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 0x7f752406c8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7534036000 tx=0x7f753402e070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:17.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:17.825+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f753c054480 con 0x7f752c0047c0 2024-02-02T13:27:18.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.085+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f754c06c7b0 con 0x7f752c0047c0 2024-02-02T13:27:18.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.585+0000 7f7549600640 1 -- 172.21.15.91:0/2690021390 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v165) v1 ==== 137+0+0 (secure 0 0 0) 0x7f753c057aa0 con 0x7f752c0047c0 2024-02-02T13:27:18.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.589+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 msgr2=0x7f752406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:18.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.589+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 0x7f752406c8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7534036000 tx=0x7f753402e070 comp rx=0 tx=0).stop 2024-02-02T13:27:18.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.589+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 msgr2=0x7f754c0c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.589+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f753c010860 tx=0x7f753c010d20 comp rx=0 tx=0).stop 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 shutdown_connections 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f752406a430 0x7f752406c8f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f754c0c5cc0 0x7f754c150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f754c0c2720 0x7f754c0c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 --2- 172.21.15.91:0/2690021390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f752c0047c0 0x7f754c0c8490 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 >> 172.21.15.91:0/2690021390 conn(0x7f754c0cec80 msgr2=0x7f754c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 shutdown_connections 2024-02-02T13:27:18.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:18.590+0000 7f7554200640 1 -- 172.21.15.91:0/2690021390 wait complete. 2024-02-02T13:27:18.611 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-02-02T13:27:18.831 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_objectstore":"memstore"} 2024-02-02T13:27:18.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-02-02T13:27:18.832 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:27:19.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.044+0000 7fa1f7200640 1 Processor -- start 2024-02-02T13:27:19.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.044+0000 7fa1f7200640 1 -- start start 2024-02-02T13:27:19.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1f7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 0x7fa1f00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1f7200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00cc7c0 con 0x7fa1f015a010 2024-02-02T13:27:19.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1f7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00ce930 con 0x7fa1f00ce130 2024-02-02T13:27:19.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1f7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00cd640 con 0x7fa1f00b0e10 2024-02-02T13:27:19.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1efe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa1f015a010 0x7fa1f00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45600/0 (socket says 172.21.15.91:45600) 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1efe00640 1 -- 172.21.15.91:0/3780624087 learned_addr learned my addr 172.21.15.91:0/3780624087 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1f5200640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 0x7fa1f00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211238743 0 0) 0x7fa1f00cc7c0 con 0x7fa1f015a010 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.046+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d0003520 con 0x7fa1f015a010 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216185752 0 0) 0x7fa1d0003520 con 0x7fa1f015a010 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 msgr2=0x7fa1f00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 0x7fa1f00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 >> v1:172.21.15.113:6789/0 conn(0x7fa1f00ce130 legacy=0x7fa1f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f006c890 con 0x7fa1f015a010 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa1e0001670 con 0x7fa1f015a010 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 >> v1:172.21.15.91:6789/0 conn(0x7fa1f015a010 legacy=0x7fa1f00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00039a0 0x7fa1d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 0x7fa1d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00cd640 con 0x7fa1d00047c0 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d000ba90 con 0x7fa1d00039a0 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d000bc50 con 0x7fa1f00b0e10 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.047+0000 7fa1ef400640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1efe00640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00039a0 0x7fa1d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1ef400640 1 -- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 msgr2=0x7fa1d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1ef400640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b0e10 0x7fa1d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1ef400640 1 -- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00039a0 msgr2=0x7fa1d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1ef400640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00039a0 0x7fa1d0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.048+0000 7fa1ef400640 1 -- 172.21.15.91:0/3780624087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0003520 con 0x7fa1d00047c0 2024-02-02T13:27:19.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.049+0000 7fa1ef400640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1d0004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8001720 tx=0x7fa1d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b540d44e9178939 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.049+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80052d0 con 0x7fa1d00047c0 2024-02-02T13:27:19.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.049+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d802e050 con 0x7fa1d00047c0 2024-02-02T13:27:19.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.049+0000 7fa1eea00640 1 -- 172.21.15.91:0/3780624087 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80035f0 con 0x7fa1d00047c0 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 -- 172.21.15.91:0/3780624087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 msgr2=0x7fa1d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 --2- 172.21.15.91:0/3780624087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1d0004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8001720 tx=0x7fa1d00057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f5200640 1 -- 172.21.15.91:0/3780624087 reap_dead start 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 -- 172.21.15.91:0/3780624087 shutdown_connections 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 -- 172.21.15.91:0/3780624087 >> 172.21.15.91:0/3780624087 conn(0x7fa1f00cec80 msgr2=0x7fa1f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 -- 172.21.15.91:0/3780624087 shutdown_connections 2024-02-02T13:27:19.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.050+0000 7fa1f7200640 1 -- 172.21.15.91:0/3780624087 wait complete. 2024-02-02T13:27:19.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.052+0000 7fa1f7200640 1 Processor -- start 2024-02-02T13:27:19.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.052+0000 7fa1f7200640 1 -- start start 2024-02-02T13:27:19.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f015d090 0x7fa1f00c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f01366b0 0x7fa1f015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c0850 con 0x7fa1d00047c0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f014d2b0 con 0x7fa1f01366b0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1f7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c5b30 con 0x7fa1f015d090 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54264/0 (socket says 172.21.15.91:54264) 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 -- 172.21.15.91:0/2719257997 learned_addr learned my addr 172.21.15.91:0/2719257997 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 -- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f015d090 msgr2=0x7fa1f00c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f015d090 0x7fa1f00c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 -- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f01366b0 msgr2=0x7fa1f015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f01366b0 0x7fa1f015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 -- 172.21.15.91:0/2719257997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00bffb0 con 0x7fa1d00047c0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.053+0000 7fa1efe00640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d7d0 tx=0x7fa1e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.054+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00072d0 con 0x7fa1d00047c0 2024-02-02T13:27:19.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.054+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e0007430 con 0x7fa1d00047c0 2024-02-02T13:27:19.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.054+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00047e0 con 0x7fa1d00047c0 2024-02-02T13:27:19.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.054+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f00c0290 con 0x7fa1d00047c0 2024-02-02T13:27:19.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.054+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f00c0e60 con 0x7fa1d00047c0 2024-02-02T13:27:19.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.055+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f00c0cc0 con 0x7fa1d00047c0 2024-02-02T13:27:19.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.056+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa1e0004940 con 0x7fa1d00047c0 2024-02-02T13:27:19.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.057+0000 7fa1e7e00640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 0x7fa1c806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.057+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1e0088090 con 0x7fa1d00047c0 2024-02-02T13:27:19.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.057+0000 7fa1ef400640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 0x7fa1c806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.058+0000 7fa1ef400640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 0x7fa1c806c920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8033000 tx=0x7fa1d802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.061+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa1f00c0cc0 con 0x7fa1d00047c0 2024-02-02T13:27:19.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.252+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa1f00c0cc0 con 0x7fa1d00047c0 2024-02-02T13:27:19.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.261+0000 7fa1e7e00640 1 -- 172.21.15.91:0/2719257997 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1f00c0cc0 con 0x7fa1d00047c0 2024-02-02T13:27:19.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.263+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 msgr2=0x7fa1c806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.263+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 0x7fa1c806c920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8033000 tx=0x7fa1d802f000 comp rx=0 tx=0).stop 2024-02-02T13:27:19.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.263+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 msgr2=0x7fa1f00bf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.263+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d7d0 tx=0x7fa1e000dc90 comp rx=0 tx=0).stop 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 shutdown_connections 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa1c806a460 0x7fa1c806c920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f015d090 0x7fa1f00c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f01366b0 0x7fa1f015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 --2- 172.21.15.91:0/2719257997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1d00047c0 0x7fa1f00bf860 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 >> 172.21.15.91:0/2719257997 conn(0x7fa1f00cec80 msgr2=0x7fa1f015b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 shutdown_connections 2024-02-02T13:27:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.264+0000 7fa1f7200640 1 -- 172.21.15.91:0/2719257997 wait complete. 2024-02-02T13:27:19.283 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:27:19.283 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:27:19.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:27:19.500 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:27:19.500 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:19.500 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:19.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:27:19.558 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:19.559 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:19.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:27:19.615 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:19.615 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:19.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:27:19.672 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:19.673 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:19.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:27:19.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:27:19.728 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:19.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.938+0000 7fc94a800640 1 Processor -- start 2024-02-02T13:27:19.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.938+0000 7fc94a800640 1 -- start start 2024-02-02T13:27:19.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.939+0000 7fc94a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.939+0000 7fc94a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9440b0700 0x7fc9440b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.939+0000 7fc94a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440cc7c0 con 0x7fc9440ce130 2024-02-02T13:27:19.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.939+0000 7fc94a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9440b10d0 con 0x7fc9440b0700 2024-02-02T13:27:19.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.939+0000 7fc94a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440ce930 con 0x7fc94415a010 2024-02-02T13:27:19.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.940+0000 7fc943400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.940+0000 7fc943400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:42564/0 (socket says 172.21.15.91:42564) 2024-02-02T13:27:19.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.940+0000 7fc943400640 1 -- 172.21.15.91:0/2281765551 learned_addr learned my addr 172.21.15.91:0/2281765551 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:19.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.940+0000 7fc943e00640 1 --2- 172.21.15.91:0/2281765551 >> v2:172.21.15.113:3300/0 conn(0x7fc9440b0700 0x7fc9440b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.941+0000 7fc943400640 1 -- 172.21.15.91:0/2281765551 >> v2:172.21.15.113:3300/0 conn(0x7fc9440b0700 msgr2=0x7fc9440b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:19.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.941+0000 7fc943400640 1 --2- 172.21.15.91:0/2281765551 >> v2:172.21.15.113:3300/0 conn(0x7fc9440b0700 0x7fc9440b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.941+0000 7fc943400640 1 -- 172.21.15.91:0/2281765551 >> v1:172.21.15.113:6789/0 conn(0x7fc9440ce130 legacy=0x7fc9440ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.941+0000 7fc943400640 1 -- 172.21.15.91:0/2281765551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440cd640 con 0x7fc94415a010 2024-02-02T13:27:19.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.942+0000 7fc943400640 1 --2- 172.21.15.91:0/2281765551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc93400bb10 tx=0x7fc93402e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292159d7f4a1c51 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.943+0000 7fc942a00640 1 -- 172.21.15.91:0/2281765551 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934036440 con 0x7fc94415a010 2024-02-02T13:27:19.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.943+0000 7fc942a00640 1 -- 172.21.15.91:0/2281765551 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9340365a0 con 0x7fc94415a010 2024-02-02T13:27:19.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.943+0000 7fc942a00640 1 -- 172.21.15.91:0/2281765551 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934036930 con 0x7fc94415a010 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 -- 172.21.15.91:0/2281765551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 msgr2=0x7fc9440c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 --2- 172.21.15.91:0/2281765551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc93400bb10 tx=0x7fc93402e150 comp rx=0 tx=0).stop 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 -- 172.21.15.91:0/2281765551 shutdown_connections 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 --2- 172.21.15.91:0/2281765551 >> v2:172.21.15.113:3300/0 conn(0x7fc9440b0700 0x7fc9440b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 --2- 172.21.15.91:0/2281765551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94415a010 0x7fc9440c9670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 -- 172.21.15.91:0/2281765551 >> 172.21.15.91:0/2281765551 conn(0x7fc9440cec80 msgr2=0x7fc9440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 -- 172.21.15.91:0/2281765551 shutdown_connections 2024-02-02T13:27:19.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.944+0000 7fc94a800640 1 -- 172.21.15.91:0/2281765551 wait complete. 2024-02-02T13:27:19.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.946+0000 7fc94a800640 1 Processor -- start 2024-02-02T13:27:19.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.946+0000 7fc94a800640 1 -- start start 2024-02-02T13:27:19.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9440ce130 0x7fc944159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 0x7fc94415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440c5cc0 con 0x7fc9440b0700 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440c7210 con 0x7fc94415a010 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc94a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440c5f10 con 0x7fc9440ce130 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43770/0 (socket says 172.21.15.91:43770) 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 -- 172.21.15.91:0/1505071954 learned_addr learned my addr 172.21.15.91:0/1505071954 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc93be00640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9440ce130 0x7fc944159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943e00640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 0x7fc94415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 -- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9440ce130 msgr2=0x7fc944159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9440ce130 0x7fc944159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.947+0000 7fc943400640 1 -- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 msgr2=0x7fc94415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc943400640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 0x7fc94415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc943400640 1 -- 172.21.15.91:0/1505071954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93400b770 con 0x7fc9440b0700 2024-02-02T13:27:19.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc943e00640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 0x7fc94415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:19.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc943400640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc93400b2c0 tx=0x7fc934004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934036440 con 0x7fc9440b0700 2024-02-02T13:27:19.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934034ba0 con 0x7fc9440b0700 2024-02-02T13:27:19.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934036e20 con 0x7fc9440b0700 2024-02-02T13:27:19.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440c61f0 con 0x7fc9440b0700 2024-02-02T13:27:19.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.948+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440c67b0 con 0x7fc9440b0700 2024-02-02T13:27:19.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.949+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94415e7e0 con 0x7fc9440b0700 2024-02-02T13:27:19.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.950+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc934041480 con 0x7fc9440b0700 2024-02-02T13:27:19.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.951+0000 7fc93b400640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 0x7fc91c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:19.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.951+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc9340ab6e0 con 0x7fc9440b0700 2024-02-02T13:27:19.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.951+0000 7fc93be00640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 0x7fc91c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:19.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.952+0000 7fc93be00640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 0x7fc91c06c8f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9440b48e0 tx=0x7fc92c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:19.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:19.956+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc9340770e0 con 0x7fc9440b0700 2024-02-02T13:27:20.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.187+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9440c69a0 con 0x7fc9440b0700 2024-02-02T13:27:20.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.188+0000 7fc93b400640 1 -- 172.21.15.91:0/1505071954 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc93403bde0 con 0x7fc9440b0700 2024-02-02T13:27:20.190 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:20.190 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2656543698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2656543698},{"type":"v1","addr":"172.21.15.113:6835","nonce":2656543698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/294703708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":294703708},{"type":"v1","addr":"172.21.15.113:6834","nonce":294703708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2591108807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2591108807},{"type":"v1","addr":"172.21.15.91:6835","nonce":2591108807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/773257891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":773257891},{"type":"v1","addr":"172.21.15.91:6837","nonce":773257891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-02-02T13:27:20.190 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 26 2024-02-02T13:27:20.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 msgr2=0x7fc91c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:20.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 0x7fc91c06c8f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9440b48e0 tx=0x7fc92c009040 comp rx=0 tx=0).stop 2024-02-02T13:27:20.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 msgr2=0x7fc9440c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:20.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc93400b2c0 tx=0x7fc934004730 comp rx=0 tx=0).stop 2024-02-02T13:27:20.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 shutdown_connections 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.191+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc91c06a430 0x7fc91c06c8f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9440ce130 0x7fc944159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc94415a010 0x7fc94415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 --2- 172.21.15.91:0/1505071954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9440b0700 0x7fc9440c6cd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 >> 172.21.15.91:0/1505071954 conn(0x7fc9440cec80 msgr2=0x7fc9440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 shutdown_connections 2024-02-02T13:27:20.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.192+0000 7fc94a800640 1 -- 172.21.15.91:0/1505071954 wait complete. 2024-02-02T13:27:20.213 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:27:20.213 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:27:20.213 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:27:20.213 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:27:20.214 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:27:20.214 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:27:20.214 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:27:20.214 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:27:20.214 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:27:20.214 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:27:20.214 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:27:20.214 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:27:20.215 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:27:20.215 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:27:20.215 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:27:20.215 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:27:20.238 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:27:20.239 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:27:20.241 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:27:20.241 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:27:20.242 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:27:20.242 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:27:20.244 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:27:20.244 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:27:20.289 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:27:20.290 DEBUG:teuthology.parallel:result is None 2024-02-02T13:27:20.292 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:27:20.292 DEBUG:teuthology.parallel:result is None 2024-02-02T13:27:20.295 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:27:20.295 DEBUG:teuthology.parallel:result is None 2024-02-02T13:27:20.298 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:27:20.298 DEBUG:teuthology.parallel:result is None 2024-02-02T13:27:20.298 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:27:20.446 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:27:20.449 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:27:20.449 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:27:20.450 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:27:20.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.484+0000 7f2dd7a00640 1 Processor -- start 2024-02-02T13:27:20.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.484+0000 7f2dd7a00640 1 -- start start 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2dd015b130 0x7f2dd00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dd00cf470 0x7f2dd00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd00cfe40 con 0x7f2dd015b130 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd00cf1b0 con 0x7f2dd00cf470 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.485+0000 7f2dd7a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00b0510 con 0x7f2dd00cd4a0 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2dd015b130 0x7f2dd00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2dd015b130 0x7f2dd00cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43824/0 (socket says 172.21.15.91:43824) 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 -- 172.21.15.91:0/747001163 learned_addr learned my addr 172.21.15.91:0/747001163 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dc7e00640 1 --2- 172.21.15.91:0/747001163 >> v2:172.21.15.113:3300/0 conn(0x7f2dd00cf470 0x7f2dd00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5a00640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 -- 172.21.15.91:0/747001163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 msgr2=0x7f2dd00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 -- 172.21.15.91:0/747001163 >> v2:172.21.15.113:3300/0 conn(0x7f2dd00cf470 msgr2=0x7f2dd00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:20.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 --2- 172.21.15.91:0/747001163 >> v2:172.21.15.113:3300/0 conn(0x7f2dd00cf470 0x7f2dd00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.486+0000 7f2dd5000640 1 -- 172.21.15.91:0/747001163 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd00d1980 con 0x7f2dd015b130 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.487+0000 7f2dd5000640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd015b130 0x7f2dd00cec70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2dd009a570 tx=0x7f2dc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19b5119b169376c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dcfe00640 1 -- 172.21.15.91:0/747001163 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0031870 con 0x7f2dd015b130 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dcfe00640 1 -- 172.21.15.91:0/747001163 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00319d0 con 0x7f2dd015b130 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dcfe00640 1 -- 172.21.15.91:0/747001163 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0031d60 con 0x7f2dd015b130 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 -- 172.21.15.91:0/747001163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd015b130 msgr2=0x7f2dd00cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd015b130 0x7f2dd00cec70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2dd009a570 tx=0x7f2dc002b910 comp rx=0 tx=0).stop 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 -- 172.21.15.91:0/747001163 shutdown_connections 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/747001163 >> v2:172.21.15.113:3300/0 conn(0x7f2dd00cf470 0x7f2dd00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/747001163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd015b130 0x7f2dd00cec70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.488+0000 7f2dd7a00640 1 -- 172.21.15.91:0/747001163 >> 172.21.15.91:0/747001163 conn(0x7f2dd009a160 msgr2=0x7f2dd00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.489+0000 7f2dd7a00640 1 -- 172.21.15.91:0/747001163 shutdown_connections 2024-02-02T13:27:20.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.489+0000 7f2dd7a00640 1 -- 172.21.15.91:0/747001163 wait complete. 2024-02-02T13:27:20.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.490+0000 7f2dd7a00640 1 Processor -- start 2024-02-02T13:27:20.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 -- start start 2024-02-02T13:27:20.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dd015b130 0x7f2dd00c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0157910 con 0x7f2dd00cf470 2024-02-02T13:27:20.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0157da0 con 0x7f2dd015b130 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.491+0000 7f2dd7a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0156d00 con 0x7f2dd00cd4a0 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43832/0 (socket says 172.21.15.91:43832) 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 -- 172.21.15.91:0/2538032691 learned_addr learned my addr 172.21.15.91:0/2538032691 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dd5000640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dd5a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dd015b130 0x7f2dd00c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 -- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 msgr2=0x7f2dd0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 -- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dd015b130 msgr2=0x7f2dd00c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dd015b130 0x7f2dd00c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dc7e00640 1 -- 172.21.15.91:0/2538032691 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00094d0 con 0x7f2dd00cf470 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.492+0000 7f2dd5000640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dc7e00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2db800e870 tx=0x7f2db800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db80094f0 con 0x7f2dd00cf470 2024-02-02T13:27:20.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800c940 con 0x7f2dd00cf470 2024-02-02T13:27:20.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800ccf0 con 0x7f2dd00cf470 2024-02-02T13:27:20.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd0156fe0 con 0x7f2dd00cf470 2024-02-02T13:27:20.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.493+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd01575a0 con 0x7f2dd00cf470 2024-02-02T13:27:20.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.494+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd0157a80 con 0x7f2dd00cf470 2024-02-02T13:27:20.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.495+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2db8013470 con 0x7f2dd00cf470 2024-02-02T13:27:20.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.496+0000 7f2dcd600640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 0x7f2da806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:20.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.496+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2db8088730 con 0x7f2dd00cf470 2024-02-02T13:27:20.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.496+0000 7f2dd5000640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 0x7f2da806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:20.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.497+0000 7f2dd5000640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 0x7f2da806c8f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036000 tx=0x7f2dc003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:20.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.501+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2db8054030 con 0x7f2dd00cf470 2024-02-02T13:27:20.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:20.708+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2dd0066cf0 con 0x7f2dd00cf470 2024-02-02T13:27:21.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.613+0000 7f2dcd600640 1 -- 172.21.15.91:0/2538032691 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2db8057650 con 0x7f2dd00cf470 2024-02-02T13:27:21.615 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:27:21.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 msgr2=0x7f2da806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 0x7f2da806c8f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036000 tx=0x7f2dc003c040 comp rx=0 tx=0).stop 2024-02-02T13:27:21.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 msgr2=0x7f2dd00c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2db800e870 tx=0x7f2db800ed30 comp rx=0 tx=0).stop 2024-02-02T13:27:21.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 shutdown_connections 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2da806a430 0x7f2da806c8f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dd015b130 0x7f2dd00c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2dd00cf470 0x7f2dd00c87e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 --2- 172.21.15.91:0/2538032691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dd00cd4a0 0x7f2dd0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.617+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 >> 172.21.15.91:0/2538032691 conn(0x7f2dd009a160 msgr2=0x7f2dd01e8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.618+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 shutdown_connections 2024-02-02T13:27:21.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.618+0000 7f2dd7a00640 1 -- 172.21.15.91:0/2538032691 wait complete. 2024-02-02T13:27:21.640 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:27:21.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.845+0000 7fee74a00640 1 Processor -- start 2024-02-02T13:27:21.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.845+0000 7fee74a00640 1 -- start start 2024-02-02T13:27:21.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee74a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee6c0cb580 0x7fee6c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee74a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fee6c0ac960 0x7fee6c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee74a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c0c94a0 con 0x7fee6c15d3a0 2024-02-02T13:27:21.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee74a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c0ad6e0 con 0x7fee6c0ac960 2024-02-02T13:27:21.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee74a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c0c96f0 con 0x7fee6c0cb580 2024-02-02T13:27:21.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee71600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fee6c15d3a0 0x7fee6c0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45292/0 (socket says 172.21.15.91:45292) 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee71600640 1 -- 172.21.15.91:0/3405692136 learned_addr learned my addr 172.21.15.91:0/3405692136 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.847+0000 7fee72a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fee6c0ac960 0x7fee6c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee72000640 1 --2- 172.21.15.91:0/3405692136 >> v2:172.21.15.113:3300/0 conn(0x7fee6c0cb580 0x7fee6c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee72a00640 1 -- 172.21.15.91:0/3405692136 >> v2:172.21.15.113:3300/0 conn(0x7fee6c0cb580 msgr2=0x7fee6c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee72a00640 1 --2- 172.21.15.91:0/3405692136 >> v2:172.21.15.113:3300/0 conn(0x7fee6c0cb580 0x7fee6c0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee72a00640 1 -- 172.21.15.91:0/3405692136 >> v1:172.21.15.91:6789/0 conn(0x7fee6c15d3a0 legacy=0x7fee6c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee72a00640 1 -- 172.21.15.91:0/3405692136 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0c99d0 con 0x7fee6c0ac960 2024-02-02T13:27:21.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.848+0000 7fee70c00640 1 -- 172.21.15.91:0/3405692136 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515510435 0 0) 0x7fee6c0c94a0 con 0x7fee6c15d3a0 2024-02-02T13:27:21.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.849+0000 7fee72a00640 1 --2- 172.21.15.91:0/3405692136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0ac960 0x7fee6c0acdf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0cc320 tx=0x7fee6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c9fddd494f36b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:21.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.849+0000 7fee70c00640 1 -- 172.21.15.91:0/3405692136 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6402cc80 con 0x7fee6c0ac960 2024-02-02T13:27:21.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.849+0000 7fee70c00640 1 -- 172.21.15.91:0/3405692136 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6402cde0 con 0x7fee6c0ac960 2024-02-02T13:27:21.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.849+0000 7fee70c00640 1 -- 172.21.15.91:0/3405692136 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64034660 con 0x7fee6c0ac960 2024-02-02T13:27:21.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 -- 172.21.15.91:0/3405692136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0ac960 msgr2=0x7fee6c0acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 --2- 172.21.15.91:0/3405692136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0ac960 0x7fee6c0acdf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0cc320 tx=0x7fee6402bed0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 -- 172.21.15.91:0/3405692136 shutdown_connections 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 --2- 172.21.15.91:0/3405692136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0ac960 0x7fee6c0acdf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 --2- 172.21.15.91:0/3405692136 >> v2:172.21.15.113:3300/0 conn(0x7fee6c0cb580 0x7fee6c0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 -- 172.21.15.91:0/3405692136 >> 172.21.15.91:0/3405692136 conn(0x7fee6c0b0700 msgr2=0x7fee6c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 -- 172.21.15.91:0/3405692136 shutdown_connections 2024-02-02T13:27:21.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.850+0000 7fee74a00640 1 -- 172.21.15.91:0/3405692136 wait complete. 2024-02-02T13:27:21.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.851+0000 7fee74a00640 1 Processor -- start 2024-02-02T13:27:21.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 -- start start 2024-02-02T13:27:21.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6c0ac960 0x7fee6c15e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6c15d3a0 0x7fee6c0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c15f080 con 0x7fee6c0cb580 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c15f360 con 0x7fee6c0ac960 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.852+0000 7fee74a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c15f640 con 0x7fee6c15d3a0 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43858/0 (socket says 172.21.15.91:43858) 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 -- 172.21.15.91:0/1448538598 learned_addr learned my addr 172.21.15.91:0/1448538598 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 -- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6c15d3a0 msgr2=0x7fee6c0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6c15d3a0 0x7fee6c0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 -- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6c0ac960 msgr2=0x7fee6c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6c0ac960 0x7fee6c15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 -- 172.21.15.91:0/1448538598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009a70 con 0x7fee6c0cb580 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.853+0000 7fee71600640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c430 tx=0x7fee5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.854+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540074e0 con 0x7fee6c0cb580 2024-02-02T13:27:21.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.854+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54007660 con 0x7fee6c0cb580 2024-02-02T13:27:21.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.854+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c0ad4b0 con 0x7fee6c0cb580 2024-02-02T13:27:21.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.854+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54004a20 con 0x7fee6c0cb580 2024-02-02T13:27:21.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.854+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c15fc00 con 0x7fee6c0cb580 2024-02-02T13:27:21.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.855+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c15fe20 con 0x7fee6c0cb580 2024-02-02T13:27:21.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.856+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fee54004b80 con 0x7fee6c0cb580 2024-02-02T13:27:21.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.857+0000 7fee6a000640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 0x7fee4406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:21.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.857+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fee540883e0 con 0x7fee6c0cb580 2024-02-02T13:27:21.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.857+0000 7fee72000640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 0x7fee4406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:21.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.858+0000 7fee72000640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 0x7fee4406c920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee6c160450 tx=0x7fee5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:21.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:21.861+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fee54053d10 con 0x7fee6c0cb580 2024-02-02T13:27:22.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.101+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fee6c066cf0 con 0x7fee6c0cb580 2024-02-02T13:27:22.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.102+0000 7fee6a000640 1 -- 172.21.15.91:0/1448538598 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fee54057330 con 0x7fee6c0cb580 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.104 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:27:22.105 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.106 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.107 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.108 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:27:22.109 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.110 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.111 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.112 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:27:22.113 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:27:22.114 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:22.115 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:27:22.116 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 msgr2=0x7fee4406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 0x7fee4406c920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee6c160450 tx=0x7fee5c008040 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 msgr2=0x7fee6c14f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c430 tx=0x7fee5400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 shutdown_connections 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee4406a460 0x7fee4406c920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6c15d3a0 0x7fee6c0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fee6c0cb580 0x7fee6c14f310 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 --2- 172.21.15.91:0/1448538598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6c0ac960 0x7fee6c15e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.104+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 >> 172.21.15.91:0/1448538598 conn(0x7fee6c0b0700 msgr2=0x7fee6c0ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.105+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 shutdown_connections 2024-02-02T13:27:22.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.105+0000 7fee74a00640 1 -- 172.21.15.91:0/1448538598 wait complete. 2024-02-02T13:27:22.123 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:22.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.320+0000 7f4949600640 1 Processor -- start 2024-02-02T13:27:22.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.321+0000 7f4949600640 1 -- start start 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.322+0000 7f4949600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494415aec0 0x7f49440cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.322+0000 7f4949600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494415b2a0 0x7f494406c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.322+0000 7f4949600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494406c1c0 con 0x7f494406cb90 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.322+0000 7f4949600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494406b5f0 con 0x7f494415b2a0 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.322+0000 7f4949600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494406b890 con 0x7f494415aec0 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4948c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f494406cb90 0x7f49440ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45304/0 (socket says 172.21.15.91:45304) 2024-02-02T13:27:22.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f494415b2a0 0x7f494406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4948c00640 1 -- 172.21.15.91:0/2959769267 learned_addr learned my addr 172.21.15.91:0/2959769267 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:22.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943e00640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494415aec0 0x7f49440cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4942a00640 1 -- 172.21.15.91:0/2959769267 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614318857 0 0) 0x7f494406c1c0 con 0x7f494406cb90 2024-02-02T13:27:22.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4942a00640 1 -- 172.21.15.91:0/2959769267 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4924003520 con 0x7f494406cb90 2024-02-02T13:27:22.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943400640 1 -- 172.21.15.91:0/2959769267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494415aec0 msgr2=0x7f49440cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:22.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943400640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494415aec0 0x7f49440cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943400640 1 -- 172.21.15.91:0/2959769267 >> v1:172.21.15.91:6789/0 conn(0x7f494406cb90 legacy=0x7f49440ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.323+0000 7f4943400640 1 -- 172.21.15.91:0/2959769267 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494406bb70 con 0x7f494415b2a0 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.324+0000 7f4943400640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f494406c650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f492c0097a0 tx=0x7f492c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe0ffcc99f3b0dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4942a00640 1 -- 172.21.15.91:0/2959769267 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c030610 con 0x7f494415b2a0 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4942a00640 1 -- 172.21.15.91:0/2959769267 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c030770 con 0x7f494415b2a0 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4942a00640 1 -- 172.21.15.91:0/2959769267 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c030b00 con 0x7f494415b2a0 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4949600640 1 -- 172.21.15.91:0/2959769267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 msgr2=0x7f494406c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4949600640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f494406c650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f492c0097a0 tx=0x7f492c02b960 comp rx=0 tx=0).stop 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4949600640 1 -- 172.21.15.91:0/2959769267 shutdown_connections 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.325+0000 7f4949600640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f494406c650 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.326+0000 7f4949600640 1 --2- 172.21.15.91:0/2959769267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494415aec0 0x7f49440cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.326+0000 7f4949600640 1 -- 172.21.15.91:0/2959769267 >> 172.21.15.91:0/2959769267 conn(0x7f49440cea90 msgr2=0x7f49440ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.326+0000 7f4949600640 1 -- 172.21.15.91:0/2959769267 shutdown_connections 2024-02-02T13:27:22.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.326+0000 7f4949600640 1 -- 172.21.15.91:0/2959769267 wait complete. 2024-02-02T13:27:22.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.327+0000 7f4949600640 1 Processor -- start 2024-02-02T13:27:22.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.327+0000 7f4949600640 1 -- start start 2024-02-02T13:27:22.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494406cb90 0x7f49441f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494415aec0 0x7f49440cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441f28f0 con 0x7f494415b2a0 2024-02-02T13:27:22.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c96d0 con 0x7f494415aec0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.328+0000 7f4949600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c9970 con 0x7f494406cb90 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43876/0 (socket says 172.21.15.91:43876) 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 -- 172.21.15.91:0/698479954 learned_addr learned my addr 172.21.15.91:0/698479954 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 -- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494406cb90 msgr2=0x7f49441f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4943e00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494406cb90 0x7f49441f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494406cb90 0x7f49441f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 -- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494415aec0 msgr2=0x7f49440cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4943400640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494415aec0 0x7f49440cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494415aec0 0x7f49440cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 -- 172.21.15.91:0/698479954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c009400 con 0x7f494415b2a0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.329+0000 7f4948c00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f493c00caa0 tx=0x7f493c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.330+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00ac00 con 0x7f494415b2a0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.330+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00ad60 con 0x7f494415b2a0 2024-02-02T13:27:22.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.330+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440c9c50 con 0x7f494415b2a0 2024-02-02T13:27:22.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.330+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c012620 con 0x7f494415b2a0 2024-02-02T13:27:22.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.330+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440ca210 con 0x7f494415b2a0 2024-02-02T13:27:22.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.331+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494404dff0 con 0x7f494415b2a0 2024-02-02T13:27:22.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.332+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f493c007070 con 0x7f494415b2a0 2024-02-02T13:27:22.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.333+0000 7f493be00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 0x7f491c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.333+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f493c088650 con 0x7f494415b2a0 2024-02-02T13:27:22.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.333+0000 7f4943e00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 0x7f491c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.334+0000 7f4943e00640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 0x7f491c06c8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4934009a10 tx=0x7f4934008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.337+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f493c053e60 con 0x7f494415b2a0 2024-02-02T13:27:22.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.528+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49440ae100 con 0x7f494415b2a0 2024-02-02T13:27:22.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.529+0000 7f493be00640 1 -- 172.21.15.91:0/698479954 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f493c017020 con 0x7f494415b2a0 2024-02-02T13:27:22.531 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:22.531 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:21.594439+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:22.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 msgr2=0x7f491c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 0x7f491c06c8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4934009a10 tx=0x7f4934008040 comp rx=0 tx=0).stop 2024-02-02T13:27:22.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 msgr2=0x7f49440cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f493c00caa0 tx=0x7f493c00cf60 comp rx=0 tx=0).stop 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 shutdown_connections 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f491c06a430 0x7f491c06c8f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f494415b2a0 0x7f49440cf3b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f494415aec0 0x7f49440cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 --2- 172.21.15.91:0/698479954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f494406cb90 0x7f49441f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.531+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 >> 172.21.15.91:0/698479954 conn(0x7f49440cea90 msgr2=0x7f49441e52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.532+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 shutdown_connections 2024-02-02T13:27:22.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.532+0000 7f4949600640 1 -- 172.21.15.91:0/698479954 wait complete. 2024-02-02T13:27:22.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:27:22.553 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:27:22.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.745+0000 7f3719800640 1 Processor -- start 2024-02-02T13:27:22.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.745+0000 7f3719800640 1 -- start start 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3719800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f37140acfe0 0x7f371406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3719800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371415d870 con 0x7f37140b0700 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3719800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371406bce0 con 0x7f371415d490 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3719800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3714161220 con 0x7f37140acfe0 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3712000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f37140acfe0 0x7f371406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3713400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f37140b0700 0x7f37140b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45306/0 (socket says 172.21.15.91:45306) 2024-02-02T13:27:22.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.747+0000 7f3713400640 1 -- 172.21.15.91:0/2179756177 learned_addr learned my addr 172.21.15.91:0/2179756177 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:22.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.748+0000 7f3712000640 1 -- 172.21.15.91:0/2179756177 >> v1:172.21.15.113:6789/0 conn(0x7f371415d490 legacy=0x7f37140ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.748+0000 7f3712000640 1 -- 172.21.15.91:0/2179756177 >> v1:172.21.15.91:6789/0 conn(0x7f37140b0700 legacy=0x7f37140b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.748+0000 7f3712000640 1 -- 172.21.15.91:0/2179756177 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140ad490 con 0x7f37140acfe0 2024-02-02T13:27:22.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.748+0000 7f3711600640 1 -- 172.21.15.91:0/2179756177 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756122200 0 0) 0x7f371415d870 con 0x7f37140b0700 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.749+0000 7f3712000640 1 --2- 172.21.15.91:0/2179756177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f371406c170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009d90 tx=0x7f36fc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9efccf7d537a823 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.749+0000 7f3711600640 1 -- 172.21.15.91:0/2179756177 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc02cd00 con 0x7f37140acfe0 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.749+0000 7f3711600640 1 -- 172.21.15.91:0/2179756177 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc02ce60 con 0x7f37140acfe0 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.749+0000 7f3711600640 1 -- 172.21.15.91:0/2179756177 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc034660 con 0x7f37140acfe0 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 -- 172.21.15.91:0/2179756177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 msgr2=0x7f371406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 --2- 172.21.15.91:0/2179756177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f371406c170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009d90 tx=0x7f36fc02bf50 comp rx=0 tx=0).stop 2024-02-02T13:27:22.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 -- 172.21.15.91:0/2179756177 shutdown_connections 2024-02-02T13:27:22.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 --2- 172.21.15.91:0/2179756177 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f371406c170 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 -- 172.21.15.91:0/2179756177 >> 172.21.15.91:0/2179756177 conn(0x7f37140cec80 msgr2=0x7f37140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:22.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 -- 172.21.15.91:0/2179756177 shutdown_connections 2024-02-02T13:27:22.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.750+0000 7f3719800640 1 -- 172.21.15.91:0/2179756177 wait complete. 2024-02-02T13:27:22.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.752+0000 7f3719800640 1 Processor -- start 2024-02-02T13:27:22.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.752+0000 7f3719800640 1 -- start start 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37140b0700 0x7f37141f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f371415d490 0x7f37140c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c4e60 con 0x7f37140acfe0 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c5140 con 0x7f37140b0700 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3719800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c5420 con 0x7f371415d490 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43902/0 (socket says 172.21.15.91:43902) 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 -- 172.21.15.91:0/2971808781 learned_addr learned my addr 172.21.15.91:0/2971808781 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 -- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f371415d490 msgr2=0x7f37140c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f371415d490 0x7f37140c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 -- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37140b0700 msgr2=0x7f37141f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37140b0700 0x7f37141f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.753+0000 7f3712a00640 1 -- 172.21.15.91:0/2971808781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009a70 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f3712a00640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f370400c450 tx=0x7f370400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040052d0 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704005430 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704012870 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140c5700 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.754+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37140c4980 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.755+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37140c4bd0 con 0x7f37140acfe0 2024-02-02T13:27:22.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.756+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f37040129d0 con 0x7f37140acfe0 2024-02-02T13:27:22.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.757+0000 7f370aa00640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 0x7f36ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:22.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.757+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3704088140 con 0x7f37140acfe0 2024-02-02T13:27:22.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.757+0000 7f3712000640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 0x7f36ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:22.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.758+0000 7f3712000640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 0x7f36ec06c8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36fc036000 tx=0x7f36fc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:22.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.761+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3704053a70 con 0x7f37140acfe0 2024-02-02T13:27:22.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.935+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3714066cf0 con 0x7f37140acfe0 2024-02-02T13:27:22.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.935+0000 7f370aa00640 1 -- 172.21.15.91:0/2971808781 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3704057090 con 0x7f37140acfe0 2024-02-02T13:27:22.937 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 msgr2=0x7f36ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 0x7f36ec06c8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36fc036000 tx=0x7f36fc02e000 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 msgr2=0x7f37141f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f370400c450 tx=0x7f370400c910 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 shutdown_connections 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36ec06a430 0x7f36ec06c8f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f371415d490 0x7f37140c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37140b0700 0x7f37141f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 --2- 172.21.15.91:0/2971808781 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37140acfe0 0x7f37141f2c80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 >> 172.21.15.91:0/2971808781 conn(0x7f37140cec80 msgr2=0x7f3714157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:22.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 shutdown_connections 2024-02-02T13:27:22.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:22.938+0000 7f3719800640 1 -- 172.21.15.91:0/2971808781 wait complete. 2024-02-02T13:27:22.958 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:27:23.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.157+0000 7f0cff200640 1 Processor -- start 2024-02-02T13:27:23.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.158+0000 7f0cff200640 1 -- start start 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cff200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d0015aef0 0x7f0d000b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cff200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0006ce50 0x7f0d00070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cff200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000b0540 con 0x7f0d000adf20 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cff200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0015b2d0 con 0x7f0d0015aef0 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cff200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000ca410 con 0x7f0d0006ce50 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cfde00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d0015aef0 0x7f0d000b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cfde00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d0015aef0 0x7f0d000b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43912/0 (socket says 172.21.15.91:43912) 2024-02-02T13:27:23.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.159+0000 7f0cfde00640 1 -- 172.21.15.91:0/3911212039 learned_addr learned my addr 172.21.15.91:0/3911212039 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.160+0000 7f0cf5200640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0006ce50 0x7f0d00070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.160+0000 7f0cfde00640 1 -- 172.21.15.91:0/3911212039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0006ce50 msgr2=0x7f0d00070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.160+0000 7f0cfde00640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0006ce50 0x7f0d00070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.160+0000 7f0cfde00640 1 -- 172.21.15.91:0/3911212039 >> v1:172.21.15.113:6789/0 conn(0x7f0d000adf20 legacy=0x7f0d0006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.160+0000 7f0cfde00640 1 -- 172.21.15.91:0/3911212039 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000b07e0 con 0x7f0d0015aef0 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.161+0000 7f0cfde00640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0015aef0 0x7f0d000b0d00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727271eaa561aaec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.161+0000 7f0cfd400640 1 -- 172.21.15.91:0/3911212039 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0031640 con 0x7f0d0015aef0 2024-02-02T13:27:23.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.161+0000 7f0cfd400640 1 -- 172.21.15.91:0/3911212039 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf00317a0 con 0x7f0d0015aef0 2024-02-02T13:27:23.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.161+0000 7f0cfd400640 1 -- 172.21.15.91:0/3911212039 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0031b30 con 0x7f0d0015aef0 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 -- 172.21.15.91:0/3911212039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0015aef0 msgr2=0x7f0d000b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0015aef0 0x7f0d000b0d00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 -- 172.21.15.91:0/3911212039 shutdown_connections 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0006ce50 0x7f0d00070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 --2- 172.21.15.91:0/3911212039 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0015aef0 0x7f0d000b0d00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 -- 172.21.15.91:0/3911212039 >> 172.21.15.91:0/3911212039 conn(0x7f0d000cb4a0 msgr2=0x7f0d000cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 -- 172.21.15.91:0/3911212039 shutdown_connections 2024-02-02T13:27:23.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.162+0000 7f0cff200640 1 -- 172.21.15.91:0/3911212039 wait complete. 2024-02-02T13:27:23.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.164+0000 7f0cff200640 1 Processor -- start 2024-02-02T13:27:23.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.164+0000 7f0cff200640 1 -- start start 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000adf20 0x7f0d000ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0015aef0 0x7f0d000ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0015bc60 con 0x7f0d0006ce50 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0015ba80 con 0x7f0d000adf20 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.165+0000 7f0cff200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000ce630 con 0x7f0d0015aef0 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43916/0 (socket says 172.21.15.91:43916) 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 -- 172.21.15.91:0/2061023603 learned_addr learned my addr 172.21.15.91:0/2061023603 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 -- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0015aef0 msgr2=0x7f0d000ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0015aef0 0x7f0d000ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 -- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000adf20 msgr2=0x7f0d000ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000adf20 0x7f0d000ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 -- 172.21.15.91:0/2061023603 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00094d0 con 0x7f0d0006ce50 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.166+0000 7f0cfde00640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.167+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf003ac60 con 0x7f0d0006ce50 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.167+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf003adc0 con 0x7f0d0006ce50 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.167+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0044610 con 0x7f0d0006ce50 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.167+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d000cd240 con 0x7f0d0006ce50 2024-02-02T13:27:23.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.167+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000cd800 con 0x7f0d0006ce50 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.168+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0015bda0 con 0x7f0d0006ce50 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.169+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0cf002f050 con 0x7f0d0006ce50 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.169+0000 7f0cf6a00640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 0x7f0cd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.169+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0cf00aa650 con 0x7f0d0006ce50 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.170+0000 7f0cf5200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 0x7f0cd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.170+0000 7f0cf5200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 0x7f0cd806c8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006fd0 tx=0x7f0ce8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.174+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d0015bda0 con 0x7f0d0006ce50 2024-02-02T13:27:23.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.350+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0d0015bda0 con 0x7f0d0006ce50 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.350+0000 7f0cf6a00640 1 -- 172.21.15.91:0/2061023603 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0d0015bda0 con 0x7f0d0006ce50 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:27:23.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.352+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 msgr2=0x7f0cd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.352+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 0x7f0cd806c8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006fd0 tx=0x7f0ce8008040 comp rx=0 tx=0).stop 2024-02-02T13:27:23.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 msgr2=0x7f0d0015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf00046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 shutdown_connections 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cd806a430 0x7f0cd806c8f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d0015aef0 0x7f0d000ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000adf20 0x7f0d000ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 --2- 172.21.15.91:0/2061023603 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d0006ce50 0x7f0d0015ddb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 >> 172.21.15.91:0/2061023603 conn(0x7f0d000cb4a0 msgr2=0x7f0d0006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 shutdown_connections 2024-02-02T13:27:23.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.353+0000 7f0cff200640 1 -- 172.21.15.91:0/2061023603 wait complete. 2024-02-02T13:27:23.372 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:27:23.372 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:27:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.577+0000 7f716a600640 1 Processor -- start 2024-02-02T13:27:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.577+0000 7f716a600640 1 -- start start 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.578+0000 7f716a600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f716406f510 0x7f71640b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.578+0000 7f716a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71640cd840 0x7f71640cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.578+0000 7f716a600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71640ce1b0 con 0x7f71640b62f0 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.578+0000 7f716a600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71640b2df0 con 0x7f716406f510 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.578+0000 7f716a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716406f9a0 con 0x7f71640cd840 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f716406f510 0x7f71640b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7163400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f71640b62f0 0x7f716415bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45322/0 (socket says 172.21.15.91:45322) 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7163400640 1 -- 172.21.15.91:0/3636300093 learned_addr learned my addr 172.21.15.91:0/3636300093 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7163e00640 1 --2- 172.21.15.91:0/3636300093 >> v2:172.21.15.113:3300/0 conn(0x7f71640cd840 0x7f71640cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162000640 1 -- 172.21.15.91:0/3636300093 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915425533 0 0) 0x7f71640ce1b0 con 0x7f71640b62f0 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162000640 1 -- 172.21.15.91:0/3636300093 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7144003520 con 0x7f71640b62f0 2024-02-02T13:27:23.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162a00640 1 -- 172.21.15.91:0/3636300093 >> v2:172.21.15.113:3300/0 conn(0x7f71640cd840 msgr2=0x7f71640cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162a00640 1 --2- 172.21.15.91:0/3636300093 >> v2:172.21.15.113:3300/0 conn(0x7f71640cd840 0x7f71640cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162a00640 1 -- 172.21.15.91:0/3636300093 >> v1:172.21.15.91:6789/0 conn(0x7f71640b62f0 legacy=0x7f716415bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.579+0000 7f7162a00640 1 -- 172.21.15.91:0/3636300093 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716415b470 con 0x7f716406f510 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.580+0000 7f7162a00640 1 --2- 172.21.15.91:0/3636300093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f71640b28b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71640cb410 tx=0x7f714c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3d4791db59230b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.581+0000 7f7162000640 1 -- 172.21.15.91:0/3636300093 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c02cc80 con 0x7f716406f510 2024-02-02T13:27:23.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.581+0000 7f7162000640 1 -- 172.21.15.91:0/3636300093 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c02cde0 con 0x7f716406f510 2024-02-02T13:27:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.581+0000 7f7162000640 1 -- 172.21.15.91:0/3636300093 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c034660 con 0x7f716406f510 2024-02-02T13:27:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.581+0000 7f716a600640 1 -- 172.21.15.91:0/3636300093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 msgr2=0x7f71640b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.581+0000 7f716a600640 1 --2- 172.21.15.91:0/3636300093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f71640b28b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71640cb410 tx=0x7f714c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 -- 172.21.15.91:0/3636300093 shutdown_connections 2024-02-02T13:27:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 --2- 172.21.15.91:0/3636300093 >> v2:172.21.15.113:3300/0 conn(0x7f71640cd840 0x7f71640cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 --2- 172.21.15.91:0/3636300093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f71640b28b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 -- 172.21.15.91:0/3636300093 >> 172.21.15.91:0/3636300093 conn(0x7f716406d840 msgr2=0x7f716406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 -- 172.21.15.91:0/3636300093 shutdown_connections 2024-02-02T13:27:23.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.582+0000 7f716a600640 1 -- 172.21.15.91:0/3636300093 wait complete. 2024-02-02T13:27:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.584+0000 7f716a600640 1 Processor -- start 2024-02-02T13:27:23.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.584+0000 7f716a600640 1 -- start start 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71640b62f0 0x7f7164144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71640cd840 0x7f71640d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164147d40 con 0x7f716406f510 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164147280 con 0x7f71640b62f0 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.585+0000 7f716a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71640c4610 con 0x7f71640cd840 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43922/0 (socket says 172.21.15.91:43922) 2024-02-02T13:27:23.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 -- 172.21.15.91:0/753599915 learned_addr learned my addr 172.21.15.91:0/753599915 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7162a00640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71640b62f0 0x7f7164144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 -- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71640cd840 msgr2=0x7f71640d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71640cd840 0x7f71640d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 -- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71640b62f0 msgr2=0x7f7164144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71640b62f0 0x7f7164144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 -- 172.21.15.91:0/753599915 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c009a70 con 0x7f716406f510 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.586+0000 7f7163400640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f715400c4d0 tx=0x7f715400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.587+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154005ce0 con 0x7f716406f510 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.587+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154005e40 con 0x7f716406f510 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.587+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71640ca170 con 0x7f716406f510 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.587+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154007270 con 0x7f716406f510 2024-02-02T13:27:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.587+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7164147780 con 0x7f716406f510 2024-02-02T13:27:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.588+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71640c9ef0 con 0x7f716406f510 2024-02-02T13:27:23.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.589+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f71540073d0 con 0x7f716406f510 2024-02-02T13:27:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.590+0000 7f715b400640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 0x7f713c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.590+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7154087b00 con 0x7f716406f510 2024-02-02T13:27:23.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.590+0000 7f7162a00640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 0x7f713c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:23.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.591+0000 7f7162a00640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 0x7f713c06c8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f714c036000 tx=0x7f714c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:23.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.594+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7154089e60 con 0x7f716406f510 2024-02-02T13:27:23.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:23.788+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7164067400 con 0x7f716406f510 2024-02-02T13:27:24.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.634+0000 7f715b400640 1 -- 172.21.15.91:0/753599915 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f71540170b0 con 0x7f716406f510 2024-02-02T13:27:24.637 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:27:24.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 msgr2=0x7f713c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:24.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 0x7f713c06c8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f714c036000 tx=0x7f714c030000 comp rx=0 tx=0).stop 2024-02-02T13:27:24.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 msgr2=0x7f7164146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:24.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f715400c4d0 tx=0x7f715400c990 comp rx=0 tx=0).stop 2024-02-02T13:27:24.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 shutdown_connections 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f713c06a430 0x7f713c06c8f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71640cd840 0x7f71640d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71640b62f0 0x7f7164144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 --2- 172.21.15.91:0/753599915 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f716406f510 0x7f7164146d40 secure :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f715400c4d0 tx=0x7f715400c990 comp rx=0 tx=0).stop 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.640+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 >> 172.21.15.91:0/753599915 conn(0x7f716406d840 msgr2=0x7f71640b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:24.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.642+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 shutdown_connections 2024-02-02T13:27:24.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.642+0000 7f716a600640 1 -- 172.21.15.91:0/753599915 wait complete. 2024-02-02T13:27:24.667 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:27:24.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.873+0000 7ff115600640 1 Processor -- start 2024-02-02T13:27:24.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.873+0000 7ff115600640 1 -- start start 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.874+0000 7ff115600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1100cdeb0 0x7ff1100d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.874+0000 7ff115600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cbb30 0x7ff1100cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.874+0000 7ff115600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100cfea0 con 0x7ff1100cfa40 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.874+0000 7ff115600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100cccd0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.874+0000 7ff115600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100cc0c0 con 0x7ff1100cbb30 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1100cdeb0 0x7ff1100d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1100cdeb0 0x7ff1100d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43928/0 (socket says 172.21.15.91:43928) 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 -- 172.21.15.91:0/3573821467 learned_addr learned my addr 172.21.15.91:0/3573821467 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10e000640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cbb30 0x7ff1100cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 -- 172.21.15.91:0/3573821467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cbb30 msgr2=0x7ff1100cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cbb30 0x7ff1100cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 -- 172.21.15.91:0/3573821467 >> v1:172.21.15.113:6789/0 conn(0x7ff1100cfa40 legacy=0x7ff11015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:24.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.875+0000 7ff10ea00640 1 -- 172.21.15.91:0/3573821467 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100cc3a0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.876+0000 7ff10ea00640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff1100d19d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff10002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d244d6d604ffc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:24.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.877+0000 7ff10d600640 1 -- 172.21.15.91:0/3573821467 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100031640 con 0x7ff1100cdeb0 2024-02-02T13:27:24.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.877+0000 7ff10d600640 1 -- 172.21.15.91:0/3573821467 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000317a0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.877+0000 7ff10d600640 1 -- 172.21.15.91:0/3573821467 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100031b30 con 0x7ff1100cdeb0 2024-02-02T13:27:24.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.877+0000 7ff115600640 1 -- 172.21.15.91:0/3573821467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 msgr2=0x7ff1100d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:24.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.877+0000 7ff115600640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff1100d19d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff10002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 -- 172.21.15.91:0/3573821467 shutdown_connections 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cbb30 0x7ff1100cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 --2- 172.21.15.91:0/3573821467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff1100d19d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 -- 172.21.15.91:0/3573821467 >> 172.21.15.91:0/3573821467 conn(0x7ff11009a1f0 msgr2=0x7ff1100d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 -- 172.21.15.91:0/3573821467 shutdown_connections 2024-02-02T13:27:24.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.878+0000 7ff115600640 1 -- 172.21.15.91:0/3573821467 wait complete. 2024-02-02T13:27:24.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.879+0000 7ff115600640 1 Processor -- start 2024-02-02T13:27:24.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.879+0000 7ff115600640 1 -- start start 2024-02-02T13:27:24.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1100cbb30 0x7ff11014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cfa40 0x7ff1100c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015ba70 con 0x7ff1100cdeb0 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015bd50 con 0x7ff1100cbb30 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.880+0000 7ff115600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015c030 con 0x7ff1100cfa40 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43944/0 (socket says 172.21.15.91:43944) 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 -- 172.21.15.91:0/3283181811 learned_addr learned my addr 172.21.15.91:0/3283181811 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10ea00640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1100cbb30 0x7ff11014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10f400640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cfa40 0x7ff1100c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 -- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cfa40 msgr2=0x7ff1100c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cfa40 0x7ff1100c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 -- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1100cbb30 msgr2=0x7ff11014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:24.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1100cbb30 0x7ff11014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.881+0000 7ff10e000640 1 -- 172.21.15.91:0/3283181811 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000094d0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff10e000640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8007e40 tx=0x7ff0f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8014620 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800ee90 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100c7ff0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80149c0 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.882+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11014d770 con 0x7ff1100cdeb0 2024-02-02T13:27:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.883+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff110067400 con 0x7ff1100cdeb0 2024-02-02T13:27:24.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.884+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff0f8014b20 con 0x7ff1100cdeb0 2024-02-02T13:27:24.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.885+0000 7ff106a00640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 0x7ff0e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:24.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.885+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff0f8089150 con 0x7ff1100cdeb0 2024-02-02T13:27:24.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.885+0000 7ff10ea00640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 0x7ff0e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:24.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.886+0000 7ff10ea00640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 0x7ff0e806c8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff100036000 tx=0x7ff10003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:24.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:24.890+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff0f8054880 con 0x7ff1100cdeb0 2024-02-02T13:27:25.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.074+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff110073010 con 0x7ff1100cdeb0 2024-02-02T13:27:25.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.647+0000 7ff106a00640 1 -- 172.21.15.91:0/3283181811 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff0f8057ea0 con 0x7ff1100cdeb0 2024-02-02T13:27:25.650 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:27:25.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.656+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 msgr2=0x7ff0e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.656+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 0x7ff0e806c8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff100036000 tx=0x7ff10003c040 comp rx=0 tx=0).stop 2024-02-02T13:27:25.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.656+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 msgr2=0x7ff11015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.656+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8007e40 tx=0x7ff0f800e4b0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 shutdown_connections 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0e806a430 0x7ff0e806c8f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1100cfa40 0x7ff1100c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1100cdeb0 0x7ff11015b0d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 --2- 172.21.15.91:0/3283181811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1100cbb30 0x7ff11014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.658+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 >> 172.21.15.91:0/3283181811 conn(0x7ff11009a1f0 msgr2=0x7ff11015c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:25.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.659+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 shutdown_connections 2024-02-02T13:27:25.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.661+0000 7ff115600640 1 -- 172.21.15.91:0/3283181811 wait complete. 2024-02-02T13:27:25.688 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:27:25.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.864+0000 7f35df200640 1 Processor -- start 2024-02-02T13:27:25.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.864+0000 7f35df200640 1 -- start start 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.865+0000 7f35df200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f35d815d490 0x7f35d806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.865+0000 7f35df200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d815d870 con 0x7f35d80ac8d0 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.865+0000 7f35df200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d806bce0 con 0x7f35d80b0e10 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.865+0000 7f35df200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d8161220 con 0x7f35d815d490 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f35d815d490 0x7f35d806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f35d80ac8d0 0x7f35d80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45324/0 (socket says 172.21.15.91:45324) 2024-02-02T13:27:25.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7400640 1 -- 172.21.15.91:0/275171883 learned_addr learned my addr 172.21.15.91:0/275171883 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d6a00640 1 -- 172.21.15.91:0/275171883 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316155151 0 0) 0x7f35d815d870 con 0x7f35d80ac8d0 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d6a00640 1 -- 172.21.15.91:0/275171883 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b8003520 con 0x7f35d80ac8d0 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7e00640 1 -- 172.21.15.91:0/275171883 >> v1:172.21.15.113:6789/0 conn(0x7f35d80b0e10 legacy=0x7f35d80c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7e00640 1 -- 172.21.15.91:0/275171883 >> v1:172.21.15.91:6789/0 conn(0x7f35d80ac8d0 legacy=0x7f35d80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.866+0000 7f35d7e00640 1 -- 172.21.15.91:0/275171883 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80ad490 con 0x7f35d815d490 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.867+0000 7f35d7e00640 1 --2- 172.21.15.91:0/275171883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d815d490 0x7f35d806c170 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097a0 tx=0x7f35c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38676579370b750 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.867+0000 7f35d6a00640 1 -- 172.21.15.91:0/275171883 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8031830 con 0x7f35d815d490 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35d6a00640 1 -- 172.21.15.91:0/275171883 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8031990 con 0x7f35d815d490 2024-02-02T13:27:25.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35d6a00640 1 -- 172.21.15.91:0/275171883 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8031d20 con 0x7f35d815d490 2024-02-02T13:27:25.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 -- 172.21.15.91:0/275171883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d815d490 msgr2=0x7f35d806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 --2- 172.21.15.91:0/275171883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d815d490 0x7f35d806c170 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097a0 tx=0x7f35c802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 -- 172.21.15.91:0/275171883 shutdown_connections 2024-02-02T13:27:25.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 --2- 172.21.15.91:0/275171883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d815d490 0x7f35d806c170 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 -- 172.21.15.91:0/275171883 >> 172.21.15.91:0/275171883 conn(0x7f35d80cec80 msgr2=0x7f35d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 -- 172.21.15.91:0/275171883 shutdown_connections 2024-02-02T13:27:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.868+0000 7f35df200640 1 -- 172.21.15.91:0/275171883 wait complete. 2024-02-02T13:27:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.870+0000 7f35df200640 1 Processor -- start 2024-02-02T13:27:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.870+0000 7f35df200640 1 -- start start 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 0x7f35d80c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 0x7f35d80c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c55b0 con 0x7f35d80ac8d0 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d815e6f0 con 0x7f35d80b0e10 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35df200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c52d0 con 0x7f35d815d490 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43956/0 (socket says 172.21.15.91:43956) 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.871+0000 7f35d7e00640 1 -- 172.21.15.91:0/1149338351 learned_addr learned my addr 172.21.15.91:0/1149338351 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7400640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 0x7f35d80c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35dd200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 0x7f35d80c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 -- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 msgr2=0x7f35d80c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 0x7f35d80c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 -- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 msgr2=0x7f35d80c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 0x7f35d80c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 -- 172.21.15.91:0/1149338351 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009400 con 0x7f35d80ac8d0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35dd200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 0x7f35d80c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7400640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 0x7f35d80c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35d7e00640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098c0 tx=0x7f35c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c803a6f0 con 0x7f35d80ac8d0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c803a870 con 0x7f35d80ac8d0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d815e0a0 con 0x7f35d80ac8d0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c803ac20 con 0x7f35d80ac8d0 2024-02-02T13:27:25.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.872+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d80cac10 con 0x7f35d80ac8d0 2024-02-02T13:27:25.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.873+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d815df00 con 0x7f35d80ac8d0 2024-02-02T13:27:25.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.875+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f35c804c3b0 con 0x7f35d80ac8d0 2024-02-02T13:27:25.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.876+0000 7f35cfe00640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 0x7f35b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:25.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.876+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f35c80aa7a0 con 0x7f35d80ac8d0 2024-02-02T13:27:25.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.876+0000 7f35d7400640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 0x7f35b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:25.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.877+0000 7f35d7400640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 0x7f35b006c8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f35d80b4360 tx=0x7f35c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:25.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:25.879+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f35c8075dd0 con 0x7f35d80ac8d0 2024-02-02T13:27:26.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.087+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35d815df00 con 0x7f35d80ac8d0 2024-02-02T13:27:26.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.097+0000 7f35cfe00640 1 -- 172.21.15.91:0/1149338351 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f35d815df00 con 0x7f35d80ac8d0 2024-02-02T13:27:26.099 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 8 and data pool 9 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 msgr2=0x7f35b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 0x7f35b006c8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f35d80b4360 tx=0x7f35c0009040 comp rx=0 tx=0).stop 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 msgr2=0x7f35d80ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098c0 tx=0x7f35c80046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 shutdown_connections 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f35b006a430 0x7f35b006c8f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d815d490 0x7f35d80c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b0e10 0x7f35d80c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 --2- 172.21.15.91:0/1149338351 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f35d80ac8d0 0x7f35d80ca6d0 secure :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098c0 tx=0x7f35c80046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.103+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 >> 172.21.15.91:0/1149338351 conn(0x7f35d80cec80 msgr2=0x7f35d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:26.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.106+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 shutdown_connections 2024-02-02T13:27:26.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.106+0000 7f35df200640 1 -- 172.21.15.91:0/1149338351 wait complete. 2024-02-02T13:27:26.132 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:26.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.306+0000 7f3363e00640 1 Processor -- start 2024-02-02T13:27:26.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.306+0000 7f3363e00640 1 -- start start 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.307+0000 7f3363e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f33640cad30 0x7f33640cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.307+0000 7f3363e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336406c1c0 con 0x7f336415b2a0 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.307+0000 7f3363e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336406b5f0 con 0x7f336415aec0 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.307+0000 7f3363e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336406cd60 con 0x7f33640cad30 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3363400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f33640cad30 0x7f33640cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3359e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f336415b2a0 0x7f33640ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45338/0 (socket says 172.21.15.91:45338) 2024-02-02T13:27:26.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3359e00640 1 -- 172.21.15.91:0/1468638207 learned_addr learned my addr 172.21.15.91:0/1468638207 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:26.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3363400640 1 -- 172.21.15.91:0/1468638207 >> v1:172.21.15.113:6789/0 conn(0x7f336415aec0 legacy=0x7f336406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:26.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3363400640 1 -- 172.21.15.91:0/1468638207 >> v1:172.21.15.91:6789/0 conn(0x7f336415b2a0 legacy=0x7f33640ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.308+0000 7f3363400640 1 -- 172.21.15.91:0/1468638207 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336406b890 con 0x7f33640cad30 2024-02-02T13:27:26.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.309+0000 7f3363400640 1 --2- 172.21.15.91:0/1468638207 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f33640cf5b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f335c009d90 tx=0x7f335c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971fdca77d21e57a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.310+0000 7f3362000640 1 -- 172.21.15.91:0/1468638207 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c02ccf0 con 0x7f33640cad30 2024-02-02T13:27:26.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.310+0000 7f3362000640 1 -- 172.21.15.91:0/1468638207 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c02ce50 con 0x7f33640cad30 2024-02-02T13:27:26.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.310+0000 7f3362000640 1 -- 172.21.15.91:0/1468638207 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c034660 con 0x7f33640cad30 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.310+0000 7f3363e00640 1 -- 172.21.15.91:0/1468638207 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 msgr2=0x7f33640cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.310+0000 7f3363e00640 1 --2- 172.21.15.91:0/1468638207 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f33640cf5b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f335c009d90 tx=0x7f335c02bf40 comp rx=0 tx=0).stop 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.311+0000 7f3363e00640 1 -- 172.21.15.91:0/1468638207 shutdown_connections 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.311+0000 7f3363e00640 1 --2- 172.21.15.91:0/1468638207 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f33640cf5b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.311+0000 7f3363e00640 1 -- 172.21.15.91:0/1468638207 >> 172.21.15.91:0/1468638207 conn(0x7f33640cea90 msgr2=0x7f33640ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:26.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.311+0000 7f3363e00640 1 -- 172.21.15.91:0/1468638207 shutdown_connections 2024-02-02T13:27:26.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.311+0000 7f3363e00640 1 -- 172.21.15.91:0/1468638207 wait complete. 2024-02-02T13:27:26.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.313+0000 7f3363e00640 1 Processor -- start 2024-02-02T13:27:26.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.313+0000 7f3363e00640 1 -- start start 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 0x7f3364147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 0x7f336415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336415d5f0 con 0x7f33640cad30 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336415d8d0 con 0x7f336415aec0 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336415dbb0 con 0x7f336415b2a0 2024-02-02T13:27:26.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3362a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3362a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43970/0 (socket says 172.21.15.91:43970) 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3362a00640 1 -- 172.21.15.91:0/165270906 learned_addr learned my addr 172.21.15.91:0/165270906 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.314+0000 7f3363400640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 0x7f336415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 -- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 msgr2=0x7f336415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3359e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 0x7f3364147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 0x7f336415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 -- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 msgr2=0x7f3364147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 0x7f3364147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 -- 172.21.15.91:0/165270906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c009a70 con 0x7f33640cad30 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3363400640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 0x7f336415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3359e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 0x7f3364147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3362a00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f335400c430 tx=0x7f335400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354011790 con 0x7f33640cad30 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33540118f0 con 0x7f33640cad30 2024-02-02T13:27:26.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354011ce0 con 0x7f33640cad30 2024-02-02T13:27:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336415de90 con 0x7f33640cad30 2024-02-02T13:27:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.315+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336415cca0 con 0x7f33640cad30 2024-02-02T13:27:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.317+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336406faf0 con 0x7f33640cad30 2024-02-02T13:27:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.317+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f335401c460 con 0x7f33640cad30 2024-02-02T13:27:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.319+0000 7f335b400640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 0x7f333c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.319+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3354088980 con 0x7f33640cad30 2024-02-02T13:27:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.319+0000 7f3359e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 0x7f333c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.320+0000 7f3359e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 0x7f333c06c8f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33640b35d0 tx=0x7f334c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.324+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3354053ed0 con 0x7f33640cad30 2024-02-02T13:27:26.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.515+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f336415d730 con 0x7f33640cad30 2024-02-02T13:27:26.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.518+0000 7f335b400640 1 -- 172.21.15.91:0/165270906 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f336415d730 con 0x7f33640cad30 2024-02-02T13:27:26.520 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:26.520 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":67,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:26.089021+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.520+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 msgr2=0x7f333c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.520+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 0x7f333c06c8f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33640b35d0 tx=0x7f334c009040 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.520+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 msgr2=0x7f336415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.520+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f335400c430 tx=0x7f335400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 shutdown_connections 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f333c06a430 0x7f333c06c8f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336415b2a0 0x7f336415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336415aec0 0x7f3364147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 --2- 172.21.15.91:0/165270906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33640cad30 0x7f336415e3b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 >> 172.21.15.91:0/165270906 conn(0x7f33640cea90 msgr2=0x7f33640b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 shutdown_connections 2024-02-02T13:27:26.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.521+0000 7f3363e00640 1 -- 172.21.15.91:0/165270906 wait complete. 2024-02-02T13:27:26.540 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:26.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.741+0000 7f7c49200640 1 Processor -- start 2024-02-02T13:27:26.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.741+0000 7f7c49200640 1 -- start start 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c49200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7c44154300 0x7f7c440b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c49200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440700b0 con 0x7f7c440c9b80 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c49200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440ca130 con 0x7f7c441546e0 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c49200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4406ff40 con 0x7f7c44154300 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c41e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7c44154300 0x7f7c440b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c41e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7c44154300 0x7f7c440b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43972/0 (socket says 172.21.15.91:43972) 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c42800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7c440c9b80 0x7f7c44070540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45350/0 (socket says 172.21.15.91:45350) 2024-02-02T13:27:26.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.742+0000 7f7c41e00640 1 -- 172.21.15.91:0/2535031465 learned_addr learned my addr 172.21.15.91:0/2535031465 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:26.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.743+0000 7f7c41e00640 1 -- 172.21.15.91:0/2535031465 >> v1:172.21.15.113:6789/0 conn(0x7f7c441546e0 legacy=0x7f7c440c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:26.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.743+0000 7f7c41e00640 1 -- 172.21.15.91:0/2535031465 >> v1:172.21.15.91:6789/0 conn(0x7f7c440c9b80 legacy=0x7f7c44070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.743+0000 7f7c41e00640 1 -- 172.21.15.91:0/2535031465 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440ca410 con 0x7f7c44154300 2024-02-02T13:27:26.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.743+0000 7f7c41400640 1 -- 172.21.15.91:0/2535031465 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602369138 0 0) 0x7f7c440700b0 con 0x7f7c440c9b80 2024-02-02T13:27:26.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.744+0000 7f7c41e00640 1 --2- 172.21.15.91:0/2535031465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c440b0720 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400bb10 tx=0x7f7c3402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f234cafc07256b6c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.744+0000 7f7c41400640 1 -- 172.21.15.91:0/2535031465 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34036440 con 0x7f7c44154300 2024-02-02T13:27:26.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.744+0000 7f7c41400640 1 -- 172.21.15.91:0/2535031465 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c340365a0 con 0x7f7c44154300 2024-02-02T13:27:26.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.744+0000 7f7c41400640 1 -- 172.21.15.91:0/2535031465 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34036930 con 0x7f7c44154300 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 -- 172.21.15.91:0/2535031465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 msgr2=0x7f7c440b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 --2- 172.21.15.91:0/2535031465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c440b0720 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400bb10 tx=0x7f7c3402e170 comp rx=0 tx=0).stop 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 -- 172.21.15.91:0/2535031465 shutdown_connections 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 --2- 172.21.15.91:0/2535031465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c440b0720 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 -- 172.21.15.91:0/2535031465 >> 172.21.15.91:0/2535031465 conn(0x7f7c440ceca0 msgr2=0x7f7c440cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 -- 172.21.15.91:0/2535031465 shutdown_connections 2024-02-02T13:27:26.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.745+0000 7f7c49200640 1 -- 172.21.15.91:0/2535031465 wait complete. 2024-02-02T13:27:26.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.747+0000 7f7c49200640 1 Processor -- start 2024-02-02T13:27:26.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.747+0000 7f7c49200640 1 -- start start 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c440c9b80 0x7f7c440c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c441546e0 0x7f7c440c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440c7cd0 con 0x7f7c44154300 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440c6530 con 0x7f7c440c9b80 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c49200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440c6810 con 0x7f7c441546e0 2024-02-02T13:27:26.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43976/0 (socket says 172.21.15.91:43976) 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 -- 172.21.15.91:0/352983455 learned_addr learned my addr 172.21.15.91:0/352983455 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 -- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c441546e0 msgr2=0x7f7c440c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c441546e0 0x7f7c440c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 -- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c440c9b80 msgr2=0x7f7c440c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c440c9b80 0x7f7c440c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.748+0000 7f7c39200640 1 -- 172.21.15.91:0/352983455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3400b770 con 0x7f7c44154300 2024-02-02T13:27:26.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c39200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b3f0 tx=0x7f7c2c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011830 con 0x7f7c44154300 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c011990 con 0x7f7c44154300 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011d40 con 0x7f7c44154300 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c440c7f70 con 0x7f7c44154300 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.749+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c440c8530 con 0x7f7c44154300 2024-02-02T13:27:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.750+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c44073830 con 0x7f7c44154300 2024-02-02T13:27:26.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.751+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c2c01a400 con 0x7f7c44154300 2024-02-02T13:27:26.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.752+0000 7f7c3aa00640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:26.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.752+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c2c088840 con 0x7f7c44154300 2024-02-02T13:27:26.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.752+0000 7f7c41e00640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:26.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.753+0000 7f7c41e00640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c34038000 tx=0x7f7c34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:26.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.756+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c2c053d90 con 0x7f7c44154300 2024-02-02T13:27:26.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.947+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c440ca6c0 con 0x7f7c44154300 2024-02-02T13:27:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.949+0000 7f7c3aa00640 1 -- 172.21.15.91:0/352983455 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f7c2c0573b0 con 0x7f7c44154300 2024-02-02T13:27:26.951 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:26.951 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":67,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:26.089021+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 msgr2=0x7f7c1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c34038000 tx=0x7f7c34008040 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 msgr2=0x7f7c4413c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b3f0 tx=0x7f7c2c00b8b0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 shutdown_connections 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c441546e0 0x7f7c440c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7c44154300 0x7f7c4413c0c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 --2- 172.21.15.91:0/352983455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c440c9b80 0x7f7c440c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 >> 172.21.15.91:0/352983455 conn(0x7f7c440ceca0 msgr2=0x7f7c440b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 shutdown_connections 2024-02-02T13:27:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:26.952+0000 7f7c49200640 1 -- 172.21.15.91:0/352983455 wait complete. 2024-02-02T13:27:26.972 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:27:27.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.176+0000 7fbea2c00640 1 Processor -- start 2024-02-02T13:27:27.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.177+0000 7fbea2c00640 1 -- start start 2024-02-02T13:27:27.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea2c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe9c160610 0x7fbe9c0d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:27.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea2c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c0b4930 con 0x7fbe9c0d2890 2024-02-02T13:27:27.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c0b4c10 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea2c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b4ef0 con 0x7fbe9c160610 2024-02-02T13:27:27.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea1800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe9c160610 0x7fbe9c0d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea0e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbe9c0d2890 0x7fbe9c0d2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45364/0 (socket says 172.21.15.91:45364) 2024-02-02T13:27:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea0e00640 1 -- 172.21.15.91:0/2624048336 learned_addr learned my addr 172.21.15.91:0/2624048336 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea1800640 1 -- 172.21.15.91:0/2624048336 >> v1:172.21.15.113:6789/0 conn(0x7fbe9c0d34b0 legacy=0x7fbe9c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbea1800640 1 -- 172.21.15.91:0/2624048336 >> v1:172.21.15.91:6789/0 conn(0x7fbe9c0d2890 legacy=0x7fbe9c0d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.178+0000 7fbe9be00640 1 -- 172.21.15.91:0/2624048336 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56194207 0 0) 0x7fbe9c0b4930 con 0x7fbe9c0d2890 2024-02-02T13:27:27.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.179+0000 7fbea1800640 1 -- 172.21.15.91:0/2624048336 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c0b51d0 con 0x7fbe9c160610 2024-02-02T13:27:27.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbea1800640 1 --2- 172.21.15.91:0/2624048336 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c160610 0x7fbe9c0d2350 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097a0 tx=0x7fbe8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf7932fc5e58dff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:27.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbe9be00640 1 -- 172.21.15.91:0/2624048336 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c031830 con 0x7fbe9c160610 2024-02-02T13:27:27.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbe9be00640 1 -- 172.21.15.91:0/2624048336 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c031990 con 0x7fbe9c160610 2024-02-02T13:27:27.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbe9be00640 1 -- 172.21.15.91:0/2624048336 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c031d20 con 0x7fbe9c160610 2024-02-02T13:27:27.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbea2c00640 1 -- 172.21.15.91:0/2624048336 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c160610 msgr2=0x7fbe9c0d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:27.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.180+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2624048336 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c160610 0x7fbe9c0d2350 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097a0 tx=0x7fbe8c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:27:27.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.181+0000 7fbea2c00640 1 -- 172.21.15.91:0/2624048336 shutdown_connections 2024-02-02T13:27:27.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.181+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2624048336 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c160610 0x7fbe9c0d2350 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:27.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.181+0000 7fbea2c00640 1 -- 172.21.15.91:0/2624048336 >> 172.21.15.91:0/2624048336 conn(0x7fbe9c070690 msgr2=0x7fbe9c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:27.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.181+0000 7fbea2c00640 1 -- 172.21.15.91:0/2624048336 shutdown_connections 2024-02-02T13:27:27.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.181+0000 7fbea2c00640 1 -- 172.21.15.91:0/2624048336 wait complete. 2024-02-02T13:27:27.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.182+0000 7fbea2c00640 1 Processor -- start 2024-02-02T13:27:27.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.182+0000 7fbea2c00640 1 -- start start 2024-02-02T13:27:27.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0d2890 0x7fbe9c0af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:27.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:27.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe9c160610 0x7fbe9c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:27.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0aa9a0 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0aac40 con 0x7fbe9c0d2890 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0aaf20 con 0x7fbe9c160610 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.183+0000 7fbea0e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44002/0 (socket says 172.21.15.91:44002) 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 -- 172.21.15.91:0/2339433365 learned_addr learned my addr 172.21.15.91:0/2339433365 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 -- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe9c160610 msgr2=0x7fbe9c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe9c160610 0x7fbe9c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 -- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0d2890 msgr2=0x7fbe9c0af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0d2890 0x7fbe9c0af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:27.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 -- 172.21.15.91:0/2339433365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009400 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea0e00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c890 tx=0x7fbe8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:27.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400a640 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8400a7c0 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c0ab200 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.184+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400ab70 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.185+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c0afa30 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.186+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c0ab720 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.186+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbe8400acd0 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.187+0000 7fbe99600640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 0x7fbe7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:27.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.187+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe84087e00 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.188+0000 7fbea1800640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 0x7fbe7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:27.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.188+0000 7fbea1800640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 0x7fbe7406c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c036000 tx=0x7fbe8c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:27.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.191+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbe9c0ab720 con 0x7fbe9c0d34b0 2024-02-02T13:27:27.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:27.421+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbe9c0ab720 con 0x7fbe9c0d34b0 2024-02-02T13:27:28.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.121+0000 7fbe99600640 1 -- 172.21.15.91:0/2339433365 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbe9c0ab720 con 0x7fbe9c0d34b0 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 msgr2=0x7fbe7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 0x7fbe7406c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c036000 tx=0x7fbe8c030000 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 msgr2=0x7fbe9c0b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c890 tx=0x7fbe8400cd50 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 shutdown_connections 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbe7406a430 0x7fbe7406c8f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe9c160610 0x7fbe9c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe9c0d34b0 0x7fbe9c0b1700 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 --2- 172.21.15.91:0/2339433365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0d2890 0x7fbe9c0af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 >> 172.21.15.91:0/2339433365 conn(0x7fbe9c070690 msgr2=0x7fbe9c0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:28.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 shutdown_connections 2024-02-02T13:27:28.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.124+0000 7fbea2c00640 1 -- 172.21.15.91:0/2339433365 wait complete. 2024-02-02T13:27:28.147 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:27:28.147 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:27:28.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.358+0000 7f1f47800640 1 Processor -- start 2024-02-02T13:27:28.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.358+0000 7f1f47800640 1 -- start start 2024-02-02T13:27:28.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f47800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 0x7f1f40070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f47800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f400b24b0 0x7f1f400707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f47800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f400cd930 con 0x7f1f400b1f60 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f47800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400cde60 con 0x7f1f400b24b0 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f47800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400ce520 con 0x7f1f4015d3c0 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.360+0000 7f1f44e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 0x7f1f40070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f45800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1f400b1f60 0x7f1f40160f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:58806/0 (socket says 172.21.15.91:58806) 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f3fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f400b24b0 0x7f1f400707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f45800640 1 -- 172.21.15.91:0/4185115633 learned_addr learned my addr 172.21.15.91:0/4185115633 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:28.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f44e00640 1 -- 172.21.15.91:0/4185115633 >> v2:172.21.15.113:3300/0 conn(0x7f1f400b24b0 msgr2=0x7f1f400707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:28.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f44e00640 1 --2- 172.21.15.91:0/4185115633 >> v2:172.21.15.113:3300/0 conn(0x7f1f400b24b0 0x7f1f400707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f3f400640 1 -- 172.21.15.91:0/4185115633 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162050031 0 0) 0x7f1f400cd930 con 0x7f1f400b1f60 2024-02-02T13:27:28.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f44e00640 1 -- 172.21.15.91:0/4185115633 >> v1:172.21.15.113:6789/0 conn(0x7f1f400b1f60 legacy=0x7f1f40160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.361+0000 7f1f44e00640 1 -- 172.21.15.91:0/4185115633 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400cdb80 con 0x7f1f4015d3c0 2024-02-02T13:27:28.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.363+0000 7f1f44e00640 1 --2- 172.21.15.91:0/4185115633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 0x7f1f40070270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f3002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c076b855e2a85f1f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:28.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.363+0000 7f1f3f400640 1 -- 172.21.15.91:0/4185115633 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3002e070 con 0x7f1f4015d3c0 2024-02-02T13:27:28.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.363+0000 7f1f3f400640 1 -- 172.21.15.91:0/4185115633 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f300315a0 con 0x7f1f4015d3c0 2024-02-02T13:27:28.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.363+0000 7f1f3f400640 1 -- 172.21.15.91:0/4185115633 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30031930 con 0x7f1f4015d3c0 2024-02-02T13:27:28.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 -- 172.21.15.91:0/4185115633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 msgr2=0x7f1f40070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 --2- 172.21.15.91:0/4185115633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 0x7f1f40070270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f3002b840 comp rx=0 tx=0).stop 2024-02-02T13:27:28.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 -- 172.21.15.91:0/4185115633 shutdown_connections 2024-02-02T13:27:28.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 --2- 172.21.15.91:0/4185115633 >> v2:172.21.15.113:3300/0 conn(0x7f1f400b24b0 0x7f1f400707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 --2- 172.21.15.91:0/4185115633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f4015d3c0 0x7f1f40070270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 -- 172.21.15.91:0/4185115633 >> 172.21.15.91:0/4185115633 conn(0x7f1f400cfa30 msgr2=0x7f1f400cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:28.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 -- 172.21.15.91:0/4185115633 shutdown_connections 2024-02-02T13:27:28.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.364+0000 7f1f47800640 1 -- 172.21.15.91:0/4185115633 wait complete. 2024-02-02T13:27:28.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.366+0000 7f1f47800640 1 Processor -- start 2024-02-02T13:27:28.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.366+0000 7f1f47800640 1 -- start start 2024-02-02T13:27:28.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f400b24b0 0x7f1f40154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 0x7f1f400b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f401f6da0 con 0x7f1f400b1f60 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f40155d50 con 0x7f1f4015d3c0 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f47800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f40155fb0 con 0x7f1f400b24b0 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44010/0 (socket says 172.21.15.91:44010) 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 -- 172.21.15.91:0/15457976 learned_addr learned my addr 172.21.15.91:0/15457976 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f3fe00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f400b24b0 0x7f1f40154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 -- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f400b24b0 msgr2=0x7f1f40154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f45800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 0x7f1f400b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f400b24b0 0x7f1f40154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.367+0000 7f1f44e00640 1 -- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 msgr2=0x7f1f400b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f44e00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 0x7f1f400b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:28.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f44e00640 1 -- 172.21.15.91:0/15457976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009400 con 0x7f1f400b1f60 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f45800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 0x7f1f400b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f44e00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004230 con 0x7f1f400b1f60 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30004390 con 0x7f1f400b1f60 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30003850 con 0x7f1f400b1f60 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f40156270 con 0x7f1f400b1f60 2024-02-02T13:27:28.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.368+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f40156830 con 0x7f1f400b1f60 2024-02-02T13:27:28.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.369+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f40066cf0 con 0x7f1f400b1f60 2024-02-02T13:27:28.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.370+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f300039b0 con 0x7f1f400b1f60 2024-02-02T13:27:28.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.371+0000 7f1f3cc00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 0x7f1f1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:28.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.371+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f300aaa60 con 0x7f1f400b1f60 2024-02-02T13:27:28.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.371+0000 7f1f3fe00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 0x7f1f1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:28.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.372+0000 7f1f3fe00640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 0x7f1f1806c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b4a40 tx=0x7f1f28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:28.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.376+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f30076070 con 0x7f1f400b1f60 2024-02-02T13:27:28.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:28.604+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1f400c9b00 con 0x7f1f400b1f60 2024-02-02T13:27:29.111 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:27:29.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:27:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.124+0000 7f1f3cc00640 1 -- 172.21.15.91:0/15457976 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f30044340 con 0x7f1f400b1f60 2024-02-02T13:27:29.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.128+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 msgr2=0x7f1f1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.128+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 0x7f1f1806c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b4a40 tx=0x7f1f28008040 comp rx=0 tx=0).stop 2024-02-02T13:27:29.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.128+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 msgr2=0x7f1f401f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.128+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f300046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 shutdown_connections 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f1806a430 0x7f1f1806c8f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f400b24b0 0x7f1f40154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f4015d3c0 0x7f1f400b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 --2- 172.21.15.91:0/15457976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f400b1f60 0x7f1f401f6860 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 >> 172.21.15.91:0/15457976 conn(0x7f1f400cfa30 msgr2=0x7f1f40158290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 shutdown_connections 2024-02-02T13:27:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.129+0000 7f1f47800640 1 -- 172.21.15.91:0/15457976 wait complete. 2024-02-02T13:27:29.151 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:27:29.151 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:29.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.389+0000 7f5b42200640 1 Processor -- start 2024-02-02T13:27:29.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.389+0000 7f5b42200640 1 -- start start 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b42200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b3c0b0e10 0x7f5b3c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b42200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c0cc7c0 con 0x7f5b3c15a010 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b42200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c0ce930 con 0x7f5b3c0ce130 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b42200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c0cd640 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b3b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5b3c15a010 0x7f5b3c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45372/0 (socket says 172.21.15.91:45372) 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b3b400640 1 -- 172.21.15.91:0/2218382835 learned_addr learned my addr 172.21.15.91:0/2218382835 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:29.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.391+0000 7f5b3be00640 1 --2- 172.21.15.91:0/2218382835 >> v2:172.21.15.91:3300/0 conn(0x7f5b3c0b0e10 0x7f5b3c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.392+0000 7f5b3be00640 1 -- 172.21.15.91:0/2218382835 >> v1:172.21.15.113:6789/0 conn(0x7f5b3c0ce130 legacy=0x7f5b3c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:29.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.392+0000 7f5b3be00640 1 -- 172.21.15.91:0/2218382835 >> v1:172.21.15.91:6789/0 conn(0x7f5b3c15a010 legacy=0x7f5b3c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.392+0000 7f5b3be00640 1 -- 172.21.15.91:0/2218382835 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c06c890 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.392+0000 7f5b3aa00640 1 -- 172.21.15.91:0/2218382835 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941694182 0 0) 0x7f5b3c0cc7c0 con 0x7f5b3c15a010 2024-02-02T13:27:29.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.393+0000 7f5b3be00640 1 --2- 172.21.15.91:0/2218382835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c9670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009d90 tx=0x7f5b3402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e9abdd26812375 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.393+0000 7f5b3aa00640 1 -- 172.21.15.91:0/2218382835 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402cbd0 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.393+0000 7f5b3aa00640 1 -- 172.21.15.91:0/2218382835 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3402cd30 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.393+0000 7f5b3aa00640 1 -- 172.21.15.91:0/2218382835 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402cea0 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 -- 172.21.15.91:0/2218382835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0b0e10 msgr2=0x7f5b3c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 --2- 172.21.15.91:0/2218382835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c9670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009d90 tx=0x7f5b3402be20 comp rx=0 tx=0).stop 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 -- 172.21.15.91:0/2218382835 shutdown_connections 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 --2- 172.21.15.91:0/2218382835 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c9670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 -- 172.21.15.91:0/2218382835 >> 172.21.15.91:0/2218382835 conn(0x7f5b3c0cec80 msgr2=0x7f5b3c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 -- 172.21.15.91:0/2218382835 shutdown_connections 2024-02-02T13:27:29.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.394+0000 7f5b42200640 1 -- 172.21.15.91:0/2218382835 wait complete. 2024-02-02T13:27:29.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.396+0000 7f5b42200640 1 Processor -- start 2024-02-02T13:27:29.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.396+0000 7f5b42200640 1 -- start start 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b3c15a010 0x7f5b3c14f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c0b1aa0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c150390 con 0x7f5b3c15a010 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b42200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c14faf0 con 0x7f5b3c0b0e10 2024-02-02T13:27:29.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44036/0 (socket says 172.21.15.91:44036) 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 -- 172.21.15.91:0/1031753562 learned_addr learned my addr 172.21.15.91:0/1031753562 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b3b400640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b3be00640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b3c15a010 0x7f5b3c14f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 -- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b3c0b0e10 msgr2=0x7f5b3c0c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 -- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b3c15a010 msgr2=0x7f5b3c14f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.397+0000 7f5b33e00640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b3c15a010 0x7f5b3c14f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b33e00640 1 -- 172.21.15.91:0/1031753562 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b34009a70 con 0x7f5b3c0ce130 2024-02-02T13:27:29.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b33e00640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c430 tx=0x7f5b2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24011790 con 0x7f5b3c0ce130 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b240118f0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24011ce0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c14fdd0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.398+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c0c8b00 con 0x7f5b3c0ce130 2024-02-02T13:27:29.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.399+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c066cf0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.400+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5b2401c460 con 0x7f5b3c0ce130 2024-02-02T13:27:29.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.401+0000 7f5b33400640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 0x7f5b1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.401+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b24088800 con 0x7f5b3c0ce130 2024-02-02T13:27:29.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.401+0000 7f5b3b400640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 0x7f5b1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.402+0000 7f5b3b400640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 0x7f5b1406c8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f60 tx=0x7f5b2c007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.406+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5b24053e80 con 0x7f5b3c0ce130 2024-02-02T13:27:29.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.632+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b3c0acb20 con 0x7f5b3c0ce130 2024-02-02T13:27:29.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.633+0000 7f5b33400640 1 -- 172.21.15.91:0/1031753562 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f5b240574a0 con 0x7f5b3c0ce130 2024-02-02T13:27:29.685 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:29.685 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:27:29.686 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:27:29.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 msgr2=0x7f5b1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 0x7f5b1406c8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f60 tx=0x7f5b2c007c00 comp rx=0 tx=0).stop 2024-02-02T13:27:29.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 msgr2=0x7f5b3c14efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c430 tx=0x7f5b2400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 shutdown_connections 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5b1406a430 0x7f5b1406c8f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b3c15a010 0x7f5b3c14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b3c0ce130 0x7f5b3c14efc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 --2- 172.21.15.91:0/1031753562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b3c0b0e10 0x7f5b3c0c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.635+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 >> 172.21.15.91:0/1031753562 conn(0x7f5b3c0cec80 msgr2=0x7f5b3c0c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.636+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 shutdown_connections 2024-02-02T13:27:29.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.636+0000 7f5b42200640 1 -- 172.21.15.91:0/1031753562 wait complete. 2024-02-02T13:27:29.687 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.899+0000 7f54c3c00640 1 Processor -- start 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.899+0000 7f54c3c00640 1 -- start start 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54bc06f4f0 0x7f54bc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 0x7f54bc0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc0b10d0 con 0x7f54bc06f4f0 2024-02-02T13:27:29.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc070f80 con 0x7f54bc0b0c40 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.900+0000 7f54c3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc152370 con 0x7f54bc151f90 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54bc06f4f0 0x7f54bc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54bc06f4f0 0x7f54bc0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57324/0 (socket says 172.21.15.91:57324) 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 -- 172.21.15.91:0/1979707489 learned_addr learned my addr 172.21.15.91:0/1979707489 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54c1200640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54c1c00640 1 --2- 172.21.15.91:0/1979707489 >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 0x7f54bc0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 -- 172.21.15.91:0/1979707489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 msgr2=0x7f54bc15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 -- 172.21.15.91:0/1979707489 >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 msgr2=0x7f54bc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1979707489 >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 0x7f54bc0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54bbe00640 1 -- 172.21.15.91:0/1979707489 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0cb010 con 0x7f54bc06f4f0 2024-02-02T13:27:29.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.901+0000 7f54c1c00640 1 --2- 172.21.15.91:0/1979707489 >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 0x7f54bc0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:29.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.902+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc06f4f0 0x7f54bc0b0700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0cfcd0 tx=0x7f54a402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60712c274a506371 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54bb400640 1 -- 172.21.15.91:0/1979707489 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4031860 con 0x7f54bc06f4f0 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54bb400640 1 -- 172.21.15.91:0/1979707489 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a40319c0 con 0x7f54bc06f4f0 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54bb400640 1 -- 172.21.15.91:0/1979707489 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4031d50 con 0x7f54bc06f4f0 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 -- 172.21.15.91:0/1979707489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc06f4f0 msgr2=0x7f54bc0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc06f4f0 0x7f54bc0b0700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0cfcd0 tx=0x7f54a402b900 comp rx=0 tx=0).stop 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 -- 172.21.15.91:0/1979707489 shutdown_connections 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1979707489 >> v2:172.21.15.113:3300/0 conn(0x7f54bc0b0c40 0x7f54bc0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc06f4f0 0x7f54bc0b0700 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1979707489 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 -- 172.21.15.91:0/1979707489 >> 172.21.15.91:0/1979707489 conn(0x7f54bc0cec80 msgr2=0x7f54bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.903+0000 7f54c3c00640 1 -- 172.21.15.91:0/1979707489 shutdown_connections 2024-02-02T13:27:29.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.904+0000 7f54c3c00640 1 -- 172.21.15.91:0/1979707489 wait complete. 2024-02-02T13:27:29.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.905+0000 7f54c3c00640 1 Processor -- start 2024-02-02T13:27:29.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.905+0000 7f54c3c00640 1 -- start start 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54bc06f4f0 0x7f54bc14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc14f160 con 0x7f54bc0b0c40 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc155300 con 0x7f54bc06f4f0 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.906+0000 7f54c3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0b0060 con 0x7f54bc151f90 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57334/0 (socket says 172.21.15.91:57334) 2024-02-02T13:27:29.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 -- 172.21.15.91:0/1741500217 learned_addr learned my addr 172.21.15.91:0/1741500217 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:29.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54c1c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 -- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 msgr2=0x7f54bc1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 -- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54bc06f4f0 msgr2=0x7f54bc14ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54c1200640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54bc06f4f0 0x7f54bc14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54bc06f4f0 0x7f54bc14ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54bbe00640 1 -- 172.21.15.91:0/1741500217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40094d0 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.907+0000 7f54c1c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54bbe00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f54a4000c00 tx=0x7f54a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a403ac60 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a403adc0 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4044610 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc1549d0 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.908+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc154f90 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.909+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc0c5cc0 con 0x7f54bc0b0c40 2024-02-02T13:27:29.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.910+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f54a404d450 con 0x7f54bc0b0c40 2024-02-02T13:27:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.911+0000 7f54b8c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 0x7f549406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.911+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54a40aafd0 con 0x7f54bc0b0c40 2024-02-02T13:27:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.911+0000 7f54c1200640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 0x7f549406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.912+0000 7f54c1200640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 0x7f549406c920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:29.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:29.916+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f54a4076490 con 0x7f54bc0b0c40 2024-02-02T13:27:30.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.106+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54bc066cf0 con 0x7f54bc0b0c40 2024-02-02T13:27:30.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.109+0000 7f54b8c00640 1 -- 172.21.15.91:0/1741500217 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f54a4079ab0 con 0x7f54bc0b0c40 2024-02-02T13:27:30.114 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:30.114 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":68,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:27.092655+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 msgr2=0x7f549406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 0x7f549406c920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac008040 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 msgr2=0x7f54bc153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f54a4000c00 tx=0x7f54a40046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 shutdown_connections 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549406a460 0x7f549406c920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54bc151f90 0x7f54bc1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54bc0b0c40 0x7f54bc153c70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.111+0000 7f54c3c00640 1 --2- 172.21.15.91:0/1741500217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54bc06f4f0 0x7f54bc14ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.112+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 >> 172.21.15.91:0/1741500217 conn(0x7f54bc0cec80 msgr2=0x7f54bc0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.112+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 shutdown_connections 2024-02-02T13:27:30.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.112+0000 7f54c3c00640 1 -- 172.21.15.91:0/1741500217 wait complete. 2024-02-02T13:27:30.134 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:27:30.134 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:27:30.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.342+0000 7fc4d7e00640 1 Processor -- start 2024-02-02T13:27:30.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.342+0000 7fc4d7e00640 1 -- start start 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.343+0000 7fc4d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4d815aec0 0x7fc4d80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d80ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d806c1c0 con 0x7fc4d815b2a0 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d7e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d806b5f0 con 0x7fc4d815aec0 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d806cd60 con 0x7fc4d806f3f0 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4d815aec0 0x7fc4d80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4d815aec0 0x7fc4d80cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57344/0 (socket says 172.21.15.91:57344) 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 -- 172.21.15.91:0/4261230700 learned_addr learned my addr 172.21.15.91:0/4261230700 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 -- 172.21.15.91:0/4261230700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 msgr2=0x7fc4d80ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 --2- 172.21.15.91:0/4261230700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d80ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 -- 172.21.15.91:0/4261230700 >> v1:172.21.15.113:6789/0 conn(0x7fc4d815b2a0 legacy=0x7fc4d806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:30.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.344+0000 7fc4d6a00640 1 -- 172.21.15.91:0/4261230700 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d806b890 con 0x7fc4d815aec0 2024-02-02T13:27:30.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.345+0000 7fc4d6a00640 1 --2- 172.21.15.91:0/4261230700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d80cf5b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009870 tx=0x7fc4c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eda458c5cccffd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.345+0000 7fc4d6000640 1 -- 172.21.15.91:0/4261230700 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8030840 con 0x7fc4d815aec0 2024-02-02T13:27:30.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d6000640 1 -- 172.21.15.91:0/4261230700 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c80309a0 con 0x7fc4d815aec0 2024-02-02T13:27:30.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d6000640 1 -- 172.21.15.91:0/4261230700 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8030d30 con 0x7fc4d815aec0 2024-02-02T13:27:30.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 -- 172.21.15.91:0/4261230700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 msgr2=0x7fc4d80cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/4261230700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d80cf5b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009870 tx=0x7fc4c802b980 comp rx=0 tx=0).stop 2024-02-02T13:27:30.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 -- 172.21.15.91:0/4261230700 shutdown_connections 2024-02-02T13:27:30.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/4261230700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d80ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/4261230700 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d80cf5b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 -- 172.21.15.91:0/4261230700 >> 172.21.15.91:0/4261230700 conn(0x7fc4d80cea90 msgr2=0x7fc4d80ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:30.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 -- 172.21.15.91:0/4261230700 shutdown_connections 2024-02-02T13:27:30.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.346+0000 7fc4d7e00640 1 -- 172.21.15.91:0/4261230700 wait complete. 2024-02-02T13:27:30.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.348+0000 7fc4d7e00640 1 Processor -- start 2024-02-02T13:27:30.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.348+0000 7fc4d7e00640 1 -- start start 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d81571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d815b2a0 0x7fc4d81f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81f4f50 con 0x7fc4d815aec0 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81f4340 con 0x7fc4d815b2a0 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81f4620 con 0x7fc4d806f3f0 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57346/0 (socket says 172.21.15.91:57346) 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.349+0000 7fc4cfe00640 1 -- 172.21.15.91:0/1523013519 learned_addr learned my addr 172.21.15.91:0/1523013519 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 -- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 msgr2=0x7fc4d81571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d81571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 -- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d815b2a0 msgr2=0x7fc4d81f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d815b2a0 0x7fc4d81f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 -- 172.21.15.91:0/1523013519 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80094d0 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cfe00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b3e0 tx=0x7fc4c000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0011830 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c00119b0 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0011d60 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d81f4900 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.350+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d8070cf0 con 0x7fc4d815aec0 2024-02-02T13:27:30.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.352+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d804dff0 con 0x7fc4d815aec0 2024-02-02T13:27:30.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.352+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc4c001a400 con 0x7fc4d815aec0 2024-02-02T13:27:30.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.354+0000 7fc4cf400640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 0x7fc4b006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.354+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4c0088bb0 con 0x7fc4d815aec0 2024-02-02T13:27:30.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.354+0000 7fc4d6a00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 0x7fc4b006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.355+0000 7fc4d6a00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 0x7fc4b006c920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8036000 tx=0x7fc4c803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.358+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc4c0054100 con 0x7fc4d815aec0 2024-02-02T13:27:30.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.545+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4d806be50 con 0x7fc4b006a460 2024-02-02T13:27:30.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.557+0000 7fc4cf400640 1 -- 172.21.15.91:0/1523013519 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282230 (secure 0 0 0) 0x7fc4c8030b90 con 0x7fc4b006a460 2024-02-02T13:27:30.562 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:30.564 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:27:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 msgr2=0x7fc4b006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 0x7fc4b006c920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8036000 tx=0x7fc4c803b040 comp rx=0 tx=0).stop 2024-02-02T13:27:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 msgr2=0x7fc4d815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b3e0 tx=0x7fc4c000b8a0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 shutdown_connections 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc4b006a460 0x7fc4b006c920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d815b2a0 0x7fc4d81f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4d815aec0 0x7fc4d815b6a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 --2- 172.21.15.91:0/1523013519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d806f3f0 0x7fc4d81571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 >> 172.21.15.91:0/1523013519 conn(0x7fc4d80cea90 msgr2=0x7fc4d80aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 shutdown_connections 2024-02-02T13:27:30.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.564+0000 7fc4d7e00640 1 -- 172.21.15.91:0/1523013519 wait complete. 2024-02-02T13:27:30.585 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-02-02T13:27:30.289609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558740,"statfs":{"total":1600000000,"available":1596150236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.657214"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.104327+0000","last_change":"2024-02-02T13:27:26.104151+0000","last_active":"2024-02-02T13:27:26.104327+0000","last_peered":"2024-02-02T13:27:26.104327+0000","last_clean":"2024-02-02T13:27:26.104327+0000","last_became_active":"2024-02-02T13:27:26.102947+0000","last_became_peered":"2024-02-02T13:27:26.102947+0000","last_unstale":"2024-02-02T13:27:26.104327+0000","last_undegraded":"2024-02-02T13:27:26.104327+0000","last_fullsized":"2024-02-02T13:27:26.104327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:40:46.350142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.107947+0000","last_change":"2024-02-02T13:27:26.107856+0000","last_active":"2024-02-02T13:27:26.107947+0000","last_peered":"2024-02-02T13:27:26.107947+0000","last_clean":"2024-02-02T13:27:26.107947+0000","last_became_active":"2024-02-02T13:27:26.106972+0000","last_became_peered":"2024-02-02T13:27:26.106972+0000","last_unstale":"2024-02-02T13:27:26.107947+0000","last_undegraded":"2024-02-02T13:27:26.107947+0000","last_fullsized":"2024-02-02T13:27:26.107947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:34.542970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122754+0000","last_change":"2024-02-02T13:27:26.117073+0000","last_active":"2024-02-02T13:27:26.122754+0000","last_peered":"2024-02-02T13:27:26.122754+0000","last_clean":"2024-02-02T13:27:26.122754+0000","last_became_active":"2024-02-02T13:27:26.116211+0000","last_became_peered":"2024-02-02T13:27:26.116211+0000","last_unstale":"2024-02-02T13:27:26.122754+0000","last_undegraded":"2024-02-02T13:27:26.122754+0000","last_fullsized":"2024-02-02T13:27:26.122754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:18:00.421164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120452+0000","last_change":"2024-02-02T13:27:26.119108+0000","last_active":"2024-02-02T13:27:26.120452+0000","last_peered":"2024-02-02T13:27:26.120452+0000","last_clean":"2024-02-02T13:27:26.120452+0000","last_became_active":"2024-02-02T13:27:26.112891+0000","last_became_peered":"2024-02-02T13:27:26.112891+0000","last_unstale":"2024-02-02T13:27:26.120452+0000","last_undegraded":"2024-02-02T13:27:26.120452+0000","last_fullsized":"2024-02-02T13:27:26.120452+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105067+0000","last_change":"2024-02-02T13:27:26.104951+0000","last_active":"2024-02-02T13:27:26.105067+0000","last_peered":"2024-02-02T13:27:26.105067+0000","last_clean":"2024-02-02T13:27:26.105067+0000","last_became_active":"2024-02-02T13:27:26.104593+0000","last_became_peered":"2024-02-02T13:27:26.104593+0000","last_unstale":"2024-02-02T13:27:26.105067+0000","last_undegraded":"2024-02-02T13:27:26.105067+0000","last_fullsized":"2024-02-02T13:27:26.105067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:57.564005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114672+0000","last_change":"2024-02-02T13:27:26.113047+0000","last_active":"2024-02-02T13:27:26.114672+0000","last_peered":"2024-02-02T13:27:26.114672+0000","last_clean":"2024-02-02T13:27:26.114672+0000","last_became_active":"2024-02-02T13:27:26.112790+0000","last_became_peered":"2024-02-02T13:27:26.112790+0000","last_unstale":"2024-02-02T13:27:26.114672+0000","last_undegraded":"2024-02-02T13:27:26.114672+0000","last_fullsized":"2024-02-02T13:27:26.114672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:25.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121475+0000","last_change":"2024-02-02T13:27:26.118900+0000","last_active":"2024-02-02T13:27:26.121475+0000","last_peered":"2024-02-02T13:27:26.121475+0000","last_clean":"2024-02-02T13:27:26.121475+0000","last_became_active":"2024-02-02T13:27:26.118509+0000","last_became_peered":"2024-02-02T13:27:26.118509+0000","last_unstale":"2024-02-02T13:27:26.121475+0000","last_undegraded":"2024-02-02T13:27:26.121475+0000","last_fullsized":"2024-02-02T13:27:26.121475+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:50.453938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.107962+0000","last_change":"2024-02-02T13:27:26.107878+0000","last_active":"2024-02-02T13:27:26.107962+0000","last_peered":"2024-02-02T13:27:26.107962+0000","last_clean":"2024-02-02T13:27:26.107962+0000","last_became_active":"2024-02-02T13:27:26.107654+0000","last_became_peered":"2024-02-02T13:27:26.107654+0000","last_unstale":"2024-02-02T13:27:26.107962+0000","last_undegraded":"2024-02-02T13:27:26.107962+0000","last_fullsized":"2024-02-02T13:27:26.107962+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121431+0000","last_change":"2024-02-02T13:27:26.118746+0000","last_active":"2024-02-02T13:27:26.121431+0000","last_peered":"2024-02-02T13:27:26.121431+0000","last_clean":"2024-02-02T13:27:26.121431+0000","last_became_active":"2024-02-02T13:27:26.118070+0000","last_became_peered":"2024-02-02T13:27:26.118070+0000","last_unstale":"2024-02-02T13:27:26.121431+0000","last_undegraded":"2024-02-02T13:27:26.121431+0000","last_fullsized":"2024-02-02T13:27:26.121431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:17.584842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127416+0000","last_change":"2024-02-02T13:27:26.127346+0000","last_active":"2024-02-02T13:27:26.127416+0000","last_peered":"2024-02-02T13:27:26.127416+0000","last_clean":"2024-02-02T13:27:26.127416+0000","last_became_active":"2024-02-02T13:27:26.127116+0000","last_became_peered":"2024-02-02T13:27:26.127116+0000","last_unstale":"2024-02-02T13:27:26.127416+0000","last_undegraded":"2024-02-02T13:27:26.127416+0000","last_fullsized":"2024-02-02T13:27:26.127416+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:52.722156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.108042+0000","last_change":"2024-02-02T13:27:26.107979+0000","last_active":"2024-02-02T13:27:26.108042+0000","last_peered":"2024-02-02T13:27:26.108042+0000","last_clean":"2024-02-02T13:27:26.108042+0000","last_became_active":"2024-02-02T13:27:26.107752+0000","last_became_peered":"2024-02-02T13:27:26.107752+0000","last_unstale":"2024-02-02T13:27:26.108042+0000","last_undegraded":"2024-02-02T13:27:26.108042+0000","last_fullsized":"2024-02-02T13:27:26.108042+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:32.309386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120489+0000","last_change":"2024-02-02T13:27:26.119194+0000","last_active":"2024-02-02T13:27:26.120489+0000","last_peered":"2024-02-02T13:27:26.120489+0000","last_clean":"2024-02-02T13:27:26.120489+0000","last_became_active":"2024-02-02T13:27:26.118830+0000","last_became_peered":"2024-02-02T13:27:26.118830+0000","last_unstale":"2024-02-02T13:27:26.120489+0000","last_undegraded":"2024-02-02T13:27:26.120489+0000","last_fullsized":"2024-02-02T13:27:26.120489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126850+0000","last_change":"2024-02-02T13:27:26.124370+0000","last_active":"2024-02-02T13:27:26.126850+0000","last_peered":"2024-02-02T13:27:26.126850+0000","last_clean":"2024-02-02T13:27:26.126850+0000","last_became_active":"2024-02-02T13:27:26.124005+0000","last_became_peered":"2024-02-02T13:27:26.124005+0000","last_unstale":"2024-02-02T13:27:26.126850+0000","last_undegraded":"2024-02-02T13:27:26.126850+0000","last_fullsized":"2024-02-02T13:27:26.126850+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:47.487227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110124+0000","last_change":"2024-02-02T13:27:26.109122+0000","last_active":"2024-02-02T13:27:26.110124+0000","last_peered":"2024-02-02T13:27:26.110124+0000","last_clean":"2024-02-02T13:27:26.110124+0000","last_became_active":"2024-02-02T13:27:26.108546+0000","last_became_peered":"2024-02-02T13:27:26.108546+0000","last_unstale":"2024-02-02T13:27:26.110124+0000","last_undegraded":"2024-02-02T13:27:26.110124+0000","last_fullsized":"2024-02-02T13:27:26.110124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:42:33.770919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110168+0000","last_change":"2024-02-02T13:27:26.109683+0000","last_active":"2024-02-02T13:27:26.110168+0000","last_peered":"2024-02-02T13:27:26.110168+0000","last_clean":"2024-02-02T13:27:26.110168+0000","last_became_active":"2024-02-02T13:27:26.109023+0000","last_became_peered":"2024-02-02T13:27:26.109023+0000","last_unstale":"2024-02-02T13:27:26.110168+0000","last_undegraded":"2024-02-02T13:27:26.110168+0000","last_fullsized":"2024-02-02T13:27:26.110168+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:15.560754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124273+0000","last_change":"2024-02-02T13:27:26.124214+0000","last_active":"2024-02-02T13:27:26.124273+0000","last_peered":"2024-02-02T13:27:26.124273+0000","last_clean":"2024-02-02T13:27:26.124273+0000","last_became_active":"2024-02-02T13:27:26.123994+0000","last_became_peered":"2024-02-02T13:27:26.123994+0000","last_unstale":"2024-02-02T13:27:26.124273+0000","last_undegraded":"2024-02-02T13:27:26.124273+0000","last_fullsized":"2024-02-02T13:27:26.124273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124437+0000","last_change":"2024-02-02T13:27:26.124370+0000","last_active":"2024-02-02T13:27:26.124437+0000","last_peered":"2024-02-02T13:27:26.124437+0000","last_clean":"2024-02-02T13:27:26.124437+0000","last_became_active":"2024-02-02T13:27:26.124136+0000","last_became_peered":"2024-02-02T13:27:26.124136+0000","last_unstale":"2024-02-02T13:27:26.124437+0000","last_undegraded":"2024-02-02T13:27:26.124437+0000","last_fullsized":"2024-02-02T13:27:26.124437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:59:54.602035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124691+0000","last_change":"2024-02-02T13:27:26.123390+0000","last_active":"2024-02-02T13:27:26.124691+0000","last_peered":"2024-02-02T13:27:26.124691+0000","last_clean":"2024-02-02T13:27:26.124691+0000","last_became_active":"2024-02-02T13:27:26.122978+0000","last_became_peered":"2024-02-02T13:27:26.122978+0000","last_unstale":"2024-02-02T13:27:26.124691+0000","last_undegraded":"2024-02-02T13:27:26.124691+0000","last_fullsized":"2024-02-02T13:27:26.124691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:14:31.975198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.104306+0000","last_change":"2024-02-02T13:27:26.103837+0000","last_active":"2024-02-02T13:27:26.104306+0000","last_peered":"2024-02-02T13:27:26.104306+0000","last_clean":"2024-02-02T13:27:26.104306+0000","last_became_active":"2024-02-02T13:27:26.103628+0000","last_became_peered":"2024-02-02T13:27:26.103628+0000","last_unstale":"2024-02-02T13:27:26.104306+0000","last_undegraded":"2024-02-02T13:27:26.104306+0000","last_fullsized":"2024-02-02T13:27:26.104306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:40.405342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114896+0000","last_change":"2024-02-02T13:27:26.114768+0000","last_active":"2024-02-02T13:27:26.114896+0000","last_peered":"2024-02-02T13:27:26.114896+0000","last_clean":"2024-02-02T13:27:26.114896+0000","last_became_active":"2024-02-02T13:27:26.114555+0000","last_became_peered":"2024-02-02T13:27:26.114555+0000","last_unstale":"2024-02-02T13:27:26.114896+0000","last_undegraded":"2024-02-02T13:27:26.114896+0000","last_fullsized":"2024-02-02T13:27:26.114896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:03:41.987049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126696+0000","last_change":"2024-02-02T13:27:26.126133+0000","last_active":"2024-02-02T13:27:26.126696+0000","last_peered":"2024-02-02T13:27:26.126696+0000","last_clean":"2024-02-02T13:27:26.126696+0000","last_became_active":"2024-02-02T13:27:26.125689+0000","last_became_peered":"2024-02-02T13:27:26.125689+0000","last_unstale":"2024-02-02T13:27:26.126696+0000","last_undegraded":"2024-02-02T13:27:26.126696+0000","last_fullsized":"2024-02-02T13:27:26.126696+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:23.851535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103155+0000","last_change":"2024-02-02T13:27:26.103084+0000","last_active":"2024-02-02T13:27:26.103155+0000","last_peered":"2024-02-02T13:27:26.103155+0000","last_clean":"2024-02-02T13:27:26.103155+0000","last_became_active":"2024-02-02T13:27:26.102874+0000","last_became_peered":"2024-02-02T13:27:26.102874+0000","last_unstale":"2024-02-02T13:27:26.103155+0000","last_undegraded":"2024-02-02T13:27:26.103155+0000","last_fullsized":"2024-02-02T13:27:26.103155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:16:58.378414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115929+0000","last_change":"2024-02-02T13:27:26.115822+0000","last_active":"2024-02-02T13:27:26.115929+0000","last_peered":"2024-02-02T13:27:26.115929+0000","last_clean":"2024-02-02T13:27:26.115929+0000","last_became_active":"2024-02-02T13:27:26.115466+0000","last_became_peered":"2024-02-02T13:27:26.115466+0000","last_unstale":"2024-02-02T13:27:26.115929+0000","last_undegraded":"2024-02-02T13:27:26.115929+0000","last_fullsized":"2024-02-02T13:27:26.115929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126869+0000","last_change":"2024-02-02T13:27:26.125968+0000","last_active":"2024-02-02T13:27:26.126869+0000","last_peered":"2024-02-02T13:27:26.126869+0000","last_clean":"2024-02-02T13:27:26.126869+0000","last_became_active":"2024-02-02T13:27:26.125646+0000","last_became_peered":"2024-02-02T13:27:26.125646+0000","last_unstale":"2024-02-02T13:27:26.126869+0000","last_undegraded":"2024-02-02T13:27:26.126869+0000","last_fullsized":"2024-02-02T13:27:26.126869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:14.418710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101459+0000","last_change":"2024-02-02T13:27:26.101268+0000","last_active":"2024-02-02T13:27:26.101459+0000","last_peered":"2024-02-02T13:27:26.101459+0000","last_clean":"2024-02-02T13:27:26.101459+0000","last_became_active":"2024-02-02T13:27:26.101044+0000","last_became_peered":"2024-02-02T13:27:26.101044+0000","last_unstale":"2024-02-02T13:27:26.101459+0000","last_undegraded":"2024-02-02T13:27:26.101459+0000","last_fullsized":"2024-02-02T13:27:26.101459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:00.524308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120580+0000","last_change":"2024-02-02T13:27:26.120174+0000","last_active":"2024-02-02T13:27:26.120580+0000","last_peered":"2024-02-02T13:27:26.120580+0000","last_clean":"2024-02-02T13:27:26.120580+0000","last_became_active":"2024-02-02T13:27:26.118598+0000","last_became_peered":"2024-02-02T13:27:26.118598+0000","last_unstale":"2024-02-02T13:27:26.120580+0000","last_undegraded":"2024-02-02T13:27:26.120580+0000","last_fullsized":"2024-02-02T13:27:26.120580+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.123886+0000","last_change":"2024-02-02T13:27:26.123815+0000","last_active":"2024-02-02T13:27:26.123886+0000","last_peered":"2024-02-02T13:27:26.123886+0000","last_clean":"2024-02-02T13:27:26.123886+0000","last_became_active":"2024-02-02T13:27:26.123643+0000","last_became_peered":"2024-02-02T13:27:26.123643+0000","last_unstale":"2024-02-02T13:27:26.123886+0000","last_undegraded":"2024-02-02T13:27:26.123886+0000","last_fullsized":"2024-02-02T13:27:26.123886+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105042+0000","last_change":"2024-02-02T13:27:26.104849+0000","last_active":"2024-02-02T13:27:26.105042+0000","last_peered":"2024-02-02T13:27:26.105042+0000","last_clean":"2024-02-02T13:27:26.105042+0000","last_became_active":"2024-02-02T13:27:26.104597+0000","last_became_peered":"2024-02-02T13:27:26.104597+0000","last_unstale":"2024-02-02T13:27:26.105042+0000","last_undegraded":"2024-02-02T13:27:26.105042+0000","last_fullsized":"2024-02-02T13:27:26.105042+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:46:08.882471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.106564+0000","last_change":"2024-02-02T13:27:26.106491+0000","last_active":"2024-02-02T13:27:26.106564+0000","last_peered":"2024-02-02T13:27:26.106564+0000","last_clean":"2024-02-02T13:27:26.106564+0000","last_became_active":"2024-02-02T13:27:26.104134+0000","last_became_peered":"2024-02-02T13:27:26.104134+0000","last_unstale":"2024-02-02T13:27:26.106564+0000","last_undegraded":"2024-02-02T13:27:26.106564+0000","last_fullsized":"2024-02-02T13:27:26.106564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:20:25.099877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122679+0000","last_change":"2024-02-02T13:27:26.121973+0000","last_active":"2024-02-02T13:27:26.122679+0000","last_peered":"2024-02-02T13:27:26.122679+0000","last_clean":"2024-02-02T13:27:26.122679+0000","last_became_active":"2024-02-02T13:27:26.120842+0000","last_became_peered":"2024-02-02T13:27:26.120842+0000","last_unstale":"2024-02-02T13:27:26.122679+0000","last_undegraded":"2024-02-02T13:27:26.122679+0000","last_fullsized":"2024-02-02T13:27:26.122679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:49.348655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105543+0000","last_change":"2024-02-02T13:27:26.105374+0000","last_active":"2024-02-02T13:27:26.105543+0000","last_peered":"2024-02-02T13:27:26.105543+0000","last_clean":"2024-02-02T13:27:26.105543+0000","last_became_active":"2024-02-02T13:27:26.105169+0000","last_became_peered":"2024-02-02T13:27:26.105169+0000","last_unstale":"2024-02-02T13:27:26.105543+0000","last_undegraded":"2024-02-02T13:27:26.105543+0000","last_fullsized":"2024-02-02T13:27:26.105543+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:31:17.688156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114712+0000","last_change":"2024-02-02T13:27:26.113559+0000","last_active":"2024-02-02T13:27:26.114712+0000","last_peered":"2024-02-02T13:27:26.114712+0000","last_clean":"2024-02-02T13:27:26.114712+0000","last_became_active":"2024-02-02T13:27:26.113297+0000","last_became_peered":"2024-02-02T13:27:26.113297+0000","last_unstale":"2024-02-02T13:27:26.114712+0000","last_undegraded":"2024-02-02T13:27:26.114712+0000","last_fullsized":"2024-02-02T13:27:26.114712+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:23.851535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117777+0000","last_change":"2024-02-02T13:27:26.117717+0000","last_active":"2024-02-02T13:27:26.117777+0000","last_peered":"2024-02-02T13:27:26.117777+0000","last_clean":"2024-02-02T13:27:26.117777+0000","last_became_active":"2024-02-02T13:27:26.117487+0000","last_became_peered":"2024-02-02T13:27:26.117487+0000","last_unstale":"2024-02-02T13:27:26.117777+0000","last_undegraded":"2024-02-02T13:27:26.117777+0000","last_fullsized":"2024-02-02T13:27:26.117777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:29.407482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122547+0000","last_change":"2024-02-02T13:27:25.666042+0000","last_active":"2024-02-02T13:27:26.122547+0000","last_peered":"2024-02-02T13:27:26.122547+0000","last_clean":"2024-02-02T13:27:26.122547+0000","last_became_active":"2024-02-02T13:27:25.665568+0000","last_became_peered":"2024-02-02T13:27:25.665568+0000","last_unstale":"2024-02-02T13:27:26.122547+0000","last_undegraded":"2024-02-02T13:27:26.122547+0000","last_fullsized":"2024-02-02T13:27:26.122547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:14.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117327+0000","last_change":"2024-02-02T13:27:26.117162+0000","last_active":"2024-02-02T13:27:26.117327+0000","last_peered":"2024-02-02T13:27:26.117327+0000","last_clean":"2024-02-02T13:27:26.117327+0000","last_became_active":"2024-02-02T13:27:26.116939+0000","last_became_peered":"2024-02-02T13:27:26.116939+0000","last_unstale":"2024-02-02T13:27:26.117327+0000","last_undegraded":"2024-02-02T13:27:26.117327+0000","last_fullsized":"2024-02-02T13:27:26.117327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:45.570715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124007+0000","last_change":"2024-02-02T13:27:25.662294+0000","last_active":"2024-02-02T13:27:26.124007+0000","last_peered":"2024-02-02T13:27:26.124007+0000","last_clean":"2024-02-02T13:27:26.124007+0000","last_became_active":"2024-02-02T13:27:25.661816+0000","last_became_peered":"2024-02-02T13:27:25.661816+0000","last_unstale":"2024-02-02T13:27:26.124007+0000","last_undegraded":"2024-02-02T13:27:26.124007+0000","last_fullsized":"2024-02-02T13:27:26.124007+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:56.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116570+0000","last_change":"2024-02-02T13:27:26.116515+0000","last_active":"2024-02-02T13:27:26.116570+0000","last_peered":"2024-02-02T13:27:26.116570+0000","last_clean":"2024-02-02T13:27:26.116570+0000","last_became_active":"2024-02-02T13:27:26.116267+0000","last_became_peered":"2024-02-02T13:27:26.116267+0000","last_unstale":"2024-02-02T13:27:26.116570+0000","last_undegraded":"2024-02-02T13:27:26.116570+0000","last_fullsized":"2024-02-02T13:27:26.116570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:49.348655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.118886+0000","last_change":"2024-02-02T13:27:25.665068+0000","last_active":"2024-02-02T13:27:26.118886+0000","last_peered":"2024-02-02T13:27:26.118886+0000","last_clean":"2024-02-02T13:27:26.118886+0000","last_became_active":"2024-02-02T13:27:25.664511+0000","last_became_peered":"2024-02-02T13:27:25.664511+0000","last_unstale":"2024-02-02T13:27:26.118886+0000","last_undegraded":"2024-02-02T13:27:26.118886+0000","last_fullsized":"2024-02-02T13:27:26.118886+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:44.560009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115953+0000","last_change":"2024-02-02T13:27:26.115881+0000","last_active":"2024-02-02T13:27:26.115953+0000","last_peered":"2024-02-02T13:27:26.115953+0000","last_clean":"2024-02-02T13:27:26.115953+0000","last_became_active":"2024-02-02T13:27:26.115461+0000","last_became_peered":"2024-02-02T13:27:26.115461+0000","last_unstale":"2024-02-02T13:27:26.115953+0000","last_undegraded":"2024-02-02T13:27:26.115953+0000","last_fullsized":"2024-02-02T13:27:26.115953+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127622+0000","last_change":"2024-02-02T13:27:25.669597+0000","last_active":"2024-02-02T13:27:26.127622+0000","last_peered":"2024-02-02T13:27:26.127622+0000","last_clean":"2024-02-02T13:27:26.127622+0000","last_became_active":"2024-02-02T13:27:25.668620+0000","last_became_peered":"2024-02-02T13:27:25.668620+0000","last_unstale":"2024-02-02T13:27:26.127622+0000","last_undegraded":"2024-02-02T13:27:26.127622+0000","last_fullsized":"2024-02-02T13:27:26.127622+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:16.574135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103791+0000","last_change":"2024-02-02T13:27:26.103729+0000","last_active":"2024-02-02T13:27:26.103791+0000","last_peered":"2024-02-02T13:27:26.103791+0000","last_clean":"2024-02-02T13:27:26.103791+0000","last_became_active":"2024-02-02T13:27:26.103510+0000","last_became_peered":"2024-02-02T13:27:26.103510+0000","last_unstale":"2024-02-02T13:27:26.103791+0000","last_undegraded":"2024-02-02T13:27:26.103791+0000","last_fullsized":"2024-02-02T13:27:26.103791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:17.038957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.118127+0000","last_change":"2024-02-02T13:27:25.665909+0000","last_active":"2024-02-02T13:27:26.118127+0000","last_peered":"2024-02-02T13:27:26.118127+0000","last_clean":"2024-02-02T13:27:26.118127+0000","last_became_active":"2024-02-02T13:27:25.665647+0000","last_became_peered":"2024-02-02T13:27:25.665647+0000","last_unstale":"2024-02-02T13:27:26.118127+0000","last_undegraded":"2024-02-02T13:27:26.118127+0000","last_fullsized":"2024-02-02T13:27:26.118127+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:13.408003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115392+0000","last_change":"2024-02-02T13:27:26.115206+0000","last_active":"2024-02-02T13:27:26.115392+0000","last_peered":"2024-02-02T13:27:26.115392+0000","last_clean":"2024-02-02T13:27:26.115392+0000","last_became_active":"2024-02-02T13:27:26.114887+0000","last_became_peered":"2024-02-02T13:27:26.114887+0000","last_unstale":"2024-02-02T13:27:26.115392+0000","last_undegraded":"2024-02-02T13:27:26.115392+0000","last_fullsized":"2024-02-02T13:27:26.115392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:42:18.679208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.123593+0000","last_change":"2024-02-02T13:27:25.665273+0000","last_active":"2024-02-02T13:27:26.123593+0000","last_peered":"2024-02-02T13:27:26.123593+0000","last_clean":"2024-02-02T13:27:26.123593+0000","last_became_active":"2024-02-02T13:27:25.664723+0000","last_became_peered":"2024-02-02T13:27:25.664723+0000","last_unstale":"2024-02-02T13:27:26.123593+0000","last_undegraded":"2024-02-02T13:27:26.123593+0000","last_fullsized":"2024-02-02T13:27:26.123593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:16.574135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126885+0000","last_change":"2024-02-02T13:27:26.126739+0000","last_active":"2024-02-02T13:27:26.126885+0000","last_peered":"2024-02-02T13:27:26.126885+0000","last_clean":"2024-02-02T13:27:26.126885+0000","last_became_active":"2024-02-02T13:27:26.126412+0000","last_became_peered":"2024-02-02T13:27:26.126412+0000","last_unstale":"2024-02-02T13:27:26.126885+0000","last_undegraded":"2024-02-02T13:27:26.126885+0000","last_fullsized":"2024-02-02T13:27:26.126885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:02.103857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101153+0000","last_change":"2024-02-02T13:27:25.669110+0000","last_active":"2024-02-02T13:27:26.101153+0000","last_peered":"2024-02-02T13:27:26.101153+0000","last_clean":"2024-02-02T13:27:26.101153+0000","last_became_active":"2024-02-02T13:27:25.668648+0000","last_became_peered":"2024-02-02T13:27:25.668648+0000","last_unstale":"2024-02-02T13:27:26.101153+0000","last_undegraded":"2024-02-02T13:27:26.101153+0000","last_fullsized":"2024-02-02T13:27:26.101153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:14.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127669+0000","last_change":"2024-02-02T13:27:26.127605+0000","last_active":"2024-02-02T13:27:26.127669+0000","last_peered":"2024-02-02T13:27:26.127669+0000","last_clean":"2024-02-02T13:27:26.127669+0000","last_became_active":"2024-02-02T13:27:26.127381+0000","last_became_peered":"2024-02-02T13:27:26.127381+0000","last_unstale":"2024-02-02T13:27:26.127669+0000","last_undegraded":"2024-02-02T13:27:26.127669+0000","last_fullsized":"2024-02-02T13:27:26.127669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:05.755303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103063+0000","last_change":"2024-02-02T13:27:25.666396+0000","last_active":"2024-02-02T13:27:26.103063+0000","last_peered":"2024-02-02T13:27:26.103063+0000","last_clean":"2024-02-02T13:27:26.103063+0000","last_became_active":"2024-02-02T13:27:25.664470+0000","last_became_peered":"2024-02-02T13:27:25.664470+0000","last_unstale":"2024-02-02T13:27:26.103063+0000","last_undegraded":"2024-02-02T13:27:26.103063+0000","last_fullsized":"2024-02-02T13:27:26.103063+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:01.093150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114673+0000","last_change":"2024-02-02T13:27:26.111554+0000","last_active":"2024-02-02T13:27:26.114673+0000","last_peered":"2024-02-02T13:27:26.114673+0000","last_clean":"2024-02-02T13:27:26.114673+0000","last_became_active":"2024-02-02T13:27:26.110890+0000","last_became_peered":"2024-02-02T13:27:26.110890+0000","last_unstale":"2024-02-02T13:27:26.114673+0000","last_undegraded":"2024-02-02T13:27:26.114673+0000","last_fullsized":"2024-02-02T13:27:26.114673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:09.075185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113750+0000","last_change":"2024-02-02T13:27:25.666478+0000","last_active":"2024-02-02T13:27:26.113750+0000","last_peered":"2024-02-02T13:27:26.113750+0000","last_clean":"2024-02-02T13:27:26.113750+0000","last_became_active":"2024-02-02T13:27:25.665736+0000","last_became_peered":"2024-02-02T13:27:25.665736+0000","last_unstale":"2024-02-02T13:27:26.113750+0000","last_undegraded":"2024-02-02T13:27:26.113750+0000","last_fullsized":"2024-02-02T13:27:26.113750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122718+0000","last_change":"2024-02-02T13:27:26.122308+0000","last_active":"2024-02-02T13:27:26.122718+0000","last_peered":"2024-02-02T13:27:26.122718+0000","last_clean":"2024-02-02T13:27:26.122718+0000","last_became_active":"2024-02-02T13:27:26.121743+0000","last_became_peered":"2024-02-02T13:27:26.121743+0000","last_unstale":"2024-02-02T13:27:26.122718+0000","last_undegraded":"2024-02-02T13:27:26.122718+0000","last_fullsized":"2024-02-02T13:27:26.122718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:04:06.503908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.099472+0000","last_change":"2024-02-02T13:27:25.669823+0000","last_active":"2024-02-02T13:27:26.099472+0000","last_peered":"2024-02-02T13:27:26.099472+0000","last_clean":"2024-02-02T13:27:26.099472+0000","last_became_active":"2024-02-02T13:27:25.669597+0000","last_became_peered":"2024-02-02T13:27:25.669597+0000","last_unstale":"2024-02-02T13:27:26.099472+0000","last_undegraded":"2024-02-02T13:27:26.099472+0000","last_fullsized":"2024-02-02T13:27:26.099472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:28.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113388+0000","last_change":"2024-02-02T13:27:26.112364+0000","last_active":"2024-02-02T13:27:26.113388+0000","last_peered":"2024-02-02T13:27:26.113388+0000","last_clean":"2024-02-02T13:27:26.113388+0000","last_became_active":"2024-02-02T13:27:26.112156+0000","last_became_peered":"2024-02-02T13:27:26.112156+0000","last_unstale":"2024-02-02T13:27:26.113388+0000","last_undegraded":"2024-02-02T13:27:26.113388+0000","last_fullsized":"2024-02-02T13:27:26.113388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:52.843663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.131479+0000","last_change":"2024-02-02T13:27:25.651369+0000","last_active":"2024-02-02T13:27:26.131479+0000","last_peered":"2024-02-02T13:27:26.131479+0000","last_clean":"2024-02-02T13:27:26.131479+0000","last_became_active":"2024-02-02T13:27:25.650186+0000","last_became_peered":"2024-02-02T13:27:25.650186+0000","last_unstale":"2024-02-02T13:27:26.131479+0000","last_undegraded":"2024-02-02T13:27:26.131479+0000","last_fullsized":"2024-02-02T13:27:26.131479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:31.298679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122815+0000","last_change":"2024-02-02T13:27:26.122729+0000","last_active":"2024-02-02T13:27:26.122815+0000","last_peered":"2024-02-02T13:27:26.122815+0000","last_clean":"2024-02-02T13:27:26.122815+0000","last_became_active":"2024-02-02T13:27:26.116728+0000","last_became_peered":"2024-02-02T13:27:26.116728+0000","last_unstale":"2024-02-02T13:27:26.122815+0000","last_undegraded":"2024-02-02T13:27:26.122815+0000","last_fullsized":"2024-02-02T13:27:26.122815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127797+0000","last_change":"2024-02-02T13:27:25.663185+0000","last_active":"2024-02-02T13:27:26.127797+0000","last_peered":"2024-02-02T13:27:26.127797+0000","last_clean":"2024-02-02T13:27:26.127797+0000","last_became_active":"2024-02-02T13:27:25.662468+0000","last_became_peered":"2024-02-02T13:27:25.662468+0000","last_unstale":"2024-02-02T13:27:26.127797+0000","last_undegraded":"2024-02-02T13:27:26.127797+0000","last_fullsized":"2024-02-02T13:27:26.127797+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:15.800371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120560+0000","last_change":"2024-02-02T13:27:26.120001+0000","last_active":"2024-02-02T13:27:26.120560+0000","last_peered":"2024-02-02T13:27:26.120560+0000","last_clean":"2024-02-02T13:27:26.120560+0000","last_became_active":"2024-02-02T13:27:26.116707+0000","last_became_peered":"2024-02-02T13:27:26.116707+0000","last_unstale":"2024-02-02T13:27:26.120560+0000","last_undegraded":"2024-02-02T13:27:26.120560+0000","last_fullsized":"2024-02-02T13:27:26.120560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115365+0000","last_change":"2024-02-02T13:27:25.669498+0000","last_active":"2024-02-02T13:27:26.115365+0000","last_peered":"2024-02-02T13:27:26.115365+0000","last_clean":"2024-02-02T13:27:26.115365+0000","last_became_active":"2024-02-02T13:27:25.669233+0000","last_became_peered":"2024-02-02T13:27:25.669233+0000","last_unstale":"2024-02-02T13:27:26.115365+0000","last_undegraded":"2024-02-02T13:27:26.115365+0000","last_fullsized":"2024-02-02T13:27:26.115365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:26.481824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.129021+0000","last_change":"2024-02-02T13:27:26.128706+0000","last_active":"2024-02-02T13:27:26.129021+0000","last_peered":"2024-02-02T13:27:26.129021+0000","last_clean":"2024-02-02T13:27:26.129021+0000","last_became_active":"2024-02-02T13:27:26.128328+0000","last_became_peered":"2024-02-02T13:27:26.128328+0000","last_unstale":"2024-02-02T13:27:26.129021+0000","last_undegraded":"2024-02-02T13:27:26.129021+0000","last_fullsized":"2024-02-02T13:27:26.129021+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:25.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122777+0000","last_change":"2024-02-02T13:27:25.661775+0000","last_active":"2024-02-02T13:27:26.122777+0000","last_peered":"2024-02-02T13:27:26.122777+0000","last_clean":"2024-02-02T13:27:26.122777+0000","last_became_active":"2024-02-02T13:27:25.661539+0000","last_became_peered":"2024-02-02T13:27:25.661539+0000","last_unstale":"2024-02-02T13:27:26.122777+0000","last_undegraded":"2024-02-02T13:27:26.122777+0000","last_fullsized":"2024-02-02T13:27:26.122777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:27.586262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116200+0000","last_change":"2024-02-02T13:27:26.115961+0000","last_active":"2024-02-02T13:27:26.116200+0000","last_peered":"2024-02-02T13:27:26.116200+0000","last_clean":"2024-02-02T13:27:26.116200+0000","last_became_active":"2024-02-02T13:27:26.115696+0000","last_became_peered":"2024-02-02T13:27:26.115696+0000","last_unstale":"2024-02-02T13:27:26.116200+0000","last_undegraded":"2024-02-02T13:27:26.116200+0000","last_fullsized":"2024-02-02T13:27:26.116200+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:53.881018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.099377+0000","last_change":"2024-02-02T13:27:25.665929+0000","last_active":"2024-02-02T13:27:26.099377+0000","last_peered":"2024-02-02T13:27:26.099377+0000","last_clean":"2024-02-02T13:27:26.099377+0000","last_became_active":"2024-02-02T13:27:25.665707+0000","last_became_peered":"2024-02-02T13:27:25.665707+0000","last_unstale":"2024-02-02T13:27:26.099377+0000","last_undegraded":"2024-02-02T13:27:26.099377+0000","last_fullsized":"2024-02-02T13:27:26.099377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:31.298679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126663+0000","last_change":"2024-02-02T13:27:26.125559+0000","last_active":"2024-02-02T13:27:26.126663+0000","last_peered":"2024-02-02T13:27:26.126663+0000","last_clean":"2024-02-02T13:27:26.126663+0000","last_became_active":"2024-02-02T13:27:26.125274+0000","last_became_peered":"2024-02-02T13:27:26.125274+0000","last_unstale":"2024-02-02T13:27:26.126663+0000","last_undegraded":"2024-02-02T13:27:26.126663+0000","last_fullsized":"2024-02-02T13:27:26.126663+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.130842+0000","last_change":"2024-02-02T13:27:25.667108+0000","last_active":"2024-02-02T13:27:26.130842+0000","last_peered":"2024-02-02T13:27:26.130842+0000","last_clean":"2024-02-02T13:27:26.130842+0000","last_became_active":"2024-02-02T13:27:25.665605+0000","last_became_peered":"2024-02-02T13:27:25.665605+0000","last_unstale":"2024-02-02T13:27:26.130842+0000","last_undegraded":"2024-02-02T13:27:26.130842+0000","last_fullsized":"2024-02-02T13:27:26.130842+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:52.870311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122700+0000","last_change":"2024-02-02T13:27:26.122160+0000","last_active":"2024-02-02T13:27:26.122700+0000","last_peered":"2024-02-02T13:27:26.122700+0000","last_clean":"2024-02-02T13:27:26.122700+0000","last_became_active":"2024-02-02T13:27:26.121180+0000","last_became_peered":"2024-02-02T13:27:26.121180+0000","last_unstale":"2024-02-02T13:27:26.122700+0000","last_undegraded":"2024-02-02T13:27:26.122700+0000","last_fullsized":"2024-02-02T13:27:26.122700+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114044+0000","last_change":"2024-02-02T13:27:25.668931+0000","last_active":"2024-02-02T13:27:26.114044+0000","last_peered":"2024-02-02T13:27:26.114044+0000","last_clean":"2024-02-02T13:27:26.114044+0000","last_became_active":"2024-02-02T13:27:25.668721+0000","last_became_peered":"2024-02-02T13:27:25.668721+0000","last_unstale":"2024-02-02T13:27:26.114044+0000","last_undegraded":"2024-02-02T13:27:26.114044+0000","last_fullsized":"2024-02-02T13:27:26.114044+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:10.622871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120509+0000","last_change":"2024-02-02T13:27:26.119432+0000","last_active":"2024-02-02T13:27:26.120509+0000","last_peered":"2024-02-02T13:27:26.120509+0000","last_clean":"2024-02-02T13:27:26.120509+0000","last_became_active":"2024-02-02T13:27:26.116931+0000","last_became_peered":"2024-02-02T13:27:26.116931+0000","last_unstale":"2024-02-02T13:27:26.120509+0000","last_undegraded":"2024-02-02T13:27:26.120509+0000","last_fullsized":"2024-02-02T13:27:26.120509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:04:06.503908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116444+0000","last_change":"2024-02-02T13:27:25.668142+0000","last_active":"2024-02-02T13:27:26.116444+0000","last_peered":"2024-02-02T13:27:26.116444+0000","last_clean":"2024-02-02T13:27:26.116444+0000","last_became_active":"2024-02-02T13:27:25.667863+0000","last_became_peered":"2024-02-02T13:27:25.667863+0000","last_unstale":"2024-02-02T13:27:26.116444+0000","last_undegraded":"2024-02-02T13:27:26.116444+0000","last_fullsized":"2024-02-02T13:27:26.116444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:07:07.246959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114880+0000","last_change":"2024-02-02T13:27:26.114679+0000","last_active":"2024-02-02T13:27:26.114880+0000","last_peered":"2024-02-02T13:27:26.114880+0000","last_clean":"2024-02-02T13:27:26.114880+0000","last_became_active":"2024-02-02T13:27:26.114409+0000","last_became_peered":"2024-02-02T13:27:26.114409+0000","last_unstale":"2024-02-02T13:27:26.114880+0000","last_undegraded":"2024-02-02T13:27:26.114880+0000","last_fullsized":"2024-02-02T13:27:26.114880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:14.091987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.129250+0000","last_change":"2024-02-02T13:27:25.669444+0000","last_active":"2024-02-02T13:27:26.129250+0000","last_peered":"2024-02-02T13:27:26.129250+0000","last_clean":"2024-02-02T13:27:26.129250+0000","last_became_active":"2024-02-02T13:27:25.668306+0000","last_became_peered":"2024-02-02T13:27:25.668306+0000","last_unstale":"2024-02-02T13:27:26.129250+0000","last_undegraded":"2024-02-02T13:27:26.129250+0000","last_fullsized":"2024-02-02T13:27:26.129250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122734+0000","last_change":"2024-02-02T13:27:26.122446+0000","last_active":"2024-02-02T13:27:26.122734+0000","last_peered":"2024-02-02T13:27:26.122734+0000","last_clean":"2024-02-02T13:27:26.122734+0000","last_became_active":"2024-02-02T13:27:26.122080+0000","last_became_peered":"2024-02-02T13:27:26.122080+0000","last_unstale":"2024-02-02T13:27:26.122734+0000","last_undegraded":"2024-02-02T13:27:26.122734+0000","last_fullsized":"2024-02-02T13:27:26.122734+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:48.938964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113164+0000","last_change":"2024-02-02T13:27:25.666285+0000","last_active":"2024-02-02T13:27:26.113164+0000","last_peered":"2024-02-02T13:27:26.113164+0000","last_clean":"2024-02-02T13:27:26.113164+0000","last_became_active":"2024-02-02T13:27:25.666036+0000","last_became_peered":"2024-02-02T13:27:25.666036+0000","last_unstale":"2024-02-02T13:27:26.113164+0000","last_undegraded":"2024-02-02T13:27:26.113164+0000","last_fullsized":"2024-02-02T13:27:26.113164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:51.711449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120530+0000","last_change":"2024-02-02T13:27:26.119637+0000","last_active":"2024-02-02T13:27:26.120530+0000","last_peered":"2024-02-02T13:27:26.120530+0000","last_clean":"2024-02-02T13:27:26.120530+0000","last_became_active":"2024-02-02T13:27:26.118707+0000","last_became_peered":"2024-02-02T13:27:26.118707+0000","last_unstale":"2024-02-02T13:27:26.120530+0000","last_undegraded":"2024-02-02T13:27:26.120530+0000","last_fullsized":"2024-02-02T13:27:26.120530+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:48.938964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115921+0000","last_change":"2024-02-02T13:27:25.665610+0000","last_active":"2024-02-02T13:27:26.115921+0000","last_peered":"2024-02-02T13:27:26.115921+0000","last_clean":"2024-02-02T13:27:26.115921+0000","last_became_active":"2024-02-02T13:27:25.664118+0000","last_became_peered":"2024-02-02T13:27:25.664118+0000","last_unstale":"2024-02-02T13:27:26.115921+0000","last_undegraded":"2024-02-02T13:27:26.115921+0000","last_fullsized":"2024-02-02T13:27:26.115921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:47.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127916+0000","last_change":"2024-02-02T13:27:26.127752+0000","last_active":"2024-02-02T13:27:26.127916+0000","last_peered":"2024-02-02T13:27:26.127916+0000","last_clean":"2024-02-02T13:27:26.127916+0000","last_became_active":"2024-02-02T13:27:26.127505+0000","last_became_peered":"2024-02-02T13:27:26.127505+0000","last_unstale":"2024-02-02T13:27:26.127916+0000","last_undegraded":"2024-02-02T13:27:26.127916+0000","last_fullsized":"2024-02-02T13:27:26.127916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:18:00.421164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.100512+0000","last_change":"2024-02-02T13:27:25.667490+0000","last_active":"2024-02-02T13:27:26.100512+0000","last_peered":"2024-02-02T13:27:26.100512+0000","last_clean":"2024-02-02T13:27:26.100512+0000","last_became_active":"2024-02-02T13:27:25.667277+0000","last_became_peered":"2024-02-02T13:27:25.667277+0000","last_unstale":"2024-02-02T13:27:26.100512+0000","last_undegraded":"2024-02-02T13:27:26.100512+0000","last_fullsized":"2024-02-02T13:27:26.100512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:33:16.075836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117672+0000","last_change":"2024-02-02T13:27:26.117401+0000","last_active":"2024-02-02T13:27:26.117672+0000","last_peered":"2024-02-02T13:27:26.117672+0000","last_clean":"2024-02-02T13:27:26.117672+0000","last_became_active":"2024-02-02T13:27:26.117203+0000","last_became_peered":"2024-02-02T13:27:26.117203+0000","last_unstale":"2024-02-02T13:27:26.117672+0000","last_undegraded":"2024-02-02T13:27:26.117672+0000","last_fullsized":"2024-02-02T13:27:26.117672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101348+0000","last_change":"2024-02-02T13:27:25.665019+0000","last_active":"2024-02-02T13:27:26.101348+0000","last_peered":"2024-02-02T13:27:26.101348+0000","last_clean":"2024-02-02T13:27:26.101348+0000","last_became_active":"2024-02-02T13:27:25.664754+0000","last_became_peered":"2024-02-02T13:27:25.664754+0000","last_unstale":"2024-02-02T13:27:26.101348+0000","last_undegraded":"2024-02-02T13:27:26.101348+0000","last_fullsized":"2024-02-02T13:27:26.101348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:52.870311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110842+0000","last_change":"2024-02-02T13:27:26.109392+0000","last_active":"2024-02-02T13:27:26.110842+0000","last_peered":"2024-02-02T13:27:26.110842+0000","last_clean":"2024-02-02T13:27:26.110842+0000","last_became_active":"2024-02-02T13:27:26.108813+0000","last_became_peered":"2024-02-02T13:27:26.108813+0000","last_unstale":"2024-02-02T13:27:26.110842+0000","last_undegraded":"2024-02-02T13:27:26.110842+0000","last_fullsized":"2024-02-02T13:27:26.110842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:29.407482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110166+0000","last_change":"2024-02-02T13:27:25.651712+0000","last_active":"2024-02-02T13:27:26.110166+0000","last_peered":"2024-02-02T13:27:26.110166+0000","last_clean":"2024-02-02T13:27:26.110166+0000","last_became_active":"2024-02-02T13:27:25.650994+0000","last_became_peered":"2024-02-02T13:27:25.650994+0000","last_unstale":"2024-02-02T13:27:26.110166+0000","last_undegraded":"2024-02-02T13:27:26.110166+0000","last_fullsized":"2024-02-02T13:27:26.110166+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:24.038059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120409+0000","last_change":"2024-02-02T13:27:26.117981+0000","last_active":"2024-02-02T13:27:26.120409+0000","last_peered":"2024-02-02T13:27:26.120409+0000","last_clean":"2024-02-02T13:27:26.120409+0000","last_became_active":"2024-02-02T13:27:26.115039+0000","last_became_peered":"2024-02-02T13:27:26.115039+0000","last_unstale":"2024-02-02T13:27:26.120409+0000","last_undegraded":"2024-02-02T13:27:26.120409+0000","last_fullsized":"2024-02-02T13:27:26.120409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.111599+0000","last_change":"2024-02-02T13:27:25.665128+0000","last_active":"2024-02-02T13:27:26.111599+0000","last_peered":"2024-02-02T13:27:26.111599+0000","last_clean":"2024-02-02T13:27:26.111599+0000","last_became_active":"2024-02-02T13:27:25.664377+0000","last_became_peered":"2024-02-02T13:27:25.664377+0000","last_unstale":"2024-02-02T13:27:26.111599+0000","last_undegraded":"2024-02-02T13:27:26.111599+0000","last_fullsized":"2024-02-02T13:27:26.111599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113373+0000","last_change":"2024-02-02T13:27:26.112360+0000","last_active":"2024-02-02T13:27:26.113373+0000","last_peered":"2024-02-02T13:27:26.113373+0000","last_clean":"2024-02-02T13:27:26.113373+0000","last_became_active":"2024-02-02T13:27:26.112071+0000","last_became_peered":"2024-02-02T13:27:26.112071+0000","last_unstale":"2024-02-02T13:27:26.113373+0000","last_undegraded":"2024-02-02T13:27:26.113373+0000","last_fullsized":"2024-02-02T13:27:26.113373+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:29:57.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122268+0000","last_change":"2024-02-02T13:27:25.663143+0000","last_active":"2024-02-02T13:27:26.122268+0000","last_peered":"2024-02-02T13:27:26.122268+0000","last_clean":"2024-02-02T13:27:26.122268+0000","last_became_active":"2024-02-02T13:27:25.662655+0000","last_became_peered":"2024-02-02T13:27:25.662655+0000","last_unstale":"2024-02-02T13:27:26.122268+0000","last_undegraded":"2024-02-02T13:27:26.122268+0000","last_fullsized":"2024-02-02T13:27:26.122268+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:42:34.990661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122832+0000","last_change":"2024-02-02T13:27:26.122794+0000","last_active":"2024-02-02T13:27:26.122832+0000","last_peered":"2024-02-02T13:27:26.122832+0000","last_clean":"2024-02-02T13:27:26.122832+0000","last_became_active":"2024-02-02T13:27:26.122630+0000","last_became_peered":"2024-02-02T13:27:26.122630+0000","last_unstale":"2024-02-02T13:27:26.122832+0000","last_undegraded":"2024-02-02T13:27:26.122832+0000","last_fullsized":"2024-02-02T13:27:26.122832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.100991+0000","last_change":"2024-02-02T13:27:25.668633+0000","last_active":"2024-02-02T13:27:26.100991+0000","last_peered":"2024-02-02T13:27:26.100991+0000","last_clean":"2024-02-02T13:27:26.100991+0000","last_became_active":"2024-02-02T13:27:25.667737+0000","last_became_peered":"2024-02-02T13:27:25.667737+0000","last_unstale":"2024-02-02T13:27:26.100991+0000","last_undegraded":"2024-02-02T13:27:26.100991+0000","last_fullsized":"2024-02-02T13:27:26.100991+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:08.064478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.108237+0000","last_change":"2024-02-02T13:27:26.107991+0000","last_active":"2024-02-02T13:27:26.108237+0000","last_peered":"2024-02-02T13:27:26.108237+0000","last_clean":"2024-02-02T13:27:26.108237+0000","last_became_active":"2024-02-02T13:27:26.107300+0000","last_became_peered":"2024-02-02T13:27:26.107300+0000","last_unstale":"2024-02-02T13:27:26.108237+0000","last_undegraded":"2024-02-02T13:27:26.108237+0000","last_fullsized":"2024-02-02T13:27:26.108237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:16:56.636011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103366+0000","last_change":"2024-02-02T13:27:25.666659+0000","last_active":"2024-02-02T13:27:26.103366+0000","last_peered":"2024-02-02T13:27:26.103366+0000","last_clean":"2024-02-02T13:27:26.103366+0000","last_became_active":"2024-02-02T13:27:25.666387+0000","last_became_peered":"2024-02-02T13:27:25.666387+0000","last_unstale":"2024-02-02T13:27:26.103366+0000","last_undegraded":"2024-02-02T13:27:26.103366+0000","last_fullsized":"2024-02-02T13:27:26.103366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:42:34.990661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126646+0000","last_change":"2024-02-02T13:27:26.124998+0000","last_active":"2024-02-02T13:27:26.126646+0000","last_peered":"2024-02-02T13:27:26.126646+0000","last_clean":"2024-02-02T13:27:26.126646+0000","last_became_active":"2024-02-02T13:27:26.124577+0000","last_became_peered":"2024-02-02T13:27:26.124577+0000","last_unstale":"2024-02-02T13:27:26.126646+0000","last_undegraded":"2024-02-02T13:27:26.126646+0000","last_fullsized":"2024-02-02T13:27:26.126646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.106250+0000","last_change":"2024-02-02T13:27:25.663167+0000","last_active":"2024-02-02T13:27:26.106250+0000","last_peered":"2024-02-02T13:27:26.106250+0000","last_clean":"2024-02-02T13:27:26.106250+0000","last_became_active":"2024-02-02T13:27:25.662712+0000","last_became_peered":"2024-02-02T13:27:25.662712+0000","last_unstale":"2024-02-02T13:27:26.106250+0000","last_undegraded":"2024-02-02T13:27:26.106250+0000","last_fullsized":"2024-02-02T13:27:26.106250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:46.476521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127434+0000","last_change":"2024-02-02T13:27:26.127356+0000","last_active":"2024-02-02T13:27:26.127434+0000","last_peered":"2024-02-02T13:27:26.127434+0000","last_clean":"2024-02-02T13:27:26.127434+0000","last_became_active":"2024-02-02T13:27:26.127123+0000","last_became_peered":"2024-02-02T13:27:26.127123+0000","last_unstale":"2024-02-02T13:27:26.127434+0000","last_undegraded":"2024-02-02T13:27:26.127434+0000","last_fullsized":"2024-02-02T13:27:26.127434+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:53.881018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.102194+0000","last_change":"2024-02-02T13:27:25.664917+0000","last_active":"2024-02-02T13:27:26.102194+0000","last_peered":"2024-02-02T13:27:26.102194+0000","last_clean":"2024-02-02T13:27:26.102194+0000","last_became_active":"2024-02-02T13:27:25.664676+0000","last_became_peered":"2024-02-02T13:27:25.664676+0000","last_unstale":"2024-02-02T13:27:26.102194+0000","last_undegraded":"2024-02-02T13:27:26.102194+0000","last_fullsized":"2024-02-02T13:27:26.102194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:06.846491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.125839+0000","last_change":"2024-02-02T13:27:26.125780+0000","last_active":"2024-02-02T13:27:26.125839+0000","last_peered":"2024-02-02T13:27:26.125839+0000","last_clean":"2024-02-02T13:27:26.125839+0000","last_became_active":"2024-02-02T13:27:26.125632+0000","last_became_peered":"2024-02-02T13:27:26.125632+0000","last_unstale":"2024-02-02T13:27:26.125839+0000","last_undegraded":"2024-02-02T13:27:26.125839+0000","last_fullsized":"2024-02-02T13:27:26.125839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:56:09.722914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.096756+0000","last_change":"2024-02-02T13:27:25.666195+0000","last_active":"2024-02-02T13:27:26.096756+0000","last_peered":"2024-02-02T13:27:26.096756+0000","last_clean":"2024-02-02T13:27:26.096756+0000","last_became_active":"2024-02-02T13:27:25.665190+0000","last_became_peered":"2024-02-02T13:27:25.665190+0000","last_unstale":"2024-02-02T13:27:26.096756+0000","last_undegraded":"2024-02-02T13:27:26.096756+0000","last_fullsized":"2024-02-02T13:27:26.096756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:47.345920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115690+0000","last_change":"2024-02-02T13:27:26.115626+0000","last_active":"2024-02-02T13:27:26.115690+0000","last_peered":"2024-02-02T13:27:26.115690+0000","last_clean":"2024-02-02T13:27:26.115690+0000","last_became_active":"2024-02-02T13:27:26.114862+0000","last_became_peered":"2024-02-02T13:27:26.114862+0000","last_unstale":"2024-02-02T13:27:26.115690+0000","last_undegraded":"2024-02-02T13:27:26.115690+0000","last_fullsized":"2024-02-02T13:27:26.115690+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:58:01.195901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121082+0000","last_change":"2024-02-02T13:27:25.670077+0000","last_active":"2024-02-02T13:27:26.121082+0000","last_peered":"2024-02-02T13:27:26.121082+0000","last_clean":"2024-02-02T13:27:26.121082+0000","last_became_active":"2024-02-02T13:27:25.669809+0000","last_became_peered":"2024-02-02T13:27:25.669809+0000","last_unstale":"2024-02-02T13:27:26.121082+0000","last_undegraded":"2024-02-02T13:27:26.121082+0000","last_fullsized":"2024-02-02T13:27:26.121082+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:43.341036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":209,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-02-02T13:27:25.643137+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:27:25.643137+0000","last_peered":"2024-02-02T13:27:25.643137+0000","last_clean":"2024-02-02T13:27:25.643137+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:27:25.643137+0000","last_undegraded":"2024-02-02T13:27:25.643137+0000","last_fullsized":"2024-02-02T13:27:25.643137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065509,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0580000000000001,"15min":1.052},"min":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5880000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.123,"15min":1.1299999999999999},"min":{"1min":0.76900000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5589999999999999,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.2010000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.083},"min":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":1.893,"15min":1.893},"last":1.268},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.105,"15min":1.099},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4099999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.365}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.208,"15min":1.2110000000000001},"min":{"1min":0.78100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.522},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.159,"15min":1.181},"min":{"1min":0.67500000000000004,"5min":0.623,"15min":0.623},"max":{"1min":1.615,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.585}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.327,"5min":1.819,"15min":1.819},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.661,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.647}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.155,"15min":1.159},"min":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.302,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6619999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.621}]},{"osd":5,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.347,"15min":1.343},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.6799999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7130000000000001,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.8200000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5209999999999999,"15min":1.5409999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8049999999999999,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.6299999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.788}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.012,"15min":0.996},"min":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5089999999999999,"5min":1.577,"15min":1.577},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.038,"5min":1.002,"15min":0.996},"min":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.488,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.242,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.0589999999999999,"15min":1.048},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.048,"15min":1.038},"min":{"1min":0.80200000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.385},{"interface":"front","average":{"1min":1.137,"5min":1.143,"15min":1.1439999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4079999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5049999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.137},{"interface":"front","average":{"1min":1.149,"5min":1.125,"15min":1.121},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.98299999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.637},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.389,"15min":1.3819999999999999},"min":{"1min":0.89900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.696,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.54}]},{"osd":5,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.77900000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.615,"5min":1.645,"15min":1.645},"last":1.482},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.339,"15min":1.327},"min":{"1min":1.099,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4399999999999999,"15min":1.4419999999999999},"min":{"1min":1.0049999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.694,"5min":1.835,"15min":1.835},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.419,"15min":1.4279999999999999},"min":{"1min":1.1439999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.655,"5min":1.804,"15min":1.804},"last":1.671}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194989,"statfs":{"total":200000000,"available":199669079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.022,"15min":1.018},"min":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0249999999999999,"15min":1.0329999999999999},"min":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.7150000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2629999999999999,"15min":1.2969999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.4990000000000001,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.655},{"interface":"front","average":{"1min":1.075,"5min":1.153,"15min":1.1659999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.444,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.296}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.171,"15min":1.169},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1799999999999999,"15min":1.1830000000000001},"min":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.512,"5min":1.744,"15min":1.744},"last":1.139}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2330000000000001,"15min":1.2350000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2729999999999999,"15min":1.2789999999999999},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.575}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3029999999999999,"15min":1.2929999999999999},"min":{"1min":1.042,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.397},{"interface":"front","average":{"1min":1.389,"5min":1.351,"15min":1.345},"min":{"1min":0.92200000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3979999999999999,"15min":1.397},"min":{"1min":0.91000000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.762},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3640000000000001,"15min":1.353},"min":{"1min":0.88600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.5289999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.96099999999999997,"15min":0.94799999999999995},"min":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.0629999999999999,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.554,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1359999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.78700000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5,"5min":1.506,"15min":1.506},"last":1.298},{"interface":"front","average":{"1min":0.995,"5min":0.999,"15min":1},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.27,"5min":1.345,"15min":1.345},"last":0.93100000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.0660000000000001,"15min":1.0580000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4339999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.327},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0589999999999999,"15min":1.038},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.111}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.161,"15min":1.145},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4490000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.385},{"interface":"front","average":{"1min":1.292,"5min":1.2210000000000001,"15min":1.2090000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.5920000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.236}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.117,"15min":1.1040000000000001},"min":{"1min":0.98199999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.335,"5min":1.2609999999999999,"15min":1.2490000000000001},"min":{"1min":1.008,"5min":0.751,"15min":0.751},"max":{"1min":1.6879999999999999,"5min":2.069,"15min":2.069},"last":1.27}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.90700000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3640000000000001,"15min":1.361},"min":{"1min":1.123,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3580000000000001,"15min":1.3500000000000001},"min":{"1min":1.1659999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.6830000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.90100000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.675,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.349}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194989,"statfs":{"total":200000000,"available":199669079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3959999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":0.998},"min":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3540000000000001,"5min":2.125,"15min":2.125},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.94299999999999995,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.181,"15min":1.177},"min":{"1min":0.91400000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6180000000000001,"5min":3.585,"15min":3.585},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.198,"5min":1.1859999999999999,"15min":1.1839999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6659999999999999,"5min":1.877,"15min":1.877},"last":1.034}]},{"osd":5,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.222,"15min":1.224},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.758,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.163,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.554,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.234,"15min":1.23},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.744,"5min":1.841,"15min":1.841},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.302,"15min":1.306},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6299999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.046}]},{"osd":7,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5720000000000001,"5min":3.391,"15min":3.391},"last":0.998},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6479999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.85999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194983,"statfs":{"total":200000000,"available":199662812,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.755,"15min":0.77200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1120000000000001,"5min":1.29,"15min":1.29},"last":1.506},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.38200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2549999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.87,"15min":0.88},"min":{"1min":0.50900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.2170000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.246,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.93899999999999995,"15min":0.94999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.3480000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.1379999999999999,"15min":1.129},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.3939999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0589999999999999,"15min":1.0409999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.25},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.669}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.173,"15min":1.1699999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.2829999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1639999999999999,"15min":1.1579999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.4159999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.022,"15min":1.0649999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":1.391,"15min":1.4910000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5029999999999999,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.371}]},{"osd":2,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.639,"15min":1.639},"last":0.503},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.091,"15min":1.121},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3680000000000001,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.25,"15min":1.3080000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4830000000000001,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.2949999999999999,"15min":1.357},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.528,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.335,"15min":1.377},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3799999999999999,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.3580000000000001,"15min":1.3939999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.4930000000000001,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.472,"15min":1.508},"min":{"1min":1.0609999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.532,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.4470000000000001,"15min":1.4850000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.8959999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.5880000000000001,"15min":1.653},"min":{"1min":0.85099999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.661,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4950000000000001,"15min":1.5429999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.806,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5609999999999999,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.496,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.6299999999999999,"15min":1.6890000000000001},"min":{"1min":0.97399999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5880000000000001,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.95899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":1.161,"15min":1.2290000000000001},"min":{"1min":0.44,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1899999999999999,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.5329999999999999,"15min":1.6559999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.157,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.333}]},{"osd":2,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.179,"15min":1.234},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2390000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.534,"15min":1.6419999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.385,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.268,"15min":1.3320000000000001},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2629999999999999,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.3340000000000001,"15min":1.3939999999999999},"min":{"1min":0.628,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4490000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.359,"15min":1.3939999999999999},"min":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4390000000000001,"5min":3.669,"15min":3.669},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.80000000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.6359999999999999,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.885,"15min":1.9930000000000001},"min":{"1min":0.83199999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6060000000000001,"5min":10.651,"15min":10.651},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.4470000000000001,"15min":1.484},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.589,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.492,"15min":1.526},"min":{"1min":0.86899999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.972,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.9259999999999999,"15min":2.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5740000000000001,"5min":10.708,"15min":10.708},"last":1.431}]},{"osd":7,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.9410000000000001,"15min":2.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7849999999999999,"5min":10.74,"15min":10.74},"last":1.397},{"interface":"front","average":{"1min":1.252,"5min":1.4610000000000001,"15min":1.496},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.613,"5min":3.863,"15min":3.863},"last":1.2549999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:27:30.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:27:30.609 INFO:tasks.ceph.mds.c.smithi091.stderr:2024-02-02T13:27:30.607+0000 7f7ec0a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:27:30.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.815+0000 7f1f3ac00640 1 Processor -- start 2024-02-02T13:27:30.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.815+0000 7f1f3ac00640 1 -- start start 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f3ac00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f34151f90 0x7f1f3415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f3ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 0x7f1f340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f3ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f34070f80 con 0x7f1f340b0c40 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f3ac00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340b10d0 con 0x7f1f34151f90 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f3ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34152370 con 0x7f1f3406f4f0 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f34151f90 0x7f1f3415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f34151f90 0x7f1f3415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57356/0 (socket says 172.21.15.91:57356) 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 -- 172.21.15.91:0/2816845746 learned_addr learned my addr 172.21.15.91:0/2816845746 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f2b200640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 0x7f1f340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 -- 172.21.15.91:0/2816845746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 msgr2=0x7f1f340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 0x7f1f340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.817+0000 7f1f33e00640 1 -- 172.21.15.91:0/2816845746 >> v1:172.21.15.113:6789/0 conn(0x7f1f340b0c40 legacy=0x7f1f340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.818+0000 7f1f33e00640 1 -- 172.21.15.91:0/2816845746 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340cb010 con 0x7f1f34151f90 2024-02-02T13:27:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.818+0000 7f1f2b200640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 0x7f1f340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f33e00640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f34151f90 0x7f1f3415f5a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009870 tx=0x7f1f2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b761dcc3164462a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f33400640 1 -- 172.21.15.91:0/2816845746 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24031640 con 0x7f1f34151f90 2024-02-02T13:27:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f33400640 1 -- 172.21.15.91:0/2816845746 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f240317a0 con 0x7f1f34151f90 2024-02-02T13:27:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f33400640 1 -- 172.21.15.91:0/2816845746 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24031b30 con 0x7f1f34151f90 2024-02-02T13:27:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f3ac00640 1 -- 172.21.15.91:0/2816845746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f34151f90 msgr2=0x7f1f3415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f34151f90 0x7f1f3415f5a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009870 tx=0x7f1f2402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f3ac00640 1 -- 172.21.15.91:0/2816845746 shutdown_connections 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f3406f4f0 0x7f1f340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.819+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/2816845746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f34151f90 0x7f1f3415f5a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.820+0000 7f1f3ac00640 1 -- 172.21.15.91:0/2816845746 >> 172.21.15.91:0/2816845746 conn(0x7f1f340cec80 msgr2=0x7f1f340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.820+0000 7f1f3ac00640 1 -- 172.21.15.91:0/2816845746 shutdown_connections 2024-02-02T13:27:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.820+0000 7f1f3ac00640 1 -- 172.21.15.91:0/2816845746 wait complete. 2024-02-02T13:27:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.821+0000 7f1f3ac00640 1 Processor -- start 2024-02-02T13:27:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.821+0000 7f1f3ac00640 1 -- start start 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f340b0c40 0x7f1f340c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34151f90 0x7f1f340c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b1c10 con 0x7f1f3406f4f0 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b1ef0 con 0x7f1f34151f90 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f3ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b21d0 con 0x7f1f340b0c40 2024-02-02T13:27:30.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.822+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57364/0 (socket says 172.21.15.91:57364) 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 -- 172.21.15.91:0/361593701 learned_addr learned my addr 172.21.15.91:0/361593701 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 -- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f340b0c40 msgr2=0x7f1f340c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f340b0c40 0x7f1f340c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 -- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34151f90 msgr2=0x7f1f340c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34151f90 0x7f1f340c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 -- 172.21.15.91:0/361593701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240094d0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f33e00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2403ac60 con 0x7f1f3406f4f0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2403ade0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.823+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24044630 con 0x7f1f3406f4f0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.824+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f340b24b0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.824+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f340b18a0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.825+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f340b29d0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.826+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f2402f050 con 0x7f1f3406f4f0 2024-02-02T13:27:30.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.826+0000 7f1f30c00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 0x7f1f0c06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:30.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.827+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f240aa9c0 con 0x7f1f3406f4f0 2024-02-02T13:27:30.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.827+0000 7f1f2b200640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 0x7f1f0c06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:30.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.827+0000 7f1f2b200640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 0x7f1f0c06c8c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:30.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:30.831+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f340b29d0 con 0x7f1f3406f4f0 2024-02-02T13:27:31.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.000+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f34071200 con 0x7f1f0c06a400 2024-02-02T13:27:31.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.021+0000 7f1f30c00640 1 -- 172.21.15.91:0/361593701 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282230 (secure 0 0 0) 0x7f1f1c00c7a0 con 0x7f1f0c06a400 2024-02-02T13:27:31.027 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.026+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 msgr2=0x7f1f0c06c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.026+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 0x7f1f0c06c8c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c008040 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.026+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 msgr2=0x7f1f34151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.026+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f240046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 shutdown_connections 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f0c06a400 0x7f1f0c06c8c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f340b0c40 0x7f1f340c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f34151f90 0x7f1f340c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 --2- 172.21.15.91:0/361593701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3406f4f0 0x7f1f34151040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 >> 172.21.15.91:0/361593701 conn(0x7f1f340cec80 msgr2=0x7f1f340cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:31.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 shutdown_connections 2024-02-02T13:27:31.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.027+0000 7f1f3ac00640 1 -- 172.21.15.91:0/361593701 wait complete. 2024-02-02T13:27:31.045 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-02-02T13:27:30.289609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558740,"statfs":{"total":1600000000,"available":1596150236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.657214"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.104327+0000","last_change":"2024-02-02T13:27:26.104151+0000","last_active":"2024-02-02T13:27:26.104327+0000","last_peered":"2024-02-02T13:27:26.104327+0000","last_clean":"2024-02-02T13:27:26.104327+0000","last_became_active":"2024-02-02T13:27:26.102947+0000","last_became_peered":"2024-02-02T13:27:26.102947+0000","last_unstale":"2024-02-02T13:27:26.104327+0000","last_undegraded":"2024-02-02T13:27:26.104327+0000","last_fullsized":"2024-02-02T13:27:26.104327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:40:46.350142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.107947+0000","last_change":"2024-02-02T13:27:26.107856+0000","last_active":"2024-02-02T13:27:26.107947+0000","last_peered":"2024-02-02T13:27:26.107947+0000","last_clean":"2024-02-02T13:27:26.107947+0000","last_became_active":"2024-02-02T13:27:26.106972+0000","last_became_peered":"2024-02-02T13:27:26.106972+0000","last_unstale":"2024-02-02T13:27:26.107947+0000","last_undegraded":"2024-02-02T13:27:26.107947+0000","last_fullsized":"2024-02-02T13:27:26.107947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:43:34.542970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122754+0000","last_change":"2024-02-02T13:27:26.117073+0000","last_active":"2024-02-02T13:27:26.122754+0000","last_peered":"2024-02-02T13:27:26.122754+0000","last_clean":"2024-02-02T13:27:26.122754+0000","last_became_active":"2024-02-02T13:27:26.116211+0000","last_became_peered":"2024-02-02T13:27:26.116211+0000","last_unstale":"2024-02-02T13:27:26.122754+0000","last_undegraded":"2024-02-02T13:27:26.122754+0000","last_fullsized":"2024-02-02T13:27:26.122754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:18:00.421164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120452+0000","last_change":"2024-02-02T13:27:26.119108+0000","last_active":"2024-02-02T13:27:26.120452+0000","last_peered":"2024-02-02T13:27:26.120452+0000","last_clean":"2024-02-02T13:27:26.120452+0000","last_became_active":"2024-02-02T13:27:26.112891+0000","last_became_peered":"2024-02-02T13:27:26.112891+0000","last_unstale":"2024-02-02T13:27:26.120452+0000","last_undegraded":"2024-02-02T13:27:26.120452+0000","last_fullsized":"2024-02-02T13:27:26.120452+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105067+0000","last_change":"2024-02-02T13:27:26.104951+0000","last_active":"2024-02-02T13:27:26.105067+0000","last_peered":"2024-02-02T13:27:26.105067+0000","last_clean":"2024-02-02T13:27:26.105067+0000","last_became_active":"2024-02-02T13:27:26.104593+0000","last_became_peered":"2024-02-02T13:27:26.104593+0000","last_unstale":"2024-02-02T13:27:26.105067+0000","last_undegraded":"2024-02-02T13:27:26.105067+0000","last_fullsized":"2024-02-02T13:27:26.105067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:57.564005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114672+0000","last_change":"2024-02-02T13:27:26.113047+0000","last_active":"2024-02-02T13:27:26.114672+0000","last_peered":"2024-02-02T13:27:26.114672+0000","last_clean":"2024-02-02T13:27:26.114672+0000","last_became_active":"2024-02-02T13:27:26.112790+0000","last_became_peered":"2024-02-02T13:27:26.112790+0000","last_unstale":"2024-02-02T13:27:26.114672+0000","last_undegraded":"2024-02-02T13:27:26.114672+0000","last_fullsized":"2024-02-02T13:27:26.114672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:25.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121475+0000","last_change":"2024-02-02T13:27:26.118900+0000","last_active":"2024-02-02T13:27:26.121475+0000","last_peered":"2024-02-02T13:27:26.121475+0000","last_clean":"2024-02-02T13:27:26.121475+0000","last_became_active":"2024-02-02T13:27:26.118509+0000","last_became_peered":"2024-02-02T13:27:26.118509+0000","last_unstale":"2024-02-02T13:27:26.121475+0000","last_undegraded":"2024-02-02T13:27:26.121475+0000","last_fullsized":"2024-02-02T13:27:26.121475+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:50.453938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.107962+0000","last_change":"2024-02-02T13:27:26.107878+0000","last_active":"2024-02-02T13:27:26.107962+0000","last_peered":"2024-02-02T13:27:26.107962+0000","last_clean":"2024-02-02T13:27:26.107962+0000","last_became_active":"2024-02-02T13:27:26.107654+0000","last_became_peered":"2024-02-02T13:27:26.107654+0000","last_unstale":"2024-02-02T13:27:26.107962+0000","last_undegraded":"2024-02-02T13:27:26.107962+0000","last_fullsized":"2024-02-02T13:27:26.107962+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121431+0000","last_change":"2024-02-02T13:27:26.118746+0000","last_active":"2024-02-02T13:27:26.121431+0000","last_peered":"2024-02-02T13:27:26.121431+0000","last_clean":"2024-02-02T13:27:26.121431+0000","last_became_active":"2024-02-02T13:27:26.118070+0000","last_became_peered":"2024-02-02T13:27:26.118070+0000","last_unstale":"2024-02-02T13:27:26.121431+0000","last_undegraded":"2024-02-02T13:27:26.121431+0000","last_fullsized":"2024-02-02T13:27:26.121431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:17.584842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127416+0000","last_change":"2024-02-02T13:27:26.127346+0000","last_active":"2024-02-02T13:27:26.127416+0000","last_peered":"2024-02-02T13:27:26.127416+0000","last_clean":"2024-02-02T13:27:26.127416+0000","last_became_active":"2024-02-02T13:27:26.127116+0000","last_became_peered":"2024-02-02T13:27:26.127116+0000","last_unstale":"2024-02-02T13:27:26.127416+0000","last_undegraded":"2024-02-02T13:27:26.127416+0000","last_fullsized":"2024-02-02T13:27:26.127416+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:52.722156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.108042+0000","last_change":"2024-02-02T13:27:26.107979+0000","last_active":"2024-02-02T13:27:26.108042+0000","last_peered":"2024-02-02T13:27:26.108042+0000","last_clean":"2024-02-02T13:27:26.108042+0000","last_became_active":"2024-02-02T13:27:26.107752+0000","last_became_peered":"2024-02-02T13:27:26.107752+0000","last_unstale":"2024-02-02T13:27:26.108042+0000","last_undegraded":"2024-02-02T13:27:26.108042+0000","last_fullsized":"2024-02-02T13:27:26.108042+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:32.309386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120489+0000","last_change":"2024-02-02T13:27:26.119194+0000","last_active":"2024-02-02T13:27:26.120489+0000","last_peered":"2024-02-02T13:27:26.120489+0000","last_clean":"2024-02-02T13:27:26.120489+0000","last_became_active":"2024-02-02T13:27:26.118830+0000","last_became_peered":"2024-02-02T13:27:26.118830+0000","last_unstale":"2024-02-02T13:27:26.120489+0000","last_undegraded":"2024-02-02T13:27:26.120489+0000","last_fullsized":"2024-02-02T13:27:26.120489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126850+0000","last_change":"2024-02-02T13:27:26.124370+0000","last_active":"2024-02-02T13:27:26.126850+0000","last_peered":"2024-02-02T13:27:26.126850+0000","last_clean":"2024-02-02T13:27:26.126850+0000","last_became_active":"2024-02-02T13:27:26.124005+0000","last_became_peered":"2024-02-02T13:27:26.124005+0000","last_unstale":"2024-02-02T13:27:26.126850+0000","last_undegraded":"2024-02-02T13:27:26.126850+0000","last_fullsized":"2024-02-02T13:27:26.126850+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:47.487227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110124+0000","last_change":"2024-02-02T13:27:26.109122+0000","last_active":"2024-02-02T13:27:26.110124+0000","last_peered":"2024-02-02T13:27:26.110124+0000","last_clean":"2024-02-02T13:27:26.110124+0000","last_became_active":"2024-02-02T13:27:26.108546+0000","last_became_peered":"2024-02-02T13:27:26.108546+0000","last_unstale":"2024-02-02T13:27:26.110124+0000","last_undegraded":"2024-02-02T13:27:26.110124+0000","last_fullsized":"2024-02-02T13:27:26.110124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:42:33.770919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110168+0000","last_change":"2024-02-02T13:27:26.109683+0000","last_active":"2024-02-02T13:27:26.110168+0000","last_peered":"2024-02-02T13:27:26.110168+0000","last_clean":"2024-02-02T13:27:26.110168+0000","last_became_active":"2024-02-02T13:27:26.109023+0000","last_became_peered":"2024-02-02T13:27:26.109023+0000","last_unstale":"2024-02-02T13:27:26.110168+0000","last_undegraded":"2024-02-02T13:27:26.110168+0000","last_fullsized":"2024-02-02T13:27:26.110168+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:15.560754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124273+0000","last_change":"2024-02-02T13:27:26.124214+0000","last_active":"2024-02-02T13:27:26.124273+0000","last_peered":"2024-02-02T13:27:26.124273+0000","last_clean":"2024-02-02T13:27:26.124273+0000","last_became_active":"2024-02-02T13:27:26.123994+0000","last_became_peered":"2024-02-02T13:27:26.123994+0000","last_unstale":"2024-02-02T13:27:26.124273+0000","last_undegraded":"2024-02-02T13:27:26.124273+0000","last_fullsized":"2024-02-02T13:27:26.124273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124437+0000","last_change":"2024-02-02T13:27:26.124370+0000","last_active":"2024-02-02T13:27:26.124437+0000","last_peered":"2024-02-02T13:27:26.124437+0000","last_clean":"2024-02-02T13:27:26.124437+0000","last_became_active":"2024-02-02T13:27:26.124136+0000","last_became_peered":"2024-02-02T13:27:26.124136+0000","last_unstale":"2024-02-02T13:27:26.124437+0000","last_undegraded":"2024-02-02T13:27:26.124437+0000","last_fullsized":"2024-02-02T13:27:26.124437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:59:54.602035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124691+0000","last_change":"2024-02-02T13:27:26.123390+0000","last_active":"2024-02-02T13:27:26.124691+0000","last_peered":"2024-02-02T13:27:26.124691+0000","last_clean":"2024-02-02T13:27:26.124691+0000","last_became_active":"2024-02-02T13:27:26.122978+0000","last_became_peered":"2024-02-02T13:27:26.122978+0000","last_unstale":"2024-02-02T13:27:26.124691+0000","last_undegraded":"2024-02-02T13:27:26.124691+0000","last_fullsized":"2024-02-02T13:27:26.124691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:14:31.975198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.104306+0000","last_change":"2024-02-02T13:27:26.103837+0000","last_active":"2024-02-02T13:27:26.104306+0000","last_peered":"2024-02-02T13:27:26.104306+0000","last_clean":"2024-02-02T13:27:26.104306+0000","last_became_active":"2024-02-02T13:27:26.103628+0000","last_became_peered":"2024-02-02T13:27:26.103628+0000","last_unstale":"2024-02-02T13:27:26.104306+0000","last_undegraded":"2024-02-02T13:27:26.104306+0000","last_fullsized":"2024-02-02T13:27:26.104306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:40.405342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114896+0000","last_change":"2024-02-02T13:27:26.114768+0000","last_active":"2024-02-02T13:27:26.114896+0000","last_peered":"2024-02-02T13:27:26.114896+0000","last_clean":"2024-02-02T13:27:26.114896+0000","last_became_active":"2024-02-02T13:27:26.114555+0000","last_became_peered":"2024-02-02T13:27:26.114555+0000","last_unstale":"2024-02-02T13:27:26.114896+0000","last_undegraded":"2024-02-02T13:27:26.114896+0000","last_fullsized":"2024-02-02T13:27:26.114896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:03:41.987049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126696+0000","last_change":"2024-02-02T13:27:26.126133+0000","last_active":"2024-02-02T13:27:26.126696+0000","last_peered":"2024-02-02T13:27:26.126696+0000","last_clean":"2024-02-02T13:27:26.126696+0000","last_became_active":"2024-02-02T13:27:26.125689+0000","last_became_peered":"2024-02-02T13:27:26.125689+0000","last_unstale":"2024-02-02T13:27:26.126696+0000","last_undegraded":"2024-02-02T13:27:26.126696+0000","last_fullsized":"2024-02-02T13:27:26.126696+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:23.851535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103155+0000","last_change":"2024-02-02T13:27:26.103084+0000","last_active":"2024-02-02T13:27:26.103155+0000","last_peered":"2024-02-02T13:27:26.103155+0000","last_clean":"2024-02-02T13:27:26.103155+0000","last_became_active":"2024-02-02T13:27:26.102874+0000","last_became_peered":"2024-02-02T13:27:26.102874+0000","last_unstale":"2024-02-02T13:27:26.103155+0000","last_undegraded":"2024-02-02T13:27:26.103155+0000","last_fullsized":"2024-02-02T13:27:26.103155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:16:58.378414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115929+0000","last_change":"2024-02-02T13:27:26.115822+0000","last_active":"2024-02-02T13:27:26.115929+0000","last_peered":"2024-02-02T13:27:26.115929+0000","last_clean":"2024-02-02T13:27:26.115929+0000","last_became_active":"2024-02-02T13:27:26.115466+0000","last_became_peered":"2024-02-02T13:27:26.115466+0000","last_unstale":"2024-02-02T13:27:26.115929+0000","last_undegraded":"2024-02-02T13:27:26.115929+0000","last_fullsized":"2024-02-02T13:27:26.115929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126869+0000","last_change":"2024-02-02T13:27:26.125968+0000","last_active":"2024-02-02T13:27:26.126869+0000","last_peered":"2024-02-02T13:27:26.126869+0000","last_clean":"2024-02-02T13:27:26.126869+0000","last_became_active":"2024-02-02T13:27:26.125646+0000","last_became_peered":"2024-02-02T13:27:26.125646+0000","last_unstale":"2024-02-02T13:27:26.126869+0000","last_undegraded":"2024-02-02T13:27:26.126869+0000","last_fullsized":"2024-02-02T13:27:26.126869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:14.418710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101459+0000","last_change":"2024-02-02T13:27:26.101268+0000","last_active":"2024-02-02T13:27:26.101459+0000","last_peered":"2024-02-02T13:27:26.101459+0000","last_clean":"2024-02-02T13:27:26.101459+0000","last_became_active":"2024-02-02T13:27:26.101044+0000","last_became_peered":"2024-02-02T13:27:26.101044+0000","last_unstale":"2024-02-02T13:27:26.101459+0000","last_undegraded":"2024-02-02T13:27:26.101459+0000","last_fullsized":"2024-02-02T13:27:26.101459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:00.524308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120580+0000","last_change":"2024-02-02T13:27:26.120174+0000","last_active":"2024-02-02T13:27:26.120580+0000","last_peered":"2024-02-02T13:27:26.120580+0000","last_clean":"2024-02-02T13:27:26.120580+0000","last_became_active":"2024-02-02T13:27:26.118598+0000","last_became_peered":"2024-02-02T13:27:26.118598+0000","last_unstale":"2024-02-02T13:27:26.120580+0000","last_undegraded":"2024-02-02T13:27:26.120580+0000","last_fullsized":"2024-02-02T13:27:26.120580+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.123886+0000","last_change":"2024-02-02T13:27:26.123815+0000","last_active":"2024-02-02T13:27:26.123886+0000","last_peered":"2024-02-02T13:27:26.123886+0000","last_clean":"2024-02-02T13:27:26.123886+0000","last_became_active":"2024-02-02T13:27:26.123643+0000","last_became_peered":"2024-02-02T13:27:26.123643+0000","last_unstale":"2024-02-02T13:27:26.123886+0000","last_undegraded":"2024-02-02T13:27:26.123886+0000","last_fullsized":"2024-02-02T13:27:26.123886+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105042+0000","last_change":"2024-02-02T13:27:26.104849+0000","last_active":"2024-02-02T13:27:26.105042+0000","last_peered":"2024-02-02T13:27:26.105042+0000","last_clean":"2024-02-02T13:27:26.105042+0000","last_became_active":"2024-02-02T13:27:26.104597+0000","last_became_peered":"2024-02-02T13:27:26.104597+0000","last_unstale":"2024-02-02T13:27:26.105042+0000","last_undegraded":"2024-02-02T13:27:26.105042+0000","last_fullsized":"2024-02-02T13:27:26.105042+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:46:08.882471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.106564+0000","last_change":"2024-02-02T13:27:26.106491+0000","last_active":"2024-02-02T13:27:26.106564+0000","last_peered":"2024-02-02T13:27:26.106564+0000","last_clean":"2024-02-02T13:27:26.106564+0000","last_became_active":"2024-02-02T13:27:26.104134+0000","last_became_peered":"2024-02-02T13:27:26.104134+0000","last_unstale":"2024-02-02T13:27:26.106564+0000","last_undegraded":"2024-02-02T13:27:26.106564+0000","last_fullsized":"2024-02-02T13:27:26.106564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:20:25.099877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122679+0000","last_change":"2024-02-02T13:27:26.121973+0000","last_active":"2024-02-02T13:27:26.122679+0000","last_peered":"2024-02-02T13:27:26.122679+0000","last_clean":"2024-02-02T13:27:26.122679+0000","last_became_active":"2024-02-02T13:27:26.120842+0000","last_became_peered":"2024-02-02T13:27:26.120842+0000","last_unstale":"2024-02-02T13:27:26.122679+0000","last_undegraded":"2024-02-02T13:27:26.122679+0000","last_fullsized":"2024-02-02T13:27:26.122679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:49.348655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.105543+0000","last_change":"2024-02-02T13:27:26.105374+0000","last_active":"2024-02-02T13:27:26.105543+0000","last_peered":"2024-02-02T13:27:26.105543+0000","last_clean":"2024-02-02T13:27:26.105543+0000","last_became_active":"2024-02-02T13:27:26.105169+0000","last_became_peered":"2024-02-02T13:27:26.105169+0000","last_unstale":"2024-02-02T13:27:26.105543+0000","last_undegraded":"2024-02-02T13:27:26.105543+0000","last_fullsized":"2024-02-02T13:27:26.105543+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:31:17.688156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114712+0000","last_change":"2024-02-02T13:27:26.113559+0000","last_active":"2024-02-02T13:27:26.114712+0000","last_peered":"2024-02-02T13:27:26.114712+0000","last_clean":"2024-02-02T13:27:26.114712+0000","last_became_active":"2024-02-02T13:27:26.113297+0000","last_became_peered":"2024-02-02T13:27:26.113297+0000","last_unstale":"2024-02-02T13:27:26.114712+0000","last_undegraded":"2024-02-02T13:27:26.114712+0000","last_fullsized":"2024-02-02T13:27:26.114712+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:23.851535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117777+0000","last_change":"2024-02-02T13:27:26.117717+0000","last_active":"2024-02-02T13:27:26.117777+0000","last_peered":"2024-02-02T13:27:26.117777+0000","last_clean":"2024-02-02T13:27:26.117777+0000","last_became_active":"2024-02-02T13:27:26.117487+0000","last_became_peered":"2024-02-02T13:27:26.117487+0000","last_unstale":"2024-02-02T13:27:26.117777+0000","last_undegraded":"2024-02-02T13:27:26.117777+0000","last_fullsized":"2024-02-02T13:27:26.117777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:29.407482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122547+0000","last_change":"2024-02-02T13:27:25.666042+0000","last_active":"2024-02-02T13:27:26.122547+0000","last_peered":"2024-02-02T13:27:26.122547+0000","last_clean":"2024-02-02T13:27:26.122547+0000","last_became_active":"2024-02-02T13:27:25.665568+0000","last_became_peered":"2024-02-02T13:27:25.665568+0000","last_unstale":"2024-02-02T13:27:26.122547+0000","last_undegraded":"2024-02-02T13:27:26.122547+0000","last_fullsized":"2024-02-02T13:27:26.122547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:14.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117327+0000","last_change":"2024-02-02T13:27:26.117162+0000","last_active":"2024-02-02T13:27:26.117327+0000","last_peered":"2024-02-02T13:27:26.117327+0000","last_clean":"2024-02-02T13:27:26.117327+0000","last_became_active":"2024-02-02T13:27:26.116939+0000","last_became_peered":"2024-02-02T13:27:26.116939+0000","last_unstale":"2024-02-02T13:27:26.117327+0000","last_undegraded":"2024-02-02T13:27:26.117327+0000","last_fullsized":"2024-02-02T13:27:26.117327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:45.570715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.124007+0000","last_change":"2024-02-02T13:27:25.662294+0000","last_active":"2024-02-02T13:27:26.124007+0000","last_peered":"2024-02-02T13:27:26.124007+0000","last_clean":"2024-02-02T13:27:26.124007+0000","last_became_active":"2024-02-02T13:27:25.661816+0000","last_became_peered":"2024-02-02T13:27:25.661816+0000","last_unstale":"2024-02-02T13:27:26.124007+0000","last_undegraded":"2024-02-02T13:27:26.124007+0000","last_fullsized":"2024-02-02T13:27:26.124007+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:56.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116570+0000","last_change":"2024-02-02T13:27:26.116515+0000","last_active":"2024-02-02T13:27:26.116570+0000","last_peered":"2024-02-02T13:27:26.116570+0000","last_clean":"2024-02-02T13:27:26.116570+0000","last_became_active":"2024-02-02T13:27:26.116267+0000","last_became_peered":"2024-02-02T13:27:26.116267+0000","last_unstale":"2024-02-02T13:27:26.116570+0000","last_undegraded":"2024-02-02T13:27:26.116570+0000","last_fullsized":"2024-02-02T13:27:26.116570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:49.348655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.118886+0000","last_change":"2024-02-02T13:27:25.665068+0000","last_active":"2024-02-02T13:27:26.118886+0000","last_peered":"2024-02-02T13:27:26.118886+0000","last_clean":"2024-02-02T13:27:26.118886+0000","last_became_active":"2024-02-02T13:27:25.664511+0000","last_became_peered":"2024-02-02T13:27:25.664511+0000","last_unstale":"2024-02-02T13:27:26.118886+0000","last_undegraded":"2024-02-02T13:27:26.118886+0000","last_fullsized":"2024-02-02T13:27:26.118886+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:44.560009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115953+0000","last_change":"2024-02-02T13:27:26.115881+0000","last_active":"2024-02-02T13:27:26.115953+0000","last_peered":"2024-02-02T13:27:26.115953+0000","last_clean":"2024-02-02T13:27:26.115953+0000","last_became_active":"2024-02-02T13:27:26.115461+0000","last_became_peered":"2024-02-02T13:27:26.115461+0000","last_unstale":"2024-02-02T13:27:26.115953+0000","last_undegraded":"2024-02-02T13:27:26.115953+0000","last_fullsized":"2024-02-02T13:27:26.115953+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:06:34.432082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127622+0000","last_change":"2024-02-02T13:27:25.669597+0000","last_active":"2024-02-02T13:27:26.127622+0000","last_peered":"2024-02-02T13:27:26.127622+0000","last_clean":"2024-02-02T13:27:26.127622+0000","last_became_active":"2024-02-02T13:27:25.668620+0000","last_became_peered":"2024-02-02T13:27:25.668620+0000","last_unstale":"2024-02-02T13:27:26.127622+0000","last_undegraded":"2024-02-02T13:27:26.127622+0000","last_fullsized":"2024-02-02T13:27:26.127622+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:16.574135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103791+0000","last_change":"2024-02-02T13:27:26.103729+0000","last_active":"2024-02-02T13:27:26.103791+0000","last_peered":"2024-02-02T13:27:26.103791+0000","last_clean":"2024-02-02T13:27:26.103791+0000","last_became_active":"2024-02-02T13:27:26.103510+0000","last_became_peered":"2024-02-02T13:27:26.103510+0000","last_unstale":"2024-02-02T13:27:26.103791+0000","last_undegraded":"2024-02-02T13:27:26.103791+0000","last_fullsized":"2024-02-02T13:27:26.103791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:17.038957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.118127+0000","last_change":"2024-02-02T13:27:25.665909+0000","last_active":"2024-02-02T13:27:26.118127+0000","last_peered":"2024-02-02T13:27:26.118127+0000","last_clean":"2024-02-02T13:27:26.118127+0000","last_became_active":"2024-02-02T13:27:25.665647+0000","last_became_peered":"2024-02-02T13:27:25.665647+0000","last_unstale":"2024-02-02T13:27:26.118127+0000","last_undegraded":"2024-02-02T13:27:26.118127+0000","last_fullsized":"2024-02-02T13:27:26.118127+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:13.408003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115392+0000","last_change":"2024-02-02T13:27:26.115206+0000","last_active":"2024-02-02T13:27:26.115392+0000","last_peered":"2024-02-02T13:27:26.115392+0000","last_clean":"2024-02-02T13:27:26.115392+0000","last_became_active":"2024-02-02T13:27:26.114887+0000","last_became_peered":"2024-02-02T13:27:26.114887+0000","last_unstale":"2024-02-02T13:27:26.115392+0000","last_undegraded":"2024-02-02T13:27:26.115392+0000","last_fullsized":"2024-02-02T13:27:26.115392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:42:18.679208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.123593+0000","last_change":"2024-02-02T13:27:25.665273+0000","last_active":"2024-02-02T13:27:26.123593+0000","last_peered":"2024-02-02T13:27:26.123593+0000","last_clean":"2024-02-02T13:27:26.123593+0000","last_became_active":"2024-02-02T13:27:25.664723+0000","last_became_peered":"2024-02-02T13:27:25.664723+0000","last_unstale":"2024-02-02T13:27:26.123593+0000","last_undegraded":"2024-02-02T13:27:26.123593+0000","last_fullsized":"2024-02-02T13:27:26.123593+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:16.574135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126885+0000","last_change":"2024-02-02T13:27:26.126739+0000","last_active":"2024-02-02T13:27:26.126885+0000","last_peered":"2024-02-02T13:27:26.126885+0000","last_clean":"2024-02-02T13:27:26.126885+0000","last_became_active":"2024-02-02T13:27:26.126412+0000","last_became_peered":"2024-02-02T13:27:26.126412+0000","last_unstale":"2024-02-02T13:27:26.126885+0000","last_undegraded":"2024-02-02T13:27:26.126885+0000","last_fullsized":"2024-02-02T13:27:26.126885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:02.103857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101153+0000","last_change":"2024-02-02T13:27:25.669110+0000","last_active":"2024-02-02T13:27:26.101153+0000","last_peered":"2024-02-02T13:27:26.101153+0000","last_clean":"2024-02-02T13:27:26.101153+0000","last_became_active":"2024-02-02T13:27:25.668648+0000","last_became_peered":"2024-02-02T13:27:25.668648+0000","last_unstale":"2024-02-02T13:27:26.101153+0000","last_undegraded":"2024-02-02T13:27:26.101153+0000","last_fullsized":"2024-02-02T13:27:26.101153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:14.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127669+0000","last_change":"2024-02-02T13:27:26.127605+0000","last_active":"2024-02-02T13:27:26.127669+0000","last_peered":"2024-02-02T13:27:26.127669+0000","last_clean":"2024-02-02T13:27:26.127669+0000","last_became_active":"2024-02-02T13:27:26.127381+0000","last_became_peered":"2024-02-02T13:27:26.127381+0000","last_unstale":"2024-02-02T13:27:26.127669+0000","last_undegraded":"2024-02-02T13:27:26.127669+0000","last_fullsized":"2024-02-02T13:27:26.127669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:05.755303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103063+0000","last_change":"2024-02-02T13:27:25.666396+0000","last_active":"2024-02-02T13:27:26.103063+0000","last_peered":"2024-02-02T13:27:26.103063+0000","last_clean":"2024-02-02T13:27:26.103063+0000","last_became_active":"2024-02-02T13:27:25.664470+0000","last_became_peered":"2024-02-02T13:27:25.664470+0000","last_unstale":"2024-02-02T13:27:26.103063+0000","last_undegraded":"2024-02-02T13:27:26.103063+0000","last_fullsized":"2024-02-02T13:27:26.103063+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:01.093150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114673+0000","last_change":"2024-02-02T13:27:26.111554+0000","last_active":"2024-02-02T13:27:26.114673+0000","last_peered":"2024-02-02T13:27:26.114673+0000","last_clean":"2024-02-02T13:27:26.114673+0000","last_became_active":"2024-02-02T13:27:26.110890+0000","last_became_peered":"2024-02-02T13:27:26.110890+0000","last_unstale":"2024-02-02T13:27:26.114673+0000","last_undegraded":"2024-02-02T13:27:26.114673+0000","last_fullsized":"2024-02-02T13:27:26.114673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:09.075185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113750+0000","last_change":"2024-02-02T13:27:25.666478+0000","last_active":"2024-02-02T13:27:26.113750+0000","last_peered":"2024-02-02T13:27:26.113750+0000","last_clean":"2024-02-02T13:27:26.113750+0000","last_became_active":"2024-02-02T13:27:25.665736+0000","last_became_peered":"2024-02-02T13:27:25.665736+0000","last_unstale":"2024-02-02T13:27:26.113750+0000","last_undegraded":"2024-02-02T13:27:26.113750+0000","last_fullsized":"2024-02-02T13:27:26.113750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122718+0000","last_change":"2024-02-02T13:27:26.122308+0000","last_active":"2024-02-02T13:27:26.122718+0000","last_peered":"2024-02-02T13:27:26.122718+0000","last_clean":"2024-02-02T13:27:26.122718+0000","last_became_active":"2024-02-02T13:27:26.121743+0000","last_became_peered":"2024-02-02T13:27:26.121743+0000","last_unstale":"2024-02-02T13:27:26.122718+0000","last_undegraded":"2024-02-02T13:27:26.122718+0000","last_fullsized":"2024-02-02T13:27:26.122718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:04:06.503908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.099472+0000","last_change":"2024-02-02T13:27:25.669823+0000","last_active":"2024-02-02T13:27:26.099472+0000","last_peered":"2024-02-02T13:27:26.099472+0000","last_clean":"2024-02-02T13:27:26.099472+0000","last_became_active":"2024-02-02T13:27:25.669597+0000","last_became_peered":"2024-02-02T13:27:25.669597+0000","last_unstale":"2024-02-02T13:27:26.099472+0000","last_undegraded":"2024-02-02T13:27:26.099472+0000","last_fullsized":"2024-02-02T13:27:26.099472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:28.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113388+0000","last_change":"2024-02-02T13:27:26.112364+0000","last_active":"2024-02-02T13:27:26.113388+0000","last_peered":"2024-02-02T13:27:26.113388+0000","last_clean":"2024-02-02T13:27:26.113388+0000","last_became_active":"2024-02-02T13:27:26.112156+0000","last_became_peered":"2024-02-02T13:27:26.112156+0000","last_unstale":"2024-02-02T13:27:26.113388+0000","last_undegraded":"2024-02-02T13:27:26.113388+0000","last_fullsized":"2024-02-02T13:27:26.113388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:52.843663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.131479+0000","last_change":"2024-02-02T13:27:25.651369+0000","last_active":"2024-02-02T13:27:26.131479+0000","last_peered":"2024-02-02T13:27:26.131479+0000","last_clean":"2024-02-02T13:27:26.131479+0000","last_became_active":"2024-02-02T13:27:25.650186+0000","last_became_peered":"2024-02-02T13:27:25.650186+0000","last_unstale":"2024-02-02T13:27:26.131479+0000","last_undegraded":"2024-02-02T13:27:26.131479+0000","last_fullsized":"2024-02-02T13:27:26.131479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:31.298679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122815+0000","last_change":"2024-02-02T13:27:26.122729+0000","last_active":"2024-02-02T13:27:26.122815+0000","last_peered":"2024-02-02T13:27:26.122815+0000","last_clean":"2024-02-02T13:27:26.122815+0000","last_became_active":"2024-02-02T13:27:26.116728+0000","last_became_peered":"2024-02-02T13:27:26.116728+0000","last_unstale":"2024-02-02T13:27:26.122815+0000","last_undegraded":"2024-02-02T13:27:26.122815+0000","last_fullsized":"2024-02-02T13:27:26.122815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:09:57.959347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127797+0000","last_change":"2024-02-02T13:27:25.663185+0000","last_active":"2024-02-02T13:27:26.127797+0000","last_peered":"2024-02-02T13:27:26.127797+0000","last_clean":"2024-02-02T13:27:26.127797+0000","last_became_active":"2024-02-02T13:27:25.662468+0000","last_became_peered":"2024-02-02T13:27:25.662468+0000","last_unstale":"2024-02-02T13:27:26.127797+0000","last_undegraded":"2024-02-02T13:27:26.127797+0000","last_fullsized":"2024-02-02T13:27:26.127797+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:15.800371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120560+0000","last_change":"2024-02-02T13:27:26.120001+0000","last_active":"2024-02-02T13:27:26.120560+0000","last_peered":"2024-02-02T13:27:26.120560+0000","last_clean":"2024-02-02T13:27:26.120560+0000","last_became_active":"2024-02-02T13:27:26.116707+0000","last_became_peered":"2024-02-02T13:27:26.116707+0000","last_unstale":"2024-02-02T13:27:26.120560+0000","last_undegraded":"2024-02-02T13:27:26.120560+0000","last_fullsized":"2024-02-02T13:27:26.120560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115365+0000","last_change":"2024-02-02T13:27:25.669498+0000","last_active":"2024-02-02T13:27:26.115365+0000","last_peered":"2024-02-02T13:27:26.115365+0000","last_clean":"2024-02-02T13:27:26.115365+0000","last_became_active":"2024-02-02T13:27:25.669233+0000","last_became_peered":"2024-02-02T13:27:25.669233+0000","last_unstale":"2024-02-02T13:27:26.115365+0000","last_undegraded":"2024-02-02T13:27:26.115365+0000","last_fullsized":"2024-02-02T13:27:26.115365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:26.481824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.129021+0000","last_change":"2024-02-02T13:27:26.128706+0000","last_active":"2024-02-02T13:27:26.129021+0000","last_peered":"2024-02-02T13:27:26.129021+0000","last_clean":"2024-02-02T13:27:26.129021+0000","last_became_active":"2024-02-02T13:27:26.128328+0000","last_became_peered":"2024-02-02T13:27:26.128328+0000","last_unstale":"2024-02-02T13:27:26.129021+0000","last_undegraded":"2024-02-02T13:27:26.129021+0000","last_fullsized":"2024-02-02T13:27:26.129021+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:25.612056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122777+0000","last_change":"2024-02-02T13:27:25.661775+0000","last_active":"2024-02-02T13:27:26.122777+0000","last_peered":"2024-02-02T13:27:26.122777+0000","last_clean":"2024-02-02T13:27:26.122777+0000","last_became_active":"2024-02-02T13:27:25.661539+0000","last_became_peered":"2024-02-02T13:27:25.661539+0000","last_unstale":"2024-02-02T13:27:26.122777+0000","last_undegraded":"2024-02-02T13:27:26.122777+0000","last_fullsized":"2024-02-02T13:27:26.122777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:27.586262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116200+0000","last_change":"2024-02-02T13:27:26.115961+0000","last_active":"2024-02-02T13:27:26.116200+0000","last_peered":"2024-02-02T13:27:26.116200+0000","last_clean":"2024-02-02T13:27:26.116200+0000","last_became_active":"2024-02-02T13:27:26.115696+0000","last_became_peered":"2024-02-02T13:27:26.115696+0000","last_unstale":"2024-02-02T13:27:26.116200+0000","last_undegraded":"2024-02-02T13:27:26.116200+0000","last_fullsized":"2024-02-02T13:27:26.116200+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:53.881018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.099377+0000","last_change":"2024-02-02T13:27:25.665929+0000","last_active":"2024-02-02T13:27:26.099377+0000","last_peered":"2024-02-02T13:27:26.099377+0000","last_clean":"2024-02-02T13:27:26.099377+0000","last_became_active":"2024-02-02T13:27:25.665707+0000","last_became_peered":"2024-02-02T13:27:25.665707+0000","last_unstale":"2024-02-02T13:27:26.099377+0000","last_undegraded":"2024-02-02T13:27:26.099377+0000","last_fullsized":"2024-02-02T13:27:26.099377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:31.298679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126663+0000","last_change":"2024-02-02T13:27:26.125559+0000","last_active":"2024-02-02T13:27:26.126663+0000","last_peered":"2024-02-02T13:27:26.126663+0000","last_clean":"2024-02-02T13:27:26.126663+0000","last_became_active":"2024-02-02T13:27:26.125274+0000","last_became_peered":"2024-02-02T13:27:26.125274+0000","last_unstale":"2024-02-02T13:27:26.126663+0000","last_undegraded":"2024-02-02T13:27:26.126663+0000","last_fullsized":"2024-02-02T13:27:26.126663+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.130842+0000","last_change":"2024-02-02T13:27:25.667108+0000","last_active":"2024-02-02T13:27:26.130842+0000","last_peered":"2024-02-02T13:27:26.130842+0000","last_clean":"2024-02-02T13:27:26.130842+0000","last_became_active":"2024-02-02T13:27:25.665605+0000","last_became_peered":"2024-02-02T13:27:25.665605+0000","last_unstale":"2024-02-02T13:27:26.130842+0000","last_undegraded":"2024-02-02T13:27:26.130842+0000","last_fullsized":"2024-02-02T13:27:26.130842+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:52.870311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122700+0000","last_change":"2024-02-02T13:27:26.122160+0000","last_active":"2024-02-02T13:27:26.122700+0000","last_peered":"2024-02-02T13:27:26.122700+0000","last_clean":"2024-02-02T13:27:26.122700+0000","last_became_active":"2024-02-02T13:27:26.121180+0000","last_became_peered":"2024-02-02T13:27:26.121180+0000","last_unstale":"2024-02-02T13:27:26.122700+0000","last_undegraded":"2024-02-02T13:27:26.122700+0000","last_fullsized":"2024-02-02T13:27:26.122700+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.492531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114044+0000","last_change":"2024-02-02T13:27:25.668931+0000","last_active":"2024-02-02T13:27:26.114044+0000","last_peered":"2024-02-02T13:27:26.114044+0000","last_clean":"2024-02-02T13:27:26.114044+0000","last_became_active":"2024-02-02T13:27:25.668721+0000","last_became_peered":"2024-02-02T13:27:25.668721+0000","last_unstale":"2024-02-02T13:27:26.114044+0000","last_undegraded":"2024-02-02T13:27:26.114044+0000","last_fullsized":"2024-02-02T13:27:26.114044+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:10.622871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120509+0000","last_change":"2024-02-02T13:27:26.119432+0000","last_active":"2024-02-02T13:27:26.120509+0000","last_peered":"2024-02-02T13:27:26.120509+0000","last_clean":"2024-02-02T13:27:26.120509+0000","last_became_active":"2024-02-02T13:27:26.116931+0000","last_became_peered":"2024-02-02T13:27:26.116931+0000","last_unstale":"2024-02-02T13:27:26.120509+0000","last_undegraded":"2024-02-02T13:27:26.120509+0000","last_fullsized":"2024-02-02T13:27:26.120509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:04:06.503908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.116444+0000","last_change":"2024-02-02T13:27:25.668142+0000","last_active":"2024-02-02T13:27:26.116444+0000","last_peered":"2024-02-02T13:27:26.116444+0000","last_clean":"2024-02-02T13:27:26.116444+0000","last_became_active":"2024-02-02T13:27:25.667863+0000","last_became_peered":"2024-02-02T13:27:25.667863+0000","last_unstale":"2024-02-02T13:27:26.116444+0000","last_undegraded":"2024-02-02T13:27:26.116444+0000","last_fullsized":"2024-02-02T13:27:26.116444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:07:07.246959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.114880+0000","last_change":"2024-02-02T13:27:26.114679+0000","last_active":"2024-02-02T13:27:26.114880+0000","last_peered":"2024-02-02T13:27:26.114880+0000","last_clean":"2024-02-02T13:27:26.114880+0000","last_became_active":"2024-02-02T13:27:26.114409+0000","last_became_peered":"2024-02-02T13:27:26.114409+0000","last_unstale":"2024-02-02T13:27:26.114880+0000","last_undegraded":"2024-02-02T13:27:26.114880+0000","last_fullsized":"2024-02-02T13:27:26.114880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:14.091987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.129250+0000","last_change":"2024-02-02T13:27:25.669444+0000","last_active":"2024-02-02T13:27:26.129250+0000","last_peered":"2024-02-02T13:27:26.129250+0000","last_clean":"2024-02-02T13:27:26.129250+0000","last_became_active":"2024-02-02T13:27:25.668306+0000","last_became_peered":"2024-02-02T13:27:25.668306+0000","last_unstale":"2024-02-02T13:27:26.129250+0000","last_undegraded":"2024-02-02T13:27:26.129250+0000","last_fullsized":"2024-02-02T13:27:26.129250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122734+0000","last_change":"2024-02-02T13:27:26.122446+0000","last_active":"2024-02-02T13:27:26.122734+0000","last_peered":"2024-02-02T13:27:26.122734+0000","last_clean":"2024-02-02T13:27:26.122734+0000","last_became_active":"2024-02-02T13:27:26.122080+0000","last_became_peered":"2024-02-02T13:27:26.122080+0000","last_unstale":"2024-02-02T13:27:26.122734+0000","last_undegraded":"2024-02-02T13:27:26.122734+0000","last_fullsized":"2024-02-02T13:27:26.122734+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:48.938964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113164+0000","last_change":"2024-02-02T13:27:25.666285+0000","last_active":"2024-02-02T13:27:26.113164+0000","last_peered":"2024-02-02T13:27:26.113164+0000","last_clean":"2024-02-02T13:27:26.113164+0000","last_became_active":"2024-02-02T13:27:25.666036+0000","last_became_peered":"2024-02-02T13:27:25.666036+0000","last_unstale":"2024-02-02T13:27:26.113164+0000","last_undegraded":"2024-02-02T13:27:26.113164+0000","last_fullsized":"2024-02-02T13:27:26.113164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:51.711449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120530+0000","last_change":"2024-02-02T13:27:26.119637+0000","last_active":"2024-02-02T13:27:26.120530+0000","last_peered":"2024-02-02T13:27:26.120530+0000","last_clean":"2024-02-02T13:27:26.120530+0000","last_became_active":"2024-02-02T13:27:26.118707+0000","last_became_peered":"2024-02-02T13:27:26.118707+0000","last_unstale":"2024-02-02T13:27:26.120530+0000","last_undegraded":"2024-02-02T13:27:26.120530+0000","last_fullsized":"2024-02-02T13:27:26.120530+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:48.938964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115921+0000","last_change":"2024-02-02T13:27:25.665610+0000","last_active":"2024-02-02T13:27:26.115921+0000","last_peered":"2024-02-02T13:27:26.115921+0000","last_clean":"2024-02-02T13:27:26.115921+0000","last_became_active":"2024-02-02T13:27:25.664118+0000","last_became_peered":"2024-02-02T13:27:25.664118+0000","last_unstale":"2024-02-02T13:27:26.115921+0000","last_undegraded":"2024-02-02T13:27:26.115921+0000","last_fullsized":"2024-02-02T13:27:26.115921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:47.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127916+0000","last_change":"2024-02-02T13:27:26.127752+0000","last_active":"2024-02-02T13:27:26.127916+0000","last_peered":"2024-02-02T13:27:26.127916+0000","last_clean":"2024-02-02T13:27:26.127916+0000","last_became_active":"2024-02-02T13:27:26.127505+0000","last_became_peered":"2024-02-02T13:27:26.127505+0000","last_unstale":"2024-02-02T13:27:26.127916+0000","last_undegraded":"2024-02-02T13:27:26.127916+0000","last_fullsized":"2024-02-02T13:27:26.127916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:18:00.421164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.100512+0000","last_change":"2024-02-02T13:27:25.667490+0000","last_active":"2024-02-02T13:27:26.100512+0000","last_peered":"2024-02-02T13:27:26.100512+0000","last_clean":"2024-02-02T13:27:26.100512+0000","last_became_active":"2024-02-02T13:27:25.667277+0000","last_became_peered":"2024-02-02T13:27:25.667277+0000","last_unstale":"2024-02-02T13:27:26.100512+0000","last_undegraded":"2024-02-02T13:27:26.100512+0000","last_fullsized":"2024-02-02T13:27:26.100512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:33:16.075836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.117672+0000","last_change":"2024-02-02T13:27:26.117401+0000","last_active":"2024-02-02T13:27:26.117672+0000","last_peered":"2024-02-02T13:27:26.117672+0000","last_clean":"2024-02-02T13:27:26.117672+0000","last_became_active":"2024-02-02T13:27:26.117203+0000","last_became_peered":"2024-02-02T13:27:26.117203+0000","last_unstale":"2024-02-02T13:27:26.117672+0000","last_undegraded":"2024-02-02T13:27:26.117672+0000","last_fullsized":"2024-02-02T13:27:26.117672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:11.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.101348+0000","last_change":"2024-02-02T13:27:25.665019+0000","last_active":"2024-02-02T13:27:26.101348+0000","last_peered":"2024-02-02T13:27:26.101348+0000","last_clean":"2024-02-02T13:27:26.101348+0000","last_became_active":"2024-02-02T13:27:25.664754+0000","last_became_peered":"2024-02-02T13:27:25.664754+0000","last_unstale":"2024-02-02T13:27:26.101348+0000","last_undegraded":"2024-02-02T13:27:26.101348+0000","last_fullsized":"2024-02-02T13:27:26.101348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:52.870311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110842+0000","last_change":"2024-02-02T13:27:26.109392+0000","last_active":"2024-02-02T13:27:26.110842+0000","last_peered":"2024-02-02T13:27:26.110842+0000","last_clean":"2024-02-02T13:27:26.110842+0000","last_became_active":"2024-02-02T13:27:26.108813+0000","last_became_peered":"2024-02-02T13:27:26.108813+0000","last_unstale":"2024-02-02T13:27:26.110842+0000","last_undegraded":"2024-02-02T13:27:26.110842+0000","last_fullsized":"2024-02-02T13:27:26.110842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:29.407482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.110166+0000","last_change":"2024-02-02T13:27:25.651712+0000","last_active":"2024-02-02T13:27:26.110166+0000","last_peered":"2024-02-02T13:27:26.110166+0000","last_clean":"2024-02-02T13:27:26.110166+0000","last_became_active":"2024-02-02T13:27:25.650994+0000","last_became_peered":"2024-02-02T13:27:25.650994+0000","last_unstale":"2024-02-02T13:27:26.110166+0000","last_undegraded":"2024-02-02T13:27:26.110166+0000","last_fullsized":"2024-02-02T13:27:26.110166+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:24.038059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.120409+0000","last_change":"2024-02-02T13:27:26.117981+0000","last_active":"2024-02-02T13:27:26.120409+0000","last_peered":"2024-02-02T13:27:26.120409+0000","last_clean":"2024-02-02T13:27:26.120409+0000","last_became_active":"2024-02-02T13:27:26.115039+0000","last_became_peered":"2024-02-02T13:27:26.115039+0000","last_unstale":"2024-02-02T13:27:26.120409+0000","last_undegraded":"2024-02-02T13:27:26.120409+0000","last_fullsized":"2024-02-02T13:27:26.120409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.111599+0000","last_change":"2024-02-02T13:27:25.665128+0000","last_active":"2024-02-02T13:27:26.111599+0000","last_peered":"2024-02-02T13:27:26.111599+0000","last_clean":"2024-02-02T13:27:26.111599+0000","last_became_active":"2024-02-02T13:27:25.664377+0000","last_became_peered":"2024-02-02T13:27:25.664377+0000","last_unstale":"2024-02-02T13:27:26.111599+0000","last_undegraded":"2024-02-02T13:27:26.111599+0000","last_fullsized":"2024-02-02T13:27:26.111599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:51.832956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.113373+0000","last_change":"2024-02-02T13:27:26.112360+0000","last_active":"2024-02-02T13:27:26.113373+0000","last_peered":"2024-02-02T13:27:26.113373+0000","last_clean":"2024-02-02T13:27:26.113373+0000","last_became_active":"2024-02-02T13:27:26.112071+0000","last_became_peered":"2024-02-02T13:27:26.112071+0000","last_unstale":"2024-02-02T13:27:26.113373+0000","last_undegraded":"2024-02-02T13:27:26.113373+0000","last_fullsized":"2024-02-02T13:27:26.113373+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:29:57.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122268+0000","last_change":"2024-02-02T13:27:25.663143+0000","last_active":"2024-02-02T13:27:26.122268+0000","last_peered":"2024-02-02T13:27:26.122268+0000","last_clean":"2024-02-02T13:27:26.122268+0000","last_became_active":"2024-02-02T13:27:25.662655+0000","last_became_peered":"2024-02-02T13:27:25.662655+0000","last_unstale":"2024-02-02T13:27:26.122268+0000","last_undegraded":"2024-02-02T13:27:26.122268+0000","last_fullsized":"2024-02-02T13:27:26.122268+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:42:34.990661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.122832+0000","last_change":"2024-02-02T13:27:26.122794+0000","last_active":"2024-02-02T13:27:26.122832+0000","last_peered":"2024-02-02T13:27:26.122832+0000","last_clean":"2024-02-02T13:27:26.122832+0000","last_became_active":"2024-02-02T13:27:26.122630+0000","last_became_peered":"2024-02-02T13:27:26.122630+0000","last_unstale":"2024-02-02T13:27:26.122832+0000","last_undegraded":"2024-02-02T13:27:26.122832+0000","last_fullsized":"2024-02-02T13:27:26.122832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.100991+0000","last_change":"2024-02-02T13:27:25.668633+0000","last_active":"2024-02-02T13:27:26.100991+0000","last_peered":"2024-02-02T13:27:26.100991+0000","last_clean":"2024-02-02T13:27:26.100991+0000","last_became_active":"2024-02-02T13:27:25.667737+0000","last_became_peered":"2024-02-02T13:27:25.667737+0000","last_unstale":"2024-02-02T13:27:26.100991+0000","last_undegraded":"2024-02-02T13:27:26.100991+0000","last_fullsized":"2024-02-02T13:27:26.100991+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:08.064478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.108237+0000","last_change":"2024-02-02T13:27:26.107991+0000","last_active":"2024-02-02T13:27:26.108237+0000","last_peered":"2024-02-02T13:27:26.108237+0000","last_clean":"2024-02-02T13:27:26.108237+0000","last_became_active":"2024-02-02T13:27:26.107300+0000","last_became_peered":"2024-02-02T13:27:26.107300+0000","last_unstale":"2024-02-02T13:27:26.108237+0000","last_undegraded":"2024-02-02T13:27:26.108237+0000","last_fullsized":"2024-02-02T13:27:26.108237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:16:56.636011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.103366+0000","last_change":"2024-02-02T13:27:25.666659+0000","last_active":"2024-02-02T13:27:26.103366+0000","last_peered":"2024-02-02T13:27:26.103366+0000","last_clean":"2024-02-02T13:27:26.103366+0000","last_became_active":"2024-02-02T13:27:25.666387+0000","last_became_peered":"2024-02-02T13:27:25.666387+0000","last_unstale":"2024-02-02T13:27:26.103366+0000","last_undegraded":"2024-02-02T13:27:26.103366+0000","last_fullsized":"2024-02-02T13:27:26.103366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:42:34.990661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.126646+0000","last_change":"2024-02-02T13:27:26.124998+0000","last_active":"2024-02-02T13:27:26.126646+0000","last_peered":"2024-02-02T13:27:26.126646+0000","last_clean":"2024-02-02T13:27:26.126646+0000","last_became_active":"2024-02-02T13:27:26.124577+0000","last_became_peered":"2024-02-02T13:27:26.124577+0000","last_unstale":"2024-02-02T13:27:26.126646+0000","last_undegraded":"2024-02-02T13:27:26.126646+0000","last_fullsized":"2024-02-02T13:27:26.126646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:48.356626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.106250+0000","last_change":"2024-02-02T13:27:25.663167+0000","last_active":"2024-02-02T13:27:26.106250+0000","last_peered":"2024-02-02T13:27:26.106250+0000","last_clean":"2024-02-02T13:27:26.106250+0000","last_became_active":"2024-02-02T13:27:25.662712+0000","last_became_peered":"2024-02-02T13:27:25.662712+0000","last_unstale":"2024-02-02T13:27:26.106250+0000","last_undegraded":"2024-02-02T13:27:26.106250+0000","last_fullsized":"2024-02-02T13:27:26.106250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:17:46.476521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.127434+0000","last_change":"2024-02-02T13:27:26.127356+0000","last_active":"2024-02-02T13:27:26.127434+0000","last_peered":"2024-02-02T13:27:26.127434+0000","last_clean":"2024-02-02T13:27:26.127434+0000","last_became_active":"2024-02-02T13:27:26.127123+0000","last_became_peered":"2024-02-02T13:27:26.127123+0000","last_unstale":"2024-02-02T13:27:26.127434+0000","last_undegraded":"2024-02-02T13:27:26.127434+0000","last_fullsized":"2024-02-02T13:27:26.127434+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:46:53.881018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.102194+0000","last_change":"2024-02-02T13:27:25.664917+0000","last_active":"2024-02-02T13:27:26.102194+0000","last_peered":"2024-02-02T13:27:26.102194+0000","last_clean":"2024-02-02T13:27:26.102194+0000","last_became_active":"2024-02-02T13:27:25.664676+0000","last_became_peered":"2024-02-02T13:27:25.664676+0000","last_unstale":"2024-02-02T13:27:26.102194+0000","last_undegraded":"2024-02-02T13:27:26.102194+0000","last_fullsized":"2024-02-02T13:27:26.102194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:06.846491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.125839+0000","last_change":"2024-02-02T13:27:26.125780+0000","last_active":"2024-02-02T13:27:26.125839+0000","last_peered":"2024-02-02T13:27:26.125839+0000","last_clean":"2024-02-02T13:27:26.125839+0000","last_became_active":"2024-02-02T13:27:26.125632+0000","last_became_peered":"2024-02-02T13:27:26.125632+0000","last_unstale":"2024-02-02T13:27:26.125839+0000","last_undegraded":"2024-02-02T13:27:26.125839+0000","last_fullsized":"2024-02-02T13:27:26.125839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:56:09.722914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.096756+0000","last_change":"2024-02-02T13:27:25.666195+0000","last_active":"2024-02-02T13:27:26.096756+0000","last_peered":"2024-02-02T13:27:26.096756+0000","last_clean":"2024-02-02T13:27:26.096756+0000","last_became_active":"2024-02-02T13:27:25.665190+0000","last_became_peered":"2024-02-02T13:27:25.665190+0000","last_unstale":"2024-02-02T13:27:26.096756+0000","last_undegraded":"2024-02-02T13:27:26.096756+0000","last_fullsized":"2024-02-02T13:27:26.096756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:02:47.345920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.115690+0000","last_change":"2024-02-02T13:27:26.115626+0000","last_active":"2024-02-02T13:27:26.115690+0000","last_peered":"2024-02-02T13:27:26.115690+0000","last_clean":"2024-02-02T13:27:26.115690+0000","last_became_active":"2024-02-02T13:27:26.114862+0000","last_became_peered":"2024-02-02T13:27:26.114862+0000","last_unstale":"2024-02-02T13:27:26.115690+0000","last_undegraded":"2024-02-02T13:27:26.115690+0000","last_fullsized":"2024-02-02T13:27:26.115690+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:25.631103+0000","last_clean_scrub_stamp":"2024-02-02T13:27:25.631103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:58:01.195901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-02-02T13:27:26.121082+0000","last_change":"2024-02-02T13:27:25.670077+0000","last_active":"2024-02-02T13:27:26.121082+0000","last_peered":"2024-02-02T13:27:26.121082+0000","last_clean":"2024-02-02T13:27:26.121082+0000","last_became_active":"2024-02-02T13:27:25.669809+0000","last_became_peered":"2024-02-02T13:27:25.669809+0000","last_unstale":"2024-02-02T13:27:26.121082+0000","last_undegraded":"2024-02-02T13:27:26.121082+0000","last_fullsized":"2024-02-02T13:27:26.121082+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:27:24.620396+0000","last_clean_scrub_stamp":"2024-02-02T13:27:24.620396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:43.341036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":209,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-02-02T13:27:25.643137+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:27:25.643137+0000","last_peered":"2024-02-02T13:27:25.643137+0000","last_clean":"2024-02-02T13:27:25.643137+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:27:25.643137+0000","last_undegraded":"2024-02-02T13:27:25.643137+0000","last_fullsized":"2024-02-02T13:27:25.643137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065509,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0580000000000001,"15min":1.052},"min":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5880000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.123,"15min":1.1299999999999999},"min":{"1min":0.76900000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5589999999999999,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.2010000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.083},"min":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":1.893,"15min":1.893},"last":1.268},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.105,"15min":1.099},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4099999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.365}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.208,"15min":1.2110000000000001},"min":{"1min":0.78100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.522},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.159,"15min":1.181},"min":{"1min":0.67500000000000004,"5min":0.623,"15min":0.623},"max":{"1min":1.615,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.585}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.327,"5min":1.819,"15min":1.819},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.661,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.647}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.155,"15min":1.159},"min":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.302,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6619999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.621}]},{"osd":5,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.347,"15min":1.343},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.6799999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7130000000000001,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.8200000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5209999999999999,"15min":1.5409999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8049999999999999,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.6299999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.788}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.012,"15min":0.996},"min":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5089999999999999,"5min":1.577,"15min":1.577},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.038,"5min":1.002,"15min":0.996},"min":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.488,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.242,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.0589999999999999,"15min":1.048},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.048,"15min":1.038},"min":{"1min":0.80200000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.385},{"interface":"front","average":{"1min":1.137,"5min":1.143,"15min":1.1439999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4079999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5049999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.137},{"interface":"front","average":{"1min":1.149,"5min":1.125,"15min":1.121},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.98299999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.637},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.389,"15min":1.3819999999999999},"min":{"1min":0.89900000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.696,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.54}]},{"osd":5,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2589999999999999,"15min":1.266},"min":{"1min":0.77900000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.615,"5min":1.645,"15min":1.645},"last":1.482},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.339,"15min":1.327},"min":{"1min":1.099,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4399999999999999,"15min":1.4419999999999999},"min":{"1min":1.0049999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.694,"5min":1.835,"15min":1.835},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.419,"15min":1.4279999999999999},"min":{"1min":1.1439999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.655,"5min":1.804,"15min":1.804},"last":1.671}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194989,"statfs":{"total":200000000,"available":199669079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.022,"15min":1.018},"min":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0249999999999999,"15min":1.0329999999999999},"min":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.7150000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2629999999999999,"15min":1.2969999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.4990000000000001,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.655},{"interface":"front","average":{"1min":1.075,"5min":1.153,"15min":1.1659999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.444,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.296}]},{"osd":2,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.171,"15min":1.169},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1799999999999999,"15min":1.1830000000000001},"min":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.512,"5min":1.744,"15min":1.744},"last":1.139}]},{"osd":4,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2330000000000001,"15min":1.2350000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2729999999999999,"15min":1.2789999999999999},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.575}]},{"osd":6,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3029999999999999,"15min":1.2929999999999999},"min":{"1min":1.042,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.397},{"interface":"front","average":{"1min":1.389,"5min":1.351,"15min":1.345},"min":{"1min":0.92200000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3979999999999999,"15min":1.397},"min":{"1min":0.91000000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.762},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3640000000000001,"15min":1.353},"min":{"1min":0.88600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.5289999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.96099999999999997,"15min":0.94799999999999995},"min":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.0629999999999999,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.554,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1359999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.78700000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5,"5min":1.506,"15min":1.506},"last":1.298},{"interface":"front","average":{"1min":0.995,"5min":0.999,"15min":1},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.27,"5min":1.345,"15min":1.345},"last":0.93100000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.0660000000000001,"15min":1.0580000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4339999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.327},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0589999999999999,"15min":1.038},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.111}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.161,"15min":1.145},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4490000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.385},{"interface":"front","average":{"1min":1.292,"5min":1.2210000000000001,"15min":1.2090000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.5920000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.236}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.117,"15min":1.1040000000000001},"min":{"1min":0.98199999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.335,"5min":1.2609999999999999,"15min":1.2490000000000001},"min":{"1min":1.008,"5min":0.751,"15min":0.751},"max":{"1min":1.6879999999999999,"5min":2.069,"15min":2.069},"last":1.27}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.90700000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3640000000000001,"15min":1.361},"min":{"1min":1.123,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3580000000000001,"15min":1.3500000000000001},"min":{"1min":1.1659999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.6830000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.90100000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.675,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.349}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194989,"statfs":{"total":200000000,"available":199669079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3959999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":0.998},"min":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3540000000000001,"5min":2.125,"15min":2.125},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.94299999999999995,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.181,"15min":1.177},"min":{"1min":0.91400000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6180000000000001,"5min":3.585,"15min":3.585},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.198,"5min":1.1859999999999999,"15min":1.1839999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6659999999999999,"5min":1.877,"15min":1.877},"last":1.034}]},{"osd":5,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.222,"15min":1.224},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.758,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.163,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.554,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.234,"15min":1.23},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.744,"5min":1.841,"15min":1.841},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.302,"15min":1.306},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6299999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.046}]},{"osd":7,"last update":"Fri Feb 2 13:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5720000000000001,"5min":3.391,"15min":3.391},"last":0.998},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6479999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.85999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194983,"statfs":{"total":200000000,"available":199662812,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.755,"15min":0.77200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1120000000000001,"5min":1.29,"15min":1.29},"last":1.506},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.38200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2549999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.87,"15min":0.88},"min":{"1min":0.50900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.2170000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.246,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.93899999999999995,"15min":0.94999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.3480000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.1379999999999999,"15min":1.129},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.3939999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0589999999999999,"15min":1.0409999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.25},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.669}]},{"osd":6,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1359999999999999,"15min":1.137},"min":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.173,"15min":1.1699999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.2829999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1639999999999999,"15min":1.1579999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.194,"15min":1.1739999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.4159999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194990,"statfs":{"total":200000000,"available":199670098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.022,"15min":1.0649999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":1.391,"15min":1.4910000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5029999999999999,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.371}]},{"osd":2,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.639,"15min":1.639},"last":0.503},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.091,"15min":1.121},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3680000000000001,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.25,"15min":1.3080000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4830000000000001,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.2949999999999999,"15min":1.357},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.528,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.335,"15min":1.377},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3799999999999999,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.3580000000000001,"15min":1.3939999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.4930000000000001,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.472,"15min":1.508},"min":{"1min":1.0609999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.532,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.4470000000000001,"15min":1.4850000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.8959999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.5880000000000001,"15min":1.653},"min":{"1min":0.85099999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.661,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4950000000000001,"15min":1.5429999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.806,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5609999999999999,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.496,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.6299999999999999,"15min":1.6890000000000001},"min":{"1min":0.97399999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5880000000000001,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.95899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":1.161,"15min":1.2290000000000001},"min":{"1min":0.44,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1899999999999999,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.5329999999999999,"15min":1.6559999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.157,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.333}]},{"osd":2,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.179,"15min":1.234},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2390000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.534,"15min":1.6419999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.385,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.268,"15min":1.3320000000000001},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2629999999999999,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.3340000000000001,"15min":1.3939999999999999},"min":{"1min":0.628,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4490000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.359,"15min":1.3939999999999999},"min":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4390000000000001,"5min":3.669,"15min":3.669},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.80000000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.6359999999999999,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.885,"15min":1.9930000000000001},"min":{"1min":0.83199999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6060000000000001,"5min":10.651,"15min":10.651},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.4470000000000001,"15min":1.484},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.589,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.492,"15min":1.526},"min":{"1min":0.86899999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.972,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.9259999999999999,"15min":2.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5740000000000001,"5min":10.708,"15min":10.708},"last":1.431}]},{"osd":7,"last update":"Fri Feb 2 13:26:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.9410000000000001,"15min":2.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7849999999999999,"5min":10.74,"15min":10.74},"last":1.397},{"interface":"front","average":{"1min":1.252,"5min":1.4610000000000001,"15min":1.496},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.613,"5min":3.863,"15min":3.863},"last":1.2549999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:27:31.051 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:27:31.052 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:27:31.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.303+0000 7fab91a00640 1 Processor -- start 2024-02-02T13:27:31.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.303+0000 7fab91a00640 1 -- start start 2024-02-02T13:27:31.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab8c0704e0 0x7fab8c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fab8c0b2300 0x7fab8c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c0d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8c0b6070 con 0x7fab8c0b2300 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8c071650 con 0x7fab8c0704e0 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c0d0cb0 con 0x7fab8c0708c0 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fab8c0b2300 0x7fab8c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fab8c0b2300 0x7fab8c0d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57366/0 (socket says 172.21.15.91:57366) 2024-02-02T13:27:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 -- 172.21.15.91:0/2807669666 learned_addr learned my addr 172.21.15.91:0/2807669666 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8be00640 1 --2- 172.21.15.91:0/2807669666 >> v2:172.21.15.113:3300/0 conn(0x7fab8c0704e0 0x7fab8c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91000640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c0d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 -- 172.21.15.91:0/2807669666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 msgr2=0x7fab8c0d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c0d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 -- 172.21.15.91:0/2807669666 >> v2:172.21.15.113:3300/0 conn(0x7fab8c0704e0 msgr2=0x7fab8c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 --2- 172.21.15.91:0/2807669666 >> v2:172.21.15.113:3300/0 conn(0x7fab8c0704e0 0x7fab8c0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab8b400640 1 -- 172.21.15.91:0/2807669666 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c0cdfc0 con 0x7fab8c0b2300 2024-02-02T13:27:31.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.305+0000 7fab91000640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c0d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:31.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.306+0000 7fab8b400640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0d1190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fab8c08b640 tx=0x7fab7402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf329a7c29b12ecd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab8aa00640 1 -- 172.21.15.91:0/2807669666 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74031740 con 0x7fab8c0b2300 2024-02-02T13:27:31.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab8aa00640 1 -- 172.21.15.91:0/2807669666 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab740318a0 con 0x7fab8c0b2300 2024-02-02T13:27:31.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab8aa00640 1 -- 172.21.15.91:0/2807669666 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74031c30 con 0x7fab8c0b2300 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 -- 172.21.15.91:0/2807669666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 msgr2=0x7fab8c0d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0d1190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fab8c08b640 tx=0x7fab7402b7e0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 -- 172.21.15.91:0/2807669666 shutdown_connections 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c0d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 --2- 172.21.15.91:0/2807669666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0d1190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 --2- 172.21.15.91:0/2807669666 >> v2:172.21.15.113:3300/0 conn(0x7fab8c0704e0 0x7fab8c0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 -- 172.21.15.91:0/2807669666 >> 172.21.15.91:0/2807669666 conn(0x7fab8c0cfbf0 msgr2=0x7fab8c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 -- 172.21.15.91:0/2807669666 shutdown_connections 2024-02-02T13:27:31.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.307+0000 7fab91a00640 1 -- 172.21.15.91:0/2807669666 wait complete. 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.309+0000 7fab91a00640 1 Processor -- start 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.309+0000 7fab91a00640 1 -- start start 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8c0704e0 0x7fab8c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c1e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c1e7150 con 0x7fab8c0b2300 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c0cad10 con 0x7fab8c0704e0 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c1e6540 con 0x7fab8c0708c0 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57382/0 (socket says 172.21.15.91:57382) 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.310+0000 7fab91000640 1 -- 172.21.15.91:0/3377412047 learned_addr learned my addr 172.21.15.91:0/3377412047 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab8b400640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c1e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 -- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 msgr2=0x7fab8c1e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c1e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab8be00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8c0704e0 0x7fab8c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 -- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8c0704e0 msgr2=0x7fab8c14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8c0704e0 0x7fab8c14b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 -- 172.21.15.91:0/3377412047 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab740094d0 con 0x7fab8c0b2300 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91000640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fab8400cef0 tx=0x7fab8400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8400d840 con 0x7fab8c0b2300 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8400d9c0 con 0x7fab8c0b2300 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8400dd70 con 0x7fab8c0b2300 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.311+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c1e6820 con 0x7fab8c0b2300 2024-02-02T13:27:31.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.312+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c1e6de0 con 0x7fab8c0b2300 2024-02-02T13:27:31.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.313+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c04dff0 con 0x7fab8c0b2300 2024-02-02T13:27:31.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.314+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fab8401e3b0 con 0x7fab8c0b2300 2024-02-02T13:27:31.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.314+0000 7fab83e00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 0x7fab6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.314+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab84089480 con 0x7fab8c0b2300 2024-02-02T13:27:31.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.315+0000 7fab8be00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 0x7fab6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.315+0000 7fab8be00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 0x7fab6406c8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.319+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fab84054a90 con 0x7fab8c0b2300 2024-02-02T13:27:31.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.584+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fab8c0ce270 con 0x7fab8c0b2300 2024-02-02T13:27:31.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.592+0000 7fab83e00640 1 -- 172.21.15.91:0/3377412047 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fab840580b0 con 0x7fab8c0b2300 2024-02-02T13:27:31.594 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 msgr2=0x7fab6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 0x7fab6406c8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).stop 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 msgr2=0x7fab8c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fab8400cef0 tx=0x7fab8400a710 comp rx=0 tx=0).stop 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 shutdown_connections 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab6406a430 0x7fab6406c8f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8c0708c0 0x7fab8c1e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab8c0b2300 0x7fab8c0b2b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 --2- 172.21.15.91:0/3377412047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8c0704e0 0x7fab8c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 >> 172.21.15.91:0/3377412047 conn(0x7fab8c0cfbf0 msgr2=0x7fab8c0b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:31.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 shutdown_connections 2024-02-02T13:27:31.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.595+0000 7fab91a00640 1 -- 172.21.15.91:0/3377412047 wait complete. 2024-02-02T13:27:31.616 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:31.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.819+0000 7ff902000640 1 Processor -- start 2024-02-02T13:27:31.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.819+0000 7ff902000640 1 -- start start 2024-02-02T13:27:31.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.820+0000 7ff902000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc0adf00 0x7ff8fc06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.820+0000 7ff902000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8fc06b990 0x7ff8fc06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.820+0000 7ff902000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc15a250 con 0x7ff8fc0c9170 2024-02-02T13:27:31.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.820+0000 7ff902000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0ae390 con 0x7ff8fc06b990 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.820+0000 7ff902000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15a4b0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8fc06b990 0x7ff8fc06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8fc06b990 0x7ff8fc06be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57390/0 (socket says 172.21.15.91:57390) 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 -- 172.21.15.91:0/2741430386 learned_addr learned my addr 172.21.15.91:0/2741430386 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff8f3c00640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc0adf00 0x7ff8fc06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 -- 172.21.15.91:0/2741430386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc0adf00 msgr2=0x7ff8fc06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc0adf00 0x7ff8fc06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 -- 172.21.15.91:0/2741430386 >> v1:172.21.15.113:6789/0 conn(0x7ff8fc0c9170 legacy=0x7ff8fc159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:31.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.821+0000 7ff901600640 1 -- 172.21.15.91:0/2741430386 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc15a790 con 0x7ff8fc06b990 2024-02-02T13:27:31.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.822+0000 7ff901600640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc06b990 0x7ff8fc06be20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009e50 tx=0x7ff8f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36e5d2e779913e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff8fbe00640 1 -- 172.21.15.91:0/2741430386 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f402ca70 con 0x7ff8fc06b990 2024-02-02T13:27:31.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff8fbe00640 1 -- 172.21.15.91:0/2741430386 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f402cbd0 con 0x7ff8fc06b990 2024-02-02T13:27:31.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff8fbe00640 1 -- 172.21.15.91:0/2741430386 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40343f0 con 0x7ff8fc06b990 2024-02-02T13:27:31.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 -- 172.21.15.91:0/2741430386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc06b990 msgr2=0x7ff8fc06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc06b990 0x7ff8fc06be20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009e50 tx=0x7ff8f402bed0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 -- 172.21.15.91:0/2741430386 shutdown_connections 2024-02-02T13:27:31.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc06b990 0x7ff8fc06be20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 --2- 172.21.15.91:0/2741430386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc0adf00 0x7ff8fc06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 -- 172.21.15.91:0/2741430386 >> 172.21.15.91:0/2741430386 conn(0x7ff8fc0cea90 msgr2=0x7ff8fc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:31.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.823+0000 7ff902000640 1 -- 172.21.15.91:0/2741430386 shutdown_connections 2024-02-02T13:27:31.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.824+0000 7ff902000640 1 -- 172.21.15.91:0/2741430386 wait complete. 2024-02-02T13:27:31.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.826+0000 7ff902000640 1 Processor -- start 2024-02-02T13:27:31.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.826+0000 7ff902000640 1 -- start start 2024-02-02T13:27:31.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 0x7ff8fc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8fc0c9170 0x7ff8fc15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c5e00 con 0x7ff8fc0adf00 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1f6d20 con 0x7ff8fc0c9170 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff902000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c6010 con 0x7ff8fc06b990 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff8f3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff8f3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57398/0 (socket says 172.21.15.91:57398) 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.827+0000 7ff8f3c00640 1 -- 172.21.15.91:0/3768199175 learned_addr learned my addr 172.21.15.91:0/3768199175 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:31.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 -- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 msgr2=0x7ff8fc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff900c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 0x7ff8fc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 0x7ff8fc1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 -- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8fc0c9170 msgr2=0x7ff8fc15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff901600640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8fc0c9170 0x7ff8fc15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8fc0c9170 0x7ff8fc15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 -- 172.21.15.91:0/3768199175 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4009b30 con 0x7ff8fc0adf00 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff900c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 0x7ff8fc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f3c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d800 tx=0x7ff8e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400b6c0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400b820 con 0x7ff8fc0adf00 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400bbb0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.828+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc0c62f0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.829+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc0b2c10 con 0x7ff8fc0adf00 2024-02-02T13:27:31.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.830+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc04dff0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.830+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff8e400bd10 con 0x7ff8fc0adf00 2024-02-02T13:27:31.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.831+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 0x7ff8d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:31.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.832+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e4088ec0 con 0x7ff8fc0adf00 2024-02-02T13:27:31.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.832+0000 7ff900c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 0x7ff8d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:31.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.832+0000 7ff900c00640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 0x7ff8d406c8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00a830 tx=0x7ff8ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:31.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:31.836+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8e4054410 con 0x7ff8fc0adf00 2024-02-02T13:27:32.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.060+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8fc0b4a10 con 0x7ff8fc0adf00 2024-02-02T13:27:32.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.060+0000 7ff8f9600640 1 -- 172.21.15.91:0/3768199175 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7ff8e4057a30 con 0x7ff8fc0adf00 2024-02-02T13:27:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:32.063 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:27:32.063 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 msgr2=0x7ff8d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 0x7ff8d406c8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00a830 tx=0x7ff8ec008040 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 msgr2=0x7ff8fc0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400d800 tx=0x7ff8e400dcc0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 shutdown_connections 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8d406a430 0x7ff8d406c8f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8fc0c9170 0x7ff8fc15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8fc0adf00 0x7ff8fc0b26d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 --2- 172.21.15.91:0/3768199175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8fc06b990 0x7ff8fc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 >> 172.21.15.91:0/3768199175 conn(0x7ff8fc0cea90 msgr2=0x7ff8fc158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 shutdown_connections 2024-02-02T13:27:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.063+0000 7ff902000640 1 -- 172.21.15.91:0/3768199175 wait complete. 2024-02-02T13:27:32.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:27:26.089350+0000', 'modified': '2024-02-02T13:27:29.118027+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5934': {'gid': 5934, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/2506136501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 2506136501}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 2506136501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:27:32.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:27:32.487 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T13:27:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.705+0000 7f9fa9600640 1 Processor -- start 2024-02-02T13:27:32.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.705+0000 7f9fa9600640 1 -- start start 2024-02-02T13:27:32.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa9600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9fa415b0f0 0x7f9fa406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9fa40cb580 0x7f9fa40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa9600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa406d910 con 0x7f9fa415ad10 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa9600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa406d730 con 0x7f9fa415b0f0 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa40cbf50 con 0x7f9fa40cb580 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa2a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9fa415ad10 0x7f9fa406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52644/0 (socket says 172.21.15.91:52644) 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa2a00640 1 -- 172.21.15.91:0/2394672936 learned_addr learned my addr 172.21.15.91:0/2394672936 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.706+0000 7f9fa2000640 1 --2- 172.21.15.91:0/2394672936 >> v2:172.21.15.91:3300/0 conn(0x7f9fa415b0f0 0x7f9fa406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa1600640 1 -- 172.21.15.91:0/2394672936 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301237332 0 0) 0x7f9fa406d910 con 0x7f9fa415ad10 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa1600640 1 -- 172.21.15.91:0/2394672936 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84003520 con 0x7f9fa415ad10 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa2000640 1 -- 172.21.15.91:0/2394672936 >> v2:172.21.15.113:3300/0 conn(0x7f9fa40cb580 msgr2=0x7f9fa40cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa2000640 1 --2- 172.21.15.91:0/2394672936 >> v2:172.21.15.113:3300/0 conn(0x7f9fa40cb580 0x7f9fa40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa2000640 1 -- 172.21.15.91:0/2394672936 >> v1:172.21.15.91:6789/0 conn(0x7f9fa415ad10 legacy=0x7f9fa406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.707+0000 7f9fa2000640 1 -- 172.21.15.91:0/2394672936 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa40cc850 con 0x7f9fa415b0f0 2024-02-02T13:27:32.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.708+0000 7f9fa2000640 1 --2- 172.21.15.91:0/2394672936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415b0f0 0x7f9fa406b7a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80f4d558901e534 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa1600640 1 -- 172.21.15.91:0/2394672936 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c02e070 con 0x7f9fa415b0f0 2024-02-02T13:27:32.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa1600640 1 -- 172.21.15.91:0/2394672936 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0315a0 con 0x7f9fa415b0f0 2024-02-02T13:27:32.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa1600640 1 -- 172.21.15.91:0/2394672936 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c031910 con 0x7f9fa415b0f0 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 -- 172.21.15.91:0/2394672936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415b0f0 msgr2=0x7f9fa406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 --2- 172.21.15.91:0/2394672936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415b0f0 0x7f9fa406b7a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02b840 comp rx=0 tx=0).stop 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 -- 172.21.15.91:0/2394672936 shutdown_connections 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 --2- 172.21.15.91:0/2394672936 >> v2:172.21.15.113:3300/0 conn(0x7f9fa40cb580 0x7f9fa40cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 --2- 172.21.15.91:0/2394672936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415b0f0 0x7f9fa406b7a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.709+0000 7f9fa9600640 1 -- 172.21.15.91:0/2394672936 >> 172.21.15.91:0/2394672936 conn(0x7f9fa40b0700 msgr2=0x7f9fa40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.710+0000 7f9fa9600640 1 -- 172.21.15.91:0/2394672936 shutdown_connections 2024-02-02T13:27:32.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.710+0000 7f9fa9600640 1 -- 172.21.15.91:0/2394672936 wait complete. 2024-02-02T13:27:32.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.711+0000 7f9fa9600640 1 Processor -- start 2024-02-02T13:27:32.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.711+0000 7f9fa9600640 1 -- start start 2024-02-02T13:27:32.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fa40cb580 0x7f9fa4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fa415b0f0 0x7f9fa415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4153310 con 0x7f9fa415ad10 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4153800 con 0x7f9fa415b0f0 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4153520 con 0x7f9fa40cb580 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57418/0 (socket says 172.21.15.91:57418) 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.712+0000 7f9fa2000640 1 -- 172.21.15.91:0/1674031528 learned_addr learned my addr 172.21.15.91:0/1674031528 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 -- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fa40cb580 msgr2=0x7f9fa4152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fa40cb580 0x7f9fa4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 -- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fa415b0f0 msgr2=0x7f9fa415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fa415b0f0 0x7f9fa415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 -- 172.21.15.91:0/1674031528 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009400 con 0x7f9fa415ad10 2024-02-02T13:27:32.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9fa2000640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c02b880 tx=0x7f9f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.713+0000 7f9f9aa00640 1 -- 172.21.15.91:0/1674031528 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003810 con 0x7f9fa415ad10 2024-02-02T13:27:32.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.714+0000 7f9f9aa00640 1 -- 172.21.15.91:0/1674031528 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c040070 con 0x7f9fa415ad10 2024-02-02T13:27:32.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.714+0000 7f9f9aa00640 1 -- 172.21.15.91:0/1674031528 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003bb0 con 0x7f9fa415ad10 2024-02-02T13:27:32.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.714+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa415bb00 con 0x7f9fa415ad10 2024-02-02T13:27:32.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.714+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa415c0c0 con 0x7f9fa415ad10 2024-02-02T13:27:32.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.716+0000 7f9f9aa00640 1 -- 172.21.15.91:0/1674031528 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9f8c003d10 con 0x7f9fa415ad10 2024-02-02T13:27:32.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.716+0000 7f9f9aa00640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.717+0000 7f9f9aa00640 1 -- 172.21.15.91:0/1674031528 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f8c042070 con 0x7f9fa415ad10 2024-02-02T13:27:32.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.717+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.718+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f94001610 tx=0x7f9f94007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.718+0000 7f9fac225740 1 Processor -- start 2024-02-02T13:27:32.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.718+0000 7f9fac225740 1 -- start start 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63baed0 0x563aa63e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63bead0 0x563aa63e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa6396650 con 0x563aa63eaf50 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa632b000 con 0x563aa63bead0 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aa6328790 con 0x563aa63baed0 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57432/0 (socket says 172.21.15.91:57432) 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 -- 172.21.15.91:0/1208894976 learned_addr learned my addr 172.21.15.91:0/1208894976 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2000640 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63baed0 0x563aa63e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 -- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63baed0 msgr2=0x563aa63e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63baed0 0x563aa63e81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 -- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63bead0 msgr2=0x563aa63e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63bead0 0x563aa63e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.719+0000 7f9fa2a00640 1 -- 172.21.15.91:0/1208894976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aa62eb710 con 0x563aa63eaf50 2024-02-02T13:27:32.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f94005bd0 tx=0x7f9f9400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e1d31f4a50e447 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9f93400640 1 -- 172.21.15.91:0/1208894976 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9401b070 con 0x563aa63eaf50 2024-02-02T13:27:32.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9f93400640 1 -- 172.21.15.91:0/1208894976 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94007720 con 0x563aa63eaf50 2024-02-02T13:27:32.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9f93400640 1 -- 172.21.15.91:0/1208894976 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94012870 con 0x563aa63eaf50 2024-02-02T13:27:32.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9fac225740 1 -- 172.21.15.91:0/1208894976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 msgr2=0x563aa63eb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.720+0000 7f9fac225740 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f94005bd0 tx=0x7f9f9400f000 comp rx=0 tx=0).stop 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 -- 172.21.15.91:0/1208894976 shutdown_connections 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63baed0 0x563aa63e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63bead0 0x563aa63e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 --2- 172.21.15.91:0/1208894976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa63eaf50 0x563aa63eb330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 -- 172.21.15.91:0/1208894976 >> 172.21.15.91:0/1208894976 conn(0x563aa63bd1e0 msgr2=0x563aa6405d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 -- 172.21.15.91:0/1208894976 shutdown_connections 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 -- 172.21.15.91:0/1208894976 wait complete. 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.721+0000 7f9fac225740 1 -- start start 2024-02-02T13:27:32.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.722+0000 7f9fac225740 1 Processor -- start 2024-02-02T13:27:32.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.722+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa64053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.722+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fac225740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fac225740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa6324080 con 0x563aa6403e30 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa62c5850 con 0x563aa63eaf50 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aa62c54f0 con 0x563aa63bead0 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fac225740 10 client.0 _ll_register_callbacks cb 0x563aa631d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57444/0 (socket says 172.21.15.91:57444) 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 -- 172.21.15.91:0/3331176003 learned_addr learned my addr 172.21.15.91:0/3331176003 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa64053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa2000640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 -- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 msgr2=0x563aa64053d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa64053d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 -- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 msgr2=0x563aa63e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa3400640 1 -- 172.21.15.91:0/3331176003 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f84003520 con 0x563aa6403e30 2024-02-02T13:27:32.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.723+0000 7f9fa2000640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9fa3400640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00b7f0 tx=0x7f9f9c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9f936fae96c3a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9f8be00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00c940 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f9c00caa0 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aa634e5d0 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f9c00cca0 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9fac225740 10 client.6087 fetch_fsmap learned FSMap version 32 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x563aa6322700 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.724+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563aa634e5d0 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f9c014440 con 0x563aa6403e30 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 10 client.6087 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 10 client.6087 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 20 client.6087 populate_metadata read hostname 'smithi091' 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa63eb3e0 0x563aa6422e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 4 client.6087 mds_command: new command op to 5934 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:27:32.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 --> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aa63168d0 con 0x563aa63eb3e0 2024-02-02T13:27:32.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa63eb3e0 0x563aa6422e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa63eb3e0 0x563aa6422e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca8659534aaa5d4 server_cookie=e29615d84c9392fe in_seq=0 out_seq=0 2024-02-02T13:27:32.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.725+0000 7f9f8be00640 10 client.6087 ms_handle_connect on v2:172.21.15.91:6835/2506136501 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.726+0000 7f9f8be00640 1 -- 172.21.15.91:0/3331176003 <== mds.0 v2:172.21.15.91:6835/2506136501 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563aa63168d0 con 0x563aa63eb3e0 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.726+0000 7f9f8be00640 10 client.6087 handle_command_reply: tid=0 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.726+0000 7f9fac225740 1 client.6087 shutdown 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 msgr2=0x563aa63d7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00b7f0 tx=0x7f9f9c00bcb0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 shutdown_connections 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa63eb3e0 0x563aa6422e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63d7000 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9f8be00640 0 client.6087 ms_handle_reset on v2:172.21.15.91:6835/2506136501 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 --2- 172.21.15.91:0/3331176003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa64053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 >> 172.21.15.91:0/3331176003 conn(0x563aa63baed0 msgr2=0x563aa6409270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 shutdown_connections 2024-02-02T13:27:32.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 1 -- 172.21.15.91:0/3331176003 wait complete. 2024-02-02T13:27:32.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.727+0000 7f9fac225740 20 client.6087 trim_cache size 0 max 16384 2024-02-02T13:27:32.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.743+0000 7f9fac225740 1 Processor -- start 2024-02-02T13:27:32.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.743+0000 7f9fac225740 1 -- start start 2024-02-02T13:27:32.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa63b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa608bf20 con 0x563aa6403e30 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa63e83e0 con 0x563aa63eaf50 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aa63e5fa0 con 0x563aa63bead0 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57458/0 (socket says 172.21.15.91:57458) 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.744+0000 7f9fa3400640 1 -- 172.21.15.91:0/1833350208 learned_addr learned my addr 172.21.15.91:0/1833350208 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa63b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa2000640 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 -- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 msgr2=0x563aa63b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa63b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 -- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 msgr2=0x563aa63d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63d6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 -- 172.21.15.91:0/1833350208 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aa6358280 con 0x563aa6403e30 2024-02-02T13:27:32.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9fa3400640 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c01d040 tx=0x7f9f9c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1e305cd17ae9a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9f93e00640 1 -- 172.21.15.91:0/1833350208 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c014440 con 0x563aa6403e30 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9f93e00640 1 -- 172.21.15.91:0/1833350208 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c014b60 con 0x563aa6403e30 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.745+0000 7f9f93e00640 1 -- 172.21.15.91:0/1833350208 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0145c0 con 0x563aa6403e30 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 -- 172.21.15.91:0/1833350208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 msgr2=0x563aa63e6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c01d040 tx=0x7f9f9c00bf90 comp rx=0 tx=0).stop 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 -- 172.21.15.91:0/1833350208 shutdown_connections 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63e6610 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 --2- 172.21.15.91:0/1833350208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa63b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 -- 172.21.15.91:0/1833350208 >> 172.21.15.91:0/1833350208 conn(0x563aa63baed0 msgr2=0x563aa63ba030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 -- 172.21.15.91:0/1833350208 shutdown_connections 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.746+0000 7f9fac225740 1 -- 172.21.15.91:0/1833350208 wait complete. 2024-02-02T13:27:32.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.747+0000 7f9fac225740 1 -- start start 2024-02-02T13:27:32.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.747+0000 7f9fac225740 1 Processor -- start 2024-02-02T13:27:32.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa6455de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63ba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa6419860 con 0x563aa6403e30 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aa646ddf0 con 0x563aa63eaf50 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aa6402bd0 con 0x563aa63bead0 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fac225740 10 client.0 _ll_register_callbacks cb 0x7f9f9c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57468/0 (socket says 172.21.15.91:57468) 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.748+0000 7f9fa3400640 1 -- 172.21.15.91:0/168661761 learned_addr learned my addr 172.21.15.91:0/168661761 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 -- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 msgr2=0x563aa6455de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa6455de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 -- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 msgr2=0x563aa63ba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 -- 172.21.15.91:0/168661761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563aa6322700 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fa3400640 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000f80 tx=0x7f9f9c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4582fea4206a60f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9f8be00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c009680 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f9c014440 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f94007720 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f9c00c740 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fac225740 10 client.6093 fetch_fsmap learned FSMap version 32 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.749+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x563aa62c4a60 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94007720 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f9c01c040 con 0x563aa6403e30 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 10 client.6093 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 10 client.6093 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 20 client.6093 populate_metadata read hostname 'smithi091' 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa646cc40 0x563aa646d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 4 client.6093 mds_command: new command op to 5934 tid=0[{"prefix": "status"}] 2024-02-02T13:27:32.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 --> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563aa631d100 con 0x563aa646cc40 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.750+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa646cc40 0x563aa646d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fa2a00640 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa646cc40 0x563aa646d020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a5205543482287 server_cookie=44758fb37285b135 in_seq=0 out_seq=0 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9f8be00640 10 client.6093 ms_handle_connect on v2:172.21.15.91:6835/2506136501 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9f8be00640 1 -- 172.21.15.91:0/168661761 <== mds.0 v2:172.21.15.91:6835/2506136501 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563aa631d100 con 0x563aa646cc40 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9f8be00640 10 client.6093 handle_command_reply: tid=0 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 client.6093 shutdown 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 msgr2=0x563aa63ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000f80 tx=0x7f9f9c004bc0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 shutdown_connections 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:6835/2506136501,v1:172.21.15.91:6837/2506136501] conn(0x563aa646cc40 0x563aa646d020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x563aa6403e30 0x563aa63ba570 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9f8be00640 0 client.6093 ms_handle_reset on v2:172.21.15.91:6835/2506136501 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563aa63eaf50 0x563aa63ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 --2- 172.21.15.91:0/168661761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563aa63bead0 0x563aa6455de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 >> 172.21.15.91:0/168661761 conn(0x563aa63baed0 msgr2=0x563aa63bf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 shutdown_connections 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 1 -- 172.21.15.91:0/168661761 wait complete. 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.751+0000 7f9fac225740 20 client.6093 trim_cache size 0 max 16384 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5934, 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:27:32.757 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.6579420970000003, 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 32, 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 68, 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 68, 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 12.299485161 2024-02-02T13:27:32.758 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 msgr2=0x7f9f7c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f94001610 tx=0x7f9f94007930 comp rx=0 tx=0).stop 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 msgr2=0x7f9fa4151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c02b880 tx=0x7f9f8c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 shutdown_connections 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fa415b0f0 0x7f9fa415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9fa415ad10 0x7f9fa4151340 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 --2- 172.21.15.91:0/1674031528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fa40cb580 0x7f9fa4152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 >> 172.21.15.91:0/1674031528 conn(0x7f9fa40b0700 msgr2=0x7f9fa4153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 shutdown_connections 2024-02-02T13:27:32.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.752+0000 7f9fa9600640 1 -- 172.21.15.91:0/1674031528 wait complete. 2024-02-02T13:27:32.772 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:27:32.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.974+0000 7fc0e8200640 1 Processor -- start 2024-02-02T13:27:32.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.974+0000 7fc0e8200640 1 -- start start 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.975+0000 7fc0e8200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0e015f5a0 0x7fc0e00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.975+0000 7fc0e8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 0x7fc0e00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.975+0000 7fc0e8200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e0070f80 con 0x7fc0e0151f90 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.975+0000 7fc0e8200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e015fa30 con 0x7fc0e015f5a0 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.975+0000 7fc0e8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00b10b0 con 0x7fc0e00b0c40 2024-02-02T13:27:32.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0e5800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc0e0151f90 0x7fc0e006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52646/0 (socket says 172.21.15.91:52646) 2024-02-02T13:27:32.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0e5800640 1 -- 172.21.15.91:0/1481996403 learned_addr learned my addr 172.21.15.91:0/1481996403 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:32.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0e6200640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 0x7fc0e00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919427921 0 0) 0x7fc0e0070f80 con 0x7fc0e0151f90 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/1481996403 >> v2:172.21.15.113:3300/0 conn(0x7fc0e015f5a0 0x7fc0e00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.976+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c0003520 con 0x7fc0e0151f90 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 402837470 0 0) 0x7fc0c0003520 con 0x7fc0e0151f90 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 msgr2=0x7fc0e00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 0x7fc0e00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 >> v2:172.21.15.113:3300/0 conn(0x7fc0e015f5a0 msgr2=0x7fc0e00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 --2- 172.21.15.91:0/1481996403 >> v2:172.21.15.113:3300/0 conn(0x7fc0e015f5a0 0x7fc0e00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0152370 con 0x7fc0e0151f90 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc0d0001670 con 0x7fc0e0151f90 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 >> v1:172.21.15.91:6789/0 conn(0x7fc0e0151f90 legacy=0x7fc0e006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 0x7fc0c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015fa30 con 0x7fc0c0003970 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00b10b0 con 0x7fc0c00047c0 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c000b980 con 0x7fc0e00b0c40 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.977+0000 7fc0e5800640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e5800640 1 -- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 msgr2=0x7fc0c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e5800640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e00b0c40 0x7fc0c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e5800640 1 -- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c00047c0 msgr2=0x7fc0c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e5800640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e5800640 1 -- 172.21.15.91:0/1481996403 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0003520 con 0x7fc0c0003970 2024-02-02T13:27:32.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.978+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:32.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0e5800640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0c0003d90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c110 tx=0x7fc0c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08138dca5770e12 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0005ad0 con 0x7fc0c0003970 2024-02-02T13:27:32.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d0005c50 con 0x7fc0c0003970 2024-02-02T13:27:32.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0dfe00640 1 -- 172.21.15.91:0/1481996403 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d002c810 con 0x7fc0c0003970 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0e8200640 1 -- 172.21.15.91:0/1481996403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 msgr2=0x7fc0c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0e8200640 1 --2- 172.21.15.91:0/1481996403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0c0003d90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c110 tx=0x7fc0c00057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.979+0000 7fc0e6200640 1 -- 172.21.15.91:0/1481996403 reap_dead start 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.980+0000 7fc0e8200640 1 -- 172.21.15.91:0/1481996403 shutdown_connections 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.980+0000 7fc0e8200640 1 -- 172.21.15.91:0/1481996403 >> 172.21.15.91:0/1481996403 conn(0x7fc0e00cec80 msgr2=0x7fc0e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.980+0000 7fc0e8200640 1 -- 172.21.15.91:0/1481996403 shutdown_connections 2024-02-02T13:27:32.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.980+0000 7fc0e8200640 1 -- 172.21.15.91:0/1481996403 wait complete. 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.981+0000 7fc0e8200640 1 Processor -- start 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 -- start start 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0e01588e0 0x7fc0e00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e015b510 0x7fc0e015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0157100 con 0x7fc0c0003970 2024-02-02T13:27:32.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0156b40 con 0x7fc0e01588e0 2024-02-02T13:27:32.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.982+0000 7fc0e8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0156e20 con 0x7fc0e015b510 2024-02-02T13:27:32.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57480/0 (socket says 172.21.15.91:57480) 2024-02-02T13:27:32.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 -- 172.21.15.91:0/4769134 learned_addr learned my addr 172.21.15.91:0/4769134 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 -- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e015b510 msgr2=0x7fc0e015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e015b510 0x7fc0e015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 -- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0e01588e0 msgr2=0x7fc0e00cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0e01588e0 0x7fc0e00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0e01588e0 0x7fc0e00cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 -- 172.21.15.91:0/4769134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00c4450 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.983+0000 7fc0e5800640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c230 tx=0x7fc0d0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.984+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d002cc20 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.984+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d002cd80 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.984+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0007270 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.984+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e01f2720 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.984+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e00cd140 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.985+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e0156c80 con 0x7fc0c0003970 2024-02-02T13:27:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.986+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0d0032070 con 0x7fc0c0003970 2024-02-02T13:27:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.987+0000 7fc0dd600640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 0x7fc0b806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.987+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0d003c040 con 0x7fc0c0003970 2024-02-02T13:27:32.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.987+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 0x7fc0b806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:32.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.988+0000 7fc0e4e00640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 0x7fc0b806c920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00c2030 tx=0x7fc0c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:32.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:32.992+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0e0156c80 con 0x7fc0c0003970 2024-02-02T13:27:33.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.220+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc0e0156c80 con 0x7fc0c0003970 2024-02-02T13:27:33.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.221+0000 7fc0dd600640 1 -- 172.21.15.91:0/4769134 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc0e0156c80 con 0x7fc0c0003970 2024-02-02T13:27:33.222 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:27:33.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.223+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 msgr2=0x7fc0b806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:33.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.223+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 0x7fc0b806c920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00c2030 tx=0x7fc0c8008040 comp rx=0 tx=0).stop 2024-02-02T13:27:33.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.223+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 msgr2=0x7fc0e01f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:33.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.223+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c230 tx=0x7fc0d0001930 comp rx=0 tx=0).stop 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 shutdown_connections 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0b806a460 0x7fc0b806c920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0e015b510 0x7fc0e015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0e01588e0 0x7fc0e00cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 --2- 172.21.15.91:0/4769134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0c0003970 0x7fc0e01f21e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 >> 172.21.15.91:0/4769134 conn(0x7fc0e00cec80 msgr2=0x7fc0e00c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 shutdown_connections 2024-02-02T13:27:33.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:33.224+0000 7fc0e8200640 1 -- 172.21.15.91:0/4769134 wait complete. 2024-02-02T13:27:33.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:27:33.244 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:27:33.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.245 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:27:33.295 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42472sec preferred_lft 42472sec 2024-02-02T13:27:33.296 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:27:33.297 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:27:33.298 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:27:33.298 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:33.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.299 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:27:33.388 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:27:33.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.392 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:27:33.442 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:27:33.442 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:27:33.442 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:27:33.443 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:27:33.443 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:27:33.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:27:33.444 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:27:33.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.623 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:27:33.673 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:27:33.673 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:27:33.674 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:27:33.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.675 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:27:33.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:27:33.764 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:27:33.816 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:27:33.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:27:33.817 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:27:33.864 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:27:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:33.864 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:34.049 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:34.049 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:27:34.084 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:34.084 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:27:34.085 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:27:34.085 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:27:34.085 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:27:34.087 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:27:34.087 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:27:34.087 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:27:34.087 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:27:34.087 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:27:34.092 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:27:34.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:34.092 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:27:34.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:34.179 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:27:34.241 INFO:teuthology.orchestra.run.smithi091.stdout:client4829 2024-02-02T13:27:34.241 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:27:34.242 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:34.293 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:27:34.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.546+0000 7fe976e00640 1 Processor -- start 2024-02-02T13:27:34.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.546+0000 7fe976e00640 1 -- start start 2024-02-02T13:27:34.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe976e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe97015a400 0x7fe9700c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe976e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe9700b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe976e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9700c9780 con 0x7fe97015d1c0 2024-02-02T13:27:34.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe976e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9700b10d0 con 0x7fe97015a400 2024-02-02T13:27:34.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe976e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700c99e0 con 0x7fe9700b0700 2024-02-02T13:27:34.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe96fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe97015d1c0 0x7fe97015d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52648/0 (socket says 172.21.15.91:52648) 2024-02-02T13:27:34.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe96fe00640 1 -- 172.21.15.91:0/198215006 learned_addr learned my addr 172.21.15.91:0/198215006 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe974e00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe9700b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.548+0000 7fe967200640 1 --2- 172.21.15.91:0/198215006 >> v2:172.21.15.113:3300/0 conn(0x7fe97015a400 0x7fe9700c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182581839 0 0) 0x7fe9700c9780 con 0x7fe97015d1c0 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe950003520 con 0x7fe97015d1c0 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649566455 0 0) 0x7fe950003520 con 0x7fe97015d1c0 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 msgr2=0x7fe9700b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe9700b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 >> v2:172.21.15.113:3300/0 conn(0x7fe97015a400 msgr2=0x7fe9700c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 --2- 172.21.15.91:0/198215006 >> v2:172.21.15.113:3300/0 conn(0x7fe97015a400 0x7fe9700c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9700c9cc0 con 0x7fe97015d1c0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe9600019f0 con 0x7fe97015d1c0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 >> v1:172.21.15.91:6789/0 conn(0x7fe97015d1c0 legacy=0x7fe97015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe950003970 0x7fe950003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9500047c0 0x7fe950004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe950005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700b10d0 con 0x7fe950003970 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700c99e0 con 0x7fe9500047c0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95000db90 con 0x7fe9700b0700 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.549+0000 7fe96fe00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe950003970 0x7fe950003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe974e00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe950005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe967200640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9500047c0 0x7fe950004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe96fe00640 1 -- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 msgr2=0x7fe950005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe96fe00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9700b0700 0x7fe950005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe96fe00640 1 -- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9500047c0 msgr2=0x7fe950004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:34.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe96fe00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9500047c0 0x7fe950004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.550+0000 7fe96fe00640 1 -- 172.21.15.91:0/198215006 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950003520 con 0x7fe950003970 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.551+0000 7fe96fe00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe950003970 0x7fe950003d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe96000a860 tx=0x7fe9500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3220f19559580df server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960034630 con 0x7fe950003970 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96002f070 con 0x7fe950003970 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe96f400640 1 -- 172.21.15.91:0/198215006 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600349f0 con 0x7fe950003970 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 -- 172.21.15.91:0/198215006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe950003970 msgr2=0x7fe950003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 --2- 172.21.15.91:0/198215006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe950003970 0x7fe950003d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe96000a860 tx=0x7fe9500057f0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe974e00640 1 -- 172.21.15.91:0/198215006 reap_dead start 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 -- 172.21.15.91:0/198215006 shutdown_connections 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 -- 172.21.15.91:0/198215006 >> 172.21.15.91:0/198215006 conn(0x7fe9700cec80 msgr2=0x7fe9700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 -- 172.21.15.91:0/198215006 shutdown_connections 2024-02-02T13:27:34.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.552+0000 7fe976e00640 1 -- 172.21.15.91:0/198215006 wait complete. 2024-02-02T13:27:34.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.554+0000 7fe976e00640 1 Processor -- start 2024-02-02T13:27:34.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.554+0000 7fe976e00640 1 -- start start 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 0x7fe970156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9701f6650 0x7fe9701518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701f3300 con 0x7fe970151e10 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700c2690 con 0x7fe950003970 2024-02-02T13:27:34.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.555+0000 7fe976e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701f75d0 con 0x7fe9701f6650 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57494/0 (socket says 172.21.15.91:57494) 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 -- 172.21.15.91:0/2650601413 learned_addr learned my addr 172.21.15.91:0/2650601413 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 -- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9701f6650 msgr2=0x7fe9701518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9701f6650 0x7fe9701518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 -- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 msgr2=0x7fe970156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe96fe00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 0x7fe970156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 0x7fe970156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 -- 172.21.15.91:0/2650601413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9701f6ad0 con 0x7fe970151e10 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe96fe00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 0x7fe970156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.556+0000 7fe974e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe970157400 tx=0x7fe968011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.557+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968004440 con 0x7fe970151e10 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.557+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968003610 con 0x7fe970151e10 2024-02-02T13:27:34.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.557+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680039a0 con 0x7fe970151e10 2024-02-02T13:27:34.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.557+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9701f6d30 con 0x7fe970151e10 2024-02-02T13:27:34.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.557+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701f72f0 con 0x7fe970151e10 2024-02-02T13:27:34.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.558+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9701f3440 con 0x7fe970151e10 2024-02-02T13:27:34.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.559+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe968003b00 con 0x7fe970151e10 2024-02-02T13:27:34.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.560+0000 7fe96cc00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 0x7fe94806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:34.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.560+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe968088c30 con 0x7fe970151e10 2024-02-02T13:27:34.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.560+0000 7fe96fe00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 0x7fe94806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:34.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.561+0000 7fe96fe00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 0x7fe94806c920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe960009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:34.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.564+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe9701f3440 con 0x7fe970151e10 2024-02-02T13:27:34.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.795+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9701f3440 con 0x7fe970151e10 2024-02-02T13:27:34.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.796+0000 7fe96cc00640 1 -- 172.21.15.91:0/2650601413 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe968019060 con 0x7fe970151e10 2024-02-02T13:27:34.798 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:27:34.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.798+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 msgr2=0x7fe94806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.798+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 0x7fe94806c920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe960009030 comp rx=0 tx=0).stop 2024-02-02T13:27:34.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.798+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 msgr2=0x7fe9701f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.798+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe970157400 tx=0x7fe968011920 comp rx=0 tx=0).stop 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 shutdown_connections 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe94806a460 0x7fe94806c920 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9701f6650 0x7fe9701518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe970151e10 0x7fe9701f3720 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 --2- 172.21.15.91:0/2650601413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe950003970 0x7fe970156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 >> 172.21.15.91:0/2650601413 conn(0x7fe9700cec80 msgr2=0x7fe9701530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 shutdown_connections 2024-02-02T13:27:34.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:34.799+0000 7fe976e00640 1 -- 172.21.15.91:0/2650601413 wait complete. 2024-02-02T13:27:34.818 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:27:34.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:27:34.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:34.819 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:27:34.878 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:27:34.878 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:27:34.878 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:27:34.878 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:27:34.879 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:27:34.880 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:27:34.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:34.881 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:27:34.968 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:27:34.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:34.973 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:27:35.023 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:27:35.023 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:27:35.023 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:27:35.024 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:27:35.024 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:27:35.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:27:35.024 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:27:35.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.209 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:27:35.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:27:35.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:27:35.260 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:27:35.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.260 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:27:35.356 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:27:35.357 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:27:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:27:35.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:27:35.408 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:27:35.458 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:27:35.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:27:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:27:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:27:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:27:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:27:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:27:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:27:35.630 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:27:35.630 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:27:35.630 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:27:35.630 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:27:35.634 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:27:35.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:27:35.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:27:35.713 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:27:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:client5128 2024-02-02T13:27:35.769 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:27:35.769 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:27:35.817 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:27:36.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.033+0000 7f5a7b600640 1 Processor -- start 2024-02-02T13:27:36.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.033+0000 7f5a7b600640 1 -- start start 2024-02-02T13:27:36.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a7b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a7b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a7b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a74070f80 con 0x7f5a7406f4f0 2024-02-02T13:27:36.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a7b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a74152370 con 0x7f5a740b0e10 2024-02-02T13:27:36.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a7b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cb010 con 0x7f5a74151f90 2024-02-02T13:27:36.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a79600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.035+0000 7f5a78c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:46364/0 (socket says 172.21.15.91:46364) 2024-02-02T13:27:36.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a79600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:42626/0 (socket says 172.21.15.91:42626) 2024-02-02T13:27:36.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 -- 172.21.15.91:0/2352632177 learned_addr learned my addr 172.21.15.91:0/2352632177 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:36.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 -- 172.21.15.91:0/2352632177 >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 msgr2=0x7f5a740b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:27:36.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 --2- 172.21.15.91:0/2352632177 >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 -- 172.21.15.91:0/2352632177 >> v1:172.21.15.113:6789/0 conn(0x7f5a7406f4f0 legacy=0x7f5a740b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a78c00640 1 -- 172.21.15.91:0/2352632177 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740cb2f0 con 0x7f5a74151f90 2024-02-02T13:27:36.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.036+0000 7f5a79600640 1 --2- 172.21.15.91:0/2352632177 >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:27:36.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.038+0000 7f5a78c00640 1 --2- 172.21.15.91:0/2352632177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e50 tx=0x7f5a6402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479653441e112daf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:36.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.038+0000 7f5a73e00640 1 -- 172.21.15.91:0/2352632177 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6402cb00 con 0x7f5a74151f90 2024-02-02T13:27:36.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.038+0000 7f5a73e00640 1 -- 172.21.15.91:0/2352632177 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6402cc60 con 0x7f5a74151f90 2024-02-02T13:27:36.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.038+0000 7f5a73e00640 1 -- 172.21.15.91:0/2352632177 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640343f0 con 0x7f5a74151f90 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 -- 172.21.15.91:0/2352632177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 msgr2=0x7f5a7415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 --2- 172.21.15.91:0/2352632177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e50 tx=0x7f5a6402bf60 comp rx=0 tx=0).stop 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 -- 172.21.15.91:0/2352632177 shutdown_connections 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 --2- 172.21.15.91:0/2352632177 >> v2:172.21.15.113:3300/0 conn(0x7f5a740b0e10 0x7f5a740b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 --2- 172.21.15.91:0/2352632177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a7415f5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 -- 172.21.15.91:0/2352632177 >> 172.21.15.91:0/2352632177 conn(0x7f5a740cec80 msgr2=0x7f5a740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 -- 172.21.15.91:0/2352632177 shutdown_connections 2024-02-02T13:27:36.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.039+0000 7f5a7b600640 1 -- 172.21.15.91:0/2352632177 wait complete. 2024-02-02T13:27:36.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.041+0000 7f5a7b600640 1 Processor -- start 2024-02-02T13:27:36.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.041+0000 7f5a7b600640 1 -- start start 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7406f4f0 0x7f5a74154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a74150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740ac8d0 con 0x7f5a740b0e10 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740ccd80 con 0x7f5a7406f4f0 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a7b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cd060 con 0x7f5a74151f90 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a6bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a6bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57496/0 (socket says 172.21.15.91:57496) 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a6bc00640 1 -- 172.21.15.91:0/3381976454 learned_addr learned my addr 172.21.15.91:0/3381976454 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a79600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a74150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.042+0000 7f5a78c00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7406f4f0 0x7f5a74154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 -- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 msgr2=0x7f5a74150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a74150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 -- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7406f4f0 msgr2=0x7f5a74154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7406f4f0 0x7f5a74154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 -- 172.21.15.91:0/3381976454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c009400 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a79600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a74150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a6bc00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c400 tx=0x7f5a5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0052d0 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c005430 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a64009b30 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.043+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003830 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.044+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a740acb20 con 0x7f5a740b0e10 2024-02-02T13:27:36.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.045+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a74066cf0 con 0x7f5a740b0e10 2024-02-02T13:27:36.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.046+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5a5c003990 con 0x7f5a740b0e10 2024-02-02T13:27:36.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.046+0000 7f5a71600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 0x7f5a4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:27:36.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.047+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a5c087eb0 con 0x7f5a740b0e10 2024-02-02T13:27:36.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.047+0000 7f5a78c00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 0x7f5a4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:27:36.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.047+0000 7f5a78c00640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 0x7f5a4c06c8f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a64035000 tx=0x7f5a64030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:27:36.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.051+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5a5c0534a0 con 0x7f5a740b0e10 2024-02-02T13:27:36.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.277+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a7406c7b0 con 0x7f5a740b0e10 2024-02-02T13:27:36.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.278+0000 7f5a71600640 1 -- 172.21.15.91:0/3381976454 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f5a5c056ac0 con 0x7f5a740b0e10 2024-02-02T13:27:36.280 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:27:36.281 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:27:36.281 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:27:36.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 msgr2=0x7f5a4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 0x7f5a4c06c8f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a64035000 tx=0x7f5a64030000 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 msgr2=0x7f5a740cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c400 tx=0x7f5a5c00c8c0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 shutdown_connections 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4c06a430 0x7f5a4c06c8f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a74151f90 0x7f5a74150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a740b0e10 0x7f5a740cd930 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 --2- 172.21.15.91:0/3381976454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7406f4f0 0x7f5a74154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 >> 172.21.15.91:0/3381976454 conn(0x7f5a740cec80 msgr2=0x7f5a740c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:27:36.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 shutdown_connections 2024-02-02T13:27:36.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:27:36.281+0000 7f5a7b600640 1 -- 172.21.15.91:0/3381976454 wait complete. 2024-02-02T13:27:36.303 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-02-02T13:27:36.528 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5934,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.418179881,"mdsmap_epoch":32,"osdmap_epoch":68,"osdmap_epoch_barrier":68,"uptime":16.059735734} 2024-02-02T13:27:36.528 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5934, "mdsmap_epoch": 32, "osdmap_epoch": 68, "osdmap_epoch_barrier": 68, "rank_uptime": 10.418179881, "state": "up:active", "uptime": 16.059735734, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:27:36.529 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:27:36.752 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:27:36.752 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:27:36.753 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-02-02T13:27:36.968 INFO:teuthology.orchestra.run.smithi091.stdout:{"mon_tick_interval":"5"} 2024-02-02T13:27:36.969 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-02-02T13:27:36.969 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2024-02-02T13:27:36.969 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:27:36.969 DEBUG:teuthology.orchestra.run.smithi091:> import time 2024-02-02T13:27:36.969 DEBUG:teuthology.orchestra.run.smithi091:> import datetime 2024-02-02T13:27:36.969 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> # Write some buffered data through before going full, all should be well 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> print("writing some data through which we expect to succeed") 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> bytes = 0 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> # buffered IO, the error should always only appear from fsync and not 2024-02-02T13:27:36.970 DEBUG:teuthology.orchestra.run.smithi091:> # from write 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> full = False 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(531 * 0.9)): 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> print("done writing {0} bytes".format(bytes)) 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> # OK, now we should sneak in under the full condition 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> # due to the time it takes the OSDs to report to the 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> # mons, and get a successful fsync on our full-making data 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> # buffered write, add more dirty data to the buffer 2024-02-02T13:27:36.971 DEBUG:teuthology.orchestra.run.smithi091:> print("starting buffered write") 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(531 * 0.2)): 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(15.0) 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> except OSError: 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> pass; 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> # Wait long enough for a background flush that should fail 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(30) 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> if False: 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> # ...and check that the failed background flush is reflected in fclose 2024-02-02T13:27:36.972 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> except OSError: 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> print("close() returned an error as expected") 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("close() failed to raise error") 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> # The kernel cephfs client does not raise errors on fclose 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-02-02T13:27:36.973 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:27:40.682 INFO:teuthology.orchestra.run.smithi091.stdout:writing some data through which we expect to succeed 2024-02-02T13:27:40.682 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-02-02T13:27:40.682 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-02-02T13:27:40.682 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-02-02T13:27:40.682 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-02-02T13:27:40.683 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-02-02T13:27:40.684 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-02-02T13:27:40.685 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-02-02T13:27:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-02-02T13:27:40.687 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-02-02T13:27:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-02-02T13:27:40.689 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-02-02T13:27:40.690 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-02-02T13:27:40.691 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-02-02T13:27:40.692 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-02-02T13:27:40.693 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-02-02T13:27:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-02-02T13:27:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-02-02T13:27:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-02-02T13:27:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-02-02T13:27:40.858 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-02-02T13:27:40.859 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-02-02T13:27:40.860 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-02-02T13:27:40.861 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-02-02T13:27:40.862 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-02-02T13:27:40.863 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-02-02T13:27:40.864 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-02-02T13:27:40.865 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-02-02T13:27:40.866 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-02-02T13:27:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-02-02T13:27:40.868 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-02-02T13:27:40.869 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-02-02T13:27:40.870 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-02-02T13:27:41.028 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-02-02T13:27:41.028 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-02-02T13:27:41.029 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-02-02T13:27:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-02-02T13:27:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-02-02T13:27:41.032 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-02-02T13:27:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-02-02T13:27:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-02-02T13:27:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-02-02T13:27:41.036 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-02-02T13:27:41.037 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-02-02T13:27:41.038 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-02-02T13:27:43.337 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:27:43.335+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-02-02T13:27:49.308 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:27:49.307+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-02-02T13:27:59.558 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:27:59.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:28:29.699 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:28:29.707 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:28:42.227 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-02-02T13:28:42.228 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:done writing 500695040 bytes 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:successfully fsync'ed prior to getting full state reported 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:starting buffered write 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:28:42.229 INFO:teuthology.orchestra.run.smithi091.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-02-02T13:28:42.408 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:28:42.408 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:28:42.519 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:28:42.636 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:28:42.636 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:28:42.636 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:42.694 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:28:42.694 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:42.747 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:28:42.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.955+0000 7f8a9b600640 1 Processor -- start 2024-02-02T13:28:42.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.955+0000 7f8a9b600640 1 -- start start 2024-02-02T13:28:42.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 0x7f8a9415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a9b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a9406f4f0 0x7f8a940b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a9b600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a94070f80 con 0x7f8a940b0c40 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a9b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a940b10d0 con 0x7f8a9406f4f0 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a9b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94152370 con 0x7f8a94151f90 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a99600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8a940b0c40 0x7f8a940b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57006/0 (socket says 172.21.15.91:57006) 2024-02-02T13:28:42.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.957+0000 7f8a99600640 1 -- 172.21.15.91:0/3114941746 learned_addr learned my addr 172.21.15.91:0/3114941746 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/3114941746 >> v2:172.21.15.113:3300/0 conn(0x7f8a9406f4f0 0x7f8a940b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a98c00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 0x7f8a9415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171354102 0 0) 0x7f8a94070f80 con 0x7f8a940b0c40 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a74003520 con 0x7f8a940b0c40 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221902068 0 0) 0x7f8a74003520 con 0x7f8a940b0c40 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 msgr2=0x7f8a9415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 0x7f8a9415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 >> v2:172.21.15.113:3300/0 conn(0x7f8a9406f4f0 msgr2=0x7f8a940b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 --2- 172.21.15.91:0/3114941746 >> v2:172.21.15.113:3300/0 conn(0x7f8a9406f4f0 0x7f8a940b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a940cb010 con 0x7f8a940b0c40 2024-02-02T13:28:42.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8a8c0019f0 con 0x7f8a940b0c40 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 >> v1:172.21.15.91:6789/0 conn(0x7f8a940b0c40 legacy=0x7f8a940b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74003970 0x7f8a74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a740047c0 0x7f8a74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 0x7f8a74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940b10d0 con 0x7f8a740047c0 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94152370 con 0x7f8a74003970 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7400ba30 con 0x7f8a94151f90 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.958+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a740047c0 0x7f8a74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a98c00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74003970 0x7f8a74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a8bc00640 1 -- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 msgr2=0x7f8a74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a94151f90 0x7f8a74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a8bc00640 1 -- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74003970 msgr2=0x7f8a74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74003970 0x7f8a74003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.959+0000 7f8a8bc00640 1 -- 172.21.15.91:0/3114941746 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74003520 con 0x7f8a740047c0 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.960+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a740047c0 0x7f8a74004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00ba00 tx=0x7f8a740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d777559ad9507d4f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.960+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c032630 con 0x7f8a740047c0 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.960+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c032790 con 0x7f8a740047c0 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.960+0000 7f8a93e00640 1 -- 172.21.15.91:0/3114941746 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c032b20 con 0x7f8a740047c0 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 -- 172.21.15.91:0/3114941746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a740047c0 msgr2=0x7f8a74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 --2- 172.21.15.91:0/3114941746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a740047c0 0x7f8a74004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00ba00 tx=0x7f8a740057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a99600640 1 -- 172.21.15.91:0/3114941746 reap_dead start 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 -- 172.21.15.91:0/3114941746 shutdown_connections 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 -- 172.21.15.91:0/3114941746 >> 172.21.15.91:0/3114941746 conn(0x7f8a940cec80 msgr2=0x7f8a940cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 -- 172.21.15.91:0/3114941746 shutdown_connections 2024-02-02T13:28:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.961+0000 7f8a9b600640 1 -- 172.21.15.91:0/3114941746 wait complete. 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.963+0000 7f8a9b600640 1 Processor -- start 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.963+0000 7f8a9b600640 1 -- start start 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740047c0 0x7f8a9414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a940c83b0 0x7f8a940c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415b8f0 con 0x7f8a9414b590 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415bbd0 con 0x7f8a740047c0 2024-02-02T13:28:42.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a9b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415beb0 con 0x7f8a940c83b0 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57842/0 (socket says 172.21.15.91:57842) 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 -- 172.21.15.91:0/630137875 learned_addr learned my addr 172.21.15.91:0/630137875 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 -- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a940c83b0 msgr2=0x7f8a940c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a940c83b0 0x7f8a940c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 -- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740047c0 msgr2=0x7f8a9414b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a98c00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740047c0 0x7f8a9414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.964+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740047c0 0x7f8a9414b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a8bc00640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9415c190 con 0x7f8a9414b590 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a8bc00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c02beb0 tx=0x7f8a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c03ca40 con 0x7f8a9414b590 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c03cbc0 con 0x7f8a9414b590 2024-02-02T13:28:42.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c030400 con 0x7f8a9414b590 2024-02-02T13:28:42.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9415c470 con 0x7f8a9414b590 2024-02-02T13:28:42.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.965+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a940c2c60 con 0x7f8a9414b590 2024-02-02T13:28:42.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.966+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9415c600 con 0x7f8a9414b590 2024-02-02T13:28:42.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.967+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8a7c030560 con 0x7f8a9414b590 2024-02-02T13:28:42.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.968+0000 7f8a91600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 0x7f8a6c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:42.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.968+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a7c0aad70 con 0x7f8a9414b590 2024-02-02T13:28:42.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.968+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f8a6c071f60 con 0x7f8a9414b590 2024-02-02T13:28:42.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.968+0000 7f8a98c00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 0x7f8a6c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:42.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.969+0000 7f8a98c00640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 0x7f8a6c06c8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006fd0 tx=0x7f8a84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:42.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:42.973+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8a7c0763f0 con 0x7f8a9414b590 2024-02-02T13:28:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.163+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a940ae6e0 con 0x7f8a9414b590 2024-02-02T13:28:43.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.166+0000 7f8a91600640 1 -- 172.21.15.91:0/630137875 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16155 (secure 0 0 0) 0x7f8a7c036030 con 0x7f8a9414b590 2024-02-02T13:28:43.168 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:43.168 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":71,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:45.358247+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:28:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 msgr2=0x7f8a6c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 0x7f8a6c06c8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006fd0 tx=0x7f8a84008040 comp rx=0 tx=0).stop 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 msgr2=0x7f8a940c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c02beb0 tx=0x7f8a7c004730 comp rx=0 tx=0).stop 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 shutdown_connections 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a6c06a430 0x7f8a6c06c8f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a940c83b0 0x7f8a940c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a9414b590 0x7f8a940c7e70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 --2- 172.21.15.91:0/630137875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a740047c0 0x7f8a9414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 >> 172.21.15.91:0/630137875 conn(0x7f8a940cec80 msgr2=0x7f8a9406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.168+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 shutdown_connections 2024-02-02T13:28:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.169+0000 7f8a9b600640 1 -- 172.21.15.91:0/630137875 wait complete. 2024-02-02T13:28:43.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:27:23.838598+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:27:25.123347+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:28:43.192 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:28:43.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.407+0000 7f2c0d600640 1 Processor -- start 2024-02-02T13:28:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.407+0000 7f2c0d600640 1 -- start start 2024-02-02T13:28:43.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.408+0000 7f2c0d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2c080acfe0 0x7f2c0806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.408+0000 7f2c0d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c080b0700 0x7f2c080b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.408+0000 7f2c0d600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0815d870 con 0x7f2c0815d490 2024-02-02T13:28:43.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.408+0000 7f2c0d600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0806bce0 con 0x7f2c080acfe0 2024-02-02T13:28:43.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.408+0000 7f2c0d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c08161220 con 0x7f2c080b0700 2024-02-02T13:28:43.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2c080acfe0 0x7f2c0806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2c080acfe0 0x7f2c0806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57844/0 (socket says 172.21.15.91:57844) 2024-02-02T13:28:43.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 -- 172.21.15.91:0/147852886 learned_addr learned my addr 172.21.15.91:0/147852886 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2c07400640 1 --2- 172.21.15.91:0/147852886 >> v2:172.21.15.113:3300/0 conn(0x7f2c080b0700 0x7f2c080b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 -- 172.21.15.91:0/147852886 >> v2:172.21.15.113:3300/0 conn(0x7f2c080b0700 msgr2=0x7f2c080b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 --2- 172.21.15.91:0/147852886 >> v2:172.21.15.113:3300/0 conn(0x7f2c080b0700 0x7f2c080b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 -- 172.21.15.91:0/147852886 >> v1:172.21.15.113:6789/0 conn(0x7f2c0815d490 legacy=0x7f2c080ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.409+0000 7f2bffe00640 1 -- 172.21.15.91:0/147852886 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080b1050 con 0x7f2c080acfe0 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.410+0000 7f2bffe00640 1 --2- 172.21.15.91:0/147852886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080acfe0 0x7f2c0806c170 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2c080b1870 tx=0x7f2bf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd68e474ad71acc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c06000640 1 -- 172.21.15.91:0/147852886 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031870 con 0x7f2c080acfe0 2024-02-02T13:28:43.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c06000640 1 -- 172.21.15.91:0/147852886 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00319d0 con 0x7f2c080acfe0 2024-02-02T13:28:43.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c06000640 1 -- 172.21.15.91:0/147852886 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031d60 con 0x7f2c080acfe0 2024-02-02T13:28:43.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 -- 172.21.15.91:0/147852886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080acfe0 msgr2=0x7f2c0806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 --2- 172.21.15.91:0/147852886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080acfe0 0x7f2c0806c170 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2c080b1870 tx=0x7f2bf002b910 comp rx=0 tx=0).stop 2024-02-02T13:28:43.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 -- 172.21.15.91:0/147852886 shutdown_connections 2024-02-02T13:28:43.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 --2- 172.21.15.91:0/147852886 >> v2:172.21.15.113:3300/0 conn(0x7f2c080b0700 0x7f2c080b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 --2- 172.21.15.91:0/147852886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080acfe0 0x7f2c0806c170 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 -- 172.21.15.91:0/147852886 >> 172.21.15.91:0/147852886 conn(0x7f2c080cec80 msgr2=0x7f2c080cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:43.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 -- 172.21.15.91:0/147852886 shutdown_connections 2024-02-02T13:28:43.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.411+0000 7f2c0d600640 1 -- 172.21.15.91:0/147852886 wait complete. 2024-02-02T13:28:43.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.413+0000 7f2c0d600640 1 Processor -- start 2024-02-02T13:28:43.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.413+0000 7f2c0d600640 1 -- start start 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 0x7f2c0806f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 0x7f2c08070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080c8bb0 con 0x7f2c080b0700 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080c85f0 con 0x7f2c080acfe0 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.414+0000 7f2c0d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080c88d0 con 0x7f2c0815d490 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57860/0 (socket says 172.21.15.91:57860) 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 -- 172.21.15.91:0/2211615217 learned_addr learned my addr 172.21.15.91:0/2211615217 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2c07400640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 0x7f2c08070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2c06a00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 0x7f2c0806f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 -- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 msgr2=0x7f2c08070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 0x7f2c08070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 -- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 msgr2=0x7f2c0806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 0x7f2c0806f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf00094d0 con 0x7f2c080b0700 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2c07400640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 0x7f2c08070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2c06a00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 0x7f2c0806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:43.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.415+0000 7f2bffe00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0000c00 tx=0x7f2bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.416+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031840 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.416+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00443f0 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.416+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00447c0 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.416+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c08070010 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.416+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0806fca0 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.417+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c080c8730 con 0x7f2c080b0700 2024-02-02T13:28:43.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.418+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2bf004d450 con 0x7f2c080b0700 2024-02-02T13:28:43.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.419+0000 7f2bfea00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 0x7f2be006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.419+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf002d040 con 0x7f2c080b0700 2024-02-02T13:28:43.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.419+0000 7f2c06a00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 0x7f2be006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.419+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f2be0071f60 con 0x7f2c080b0700 2024-02-02T13:28:43.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.420+0000 7f2c06a00640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 0x7f2be006c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007fd0 tx=0x7f2bf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.423+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2c080c8730 con 0x7f2c080b0700 2024-02-02T13:28:43.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.614+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c080c8730 con 0x7f2c080b0700 2024-02-02T13:28:43.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.615+0000 7f2bfea00640 1 -- 172.21.15.91:0/2211615217 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c080c8730 con 0x7f2c080b0700 2024-02-02T13:28:43.617 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:28:43.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 msgr2=0x7f2be006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 0x7f2be006c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007fd0 tx=0x7f2bf8009040 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 msgr2=0x7f2c08140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0000c00 tx=0x7f2bf00046c0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 shutdown_connections 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2be006a430 0x7f2be006c8f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c0815d490 0x7f2c08070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c080b0700 0x7f2c08140c50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 --2- 172.21.15.91:0/2211615217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c080acfe0 0x7f2c0806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 >> 172.21.15.91:0/2211615217 conn(0x7f2c080cec80 msgr2=0x7f2c080c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 shutdown_connections 2024-02-02T13:28:43.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.618+0000 7f2c0d600640 1 -- 172.21.15.91:0/2211615217 wait complete. 2024-02-02T13:28:43.638 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:28:43.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.850+0000 7fe850a00640 1 Processor -- start 2024-02-02T13:28:43.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.851+0000 7fe850a00640 1 -- start start 2024-02-02T13:28:43.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.852+0000 7fe850a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe848151f90 0x7fe84815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.852+0000 7fe850a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe84806f4f0 0x7fe8480b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.852+0000 7fe850a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe848070f80 con 0x7fe8480b0c40 2024-02-02T13:28:43.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.852+0000 7fe850a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8480b10d0 con 0x7fe848151f90 2024-02-02T13:28:43.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.852+0000 7fe850a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe848152370 con 0x7fe84806f4f0 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe8480b0c40 0x7fe8480b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57012/0 (socket says 172.21.15.91:57012) 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84ea00640 1 -- 172.21.15.91:0/1544131163 learned_addr learned my addr 172.21.15.91:0/1544131163 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe84806f4f0 0x7fe8480b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84e000640 1 --2- 172.21.15.91:0/1544131163 >> v2:172.21.15.113:3300/0 conn(0x7fe848151f90 0x7fe84815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889475400 0 0) 0x7fe848070f80 con 0x7fe8480b0c40 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe828003520 con 0x7fe8480b0c40 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699162778 0 0) 0x7fe828003520 con 0x7fe8480b0c40 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe84806f4f0 msgr2=0x7fe8480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.853+0000 7fe84cc00640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe84806f4f0 0x7fe8480b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 >> v2:172.21.15.113:3300/0 conn(0x7fe848151f90 msgr2=0x7fe84815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 --2- 172.21.15.91:0/1544131163 >> v2:172.21.15.113:3300/0 conn(0x7fe848151f90 0x7fe84815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480cb010 con 0x7fe8480b0c40 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe8380019f0 con 0x7fe8480b0c40 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe84806f4f0 0x7fe8480b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 >> v1:172.21.15.91:6789/0 conn(0x7fe8480b0c40 legacy=0x7fe8480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828003970 0x7fe828003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe828004360 0x7fe828004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe828004eb0 0x7fe82800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480b10d0 con 0x7fe828004360 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe848152370 con 0x7fe828004eb0 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82800bec0 con 0x7fe828003970 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe828004360 0x7fe828004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84e000640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828003970 0x7fe828003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 -- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828003970 msgr2=0x7fe828003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828003970 0x7fe828003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 -- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe828004eb0 msgr2=0x7fe82800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:43.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe828004eb0 0x7fe82800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84d600640 1 -- 172.21.15.91:0/1544131163 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828003520 con 0x7fe828004360 2024-02-02T13:28:43.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.854+0000 7fe84e000640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828003970 0x7fe828003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.855+0000 7fe84d600640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe828004360 0x7fe828004840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe830009b20 tx=0x7fe8280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f689692d96328b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830004030 con 0x7fe828004360 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830004190 con 0x7fe828004360 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe84cc00640 1 -- 172.21.15.91:0/1544131163 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830004500 con 0x7fe828004360 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 -- 172.21.15.91:0/1544131163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe828004360 msgr2=0x7fe828004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 --2- 172.21.15.91:0/1544131163 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe828004360 0x7fe828004840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe830009b20 tx=0x7fe8280057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe84ea00640 1 -- 172.21.15.91:0/1544131163 reap_dead start 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 -- 172.21.15.91:0/1544131163 shutdown_connections 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 -- 172.21.15.91:0/1544131163 >> 172.21.15.91:0/1544131163 conn(0x7fe8480cec80 msgr2=0x7fe8480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 -- 172.21.15.91:0/1544131163 shutdown_connections 2024-02-02T13:28:43.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.856+0000 7fe850a00640 1 -- 172.21.15.91:0/1544131163 wait complete. 2024-02-02T13:28:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.858+0000 7fe850a00640 1 Processor -- start 2024-02-02T13:28:43.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.858+0000 7fe850a00640 1 -- start start 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828004360 0x7fe8481585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe84814c510 0x7fe8480af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480b00f0 con 0x7fe8480afc70 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480c2a90 con 0x7fe84814c510 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.859+0000 7fe850a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480c8440 con 0x7fe828004360 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57882/0 (socket says 172.21.15.91:57882) 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 -- 172.21.15.91:0/3523524849 learned_addr learned my addr 172.21.15.91:0/3523524849 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84e000640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828004360 0x7fe8481585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84d600640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe84814c510 0x7fe8480af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 -- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828004360 msgr2=0x7fe8481585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828004360 0x7fe8481585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 -- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe84814c510 msgr2=0x7fe8480af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe84814c510 0x7fe8480af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480c8720 con 0x7fe8480afc70 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe84ea00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe8480b1870 tx=0x7fe83800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.860+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800ce50 con 0x7fe8480afc70 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.861+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838007110 con 0x7fe8480afc70 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.861+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380074c0 con 0x7fe8480afc70 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.861+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8480c8a00 con 0x7fe8480afc70 2024-02-02T13:28:43.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.861+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8480c8060 con 0x7fe8480afc70 2024-02-02T13:28:43.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.862+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe848066cf0 con 0x7fe8480afc70 2024-02-02T13:28:43.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.863+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe838007620 con 0x7fe8480afc70 2024-02-02T13:28:43.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.863+0000 7fe846000640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 0x7fe82006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:43.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.864+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8380884a0 con 0x7fe8480afc70 2024-02-02T13:28:43.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.864+0000 7fe84e000640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 0x7fe82006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:43.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.864+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fe820071f60 con 0x7fe8480afc70 2024-02-02T13:28:43.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.864+0000 7fe84e000640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 0x7fe82006c8f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe840009a40 tx=0x7fe840008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:43.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:43.868+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe8380539f0 con 0x7fe8480afc70 2024-02-02T13:28:44.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.055+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe84806c7b0 con 0x7fe8480afc70 2024-02-02T13:28:44.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.056+0000 7fe846000640 1 -- 172.21.15.91:0/3523524849 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe83801a020 con 0x7fe8480afc70 2024-02-02T13:28:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:28:44.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.058+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 msgr2=0x7fe82006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.058+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 0x7fe82006c8f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe840009a40 tx=0x7fe840008040 comp rx=0 tx=0).stop 2024-02-02T13:28:44.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.058+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 msgr2=0x7fe8480c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.058+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe8480b1870 tx=0x7fe83800cc00 comp rx=0 tx=0).stop 2024-02-02T13:28:44.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 shutdown_connections 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe82006a430 0x7fe82006c8f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8480afc70 0x7fe8480c5cc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe84814c510 0x7fe8480af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 --2- 172.21.15.91:0/3523524849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe828004360 0x7fe8481585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 >> 172.21.15.91:0/3523524849 conn(0x7fe8480cec80 msgr2=0x7fe84806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 shutdown_connections 2024-02-02T13:28:44.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.059+0000 7fe850a00640 1 -- 172.21.15.91:0/3523524849 wait complete. 2024-02-02T13:28:44.080 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:28:44.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.288+0000 7fb52a400640 1 Processor -- start 2024-02-02T13:28:44.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.289+0000 7fb52a400640 1 -- start start 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb52a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb52415d330 0x7fb5240b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb52a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb5240d3850 0x7fb5240d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb52a400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb524161090 con 0x7fb5240b2470 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb52a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb524161cd0 con 0x7fb5240d3850 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb52a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb524161330 con 0x7fb52415d330 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb522a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb5240b2470 0x7fb5240d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57028/0 (socket says 172.21.15.91:57028) 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb522a00640 1 -- 172.21.15.91:0/2304728890 learned_addr learned my addr 172.21.15.91:0/2304728890 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:44.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.290+0000 7fb523e00640 1 --2- 172.21.15.91:0/2304728890 >> v2:172.21.15.91:3300/0 conn(0x7fb5240d3850 0x7fb5240d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb523400640 1 --2- 172.21.15.91:0/2304728890 >> v2:172.21.15.113:3300/0 conn(0x7fb52415d330 0x7fb5240b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb522000640 1 -- 172.21.15.91:0/2304728890 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759766019 0 0) 0x7fb524161090 con 0x7fb5240b2470 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb522000640 1 -- 172.21.15.91:0/2304728890 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003520 con 0x7fb5240b2470 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb523e00640 1 -- 172.21.15.91:0/2304728890 >> v2:172.21.15.113:3300/0 conn(0x7fb52415d330 msgr2=0x7fb5240b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb523e00640 1 --2- 172.21.15.91:0/2304728890 >> v2:172.21.15.113:3300/0 conn(0x7fb52415d330 0x7fb5240b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb523e00640 1 -- 172.21.15.91:0/2304728890 >> v1:172.21.15.91:6789/0 conn(0x7fb5240b2470 legacy=0x7fb5240d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.291+0000 7fb523e00640 1 -- 172.21.15.91:0/2304728890 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524161610 con 0x7fb5240d3850 2024-02-02T13:28:44.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.292+0000 7fb523e00640 1 --2- 172.21.15.91:0/2304728890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb5240d1610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb5240d3d00 tx=0x7fb51c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6832884cfa5c8062 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.292+0000 7fb522000640 1 -- 172.21.15.91:0/2304728890 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c02cb60 con 0x7fb5240d3850 2024-02-02T13:28:44.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.292+0000 7fb522000640 1 -- 172.21.15.91:0/2304728890 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c02ccc0 con 0x7fb5240d3850 2024-02-02T13:28:44.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.292+0000 7fb522000640 1 -- 172.21.15.91:0/2304728890 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c034660 con 0x7fb5240d3850 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 -- 172.21.15.91:0/2304728890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 msgr2=0x7fb5240d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 --2- 172.21.15.91:0/2304728890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb5240d1610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb5240d3d00 tx=0x7fb51c02bdb0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 -- 172.21.15.91:0/2304728890 shutdown_connections 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 --2- 172.21.15.91:0/2304728890 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb5240d1610 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 --2- 172.21.15.91:0/2304728890 >> v2:172.21.15.113:3300/0 conn(0x7fb52415d330 0x7fb5240b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 -- 172.21.15.91:0/2304728890 >> 172.21.15.91:0/2304728890 conn(0x7fb524070660 msgr2=0x7fb524070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 -- 172.21.15.91:0/2304728890 shutdown_connections 2024-02-02T13:28:44.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.293+0000 7fb52a400640 1 -- 172.21.15.91:0/2304728890 wait complete. 2024-02-02T13:28:44.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.295+0000 7fb52a400640 1 Processor -- start 2024-02-02T13:28:44.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.295+0000 7fb52a400640 1 -- start start 2024-02-02T13:28:44.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5240b2470 0x7fb524158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb52415d330 0x7fb52415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524160830 con 0x7fb5240d3850 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524158f70 con 0x7fb5240b2470 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb52a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240b4740 con 0x7fb52415d330 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57902/0 (socket says 172.21.15.91:57902) 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 -- 172.21.15.91:0/2505193140 learned_addr learned my addr 172.21.15.91:0/2505193140 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb523400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5240b2470 0x7fb524158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 -- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb52415d330 msgr2=0x7fb52415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb52415d330 0x7fb52415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 -- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5240b2470 msgr2=0x7fb524158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5240b2470 0x7fb524158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.296+0000 7fb522a00640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c009a70 con 0x7fb5240d3850 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb522a00640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c410 tx=0x7fb50c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c005ce0 con 0x7fb5240d3850 2024-02-02T13:28:44.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c005e40 con 0x7fb5240d3850 2024-02-02T13:28:44.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5240b3e10 con 0x7fb5240d3850 2024-02-02T13:28:44.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0072f0 con 0x7fb5240d3850 2024-02-02T13:28:44.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.297+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5240b43d0 con 0x7fb5240d3850 2024-02-02T13:28:44.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.298+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524067400 con 0x7fb5240d3850 2024-02-02T13:28:44.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.299+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb50c007450 con 0x7fb5240d3850 2024-02-02T13:28:44.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.300+0000 7fb51b400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 0x7fb4fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.300+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb50c087ff0 con 0x7fb5240d3850 2024-02-02T13:28:44.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.300+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fb4fc071f90 con 0x7fb5240d3850 2024-02-02T13:28:44.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.300+0000 7fb523400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 0x7fb4fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.301+0000 7fb523400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 0x7fb4fc06c920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb514009770 tx=0x7fb514008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.304+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb50c0baca0 con 0x7fb5240d3850 2024-02-02T13:28:44.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.493+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5240b4880 con 0x7fb5240d3850 2024-02-02T13:28:44.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.495+0000 7fb51b400640 1 -- 172.21.15.91:0/2505193140 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb50c017030 con 0x7fb5240d3850 2024-02-02T13:28:44.496 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 msgr2=0x7fb4fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 0x7fb4fc06c920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb514009770 tx=0x7fb514008040 comp rx=0 tx=0).stop 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 msgr2=0x7fb52415a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c410 tx=0x7fb50c00c8d0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 shutdown_connections 2024-02-02T13:28:44.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb4fc06a460 0x7fb4fc06c920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb52415d330 0x7fb52415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb5240d3850 0x7fb52415a800 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 --2- 172.21.15.91:0/2505193140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5240b2470 0x7fb524158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 >> 172.21.15.91:0/2505193140 conn(0x7fb524070660 msgr2=0x7fb5240d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.497+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 shutdown_connections 2024-02-02T13:28:44.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.498+0000 7fb52a400640 1 -- 172.21.15.91:0/2505193140 wait complete. 2024-02-02T13:28:44.517 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:28:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.719+0000 7f9a18000640 1 Processor -- start 2024-02-02T13:28:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.719+0000 7f9a18000640 1 -- start start 2024-02-02T13:28:44.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.720+0000 7f9a18000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a101609b0 0x7f9a100cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.720+0000 7f9a18000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 0x7f9a10070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a18000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a10160d90 con 0x7f9a100707b0 2024-02-02T13:28:44.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a18000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a10070be0 con 0x7f9a101609b0 2024-02-02T13:28:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a18000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a10070fe0 con 0x7f9a100ce530 2024-02-02T13:28:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a15600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a101609b0 0x7f9a100cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a16000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9a100707b0 0x7f9a100d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57042/0 (socket says 172.21.15.91:57042) 2024-02-02T13:28:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a16000640 1 -- 172.21.15.91:0/214678397 learned_addr learned my addr 172.21.15.91:0/214678397 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a14c00640 1 --2- 172.21.15.91:0/214678397 >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 0x7f9a10070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a15600640 1 -- 172.21.15.91:0/214678397 >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 msgr2=0x7f9a10070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a15600640 1 --2- 172.21.15.91:0/214678397 >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 0x7f9a10070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a15600640 1 -- 172.21.15.91:0/214678397 >> v1:172.21.15.91:6789/0 conn(0x7f9a100707b0 legacy=0x7f9a100d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a15600640 1 -- 172.21.15.91:0/214678397 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100d2940 con 0x7f9a101609b0 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.721+0000 7f9a14c00640 1 --2- 172.21.15.91:0/214678397 >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 0x7f9a10070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:44.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.723+0000 7f9a15600640 1 --2- 172.21.15.91:0/214678397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a101609b0 0x7f9a100cdff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9a10067830 tx=0x7f9a0002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385dc535c70660ba server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.723+0000 7f9a0fe00640 1 -- 172.21.15.91:0/214678397 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00031710 con 0x7f9a101609b0 2024-02-02T13:28:44.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.723+0000 7f9a0fe00640 1 -- 172.21.15.91:0/214678397 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0002f050 con 0x7f9a101609b0 2024-02-02T13:28:44.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.723+0000 7f9a0fe00640 1 -- 172.21.15.91:0/214678397 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00031a90 con 0x7f9a101609b0 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 -- 172.21.15.91:0/214678397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a101609b0 msgr2=0x7f9a100cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 --2- 172.21.15.91:0/214678397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a101609b0 0x7f9a100cdff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9a10067830 tx=0x7f9a0002b7b0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 -- 172.21.15.91:0/214678397 shutdown_connections 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 --2- 172.21.15.91:0/214678397 >> v2:172.21.15.113:3300/0 conn(0x7f9a100ce530 0x7f9a10070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 --2- 172.21.15.91:0/214678397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a101609b0 0x7f9a100cdff0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 -- 172.21.15.91:0/214678397 >> 172.21.15.91:0/214678397 conn(0x7f9a100d3020 msgr2=0x7f9a100d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 -- 172.21.15.91:0/214678397 shutdown_connections 2024-02-02T13:28:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.724+0000 7f9a18000640 1 -- 172.21.15.91:0/214678397 wait complete. 2024-02-02T13:28:44.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.726+0000 7f9a18000640 1 Processor -- start 2024-02-02T13:28:44.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.726+0000 7f9a18000640 1 -- start start 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a100707b0 0x7f9a100b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a101609b0 0x7f9a100b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100b7590 con 0x7f9a100ce530 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1015e8f0 con 0x7f9a100707b0 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a18000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1015ebd0 con 0x7f9a101609b0 2024-02-02T13:28:44.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57914/0 (socket says 172.21.15.91:57914) 2024-02-02T13:28:44.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.727+0000 7f9a14c00640 1 -- 172.21.15.91:0/1765849198 learned_addr learned my addr 172.21.15.91:0/1765849198 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:44.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a16000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a101609b0 0x7f9a100b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 -- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a101609b0 msgr2=0x7f9a100b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a15600640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a100707b0 0x7f9a100b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a101609b0 0x7f9a100b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 -- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a100707b0 msgr2=0x7f9a100b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a100707b0 0x7f9a100b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009400 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a14c00640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c7f0 tx=0x7f99f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800cea0 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f8007a30 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8007dc0 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.728+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1015eeb0 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.729+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1015f470 con 0x7f9a100ce530 2024-02-02T13:28:44.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.730+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a10066cf0 con 0x7f9a100ce530 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.730+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f99f80113f0 con 0x7f9a100ce530 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.731+0000 7f9a0d600640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 0x7f99e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.731+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99f8088b50 con 0x7f9a100ce530 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.732+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f99e8071f60 con 0x7f9a100ce530 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.732+0000 7f9a15600640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 0x7f99e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:44.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.732+0000 7f9a15600640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 0x7f99e806c8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a00035000 tx=0x7f9a00008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:44.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.739+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f99f801f020 con 0x7f9a100ce530 2024-02-02T13:28:44.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.943+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a100d39e0 con 0x7f9a100ce530 2024-02-02T13:28:44.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.944+0000 7f9a0d600640 1 -- 172.21.15.91:0/1765849198 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7f99f8054090 con 0x7f9a100ce530 2024-02-02T13:28:44.946 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:28:44.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 msgr2=0x7f99e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 0x7f99e806c8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a00035000 tx=0x7f9a00008030 comp rx=0 tx=0).stop 2024-02-02T13:28:44.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 msgr2=0x7f9a1014f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f99f800c7f0 tx=0x7f99f800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 shutdown_connections 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f99e806a430 0x7f99e806c8f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a101609b0 0x7f9a100b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a100ce530 0x7f9a1014f1e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 --2- 172.21.15.91:0/1765849198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a100707b0 0x7f9a100b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 >> 172.21.15.91:0/1765849198 conn(0x7f9a100d3020 msgr2=0x7f9a1015c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 shutdown_connections 2024-02-02T13:28:44.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:44.947+0000 7f9a18000640 1 -- 172.21.15.91:0/1765849198 wait complete. 2024-02-02T13:28:44.966 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:28:44.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:28:44.966 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:28:45.073 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:28:45.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:28:45.073 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:28:45.121 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:28:45.121 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:45.177 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:28:45.178 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:45.230 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:28:45.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.469+0000 7f13a6400640 1 Processor -- start 2024-02-02T13:28:45.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.469+0000 7f13a6400640 1 -- start start 2024-02-02T13:28:45.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.471+0000 7f13a6400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f13a00ce130 0x7f13a00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.471+0000 7f13a6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a015a010 0x7f13a00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.471+0000 7f13a6400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13a00cd470 0x7f13a00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f13a6400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00cc7c0 con 0x7f13a00ce130 2024-02-02T13:28:45.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f13a6400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00ce930 con 0x7f13a00cd470 2024-02-02T13:28:45.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f13a6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a006c890 con 0x7f13a015a010 2024-02-02T13:28:45.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f139f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f13a00ce130 0x7f13a00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f139f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f13a00ce130 0x7f13a00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57918/0 (socket says 172.21.15.91:57918) 2024-02-02T13:28:45.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f139f400640 1 -- 172.21.15.91:0/3341929246 learned_addr learned my addr 172.21.15.91:0/3341929246 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:45.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f139ea00640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a015a010 0x7f13a00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.472+0000 7f139fe00640 1 --2- 172.21.15.91:0/3341929246 >> v2:172.21.15.113:3300/0 conn(0x7f13a00cd470 0x7f13a00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.473+0000 7f139f400640 1 -- 172.21.15.91:0/3341929246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a015a010 msgr2=0x7f13a00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.473+0000 7f139f400640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a015a010 0x7f13a00ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.473+0000 7f139f400640 1 -- 172.21.15.91:0/3341929246 >> v2:172.21.15.113:3300/0 conn(0x7f13a00cd470 msgr2=0x7f13a00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.473+0000 7f139f400640 1 --2- 172.21.15.91:0/3341929246 >> v2:172.21.15.113:3300/0 conn(0x7f13a00cd470 0x7f13a00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.473+0000 7f139f400640 1 -- 172.21.15.91:0/3341929246 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a006cb70 con 0x7f13a00ce130 2024-02-02T13:28:45.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.474+0000 7f139f400640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00c9670 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f13a00cb280 tx=0x7f1398030250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce5b5b44662f781 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.475+0000 7f139e000640 1 -- 172.21.15.91:0/3341929246 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398030df0 con 0x7f13a00ce130 2024-02-02T13:28:45.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.475+0000 7f139e000640 1 -- 172.21.15.91:0/3341929246 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13980383f0 con 0x7f13a00ce130 2024-02-02T13:28:45.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.475+0000 7f139e000640 1 -- 172.21.15.91:0/3341929246 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398038780 con 0x7f13a00ce130 2024-02-02T13:28:45.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.476+0000 7f13a6400640 1 -- 172.21.15.91:0/3341929246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 msgr2=0x7f13a00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.476+0000 7f13a6400640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00c9670 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f13a00cb280 tx=0x7f1398030250 comp rx=0 tx=0).stop 2024-02-02T13:28:45.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.476+0000 7f13a6400640 1 -- 172.21.15.91:0/3341929246 shutdown_connections 2024-02-02T13:28:45.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 --2- 172.21.15.91:0/3341929246 >> v2:172.21.15.113:3300/0 conn(0x7f13a00cd470 0x7f13a00b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a015a010 0x7f13a00ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 --2- 172.21.15.91:0/3341929246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00c9670 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 -- 172.21.15.91:0/3341929246 >> 172.21.15.91:0/3341929246 conn(0x7f13a00cec80 msgr2=0x7f13a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:45.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 -- 172.21.15.91:0/3341929246 shutdown_connections 2024-02-02T13:28:45.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.477+0000 7f13a6400640 1 -- 172.21.15.91:0/3341929246 wait complete. 2024-02-02T13:28:45.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.479+0000 7f13a6400640 1 Processor -- start 2024-02-02T13:28:45.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.480+0000 7f13a6400640 1 -- start start 2024-02-02T13:28:45.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015a010 0x7f13a00b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00b3e00 con 0x7f13a00ce130 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0158bf0 con 0x7f13a015a010 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f13a6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00b3180 con 0x7f13a01564b0 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f139fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f139fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:45336/0 (socket says 172.21.15.91:45336) 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f139fe00640 1 -- 172.21.15.91:0/339117873 learned_addr learned my addr 172.21.15.91:0/339117873 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f139f400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.481+0000 7f139ea00640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015a010 0x7f13a00b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 -- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015a010 msgr2=0x7f13a00b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015a010 0x7f13a00b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 -- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 msgr2=0x7f13a00b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009b30 con 0x7f13a01564b0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139f400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.482+0000 7f139fe00640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f139000e770 tx=0x7f139000ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.483+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900094e0 con 0x7f13a01564b0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.483+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139000c8e0 con 0x7f13a01564b0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.483+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139000cc70 con 0x7f13a01564b0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.483+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a00b3460 con 0x7f13a01564b0 2024-02-02T13:28:45.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.483+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a00b3a20 con 0x7f13a01564b0 2024-02-02T13:28:45.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.485+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a0066cf0 con 0x7f13a01564b0 2024-02-02T13:28:45.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.488+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1390005990 con 0x7f13a01564b0 2024-02-02T13:28:45.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.489+0000 7f1397400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 0x7f137806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.489+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1390088c50 con 0x7f13a01564b0 2024-02-02T13:28:45.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.490+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f1378071f60 con 0x7f13a01564b0 2024-02-02T13:28:45.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.490+0000 7f139f400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 0x7f137806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.492+0000 7f139f400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 0x7f137806c8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1398039000 tx=0x7f1398034000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.495+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1390021060 con 0x7f13a01564b0 2024-02-02T13:28:45.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.715+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13a00b4c00 con 0x7f13a01564b0 2024-02-02T13:28:45.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.718+0000 7f1397400640 1 -- 172.21.15.91:0/339117873 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16155 (secure 0 0 0) 0x7f1390017020 con 0x7f13a01564b0 2024-02-02T13:28:45.720 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:45.721 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":71,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:27:45.358247+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 msgr2=0x7f137806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 0x7f137806c8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1398039000 tx=0x7f1398034000 comp rx=0 tx=0).stop 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 msgr2=0x7f13a006b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f139000e770 tx=0x7f139000ec30 comp rx=0 tx=0).stop 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 shutdown_connections 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f137806a430 0x7f137806c8f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a01564b0 0x7f13a006b5f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015a010 0x7f13a00b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 --2- 172.21.15.91:0/339117873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13a00ce130 0x7f13a00b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 >> 172.21.15.91:0/339117873 conn(0x7f13a00cec80 msgr2=0x7f13a00ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:45.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 shutdown_connections 2024-02-02T13:28:45.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.721+0000 7f13a6400640 1 -- 172.21.15.91:0/339117873 wait complete. 2024-02-02T13:28:45.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:27:23.838598+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:27:25.123347+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:28:45.742 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:28:45.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.948+0000 7f9c19400640 1 Processor -- start 2024-02-02T13:28:45.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.948+0000 7f9c19400640 1 -- start start 2024-02-02T13:28:45.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.949+0000 7f9c19400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c140cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.949+0000 7f9c19400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c140c1c70 0x7f9c140ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c19400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140ae1b0 con 0x7f9c140cd0e0 2024-02-02T13:28:45.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c19400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1406b450 con 0x7f9c140c1c70 2024-02-02T13:28:45.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c19400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14161b30 con 0x7f9c140addd0 2024-02-02T13:28:45.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c0b200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c140c1c70 0x7f9c140ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c18a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9c140cd0e0 0x7f9c141651b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57046/0 (socket says 172.21.15.91:57046) 2024-02-02T13:28:45.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c18a00640 1 -- 172.21.15.91:0/2048641892 learned_addr learned my addr 172.21.15.91:0/2048641892 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c13e00640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c140cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c0b200640 1 -- 172.21.15.91:0/2048641892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 msgr2=0x7f9c140cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c0b200640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c140cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c0b200640 1 -- 172.21.15.91:0/2048641892 >> v1:172.21.15.91:6789/0 conn(0x7f9c140cd0e0 legacy=0x7f9c141651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.950+0000 7f9c0b200640 1 -- 172.21.15.91:0/2048641892 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140c9430 con 0x7f9c140c1c70 2024-02-02T13:28:45.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.951+0000 7f9c13e00640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c140cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:28:45.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c0b200640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140ccba0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0097a0 tx=0x7f9bfc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5b96dab0ad220d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c13400640 1 -- 172.21.15.91:0/2048641892 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc02d040 con 0x7f9c140c1c70 2024-02-02T13:28:45.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c13400640 1 -- 172.21.15.91:0/2048641892 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc031810 con 0x7f9c140c1c70 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c13400640 1 -- 172.21.15.91:0/2048641892 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc031ba0 con 0x7f9c140c1c70 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c19400640 1 -- 172.21.15.91:0/2048641892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 msgr2=0x7f9c140ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.952+0000 7f9c19400640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140ccba0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0097a0 tx=0x7f9bfc02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 -- 172.21.15.91:0/2048641892 shutdown_connections 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140ccba0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 --2- 172.21.15.91:0/2048641892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c140cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 -- 172.21.15.91:0/2048641892 >> 172.21.15.91:0/2048641892 conn(0x7f9c140cea90 msgr2=0x7f9c140ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 -- 172.21.15.91:0/2048641892 shutdown_connections 2024-02-02T13:28:45.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.953+0000 7f9c19400640 1 -- 172.21.15.91:0/2048641892 wait complete. 2024-02-02T13:28:45.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.954+0000 7f9c19400640 1 Processor -- start 2024-02-02T13:28:45.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.955+0000 7f9c19400640 1 -- start start 2024-02-02T13:28:45.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.955+0000 7f9c19400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c1415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.955+0000 7f9c19400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c19400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c140cd0e0 0x7f9c140b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c19400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1415ca70 con 0x7f9c140c1c70 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c19400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c141615c0 con 0x7f9c140cd0e0 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c19400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c140c41f0 con 0x7f9c140addd0 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57940/0 (socket says 172.21.15.91:57940) 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 -- 172.21.15.91:0/715693019 learned_addr learned my addr 172.21.15.91:0/715693019 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c13e00640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c1415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c18a00640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c140cd0e0 0x7f9c140b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 -- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 msgr2=0x7f9c1415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c1415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 -- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c140cd0e0 msgr2=0x7f9c140b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c140cd0e0 0x7f9c140b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:45.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.956+0000 7f9c0b200640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc009400 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c0b200640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0096f0 tx=0x7f9bfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc03ad60 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc044400 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c140c44d0 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0447b0 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.957+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c140c4a90 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.958+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c140ce7d0 con 0x7f9c140c1c70 2024-02-02T13:28:45.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.959+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9bfc04d440 con 0x7f9c140c1c70 2024-02-02T13:28:45.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.960+0000 7f9c10c00640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 0x7f9bec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:45.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.960+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bfc0aacc0 con 0x7f9c140c1c70 2024-02-02T13:28:45.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.960+0000 7f9c13e00640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 0x7f9bec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:45.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.960+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f9bec071f60 con 0x7f9c140c1c70 2024-02-02T13:28:45.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.961+0000 7f9c13e00640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 0x7f9bec06c8f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c04006fd0 tx=0x7f9c04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:45.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:45.964+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9bfc0b1080 con 0x7f9c140c1c70 2024-02-02T13:28:46.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.156+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c140c90e0 con 0x7f9c140c1c70 2024-02-02T13:28:46.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.157+0000 7f9c10c00640 1 -- 172.21.15.91:0/715693019 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bfc035070 con 0x7f9c140c1c70 2024-02-02T13:28:46.158 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:28:46.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 msgr2=0x7f9bec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 0x7f9bec06c8f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c04006fd0 tx=0x7f9c04008040 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 msgr2=0x7f9c140afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0096f0 tx=0x7f9bfc0046c0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 shutdown_connections 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bec06a430 0x7f9bec06c8f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c140cd0e0 0x7f9c140b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c140c1c70 0x7f9c140afbb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 --2- 172.21.15.91:0/715693019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c140addd0 0x7f9c1415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 >> 172.21.15.91:0/715693019 conn(0x7f9c140cea90 msgr2=0x7f9c14160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 shutdown_connections 2024-02-02T13:28:46.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.160+0000 7f9c19400640 1 -- 172.21.15.91:0/715693019 wait complete. 2024-02-02T13:28:46.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:28:46.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.386+0000 7fd7f5200640 1 Processor -- start 2024-02-02T13:28:46.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.386+0000 7fd7f5200640 1 -- start start 2024-02-02T13:28:46.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.387+0000 7fd7f5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f0151f90 0x7fd7f015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.387+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f00b0e10 0x7fd7f00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7f5200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f0070f80 con 0x7fd7f006f4f0 2024-02-02T13:28:46.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7f5200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f0152370 con 0x7fd7f0151f90 2024-02-02T13:28:46.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7f5200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00cb010 con 0x7fd7f00b0e10 2024-02-02T13:28:46.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7ede00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f0151f90 0x7fd7f015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7ede00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f0151f90 0x7fd7f015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:60998/0 (socket says 172.21.15.91:60998) 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7ede00640 1 -- 172.21.15.91:0/322957122 learned_addr learned my addr 172.21.15.91:0/322957122 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7e5200640 1 --1- 172.21.15.91:0/322957122 >> v1:172.21.15.113:6789/0 conn(0x7fd7f006f4f0 0x7fd7f00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:39752/0 (socket says 172.21.15.91:39752) 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.388+0000 7fd7ee800640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f00b0e10 0x7fd7f00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.389+0000 7fd7ede00640 1 -- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f00b0e10 msgr2=0x7fd7f00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.389+0000 7fd7ede00640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f00b0e10 0x7fd7f00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.389+0000 7fd7ede00640 1 -- 172.21.15.91:0/322957122 >> v1:172.21.15.113:6789/0 conn(0x7fd7f006f4f0 legacy=0x7fd7f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.389+0000 7fd7ede00640 1 -- 172.21.15.91:0/322957122 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00cb2f0 con 0x7fd7f0151f90 2024-02-02T13:28:46.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.390+0000 7fd7ede00640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f0151f90 0x7fd7f015f5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009870 tx=0x7fd7e002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=659133d4298027b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.391+0000 7fd7ed400640 1 -- 172.21.15.91:0/322957122 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0030840 con 0x7fd7f0151f90 2024-02-02T13:28:46.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.391+0000 7fd7ed400640 1 -- 172.21.15.91:0/322957122 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e00309a0 con 0x7fd7f0151f90 2024-02-02T13:28:46.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.391+0000 7fd7ed400640 1 -- 172.21.15.91:0/322957122 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0030d30 con 0x7fd7f0151f90 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 -- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f0151f90 msgr2=0x7fd7f015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f0151f90 0x7fd7f015f5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009870 tx=0x7fd7e002b980 comp rx=0 tx=0).stop 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 -- 172.21.15.91:0/322957122 shutdown_connections 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f00b0e10 0x7fd7f00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 --2- 172.21.15.91:0/322957122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f0151f90 0x7fd7f015f5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 -- 172.21.15.91:0/322957122 >> 172.21.15.91:0/322957122 conn(0x7fd7f00cec80 msgr2=0x7fd7f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 -- 172.21.15.91:0/322957122 shutdown_connections 2024-02-02T13:28:46.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.392+0000 7fd7f5200640 1 -- 172.21.15.91:0/322957122 wait complete. 2024-02-02T13:28:46.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.394+0000 7fd7f5200640 1 Processor -- start 2024-02-02T13:28:46.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.394+0000 7fd7f5200640 1 -- start start 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 0x7fd7f0150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0151f90 0x7fd7f00c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0150c30 con 0x7fd7f00b0e10 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c7640 con 0x7fd7f006f4f0 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7f5200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c78e0 con 0x7fd7f0151f90 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7e5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7e5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57948/0 (socket says 172.21.15.91:57948) 2024-02-02T13:28:46.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7e5200640 1 -- 172.21.15.91:0/861401788 learned_addr learned my addr 172.21.15.91:0/861401788 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:46.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7ede00640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 0x7fd7f0150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.395+0000 7fd7ee800640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0151f90 0x7fd7f00c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 -- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0151f90 msgr2=0x7fd7f00c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0151f90 0x7fd7f00c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 -- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 msgr2=0x7fd7f0150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 0x7fd7f0150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00094d0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7ede00640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 0x7fd7f0150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c3c0 tx=0x7fd7d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80074e0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8007640 con 0x7fd7f00b0e10 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8004a00 con 0x7fd7f00b0e10 2024-02-02T13:28:46.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.396+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00c7bc0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.397+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00c8180 con 0x7fd7f00b0e10 2024-02-02T13:28:46.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.398+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0066cf0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.399+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7d8004b60 con 0x7fd7f00b0e10 2024-02-02T13:28:46.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.400+0000 7fd7e6a00640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 0x7fd7c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.400+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7d80882f0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.400+0000 7fd7ede00640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 0x7fd7c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.400+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fd7c8071f60 con 0x7fd7f00b0e10 2024-02-02T13:28:46.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.401+0000 7fd7ede00640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 0x7fd7c806c8f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0036000 tx=0x7fd7e0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.404+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7d8053990 con 0x7fd7f00b0e10 2024-02-02T13:28:46.453 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:28:46.451+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-02-02T13:28:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.457+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(72..72 src has 1..72) v4 ==== 296+0+0 (secure 0 0 0) 0x7fd7d8057650 con 0x7fd7f00b0e10 2024-02-02T13:28:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.457+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fd7c8072810 con 0x7fd7f00b0e10 2024-02-02T13:28:46.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.597+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7f015e8e0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.598+0000 7fd7e6a00640 1 -- 172.21.15.91:0/861401788 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7d8056fb0 con 0x7fd7f00b0e10 2024-02-02T13:28:46.600 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:28:46.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.600+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 msgr2=0x7fd7c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.600+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 0x7fd7c806c8f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0036000 tx=0x7fd7e0008030 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.600+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 msgr2=0x7fd7f01506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.600+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c3c0 tx=0x7fd7d800c880 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 shutdown_connections 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7c806a430 0x7fd7c806c8f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0151f90 0x7fd7f00c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7f00b0e10 0x7fd7f01506f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 --2- 172.21.15.91:0/861401788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f006f4f0 0x7fd7f0150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 >> 172.21.15.91:0/861401788 conn(0x7fd7f00cec80 msgr2=0x7fd7f00c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 shutdown_connections 2024-02-02T13:28:46.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.601+0000 7fd7f5200640 1 -- 172.21.15.91:0/861401788 wait complete. 2024-02-02T13:28:46.619 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:28:46.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.824+0000 7f3b2c400640 1 Processor -- start 2024-02-02T13:28:46.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.824+0000 7f3b2c400640 1 -- start start 2024-02-02T13:28:46.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b2c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b240b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b2c400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b240b0c40 0x7f3b240b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b2c400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2415fa00 con 0x7f3b24151f90 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b2c400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240b10d0 con 0x7f3b240b0c40 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b2c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24070f80 con 0x7f3b2415f5a0 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b29a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3b24151f90 0x7f3b2406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57052/0 (socket says 172.21.15.91:57052) 2024-02-02T13:28:46.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b29a00640 1 -- 172.21.15.91:0/3495969046 learned_addr learned my addr 172.21.15.91:0/3495969046 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.826+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636186661 0 0) 0x7f3b14001110 con 0x7f3b24151f90 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b04003520 con 0x7f3b24151f90 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b29000640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b240b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b2a400640 1 --2- 172.21.15.91:0/3495969046 >> v2:172.21.15.113:3300/0 conn(0x7f3b240b0c40 0x7f3b240b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933031890 0 0) 0x7f3b04003520 con 0x7f3b24151f90 2024-02-02T13:28:46.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 msgr2=0x7f3b240b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b240b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 >> v2:172.21.15.113:3300/0 conn(0x7f3b240b0c40 msgr2=0x7f3b240b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 --2- 172.21.15.91:0/3495969046 >> v2:172.21.15.113:3300/0 conn(0x7f3b240b0c40 0x7f3b240b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24152370 con 0x7f3b24151f90 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3b14001800 con 0x7f3b24151f90 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 >> v1:172.21.15.91:6789/0 conn(0x7f3b24151f90 legacy=0x7f3b2406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b04003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b04003fa0 0x7f3b040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240b10d0 con 0x7f3b04004d00 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24070f80 con 0x7f3b04003fa0 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0400ba80 con 0x7f3b2415f5a0 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.827+0000 7f3b2a400640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b29000640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b04003fa0 0x7f3b040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b29a00640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b04003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b2a400640 1 -- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 msgr2=0x7f3b04003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b2a400640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b2415f5a0 0x7f3b04003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b2a400640 1 -- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b04003fa0 msgr2=0x7f3b040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b2a400640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b04003fa0 0x7f3b040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.828+0000 7f3b2a400640 1 -- 172.21.15.91:0/3495969046 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04003520 con 0x7f3b04004d00 2024-02-02T13:28:46.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.829+0000 7f3b2a400640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b04005180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007360 tx=0x7f3b040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfb6efdea7c6b94 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.829+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c02ccf0 con 0x7f3b04004d00 2024-02-02T13:28:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.829+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c02ce50 con 0x7f3b04004d00 2024-02-02T13:28:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.829+0000 7f3b23e00640 1 -- 172.21.15.91:0/3495969046 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c034660 con 0x7f3b04004d00 2024-02-02T13:28:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 -- 172.21.15.91:0/3495969046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 msgr2=0x7f3b04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 --2- 172.21.15.91:0/3495969046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b04005180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007360 tx=0x7f3b040057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2a400640 1 -- 172.21.15.91:0/3495969046 reap_dead start 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 -- 172.21.15.91:0/3495969046 shutdown_connections 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 -- 172.21.15.91:0/3495969046 >> 172.21.15.91:0/3495969046 conn(0x7f3b240cec80 msgr2=0x7f3b240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 -- 172.21.15.91:0/3495969046 shutdown_connections 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.830+0000 7f3b2c400640 1 -- 172.21.15.91:0/3495969046 wait complete. 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.832+0000 7f3b2c400640 1 Processor -- start 2024-02-02T13:28:46.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.832+0000 7f3b2c400640 1 -- start start 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b241f5ce0 0x7f3b240c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b240c7100 0x7f3b241593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240c5cc0 con 0x7f3b04004d00 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240c5f60 con 0x7f3b240c7100 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b2c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b241f64c0 con 0x7f3b241f5ce0 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b29a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b29a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57972/0 (socket says 172.21.15.91:57972) 2024-02-02T13:28:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.833+0000 7f3b29a00640 1 -- 172.21.15.91:0/2260469435 learned_addr learned my addr 172.21.15.91:0/2260469435 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29000640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b241f5ce0 0x7f3b240c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 -- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b241f5ce0 msgr2=0x7f3b240c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b241f5ce0 0x7f3b240c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 -- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b240c7100 msgr2=0x7f3b241593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b240c7100 0x7f3b241593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b241f67a0 con 0x7f3b04004d00 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b29a00640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007360 tx=0x7f3b14007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400e870 con 0x7f3b04004d00 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1400e9d0 con 0x7f3b04004d00 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400ed80 con 0x7f3b04004d00 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b241f6a80 con 0x7f3b04004d00 2024-02-02T13:28:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.834+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b241f6160 con 0x7f3b04004d00 2024-02-02T13:28:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.836+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b241f6c70 con 0x7f3b04004d00 2024-02-02T13:28:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.836+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3b1401d460 con 0x7f3b04004d00 2024-02-02T13:28:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.837+0000 7f3b21600640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 0x7f3afc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.837+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b140888d0 con 0x7f3b04004d00 2024-02-02T13:28:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.838+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f3afc071f90 con 0x7f3b04004d00 2024-02-02T13:28:46.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.838+0000 7f3b29000640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 0x7f3afc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:46.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.838+0000 7f3b29000640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 0x7f3afc06c920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b2415a9a0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:46.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:46.842+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3b241f6c70 con 0x7f3b04004d00 2024-02-02T13:28:47.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.029+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b2415fa00 con 0x7f3b04004d00 2024-02-02T13:28:47.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.031+0000 7f3b21600640 1 -- 172.21.15.91:0/2260469435 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b2415fa00 con 0x7f3b04004d00 2024-02-02T13:28:47.032 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 msgr2=0x7f3afc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 0x7f3afc06c920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b2415a9a0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 msgr2=0x7f3b241f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007360 tx=0x7f3b14007820 comp rx=0 tx=0).stop 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 shutdown_connections 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3afc06a460 0x7f3afc06c920 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b241f5ce0 0x7f3b240c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b240c7100 0x7f3b241593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 --2- 172.21.15.91:0/2260469435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b04004d00 0x7f3b241f3290 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.033+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 >> 172.21.15.91:0/2260469435 conn(0x7f3b240cec80 msgr2=0x7f3b240cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:47.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.034+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 shutdown_connections 2024-02-02T13:28:47.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.034+0000 7f3b2c400640 1 -- 172.21.15.91:0/2260469435 wait complete. 2024-02-02T13:28:47.054 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:28:47.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.262+0000 7f927f000640 1 Processor -- start 2024-02-02T13:28:47.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.262+0000 7f927f000640 1 -- start start 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f927f000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f927815ad10 0x7f927806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f927f000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f927806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f927f000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927806d910 con 0x7f92780cb580 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f927f000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92780cd610 con 0x7f927815ad10 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f927f000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92780cc850 con 0x7f927815b0f0 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f927815ad10 0x7f927806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f927815ad10 0x7f927806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57976/0 (socket says 172.21.15.91:57976) 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 -- 172.21.15.91:0/3878411120 learned_addr learned my addr 172.21.15.91:0/3878411120 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 -- 172.21.15.91:0/3878411120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 msgr2=0x7f927806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:47.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277400640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f927806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f927806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 -- 172.21.15.91:0/3878411120 >> v1:172.21.15.113:6789/0 conn(0x7f92780cb580 legacy=0x7f927806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.264+0000 7f9277e00640 1 -- 172.21.15.91:0/3878411120 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780ccb30 con 0x7f927815ad10 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.265+0000 7f9277400640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f927806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.266+0000 7f9277e00640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f927806b7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584cf93be9cb2084 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.266+0000 7f9276a00640 1 -- 172.21.15.91:0/3878411120 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268031640 con 0x7f927815ad10 2024-02-02T13:28:47.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.266+0000 7f9276a00640 1 -- 172.21.15.91:0/3878411120 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92680317a0 con 0x7f927815ad10 2024-02-02T13:28:47.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.266+0000 7f9276a00640 1 -- 172.21.15.91:0/3878411120 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268031b30 con 0x7f927815ad10 2024-02-02T13:28:47.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 -- 172.21.15.91:0/3878411120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 msgr2=0x7f927806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f927806b7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 -- 172.21.15.91:0/3878411120 shutdown_connections 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f927806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 --2- 172.21.15.91:0/3878411120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f927806b7a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 -- 172.21.15.91:0/3878411120 >> 172.21.15.91:0/3878411120 conn(0x7f92780b0700 msgr2=0x7f92780b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 -- 172.21.15.91:0/3878411120 shutdown_connections 2024-02-02T13:28:47.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.267+0000 7f927f000640 1 -- 172.21.15.91:0/3878411120 wait complete. 2024-02-02T13:28:47.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.269+0000 7f927f000640 1 Processor -- start 2024-02-02T13:28:47.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.269+0000 7f927f000640 1 -- start start 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.270+0000 7f927f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92780cb580 0x7f9278153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.270+0000 7f927f000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f927f000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f92780c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f927f000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278150d40 con 0x7f927815ad10 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f927f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780c7d80 con 0x7f92780cb580 2024-02-02T13:28:47.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f927f000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92780c5d70 con 0x7f927815b0f0 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57978/0 (socket says 172.21.15.91:57978) 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 -- 172.21.15.91:0/1954163711 learned_addr learned my addr 172.21.15.91:0/1954163711 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277e00640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92780cb580 0x7f9278153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f927d000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f92780c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 -- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 msgr2=0x7f92780c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f92780c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 -- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92780cb580 msgr2=0x7f9278153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92780cb580 0x7f9278153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.271+0000 7f9277400640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680094d0 con 0x7f927815ad10 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f9277400640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f926000c7f0 tx=0x7f926000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260007820 con 0x7f927815ad10 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260007980 con 0x7f927815ad10 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260007cf0 con 0x7f927815ad10 2024-02-02T13:28:47.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92780c6050 con 0x7f927815ad10 2024-02-02T13:28:47.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.272+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92780c6610 con 0x7f927815ad10 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.273+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92780c67d0 con 0x7f927815ad10 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.274+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92600113f0 con 0x7f927815ad10 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.275+0000 7f926fe00640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 0x7f925006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.275+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9260088c50 con 0x7f927815ad10 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.275+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f9250071f60 con 0x7f927815ad10 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.275+0000 7f9277e00640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 0x7f925006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.276+0000 7f9277e00640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 0x7f925006c8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f926803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.280+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f926001f060 con 0x7f927815ad10 2024-02-02T13:28:47.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.481+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92780ccdf0 con 0x7f927815ad10 2024-02-02T13:28:47.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.481+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(73..73 src has 1..73) v4 ==== 304+0+0 (secure 0 0 0) 0x7f92600585f0 con 0x7f927815ad10 2024-02-02T13:28:47.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.481+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f9250072810 con 0x7f927815ad10 2024-02-02T13:28:47.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.487+0000 7f926fe00640 1 -- 172.21.15.91:0/1954163711 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92780ccdf0 con 0x7f927815ad10 2024-02-02T13:28:47.489 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 msgr2=0x7f925006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 0x7f925006c8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f926803c040 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 msgr2=0x7f92780c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f926000c7f0 tx=0x7f926000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 shutdown_connections 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f925006a430 0x7f925006c8f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f927815b0f0 0x7f92780c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f927815ad10 0x7f92780c7550 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 --2- 172.21.15.91:0/1954163711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92780cb580 0x7f9278153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 >> 172.21.15.91:0/1954163711 conn(0x7f92780b0700 msgr2=0x7f92780b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 shutdown_connections 2024-02-02T13:28:47.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.490+0000 7f927f000640 1 -- 172.21.15.91:0/1954163711 wait complete. 2024-02-02T13:28:47.510 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:28:47.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:28:47.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:28:47.601 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:28:47.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:28:47.601 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:28:47.651 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:28:47.651 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:47.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:28:47.709 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:28:47.709 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:47.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:28:47.764 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:28:47.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.970+0000 7f606a800640 1 Processor -- start 2024-02-02T13:28:47.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.970+0000 7f606a800640 1 -- start start 2024-02-02T13:28:47.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.971+0000 7f606a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 0x7f606415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f606a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f60640b0e10 0x7f60640b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f606a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6064070f80 con 0x7f606406f4f0 2024-02-02T13:28:47.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f606a800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6064152370 con 0x7f60640b0e10 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f606a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640cb010 con 0x7f6064151f90 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f60640b0e10 0x7f60640b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f60640b0e10 0x7f60640b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57982/0 (socket says 172.21.15.91:57982) 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 -- 172.21.15.91:0/750458191 learned_addr learned my addr 172.21.15.91:0/750458191 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063400640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 0x7f606415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 -- 172.21.15.91:0/750458191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 msgr2=0x7f606415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 0x7f606415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 -- 172.21.15.91:0/750458191 >> v1:172.21.15.113:6789/0 conn(0x7f606406f4f0 legacy=0x7f60640b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:47.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063e00640 1 -- 172.21.15.91:0/750458191 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640cb2f0 con 0x7f60640b0e10 2024-02-02T13:28:47.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.972+0000 7f6063400640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 0x7f606415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:47.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.973+0000 7f6063e00640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60640b0e10 0x7f60640b3af0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bc9347b5e1722b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.974+0000 7f6062a00640 1 -- 172.21.15.91:0/750458191 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02ca70 con 0x7f60640b0e10 2024-02-02T13:28:47.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.974+0000 7f6062a00640 1 -- 172.21.15.91:0/750458191 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c02cbd0 con 0x7f60640b0e10 2024-02-02T13:28:47.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.974+0000 7f6062a00640 1 -- 172.21.15.91:0/750458191 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0343f0 con 0x7f60640b0e10 2024-02-02T13:28:47.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.974+0000 7f606a800640 1 -- 172.21.15.91:0/750458191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60640b0e10 msgr2=0x7f60640b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.974+0000 7f606a800640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60640b0e10 0x7f60640b3af0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 -- 172.21.15.91:0/750458191 shutdown_connections 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60640b0e10 0x7f60640b3af0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 --2- 172.21.15.91:0/750458191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6064151f90 0x7f606415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 -- 172.21.15.91:0/750458191 >> 172.21.15.91:0/750458191 conn(0x7f60640cec80 msgr2=0x7f60640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 -- 172.21.15.91:0/750458191 shutdown_connections 2024-02-02T13:28:47.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.975+0000 7f606a800640 1 -- 172.21.15.91:0/750458191 wait complete. 2024-02-02T13:28:47.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.976+0000 7f606a800640 1 Processor -- start 2024-02-02T13:28:47.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.976+0000 7f606a800640 1 -- start start 2024-02-02T13:28:47.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60640b0e10 0x7f60640b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6064151f90 0x7f60640ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b3670 con 0x7f606406f4f0 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b2a60 con 0x7f6064151f90 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.977+0000 7f606a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b2d40 con 0x7f60640b0e10 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57992/0 (socket says 172.21.15.91:57992) 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 -- 172.21.15.91:0/1965418983 learned_addr learned my addr 172.21.15.91:0/1965418983 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 -- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60640b0e10 msgr2=0x7f60640b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60640b0e10 0x7f60640b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 -- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6064151f90 msgr2=0x7f60640ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6064151f90 0x7f60640ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c009b30 con 0x7f606406f4f0 2024-02-02T13:28:47.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.978+0000 7f6063400640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f605400c7f0 tx=0x7f605400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.979+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054007a30 con 0x7f606406f4f0 2024-02-02T13:28:47.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.979+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6054007b90 con 0x7f606406f4f0 2024-02-02T13:28:47.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.979+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605400ce40 con 0x7f606406f4f0 2024-02-02T13:28:47.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.979+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60640b3020 con 0x7f606406f4f0 2024-02-02T13:28:47.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.979+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60641f3960 con 0x7f606406f4f0 2024-02-02T13:28:47.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.980+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606406f930 con 0x7f606406f4f0 2024-02-02T13:28:47.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.981+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f60540113f0 con 0x7f606406f4f0 2024-02-02T13:28:47.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.981+0000 7f605b400640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 0x7f603c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:47.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.982+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6054088aa0 con 0x7f606406f4f0 2024-02-02T13:28:47.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.982+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f603c071f90 con 0x7f606406f4f0 2024-02-02T13:28:47.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.982+0000 7f605be00640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 0x7f603c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:47.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.983+0000 7f605be00640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 0x7f603c06c920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f604c006fd0 tx=0x7f604c006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:47.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:47.986+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f605401f060 con 0x7f606406f4f0 2024-02-02T13:28:48.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.212+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f606415eb90 con 0x7f606406f4f0 2024-02-02T13:28:48.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.213+0000 7f605b400640 1 -- 172.21.15.91:0/1965418983 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f605401f020 con 0x7f606406f4f0 2024-02-02T13:28:48.215 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:48.215 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:28:48.215 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 msgr2=0x7f603c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 0x7f603c06c920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f604c006fd0 tx=0x7f604c006bc0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 msgr2=0x7f60641f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f605400c7f0 tx=0x7f605400cbf0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 shutdown_connections 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f603c06a460 0x7f603c06c920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60640b0e10 0x7f60640b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6064151f90 0x7f60640ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 --2- 172.21.15.91:0/1965418983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f606406f4f0 0x7f60641f3420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 >> 172.21.15.91:0/1965418983 conn(0x7f60640cec80 msgr2=0x7f60640b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 shutdown_connections 2024-02-02T13:28:48.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.216+0000 7f606a800640 1 -- 172.21.15.91:0/1965418983 wait complete. 2024-02-02T13:28:48.237 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:28:48.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.432+0000 7fd3e5600640 1 Processor -- start 2024-02-02T13:28:48.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.432+0000 7fd3e5600640 1 -- start start 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.433+0000 7fd3e5600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3e015f5a0 0x7fd3e00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.433+0000 7fd3e5600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.433+0000 7fd3e5600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e0070f80 con 0x7fd3e0151f90 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.433+0000 7fd3e5600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e015fa30 con 0x7fd3e015f5a0 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.433+0000 7fd3e5600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00b10b0 con 0x7fd3e00b0c40 2024-02-02T13:28:48.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3e015f5a0 0x7fd3e00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3dea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd3e0151f90 0x7fd3e006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57062/0 (socket says 172.21.15.91:57062) 2024-02-02T13:28:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3dea00640 1 -- 172.21.15.91:0/1235275614 learned_addr learned my addr 172.21.15.91:0/1235275614 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3df400640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3d7e00640 1 -- 172.21.15.91:0/1235275614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 msgr2=0x7fd3e00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:48.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3d7e00640 1 -- 172.21.15.91:0/1235275614 >> v1:172.21.15.91:6789/0 conn(0x7fd3e0151f90 legacy=0x7fd3e006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3d7e00640 1 -- 172.21.15.91:0/1235275614 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0152370 con 0x7fd3e015f5a0 2024-02-02T13:28:48.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.434+0000 7fd3df400640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:48.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.436+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e015f5a0 0x7fd3e00b0700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80097a0 tx=0x7fd3c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee1d13edb5b10b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.436+0000 7fd3de000640 1 -- 172.21.15.91:0/1235275614 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802d040 con 0x7fd3e015f5a0 2024-02-02T13:28:48.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.436+0000 7fd3de000640 1 -- 172.21.15.91:0/1235275614 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c8031810 con 0x7fd3e015f5a0 2024-02-02T13:28:48.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.436+0000 7fd3de000640 1 -- 172.21.15.91:0/1235275614 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8031ba0 con 0x7fd3e015f5a0 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 -- 172.21.15.91:0/1235275614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e015f5a0 msgr2=0x7fd3e00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e015f5a0 0x7fd3e00b0700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80097a0 tx=0x7fd3c802b8b0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 -- 172.21.15.91:0/1235275614 shutdown_connections 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 --2- 172.21.15.91:0/1235275614 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e015f5a0 0x7fd3e00b0700 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 -- 172.21.15.91:0/1235275614 >> 172.21.15.91:0/1235275614 conn(0x7fd3e00cec80 msgr2=0x7fd3e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 -- 172.21.15.91:0/1235275614 shutdown_connections 2024-02-02T13:28:48.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.437+0000 7fd3e5600640 1 -- 172.21.15.91:0/1235275614 wait complete. 2024-02-02T13:28:48.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.439+0000 7fd3e5600640 1 Processor -- start 2024-02-02T13:28:48.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.439+0000 7fd3e5600640 1 -- start start 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e015f5a0 0x7fd3e015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c6210 con 0x7fd3e0151f90 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c64f0 con 0x7fd3e015f5a0 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3e5600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c67d0 con 0x7fd3e00b0c40 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58010/0 (socket says 172.21.15.91:58010) 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 -- 172.21.15.91:0/3779752213 learned_addr learned my addr 172.21.15.91:0/3779752213 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3dea00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 -- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 msgr2=0x7fd3e00c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 -- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e015f5a0 msgr2=0x7fd3e015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e015f5a0 0x7fd3e015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.440+0000 7fd3d7e00640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8009400 con 0x7fd3e0151f90 2024-02-02T13:28:48.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3d7e00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c803ad60 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c80443f0 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80447a0 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e00c6ab0 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.441+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e00c7fe0 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.442+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e00c6c70 con 0x7fd3e0151f90 2024-02-02T13:28:48.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.443+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3c804d450 con 0x7fd3e0151f90 2024-02-02T13:28:48.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.444+0000 7fd3d6a00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 0x7fd3b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.444+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3c80aacd0 con 0x7fd3e0151f90 2024-02-02T13:28:48.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.444+0000 7fd3dea00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 0x7fd3b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.444+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fd3b8071f60 con 0x7fd3e0151f90 2024-02-02T13:28:48.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.445+0000 7fd3dea00640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 0x7fd3b806c8f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005880 tx=0x7fd3d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.448+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd3c80762c0 con 0x7fd3e0151f90 2024-02-02T13:28:48.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.641+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3e0066cf0 con 0x7fd3e0151f90 2024-02-02T13:28:48.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.644+0000 7fd3d6a00640 1 -- 172.21.15.91:0/3779752213 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16126 (secure 0 0 0) 0x7fd3c8035030 con 0x7fd3e0151f90 2024-02-02T13:28:48.645 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:48.646 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":73,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:28:47.471605+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 msgr2=0x7fd3b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 0x7fd3b806c8f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005880 tx=0x7fd3d0009040 comp rx=0 tx=0).stop 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 msgr2=0x7fd3e01510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c80046c0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 shutdown_connections 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3b806a430 0x7fd3b806c8f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e015f5a0 0x7fd3e015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3e0151f90 0x7fd3e01510b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 --2- 172.21.15.91:0/3779752213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3e00b0c40 0x7fd3e00c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 >> 172.21.15.91:0/3779752213 conn(0x7fd3e00cec80 msgr2=0x7fd3e00ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 shutdown_connections 2024-02-02T13:28:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.646+0000 7fd3e5600640 1 -- 172.21.15.91:0/3779752213 wait complete. 2024-02-02T13:28:48.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:27:23.838598+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:27:25.123347+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:28:48.666 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:28:48.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.867+0000 7f291d600640 1 Processor -- start 2024-02-02T13:28:48.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.867+0000 7f291d600640 1 -- start start 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918160830 0x7f29180b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 0x7f29180711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2918070270 0x7f2918070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2918070c40 con 0x7f2918070270 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180ce210 con 0x7f29180b1d70 2024-02-02T13:28:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.868+0000 7f291d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180ce4f0 con 0x7f2918160830 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2918070270 0x7f2918070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2918070270 0x7f2918070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58018/0 (socket says 172.21.15.91:58018) 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 -- 172.21.15.91:0/780265751 learned_addr learned my addr 172.21.15.91:0/780265751 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2916a00640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918160830 0x7f29180b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f290de00640 1 --2- 172.21.15.91:0/780265751 >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 0x7f29180711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 -- 172.21.15.91:0/780265751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918160830 msgr2=0x7f29180b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918160830 0x7f29180b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 -- 172.21.15.91:0/780265751 >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 msgr2=0x7f29180711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 --2- 172.21.15.91:0/780265751 >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 0x7f29180711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f2917400640 1 -- 172.21.15.91:0/780265751 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180ce7d0 con 0x7f2918070270 2024-02-02T13:28:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.869+0000 7f290de00640 1 --2- 172.21.15.91:0/780265751 >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 0x7f29180711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.870+0000 7f2917400640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2918070270 0x7f2918070700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f291808b100 tx=0x7f291002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b06e4e73722b9ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f2916000640 1 -- 172.21.15.91:0/780265751 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291002c9e0 con 0x7f2918070270 2024-02-02T13:28:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f2916000640 1 -- 172.21.15.91:0/780265751 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291002cb40 con 0x7f2918070270 2024-02-02T13:28:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f2916000640 1 -- 172.21.15.91:0/780265751 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100343f0 con 0x7f2918070270 2024-02-02T13:28:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 -- 172.21.15.91:0/780265751 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2918070270 msgr2=0x7f2918070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2918070270 0x7f2918070700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f291808b100 tx=0x7f291002be40 comp rx=0 tx=0).stop 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 -- 172.21.15.91:0/780265751 shutdown_connections 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2918070270 0x7f2918070700 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 --2- 172.21.15.91:0/780265751 >> v2:172.21.15.113:3300/0 conn(0x7f29180b1d70 0x7f29180711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 --2- 172.21.15.91:0/780265751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918160830 0x7f29180b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 -- 172.21.15.91:0/780265751 >> 172.21.15.91:0/780265751 conn(0x7f29180d3030 msgr2=0x7f29180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 -- 172.21.15.91:0/780265751 shutdown_connections 2024-02-02T13:28:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.871+0000 7f291d600640 1 -- 172.21.15.91:0/780265751 wait complete. 2024-02-02T13:28:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.873+0000 7f291d600640 1 Processor -- start 2024-02-02T13:28:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.873+0000 7f291d600640 1 -- start start 2024-02-02T13:28:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918070270 0x7f29181f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 0x7f29180c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180ca100 con 0x7f29180b1d70 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29181f6f00 con 0x7f2918160830 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f291d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180ccb60 con 0x7f2918070270 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f290de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f290de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58034/0 (socket says 172.21.15.91:58034) 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.874+0000 7f290de00640 1 -- 172.21.15.91:0/867750817 learned_addr learned my addr 172.21.15.91:0/867750817 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f2916a00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918070270 0x7f29181f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f2917400640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 0x7f29180c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 -- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918070270 msgr2=0x7f29181f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918070270 0x7f29181f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 -- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 msgr2=0x7f29180c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 0x7f29180c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910009b30 con 0x7f29180b1d70 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f2917400640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 0x7f29180c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290de00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f290000c7f0 tx=0x7f290000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900007820 con 0x7f29180b1d70 2024-02-02T13:28:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2900007980 con 0x7f29180b1d70 2024-02-02T13:28:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900007cf0 con 0x7f29180b1d70 2024-02-02T13:28:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.875+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29180cce40 con 0x7f29180b1d70 2024-02-02T13:28:48.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.876+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29180ca5c0 con 0x7f29180b1d70 2024-02-02T13:28:48.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.877+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29180cd080 con 0x7f29180b1d70 2024-02-02T13:28:48.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.877+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f29000113f0 con 0x7f29180b1d70 2024-02-02T13:28:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.878+0000 7f290f400640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 0x7f28f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.878+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2900088ce0 con 0x7f29180b1d70 2024-02-02T13:28:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.879+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f28f0071f60 con 0x7f29180b1d70 2024-02-02T13:28:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.879+0000 7f2916a00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 0x7f28f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.879+0000 7f2916a00640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 0x7f28f006c8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2908006fd0 tx=0x7f2908008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:48.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:48.883+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f290001f060 con 0x7f29180b1d70 2024-02-02T13:28:49.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.055+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2918066cf0 con 0x7f29180b1d70 2024-02-02T13:28:49.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.056+0000 7f290f400640 1 -- 172.21.15.91:0/867750817 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29180cd080 con 0x7f29180b1d70 2024-02-02T13:28:49.058 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 msgr2=0x7f28f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 0x7f28f006c8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2908006fd0 tx=0x7f2908008040 comp rx=0 tx=0).stop 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 msgr2=0x7f2918158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f290000c7f0 tx=0x7f290000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 shutdown_connections 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a430 0x7f28f006c8f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2918160830 0x7f29180c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b1d70 0x7f2918158e00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 --2- 172.21.15.91:0/867750817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2918070270 0x7f29181f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.059+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 >> 172.21.15.91:0/867750817 conn(0x7f29180d3030 msgr2=0x7f29180b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.060+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 shutdown_connections 2024-02-02T13:28:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.060+0000 7f291d600640 1 -- 172.21.15.91:0/867750817 wait complete. 2024-02-02T13:28:49.079 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:28:49.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.287+0000 7f73b4000640 1 Processor -- start 2024-02-02T13:28:49.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.288+0000 7f73b4000640 1 -- start start 2024-02-02T13:28:49.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.289+0000 7f73b4000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f73ac15a010 0x7f73ac0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.289+0000 7f73b4000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 0x7f73ac0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.289+0000 7f73b4000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0cc7c0 con 0x7f73ac0cd470 2024-02-02T13:28:49.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.289+0000 7f73b4000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac0ce930 con 0x7f73ac15a010 2024-02-02T13:28:49.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.289+0000 7f73b4000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac06c890 con 0x7f73ac0ce130 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b2000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f73ac0cd470 0x7f73ac0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57072/0 (socket says 172.21.15.91:57072) 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b2000640 1 -- 172.21.15.91:0/180019003 learned_addr learned my addr 172.21.15.91:0/180019003 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b1600640 1 --2- 172.21.15.91:0/180019003 >> v2:172.21.15.91:3300/0 conn(0x7f73ac15a010 0x7f73ac0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b0c00640 1 --2- 172.21.15.91:0/180019003 >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 0x7f73ac0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b1600640 1 -- 172.21.15.91:0/180019003 >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 msgr2=0x7f73ac0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b1600640 1 --2- 172.21.15.91:0/180019003 >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 0x7f73ac0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b1600640 1 -- 172.21.15.91:0/180019003 >> v1:172.21.15.91:6789/0 conn(0x7f73ac0cd470 legacy=0x7f73ac0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b1600640 1 -- 172.21.15.91:0/180019003 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac06cb70 con 0x7f73ac15a010 2024-02-02T13:28:49.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73b0c00640 1 --2- 172.21.15.91:0/180019003 >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 0x7f73ac0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:49.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.290+0000 7f73abe00640 1 -- 172.21.15.91:0/180019003 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147645035 0 0) 0x7f73ac0cc7c0 con 0x7f73ac0cd470 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.291+0000 7f73b1600640 1 --2- 172.21.15.91:0/180019003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac15a010 0x7f73ac0c9670 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0b1870 tx=0x7f739c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654935d3bdb679c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.291+0000 7f73abe00640 1 -- 172.21.15.91:0/180019003 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c02e070 con 0x7f73ac15a010 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73abe00640 1 -- 172.21.15.91:0/180019003 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0315a0 con 0x7f73ac15a010 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73abe00640 1 -- 172.21.15.91:0/180019003 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c031930 con 0x7f73ac15a010 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 -- 172.21.15.91:0/180019003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac15a010 msgr2=0x7f73ac0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 --2- 172.21.15.91:0/180019003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac15a010 0x7f73ac0c9670 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0b1870 tx=0x7f739c02b840 comp rx=0 tx=0).stop 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 -- 172.21.15.91:0/180019003 shutdown_connections 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 --2- 172.21.15.91:0/180019003 >> v2:172.21.15.113:3300/0 conn(0x7f73ac0ce130 0x7f73ac0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 --2- 172.21.15.91:0/180019003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac15a010 0x7f73ac0c9670 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 -- 172.21.15.91:0/180019003 >> 172.21.15.91:0/180019003 conn(0x7f73ac0cec80 msgr2=0x7f73ac0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 -- 172.21.15.91:0/180019003 shutdown_connections 2024-02-02T13:28:49.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.292+0000 7f73b4000640 1 -- 172.21.15.91:0/180019003 wait complete. 2024-02-02T13:28:49.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.294+0000 7f73b4000640 1 Processor -- start 2024-02-02T13:28:49.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.294+0000 7f73b4000640 1 -- start start 2024-02-02T13:28:49.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73ac0ce130 0x7f73ac1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ac15a010 0x7f73ac0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0c3d40 con 0x7f73ac0c4160 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0c33a0 con 0x7f73ac15a010 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b4000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0c3680 con 0x7f73ac0ce130 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.295+0000 7f73b2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58052/0 (socket says 172.21.15.91:58052) 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 -- 172.21.15.91:0/2957591295 learned_addr learned my addr 172.21.15.91:0/2957591295 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 -- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73ac0ce130 msgr2=0x7f73ac1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b0c00640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ac15a010 0x7f73ac0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73ac0ce130 0x7f73ac1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 -- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ac15a010 msgr2=0x7f73ac0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ac15a010 0x7f73ac0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009400 con 0x7f73ac0c4160 2024-02-02T13:28:49.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73b2000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f73a4001560 tx=0x7f73a400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.296+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ce30 con 0x7f73ac0c4160 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.297+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400a610 con 0x7f73ac0c4160 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.297+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac0c3960 con 0x7f73ac0c4160 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.297+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400a9a0 con 0x7f73ac0c4160 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.297+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac0c3050 con 0x7f73ac0c4160 2024-02-02T13:28:49.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.298+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac0c3e80 con 0x7f73ac0c4160 2024-02-02T13:28:49.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.299+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f73a400ab00 con 0x7f73ac0c4160 2024-02-02T13:28:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.299+0000 7f73a9600640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 0x7f738406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.299+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73a4087fd0 con 0x7f73ac0c4160 2024-02-02T13:28:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.300+0000 7f73b1600640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 0x7f738406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.300+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f7384071f90 con 0x7f73ac0c4160 2024-02-02T13:28:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.300+0000 7f73b1600640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 0x7f738406c920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f739c036000 tx=0x7f739c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.304+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f73a4018060 con 0x7f73ac0c4160 2024-02-02T13:28:49.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.487+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(74..74 src has 1..74) v4 ==== 1476+0+0 (secure 0 0 0) 0x7f73a4057a10 con 0x7f73ac0c4160 2024-02-02T13:28:49.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.487+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f7384073140 con 0x7f73ac0c4160 2024-02-02T13:28:49.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.500+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73ac066cf0 con 0x7f73ac0c4160 2024-02-02T13:28:49.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.501+0000 7f73a9600640 1 -- 172.21.15.91:0/2957591295 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73a4018070 con 0x7f73ac0c4160 2024-02-02T13:28:49.503 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:28:49.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 msgr2=0x7f738406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 0x7f738406c920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f739c036000 tx=0x7f739c030000 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 msgr2=0x7f73ac0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f73a4001560 tx=0x7f73a400cbe0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 shutdown_connections 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f738406a460 0x7f738406c920 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73ac0c4160 0x7f73ac0b3dc0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ac15a010 0x7f73ac0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 --2- 172.21.15.91:0/2957591295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73ac0ce130 0x7f73ac1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 >> 172.21.15.91:0/2957591295 conn(0x7f73ac0cec80 msgr2=0x7f73ac0c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:49.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 shutdown_connections 2024-02-02T13:28:49.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.504+0000 7f73b4000640 1 -- 172.21.15.91:0/2957591295 wait complete. 2024-02-02T13:28:49.522 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:28:49.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.716+0000 7f0a9bc00640 1 Processor -- start 2024-02-02T13:28:49.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.717+0000 7f0a9bc00640 1 -- start start 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a9bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0a9415a010 0x7f0a940c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a9bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a9bc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940cc7c0 con 0x7f0a940ce130 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a9bc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a940b10d0 con 0x7f0a9415a010 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a9bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940ce930 con 0x7f0a940b0700 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.718+0000 7f0a99200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0a9415a010 0x7f0a940c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0a9415a010 0x7f0a940c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58066/0 (socket says 172.21.15.91:58066) 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 -- 172.21.15.91:0/4117259775 learned_addr learned my addr 172.21.15.91:0/4117259775 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 -- 172.21.15.91:0/4117259775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 msgr2=0x7f0a940b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 --2- 172.21.15.91:0/4117259775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 -- 172.21.15.91:0/4117259775 >> v1:172.21.15.113:6789/0 conn(0x7f0a940ce130 legacy=0x7f0a940ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.719+0000 7f0a99200640 1 -- 172.21.15.91:0/4117259775 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940cd640 con 0x7f0a9415a010 2024-02-02T13:28:49.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.720+0000 7f0a99200640 1 --2- 172.21.15.91:0/4117259775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a9415a010 0x7f0a940c9670 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d553def7970f3d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.720+0000 7f0a93400640 1 -- 172.21.15.91:0/4117259775 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8402f8c0 con 0x7f0a9415a010 2024-02-02T13:28:49.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.720+0000 7f0a93400640 1 -- 172.21.15.91:0/4117259775 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8402fa20 con 0x7f0a9415a010 2024-02-02T13:28:49.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.720+0000 7f0a93400640 1 -- 172.21.15.91:0/4117259775 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8402fdb0 con 0x7f0a9415a010 2024-02-02T13:28:49.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 -- 172.21.15.91:0/4117259775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a9415a010 msgr2=0x7f0a940c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/4117259775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a9415a010 0x7f0a940c9670 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402db00 comp rx=0 tx=0).stop 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 -- 172.21.15.91:0/4117259775 shutdown_connections 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/4117259775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/4117259775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a9415a010 0x7f0a940c9670 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 -- 172.21.15.91:0/4117259775 >> 172.21.15.91:0/4117259775 conn(0x7f0a940cec80 msgr2=0x7f0a940cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 -- 172.21.15.91:0/4117259775 shutdown_connections 2024-02-02T13:28:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.721+0000 7f0a9bc00640 1 -- 172.21.15.91:0/4117259775 wait complete. 2024-02-02T13:28:49.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.723+0000 7f0a9bc00640 1 Processor -- start 2024-02-02T13:28:49.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.723+0000 7f0a9bc00640 1 -- start start 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9415a010 0x7f0a94148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940c62a0 con 0x7f0a940ce130 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940c6580 con 0x7f0a9415a010 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a9bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940c6860 con 0x7f0a940b0700 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58068/0 (socket says 172.21.15.91:58068) 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 -- 172.21.15.91:0/3779227319 learned_addr learned my addr 172.21.15.91:0/3779227319 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a99200640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a99c00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9415a010 0x7f0a94148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 -- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 msgr2=0x7f0a940c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 -- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9415a010 msgr2=0x7f0a94148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9415a010 0x7f0a94148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.725+0000 7f0a93e00640 1 -- 172.21.15.91:0/3779227319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840094d0 con 0x7f0a940ce130 2024-02-02T13:28:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a93e00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c410 tx=0x7f0a7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0035f0 con 0x7f0a940ce130 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c003750 con 0x7f0a940ce130 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c003b40 con 0x7f0a940ce130 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a940c6b40 con 0x7f0a940ce130 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.726+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9415d350 con 0x7f0a940ce130 2024-02-02T13:28:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.727+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94066cf0 con 0x7f0a940ce130 2024-02-02T13:28:49.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.728+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a7c003ca0 con 0x7f0a940ce130 2024-02-02T13:28:49.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.729+0000 7f0a90c00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:49.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.729+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a7c087be0 con 0x7f0a940ce130 2024-02-02T13:28:49.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.729+0000 7f0a99200640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:49.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.730+0000 7f0a99200640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a84038000 tx=0x7f0a84008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:49.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.734+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a7c053150 con 0x7f0a940ce130 2024-02-02T13:28:49.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.921+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a940b4c00 con 0x7f0a940ce130 2024-02-02T13:28:49.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.922+0000 7f0a90c00640 1 -- 172.21.15.91:0/3779227319 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a7c056770 con 0x7f0a940ce130 2024-02-02T13:28:49.924 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 msgr2=0x7f0a6c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a84038000 tx=0x7f0a84008030 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 msgr2=0x7f0a9415ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c410 tx=0x7f0a7c00c8d0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 shutdown_connections 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9415a010 0x7f0a94148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940ce130 0x7f0a9415ce10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 --2- 172.21.15.91:0/3779227319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0700 0x7f0a940c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 >> 172.21.15.91:0/3779227319 conn(0x7f0a940cec80 msgr2=0x7f0a9415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 shutdown_connections 2024-02-02T13:28:49.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:49.925+0000 7f0a9bc00640 1 -- 172.21.15.91:0/3779227319 wait complete. 2024-02-02T13:28:49.945 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:28:50.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.151+0000 7efdf0c00640 1 Processor -- start 2024-02-02T13:28:50.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.151+0000 7efdf0c00640 1 -- start start 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdf0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efdec06c820 0x7efdec0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdf0c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdf0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0ad2e0 con 0x7efdec15d2c0 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdf0c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0ad5c0 con 0x7efdec06c820 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdf0c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06b4f0 con 0x7efdec06cd50 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdebe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdebe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:53934/0 (socket says 172.21.15.91:53934) 2024-02-02T13:28:50.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efdebe00640 1 -- 172.21.15.91:0/1425859594 learned_addr learned my addr 172.21.15.91:0/1425859594 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:50.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.153+0000 7efde3e00640 1 --2- 172.21.15.91:0/1425859594 >> v2:172.21.15.113:3300/0 conn(0x7efdec06c820 0x7efdec0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.154+0000 7efdebe00640 1 -- 172.21.15.91:0/1425859594 >> v2:172.21.15.113:3300/0 conn(0x7efdec06c820 msgr2=0x7efdec0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:50.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.154+0000 7efdebe00640 1 --2- 172.21.15.91:0/1425859594 >> v2:172.21.15.113:3300/0 conn(0x7efdec06c820 0x7efdec0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.154+0000 7efdebe00640 1 -- 172.21.15.91:0/1425859594 >> v1:172.21.15.113:6789/0 conn(0x7efdec15d2c0 legacy=0x7efdec06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.154+0000 7efdebe00640 1 -- 172.21.15.91:0/1425859594 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec06b7d0 con 0x7efdec06cd50 2024-02-02T13:28:50.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.156+0000 7efdebe00640 1 --2- 172.21.15.91:0/1425859594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efde4009e50 tx=0x7efde402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93658d6792ed063 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.156+0000 7efdeaa00640 1 -- 172.21.15.91:0/1425859594 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde402cb00 con 0x7efdec06cd50 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.156+0000 7efdeaa00640 1 -- 172.21.15.91:0/1425859594 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde402cc60 con 0x7efdec06cd50 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.156+0000 7efdeaa00640 1 -- 172.21.15.91:0/1425859594 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40343f0 con 0x7efdec06cd50 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 -- 172.21.15.91:0/1425859594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 msgr2=0x7efdec0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 --2- 172.21.15.91:0/1425859594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efde4009e50 tx=0x7efde402bf60 comp rx=0 tx=0).stop 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 -- 172.21.15.91:0/1425859594 shutdown_connections 2024-02-02T13:28:50.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 --2- 172.21.15.91:0/1425859594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0ccba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 --2- 172.21.15.91:0/1425859594 >> v2:172.21.15.113:3300/0 conn(0x7efdec06c820 0x7efdec0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 -- 172.21.15.91:0/1425859594 >> 172.21.15.91:0/1425859594 conn(0x7efdec0cea90 msgr2=0x7efdec0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:50.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 -- 172.21.15.91:0/1425859594 shutdown_connections 2024-02-02T13:28:50.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.157+0000 7efdf0c00640 1 -- 172.21.15.91:0/1425859594 wait complete. 2024-02-02T13:28:50.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.159+0000 7efdf0c00640 1 Processor -- start 2024-02-02T13:28:50.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.159+0000 7efdf0c00640 1 -- start start 2024-02-02T13:28:50.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 0x7efdec153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0b1450 con 0x7efdec15d2c0 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0b11b0 con 0x7efdec06c820 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdf0c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0b00c0 con 0x7efdec06cd50 2024-02-02T13:28:50.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.160+0000 7efdebe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48876/0 (socket says 172.21.15.91:48876) 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 -- 172.21.15.91:0/897731115 learned_addr learned my addr 172.21.15.91:0/897731115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efde3e00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdeb400640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 0x7efdec153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 -- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 msgr2=0x7efdec0b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 -- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 msgr2=0x7efdec153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 0x7efdec153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 -- 172.21.15.91:0/897731115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4009b30 con 0x7efdec15d2c0 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdeb400640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 0x7efdec153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.161+0000 7efdebe00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efde4009e50 tx=0x7efde4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.162+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4009510 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.162+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde403e450 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.162+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde403e800 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.162+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec15d820 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.162+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec0afb00 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.163+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec0c6bb0 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.164+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efde403e960 con 0x7efdec15d2c0 2024-02-02T13:28:50.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.164+0000 7efde3400640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.165+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efde40ab580 con 0x7efdec15d2c0 2024-02-02T13:28:50.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.165+0000 7efdeb400640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.165+0000 7efdeb400640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 0x7efdc406c8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efddc000f60 tx=0x7efddc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.169+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efdec0c6bb0 con 0x7efdec15d2c0 2024-02-02T13:28:50.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.396+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdec0c6bb0 con 0x7efdec15d2c0 2024-02-02T13:28:50.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.397+0000 7efde3400640 1 -- 172.21.15.91:0/897731115 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7efdec0c6bb0 con 0x7efdec15d2c0 2024-02-02T13:28:50.399 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:50.399 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:28:50.399 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 msgr2=0x7efdc406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 0x7efdc406c8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efddc000f60 tx=0x7efddc009040 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 msgr2=0x7efdec0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efde4009e50 tx=0x7efde4004a40 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 shutdown_connections 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdec06cd50 0x7efdec0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdec15d2c0 0x7efdec0b0c70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 --2- 172.21.15.91:0/897731115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdec06c820 0x7efdec153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 >> 172.21.15.91:0/897731115 conn(0x7efdec0cea90 msgr2=0x7efdec0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:50.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 shutdown_connections 2024-02-02T13:28:50.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.400+0000 7efdf0c00640 1 -- 172.21.15.91:0/897731115 wait complete. 2024-02-02T13:28:50.421 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:28:50.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.626+0000 7f213b000640 1 Processor -- start 2024-02-02T13:28:50.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.626+0000 7f213b000640 1 -- start start 2024-02-02T13:28:50.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.627+0000 7f213b000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f213415a010 0x7f21340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.627+0000 7f213b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.627+0000 7f213b000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21340cc7c0 con 0x7f21340ce130 2024-02-02T13:28:50.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.627+0000 7f213b000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340b10d0 con 0x7f213415a010 2024-02-02T13:28:50.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.627+0000 7f213b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21340ce930 con 0x7f21340b0700 2024-02-02T13:28:50.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.628+0000 7f2139000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.628+0000 7f2133e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f213415a010 0x7f21340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.628+0000 7f212b200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21340ce130 0x7f21340ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:60510/0 (socket says 172.21.15.91:60510) 2024-02-02T13:28:50.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.628+0000 7f2139000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:53960/0 (socket says 172.21.15.91:53960) 2024-02-02T13:28:50.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.628+0000 7f212b200640 1 -- 172.21.15.91:0/2413967985 learned_addr learned my addr 172.21.15.91:0/2413967985 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:50.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.629+0000 7f2139000640 1 -- 172.21.15.91:0/2413967985 >> v2:172.21.15.113:3300/0 conn(0x7f213415a010 msgr2=0x7f21340c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:50.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.629+0000 7f2139000640 1 --2- 172.21.15.91:0/2413967985 >> v2:172.21.15.113:3300/0 conn(0x7f213415a010 0x7f21340c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.629+0000 7f2139000640 1 -- 172.21.15.91:0/2413967985 >> v1:172.21.15.113:6789/0 conn(0x7f21340ce130 legacy=0x7f21340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.629+0000 7f2139000640 1 -- 172.21.15.91:0/2413967985 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340cd640 con 0x7f21340b0700 2024-02-02T13:28:50.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.630+0000 7f2139000640 1 --2- 172.21.15.91:0/2413967985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f212c009d90 tx=0x7f212c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5285655e1da7268 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f2133400640 1 -- 172.21.15.91:0/2413967985 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c02cc50 con 0x7f21340b0700 2024-02-02T13:28:50.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f2133400640 1 -- 172.21.15.91:0/2413967985 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c02cdb0 con 0x7f21340b0700 2024-02-02T13:28:50.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f2133400640 1 -- 172.21.15.91:0/2413967985 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c034660 con 0x7f21340b0700 2024-02-02T13:28:50.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 -- 172.21.15.91:0/2413967985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 msgr2=0x7f21340b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 --2- 172.21.15.91:0/2413967985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f212c009d90 tx=0x7f212c02bea0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 -- 172.21.15.91:0/2413967985 shutdown_connections 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 --2- 172.21.15.91:0/2413967985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21340b0700 0x7f21340b0b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 --2- 172.21.15.91:0/2413967985 >> v2:172.21.15.113:3300/0 conn(0x7f213415a010 0x7f21340c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 -- 172.21.15.91:0/2413967985 >> 172.21.15.91:0/2413967985 conn(0x7f21340cec80 msgr2=0x7f21340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 -- 172.21.15.91:0/2413967985 shutdown_connections 2024-02-02T13:28:50.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.631+0000 7f213b000640 1 -- 172.21.15.91:0/2413967985 wait complete. 2024-02-02T13:28:50.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.633+0000 7f213b000640 1 Processor -- start 2024-02-02T13:28:50.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.633+0000 7f213b000640 1 -- start start 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21340b0700 0x7f21341501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f213415a010 0x7f21340aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b3db0 con 0x7f21340ce130 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b31a0 con 0x7f21340b0700 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f213b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b3480 con 0x7f213415a010 2024-02-02T13:28:50.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f212b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f212b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48884/0 (socket says 172.21.15.91:48884) 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.634+0000 7f212b200640 1 -- 172.21.15.91:0/3737352358 learned_addr learned my addr 172.21.15.91:0/3737352358 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f2139000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f213415a010 0x7f21340aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 -- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f213415a010 msgr2=0x7f21340aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f213415a010 0x7f21340aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 -- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21340b0700 msgr2=0x7f21341501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21340b0700 0x7f21341501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 -- 172.21.15.91:0/3737352358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c009a70 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f212b200640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c4a0 tx=0x7f211c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c005ce0 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c005e40 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c007270 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.635+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21340b3760 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.636+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21341506e0 con 0x7f21340ce130 2024-02-02T13:28:50.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.637+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21340b4380 con 0x7f21340ce130 2024-02-02T13:28:50.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.637+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f211c007490 con 0x7f21340ce130 2024-02-02T13:28:50.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.638+0000 7f2130c00640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 0x7f210c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:50.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.638+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f211c087e50 con 0x7f21340ce130 2024-02-02T13:28:50.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.639+0000 7f2133e00640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 0x7f210c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:50.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.639+0000 7f2133e00640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 0x7f210c06c8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2124006a90 tx=0x7f2124006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:50.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.643+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f21340b4380 con 0x7f21340ce130 2024-02-02T13:28:50.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.832+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21340b4380 con 0x7f21340ce130 2024-02-02T13:28:50.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.834+0000 7f2130c00640 1 -- 172.21.15.91:0/3737352358 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f21340b4380 con 0x7f21340ce130 2024-02-02T13:28:50.836 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:50.836 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":74,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:28:49.484105+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 msgr2=0x7f210c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 0x7f210c06c8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2124006a90 tx=0x7f2124006a20 comp rx=0 tx=0).stop 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 msgr2=0x7f21340ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c4a0 tx=0x7f211c00c960 comp rx=0 tx=0).stop 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 shutdown_connections 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f210c06a430 0x7f210c06c8f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f213415a010 0x7f21340aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21340ce130 0x7f21340ae6e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 --2- 172.21.15.91:0/3737352358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21340b0700 0x7f21341501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:50.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 >> 172.21.15.91:0/3737352358 conn(0x7f21340cec80 msgr2=0x7f213415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:50.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 shutdown_connections 2024-02-02T13:28:50.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:50.837+0000 7f213b000640 1 -- 172.21.15.91:0/3737352358 wait complete. 2024-02-02T13:28:50.856 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:28:50.857 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:28:51.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.063+0000 7fb21b000640 1 Processor -- start 2024-02-02T13:28:51.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.063+0000 7fb21b000640 1 -- start start 2024-02-02T13:28:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21b000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb21415cee0 0x7fb2140b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21b000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2140b6990 0x7fb2140cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21b000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2140b7d40 con 0x7fb2140ce000 2024-02-02T13:28:51.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21b000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2140ce400 con 0x7fb21415cee0 2024-02-02T13:28:51.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21b000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2140b81d0 con 0x7fb2140b6990 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb219c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb21415cee0 0x7fb2140b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21a600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb2140ce000 0x7fb2140b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54798/0 (socket says 172.21.15.91:54798) 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.065+0000 7fb21a600640 1 -- 172.21.15.91:0/3154815412 learned_addr learned my addr 172.21.15.91:0/3154815412 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb219200640 1 --2- 172.21.15.91:0/3154815412 >> v2:172.21.15.113:3300/0 conn(0x7fb2140b6990 0x7fb2140cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb219c00640 1 -- 172.21.15.91:0/3154815412 >> v2:172.21.15.113:3300/0 conn(0x7fb2140b6990 msgr2=0x7fb2140cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb219c00640 1 --2- 172.21.15.91:0/3154815412 >> v2:172.21.15.113:3300/0 conn(0x7fb2140b6990 0x7fb2140cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb219c00640 1 -- 172.21.15.91:0/3154815412 >> v1:172.21.15.91:6789/0 conn(0x7fb2140ce000 legacy=0x7fb2140b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb213e00640 1 -- 172.21.15.91:0/3154815412 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17347456 0 0) 0x7fb2140b7d40 con 0x7fb2140ce000 2024-02-02T13:28:51.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.066+0000 7fb219c00640 1 -- 172.21.15.91:0/3154815412 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2140b7130 con 0x7fb21415cee0 2024-02-02T13:28:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.067+0000 7fb219c00640 1 --2- 172.21.15.91:0/3154815412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140b8990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb2140d1080 tx=0x7fb20402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf202799f424e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb213e00640 1 -- 172.21.15.91:0/3154815412 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204031830 con 0x7fb21415cee0 2024-02-02T13:28:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb213e00640 1 -- 172.21.15.91:0/3154815412 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204031990 con 0x7fb21415cee0 2024-02-02T13:28:51.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb213e00640 1 -- 172.21.15.91:0/3154815412 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204031d20 con 0x7fb21415cee0 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 -- 172.21.15.91:0/3154815412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 msgr2=0x7fb2140b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 --2- 172.21.15.91:0/3154815412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140b8990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb2140d1080 tx=0x7fb20402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 -- 172.21.15.91:0/3154815412 shutdown_connections 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 --2- 172.21.15.91:0/3154815412 >> v2:172.21.15.113:3300/0 conn(0x7fb2140b6990 0x7fb2140cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 --2- 172.21.15.91:0/3154815412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140b8990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 -- 172.21.15.91:0/3154815412 >> 172.21.15.91:0/3154815412 conn(0x7fb214070690 msgr2=0x7fb214070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.068+0000 7fb21b000640 1 -- 172.21.15.91:0/3154815412 shutdown_connections 2024-02-02T13:28:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.069+0000 7fb21b000640 1 -- 172.21.15.91:0/3154815412 wait complete. 2024-02-02T13:28:51.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.070+0000 7fb21b000640 1 Processor -- start 2024-02-02T13:28:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.070+0000 7fb21b000640 1 -- start start 2024-02-02T13:28:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140b6990 0x7fb21415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140ce000 0x7fb2140d2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f42a0 con 0x7fb21415cee0 2024-02-02T13:28:51.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f3670 con 0x7fb2140b6990 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.071+0000 7fb21b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f3970 con 0x7fb2140ce000 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48904/0 (socket says 172.21.15.91:48904) 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 -- 172.21.15.91:0/3147149114 learned_addr learned my addr 172.21.15.91:0/3147149114 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb219200640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140ce000 0x7fb2140d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb219c00640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140b6990 0x7fb21415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 -- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140ce000 msgr2=0x7fb2140d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140ce000 0x7fb2140d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 -- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140b6990 msgr2=0x7fb21415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140b6990 0x7fb21415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 -- 172.21.15.91:0/3147149114 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009400 con 0x7fb21415cee0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.072+0000 7fb21a600640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c7e0 tx=0x7fb20c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.073+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00a640 con 0x7fb21415cee0 2024-02-02T13:28:51.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.073+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c00a7c0 con 0x7fb21415cee0 2024-02-02T13:28:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.073+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00ab70 con 0x7fb21415cee0 2024-02-02T13:28:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.073+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2141f3c50 con 0x7fb21415cee0 2024-02-02T13:28:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.073+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21415ffe0 con 0x7fb21415cee0 2024-02-02T13:28:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.074+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2141f4410 con 0x7fb21415cee0 2024-02-02T13:28:51.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.075+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb20c00acd0 con 0x7fb21415cee0 2024-02-02T13:28:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.076+0000 7fb211600640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 0x7fb1ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.076+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb20c087de0 con 0x7fb21415cee0 2024-02-02T13:28:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.076+0000 7fb219c00640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 0x7fb1ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.077+0000 7fb219c00640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 0x7fb1ec06c920 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb204036000 tx=0x7fb20403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.080+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb20c0533c0 con 0x7fb21415cee0 2024-02-02T13:28:51.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.308+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb21404dff0 con 0x7fb21415cee0 2024-02-02T13:28:51.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.309+0000 7fb211600640 1 -- 172.21.15.91:0/3147149114 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fb20c018090 con 0x7fb21415cee0 2024-02-02T13:28:51.311 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:51.311 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:28:51.311 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:28:51.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.313+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 msgr2=0x7fb1ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.313+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 0x7fb1ec06c920 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb204036000 tx=0x7fb20403c040 comp rx=0 tx=0).stop 2024-02-02T13:28:51.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.313+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 msgr2=0x7fb2140d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.313+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c7e0 tx=0x7fb20c00cca0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 shutdown_connections 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1ec06a460 0x7fb1ec06c920 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140ce000 0x7fb2140d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb21415cee0 0x7fb2140d2680 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 --2- 172.21.15.91:0/3147149114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140b6990 0x7fb21415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 >> 172.21.15.91:0/3147149114 conn(0x7fb214070690 msgr2=0x7fb2140c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 shutdown_connections 2024-02-02T13:28:51.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.314+0000 7fb21b000640 1 -- 172.21.15.91:0/3147149114 wait complete. 2024-02-02T13:28:51.333 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:28:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.545+0000 7f50f3800640 1 Processor -- start 2024-02-02T13:28:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.545+0000 7f50f3800640 1 -- start start 2024-02-02T13:28:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.546+0000 7f50f3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50ec15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f3800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec070f80 con 0x7f50ec0b0e10 2024-02-02T13:28:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f3800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec06c600 con 0x7f50ec06f4f0 2024-02-02T13:28:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec152370 con 0x7f50ec151f90 2024-02-02T13:28:51.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f1800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f50ec0b0e10 0x7f50ec0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54812/0 (socket says 172.21.15.91:54812) 2024-02-02T13:28:51.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f1800640 1 -- 172.21.15.91:0/1207168071 learned_addr learned my addr 172.21.15.91:0/1207168071 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:51.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50f0e00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50ec15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064908799 0 0) 0x7f50ec070f80 con 0x7f50ec0b0e10 2024-02-02T13:28:51.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.547+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50cc003520 con 0x7f50ec0b0e10 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284234538 0 0) 0x7f50cc003520 con 0x7f50ec0b0e10 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 msgr2=0x7f50ec15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50ec15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 >> v1:172.21.15.113:6789/0 conn(0x7f50ec06f4f0 legacy=0x7f50ec0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0cb010 con 0x7f50ec0b0e10 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f50e4001670 con 0x7f50ec0b0e10 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 >> v1:172.21.15.91:6789/0 conn(0x7f50ec0b0e10 legacy=0x7f50ec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50cc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50cc003f70 0x7f50cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50cc004d00 0x7f50cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec152370 con 0x7f50cc003f70 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc00bac0 con 0x7f50cc004d00 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc00bc80 con 0x7f50ec151f90 2024-02-02T13:28:51.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.548+0000 7f50ebe00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50cc003f70 0x7f50cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50f1800640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50cc004d00 0x7f50cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50ebe00640 1 -- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 msgr2=0x7f50cc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50f0e00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50cc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50ebe00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50ec151f90 0x7f50cc0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50ebe00640 1 -- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50cc004d00 msgr2=0x7f50cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50ebe00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50cc004d00 0x7f50cc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.549+0000 7f50ebe00640 1 -- 172.21.15.91:0/1207168071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc003520 con 0x7f50cc003f70 2024-02-02T13:28:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.550+0000 7f50ebe00640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50cc003f70 0x7f50cc0047c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f50dc001e50 tx=0x7f50cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a2614624ba4965 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.550+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003b70 con 0x7f50cc003f70 2024-02-02T13:28:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.550+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc003cd0 con 0x7f50cc003f70 2024-02-02T13:28:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.550+0000 7f50eb400640 1 -- 172.21.15.91:0/1207168071 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc005310 con 0x7f50cc003f70 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 -- 172.21.15.91:0/1207168071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50cc003f70 msgr2=0x7f50cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 --2- 172.21.15.91:0/1207168071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50cc003f70 0x7f50cc0047c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f50dc001e50 tx=0x7f50cc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f1800640 1 -- 172.21.15.91:0/1207168071 reap_dead start 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 -- 172.21.15.91:0/1207168071 shutdown_connections 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 -- 172.21.15.91:0/1207168071 >> 172.21.15.91:0/1207168071 conn(0x7f50ec0cec80 msgr2=0x7f50ec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 -- 172.21.15.91:0/1207168071 shutdown_connections 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.551+0000 7f50f3800640 1 -- 172.21.15.91:0/1207168071 wait complete. 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.553+0000 7f50f3800640 1 Processor -- start 2024-02-02T13:28:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.553+0000 7f50f3800640 1 -- start start 2024-02-02T13:28:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50cc003f70 0x7f50ec154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50ec1f2aa0 0x7f50ec0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15ae10 con 0x7f50ec15a450 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec1547f0 con 0x7f50ec1f2aa0 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec154ad0 con 0x7f50cc003f70 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48932/0 (socket says 172.21.15.91:48932) 2024-02-02T13:28:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 -- 172.21.15.91:0/4074704017 learned_addr learned my addr 172.21.15.91:0/4074704017 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 -- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50cc003f70 msgr2=0x7f50ec154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50cc003f70 0x7f50ec154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 -- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50ec1f2aa0 msgr2=0x7f50ec0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50ec1f2aa0 0x7f50ec0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.554+0000 7f50f1800640 1 -- 172.21.15.91:0/4074704017 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec154db0 con 0x7f50ec15a450 2024-02-02T13:28:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50f1800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0c5c50 tx=0x7f50e400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e40074e0 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4007660 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4004a20 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec155090 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.555+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec0c2f80 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.556+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec066cf0 con 0x7f50ec15a450 2024-02-02T13:28:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.557+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f50e4004b80 con 0x7f50ec15a450 2024-02-02T13:28:51.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.559+0000 7f50e8c00640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 0x7f50c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:51.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.559+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50e40885d0 con 0x7f50ec15a450 2024-02-02T13:28:51.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.559+0000 7f50f0e00640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 0x7f50c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:51.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.559+0000 7f50f0e00640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 0x7f50c406c920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009950 tx=0x7f50d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:51.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.563+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f50e4053ba0 con 0x7f50ec15a450 2024-02-02T13:28:51.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.789+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50ec1554e0 con 0x7f50ec15a450 2024-02-02T13:28:51.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.790+0000 7f50e8c00640 1 -- 172.21.15.91:0/4074704017 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f50e40571c0 con 0x7f50ec15a450 2024-02-02T13:28:51.795 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:51.795 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:27:26.089350+0000","modified":"2024-02-02T13:27:29.118027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2506136501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2506136501},{"type":"v1","addr":"172.21.15.91:6837","nonce":2506136501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 32 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 msgr2=0x7f50c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 0x7f50c406c920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009950 tx=0x7f50d4008040 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 msgr2=0x7f50ec15a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0c5c50 tx=0x7f50e400d9d0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 shutdown_connections 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f50c406a460 0x7f50c406c920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50ec15a450 0x7f50ec15a8d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50ec1f2aa0 0x7f50ec0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 --2- 172.21.15.91:0/4074704017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50cc003f70 0x7f50ec154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 >> 172.21.15.91:0/4074704017 conn(0x7f50ec0cec80 msgr2=0x7f50ec0aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 shutdown_connections 2024-02-02T13:28:51.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:51.793+0000 7f50f3800640 1 -- 172.21.15.91:0/4074704017 wait complete. 2024-02-02T13:28:51.813 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:28:52.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.023+0000 7ff8dda00640 1 Processor -- start 2024-02-02T13:28:52.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.023+0000 7ff8dda00640 1 -- start start 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.024+0000 7ff8dda00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 0x7ff8d80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8dda00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cc7c0 con 0x7ff8d80b0700 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8dda00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80ce930 con 0x7ff8d80ce130 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8dda00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80cd640 con 0x7ff8d815a010 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8d7400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff8d80b0700 0x7ff8d806ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54814/0 (socket says 172.21.15.91:54814) 2024-02-02T13:28:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8d7400640 1 -- 172.21.15.91:0/1036384166 learned_addr learned my addr 172.21.15.91:0/1036384166 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:52.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156096627 0 0) 0x7ff8d80cc7c0 con 0x7ff8d80b0700 2024-02-02T13:28:52.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8d6a00640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 0x7ff8d80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.025+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b8005730 con 0x7ff8d80b0700 2024-02-02T13:28:52.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463197099 0 0) 0x7ff8b8005730 con 0x7ff8d80b0700 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 msgr2=0x7ff8d80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 0x7ff8d80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 >> v1:172.21.15.113:6789/0 conn(0x7ff8d80ce130 legacy=0x7ff8d80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d806c890 con 0x7ff8d80b0700 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff8d00019f0 con 0x7ff8d80b0700 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 >> v1:172.21.15.91:6789/0 conn(0x7ff8d80b0700 legacy=0x7ff8d806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b8005bb0 0x7ff8b8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 0x7ff8b80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8b8006f10 0x7ff8b8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80cd640 con 0x7ff8b8006f10 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b800dc70 con 0x7ff8b8005bb0 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b800de30 con 0x7ff8d815a010 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.026+0000 7ff8d7400640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8b8006f10 0x7ff8b8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.027+0000 7ff8d7400640 1 -- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 msgr2=0x7ff8b80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.027+0000 7ff8d7400640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8d815a010 0x7ff8b80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.027+0000 7ff8d7400640 1 -- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b8005bb0 msgr2=0x7ff8b8005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.027+0000 7ff8d7400640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b8005bb0 0x7ff8b8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.027+0000 7ff8d7400640 1 -- 172.21.15.91:0/1036384166 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8005730 con 0x7ff8b8006f10 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8d7400640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8b8006f10 0x7ff8b8007390 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0002230 tx=0x7ff8b8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c73b35dce3d4ec9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0034630 con 0x7ff8b8006f10 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d002f070 con 0x7ff8b8006f10 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8d5600640 1 -- 172.21.15.91:0/1036384166 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0034a10 con 0x7ff8b8006f10 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8dda00640 1 -- 172.21.15.91:0/1036384166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8b8006f10 msgr2=0x7ff8b8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8dda00640 1 --2- 172.21.15.91:0/1036384166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8b8006f10 0x7ff8b8007390 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0002230 tx=0x7ff8b8007a00 comp rx=0 tx=0).stop 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8d7400640 1 -- 172.21.15.91:0/1036384166 reap_dead start 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.028+0000 7ff8dda00640 1 -- 172.21.15.91:0/1036384166 shutdown_connections 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.029+0000 7ff8dda00640 1 -- 172.21.15.91:0/1036384166 >> 172.21.15.91:0/1036384166 conn(0x7ff8d80cec80 msgr2=0x7ff8d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.029+0000 7ff8dda00640 1 -- 172.21.15.91:0/1036384166 shutdown_connections 2024-02-02T13:28:52.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.029+0000 7ff8dda00640 1 -- 172.21.15.91:0/1036384166 wait complete. 2024-02-02T13:28:52.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.030+0000 7ff8dda00640 1 Processor -- start 2024-02-02T13:28:52.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.030+0000 7ff8dda00640 1 -- start start 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8b8006f10 0x7ff8d81f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b2230 0x7ff8d80b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8071080 con 0x7ff8d8070690 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81f27a0 con 0x7ff8d80b2230 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.031+0000 7ff8dda00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b2d20 con 0x7ff8b8006f10 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48950/0 (socket says 172.21.15.91:48950) 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 -- 172.21.15.91:0/4011097664 learned_addr learned my addr 172.21.15.91:0/4011097664 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 -- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8b8006f10 msgr2=0x7ff8d81f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8b8006f10 0x7ff8d81f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 -- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b2230 msgr2=0x7ff8d80b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b2230 0x7ff8d80b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 -- 172.21.15.91:0/4011097664 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80b3000 con 0x7ff8d8070690 2024-02-02T13:28:52.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.032+0000 7ff8d7400640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff8d002ca70 tx=0x7ff8d0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.033+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0034840 con 0x7ff8d8070690 2024-02-02T13:28:52.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.033+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d0001250 con 0x7ff8d8070690 2024-02-02T13:28:52.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.033+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0034c30 con 0x7ff8d8070690 2024-02-02T13:28:52.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.033+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d80b32e0 con 0x7ff8d8070690 2024-02-02T13:28:52.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.033+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80c4030 con 0x7ff8d8070690 2024-02-02T13:28:52.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.034+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d81f2c80 con 0x7ff8d8070690 2024-02-02T13:28:52.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.035+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff8d0031030 con 0x7ff8d8070690 2024-02-02T13:28:52.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.036+0000 7ff8cea00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 0x7ff8b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.036+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8d00aad90 con 0x7ff8d8070690 2024-02-02T13:28:52.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.036+0000 7ff8d6a00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 0x7ff8b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.037+0000 7ff8d6a00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 0x7ff8b006c8f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff8d815dd00 tx=0x7ff8c8009500 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.040+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8d81f2c80 con 0x7ff8d8070690 2024-02-02T13:28:52.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.230+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8d81f2c80 con 0x7ff8d8070690 2024-02-02T13:28:52.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.233+0000 7ff8cea00640 1 -- 172.21.15.91:0/4011097664 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16097 (secure 0 0 0) 0x7ff8d81f2c80 con 0x7ff8d8070690 2024-02-02T13:28:52.235 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:52.236 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":74,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:28:49.484105+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:27:23.838598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-02T13:27:25.123347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 msgr2=0x7ff8b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 0x7ff8b006c8f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff8d815dd00 tx=0x7ff8c8009500 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 msgr2=0x7ff8d8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff8d002ca70 tx=0x7ff8d0001cb0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 shutdown_connections 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8b006a430 0x7ff8b006c8f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8d8070690 0x7ff8d8070b40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b2230 0x7ff8d80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 --2- 172.21.15.91:0/4011097664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8b8006f10 0x7ff8d81f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 >> 172.21.15.91:0/4011097664 conn(0x7ff8d80cec80 msgr2=0x7ff8d815e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:52.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 shutdown_connections 2024-02-02T13:28:52.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.236+0000 7ff8dda00640 1 -- 172.21.15.91:0/4011097664 wait complete. 2024-02-02T13:28:52.256 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:28:52.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.467+0000 7ff6de200640 1 Processor -- start 2024-02-02T13:28:52.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.467+0000 7ff6de200640 1 -- start start 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6de200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 0x7ff6d80d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6de200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6d8069ff0 0x7ff6d806a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6de200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d815db10 con 0x7ff6d806a970 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6de200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d80d1d40 con 0x7ff6d8069ff0 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6de200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d80cfca0 con 0x7ff6d815d730 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6d7e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff6d806a970 0x7ff6d81611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54826/0 (socket says 172.21.15.91:54826) 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6d7e00640 1 -- 172.21.15.91:0/1022437812 learned_addr learned my addr 172.21.15.91:0/1022437812 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.469+0000 7ff6ce800640 1 --2- 172.21.15.91:0/1022437812 >> v2:172.21.15.91:3300/0 conn(0x7ff6d8069ff0 0x7ff6d806a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6d7400640 1 --2- 172.21.15.91:0/1022437812 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 0x7ff6d80d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6ce800640 1 -- 172.21.15.91:0/1022437812 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 msgr2=0x7ff6d80d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6ce800640 1 --2- 172.21.15.91:0/1022437812 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 0x7ff6d80d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6ce800640 1 -- 172.21.15.91:0/1022437812 >> v1:172.21.15.91:6789/0 conn(0x7ff6d806a970 legacy=0x7ff6d81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6ce800640 1 -- 172.21.15.91:0/1022437812 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80cb490 con 0x7ff6d8069ff0 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6d6a00640 1 -- 172.21.15.91:0/1022437812 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327423728 0 0) 0x7ff6d815db10 con 0x7ff6d806a970 2024-02-02T13:28:52.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.470+0000 7ff6d7400640 1 --2- 172.21.15.91:0/1022437812 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 0x7ff6d80d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:28:52.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.471+0000 7ff6ce800640 1 --2- 172.21.15.91:0/1022437812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d8069ff0 0x7ff6d806a430 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8066800 tx=0x7ff6c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17735ce04fe0141a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.471+0000 7ff6d6a00640 1 -- 172.21.15.91:0/1022437812 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0031830 con 0x7ff6d8069ff0 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.471+0000 7ff6d6a00640 1 -- 172.21.15.91:0/1022437812 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0031990 con 0x7ff6d8069ff0 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.471+0000 7ff6d6a00640 1 -- 172.21.15.91:0/1022437812 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0031d20 con 0x7ff6d8069ff0 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 -- 172.21.15.91:0/1022437812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d8069ff0 msgr2=0x7ff6d806a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 --2- 172.21.15.91:0/1022437812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d8069ff0 0x7ff6d806a430 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8066800 tx=0x7ff6c002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 -- 172.21.15.91:0/1022437812 shutdown_connections 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 --2- 172.21.15.91:0/1022437812 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d8069ff0 0x7ff6d806a430 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 --2- 172.21.15.91:0/1022437812 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d730 0x7ff6d80d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 -- 172.21.15.91:0/1022437812 >> 172.21.15.91:0/1022437812 conn(0x7ff6d809a110 msgr2=0x7ff6d80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 -- 172.21.15.91:0/1022437812 shutdown_connections 2024-02-02T13:28:52.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.472+0000 7ff6de200640 1 -- 172.21.15.91:0/1022437812 wait complete. 2024-02-02T13:28:52.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.474+0000 7ff6de200640 1 Processor -- start 2024-02-02T13:28:52.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.474+0000 7ff6de200640 1 -- start start 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069ff0 0x7ff6d8158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d806a970 0x7ff6d80b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8158e50 con 0x7ff6d815d730 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80b4870 con 0x7ff6d806a970 2024-02-02T13:28:52.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6de200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80b3fd0 con 0x7ff6d8069ff0 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48964/0 (socket says 172.21.15.91:48964) 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.475+0000 7ff6d7e00640 1 -- 172.21.15.91:0/3971629642 learned_addr learned my addr 172.21.15.91:0/3971629642 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7400640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069ff0 0x7ff6d8158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6ce800640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d806a970 0x7ff6d80b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 -- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069ff0 msgr2=0x7ff6d8158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069ff0 0x7ff6d8158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 -- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d806a970 msgr2=0x7ff6d80b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d806a970 0x7ff6d80b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 -- 172.21.15.91:0/3971629642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0009400 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6d7e00640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c780 tx=0x7ff6d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000ce90 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000a640 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d80b42b0 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.476+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000a9d0 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.477+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d80c5b20 con 0x7ff6d815d730 2024-02-02T13:28:52.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.478+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d80c5d10 con 0x7ff6d815d730 2024-02-02T13:28:52.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.479+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff6d000ab30 con 0x7ff6d815d730 2024-02-02T13:28:52.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.479+0000 7ff6cfe00640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 0x7ff6b006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.479+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6d0087d90 con 0x7ff6d815d730 2024-02-02T13:28:52.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.480+0000 7ff6d7400640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 0x7ff6b006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.480+0000 7ff6d7400640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 0x7ff6b006c920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.484+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff6d0053370 con 0x7ff6d815d730 2024-02-02T13:28:52.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.710+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6d80d0420 con 0x7ff6d815d730 2024-02-02T13:28:52.713 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:28:52.712+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:28:52.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.732+0000 7ff6cfe00640 1 -- 172.21.15.91:0/3971629642 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff6d0018090 con 0x7ff6d815d730 2024-02-02T13:28:52.734 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:28:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 msgr2=0x7ff6b006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 0x7ff6b006c920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8008040 comp rx=0 tx=0).stop 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 msgr2=0x7ff6d80b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c780 tx=0x7ff6d000cc40 comp rx=0 tx=0).stop 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 shutdown_connections 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6b006a460 0x7ff6b006c920 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6d815d730 0x7ff6d80b32c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d806a970 0x7ff6d80b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 --2- 172.21.15.91:0/3971629642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069ff0 0x7ff6d8158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 >> 172.21.15.91:0/3971629642 conn(0x7ff6d809a110 msgr2=0x7ff6d806f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:52.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 shutdown_connections 2024-02-02T13:28:52.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.735+0000 7ff6de200640 1 -- 172.21.15.91:0/3971629642 wait complete. 2024-02-02T13:28:52.760 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:28:52.826 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:28:52.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.940+0000 7f4c04a00640 1 Processor -- start 2024-02-02T13:28:52.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.940+0000 7f4c04a00640 1 -- start start 2024-02-02T13:28:52.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.941+0000 7f4c04a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000cc600 0x7f4c000cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.941+0000 7f4c04a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4c000ac8d0 0x7f4c000acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.941+0000 7f4c04a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c000ad5c0 con 0x7f4c0015e110 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.941+0000 7f4c04a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c000ad8a0 con 0x7f4c000ac8d0 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.941+0000 7f4c04a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000ccfc0 con 0x7f4c000cc600 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4c000ac8d0 0x7f4c000acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4c000ac8d0 0x7f4c000acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48988/0 (socket says 172.21.15.91:48988) 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 -- 172.21.15.91:0/1194355580 learned_addr learned my addr 172.21.15.91:0/1194355580 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bf5c00640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000cc600 0x7f4c000cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 -- 172.21.15.91:0/1194355580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000cc600 msgr2=0x7f4c000cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000cc600 0x7f4c000cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 -- 172.21.15.91:0/1194355580 >> v1:172.21.15.113:6789/0 conn(0x7f4c0015e110 legacy=0x7f4c000cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:52.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.942+0000 7f4bff200640 1 -- 172.21.15.91:0/1194355580 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000cae90 con 0x7f4c000ac8d0 2024-02-02T13:28:52.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.943+0000 7f4bff200640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c000ac8d0 0x7f4c000acd60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009e50 tx=0x7f4bf802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ee8d7edf60b116 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4bfde00640 1 -- 172.21.15.91:0/1194355580 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf802c9f0 con 0x7f4c000ac8d0 2024-02-02T13:28:52.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4bfde00640 1 -- 172.21.15.91:0/1194355580 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf802cb50 con 0x7f4c000ac8d0 2024-02-02T13:28:52.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4bfde00640 1 -- 172.21.15.91:0/1194355580 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80343f0 con 0x7f4c000ac8d0 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 -- 172.21.15.91:0/1194355580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c000ac8d0 msgr2=0x7f4c000acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c000ac8d0 0x7f4c000acd60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009e50 tx=0x7f4bf802be50 comp rx=0 tx=0).stop 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 -- 172.21.15.91:0/1194355580 shutdown_connections 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c000ac8d0 0x7f4c000acd60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 --2- 172.21.15.91:0/1194355580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000cc600 0x7f4c000cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 -- 172.21.15.91:0/1194355580 >> 172.21.15.91:0/1194355580 conn(0x7f4c000b0560 msgr2=0x7f4c000b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 -- 172.21.15.91:0/1194355580 shutdown_connections 2024-02-02T13:28:52.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.944+0000 7f4c04a00640 1 -- 172.21.15.91:0/1194355580 wait complete. 2024-02-02T13:28:52.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.946+0000 7f4c04a00640 1 Processor -- start 2024-02-02T13:28:52.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.946+0000 7f4c04a00640 1 -- start start 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.947+0000 7f4c04a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 0x7f4c00153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.947+0000 7f4c04a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000cc600 0x7f4c0015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.947+0000 7f4c04a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4c04a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000701f0 con 0x7f4c0015e110 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4c04a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0006f360 con 0x7f4c000cc600 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4c04a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0006f570 con 0x7f4c000ac8d0 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:48994/0 (socket says 172.21.15.91:48994) 2024-02-02T13:28:52.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 -- 172.21.15.91:0/87806062 learned_addr learned my addr 172.21.15.91:0/87806062 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bf5c00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000cc600 0x7f4c0015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bfe800640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 0x7f4c00153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 -- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 msgr2=0x7f4c00153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 0x7f4c00153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 -- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000cc600 msgr2=0x7f4c0015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000cc600 0x7f4c0015f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bff200640 1 -- 172.21.15.91:0/87806062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009b30 con 0x7f4c0015e110 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.948+0000 7f4bfe800640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 0x7f4c00153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:52.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4bff200640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8036040 tx=0x7f4bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf803e450 con 0x7f4c0015e110 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf803e5d0 con 0x7f4c0015e110 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf803e980 con 0x7f4c0015e110 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0006f850 con 0x7f4c0015e110 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.949+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0006fe10 con 0x7f4c0015e110 2024-02-02T13:28:52.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.950+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c000cb130 con 0x7f4c0015e110 2024-02-02T13:28:52.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.951+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4bf803eae0 con 0x7f4c0015e110 2024-02-02T13:28:52.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.952+0000 7f4bf7400640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 0x7f4bd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:52.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.952+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4bf80ab820 con 0x7f4c0015e110 2024-02-02T13:28:52.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.953+0000 7f4bfe800640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 0x7f4bd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:52.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.953+0000 7f4bfe800640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 0x7f4bd806c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000a830 tx=0x7f4bf0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:52.957+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4bf8076e50 con 0x7f4c0015e110 2024-02-02T13:28:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.185+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4c000c70a0 con 0x7f4c0015e110 2024-02-02T13:28:53.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.737+0000 7f4bf7400640 1 -- 172.21.15.91:0/87806062 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bf807a470 con 0x7f4c0015e110 2024-02-02T13:28:53.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 msgr2=0x7f4bd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 0x7f4bd806c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000a830 tx=0x7f4bf0008040 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 msgr2=0x7f4c0015fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8036040 tx=0x7f4bf8004a40 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 shutdown_connections 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4bd806a430 0x7f4bd806c8f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c0015e110 0x7f4c0015fb20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000cc600 0x7f4c0015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 --2- 172.21.15.91:0/87806062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c000ac8d0 0x7f4c00153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 >> 172.21.15.91:0/87806062 conn(0x7f4c000b0560 msgr2=0x7f4c00073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 shutdown_connections 2024-02-02T13:28:53.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.741+0000 7f4c04a00640 1 -- 172.21.15.91:0/87806062 wait complete. 2024-02-02T13:28:53.762 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:28:53.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.974+0000 7f6b91a00640 1 Processor -- start 2024-02-02T13:28:53.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.974+0000 7f6b91a00640 1 -- start start 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.975+0000 7f6b91a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b8c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.975+0000 7f6b91a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c070f80 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b91a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c06c600 con 0x7f6b8c0b0e10 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b91a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c152370 con 0x7f6b8c151f90 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b83e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6b8c06f4f0 0x7f6b8c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54854/0 (socket says 172.21.15.91:54854) 2024-02-02T13:28:53.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b83e00640 1 -- 172.21.15.91:0/78328129 learned_addr learned my addr 172.21.15.91:0/78328129 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:53.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b8aa00640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b8c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375953506 0 0) 0x7f6b8c070f80 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.976+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b6c003520 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023060733 0 0) 0x7f6b6c003520 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 msgr2=0x7f6b8c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b8c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 >> v1:172.21.15.113:6789/0 conn(0x7f6b8c0b0e10 legacy=0x7f6b8c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0cb010 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6b74001670 con 0x7f6b8c06f4f0 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 >> v1:172.21.15.91:6789/0 conn(0x7f6b8c06f4f0 legacy=0x7f6b8c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b6c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b6c003f70 0x7f6b6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b6c004d00 0x7f6b6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c152370 con 0x7f6b6c004d00 2024-02-02T13:28:53.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c00bac0 con 0x7f6b6c003f70 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c00bc80 con 0x7f6b8c151f90 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.977+0000 7f6b8b400640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b6c004d00 0x7f6b6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8aa00640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b6c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b83e00640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b6c003f70 0x7f6b6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8b400640 1 -- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 msgr2=0x7f6b6c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8b400640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b8c151f90 0x7f6b6c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8b400640 1 -- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b6c003f70 msgr2=0x7f6b6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8b400640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b6c003f70 0x7f6b6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.978+0000 7f6b8b400640 1 -- 172.21.15.91:0/78328129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c003520 con 0x7f6b6c004d00 2024-02-02T13:28:53.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.979+0000 7f6b8b400640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b6c004d00 0x7f6b6c005180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c071a20 tx=0x7f6b6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb527f953d2fe3e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:53.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.979+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84003b70 con 0x7f6b6c004d00 2024-02-02T13:28:53.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.979+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b84003cd0 con 0x7f6b6c004d00 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.979+0000 7f6b8a000640 1 -- 172.21.15.91:0/78328129 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84005310 con 0x7f6b6c004d00 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 -- 172.21.15.91:0/78328129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b6c004d00 msgr2=0x7f6b6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 --2- 172.21.15.91:0/78328129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b6c004d00 0x7f6b6c005180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c071a20 tx=0x7f6b6c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b8b400640 1 -- 172.21.15.91:0/78328129 reap_dead start 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 -- 172.21.15.91:0/78328129 shutdown_connections 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 -- 172.21.15.91:0/78328129 >> 172.21.15.91:0/78328129 conn(0x7f6b8c0cec80 msgr2=0x7f6b8c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 -- 172.21.15.91:0/78328129 shutdown_connections 2024-02-02T13:28:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.980+0000 7f6b91a00640 1 -- 172.21.15.91:0/78328129 wait complete. 2024-02-02T13:28:53.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.982+0000 7f6b91a00640 1 Processor -- start 2024-02-02T13:28:53.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.982+0000 7f6b91a00640 1 -- start start 2024-02-02T13:28:53.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b6c004d00 0x7f6b8c0c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 0x7f6b8c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0af5a0 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0aefe0 con 0x7f6b8c155000 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b91a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0af2c0 con 0x7f6b6c004d00 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b8b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b8b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49012/0 (socket says 172.21.15.91:49012) 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b8b400640 1 -- 172.21.15.91:0/750612605 learned_addr learned my addr 172.21.15.91:0/750612605 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b8aa00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b6c004d00 0x7f6b8c0c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b83e00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 0x7f6b8c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.983+0000 7f6b8b400640 1 -- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b6c004d00 msgr2=0x7f6b8c0c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b8b400640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b6c004d00 0x7f6b8c0c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b8b400640 1 -- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 msgr2=0x7f6b8c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b8b400640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 0x7f6b8c0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b8b400640 1 -- 172.21.15.91:0/750612605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c153540 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b83e00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 0x7f6b8c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:53.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b8b400640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b8402c020 tx=0x7f6b840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:53.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8403e070 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b84003e40 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84006d50 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c153820 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.984+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c0aec20 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.985+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c066cf0 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.986+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6b84007030 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.987+0000 7f6b82a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 0x7f6b6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.987+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6b840a9b60 con 0x7f6b8c0c5cc0 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.987+0000 7f6b8aa00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 0x7f6b6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:53.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.988+0000 7f6b8aa00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 0x7f6b6406c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009a10 tx=0x7f6b7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:53.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:53.991+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6b84075100 con 0x7f6b8c0c5cc0 2024-02-02T13:28:54.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.186+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b8c06c7b0 con 0x7f6b8c0c5cc0 2024-02-02T13:28:54.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.749+0000 7f6b82a00640 1 -- 172.21.15.91:0/750612605 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6b84048020 con 0x7f6b8c0c5cc0 2024-02-02T13:28:54.752 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:28:54.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 msgr2=0x7f6b6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 0x7f6b6406c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009a10 tx=0x7f6b7c008040 comp rx=0 tx=0).stop 2024-02-02T13:28:54.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 msgr2=0x7f6b8c14fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b8402c020 tx=0x7f6b840014a0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 shutdown_connections 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b6406a430 0x7f6b6406c8f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b8c0c5cc0 0x7f6b8c14fbc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c155000 0x7f6b8c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 --2- 172.21.15.91:0/750612605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b6c004d00 0x7f6b8c0c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 >> 172.21.15.91:0/750612605 conn(0x7f6b8c0cec80 msgr2=0x7f6b8c0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 shutdown_connections 2024-02-02T13:28:54.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.754+0000 7f6b91a00640 1 -- 172.21.15.91:0/750612605 wait complete. 2024-02-02T13:28:54.776 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:28:54.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.980+0000 7f94d7e00640 1 Processor -- start 2024-02-02T13:28:54.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.981+0000 7f94d7e00640 1 -- start start 2024-02-02T13:28:54.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f94d815aec0 0x7f94d8071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:54.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d806f9f0 con 0x7f94d80b1c00 2024-02-02T13:28:54.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d806b6a0 con 0x7f94d80cb730 2024-02-02T13:28:54.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d806c600 con 0x7f94d815aec0 2024-02-02T13:28:54.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f94d815aec0 0x7f94d8071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f94d80b1c00 0x7f94d80b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54868/0 (socket says 172.21.15.91:54868) 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.982+0000 7f94d7400640 1 -- 172.21.15.91:0/1897865250 learned_addr learned my addr 172.21.15.91:0/1897865250 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.983+0000 7f94d6a00640 1 -- 172.21.15.91:0/1897865250 >> v1:172.21.15.113:6789/0 conn(0x7f94d80cb730 legacy=0x7f94d80cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.983+0000 7f94d6a00640 1 -- 172.21.15.91:0/1897865250 >> v1:172.21.15.91:6789/0 conn(0x7f94d80b1c00 legacy=0x7f94d80b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.983+0000 7f94d6a00640 1 -- 172.21.15.91:0/1897865250 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d806b980 con 0x7f94d815aec0 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.983+0000 7f94d6000640 1 -- 172.21.15.91:0/1897865250 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974008587 0 0) 0x7f94d806f9f0 con 0x7f94d80b1c00 2024-02-02T13:28:54.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.984+0000 7f94d6a00640 1 --2- 172.21.15.91:0/1897865250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d815aec0 0x7f94d8071290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f94c80097a0 tx=0x7f94c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df0aa7baecfd0be server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:54.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.984+0000 7f94d6000640 1 -- 172.21.15.91:0/1897865250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c802e070 con 0x7f94d815aec0 2024-02-02T13:28:54.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.984+0000 7f94d6000640 1 -- 172.21.15.91:0/1897865250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c80315a0 con 0x7f94d815aec0 2024-02-02T13:28:54.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.984+0000 7f94d6000640 1 -- 172.21.15.91:0/1897865250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8031930 con 0x7f94d815aec0 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 -- 172.21.15.91:0/1897865250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d815aec0 msgr2=0x7f94d8071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1897865250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d815aec0 0x7f94d8071290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f94c80097a0 tx=0x7f94c802b840 comp rx=0 tx=0).stop 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 -- 172.21.15.91:0/1897865250 shutdown_connections 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1897865250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d815aec0 0x7f94d8071290 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 -- 172.21.15.91:0/1897865250 >> 172.21.15.91:0/1897865250 conn(0x7f94d80b08e0 msgr2=0x7f94d80b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 -- 172.21.15.91:0/1897865250 shutdown_connections 2024-02-02T13:28:54.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.985+0000 7f94d7e00640 1 -- 172.21.15.91:0/1897865250 wait complete. 2024-02-02T13:28:54.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.987+0000 7f94d7e00640 1 Processor -- start 2024-02-02T13:28:54.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.987+0000 7f94d7e00640 1 -- start start 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d80b1c00 0x7f94d814e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 0x7f94d81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8152e60 con 0x7f94d80cb730 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d814eba0 con 0x7f94d80b1c00 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94d7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8152530 con 0x7f94d815aec0 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94cde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94cde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49026/0 (socket says 172.21.15.91:49026) 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.988+0000 7f94cde00640 1 -- 172.21.15.91:0/1060180962 learned_addr learned my addr 172.21.15.91:0/1060180962 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:54.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94d6a00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d80b1c00 0x7f94d814e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 -- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 msgr2=0x7f94d81532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94d7400640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 0x7f94d81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 0x7f94d81532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 -- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d80b1c00 msgr2=0x7f94d814e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d80b1c00 0x7f94d814e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 -- 172.21.15.91:0/1060180962 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c8009400 con 0x7f94d80cb730 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94d7400640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 0x7f94d81532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cde00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c430 tx=0x7f94c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0011790 con 0x7f94d80cb730 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c00118f0 con 0x7f94d80cb730 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8152810 con 0x7f94d80cb730 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0011ce0 con 0x7f94d80cb730 2024-02-02T13:28:54.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.989+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d81543e0 con 0x7f94d80cb730 2024-02-02T13:28:54.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.991+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d8152fd0 con 0x7f94d80cb730 2024-02-02T13:28:54.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.992+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f94c001c460 con 0x7f94d80cb730 2024-02-02T13:28:54.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.992+0000 7f94cfe00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 0x7f94b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:54.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.993+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f94c0088780 con 0x7f94d80cb730 2024-02-02T13:28:54.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.993+0000 7f94d6a00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 0x7f94b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:54.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.993+0000 7f94d6a00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 0x7f94b006c8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94c8036000 tx=0x7f94c8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:54.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:54.997+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f94c0053ed0 con 0x7f94d80cb730 2024-02-02T13:28:55.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.191+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94d804dff0 con 0x7f94d80cb730 2024-02-02T13:28:55.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.760+0000 7f94cfe00640 1 -- 172.21.15.91:0/1060180962 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f94c00574f0 con 0x7f94d80cb730 2024-02-02T13:28:55.764 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 msgr2=0x7f94b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 0x7f94b006c8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94c8036000 tx=0x7f94c8008030 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 msgr2=0x7f94d8155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c430 tx=0x7f94c000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 shutdown_connections 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94b006a430 0x7f94b006c8f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.765+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94d815aec0 0x7f94d81532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.766+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94d80cb730 0x7f94d8155140 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.766+0000 7f94d7e00640 1 --2- 172.21.15.91:0/1060180962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d80b1c00 0x7f94d814e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.766+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 >> 172.21.15.91:0/1060180962 conn(0x7f94d80b08e0 msgr2=0x7f94d80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.766+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 shutdown_connections 2024-02-02T13:28:55.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.766+0000 7f94d7e00640 1 -- 172.21.15.91:0/1060180962 wait complete. 2024-02-02T13:28:55.788 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:28:55.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.983+0000 7f77cca00640 1 Processor -- start 2024-02-02T13:28:55.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.983+0000 7f77cca00640 1 -- start start 2024-02-02T13:28:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.984+0000 7f77cca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.984+0000 7f77cca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77c40711f0 0x7f77c40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.984+0000 7f77cca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c415f1b0 con 0x7f77c4071650 2024-02-02T13:28:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.984+0000 7f77cca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c415fdc0 con 0x7f77c40711f0 2024-02-02T13:28:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.984+0000 7f77cca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c415b6d0 con 0x7f77c415b2f0 2024-02-02T13:28:55.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.985+0000 7f77ca000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.985+0000 7f77ca000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:54106/0 (socket says 172.21.15.91:54106) 2024-02-02T13:28:55.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.985+0000 7f77ca000640 1 -- 172.21.15.91:0/1121800657 learned_addr learned my addr 172.21.15.91:0/1121800657 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:55.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.985+0000 7f77c9600640 1 --2- 172.21.15.91:0/1121800657 >> v2:172.21.15.113:3300/0 conn(0x7f77c40711f0 0x7f77c40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.986+0000 7f77ca000640 1 -- 172.21.15.91:0/1121800657 >> v2:172.21.15.113:3300/0 conn(0x7f77c40711f0 msgr2=0x7f77c40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:55.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.986+0000 7f77ca000640 1 --2- 172.21.15.91:0/1121800657 >> v2:172.21.15.113:3300/0 conn(0x7f77c40711f0 0x7f77c40b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.986+0000 7f77ca000640 1 -- 172.21.15.91:0/1121800657 >> v1:172.21.15.113:6789/0 conn(0x7f77c4071650 legacy=0x7f77c40c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.986+0000 7f77ca000640 1 -- 172.21.15.91:0/1121800657 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c415f490 con 0x7f77c415b2f0 2024-02-02T13:28:55.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.987+0000 7f77ca000640 1 --2- 172.21.15.91:0/1121800657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009870 tx=0x7f77ac02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4158a57940b20d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:55.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77c8c00640 1 -- 172.21.15.91:0/1121800657 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac031640 con 0x7f77c415b2f0 2024-02-02T13:28:55.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77c8c00640 1 -- 172.21.15.91:0/1121800657 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac0317a0 con 0x7f77c415b2f0 2024-02-02T13:28:55.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77c8c00640 1 -- 172.21.15.91:0/1121800657 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac031b30 con 0x7f77c415b2f0 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77cca00640 1 -- 172.21.15.91:0/1121800657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 msgr2=0x7f77c40b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77cca00640 1 --2- 172.21.15.91:0/1121800657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009870 tx=0x7f77ac02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77cca00640 1 -- 172.21.15.91:0/1121800657 shutdown_connections 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77cca00640 1 --2- 172.21.15.91:0/1121800657 >> v2:172.21.15.113:3300/0 conn(0x7f77c40711f0 0x7f77c40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.988+0000 7f77cca00640 1 --2- 172.21.15.91:0/1121800657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c415b2f0 0x7f77c40b1a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.989+0000 7f77cca00640 1 -- 172.21.15.91:0/1121800657 >> 172.21.15.91:0/1121800657 conn(0x7f77c40ceb90 msgr2=0x7f77c40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:55.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.989+0000 7f77cca00640 1 -- 172.21.15.91:0/1121800657 shutdown_connections 2024-02-02T13:28:55.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.989+0000 7f77cca00640 1 -- 172.21.15.91:0/1121800657 wait complete. 2024-02-02T13:28:55.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.990+0000 7f77cca00640 1 Processor -- start 2024-02-02T13:28:55.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.990+0000 7f77cca00640 1 -- start start 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 0x7f77c41f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c415b2f0 0x7f77c41f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40ae9e0 con 0x7f77c40711f0 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414c8e0 con 0x7f77c415b2f0 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.991+0000 7f77cca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414c600 con 0x7f77c4071650 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49028/0 (socket says 172.21.15.91:49028) 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 -- 172.21.15.91:0/2251026223 learned_addr learned my addr 172.21.15.91:0/2251026223 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77c9600640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 0x7f77c41f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 -- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 msgr2=0x7f77c41f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 0x7f77c41f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77caa00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c415b2f0 0x7f77c41f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 -- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c415b2f0 msgr2=0x7f77c41f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c415b2f0 0x7f77c41f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 -- 172.21.15.91:0/2251026223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0094d0 con 0x7f77c40711f0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77c9600640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 0x7f77c41f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.992+0000 7f77ca000640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0090b0 tx=0x7f77ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.993+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac03ba00 con 0x7f77c40711f0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.993+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac03bb60 con 0x7f77c40711f0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.993+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c40aec10 con 0x7f77c40711f0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.993+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0453f0 con 0x7f77c40711f0 2024-02-02T13:28:55.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.993+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40af1d0 con 0x7f77c40711f0 2024-02-02T13:28:55.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.994+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4066cf0 con 0x7f77c40711f0 2024-02-02T13:28:55.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.995+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77ac02f050 con 0x7f77c40711f0 2024-02-02T13:28:55.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.996+0000 7f77c2000640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 0x7f779c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:55.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.996+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f77ac0aa4c0 con 0x7f77c40711f0 2024-02-02T13:28:55.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.996+0000 7f77c9600640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 0x7f779c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:55.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:55.996+0000 7f77c9600640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 0x7f779c06c8f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c41f71a0 tx=0x7f77b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:56.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.000+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77ac075f60 con 0x7f77c40711f0 2024-02-02T13:28:56.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.190+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f77c406c950 con 0x7f77c40711f0 2024-02-02T13:28:56.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.196+0000 7f77c2000640 1 -- 172.21.15.91:0/2251026223 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77ac079580 con 0x7f77c40711f0 2024-02-02T13:28:56.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.199+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 msgr2=0x7f779c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.199+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 0x7f779c06c8f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c41f71a0 tx=0x7f77b4008040 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.199+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 msgr2=0x7f77c40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.199+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0090b0 tx=0x7f77ac0046c0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 shutdown_connections 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f779c06a430 0x7f779c06c8f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c4071650 0x7f77c41f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c415b2f0 0x7f77c41f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 --2- 172.21.15.91:0/2251026223 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77c40711f0 0x7f77c40b3250 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 >> 172.21.15.91:0/2251026223 conn(0x7f77c40ceb90 msgr2=0x7f77c40b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 shutdown_connections 2024-02-02T13:28:56.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.200+0000 7f77cca00640 1 -- 172.21.15.91:0/2251026223 wait complete. 2024-02-02T13:28:56.220 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-02-02T13:28:56.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.424+0000 7f2d3a800640 1 Processor -- start 2024-02-02T13:28:56.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.424+0000 7f2d3a800640 1 -- start start 2024-02-02T13:28:56.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d3a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2d340b0e10 0x7f2d340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d3a800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d340cc7c0 con 0x7f2d3415a010 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d3a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d340ce930 con 0x7f2d340ce130 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d3a800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d340cd640 con 0x7f2d340b0e10 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d33400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2d3415a010 0x7f2d340ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54882/0 (socket says 172.21.15.91:54882) 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d33400640 1 -- 172.21.15.91:0/1837179644 learned_addr learned my addr 172.21.15.91:0/1837179644 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d33e00640 1 --2- 172.21.15.91:0/1837179644 >> v2:172.21.15.91:3300/0 conn(0x7f2d340b0e10 0x7f2d340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.426+0000 7f2d32a00640 1 -- 172.21.15.91:0/1837179644 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244809123 0 0) 0x7f2d340cc7c0 con 0x7f2d3415a010 2024-02-02T13:28:56.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.427+0000 7f2d32a00640 1 -- 172.21.15.91:0/1837179644 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d14005730 con 0x7f2d3415a010 2024-02-02T13:28:56.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.427+0000 7f2d33e00640 1 -- 172.21.15.91:0/1837179644 >> v1:172.21.15.113:6789/0 conn(0x7f2d340ce130 legacy=0x7f2d340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.427+0000 7f2d33e00640 1 -- 172.21.15.91:0/1837179644 >> v1:172.21.15.91:6789/0 conn(0x7f2d3415a010 legacy=0x7f2d340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.427+0000 7f2d33e00640 1 -- 172.21.15.91:0/1837179644 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3406c890 con 0x7f2d340b0e10 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d33e00640 1 --2- 172.21.15.91:0/1837179644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340c9670 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c165d6c106da4e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d32a00640 1 -- 172.21.15.91:0/1837179644 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c02cc60 con 0x7f2d340b0e10 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d32a00640 1 -- 172.21.15.91:0/1837179644 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c02cdc0 con 0x7f2d340b0e10 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d32a00640 1 -- 172.21.15.91:0/1837179644 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c034660 con 0x7f2d340b0e10 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d3a800640 1 -- 172.21.15.91:0/1837179644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 msgr2=0x7f2d340c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.428+0000 7f2d3a800640 1 --2- 172.21.15.91:0/1837179644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340c9670 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02beb0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.429+0000 7f2d3a800640 1 -- 172.21.15.91:0/1837179644 shutdown_connections 2024-02-02T13:28:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.429+0000 7f2d3a800640 1 --2- 172.21.15.91:0/1837179644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340c9670 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.429+0000 7f2d3a800640 1 -- 172.21.15.91:0/1837179644 >> 172.21.15.91:0/1837179644 conn(0x7f2d340cec80 msgr2=0x7f2d340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.429+0000 7f2d3a800640 1 -- 172.21.15.91:0/1837179644 shutdown_connections 2024-02-02T13:28:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.429+0000 7f2d3a800640 1 -- 172.21.15.91:0/1837179644 wait complete. 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.430+0000 7f2d3a800640 1 Processor -- start 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.430+0000 7f2d3a800640 1 -- start start 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 0x7f2d341f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d3415a010 0x7f2d340af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b3910 con 0x7f2d340b0e10 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b2960 con 0x7f2d340ce130 2024-02-02T13:28:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.431+0000 7f2d3a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b4310 con 0x7f2d3415a010 2024-02-02T13:28:56.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49048/0 (socket says 172.21.15.91:49048) 2024-02-02T13:28:56.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 -- 172.21.15.91:0/4220773493 learned_addr learned my addr 172.21.15.91:0/4220773493 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d2a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 0x7f2d341f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33e00640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d3415a010 0x7f2d340af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 -- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d3415a010 msgr2=0x7f2d340af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d3415a010 0x7f2d340af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 -- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 msgr2=0x7f2d341f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 0x7f2d341f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 -- 172.21.15.91:0/4220773493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c009a70 con 0x7f2d340b0e10 2024-02-02T13:28:56.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d2a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 0x7f2d341f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.432+0000 7f2d33400640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c440 tx=0x7f2d2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.433+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24005ce0 con 0x7f2d340b0e10 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.433+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24005e40 con 0x7f2d340b0e10 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.433+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d340b2bd0 con 0x7f2d340b0e10 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.433+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240072b0 con 0x7f2d340b0e10 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.433+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d340b45f0 con 0x7f2d340b0e10 2024-02-02T13:28:56.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.434+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d34066cf0 con 0x7f2d340b0e10 2024-02-02T13:28:56.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.435+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2d24007410 con 0x7f2d340b0e10 2024-02-02T13:28:56.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.436+0000 7f2d2be00640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 0x7f2d0c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.436+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2d24087bc0 con 0x7f2d340b0e10 2024-02-02T13:28:56.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.437+0000 7f2d2a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 0x7f2d0c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.437+0000 7f2d2a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 0x7f2d0c06c8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d341f6ce0 tx=0x7f2d1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:56.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.441+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2d24053570 con 0x7f2d340b0e10 2024-02-02T13:28:56.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.699+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f2d34070b90 con 0x7f2d340b0e10 2024-02-02T13:28:56.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.761+0000 7f2d2be00640 1 -- 172.21.15.91:0/4220773493 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v231) v1 ==== 134+0+0 (secure 0 0 0) 0x7f2d240170c0 con 0x7f2d340b0e10 2024-02-02T13:28:56.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 msgr2=0x7f2d0c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 0x7f2d0c06c8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d341f6ce0 tx=0x7f2d1c009040 comp rx=0 tx=0).stop 2024-02-02T13:28:56.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 msgr2=0x7f2d340b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c440 tx=0x7f2d2400c900 comp rx=0 tx=0).stop 2024-02-02T13:28:56.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 shutdown_connections 2024-02-02T13:28:56.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2d0c06a430 0x7f2d0c06c8f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d3415a010 0x7f2d340af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d340ce130 0x7f2d341f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 --2- 172.21.15.91:0/4220773493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d340b0e10 0x7f2d340b33d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 >> 172.21.15.91:0/4220773493 conn(0x7f2d340cec80 msgr2=0x7f2d3406fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:56.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 shutdown_connections 2024-02-02T13:28:56.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.765+0000 7f2d3a800640 1 -- 172.21.15.91:0/4220773493 wait complete. 2024-02-02T13:28:56.787 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-02-02T13:28:56.787 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-02-02T13:28:56.787 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-02-02T13:28:56.787 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-02-02T13:28:56.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.992+0000 7f0d16200640 1 Processor -- start 2024-02-02T13:28:56.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.992+0000 7f0d16200640 1 -- start start 2024-02-02T13:28:56.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.993+0000 7f0d16200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d100ce130 0x7f0d100c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:56.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.993+0000 7f0d16200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100cc7c0 con 0x7f0d100b0700 2024-02-02T13:28:56.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d16200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100ce930 con 0x7f0d1015a010 2024-02-02T13:28:56.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d16200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100cd640 con 0x7f0d100ce130 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d06800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d100ce130 0x7f0d100c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d0fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0d100b0700 0x7f0d1006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54898/0 (socket says 172.21.15.91:54898) 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d0fe00640 1 -- 172.21.15.91:0/815385264 learned_addr learned my addr 172.21.15.91:0/815385264 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d06800640 1 -- 172.21.15.91:0/815385264 >> v1:172.21.15.113:6789/0 conn(0x7f0d1015a010 legacy=0x7f0d100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d06800640 1 -- 172.21.15.91:0/815385264 >> v1:172.21.15.91:6789/0 conn(0x7f0d100b0700 legacy=0x7f0d1006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.994+0000 7f0d0ea00640 1 -- 172.21.15.91:0/815385264 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925236424 0 0) 0x7f0d100cc7c0 con 0x7f0d100b0700 2024-02-02T13:28:56.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.995+0000 7f0d06800640 1 -- 172.21.15.91:0/815385264 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1006c890 con 0x7f0d100ce130 2024-02-02T13:28:56.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d06800640 1 --2- 172.21.15.91:0/815385264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d100c9670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a17e507451d7446 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:56.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d0ea00640 1 -- 172.21.15.91:0/815385264 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8030610 con 0x7f0d100ce130 2024-02-02T13:28:56.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d0ea00640 1 -- 172.21.15.91:0/815385264 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8030770 con 0x7f0d100ce130 2024-02-02T13:28:56.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d0ea00640 1 -- 172.21.15.91:0/815385264 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8030b00 con 0x7f0d100ce130 2024-02-02T13:28:56.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d16200640 1 -- 172.21.15.91:0/815385264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 msgr2=0x7f0d100c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:56.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.996+0000 7f0d16200640 1 --2- 172.21.15.91:0/815385264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d100c9670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b960 comp rx=0 tx=0).stop 2024-02-02T13:28:56.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.997+0000 7f0d16200640 1 -- 172.21.15.91:0/815385264 shutdown_connections 2024-02-02T13:28:56.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.997+0000 7f0d16200640 1 --2- 172.21.15.91:0/815385264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d100c9670 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:56.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.997+0000 7f0d16200640 1 -- 172.21.15.91:0/815385264 >> 172.21.15.91:0/815385264 conn(0x7f0d100cec80 msgr2=0x7f0d100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:56.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.997+0000 7f0d16200640 1 -- 172.21.15.91:0/815385264 shutdown_connections 2024-02-02T13:28:56.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.997+0000 7f0d16200640 1 -- 172.21.15.91:0/815385264 wait complete. 2024-02-02T13:28:57.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.998+0000 7f0d16200640 1 Processor -- start 2024-02-02T13:28:57.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.998+0000 7f0d16200640 1 -- start start 2024-02-02T13:28:57.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 0x7f0d1015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 0x7f0d100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100c5cc0 con 0x7f0d100ce130 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015bf80 con 0x7f0d1015a010 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:56.999+0000 7f0d16200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100c5ed0 con 0x7f0d100b0700 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49062/0 (socket says 172.21.15.91:49062) 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 -- 172.21.15.91:0/1683031040 learned_addr learned my addr 172.21.15.91:0/1683031040 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d0fe00640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 0x7f0d100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d0f400640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 0x7f0d1015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:57.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 -- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 msgr2=0x7f0d1015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 0x7f0d1015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 -- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 msgr2=0x7f0d100706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 0x7f0d100706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d06800640 1 -- 172.21.15.91:0/1683031040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009400 con 0x7f0d100ce130 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d0fe00640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 0x7f0d100706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.000+0000 7f0d0f400640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 0x7f0d1015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d06800640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000c00 tx=0x7f0cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8039c90 con 0x7f0d100ce130 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf803f040 con 0x7f0d100ce130 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8004030 con 0x7f0d100ce130 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100c61b0 con 0x7f0d100ce130 2024-02-02T13:28:57.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.001+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d101556c0 con 0x7f0d100ce130 2024-02-02T13:28:57.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.002+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1015c410 con 0x7f0d100ce130 2024-02-02T13:28:57.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.003+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0cf8004190 con 0x7f0d100ce130 2024-02-02T13:28:57.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.004+0000 7f0d07e00640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 0x7f0ce806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:57.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.004+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cf80aa3d0 con 0x7f0d100ce130 2024-02-02T13:28:57.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.004+0000 7f0d0f400640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 0x7f0ce806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:57.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.005+0000 7f0d0f400640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 0x7f0ce806c8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d100cf4b0 tx=0x7f0d00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.009+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d1015c410 con 0x7f0d100ce130 2024-02-02T13:28:57.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.268+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f0d1015c410 con 0x7f0d100ce130 2024-02-02T13:28:57.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.765+0000 7f0d07e00640 1 -- 172.21.15.91:0/1683031040 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v232) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0d1015c410 con 0x7f0d100ce130 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 msgr2=0x7f0ce806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 0x7f0ce806c8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d100cf4b0 tx=0x7f0d00009040 comp rx=0 tx=0).stop 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 msgr2=0x7f0d10155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000c00 tx=0x7f0cf80046c0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 shutdown_connections 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ce806a430 0x7f0ce806c8f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1015a010 0x7f0d100706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d100ce130 0x7f0d10155180 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 --2- 172.21.15.91:0/1683031040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d100b0700 0x7f0d1015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:57.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 >> 172.21.15.91:0/1683031040 conn(0x7f0d100cec80 msgr2=0x7f0d100c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:57.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 shutdown_connections 2024-02-02T13:28:57.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:57.769+0000 7f0d16200640 1 -- 172.21.15.91:0/1683031040 wait complete. 2024-02-02T13:28:57.790 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-02-02T13:28:58.010 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_objectstore":"memstore"} 2024-02-02T13:28:58.010 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-02-02T13:28:58.011 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:28:58.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.223+0000 7f92fa400640 1 Processor -- start 2024-02-02T13:28:58.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.223+0000 7f92fa400640 1 -- start start 2024-02-02T13:28:58.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92fa400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92f406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92fa400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f415d870 con 0x7f92f40b0700 2024-02-02T13:28:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92fa400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f406bce0 con 0x7f92f415d490 2024-02-02T13:28:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92fa400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4161220 con 0x7f92f40acfe0 2024-02-02T13:28:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92f3e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f92f40b0700 0x7f92f40c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54914/0 (socket says 172.21.15.91:54914) 2024-02-02T13:28:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92f3e00640 1 -- 172.21.15.91:0/2653846144 learned_addr learned my addr 172.21.15.91:0/2653846144 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.225+0000 7f92f2a00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92f406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494750075 0 0) 0x7f92f415d870 con 0x7f92f40b0700 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d4003520 con 0x7f92f40b0700 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105917145 0 0) 0x7f92d4003520 con 0x7f92f40b0700 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 msgr2=0x7f92f406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92f406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 >> v1:172.21.15.113:6789/0 conn(0x7f92f415d490 legacy=0x7f92f40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40ad490 con 0x7f92f40b0700 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f92ec0019f0 con 0x7f92f40b0700 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 >> v1:172.21.15.91:6789/0 conn(0x7f92f40b0700 legacy=0x7f92f40c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d40039a0 0x7f92d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4161220 con 0x7f92d4004d00 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400ba60 con 0x7f92d40039a0 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.226+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400bc20 con 0x7f92f40acfe0 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3400640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d40039a0 0x7f92d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f2a00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 -- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 msgr2=0x7f92d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40acfe0 0x7f92d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 -- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d40039a0 msgr2=0x7f92d4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92d40039a0 0x7f92d4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.227+0000 7f92f3e00640 1 -- 172.21.15.91:0/2653846144 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4003520 con 0x7f92d4004d00 2024-02-02T13:28:58.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.228+0000 7f92f3e00640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92ec002230 tx=0x7f92d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c6640b1a8cf261 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:58.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.228+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec034630 con 0x7f92d4004d00 2024-02-02T13:28:58.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.228+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec02f070 con 0x7f92d4004d00 2024-02-02T13:28:58.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.228+0000 7f92f2000640 1 -- 172.21.15.91:0/2653846144 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec034a10 con 0x7f92d4004d00 2024-02-02T13:28:58.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 -- 172.21.15.91:0/2653846144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d4004d00 msgr2=0x7f92d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 --2- 172.21.15.91:0/2653846144 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92ec002230 tx=0x7f92d40057f0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92f3e00640 1 -- 172.21.15.91:0/2653846144 reap_dead start 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 -- 172.21.15.91:0/2653846144 shutdown_connections 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 -- 172.21.15.91:0/2653846144 >> 172.21.15.91:0/2653846144 conn(0x7f92f40cec80 msgr2=0x7f92f40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 -- 172.21.15.91:0/2653846144 shutdown_connections 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.229+0000 7f92fa400640 1 -- 172.21.15.91:0/2653846144 wait complete. 2024-02-02T13:28:58.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.231+0000 7f92fa400640 1 Processor -- start 2024-02-02T13:28:58.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.231+0000 7f92fa400640 1 -- start start 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d4004d00 0x7f92f40c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414d520 0x7f92f40c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40c6200 con 0x7f92f40c3cb0 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40c8cd0 con 0x7f92f414d520 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92fa400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4160300 con 0x7f92d4004d00 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92f3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92f3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49070/0 (socket says 172.21.15.91:49070) 2024-02-02T13:28:58.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.232+0000 7f92f3e00640 1 -- 172.21.15.91:0/390747337 learned_addr learned my addr 172.21.15.91:0/390747337 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f2a00640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414d520 0x7f92f40c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d4004d00 0x7f92f40c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 -- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d4004d00 msgr2=0x7f92f40c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d4004d00 0x7f92f40c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 -- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414d520 msgr2=0x7f92f40c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414d520 0x7f92f40c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 -- 172.21.15.91:0/390747337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f414e4d0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92f3e00640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000f80 tx=0x7f92ec001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec034640 con 0x7f92f40c3cb0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec0347a0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f41605a0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.233+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec034b50 con 0x7f92f40c3cb0 2024-02-02T13:28:58.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.234+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f415e4b0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.235+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f415e6a0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.236+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92ec031030 con 0x7f92f40c3cb0 2024-02-02T13:28:58.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.236+0000 7f92eb400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 0x7f92cc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:58.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.237+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92ec0aa960 con 0x7f92f40c3cb0 2024-02-02T13:28:58.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.237+0000 7f92f3400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 0x7f92cc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:58.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.237+0000 7f92f3400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 0x7f92cc06c920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007fd0 tx=0x7f92e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:58.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.241+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f92ec076250 con 0x7f92f40c3cb0 2024-02-02T13:28:58.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.430+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f92f415e6a0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.436+0000 7f92eb400640 1 -- 172.21.15.91:0/390747337 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92ec034cb0 con 0x7f92f40c3cb0 2024-02-02T13:28:58.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 msgr2=0x7f92cc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 0x7f92cc06c920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007fd0 tx=0x7f92e4009040 comp rx=0 tx=0).stop 2024-02-02T13:28:58.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 msgr2=0x7f92f415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000f80 tx=0x7f92ec001cb0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 shutdown_connections 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92cc06a460 0x7f92cc06c920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f40c3cb0 0x7f92f415e9d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f414d520 0x7f92f40c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 --2- 172.21.15.91:0/390747337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92d4004d00 0x7f92f40c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:58.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 >> 172.21.15.91:0/390747337 conn(0x7f92f40cec80 msgr2=0x7f92f4161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:58.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 shutdown_connections 2024-02-02T13:28:58.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:58.439+0000 7f92fa400640 1 -- 172.21.15.91:0/390747337 wait complete. 2024-02-02T13:28:58.460 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:28:58.460 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:28:58.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:28:58.686 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:28:58.686 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:28:58.686 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:58.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:28:58.743 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:28:58.743 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:58.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:28:58.797 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:28:58.797 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:58.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:28:58.854 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:28:58.854 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:28:58.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:28:58.911 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:28:58.911 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:28:59.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.113+0000 7f0cfa800640 1 Processor -- start 2024-02-02T13:28:59.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.113+0000 7f0cfa800640 1 -- start start 2024-02-02T13:28:59.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cfa800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161660 0x7f0cf406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cfa800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0cf40ce270 0x7f0cf40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cfa800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40b10a0 con 0x7f0cf40b0700 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cfa800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40ccdd0 con 0x7f0cf40ce270 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cfa800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40cc7c0 con 0x7f0cf4161660 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0cf40ce270 0x7f0cf40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0cf40ce270 0x7f0cf40b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49084/0 (socket says 172.21.15.91:49084) 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 -- 172.21.15.91:0/520170407 learned_addr learned my addr 172.21.15.91:0/520170407 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:59.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 -- 172.21.15.91:0/520170407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161660 msgr2=0x7f0cf406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/520170407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161660 0x7f0cf406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 -- 172.21.15.91:0/520170407 >> v1:172.21.15.113:6789/0 conn(0x7f0cf40b0700 legacy=0x7f0cf4165200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.115+0000 7f0cf3e00640 1 -- 172.21.15.91:0/520170407 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf406c540 con 0x7f0cf40ce270 2024-02-02T13:28:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.117+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/520170407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf40ce270 0x7f0cf40b0b60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009e50 tx=0x7f0cec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28478c2141641127 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.117+0000 7f0cf2a00640 1 -- 172.21.15.91:0/520170407 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec02c9e0 con 0x7f0cf40ce270 2024-02-02T13:28:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.117+0000 7f0cf2a00640 1 -- 172.21.15.91:0/520170407 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec02cb40 con 0x7f0cf40ce270 2024-02-02T13:28:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.117+0000 7f0cf2a00640 1 -- 172.21.15.91:0/520170407 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0343f0 con 0x7f0cf40ce270 2024-02-02T13:28:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 -- 172.21.15.91:0/520170407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf40ce270 msgr2=0x7f0cf40b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 --2- 172.21.15.91:0/520170407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf40ce270 0x7f0cf40b0b60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009e50 tx=0x7f0cec02be40 comp rx=0 tx=0).stop 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 -- 172.21.15.91:0/520170407 shutdown_connections 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 --2- 172.21.15.91:0/520170407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf40ce270 0x7f0cf40b0b60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 --2- 172.21.15.91:0/520170407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161660 0x7f0cf406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 -- 172.21.15.91:0/520170407 >> 172.21.15.91:0/520170407 conn(0x7f0cf40cec80 msgr2=0x7f0cf40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 -- 172.21.15.91:0/520170407 shutdown_connections 2024-02-02T13:28:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.118+0000 7f0cfa800640 1 -- 172.21.15.91:0/520170407 wait complete. 2024-02-02T13:28:59.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.120+0000 7f0cfa800640 1 Processor -- start 2024-02-02T13:28:59.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.120+0000 7f0cfa800640 1 -- start start 2024-02-02T13:28:59.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 0x7f0cf40b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40ce270 0x7f0cf40c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c3a60 con 0x7f0cf4161660 2024-02-02T13:28:59.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c3d40 con 0x7f0cf40ce270 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.121+0000 7f0cfa800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c4020 con 0x7f0cf40b0700 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49088/0 (socket says 172.21.15.91:49088) 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 -- 172.21.15.91:0/3148953896 learned_addr learned my addr 172.21.15.91:0/3148953896 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cebe00640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40ce270 0x7f0cf40c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3400640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 0x7f0cf40b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 -- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 msgr2=0x7f0cf40b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 0x7f0cf40b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 -- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40ce270 msgr2=0x7f0cf40c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40ce270 0x7f0cf40c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 -- 172.21.15.91:0/3148953896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009b30 con 0x7f0cf4161660 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3400640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 0x7f0cf40b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.122+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cec036040 tx=0x7f0cec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.123+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec009510 con 0x7f0cf4161660 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.123+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec03e450 con 0x7f0cf4161660 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.123+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec03e7e0 con 0x7f0cf4161660 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.123+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf40c4300 con 0x7f0cf4161660 2024-02-02T13:28:59.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.123+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4164a90 con 0x7f0cf4161660 2024-02-02T13:28:59.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.124+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf40c47c0 con 0x7f0cf4161660 2024-02-02T13:28:59.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.125+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0cec03e940 con 0x7f0cf4161660 2024-02-02T13:28:59.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.126+0000 7f0ceb400640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.126+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cec0ab480 con 0x7f0cf4161660 2024-02-02T13:28:59.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.126+0000 7f0cf3400640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.127+0000 7f0cf3400640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.130+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0cec076d60 con 0x7f0cf4161660 2024-02-02T13:28:59.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.357+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf4066cf0 con 0x7f0cf4161660 2024-02-02T13:28:59.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.358+0000 7f0ceb400640 1 -- 172.21.15.91:0/3148953896 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0cec043de0 con 0x7f0cf4161660 2024-02-02T13:28:59.360 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:28:59.360 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1484393297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1484393297},{"type":"v1","addr":"172.21.15.91:6836","nonce":1484393297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2617168700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2617168700},{"type":"v1","addr":"172.21.15.113:6834","nonce":2617168700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3850525880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3850525880},{"type":"v1","addr":"172.21.15.113:6835","nonce":3850525880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/109959325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":109959325},{"type":"v1","addr":"172.21.15.91:6837","nonce":109959325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2024-02-02T13:28:59.360 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 34 2024-02-02T13:28:59.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 msgr2=0x7f0ccc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 msgr2=0x7f0cf40b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cec036040 tx=0x7f0cec004a40 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 shutdown_connections 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf4161660 0x7f0cf40b3970 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40ce270 0x7f0cf40c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 --2- 172.21.15.91:0/3148953896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf40b0700 0x7f0cf40b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 >> 172.21.15.91:0/3148953896 conn(0x7f0cf40cec80 msgr2=0x7f0cf415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 shutdown_connections 2024-02-02T13:28:59.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.361+0000 7f0cfa800640 1 -- 172.21.15.91:0/3148953896 wait complete. 2024-02-02T13:28:59.381 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:28:59.381 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:28:59.381 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:28:59.381 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:28:59.381 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:28:59.381 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:28:59.382 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:28:59.382 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:28:59.382 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:28:59.382 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:28:59.382 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:28:59.382 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:28:59.382 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:28:59.382 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:28:59.383 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:28:59.383 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:28:59.396 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:28:59.397 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:28:59.399 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:28:59.399 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:28:59.402 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:28:59.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:28:59.406 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:28:59.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:28:59.446 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:28:59.447 DEBUG:teuthology.parallel:result is None 2024-02-02T13:28:59.453 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:28:59.453 DEBUG:teuthology.parallel:result is None 2024-02-02T13:28:59.454 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:28:59.454 DEBUG:teuthology.parallel:result is None 2024-02-02T13:28:59.459 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:28:59.459 DEBUG:teuthology.parallel:result is None 2024-02-02T13:28:59.460 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:28:59.600 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:28:59.601 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:28:59.616 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:28:59.620 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:28:59.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.647+0000 7f0d1ba00640 1 Processor -- start 2024-02-02T13:28:59.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.647+0000 7f0d1ba00640 1 -- start start 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d1ba00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d140d1a60 0x7f0d140cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d1ba00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140704a0 con 0x7f0d1415d3e0 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d1ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140d1f60 con 0x7f0d140ce2e0 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d1ba00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140d2800 con 0x7f0d140d1a60 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d140d1a60 0x7f0d140cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d140d1a60 0x7f0d140cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49144/0 (socket says 172.21.15.91:49144) 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 -- 172.21.15.91:0/749882097 learned_addr learned my addr 172.21.15.91:0/749882097 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d19000640 1 --1- 172.21.15.91:0/749882097 >> v1:172.21.15.91:6789/0 conn(0x7f0d1415d3e0 0x7f0d140b88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54936/0 (socket says 172.21.15.91:54936) 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 -- 172.21.15.91:0/749882097 >> v1:172.21.15.113:6789/0 conn(0x7f0d140ce2e0 legacy=0x7f0d14070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:28:59.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 -- 172.21.15.91:0/749882097 >> v1:172.21.15.91:6789/0 conn(0x7f0d1415d3e0 legacy=0x7f0d140b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.649+0000 7f0d0bc00640 1 -- 172.21.15.91:0/749882097 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140d2240 con 0x7f0d140d1a60 2024-02-02T13:28:59.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d0bc00640 1 --2- 172.21.15.91:0/749882097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d140cdda0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009e50 tx=0x7f0cfc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd241df0c7ddf54 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d13e00640 1 -- 172.21.15.91:0/749882097 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc02cb10 con 0x7f0d140d1a60 2024-02-02T13:28:59.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d13e00640 1 -- 172.21.15.91:0/749882097 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc02cc70 con 0x7f0d140d1a60 2024-02-02T13:28:59.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d13e00640 1 -- 172.21.15.91:0/749882097 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0343f0 con 0x7f0d140d1a60 2024-02-02T13:28:59.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d1ba00640 1 -- 172.21.15.91:0/749882097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 msgr2=0x7f0d140cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.651+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/749882097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d140cdda0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009e50 tx=0x7f0cfc02bf70 comp rx=0 tx=0).stop 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.652+0000 7f0d1ba00640 1 -- 172.21.15.91:0/749882097 shutdown_connections 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.652+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/749882097 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d140cdda0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.652+0000 7f0d1ba00640 1 -- 172.21.15.91:0/749882097 >> 172.21.15.91:0/749882097 conn(0x7f0d140cfa50 msgr2=0x7f0d140cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.652+0000 7f0d1ba00640 1 -- 172.21.15.91:0/749882097 shutdown_connections 2024-02-02T13:28:59.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.652+0000 7f0d1ba00640 1 -- 172.21.15.91:0/749882097 wait complete. 2024-02-02T13:28:59.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.653+0000 7f0d1ba00640 1 Processor -- start 2024-02-02T13:28:59.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.653+0000 7f0d1ba00640 1 -- start start 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 0x7f0d14156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 0x7f0d14156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415dcd0 con 0x7f0d140d1a60 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415e630 con 0x7f0d1415d3e0 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.654+0000 7f0d1ba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415edc0 con 0x7f0d140ce2e0 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49150/0 (socket says 172.21.15.91:49150) 2024-02-02T13:28:59.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 -- 172.21.15.91:0/1624285419 learned_addr learned my addr 172.21.15.91:0/1624285419 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:28:59.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 -- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 msgr2=0x7f0d14156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d19000640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 0x7f0d14156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d19a00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 0x7f0d14156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 0x7f0d14156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 -- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 msgr2=0x7f0d14156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 0x7f0d14156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d0bc00640 1 -- 172.21.15.91:0/1624285419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009b30 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d19000640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 0x7f0d14156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.655+0000 7f0d19a00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 0x7f0d14156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d0bc00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc000f80 tx=0x7f0cfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc03e450 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc03e5b0 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1415f0a0 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc03e960 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.656+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14156c50 con 0x7f0d140d1a60 2024-02-02T13:28:59.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.657+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1415de10 con 0x7f0d140d1a60 2024-02-02T13:28:59.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.658+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0cfc03eac0 con 0x7f0d140d1a60 2024-02-02T13:28:59.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.659+0000 7f0d11600640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 0x7f0cec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:28:59.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.659+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cfc0ab6c0 con 0x7f0d140d1a60 2024-02-02T13:28:59.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.660+0000 7f0d19000640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 0x7f0cec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:28:59.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.660+0000 7f0d19000640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 0x7f0cec06c8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d04001020 tx=0x7f0d04009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:28:59.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.666+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0cfc077140 con 0x7f0d140d1a60 2024-02-02T13:28:59.839 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:28:59.847 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:28:59.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:28:59.856+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0d1415de10 con 0x7f0d140d1a60 2024-02-02T13:29:00.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.646+0000 7f0d11600640 1 -- 172.21.15.91:0/1624285419 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0d1415de10 con 0x7f0d140d1a60 2024-02-02T13:29:00.648 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 msgr2=0x7f0cec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 0x7f0cec06c8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d04001020 tx=0x7f0d04009040 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 msgr2=0x7f0d1415e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc000f80 tx=0x7f0cfc004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 shutdown_connections 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0cec06a430 0x7f0cec06c8f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1415d3e0 0x7f0d14156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d140d1a60 0x7f0d1415e0f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 --2- 172.21.15.91:0/1624285419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d140ce2e0 0x7f0d14156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 >> 172.21.15.91:0/1624285419 conn(0x7f0d140cfa50 msgr2=0x7f0d140b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 shutdown_connections 2024-02-02T13:29:00.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.650+0000 7f0d1ba00640 1 -- 172.21.15.91:0/1624285419 wait complete. 2024-02-02T13:29:00.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:29:00.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.870+0000 7fde3b200640 1 Processor -- start 2024-02-02T13:29:00.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.870+0000 7fde3b200640 1 -- start start 2024-02-02T13:29:00.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.871+0000 7fde3b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c15d520 0x7fde3c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.871+0000 7fde3b200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde3c0b04a0 0x7fde3c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.871+0000 7fde3b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c0cd3c0 con 0x7fde3c0ccfe0 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.871+0000 7fde3b200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c0b0e30 con 0x7fde3c0b04a0 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.871+0000 7fde3b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c15df70 con 0x7fde3c15d520 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde3c0b04a0 0x7fde3c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde3c0b04a0 0x7fde3c0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50788/0 (socket says 172.21.15.91:50788) 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 -- 172.21.15.91:0/2838500976 learned_addr learned my addr 172.21.15.91:0/2838500976 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde33e00640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c15d520 0x7fde3c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 -- 172.21.15.91:0/2838500976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c15d520 msgr2=0x7fde3c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c15d520 0x7fde3c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 -- 172.21.15.91:0/2838500976 >> v1:172.21.15.113:6789/0 conn(0x7fde3c0ccfe0 legacy=0x7fde3c0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:00.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.872+0000 7fde3a800640 1 -- 172.21.15.91:0/2838500976 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c15e250 con 0x7fde3c0b04a0 2024-02-02T13:29:00.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.873+0000 7fde3a800640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c0b04a0 0x7fde3c0b08f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fde34009e50 tx=0x7fde3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299cd3eec7c8fc6c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:00.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.873+0000 7fde39400640 1 -- 172.21.15.91:0/2838500976 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3402ca70 con 0x7fde3c0b04a0 2024-02-02T13:29:00.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde39400640 1 -- 172.21.15.91:0/2838500976 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3402cbd0 con 0x7fde3c0b04a0 2024-02-02T13:29:00.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde39400640 1 -- 172.21.15.91:0/2838500976 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340343f0 con 0x7fde3c0b04a0 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 -- 172.21.15.91:0/2838500976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c0b04a0 msgr2=0x7fde3c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c0b04a0 0x7fde3c0b08f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fde34009e50 tx=0x7fde3402bed0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 -- 172.21.15.91:0/2838500976 shutdown_connections 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c0b04a0 0x7fde3c0b08f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 --2- 172.21.15.91:0/2838500976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c15d520 0x7fde3c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 -- 172.21.15.91:0/2838500976 >> 172.21.15.91:0/2838500976 conn(0x7fde3c0cb4a0 msgr2=0x7fde3c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 -- 172.21.15.91:0/2838500976 shutdown_connections 2024-02-02T13:29:00.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.874+0000 7fde3b200640 1 -- 172.21.15.91:0/2838500976 wait complete. 2024-02-02T13:29:00.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.876+0000 7fde3b200640 1 Processor -- start 2024-02-02T13:29:00.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.876+0000 7fde3b200640 1 -- start start 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c0b04a0 0x7fde3c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 0x7fde3c1f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0c69c0 con 0x7fde3c15d520 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c158870 con 0x7fde3c0ccfe0 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c158af0 con 0x7fde3c0b04a0 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50790/0 (socket says 172.21.15.91:50790) 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3a800640 1 -- 172.21.15.91:0/2967263349 learned_addr learned my addr 172.21.15.91:0/2967263349 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde33e00640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 0x7fde3c1f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde39e00640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c0b04a0 0x7fde3c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3a800640 1 -- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c0b04a0 msgr2=0x7fde3c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.877+0000 7fde3a800640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c0b04a0 0x7fde3c15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3a800640 1 -- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 msgr2=0x7fde3c1f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3a800640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 0x7fde3c1f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3a800640 1 -- 172.21.15.91:0/2967263349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009b30 con 0x7fde3c15d520 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde33e00640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 0x7fde3c1f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3a800640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fde34036040 tx=0x7fde34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34009510 con 0x7fde3c15d520 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3403e450 con 0x7fde3c15d520 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3403e7e0 con 0x7fde3c15d520 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c158dd0 con 0x7fde3c15d520 2024-02-02T13:29:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.878+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0c6570 con 0x7fde3c15d520 2024-02-02T13:29:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.879+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c0c6b00 con 0x7fde3c15d520 2024-02-02T13:29:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.880+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde3403e940 con 0x7fde3c15d520 2024-02-02T13:29:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.881+0000 7fde32000640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 0x7fde1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.881+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fde340ab430 con 0x7fde3c15d520 2024-02-02T13:29:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.881+0000 7fde39e00640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 0x7fde1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.882+0000 7fde39e00640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 0x7fde1406c8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0015d0 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:00.885+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde3c0c6b00 con 0x7fde3c15d520 2024-02-02T13:29:01.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.145+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde3c0c6b00 con 0x7fde3c15d520 2024-02-02T13:29:01.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.146+0000 7fde32000640 1 -- 172.21.15.91:0/2967263349 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fde3c0c6b00 con 0x7fde3c15d520 2024-02-02T13:29:01.148 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:29:01.153 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.154 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:29:01.155 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:29:01.157 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.158 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.159 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:29:01.161 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 msgr2=0x7fde1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 0x7fde1406c8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0015d0 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 msgr2=0x7fde3c158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fde34036040 tx=0x7fde34004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 shutdown_connections 2024-02-02T13:29:01.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde1406a430 0x7fde1406c8f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde3c15d520 0x7fde3c158330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde3c0ccfe0 0x7fde3c1f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 --2- 172.21.15.91:0/2967263349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde3c0b04a0 0x7fde3c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 >> 172.21.15.91:0/2967263349 conn(0x7fde3c0cb4a0 msgr2=0x7fde3c0c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 shutdown_connections 2024-02-02T13:29:01.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.149+0000 7fde3b200640 1 -- 172.21.15.91:0/2967263349 wait complete. 2024-02-02T13:29:01.167 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:01.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.373+0000 7f50a3e00640 1 Processor -- start 2024-02-02T13:29:01.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.373+0000 7f50a3e00640 1 -- start start 2024-02-02T13:29:01.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f509c151f90 0x7f509c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f509c06f4f0 0x7f509c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a3e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c0b1070 con 0x7f509c0b0c40 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a3e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c070f80 con 0x7f509c151f90 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c152370 con 0x7f509c06f4f0 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f509c151f90 0x7f509c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f509c151f90 0x7f509c15f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50802/0 (socket says 172.21.15.91:50802) 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.374+0000 7f50a1400640 1 -- 172.21.15.91:0/2158438399 learned_addr learned my addr 172.21.15.91:0/2158438399 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f50a0a00640 1 --2- 172.21.15.91:0/2158438399 >> v2:172.21.15.113:3300/0 conn(0x7f509c06f4f0 0x7f509c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f50a1400640 1 -- 172.21.15.91:0/2158438399 >> v2:172.21.15.113:3300/0 conn(0x7f509c06f4f0 msgr2=0x7f509c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:01.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f50a1400640 1 --2- 172.21.15.91:0/2158438399 >> v2:172.21.15.113:3300/0 conn(0x7f509c06f4f0 0x7f509c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f50a1400640 1 -- 172.21.15.91:0/2158438399 >> v1:172.21.15.91:6789/0 conn(0x7f509c0b0c40 legacy=0x7f509c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f50a1400640 1 -- 172.21.15.91:0/2158438399 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0cb010 con 0x7f509c151f90 2024-02-02T13:29:01.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.375+0000 7f509be00640 1 -- 172.21.15.91:0/2158438399 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366543177 0 0) 0x7f509c0b1070 con 0x7f509c0b0c40 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.376+0000 7f50a1400640 1 --2- 172.21.15.91:0/2158438399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c15f5a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f509c071a00 tx=0x7f508c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664cfc0577395b45 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.376+0000 7f509be00640 1 -- 172.21.15.91:0/2158438399 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c030860 con 0x7f509c151f90 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.376+0000 7f509be00640 1 -- 172.21.15.91:0/2158438399 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c0309c0 con 0x7f509c151f90 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.376+0000 7f509be00640 1 -- 172.21.15.91:0/2158438399 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c030d50 con 0x7f509c151f90 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 -- 172.21.15.91:0/2158438399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 msgr2=0x7f509c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2158438399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c15f5a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f509c071a00 tx=0x7f508c02b9a0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 -- 172.21.15.91:0/2158438399 shutdown_connections 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2158438399 >> v2:172.21.15.113:3300/0 conn(0x7f509c06f4f0 0x7f509c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2158438399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c15f5a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 -- 172.21.15.91:0/2158438399 >> 172.21.15.91:0/2158438399 conn(0x7f509c0cec80 msgr2=0x7f509c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 -- 172.21.15.91:0/2158438399 shutdown_connections 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.377+0000 7f50a3e00640 1 -- 172.21.15.91:0/2158438399 wait complete. 2024-02-02T13:29:01.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.379+0000 7f50a3e00640 1 Processor -- start 2024-02-02T13:29:01.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.379+0000 7f50a3e00640 1 -- start start 2024-02-02T13:29:01.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c06f4f0 0x7f509c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f509c0b0c40 0x7f509c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0b1de0 con 0x7f509c151f90 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0c2b00 con 0x7f509c06f4f0 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0c2de0 con 0x7f509c0b0c40 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50814/0 (socket says 172.21.15.91:50814) 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 -- 172.21.15.91:0/2641844644 learned_addr learned my addr 172.21.15.91:0/2641844644 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1400640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c06f4f0 0x7f509c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a0a00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f509c0b0c40 0x7f509c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 -- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f509c0b0c40 msgr2=0x7f509c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f509c0b0c40 0x7f509c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 -- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c06f4f0 msgr2=0x7f509c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c06f4f0 0x7f509c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.380+0000 7f50a1e00640 1 -- 172.21.15.91:0/2641844644 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0094d0 con 0x7f509c151f90 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f50a1e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f509400c780 tx=0x7f509400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094003b70 con 0x7f509c151f90 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094003cd0 con 0x7f509c151f90 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c0c30c0 con 0x7f509c151f90 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940052b0 con 0x7f509c151f90 2024-02-02T13:29:01.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.381+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c0c3680 con 0x7f509c151f90 2024-02-02T13:29:01.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.382+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c0c38a0 con 0x7f509c151f90 2024-02-02T13:29:01.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.383+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5094005410 con 0x7f509c151f90 2024-02-02T13:29:01.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.384+0000 7f5099600640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 0x7f507406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.384+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5094087d50 con 0x7f509c151f90 2024-02-02T13:29:01.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.384+0000 7f50a1400640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 0x7f507406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.385+0000 7f50a1400640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 0x7f507406c8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f508c036000 tx=0x7f508c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.388+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5094053680 con 0x7f509c151f90 2024-02-02T13:29:01.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.579+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509c066cf0 con 0x7f509c151f90 2024-02-02T13:29:01.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.580+0000 7f5099600640 1 -- 172.21.15.91:0/2641844644 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f5094017020 con 0x7f509c151f90 2024-02-02T13:29:01.582 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:01.582 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":82,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:00.639021+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.582+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 msgr2=0x7f507406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.582+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 0x7f507406c8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f508c036000 tx=0x7f508c008030 comp rx=0 tx=0).stop 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.582+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 msgr2=0x7f509c0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.582+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f509400c780 tx=0x7f509400cc40 comp rx=0 tx=0).stop 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 shutdown_connections 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f507406a430 0x7f507406c8f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f509c0b0c40 0x7f509c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f509c151f90 0x7f509c0c7e90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 --2- 172.21.15.91:0/2641844644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c06f4f0 0x7f509c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 >> 172.21.15.91:0/2641844644 conn(0x7f509c0cec80 msgr2=0x7f509c0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:01.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 shutdown_connections 2024-02-02T13:29:01.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.583+0000 7f50a3e00640 1 -- 172.21.15.91:0/2641844644 wait complete. 2024-02-02T13:29:01.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:29:01.603 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:29:01.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.812+0000 7f9637e00640 1 Processor -- start 2024-02-02T13:29:01.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.812+0000 7f9637e00640 1 -- start start 2024-02-02T13:29:01.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96380add80 0x7f96380706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96380ce3b0 con 0x7f96380c9450 2024-02-02T13:29:01.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96380ce8e0 con 0x7f963815d3a0 2024-02-02T13:29:01.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96380adbd0 con 0x7f96380add80 2024-02-02T13:29:01.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f96380c9450 0x7f96380c98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52748/0 (socket says 172.21.15.91:52748) 2024-02-02T13:29:01.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.814+0000 7f9637400640 1 -- 172.21.15.91:0/3473149439 learned_addr learned my addr 172.21.15.91:0/3473149439 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:01.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f962fe00640 1 --2- 172.21.15.91:0/3473149439 >> v2:172.21.15.113:3300/0 conn(0x7f96380add80 0x7f96380706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120947389 0 0) 0x7f96380ce3b0 con 0x7f96380c9450 2024-02-02T13:29:01.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9618003520 con 0x7f96380c9450 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 713984038 0 0) 0x7f9618003520 con 0x7f96380c9450 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 >> v2:172.21.15.113:3300/0 conn(0x7f96380add80 msgr2=0x7f96380706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 --2- 172.21.15.91:0/3473149439 >> v2:172.21.15.113:3300/0 conn(0x7f96380add80 0x7f96380706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 >> v1:172.21.15.113:6789/0 conn(0x7f963815d3a0 legacy=0x7f963806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963815d780 con 0x7f96380c9450 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f96300019f0 con 0x7f96380c9450 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 >> v1:172.21.15.91:6789/0 conn(0x7f96380c9450 legacy=0x7f96380c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9618003970 0x7f9618003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180047c0 0x7f9618004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 0x7f961800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380adbd0 con 0x7f96180051f0 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961800c150 con 0x7f9618003970 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.815+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961800c310 con 0x7f96180047c0 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 0x7f961800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9636a00640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9618003970 0x7f9618003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f962fe00640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180047c0 0x7f9618004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 -- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180047c0 msgr2=0x7f9618004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180047c0 0x7f9618004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 -- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9618003970 msgr2=0x7f9618003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9618003970 0x7f9618003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9637400640 1 -- 172.21.15.91:0/3473149439 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618003520 con 0x7f96180051f0 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.816+0000 7f9636a00640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9618003970 0x7f9618003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.817+0000 7f9637400640 1 -- 172.21.15.91:0/3473149439 reap_dead start 2024-02-02T13:29:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.817+0000 7f9637400640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 0x7f961800b980 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f963000a830 tx=0x7f96180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354c6b0302da31d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.817+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963002cea0 con 0x7f96180051f0 2024-02-02T13:29:01.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.817+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963002f070 con 0x7f96180051f0 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.817+0000 7f9636000640 1 -- 172.21.15.91:0/3473149439 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630034890 con 0x7f96180051f0 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 -- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 msgr2=0x7f961800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 0x7f961800b980 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f963000a830 tx=0x7f96180057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 -- 172.21.15.91:0/3473149439 shutdown_connections 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 --2- 172.21.15.91:0/3473149439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96180051f0 0x7f961800b980 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 -- 172.21.15.91:0/3473149439 >> 172.21.15.91:0/3473149439 conn(0x7f96380cea90 msgr2=0x7f96380ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 -- 172.21.15.91:0/3473149439 shutdown_connections 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.818+0000 7f9637e00640 1 -- 172.21.15.91:0/3473149439 wait complete. 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.820+0000 7f9637e00640 1 Processor -- start 2024-02-02T13:29:01.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.820+0000 7f9637e00640 1 -- start start 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180051f0 0x7f96380c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96380b3d90 0x7f96380b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380c6bb0 con 0x7f96381f1550 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380c7a10 con 0x7f96380b3d90 2024-02-02T13:29:01.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96380b4750 con 0x7f96180051f0 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50834/0 (socket says 172.21.15.91:50834) 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.821+0000 7f9637400640 1 -- 172.21.15.91:0/1235738033 learned_addr learned my addr 172.21.15.91:0/1235738033 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f962fe00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96380b3d90 0x7f96380b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9636a00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180051f0 0x7f96380c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 -- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180051f0 msgr2=0x7f96380c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180051f0 0x7f96380c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 -- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96380b3d90 msgr2=0x7f96380b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96380b3d90 0x7f96380b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 -- 172.21.15.91:0/1235738033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380c6e00 con 0x7f96381f1550 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637400640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f963000a830 tx=0x7f9630001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630004de0 con 0x7f96381f1550 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630001730 con 0x7f96381f1550 2024-02-02T13:29:01.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96380c70e0 con 0x7f96381f1550 2024-02-02T13:29:01.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.822+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630001ae0 con 0x7f96381f1550 2024-02-02T13:29:01.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.823+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96380c76a0 con 0x7f96381f1550 2024-02-02T13:29:01.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.824+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96380c7ba0 con 0x7f96381f1550 2024-02-02T13:29:01.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.824+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9630031030 con 0x7f96381f1550 2024-02-02T13:29:01.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.825+0000 7f962f400640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 0x7f961006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:01.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.825+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96300a9820 con 0x7f96381f1550 2024-02-02T13:29:01.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.826+0000 7f9636a00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 0x7f961006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:01.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.826+0000 7f9636a00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 0x7f961006c8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96380cfac0 tx=0x7f9628009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:01.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:01.830+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9630075170 con 0x7f96381f1550 2024-02-02T13:29:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.016+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f963804dff0 con 0x7f96381f1550 2024-02-02T13:29:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.017+0000 7f962f400640 1 -- 172.21.15.91:0/1235738033 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9630044020 con 0x7f96381f1550 2024-02-02T13:29:02.019 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 msgr2=0x7f961006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 0x7f961006c8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96380cfac0 tx=0x7f9628009040 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 msgr2=0x7f96381f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f963000a830 tx=0x7f9630001d80 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 shutdown_connections 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f961006a430 0x7f961006c8f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96381f1550 0x7f96381f19d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96380b3d90 0x7f96380b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 --2- 172.21.15.91:0/1235738033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96180051f0 0x7f96380c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 >> 172.21.15.91:0/1235738033 conn(0x7f96380cea90 msgr2=0x7f96380ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 shutdown_connections 2024-02-02T13:29:02.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.020+0000 7f9637e00640 1 -- 172.21.15.91:0/1235738033 wait complete. 2024-02-02T13:29:02.040 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:29:02.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.251+0000 7fd31ea00640 1 Processor -- start 2024-02-02T13:29:02.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.251+0000 7fd31ea00640 1 -- start start 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.252+0000 7fd31ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3181609b0 0x7fd3180cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.252+0000 7fd31ea00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318160d90 con 0x7fd3180ce530 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.252+0000 7fd31ea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318070fe0 con 0x7fd318070980 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.252+0000 7fd31ea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3180d2940 con 0x7fd3181609b0 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.252+0000 7fd317e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3181609b0 0x7fd3180cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.253+0000 7fd317e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3181609b0 0x7fd3180cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50848/0 (socket says 172.21.15.91:50848) 2024-02-02T13:29:02.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.253+0000 7fd317e00640 1 -- 172.21.15.91:0/831012189 learned_addr learned my addr 172.21.15.91:0/831012189 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:02.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.253+0000 7fd317e00640 1 -- 172.21.15.91:0/831012189 >> v1:172.21.15.113:6789/0 conn(0x7fd318070980 legacy=0x7fd3180b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:02.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.253+0000 7fd317e00640 1 -- 172.21.15.91:0/831012189 >> v1:172.21.15.91:6789/0 conn(0x7fd3180ce530 legacy=0x7fd318070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.253+0000 7fd317e00640 1 -- 172.21.15.91:0/831012189 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180d2c20 con 0x7fd3181609b0 2024-02-02T13:29:02.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.254+0000 7fd317e00640 1 --2- 172.21.15.91:0/831012189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cdff0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd308009e50 tx=0x7fd30802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8713bc31a50410e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.254+0000 7fd316a00640 1 -- 172.21.15.91:0/831012189 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30802c9f0 con 0x7fd3181609b0 2024-02-02T13:29:02.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd316a00640 1 -- 172.21.15.91:0/831012189 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30802cb50 con 0x7fd3181609b0 2024-02-02T13:29:02.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd316a00640 1 -- 172.21.15.91:0/831012189 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3080343f0 con 0x7fd3181609b0 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 -- 172.21.15.91:0/831012189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 msgr2=0x7fd3180cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 --2- 172.21.15.91:0/831012189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cdff0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd308009e50 tx=0x7fd30802be50 comp rx=0 tx=0).stop 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 -- 172.21.15.91:0/831012189 shutdown_connections 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 --2- 172.21.15.91:0/831012189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cdff0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 -- 172.21.15.91:0/831012189 >> 172.21.15.91:0/831012189 conn(0x7fd3180d3020 msgr2=0x7fd3180d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 -- 172.21.15.91:0/831012189 shutdown_connections 2024-02-02T13:29:02.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.255+0000 7fd31ea00640 1 -- 172.21.15.91:0/831012189 wait complete. 2024-02-02T13:29:02.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.257+0000 7fd31ea00640 1 Processor -- start 2024-02-02T13:29:02.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.257+0000 7fd31ea00640 1 -- start start 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 0x7fd3181f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3180ce530 0x7fd318151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181f32a0 con 0x7fd3181609b0 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318151b70 con 0x7fd318070980 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181f2920 con 0x7fd3180ce530 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.258+0000 7fd31ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50850/0 (socket says 172.21.15.91:50850) 2024-02-02T13:29:02.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 -- 172.21.15.91:0/2660453799 learned_addr learned my addr 172.21.15.91:0/2660453799 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd317e00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 0x7fd3181f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 -- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3180ce530 msgr2=0x7fd318151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3180ce530 0x7fd318151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 -- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 msgr2=0x7fd3181f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 0x7fd3181f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 -- 172.21.15.91:0/2660453799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308009b30 con 0x7fd3181609b0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd317e00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 0x7fd3181f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd31ca00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd31000b410 tx=0x7fd31000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310004020 con 0x7fd3181609b0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd310004180 con 0x7fd3181609b0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.259+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310011630 con 0x7fd3181609b0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.260+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd318159950 con 0x7fd3181609b0 2024-02-02T13:29:02.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.260+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3181f2ec0 con 0x7fd3181609b0 2024-02-02T13:29:02.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.261+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3181f33e0 con 0x7fd3181609b0 2024-02-02T13:29:02.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.262+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd310011790 con 0x7fd3181609b0 2024-02-02T13:29:02.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.263+0000 7fd30fe00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 0x7fd2f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.263+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3100882f0 con 0x7fd3181609b0 2024-02-02T13:29:02.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.263+0000 7fd317e00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 0x7fd2f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.264+0000 7fd317e00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 0x7fd2f006c8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd308035000 tx=0x7fd308030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.267+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd3181f33e0 con 0x7fd3181609b0 2024-02-02T13:29:02.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.463+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd3181f33e0 con 0x7fd3181609b0 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.464+0000 7fd30fe00640 1 -- 172.21.15.91:0/2660453799 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd3181f33e0 con 0x7fd3181609b0 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:29:02.466 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 msgr2=0x7fd2f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 0x7fd2f006c8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd308035000 tx=0x7fd308030000 comp rx=0 tx=0).stop 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 msgr2=0x7fd3180cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd31000b410 tx=0x7fd31000b8d0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 shutdown_connections 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd2f006a430 0x7fd2f006c8f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3180ce530 0x7fd318151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3181609b0 0x7fd3180cc4e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 --2- 172.21.15.91:0/2660453799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd318070980 0x7fd3181f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.467+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 >> 172.21.15.91:0/2660453799 conn(0x7fd3180d3020 msgr2=0x7fd31815de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.468+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 shutdown_connections 2024-02-02T13:29:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.468+0000 7fd31ea00640 1 -- 172.21.15.91:0/2660453799 wait complete. 2024-02-02T13:29:02.489 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:29:02.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:29:02.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.704+0000 7fada2800640 1 Processor -- start 2024-02-02T13:29:02.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.704+0000 7fada2800640 1 -- start start 2024-02-02T13:29:02.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada2800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad9c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9c0cbab0 0x7fad9c159800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada2800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c0cbf40 con 0x7fad9c06ca20 2024-02-02T13:29:02.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada2800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0ce330 con 0x7fad9c0cbab0 2024-02-02T13:29:02.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada2800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c9910 con 0x7fad9c06ce00 2024-02-02T13:29:02.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada1400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fad9c06ca20 0x7fad9c0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52768/0 (socket says 172.21.15.91:52768) 2024-02-02T13:29:02.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada1400640 1 -- 172.21.15.91:0/3150797882 learned_addr learned my addr 172.21.15.91:0/3150797882 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada0a00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad9c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801179376 0 0) 0x7fad8c001110 con 0x7fad9c06ca20 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fada1e00640 1 --2- 172.21.15.91:0/3150797882 >> v2:172.21.15.113:3300/0 conn(0x7fad9c0cbab0 0x7fad9c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.706+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad7c005730 con 0x7fad9c06ca20 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296351691 0 0) 0x7fad7c005730 con 0x7fad9c06ca20 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 msgr2=0x7fad9c0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad9c0cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 >> v2:172.21.15.113:3300/0 conn(0x7fad9c0cbab0 msgr2=0x7fad9c159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:02.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 --2- 172.21.15.91:0/3150797882 >> v2:172.21.15.113:3300/0 conn(0x7fad9c0cbab0 0x7fad9c159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0cc180 con 0x7fad9c06ca20 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fad8c001800 con 0x7fad9c06ca20 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 >> v1:172.21.15.91:6789/0 conn(0x7fad9c06ca20 legacy=0x7fad9c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c005b80 0x7fad7c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad7c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c006f10 0x7fad7c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0ce330 con 0x7fad7c005b80 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c9910 con 0x7fad7c006f10 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c00dc70 con 0x7fad9c06ce00 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.707+0000 7fada1400640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c005b80 0x7fad7c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1400640 1 -- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 msgr2=0x7fad7c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada0a00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad7c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1e00640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c006f10 0x7fad7c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1400640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c06ce00 0x7fad7c0069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1400640 1 -- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c006f10 msgr2=0x7fad7c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1400640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c006f10 0x7fad7c007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.708+0000 7fada1400640 1 -- 172.21.15.91:0/3150797882 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c005730 con 0x7fad7c005b80 2024-02-02T13:29:02.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fada1400640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c005b80 0x7fad7c005fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c210 tx=0x7fad7c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd920267c39ce8e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c005ad0 con 0x7fad7c005b80 2024-02-02T13:29:02.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c005c50 con 0x7fad7c005b80 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fad9be00640 1 -- 172.21.15.91:0/3150797882 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c02c8d0 con 0x7fad7c005b80 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fada2800640 1 -- 172.21.15.91:0/3150797882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c005b80 msgr2=0x7fad7c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fada2800640 1 --2- 172.21.15.91:0/3150797882 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c005b80 0x7fad7c005fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c210 tx=0x7fad7c007a00 comp rx=0 tx=0).stop 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.709+0000 7fada1e00640 1 -- 172.21.15.91:0/3150797882 reap_dead start 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.710+0000 7fada2800640 1 -- 172.21.15.91:0/3150797882 shutdown_connections 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.710+0000 7fada2800640 1 -- 172.21.15.91:0/3150797882 >> 172.21.15.91:0/3150797882 conn(0x7fad9c0b08e0 msgr2=0x7fad9c0b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.710+0000 7fada2800640 1 -- 172.21.15.91:0/3150797882 shutdown_connections 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.710+0000 7fada2800640 1 -- 172.21.15.91:0/3150797882 wait complete. 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.711+0000 7fada2800640 1 Processor -- start 2024-02-02T13:29:02.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.712+0000 7fada2800640 1 -- start start 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.712+0000 7fada2800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad7c005b80 0x7fad9c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.712+0000 7fada2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c1fa630 0x7fad9c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada2800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1f6b70 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada2800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1fac00 con 0x7fad9c1fa630 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada2800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1faee0 con 0x7fad7c005b80 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50872/0 (socket says 172.21.15.91:50872) 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 -- 172.21.15.91:0/3519623124 learned_addr learned my addr 172.21.15.91:0/3519623124 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:02.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada1e00640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c1fa630 0x7fad9c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 -- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad7c005b80 msgr2=0x7fad9c155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad7c005b80 0x7fad9c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 -- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c1fa630 msgr2=0x7fad9c15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c1fa630 0x7fad9c15bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.713+0000 7fada0a00640 1 -- 172.21.15.91:0/3519623124 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c159790 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fada0a00640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e7e0 tx=0x7fad8400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400c6d0 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8400c830 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c159a70 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400cba0 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.714+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c156fa0 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.715+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c1fb120 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.716+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fad8400cd00 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.717+0000 7fad99600640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 0x7fad7406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.717+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad840883d0 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.717+0000 7fada1400640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 0x7fad7406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.718+0000 7fada1400640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 0x7fad7406c920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad8c034000 tx=0x7fad8c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:02.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.721+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fad84053d10 con 0x7fad9c1fa1b0 2024-02-02T13:29:02.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:02.910+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fad9c1595d0 con 0x7fad9c1fa1b0 2024-02-02T13:29:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.381+0000 7fad99600640 1 -- 172.21.15.91:0/3519623124 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fad84009380 con 0x7fad9c1fa1b0 2024-02-02T13:29:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:29:03.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 msgr2=0x7fad7406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 0x7fad7406c920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad8c034000 tx=0x7fad8c02e000 comp rx=0 tx=0).stop 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 msgr2=0x7fad9c15ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e7e0 tx=0x7fad8400eca0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 shutdown_connections 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fad7406a460 0x7fad7406c920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c1fa630 0x7fad9c15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9c1fa1b0 0x7fad9c15ba50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 --2- 172.21.15.91:0/3519623124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad7c005b80 0x7fad9c155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.386+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 >> 172.21.15.91:0/3519623124 conn(0x7fad9c0b08e0 msgr2=0x7fad9c15c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:03.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.387+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 shutdown_connections 2024-02-02T13:29:03.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.387+0000 7fada2800640 1 -- 172.21.15.91:0/3519623124 wait complete. 2024-02-02T13:29:03.410 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:29:03.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.622+0000 7fc76e400640 1 Processor -- start 2024-02-02T13:29:03.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.622+0000 7fc76e400640 1 -- start start 2024-02-02T13:29:03.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc76e400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 0x7fc76806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc76e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc76e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76815d870 con 0x7fc7680ac8d0 2024-02-02T13:29:03.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc76e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7680ad270 con 0x7fc7680b0e10 2024-02-02T13:29:03.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc76e400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76806bce0 con 0x7fc76815d490 2024-02-02T13:29:03.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc767e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.624+0000 7fc767400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 0x7fc76806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:40550/0 (socket says 172.21.15.91:40550) 2024-02-02T13:29:03.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767e00640 1 -- 172.21.15.91:0/813494293 learned_addr learned my addr 172.21.15.91:0/813494293 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc766000640 1 -- 172.21.15.91:0/813494293 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551733629 0 0) 0x7fc76815d870 con 0x7fc7680ac8d0 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767400640 1 -- 172.21.15.91:0/813494293 >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 msgr2=0x7fc7680acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767400640 1 --2- 172.21.15.91:0/813494293 >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767400640 1 -- 172.21.15.91:0/813494293 >> v1:172.21.15.113:6789/0 conn(0x7fc7680ac8d0 legacy=0x7fc7680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767400640 1 -- 172.21.15.91:0/813494293 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768161220 con 0x7fc76815d490 2024-02-02T13:29:03.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.625+0000 7fc767e00640 1 --2- 172.21.15.91:0/813494293 >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:29:03.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.627+0000 7fc767400640 1 --2- 172.21.15.91:0/813494293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 0x7fc76806c170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89953a9b224a5a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:03.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc766000640 1 -- 172.21.15.91:0/813494293 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75002d040 con 0x7fc76815d490 2024-02-02T13:29:03.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc766000640 1 -- 172.21.15.91:0/813494293 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750031810 con 0x7fc76815d490 2024-02-02T13:29:03.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc766000640 1 -- 172.21.15.91:0/813494293 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750031ba0 con 0x7fc76815d490 2024-02-02T13:29:03.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 -- 172.21.15.91:0/813494293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 msgr2=0x7fc76806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 --2- 172.21.15.91:0/813494293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 0x7fc76806c170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 -- 172.21.15.91:0/813494293 shutdown_connections 2024-02-02T13:29:03.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 --2- 172.21.15.91:0/813494293 >> v2:172.21.15.113:3300/0 conn(0x7fc7680b0e10 0x7fc7680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 --2- 172.21.15.91:0/813494293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc76815d490 0x7fc76806c170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 -- 172.21.15.91:0/813494293 >> 172.21.15.91:0/813494293 conn(0x7fc7680cec80 msgr2=0x7fc7680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:03.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.628+0000 7fc76e400640 1 -- 172.21.15.91:0/813494293 shutdown_connections 2024-02-02T13:29:03.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.629+0000 7fc76e400640 1 -- 172.21.15.91:0/813494293 wait complete. 2024-02-02T13:29:03.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.630+0000 7fc76e400640 1 Processor -- start 2024-02-02T13:29:03.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.630+0000 7fc76e400640 1 -- start start 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7680b0e10 0x7fc7680b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc76815d490 0x7fc7680b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b1870 con 0x7fc7680ac8d0 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b4570 con 0x7fc76815d490 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc76e400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b1b30 con 0x7fc7680b0e10 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.631+0000 7fc767400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50884/0 (socket says 172.21.15.91:50884) 2024-02-02T13:29:03.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 -- 172.21.15.91:0/3983313996 learned_addr learned my addr 172.21.15.91:0/3983313996 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:03.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc766a00640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7680b0e10 0x7fc7680b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 -- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7680b0e10 msgr2=0x7fc7680b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767e00640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc76815d490 0x7fc7680b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7680b0e10 0x7fc7680b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 -- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc76815d490 msgr2=0x7fc7680b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc76815d490 0x7fc7680b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 -- 172.21.15.91:0/3983313996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600094d0 con 0x7fc7680ac8d0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.632+0000 7fc767400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc750000c00 tx=0x7fc7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.633+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750006e10 con 0x7fc7680ac8d0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.633+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75003bae0 con 0x7fc7680ac8d0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.633+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75003be90 con 0x7fc7680ac8d0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.633+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc750009400 con 0x7fc7680ac8d0 2024-02-02T13:29:03.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.633+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7680b20f0 con 0x7fc7680ac8d0 2024-02-02T13:29:03.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.634+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680b22e0 con 0x7fc7680ac8d0 2024-02-02T13:29:03.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.635+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc750043070 con 0x7fc7680ac8d0 2024-02-02T13:29:03.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.636+0000 7fc75f400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 0x7fc74006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:03.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.636+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc7500aaab0 con 0x7fc7680ac8d0 2024-02-02T13:29:03.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.636+0000 7fc766a00640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 0x7fc74006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:03.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.637+0000 7fc766a00640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 0x7fc74006c8f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc768155570 tx=0x7fc758009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:03.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.640+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc750076300 con 0x7fc7680ac8d0 2024-02-02T13:29:03.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:03.832+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc768066cf0 con 0x7fc7680ac8d0 2024-02-02T13:29:04.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.384+0000 7fc75f400640 1 -- 172.21.15.91:0/3983313996 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc750079920 con 0x7fc7680ac8d0 2024-02-02T13:29:04.387 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:29:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.391+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 msgr2=0x7fc74006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.391+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 0x7fc74006c8f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc768155570 tx=0x7fc758009040 comp rx=0 tx=0).stop 2024-02-02T13:29:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.391+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 msgr2=0x7fc7681f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.391+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc750000c00 tx=0x7fc7500046c0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 shutdown_connections 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc74006a430 0x7fc74006c8f0 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc768155570 tx=0x7fc758009040 comp rx=0 tx=0).stop 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7680b0e10 0x7fc7680b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc76815d490 0x7fc7680b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 --2- 172.21.15.91:0/3983313996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7680ac8d0 0x7fc7681f53a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.392+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 >> 172.21.15.91:0/3983313996 conn(0x7fc7680cec80 msgr2=0x7fc768157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.396+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 shutdown_connections 2024-02-02T13:29:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.397+0000 7fc76e400640 1 -- 172.21.15.91:0/3983313996 wait complete. 2024-02-02T13:29:04.423 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:29:04.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.615+0000 7f0323a00640 1 Processor -- start 2024-02-02T13:29:04.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.615+0000 7f0323a00640 1 -- start start 2024-02-02T13:29:04.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0323a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0323a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f031c0b0e10 0x7f031c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0323a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c15d870 con 0x7f031c0ac8d0 2024-02-02T13:29:04.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0323a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c0ad270 con 0x7f031c0b0e10 2024-02-02T13:29:04.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0323a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06bce0 con 0x7f031c15d490 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0321a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f031c0b0e10 0x7f031c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0321000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0321000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:37730/0 (socket says 172.21.15.91:37730) 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.617+0000 7f0321000640 1 -- 172.21.15.91:0/177474679 learned_addr learned my addr 172.21.15.91:0/177474679 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.618+0000 7f0321000640 1 -- 172.21.15.91:0/177474679 >> v2:172.21.15.113:3300/0 conn(0x7f031c0b0e10 msgr2=0x7f031c0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.618+0000 7f0321000640 1 --2- 172.21.15.91:0/177474679 >> v2:172.21.15.113:3300/0 conn(0x7f031c0b0e10 0x7f031c0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.618+0000 7f0321000640 1 -- 172.21.15.91:0/177474679 >> v1:172.21.15.113:6789/0 conn(0x7f031c0ac8d0 legacy=0x7f031c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.620 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.618+0000 7f0321000640 1 -- 172.21.15.91:0/177474679 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c161220 con 0x7f031c15d490 2024-02-02T13:29:04.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.620+0000 7f0321000640 1 --2- 172.21.15.91:0/177474679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f030c009870 tx=0x7f030c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09fef99e5dad979 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:04.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.620+0000 7f031b400640 1 -- 172.21.15.91:0/177474679 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c030840 con 0x7f031c15d490 2024-02-02T13:29:04.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.620+0000 7f031b400640 1 -- 172.21.15.91:0/177474679 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0309a0 con 0x7f031c15d490 2024-02-02T13:29:04.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.620+0000 7f031b400640 1 -- 172.21.15.91:0/177474679 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c030d30 con 0x7f031c15d490 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 -- 172.21.15.91:0/177474679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 msgr2=0x7f031c06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 --2- 172.21.15.91:0/177474679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f030c009870 tx=0x7f030c02b980 comp rx=0 tx=0).stop 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 -- 172.21.15.91:0/177474679 shutdown_connections 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 --2- 172.21.15.91:0/177474679 >> v2:172.21.15.113:3300/0 conn(0x7f031c0b0e10 0x7f031c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 --2- 172.21.15.91:0/177474679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c15d490 0x7f031c06c170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 -- 172.21.15.91:0/177474679 >> 172.21.15.91:0/177474679 conn(0x7f031c0cec80 msgr2=0x7f031c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 -- 172.21.15.91:0/177474679 shutdown_connections 2024-02-02T13:29:04.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.621+0000 7f0323a00640 1 -- 172.21.15.91:0/177474679 wait complete. 2024-02-02T13:29:04.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.623+0000 7f0323a00640 1 Processor -- start 2024-02-02T13:29:04.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.623+0000 7f0323a00640 1 -- start start 2024-02-02T13:29:04.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.624+0000 7f0323a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.624+0000 7f0323a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 0x7f031c0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.624+0000 7f0323a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f031c15d490 0x7f031c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.624+0000 7f0323a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0b3bd0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.624+0000 7f0323a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c15da00 con 0x7f031c15d490 2024-02-02T13:29:04.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0323a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0c2f70 con 0x7f031c0b0e10 2024-02-02T13:29:04.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50894/0 (socket says 172.21.15.91:50894) 2024-02-02T13:29:04.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321000640 1 -- 172.21.15.91:0/540413782 learned_addr learned my addr 172.21.15.91:0/540413782 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f031c15d490 0x7f031c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f031be00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 0x7f031c0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321000640 1 -- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 msgr2=0x7f031c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.625+0000 7f0321000640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 0x7f031c0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0321000640 1 -- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f031c15d490 msgr2=0x7f031c0af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0321000640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f031c15d490 0x7f031c0af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0321000640 1 -- 172.21.15.91:0/540413782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0094d0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f031be00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 0x7f031c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0321000640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f030c02b9c0 tx=0x7f030c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c03e040 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c039cb0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c044630 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c0c3250 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.626+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c0c3810 con 0x7f031c0ac8d0 2024-02-02T13:29:04.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.628+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c066cf0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.629+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f030c043390 con 0x7f031c0ac8d0 2024-02-02T13:29:04.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.630+0000 7f0318c00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 0x7f02f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:04.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.630+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f030c0aad50 con 0x7f031c0ac8d0 2024-02-02T13:29:04.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.630+0000 7f031be00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 0x7f02f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:04.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.631+0000 7f031be00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 0x7f02f406c8f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0304007fd0 tx=0x7f0304009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:04.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.639+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f030c076300 con 0x7f031c0ac8d0 2024-02-02T13:29:04.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.866+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f031c15ccb0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.888 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 10 and data pool 11 2024-02-02T13:29:04.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.880+0000 7f0318c00640 1 -- 172.21.15.91:0/540413782 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f030c0340c0 con 0x7f031c0ac8d0 2024-02-02T13:29:04.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 msgr2=0x7f02f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 0x7f02f406c8f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0304007fd0 tx=0x7f0304009040 comp rx=0 tx=0).stop 2024-02-02T13:29:04.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 msgr2=0x7f031c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:04.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f030c02b9c0 tx=0x7f030c004730 comp rx=0 tx=0).stop 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 shutdown_connections 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f02f406a430 0x7f02f406c8f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f031c0b0e10 0x7f031c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f031c15d490 0x7f031c0af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 --2- 172.21.15.91:0/540413782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f031c0ac8d0 0x7f031c0b28c0 secure :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f030c02b9c0 tx=0x7f030c004730 comp rx=0 tx=0).stop 2024-02-02T13:29:04.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.889+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 >> 172.21.15.91:0/540413782 conn(0x7f031c0cec80 msgr2=0x7f031c06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:04.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.890+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 shutdown_connections 2024-02-02T13:29:04.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:04.890+0000 7f0323a00640 1 -- 172.21.15.91:0/540413782 wait complete. 2024-02-02T13:29:04.913 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:05.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.087+0000 7f8a75200640 1 Processor -- start 2024-02-02T13:29:05.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.087+0000 7f8a75200640 1 -- start start 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a75200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a7015a010 0x7f8a700c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a75200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a700ce130 0x7f8a700ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a75200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a700cc7c0 con 0x7f8a700cd470 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a75200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700ce930 con 0x7f8a7015a010 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a75200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7006c890 con 0x7f8a700ce130 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a6e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8a700cd470 0x7f8a700b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52778/0 (socket says 172.21.15.91:52778) 2024-02-02T13:29:05.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.088+0000 7f8a6e800640 1 -- 172.21.15.91:0/224777099 learned_addr learned my addr 172.21.15.91:0/224777099 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6de00640 1 --2- 172.21.15.91:0/224777099 >> v2:172.21.15.113:3300/0 conn(0x7f8a7015a010 0x7f8a700c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090556243 0 0) 0x7f8a700cc7c0 con 0x7f8a700cd470 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a65200640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a700ce130 0x7f8a700ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a50005730 con 0x7f8a700cd470 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 937121389 0 0) 0x7f8a50005730 con 0x7f8a700cd470 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a700ce130 msgr2=0x7f8a700ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a700ce130 0x7f8a700ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 >> v2:172.21.15.113:3300/0 conn(0x7f8a7015a010 msgr2=0x7f8a700c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 --2- 172.21.15.91:0/224777099 >> v2:172.21.15.113:3300/0 conn(0x7f8a7015a010 0x7f8a700c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7006cb70 con 0x7f8a700cd470 2024-02-02T13:29:05.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8a680019f0 con 0x7f8a700cd470 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 >> v1:172.21.15.91:6789/0 conn(0x7f8a700cd470 legacy=0x7f8a700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a50005b80 0x7f8a50005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a500069d0 0x7f8a50006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a5000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ce930 con 0x7f8a50007390 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7006c890 con 0x7f8a500069d0 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5000e150 con 0x7f8a50005b80 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.089+0000 7f8a65200640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a700ce130 0x7f8a700ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a5000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 -- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a50005b80 msgr2=0x7f8a50005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a50005b80 0x7f8a50005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 -- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a500069d0 msgr2=0x7f8a50006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a500069d0 0x7f8a50006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.090+0000 7f8a6e800640 1 -- 172.21.15.91:0/224777099 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50005730 con 0x7f8a50007390 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.091+0000 7f8a6e800640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a5000db90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cec30 tx=0x7f8a50007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fe37b045fd2dd9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.091+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68034630 con 0x7f8a50007390 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.091+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6802f070 con 0x7f8a50007390 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.091+0000 7f8a6d400640 1 -- 172.21.15.91:0/224777099 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a680349f0 con 0x7f8a50007390 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 -- 172.21.15.91:0/224777099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 msgr2=0x7f8a5000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 --2- 172.21.15.91:0/224777099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a5000db90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cec30 tx=0x7f8a50007a00 comp rx=0 tx=0).stop 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a6e800640 1 -- 172.21.15.91:0/224777099 reap_dead start 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 -- 172.21.15.91:0/224777099 shutdown_connections 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 -- 172.21.15.91:0/224777099 >> 172.21.15.91:0/224777099 conn(0x7f8a700cec80 msgr2=0x7f8a700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:05.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 -- 172.21.15.91:0/224777099 shutdown_connections 2024-02-02T13:29:05.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.092+0000 7f8a75200640 1 -- 172.21.15.91:0/224777099 wait complete. 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.094+0000 7f8a75200640 1 Processor -- start 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.094+0000 7f8a75200640 1 -- start start 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701f6440 0x7f8a7015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7015bc10 0x7f8a7014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700c4fb0 con 0x7f8a50007390 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701f68c0 con 0x7f8a701f6440 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a75200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015c090 con 0x7f8a7015bc10 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a6de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a6de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50912/0 (socket says 172.21.15.91:50912) 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.095+0000 7f8a6de00640 1 -- 172.21.15.91:0/1623297191 learned_addr learned my addr 172.21.15.91:0/1623297191 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 -- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7015bc10 msgr2=0x7f8a7014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7015bc10 0x7f8a7014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 -- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701f6440 msgr2=0x7f8a7015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701f6440 0x7f8a7015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 -- 172.21.15.91:0/1623297191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7015c350 con 0x7f8a50007390 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a6de00640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e7c0 tx=0x7f8a6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000eed0 con 0x7f8a50007390 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000c8c0 con 0x7f8a50007390 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000cc70 con 0x7f8a50007390 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7015c630 con 0x7f8a50007390 2024-02-02T13:29:05.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.096+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7015cbf0 con 0x7f8a50007390 2024-02-02T13:29:05.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.097+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70066cf0 con 0x7f8a50007390 2024-02-02T13:29:05.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.099+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8a60013470 con 0x7f8a50007390 2024-02-02T13:29:05.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.099+0000 7f8a66a00640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 0x7f8a4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.099+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a60088a60 con 0x7f8a50007390 2024-02-02T13:29:05.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.100+0000 7f8a65200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 0x7f8a4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.100+0000 7f8a65200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 0x7f8a4806c8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.104+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8a60053fb0 con 0x7f8a50007390 2024-02-02T13:29:05.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.294+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a700b4c00 con 0x7f8a50007390 2024-02-02T13:29:05.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.297+0000 7f8a66a00640 1 -- 172.21.15.91:0/1623297191 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16113 (secure 0 0 0) 0x7f8a60017090 con 0x7f8a50007390 2024-02-02T13:29:05.299 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:05.299 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":85,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:04.867936+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:05.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 msgr2=0x7f8a4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 0x7f8a4806c8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 msgr2=0x7f8a701f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e7c0 tx=0x7f8a6000ec80 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 shutdown_connections 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a4806a430 0x7f8a4806c8f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7015bc10 0x7f8a7014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701f6440 0x7f8a7015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 --2- 172.21.15.91:0/1623297191 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a50007390 0x7f8a701f5f00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.299+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 >> 172.21.15.91:0/1623297191 conn(0x7f8a700cec80 msgr2=0x7f8a700c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.300+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 shutdown_connections 2024-02-02T13:29:05.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.300+0000 7f8a75200640 1 -- 172.21.15.91:0/1623297191 wait complete. 2024-02-02T13:29:05.320 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:05.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.531+0000 7fc1bb400640 1 Processor -- start 2024-02-02T13:29:05.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.531+0000 7fc1bb400640 1 -- start start 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.532+0000 7fc1bb400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ce130 0x7fc1b40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.532+0000 7fc1bb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc1b40b0700 0x7fc1b40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1bb400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40cc7c0 con 0x7fc1b415a010 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1bb400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40b10d0 con 0x7fc1b40b0700 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1bb400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40ce930 con 0x7fc1b40ce130 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1b9400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc1b40b0700 0x7fc1b40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1b9400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc1b40b0700 0x7fc1b40b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50926/0 (socket says 172.21.15.91:50926) 2024-02-02T13:29:05.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1b9400640 1 -- 172.21.15.91:0/1196604954 learned_addr learned my addr 172.21.15.91:0/1196604954 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:05.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.533+0000 7fc1b3e00640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ce130 0x7fc1b40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.534+0000 7fc1b9400640 1 -- 172.21.15.91:0/1196604954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ce130 msgr2=0x7fc1b40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.534+0000 7fc1b9400640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ce130 0x7fc1b40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.534+0000 7fc1b9400640 1 -- 172.21.15.91:0/1196604954 >> v1:172.21.15.113:6789/0 conn(0x7fc1b415a010 legacy=0x7fc1b40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.534+0000 7fc1b9400640 1 -- 172.21.15.91:0/1196604954 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40cd640 con 0x7fc1b40b0700 2024-02-02T13:29:05.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.535+0000 7fc1b9400640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40b0700 0x7fc1b40b0b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009e50 tx=0x7fc1ac02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f910d5781c1e2ff3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.535+0000 7fc1b3400640 1 -- 172.21.15.91:0/1196604954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac02caf0 con 0x7fc1b40b0700 2024-02-02T13:29:05.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.535+0000 7fc1b3400640 1 -- 172.21.15.91:0/1196604954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac02cc50 con 0x7fc1b40b0700 2024-02-02T13:29:05.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.535+0000 7fc1b3400640 1 -- 172.21.15.91:0/1196604954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0343f0 con 0x7fc1b40b0700 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 -- 172.21.15.91:0/1196604954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40b0700 msgr2=0x7fc1b40b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40b0700 0x7fc1b40b0b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009e50 tx=0x7fc1ac02bf50 comp rx=0 tx=0).stop 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 -- 172.21.15.91:0/1196604954 shutdown_connections 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40b0700 0x7fc1b40b0b90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 --2- 172.21.15.91:0/1196604954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ce130 0x7fc1b40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 -- 172.21.15.91:0/1196604954 >> 172.21.15.91:0/1196604954 conn(0x7fc1b40cec80 msgr2=0x7fc1b40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 -- 172.21.15.91:0/1196604954 shutdown_connections 2024-02-02T13:29:05.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.536+0000 7fc1bb400640 1 -- 172.21.15.91:0/1196604954 wait complete. 2024-02-02T13:29:05.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.538+0000 7fc1bb400640 1 Processor -- start 2024-02-02T13:29:05.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.538+0000 7fc1bb400640 1 -- start start 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b40b0700 0x7fc1b4146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b415a010 0x7fc1b40b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40b2650 con 0x7fc1b40ce130 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40b1d40 con 0x7fc1b40b0700 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1bb400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40b3910 con 0x7fc1b415a010 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50928/0 (socket says 172.21.15.91:50928) 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 -- 172.21.15.91:0/2735273517 learned_addr learned my addr 172.21.15.91:0/2735273517 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 -- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b415a010 msgr2=0x7fc1b40b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b415a010 0x7fc1b40b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 -- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b40b0700 msgr2=0x7fc1b4146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b8a00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b40b0700 0x7fc1b4146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b40b0700 0x7fc1b4146090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.539+0000 7fc1b3e00640 1 -- 172.21.15.91:0/2735273517 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009b30 con 0x7fc1b40ce130 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1b3e00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9e0 tx=0x7fc19c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c009670 con 0x7fc1b40ce130 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00c9d0 con 0x7fc1b40ce130 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b40b1f90 con 0x7fc1b40ce130 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00cd80 con 0x7fc1b40ce130 2024-02-02T13:29:05.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.540+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b40b2a70 con 0x7fc1b40ce130 2024-02-02T13:29:05.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.541+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b4066cf0 con 0x7fc1b40ce130 2024-02-02T13:29:05.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.542+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc19c013470 con 0x7fc1b40ce130 2024-02-02T13:29:05.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.544+0000 7fc1b0c00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 0x7fc18c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.544+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc19c088b20 con 0x7fc1b40ce130 2024-02-02T13:29:05.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.544+0000 7fc1b8a00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 0x7fc18c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.544+0000 7fc1b8a00640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 0x7fc18c06c8f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000f60 tx=0x7fc1a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.548+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc19c0541a0 con 0x7fc1b40ce130 2024-02-02T13:29:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.722+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1b4159830 con 0x7fc1b40ce130 2024-02-02T13:29:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.725+0000 7fc1b0c00640 1 -- 172.21.15.91:0/2735273517 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16113 (secure 0 0 0) 0x7fc19c017090 con 0x7fc1b40ce130 2024-02-02T13:29:05.727 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:05.728 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":85,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:04.867936+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 msgr2=0x7fc18c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 0x7fc18c06c8f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000f60 tx=0x7fc1a4009040 comp rx=0 tx=0).stop 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 msgr2=0x7fc1b40b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9e0 tx=0x7fc19c00eea0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 shutdown_connections 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc18c06a430 0x7fc18c06c8f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b415a010 0x7fc1b40b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc1b40ce130 0x7fc1b40b2fa0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 --2- 172.21.15.91:0/2735273517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b40b0700 0x7fc1b4146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 >> 172.21.15.91:0/2735273517 conn(0x7fc1b40cec80 msgr2=0x7fc1b40aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:05.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 shutdown_connections 2024-02-02T13:29:05.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.728+0000 7fc1bb400640 1 -- 172.21.15.91:0/2735273517 wait complete. 2024-02-02T13:29:05.747 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:29:05.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.955+0000 7f6b82e00640 1 Processor -- start 2024-02-02T13:29:05.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.955+0000 7f6b82e00640 1 -- start start 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b82e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6b7c0b2360 0x7f6b7c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b82e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b82e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c0d2940 con 0x7f6b7c0d2bc0 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b82e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cd8f0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b82e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0d1c70 con 0x7f6b7c070980 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b7be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6b7c0b2360 0x7f6b7c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b7b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6b7c0d2bc0 0x7f6b7c070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52780/0 (socket says 172.21.15.91:52780) 2024-02-02T13:29:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.957+0000 7f6b7b400640 1 -- 172.21.15.91:0/2362954415 learned_addr learned my addr 172.21.15.91:0/2362954415 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b80e00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b7be00640 1 -- 172.21.15.91:0/2362954415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 msgr2=0x7f6b7c0b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b7be00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c0b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b7be00640 1 -- 172.21.15.91:0/2362954415 >> v1:172.21.15.91:6789/0 conn(0x7f6b7c0d2bc0 legacy=0x7f6b7c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b7be00640 1 -- 172.21.15.91:0/2362954415 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c161ba0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b80e00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.958+0000 7f6b7aa00640 1 -- 172.21.15.91:0/2362954415 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209685529 0 0) 0x7f6b7c0d2940 con 0x7f6b7c0d2bc0 2024-02-02T13:29:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.959+0000 7f6b7be00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0ce060 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009d90 tx=0x7f6b6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b701dd735adee6cc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.959+0000 7f6b7aa00640 1 -- 172.21.15.91:0/2362954415 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c02cc50 con 0x7f6b7c0b2360 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.959+0000 7f6b7aa00640 1 -- 172.21.15.91:0/2362954415 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c02cdb0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.959+0000 7f6b7aa00640 1 -- 172.21.15.91:0/2362954415 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c034660 con 0x7f6b7c0b2360 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 -- 172.21.15.91:0/2362954415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 msgr2=0x7f6b7c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0ce060 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009d90 tx=0x7f6b6c02bea0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 -- 172.21.15.91:0/2362954415 shutdown_connections 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 --2- 172.21.15.91:0/2362954415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0ce060 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 -- 172.21.15.91:0/2362954415 >> 172.21.15.91:0/2362954415 conn(0x7f6b7c0d3000 msgr2=0x7f6b7c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:05.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 -- 172.21.15.91:0/2362954415 shutdown_connections 2024-02-02T13:29:05.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.960+0000 7f6b82e00640 1 -- 172.21.15.91:0/2362954415 wait complete. 2024-02-02T13:29:05.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.962+0000 7f6b82e00640 1 Processor -- start 2024-02-02T13:29:05.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.962+0000 7f6b82e00640 1 -- start start 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c0d2bc0 0x7f6b7c0b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cc9a0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ccc80 con 0x7f6b7c0d2bc0 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b82e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1590f0 con 0x7f6b7c070980 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.963+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50950/0 (socket says 172.21.15.91:50950) 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 -- 172.21.15.91:0/3793728970 learned_addr learned my addr 172.21.15.91:0/3793728970 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b80e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c0d2bc0 0x7f6b7c0b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7be00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 -- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 msgr2=0x7f6b7c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 -- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c0d2bc0 msgr2=0x7f6b7c0b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c0d2bc0 0x7f6b7c0b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 -- 172.21.15.91:0/3793728970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009a70 con 0x7f6b7c0b2360 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b7b400640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c430 tx=0x7f6b6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b640074e0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b64007660 con 0x7f6b7c0b2360 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64004a20 con 0x7f6b7c0b2360 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.964+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c1593d0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.965+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c0cc600 con 0x7f6b7c0b2360 2024-02-02T13:29:05.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.966+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c1598f0 con 0x7f6b7c0b2360 2024-02-02T13:29:05.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.967+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6b64004b80 con 0x7f6b7c0b2360 2024-02-02T13:29:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.968+0000 7f6b73e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 0x7f6b5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.968+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b64088760 con 0x7f6b7c0b2360 2024-02-02T13:29:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.968+0000 7f6b7be00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 0x7f6b5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:05.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.968+0000 7f6b7be00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 0x7f6b5406c8f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c036000 tx=0x7f6b6c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:05.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:05.972+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6b7c1598f0 con 0x7f6b7c0b2360 2024-02-02T13:29:06.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.197+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b7c1598f0 con 0x7f6b7c0b2360 2024-02-02T13:29:06.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.899+0000 7f6b73e00640 1 -- 172.21.15.91:0/3793728970 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b7c1598f0 con 0x7f6b7c0b2360 2024-02-02T13:29:06.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 msgr2=0x7f6b5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:06.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 0x7f6b5406c8f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c036000 tx=0x7f6b6c030000 comp rx=0 tx=0).stop 2024-02-02T13:29:06.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 msgr2=0x7f6b7c0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:06.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c430 tx=0x7f6b6400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:29:06.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 shutdown_connections 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6b5406a430 0x7f6b5406c8f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b7c0d2bc0 0x7f6b7c0b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b7c0b2360 0x7f6b7c0cbb30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 --2- 172.21.15.91:0/3793728970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b7c070980 0x7f6b7c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.903+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 >> 172.21.15.91:0/3793728970 conn(0x7f6b7c0d3000 msgr2=0x7f6b7c070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.904+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 shutdown_connections 2024-02-02T13:29:06.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:06.904+0000 7f6b82e00640 1 -- 172.21.15.91:0/3793728970 wait complete. 2024-02-02T13:29:06.926 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:29:06.927 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:29:07.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.140+0000 7f95ac200640 1 Processor -- start 2024-02-02T13:29:07.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.140+0000 7f95ac200640 1 -- start start 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.141+0000 7f95ac200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95a40b1a00 0x7f95a40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.141+0000 7f95ac200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 0x7f95a40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.141+0000 7f95ac200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40b1e60 con 0x7f95a415b2f0 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.141+0000 7f95ac200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a415fdc0 con 0x7f95a40b1a00 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.141+0000 7f95ac200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40b1060 con 0x7f95a40b0bd0 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95a9800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f95a415b2f0 0x7f95a40711f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52782/0 (socket says 172.21.15.91:52782) 2024-02-02T13:29:07.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95a9800640 1 -- 172.21.15.91:0/481849029 learned_addr learned my addr 172.21.15.91:0/481849029 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95a8e00640 1 --2- 172.21.15.91:0/481849029 >> v2:172.21.15.113:3300/0 conn(0x7f95a40b1a00 0x7f95a40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95aa200640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 0x7f95a40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126896523 0 0) 0x7f95a40b1e60 con 0x7f95a415b2f0 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.142+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9584003520 con 0x7f95a415b2f0 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253298256 0 0) 0x7f9584003520 con 0x7f95a415b2f0 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 msgr2=0x7f95a40c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:07.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 0x7f95a40c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 >> v2:172.21.15.113:3300/0 conn(0x7f95a40b1a00 msgr2=0x7f95a40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 --2- 172.21.15.91:0/481849029 >> v2:172.21.15.113:3300/0 conn(0x7f95a40b1a00 0x7f95a40b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a415f1b0 con 0x7f95a415b2f0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f95940019f0 con 0x7f95a415b2f0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 >> v1:172.21.15.91:6789/0 conn(0x7f95a415b2f0 legacy=0x7f95a40711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9584003970 0x7f9584003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95840047c0 0x7f9584004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 0x7f9584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a415fdc0 con 0x7f9584003970 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40b1060 con 0x7f95840047c0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958400b9b0 con 0x7f95a40b0bd0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a9800640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9584003970 0x7f9584003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a8e00640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95840047c0 0x7f9584004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.143+0000 7f95a9800640 1 -- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 msgr2=0x7f9584005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:07.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.144+0000 7f95a9800640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a40b0bd0 0x7f9584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.144+0000 7f95a9800640 1 -- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95840047c0 msgr2=0x7f9584004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:07.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.144+0000 7f95a9800640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95840047c0 0x7f9584004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.144+0000 7f95a9800640 1 -- 172.21.15.91:0/481849029 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584003520 con 0x7f9584003970 2024-02-02T13:29:07.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95a9800640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9584003970 0x7f9584003d90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f959400a8d0 tx=0x7f95840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d5a0bbd7041526 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:07.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594034630 con 0x7f9584003970 2024-02-02T13:29:07.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959402f070 con 0x7f9584003970 2024-02-02T13:29:07.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95a3e00640 1 -- 172.21.15.91:0/481849029 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95940349d0 con 0x7f9584003970 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95ac200640 1 -- 172.21.15.91:0/481849029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9584003970 msgr2=0x7f9584003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.145+0000 7f95ac200640 1 --2- 172.21.15.91:0/481849029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9584003970 0x7f9584003d90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f959400a8d0 tx=0x7f95840057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.146+0000 7f95aa200640 1 -- 172.21.15.91:0/481849029 reap_dead start 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.146+0000 7f95ac200640 1 -- 172.21.15.91:0/481849029 shutdown_connections 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.146+0000 7f95ac200640 1 -- 172.21.15.91:0/481849029 >> 172.21.15.91:0/481849029 conn(0x7f95a40ceb90 msgr2=0x7f95a40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.146+0000 7f95ac200640 1 -- 172.21.15.91:0/481849029 shutdown_connections 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.146+0000 7f95ac200640 1 -- 172.21.15.91:0/481849029 wait complete. 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.148+0000 7f95ac200640 1 Processor -- start 2024-02-02T13:29:07.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.148+0000 7f95ac200640 1 -- start start 2024-02-02T13:29:07.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9584003970 0x7f95a40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a415e000 0x7f95a4151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40c9df0 con 0x7f95a415cac0 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40c62d0 con 0x7f9584003970 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95ac200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a415b980 con 0x7f95a415e000 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50954/0 (socket says 172.21.15.91:50954) 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 -- 172.21.15.91:0/3211025719 learned_addr learned my addr 172.21.15.91:0/3211025719 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95a9800640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9584003970 0x7f95a40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 -- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a415e000 msgr2=0x7f95a4151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a415e000 0x7f95a4151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 -- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9584003970 msgr2=0x7f95a40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9584003970 0x7f95a40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.149+0000 7f95aa200640 1 -- 172.21.15.91:0/3211025719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4151590 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95aa200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f95a40ae640 tx=0x7f959c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00c680 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00c7e0 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00cb90 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a40c6700 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.150+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a40c6cc0 con 0x7f95a415cac0 2024-02-02T13:29:07.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.151+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a4066cf0 con 0x7f95a415cac0 2024-02-02T13:29:07.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.152+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f959c00ccf0 con 0x7f95a415cac0 2024-02-02T13:29:07.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.153+0000 7f95a1600640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 0x7f957c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:07.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.153+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f959c088680 con 0x7f95a415cac0 2024-02-02T13:29:07.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.153+0000 7f95a9800640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 0x7f957c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:07.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.154+0000 7f95a9800640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 0x7f957c06c8f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9594034000 tx=0x7f9594009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.157+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f959c053c60 con 0x7f95a415cac0 2024-02-02T13:29:07.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.382+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f95a415b6d0 con 0x7f95a415cac0 2024-02-02T13:29:07.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.909+0000 7f95a1600640 1 -- 172.21.15.91:0/3211025719 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f959c057280 con 0x7f95a415cac0 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 msgr2=0x7f957c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 0x7f957c06c8f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9594034000 tx=0x7f9594009030 comp rx=0 tx=0).stop 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 msgr2=0x7f95a40ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f95a40ae640 tx=0x7f959c00ec00 comp rx=0 tx=0).stop 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 shutdown_connections 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f957c06a430 0x7f957c06c8f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95a415e000 0x7f95a4151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95a415cac0 0x7f95a40ad1e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 --2- 172.21.15.91:0/3211025719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9584003970 0x7f95a40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:07.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 >> 172.21.15.91:0/3211025719 conn(0x7f95a40ceb90 msgr2=0x7f95a406f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:07.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 shutdown_connections 2024-02-02T13:29:07.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:07.913+0000 7f95ac200640 1 -- 172.21.15.91:0/3211025719 wait complete. 2024-02-02T13:29:07.934 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:29:07.934 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:29:08.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.132+0000 7f3832e00640 1 Processor -- start 2024-02-02T13:29:08.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.132+0000 7f3832e00640 1 -- start start 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.133+0000 7f3832e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 0x7f382c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.133+0000 7f3832e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f382c0b0e10 0x7f382c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.133+0000 7f3832e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c15d870 con 0x7f382c0ac8d0 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.133+0000 7f3832e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c0ad2a0 con 0x7f382c0b0e10 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.133+0000 7f3832e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c06bce0 con 0x7f382c15d490 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3830e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f382c0b0e10 0x7f382c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3823200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f382c0ac8d0 0x7f382c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52786/0 (socket says 172.21.15.91:52786) 2024-02-02T13:29:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3823200640 1 -- 172.21.15.91:0/2408754834 learned_addr learned my addr 172.21.15.91:0/2408754834 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f382be00640 1 --2- 172.21.15.91:0/2408754834 >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 0x7f382c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3830e00640 1 -- 172.21.15.91:0/2408754834 >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 msgr2=0x7f382c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3830e00640 1 --2- 172.21.15.91:0/2408754834 >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 0x7f382c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3830e00640 1 -- 172.21.15.91:0/2408754834 >> v1:172.21.15.91:6789/0 conn(0x7f382c0ac8d0 legacy=0x7f382c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f382b400640 1 -- 172.21.15.91:0/2408754834 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929373363 0 0) 0x7f382c15d870 con 0x7f382c0ac8d0 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f3830e00640 1 -- 172.21.15.91:0/2408754834 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c161220 con 0x7f382c0b0e10 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.134+0000 7f382be00640 1 --2- 172.21.15.91:0/2408754834 >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 0x7f382c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.135+0000 7f3830e00640 1 --2- 172.21.15.91:0/2408754834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c0b0e10 0x7f382c0acd60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f382c0ae6b0 tx=0x7f382402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d886b6720650818b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f382b400640 1 -- 172.21.15.91:0/2408754834 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382402cc80 con 0x7f382c0b0e10 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f382b400640 1 -- 172.21.15.91:0/2408754834 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382402cde0 con 0x7f382c0b0e10 2024-02-02T13:29:08.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f382b400640 1 -- 172.21.15.91:0/2408754834 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824034660 con 0x7f382c0b0e10 2024-02-02T13:29:08.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 -- 172.21.15.91:0/2408754834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c0b0e10 msgr2=0x7f382c0acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 --2- 172.21.15.91:0/2408754834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c0b0e10 0x7f382c0acd60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f382c0ae6b0 tx=0x7f382402bed0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 -- 172.21.15.91:0/2408754834 shutdown_connections 2024-02-02T13:29:08.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 --2- 172.21.15.91:0/2408754834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c0b0e10 0x7f382c0acd60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 --2- 172.21.15.91:0/2408754834 >> v2:172.21.15.113:3300/0 conn(0x7f382c15d490 0x7f382c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.136+0000 7f3832e00640 1 -- 172.21.15.91:0/2408754834 >> 172.21.15.91:0/2408754834 conn(0x7f382c0cec80 msgr2=0x7f382c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:08.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.137+0000 7f3832e00640 1 -- 172.21.15.91:0/2408754834 shutdown_connections 2024-02-02T13:29:08.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.137+0000 7f3832e00640 1 -- 172.21.15.91:0/2408754834 wait complete. 2024-02-02T13:29:08.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.139+0000 7f3832e00640 1 Processor -- start 2024-02-02T13:29:08.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.139+0000 7f3832e00640 1 -- start start 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 0x7f382c0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c0b0e10 0x7f382c1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c2c60 con 0x7f382c15d490 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c159520 con 0x7f382c0ac8d0 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3832e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c159800 con 0x7f382c0b0e10 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3830e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3830e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:50984/0 (socket says 172.21.15.91:50984) 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3830e00640 1 -- 172.21.15.91:0/2546305478 learned_addr learned my addr 172.21.15.91:0/2546305478 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3823200640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c0b0e10 0x7f382c1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f382be00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 0x7f382c0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3830e00640 1 -- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c0b0e10 msgr2=0x7f382c1f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.140+0000 7f3830e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c0b0e10 0x7f382c1f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3830e00640 1 -- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 msgr2=0x7f382c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3830e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 0x7f382c0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3830e00640 1 -- 172.21.15.91:0/2546305478 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3824009a70 con 0x7f382c15d490 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f382be00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 0x7f382c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3830e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3824009e80 tx=0x7f3824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824040650 con 0x7f382c15d490 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240407b0 con 0x7f382c15d490 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c159ae0 con 0x7f382c15d490 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824040b60 con 0x7f382c15d490 2024-02-02T13:29:08.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.141+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c15a0a0 con 0x7f382c15d490 2024-02-02T13:29:08.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.143+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c0ad770 con 0x7f382c15d490 2024-02-02T13:29:08.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.144+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3824040cc0 con 0x7f382c15d490 2024-02-02T13:29:08.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.144+0000 7f3828c00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.145+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38240ab720 con 0x7f382c15d490 2024-02-02T13:29:08.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.145+0000 7f382be00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.145+0000 7f382be00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.150+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3824076dd0 con 0x7f382c15d490 2024-02-02T13:29:08.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.357+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382c066cf0 con 0x7f382c15d490 2024-02-02T13:29:08.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.358+0000 7f3828c00640 1 -- 172.21.15.91:0/2546305478 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f382407a3f0 con 0x7f382c15d490 2024-02-02T13:29:08.361 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:08.361 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:29:08.361 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 msgr2=0x7f380406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).stop 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 msgr2=0x7f382c0b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3824009e80 tx=0x7f3824004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 shutdown_connections 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c0b0e10 0x7f382c1f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f382c15d490 0x7f382c0b29c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 --2- 172.21.15.91:0/2546305478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c0ac8d0 0x7f382c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 >> 172.21.15.91:0/2546305478 conn(0x7f382c0cec80 msgr2=0x7f382c1582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:08.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.361+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 shutdown_connections 2024-02-02T13:29:08.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.362+0000 7f3832e00640 1 -- 172.21.15.91:0/2546305478 wait complete. 2024-02-02T13:29:08.381 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:08.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.576+0000 7f3712400640 1 Processor -- start 2024-02-02T13:29:08.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.577+0000 7f3712400640 1 -- start start 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.578+0000 7f3712400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f370c0b0e10 0x7f370c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.578+0000 7f3712400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c070f80 con 0x7f370c15f5a0 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.578+0000 7f3712400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c06c600 con 0x7f370c151f90 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.578+0000 7f3712400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c15fa30 con 0x7f370c0b0e10 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.578+0000 7f3702800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f370c15f5a0 0x7f370c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52790/0 (socket says 172.21.15.91:52790) 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f3702800640 1 -- 172.21.15.91:0/3040547917 learned_addr learned my addr 172.21.15.91:0/3040547917 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:08.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370be00640 1 --2- 172.21.15.91:0/3040547917 >> v2:172.21.15.113:3300/0 conn(0x7f370c0b0e10 0x7f370c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334181167 0 0) 0x7f370c070f80 con 0x7f370c15f5a0 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec003520 con 0x7f370c15f5a0 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 411858929 0 0) 0x7f36ec003520 con 0x7f370c15f5a0 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 >> v2:172.21.15.113:3300/0 conn(0x7f370c0b0e10 msgr2=0x7f370c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 --2- 172.21.15.91:0/3040547917 >> v2:172.21.15.113:3300/0 conn(0x7f370c0b0e10 0x7f370c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 >> v1:172.21.15.113:6789/0 conn(0x7f370c151f90 legacy=0x7f370c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c152370 con 0x7f370c15f5a0 2024-02-02T13:29:08.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f36f4001670 con 0x7f370c15f5a0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 >> v1:172.21.15.91:6789/0 conn(0x7f370c15f5a0 legacy=0x7f370c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36ec003970 0x7f36ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f36ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.579+0000 7f370aa00640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36ec005180 0x7f36ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15fa30 con 0x7f36ec0047c0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec00c020 con 0x7f36ec005180 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec00c1e0 con 0x7f36ec003970 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f36ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370be00640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36ec005180 0x7f36ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370b400640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36ec003970 0x7f36ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 -- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36ec003970 msgr2=0x7f36ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36ec003970 0x7f36ec003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 -- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36ec005180 msgr2=0x7f36ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36ec005180 0x7f36ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f3702800640 1 -- 172.21.15.91:0/3040547917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec003520 con 0x7f36ec0047c0 2024-02-02T13:29:08.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370be00640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36ec005180 0x7f36ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:08.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.580+0000 7f370be00640 1 -- 172.21.15.91:0/3040547917 reap_dead start 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.581+0000 7f3702800640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f36ec004c40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c240 tx=0x7f36ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c7cc057729d815 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.581+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4005ad0 con 0x7f36ec0047c0 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4005c30 con 0x7f36ec0047c0 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f370aa00640 1 -- 172.21.15.91:0/3040547917 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f402ca70 con 0x7f36ec0047c0 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 -- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 msgr2=0x7f36ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f36ec004c40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c240 tx=0x7f36ec0057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 -- 172.21.15.91:0/3040547917 shutdown_connections 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 --2- 172.21.15.91:0/3040547917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f36ec004c40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 -- 172.21.15.91:0/3040547917 >> 172.21.15.91:0/3040547917 conn(0x7f370c0cec80 msgr2=0x7f370c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 -- 172.21.15.91:0/3040547917 shutdown_connections 2024-02-02T13:29:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.582+0000 7f3712400640 1 -- 172.21.15.91:0/3040547917 wait complete. 2024-02-02T13:29:08.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.584+0000 7f3712400640 1 Processor -- start 2024-02-02T13:29:08.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.584+0000 7f3712400640 1 -- start start 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370c0c44e0 0x7f370c1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370c153df0 0x7f370c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1f2660 con 0x7f36ec0047c0 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c150f50 con 0x7f370c153df0 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.585+0000 7f3712400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c4a50 con 0x7f370c0c44e0 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51002/0 (socket says 172.21.15.91:51002) 2024-02-02T13:29:08.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 -- 172.21.15.91:0/4071087054 learned_addr learned my addr 172.21.15.91:0/4071087054 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:08.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 -- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370c0c44e0 msgr2=0x7f370c1f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370c0c44e0 0x7f370c1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 -- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370c153df0 msgr2=0x7f370c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370c153df0 0x7f370c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 -- 172.21.15.91:0/4071087054 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0c4d30 con 0x7f36ec0047c0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.586+0000 7f370b400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d4c0 tx=0x7f36fc00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.587+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc005480 con 0x7f36ec0047c0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.587+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc003610 con 0x7f36ec0047c0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.587+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0c5010 con 0x7f36ec0047c0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.587+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0039e0 con 0x7f36ec0047c0 2024-02-02T13:29:08.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.587+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1538d0 con 0x7f36ec0047c0 2024-02-02T13:29:08.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.588+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c066cf0 con 0x7f36ec0047c0 2024-02-02T13:29:08.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.589+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f36fc003b40 con 0x7f36ec0047c0 2024-02-02T13:29:08.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.590+0000 7f3703e00640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 0x7f36e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:08.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.590+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36fc088870 con 0x7f36ec0047c0 2024-02-02T13:29:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.590+0000 7f3702800640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 0x7f36e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:08.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.591+0000 7f3702800640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 0x7f36e406c8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f36f4034000 tx=0x7f36f4039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:08.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.595+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f36fc053d30 con 0x7f36ec0047c0 2024-02-02T13:29:08.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.785+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370c06c7b0 con 0x7f36ec0047c0 2024-02-02T13:29:08.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.786+0000 7f3703e00640 1 -- 172.21.15.91:0/4071087054 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f36fc057350 con 0x7f36ec0047c0 2024-02-02T13:29:08.788 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:08.788 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":86,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:05.872809+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 msgr2=0x7f36e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 0x7f36e406c8f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f36f4034000 tx=0x7f36f4039040 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 msgr2=0x7f370c0c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d4c0 tx=0x7f36fc00d980 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 shutdown_connections 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36e406a430 0x7f36e406c8f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370c0c44e0 0x7f370c1f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370c153df0 0x7f370c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 --2- 172.21.15.91:0/4071087054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36ec0047c0 0x7f370c0c3940 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 >> 172.21.15.91:0/4071087054 conn(0x7f370c0cec80 msgr2=0x7f370c06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:08.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 shutdown_connections 2024-02-02T13:29:08.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:08.789+0000 7f3712400640 1 -- 172.21.15.91:0/4071087054 wait complete. 2024-02-02T13:29:08.809 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:29:08.810 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:29:09.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.009+0000 7f3e52c00640 1 Processor -- start 2024-02-02T13:29:09.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.009+0000 7f3e52c00640 1 -- start start 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e52c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e52c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e52c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c0cc7c0 con 0x7f3e4c15a010 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e52c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b10d0 con 0x7f3e4c0ce130 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e52c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0ce930 con 0x7f3e4c0b0700 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e4be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3e4c15a010 0x7f3e4c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:52804/0 (socket says 172.21.15.91:52804) 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e4be00640 1 -- 172.21.15.91:0/1880633610 learned_addr learned my addr 172.21.15.91:0/1880633610 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:09.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.011+0000 7f3e43200640 1 --2- 172.21.15.91:0/1880633610 >> v2:172.21.15.91:3300/0 conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e4b400640 1 -- 172.21.15.91:0/1880633610 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312452768 0 0) 0x7f3e4c0cc7c0 con 0x7f3e4c15a010 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e4b400640 1 -- 172.21.15.91:0/1880633610 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e2c003520 con 0x7f3e4c15a010 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e50c00640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e43200640 1 -- 172.21.15.91:0/1880633610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 msgr2=0x7f3e4c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e43200640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e43200640 1 -- 172.21.15.91:0/1880633610 >> v1:172.21.15.91:6789/0 conn(0x7f3e4c15a010 legacy=0x7f3e4c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.012+0000 7f3e43200640 1 -- 172.21.15.91:0/1880633610 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0cd640 con 0x7f3e4c0ce130 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.013+0000 7f3e43200640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3e340097a0 tx=0x7f3e3402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1c66192875f4f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.013+0000 7f3e4b400640 1 -- 172.21.15.91:0/1880633610 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3402f8a0 con 0x7f3e4c0ce130 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.013+0000 7f3e4b400640 1 -- 172.21.15.91:0/1880633610 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3402fa00 con 0x7f3e4c0ce130 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.013+0000 7f3e4b400640 1 -- 172.21.15.91:0/1880633610 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3402fd90 con 0x7f3e4c0ce130 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 -- 172.21.15.91:0/1880633610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 msgr2=0x7f3e4c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3e340097a0 tx=0x7f3e3402dae0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 -- 172.21.15.91:0/1880633610 shutdown_connections 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 --2- 172.21.15.91:0/1880633610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 -- 172.21.15.91:0/1880633610 >> 172.21.15.91:0/1880633610 conn(0x7f3e4c0cec80 msgr2=0x7f3e4c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 -- 172.21.15.91:0/1880633610 shutdown_connections 2024-02-02T13:29:09.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.014+0000 7f3e52c00640 1 -- 172.21.15.91:0/1880633610 wait complete. 2024-02-02T13:29:09.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.016+0000 7f3e52c00640 1 Processor -- start 2024-02-02T13:29:09.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.016+0000 7f3e52c00640 1 -- start start 2024-02-02T13:29:09.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.017+0000 7f3e52c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.017+0000 7f3e52c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c1465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.017+0000 7f3e52c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.017+0000 7f3e52c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c155670 con 0x7f3e4c15a010 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.017+0000 7f3e52c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c146b20 con 0x7f3e4c0ce130 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e52c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0c8ab0 con 0x7f3e4c0b0700 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51024/0 (socket says 172.21.15.91:51024) 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 -- 172.21.15.91:0/3228097249 learned_addr learned my addr 172.21.15.91:0/3228097249 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e4be00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 -- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 msgr2=0x7f3e4c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e43200640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c1465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 -- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 msgr2=0x7f3e4c1465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c1465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e50c00640 1 -- 172.21.15.91:0/3228097249 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34009400 con 0x7f3e4c15a010 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.018+0000 7f3e43200640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c1465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e50c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3e44007d90 tx=0x7f3e4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44014420 con 0x7f3e4c15a010 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440145a0 con 0x7f3e4c15a010 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c154a60 con 0x7f3e4c15a010 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44014950 con 0x7f3e4c15a010 2024-02-02T13:29:09.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.019+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c155020 con 0x7f3e4c15a010 2024-02-02T13:29:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.020+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c066cf0 con 0x7f3e4c15a010 2024-02-02T13:29:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.021+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e44014ab0 con 0x7f3e4c15a010 2024-02-02T13:29:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.022+0000 7f3e48c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.022+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e440892d0 con 0x7f3e4c15a010 2024-02-02T13:29:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.023+0000 7f3e4be00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.023+0000 7f3e4be00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0011e0 tx=0x7f3e3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.027+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e44054820 con 0x7f3e4c15a010 2024-02-02T13:29:09.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.211+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e4c0b34e0 con 0x7f3e2406a430 2024-02-02T13:29:09.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.222+0000 7f3e48c00640 1 -- 172.21.15.91:0/3228097249 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282245 (secure 0 0 0) 0x7f3e4c0b34e0 con 0x7f3e2406a430 2024-02-02T13:29:09.228 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:09.229 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.228+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 msgr2=0x7f3e2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.228+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0011e0 tx=0x7f3e3c009040 comp rx=0 tx=0).stop 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 msgr2=0x7f3e4c0c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3e44007d90 tx=0x7f3e4400e4b0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 shutdown_connections 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c15a010 0x7f3e4c0c8570 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c1465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 --2- 172.21.15.91:0/3228097249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c0b0700 0x7f3e4c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 >> 172.21.15.91:0/3228097249 conn(0x7f3e4c0cec80 msgr2=0x7f3e4c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:09.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 shutdown_connections 2024-02-02T13:29:09.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.229+0000 7f3e52c00640 1 -- 172.21.15.91:0/3228097249 wait complete. 2024-02-02T13:29:09.249 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-02-02T13:29:08.323822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557809,"statfs":{"total":1600000000,"available":1595196868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.950463"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881546+0000","last_change":"2024-02-02T13:29:04.909851+0000","last_active":"2024-02-02T13:29:05.881546+0000","last_peered":"2024-02-02T13:29:05.881546+0000","last_clean":"2024-02-02T13:29:05.881546+0000","last_became_active":"2024-02-02T13:29:04.909111+0000","last_became_peered":"2024-02-02T13:29:04.909111+0000","last_unstale":"2024-02-02T13:29:05.881546+0000","last_undegraded":"2024-02-02T13:29:05.881546+0000","last_fullsized":"2024-02-02T13:29:05.881546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:33:58.933407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908056+0000","last_change":"2024-02-02T13:29:04.906480+0000","last_active":"2024-02-02T13:29:04.908056+0000","last_peered":"2024-02-02T13:29:04.908056+0000","last_clean":"2024-02-02T13:29:04.908056+0000","last_became_active":"2024-02-02T13:29:04.906133+0000","last_became_peered":"2024-02-02T13:29:04.906133+0000","last_unstale":"2024-02-02T13:29:04.908056+0000","last_undegraded":"2024-02-02T13:29:04.908056+0000","last_fullsized":"2024-02-02T13:29:04.908056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:45.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889086+0000","last_change":"2024-02-02T13:29:04.888505+0000","last_active":"2024-02-02T13:29:04.889086+0000","last_peered":"2024-02-02T13:29:04.889086+0000","last_clean":"2024-02-02T13:29:04.889086+0000","last_became_active":"2024-02-02T13:29:04.888296+0000","last_became_peered":"2024-02-02T13:29:04.888296+0000","last_unstale":"2024-02-02T13:29:04.889086+0000","last_undegraded":"2024-02-02T13:29:04.889086+0000","last_fullsized":"2024-02-02T13:29:04.889086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:10:32.798262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898352+0000","last_change":"2024-02-02T13:29:04.898282+0000","last_active":"2024-02-02T13:29:04.898352+0000","last_peered":"2024-02-02T13:29:04.898352+0000","last_clean":"2024-02-02T13:29:04.898352+0000","last_became_active":"2024-02-02T13:29:04.897892+0000","last_became_peered":"2024-02-02T13:29:04.897892+0000","last_unstale":"2024-02-02T13:29:04.898352+0000","last_undegraded":"2024-02-02T13:29:04.898352+0000","last_fullsized":"2024-02-02T13:29:04.898352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:55:29.138956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.909334+0000","last_change":"2024-02-02T13:29:04.909282+0000","last_active":"2024-02-02T13:29:04.909334+0000","last_peered":"2024-02-02T13:29:04.909334+0000","last_clean":"2024-02-02T13:29:04.909334+0000","last_became_active":"2024-02-02T13:29:04.909149+0000","last_became_peered":"2024-02-02T13:29:04.909149+0000","last_unstale":"2024-02-02T13:29:04.909334+0000","last_undegraded":"2024-02-02T13:29:04.909334+0000","last_fullsized":"2024-02-02T13:29:04.909334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:57.294165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908990+0000","last_change":"2024-02-02T13:29:04.908565+0000","last_active":"2024-02-02T13:29:04.908990+0000","last_peered":"2024-02-02T13:29:04.908990+0000","last_clean":"2024-02-02T13:29:04.908990+0000","last_became_active":"2024-02-02T13:29:04.908248+0000","last_became_peered":"2024-02-02T13:29:04.908248+0000","last_unstale":"2024-02-02T13:29:04.908990+0000","last_undegraded":"2024-02-02T13:29:04.908990+0000","last_fullsized":"2024-02-02T13:29:04.908990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:46.705986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.880772+0000","last_change":"2024-02-02T13:29:04.884957+0000","last_active":"2024-02-02T13:29:05.880772+0000","last_peered":"2024-02-02T13:29:05.880772+0000","last_clean":"2024-02-02T13:29:05.880772+0000","last_became_active":"2024-02-02T13:29:04.883916+0000","last_became_peered":"2024-02-02T13:29:04.883916+0000","last_unstale":"2024-02-02T13:29:05.880772+0000","last_undegraded":"2024-02-02T13:29:05.880772+0000","last_fullsized":"2024-02-02T13:29:05.880772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:08.230800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908119+0000","last_change":"2024-02-02T13:29:04.907789+0000","last_active":"2024-02-02T13:29:04.908119+0000","last_peered":"2024-02-02T13:29:04.908119+0000","last_clean":"2024-02-02T13:29:04.908119+0000","last_became_active":"2024-02-02T13:29:04.900053+0000","last_became_peered":"2024-02-02T13:29:04.900053+0000","last_unstale":"2024-02-02T13:29:04.908119+0000","last_undegraded":"2024-02-02T13:29:04.908119+0000","last_fullsized":"2024-02-02T13:29:04.908119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:21:10.301191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889387+0000","last_change":"2024-02-02T13:29:04.889001+0000","last_active":"2024-02-02T13:29:04.889387+0000","last_peered":"2024-02-02T13:29:04.889387+0000","last_clean":"2024-02-02T13:29:04.889387+0000","last_became_active":"2024-02-02T13:29:04.888734+0000","last_became_peered":"2024-02-02T13:29:04.888734+0000","last_unstale":"2024-02-02T13:29:04.889387+0000","last_undegraded":"2024-02-02T13:29:04.889387+0000","last_fullsized":"2024-02-02T13:29:04.889387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:05:57.256014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884627+0000","last_change":"2024-02-02T13:29:04.902839+0000","last_active":"2024-02-02T13:29:05.884627+0000","last_peered":"2024-02-02T13:29:05.884627+0000","last_clean":"2024-02-02T13:29:05.884627+0000","last_became_active":"2024-02-02T13:29:04.902177+0000","last_became_peered":"2024-02-02T13:29:04.902177+0000","last_unstale":"2024-02-02T13:29:05.884627+0000","last_undegraded":"2024-02-02T13:29:05.884627+0000","last_fullsized":"2024-02-02T13:29:05.884627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:27.748598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881965+0000","last_change":"2024-02-02T13:29:04.910007+0000","last_active":"2024-02-02T13:29:05.881965+0000","last_peered":"2024-02-02T13:29:05.881965+0000","last_clean":"2024-02-02T13:29:05.881965+0000","last_became_active":"2024-02-02T13:29:04.909376+0000","last_became_peered":"2024-02-02T13:29:04.909376+0000","last_unstale":"2024-02-02T13:29:05.881965+0000","last_undegraded":"2024-02-02T13:29:05.881965+0000","last_fullsized":"2024-02-02T13:29:05.881965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:53.426160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.883104+0000","last_change":"2024-02-02T13:29:04.882980+0000","last_active":"2024-02-02T13:29:04.883104+0000","last_peered":"2024-02-02T13:29:04.883104+0000","last_clean":"2024-02-02T13:29:04.883104+0000","last_became_active":"2024-02-02T13:29:04.882376+0000","last_became_peered":"2024-02-02T13:29:04.882376+0000","last_unstale":"2024-02-02T13:29:04.883104+0000","last_undegraded":"2024-02-02T13:29:04.883104+0000","last_fullsized":"2024-02-02T13:29:04.883104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:46:40.349335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.901257+0000","last_change":"2024-02-02T13:29:04.901202+0000","last_active":"2024-02-02T13:29:04.901257+0000","last_peered":"2024-02-02T13:29:04.901257+0000","last_clean":"2024-02-02T13:29:04.901257+0000","last_became_active":"2024-02-02T13:29:04.900927+0000","last_became_peered":"2024-02-02T13:29:04.900927+0000","last_unstale":"2024-02-02T13:29:04.901257+0000","last_undegraded":"2024-02-02T13:29:04.901257+0000","last_fullsized":"2024-02-02T13:29:04.901257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:45.908362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.882673+0000","last_change":"2024-02-02T13:29:04.882518+0000","last_active":"2024-02-02T13:29:04.882673+0000","last_peered":"2024-02-02T13:29:04.882673+0000","last_clean":"2024-02-02T13:29:04.882673+0000","last_became_active":"2024-02-02T13:29:04.882310+0000","last_became_peered":"2024-02-02T13:29:04.882310+0000","last_unstale":"2024-02-02T13:29:04.882673+0000","last_undegraded":"2024-02-02T13:29:04.882673+0000","last_fullsized":"2024-02-02T13:29:04.882673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:43:56.120661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890181+0000","last_change":"2024-02-02T13:29:04.890133+0000","last_active":"2024-02-02T13:29:04.890181+0000","last_peered":"2024-02-02T13:29:04.890181+0000","last_clean":"2024-02-02T13:29:04.890181+0000","last_became_active":"2024-02-02T13:29:04.889907+0000","last_became_peered":"2024-02-02T13:29:04.889907+0000","last_unstale":"2024-02-02T13:29:04.890181+0000","last_undegraded":"2024-02-02T13:29:04.890181+0000","last_fullsized":"2024-02-02T13:29:04.890181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:15.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884080+0000","last_change":"2024-02-02T13:29:04.912080+0000","last_active":"2024-02-02T13:29:05.884080+0000","last_peered":"2024-02-02T13:29:05.884080+0000","last_clean":"2024-02-02T13:29:05.884080+0000","last_became_active":"2024-02-02T13:29:04.911575+0000","last_became_peered":"2024-02-02T13:29:04.911575+0000","last_unstale":"2024-02-02T13:29:05.884080+0000","last_undegraded":"2024-02-02T13:29:05.884080+0000","last_fullsized":"2024-02-02T13:29:05.884080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:46.705986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886522+0000","last_change":"2024-02-02T13:29:04.886461+0000","last_active":"2024-02-02T13:29:04.886522+0000","last_peered":"2024-02-02T13:29:04.886522+0000","last_clean":"2024-02-02T13:29:04.886522+0000","last_became_active":"2024-02-02T13:29:04.885876+0000","last_became_peered":"2024-02-02T13:29:04.885876+0000","last_unstale":"2024-02-02T13:29:04.886522+0000","last_undegraded":"2024-02-02T13:29:04.886522+0000","last_fullsized":"2024-02-02T13:29:04.886522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:30.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.900075+0000","last_change":"2024-02-02T13:29:04.899976+0000","last_active":"2024-02-02T13:29:04.900075+0000","last_peered":"2024-02-02T13:29:04.900075+0000","last_clean":"2024-02-02T13:29:04.900075+0000","last_became_active":"2024-02-02T13:29:04.899744+0000","last_became_peered":"2024-02-02T13:29:04.899744+0000","last_unstale":"2024-02-02T13:29:04.900075+0000","last_undegraded":"2024-02-02T13:29:04.900075+0000","last_fullsized":"2024-02-02T13:29:04.900075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:09.878784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899762+0000","last_change":"2024-02-02T13:29:04.899628+0000","last_active":"2024-02-02T13:29:04.899762+0000","last_peered":"2024-02-02T13:29:04.899762+0000","last_clean":"2024-02-02T13:29:04.899762+0000","last_became_active":"2024-02-02T13:29:04.898887+0000","last_became_peered":"2024-02-02T13:29:04.898887+0000","last_unstale":"2024-02-02T13:29:04.899762+0000","last_undegraded":"2024-02-02T13:29:04.899762+0000","last_fullsized":"2024-02-02T13:29:04.899762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:47:26.089514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.893724+0000","last_change":"2024-02-02T13:29:04.893659+0000","last_active":"2024-02-02T13:29:04.893724+0000","last_peered":"2024-02-02T13:29:04.893724+0000","last_clean":"2024-02-02T13:29:04.893724+0000","last_became_active":"2024-02-02T13:29:04.893432+0000","last_became_peered":"2024-02-02T13:29:04.893432+0000","last_unstale":"2024-02-02T13:29:04.893724+0000","last_undegraded":"2024-02-02T13:29:04.893724+0000","last_fullsized":"2024-02-02T13:29:04.893724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:08.230800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884500+0000","last_change":"2024-02-02T13:29:04.895867+0000","last_active":"2024-02-02T13:29:05.884500+0000","last_peered":"2024-02-02T13:29:05.884500+0000","last_clean":"2024-02-02T13:29:05.884500+0000","last_became_active":"2024-02-02T13:29:04.894304+0000","last_became_peered":"2024-02-02T13:29:04.894304+0000","last_unstale":"2024-02-02T13:29:05.884500+0000","last_undegraded":"2024-02-02T13:29:05.884500+0000","last_fullsized":"2024-02-02T13:29:05.884500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:45.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883811+0000","last_change":"2024-02-02T13:29:04.901290+0000","last_active":"2024-02-02T13:29:05.883811+0000","last_peered":"2024-02-02T13:29:05.883811+0000","last_clean":"2024-02-02T13:29:05.883811+0000","last_became_active":"2024-02-02T13:29:04.900860+0000","last_became_peered":"2024-02-02T13:29:04.900860+0000","last_unstale":"2024-02-02T13:29:05.883811+0000","last_undegraded":"2024-02-02T13:29:05.883811+0000","last_fullsized":"2024-02-02T13:29:05.883811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:12.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881377+0000","last_change":"2024-02-02T13:29:04.909839+0000","last_active":"2024-02-02T13:29:05.881377+0000","last_peered":"2024-02-02T13:29:05.881377+0000","last_clean":"2024-02-02T13:29:05.881377+0000","last_became_active":"2024-02-02T13:29:04.909607+0000","last_became_peered":"2024-02-02T13:29:04.909607+0000","last_unstale":"2024-02-02T13:29:05.881377+0000","last_undegraded":"2024-02-02T13:29:05.881377+0000","last_fullsized":"2024-02-02T13:29:05.881377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:19:31.477807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884049+0000","last_change":"2024-02-02T13:29:04.914130+0000","last_active":"2024-02-02T13:29:05.884049+0000","last_peered":"2024-02-02T13:29:05.884049+0000","last_clean":"2024-02-02T13:29:05.884049+0000","last_became_active":"2024-02-02T13:29:04.913430+0000","last_became_peered":"2024-02-02T13:29:04.913430+0000","last_unstale":"2024-02-02T13:29:05.884049+0000","last_undegraded":"2024-02-02T13:29:05.884049+0000","last_fullsized":"2024-02-02T13:29:05.884049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:06.196541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884844+0000","last_change":"2024-02-02T13:29:04.916605+0000","last_active":"2024-02-02T13:29:05.884844+0000","last_peered":"2024-02-02T13:29:05.884844+0000","last_clean":"2024-02-02T13:29:05.884844+0000","last_became_active":"2024-02-02T13:29:04.916339+0000","last_became_peered":"2024-02-02T13:29:04.916339+0000","last_unstale":"2024-02-02T13:29:05.884844+0000","last_undegraded":"2024-02-02T13:29:05.884844+0000","last_fullsized":"2024-02-02T13:29:05.884844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:53.821488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881897+0000","last_change":"2024-02-02T13:29:04.910741+0000","last_active":"2024-02-02T13:29:05.881897+0000","last_peered":"2024-02-02T13:29:05.881897+0000","last_clean":"2024-02-02T13:29:05.881897+0000","last_became_active":"2024-02-02T13:29:04.910487+0000","last_became_peered":"2024-02-02T13:29:04.910487+0000","last_unstale":"2024-02-02T13:29:05.881897+0000","last_undegraded":"2024-02-02T13:29:05.881897+0000","last_fullsized":"2024-02-02T13:29:05.881897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:11.165171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.914530+0000","last_change":"2024-02-02T13:29:04.914450+0000","last_active":"2024-02-02T13:29:04.914530+0000","last_peered":"2024-02-02T13:29:04.914530+0000","last_clean":"2024-02-02T13:29:04.914530+0000","last_became_active":"2024-02-02T13:29:04.914063+0000","last_became_peered":"2024-02-02T13:29:04.914063+0000","last_unstale":"2024-02-02T13:29:04.914530+0000","last_undegraded":"2024-02-02T13:29:04.914530+0000","last_fullsized":"2024-02-02T13:29:04.914530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:38:37.104567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883617+0000","last_change":"2024-02-02T13:29:04.900779+0000","last_active":"2024-02-02T13:29:05.883617+0000","last_peered":"2024-02-02T13:29:05.883617+0000","last_clean":"2024-02-02T13:29:05.883617+0000","last_became_active":"2024-02-02T13:29:04.900259+0000","last_became_peered":"2024-02-02T13:29:04.900259+0000","last_unstale":"2024-02-02T13:29:05.883617+0000","last_undegraded":"2024-02-02T13:29:05.883617+0000","last_fullsized":"2024-02-02T13:29:05.883617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:31.600140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889104+0000","last_change":"2024-02-02T13:29:04.888588+0000","last_active":"2024-02-02T13:29:04.889104+0000","last_peered":"2024-02-02T13:29:04.889104+0000","last_clean":"2024-02-02T13:29:04.889104+0000","last_became_active":"2024-02-02T13:29:04.888361+0000","last_became_peered":"2024-02-02T13:29:04.888361+0000","last_unstale":"2024-02-02T13:29:04.889104+0000","last_undegraded":"2024-02-02T13:29:04.889104+0000","last_fullsized":"2024-02-02T13:29:04.889104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:46:31.174891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882575+0000","last_change":"2024-02-02T13:29:04.908772+0000","last_active":"2024-02-02T13:29:05.882575+0000","last_peered":"2024-02-02T13:29:05.882575+0000","last_clean":"2024-02-02T13:29:05.882575+0000","last_became_active":"2024-02-02T13:29:04.908508+0000","last_became_peered":"2024-02-02T13:29:04.908508+0000","last_unstale":"2024-02-02T13:29:05.882575+0000","last_undegraded":"2024-02-02T13:29:05.882575+0000","last_fullsized":"2024-02-02T13:29:05.882575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:53.541313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882121+0000","last_change":"2024-02-02T13:29:04.886830+0000","last_active":"2024-02-02T13:29:05.882121+0000","last_peered":"2024-02-02T13:29:05.882121+0000","last_clean":"2024-02-02T13:29:05.882121+0000","last_became_active":"2024-02-02T13:29:04.885982+0000","last_became_peered":"2024-02-02T13:29:04.885982+0000","last_unstale":"2024-02-02T13:29:05.882121+0000","last_undegraded":"2024-02-02T13:29:05.882121+0000","last_fullsized":"2024-02-02T13:29:05.882121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:07:25.371263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899724+0000","last_change":"2024-02-02T13:29:04.899584+0000","last_active":"2024-02-02T13:29:04.899724+0000","last_peered":"2024-02-02T13:29:04.899724+0000","last_clean":"2024-02-02T13:29:04.899724+0000","last_became_active":"2024-02-02T13:29:04.899071+0000","last_became_peered":"2024-02-02T13:29:04.899071+0000","last_unstale":"2024-02-02T13:29:04.899724+0000","last_undegraded":"2024-02-02T13:29:04.899724+0000","last_fullsized":"2024-02-02T13:29:04.899724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:27.748598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884587+0000","last_change":"2024-02-02T13:29:04.914623+0000","last_active":"2024-02-02T13:29:05.884587+0000","last_peered":"2024-02-02T13:29:05.884587+0000","last_clean":"2024-02-02T13:29:05.884587+0000","last_became_active":"2024-02-02T13:29:04.913897+0000","last_became_peered":"2024-02-02T13:29:04.913897+0000","last_unstale":"2024-02-02T13:29:05.884587+0000","last_undegraded":"2024-02-02T13:29:05.884587+0000","last_fullsized":"2024-02-02T13:29:05.884587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:20.651424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881661+0000","last_change":"2024-02-02T13:29:04.401531+0000","last_active":"2024-02-02T13:29:05.881661+0000","last_peered":"2024-02-02T13:29:05.881661+0000","last_clean":"2024-02-02T13:29:05.881661+0000","last_became_active":"2024-02-02T13:29:04.400570+0000","last_became_peered":"2024-02-02T13:29:04.400570+0000","last_unstale":"2024-02-02T13:29:05.881661+0000","last_undegraded":"2024-02-02T13:29:05.881661+0000","last_fullsized":"2024-02-02T13:29:05.881661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:22:43.166181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.887521+0000","last_change":"2024-02-02T13:29:04.887446+0000","last_active":"2024-02-02T13:29:04.887521+0000","last_peered":"2024-02-02T13:29:04.887521+0000","last_clean":"2024-02-02T13:29:04.887521+0000","last_became_active":"2024-02-02T13:29:04.887160+0000","last_became_peered":"2024-02-02T13:29:04.887160+0000","last_unstale":"2024-02-02T13:29:04.887521+0000","last_undegraded":"2024-02-02T13:29:04.887521+0000","last_fullsized":"2024-02-02T13:29:04.887521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:36.516051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890307+0000","last_change":"2024-02-02T13:29:04.396861+0000","last_active":"2024-02-02T13:29:04.890307+0000","last_peered":"2024-02-02T13:29:04.890307+0000","last_clean":"2024-02-02T13:29:04.890307+0000","last_became_active":"2024-02-02T13:29:04.395937+0000","last_became_peered":"2024-02-02T13:29:04.395937+0000","last_unstale":"2024-02-02T13:29:04.890307+0000","last_undegraded":"2024-02-02T13:29:04.890307+0000","last_fullsized":"2024-02-02T13:29:04.890307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:44.872326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.913958+0000","last_change":"2024-02-02T13:29:04.913873+0000","last_active":"2024-02-02T13:29:04.913958+0000","last_peered":"2024-02-02T13:29:04.913958+0000","last_clean":"2024-02-02T13:29:04.913958+0000","last_became_active":"2024-02-02T13:29:04.912396+0000","last_became_peered":"2024-02-02T13:29:04.912396+0000","last_unstale":"2024-02-02T13:29:04.913958+0000","last_undegraded":"2024-02-02T13:29:04.913958+0000","last_fullsized":"2024-02-02T13:29:04.913958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:54:03.744654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885227+0000","last_change":"2024-02-02T13:29:04.409703+0000","last_active":"2024-02-02T13:29:05.885227+0000","last_peered":"2024-02-02T13:29:05.885227+0000","last_clean":"2024-02-02T13:29:05.885227+0000","last_became_active":"2024-02-02T13:29:04.408780+0000","last_became_peered":"2024-02-02T13:29:04.408780+0000","last_unstale":"2024-02-02T13:29:05.885227+0000","last_undegraded":"2024-02-02T13:29:05.885227+0000","last_fullsized":"2024-02-02T13:29:05.885227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:20.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882380+0000","last_change":"2024-02-02T13:29:04.913827+0000","last_active":"2024-02-02T13:29:05.882380+0000","last_peered":"2024-02-02T13:29:05.882380+0000","last_clean":"2024-02-02T13:29:05.882380+0000","last_became_active":"2024-02-02T13:29:04.913166+0000","last_became_peered":"2024-02-02T13:29:04.913166+0000","last_unstale":"2024-02-02T13:29:05.882380+0000","last_undegraded":"2024-02-02T13:29:05.882380+0000","last_fullsized":"2024-02-02T13:29:05.882380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:38.813534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.912678+0000","last_change":"2024-02-02T13:29:04.408968+0000","last_active":"2024-02-02T13:29:04.912678+0000","last_peered":"2024-02-02T13:29:04.912678+0000","last_clean":"2024-02-02T13:29:04.912678+0000","last_became_active":"2024-02-02T13:29:04.408182+0000","last_became_peered":"2024-02-02T13:29:04.408182+0000","last_unstale":"2024-02-02T13:29:04.912678+0000","last_undegraded":"2024-02-02T13:29:04.912678+0000","last_fullsized":"2024-02-02T13:29:04.912678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:33.577918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.880784+0000","last_change":"2024-02-02T13:29:04.889303+0000","last_active":"2024-02-02T13:29:05.880784+0000","last_peered":"2024-02-02T13:29:05.880784+0000","last_clean":"2024-02-02T13:29:05.880784+0000","last_became_active":"2024-02-02T13:29:04.888641+0000","last_became_peered":"2024-02-02T13:29:04.888641+0000","last_unstale":"2024-02-02T13:29:05.880784+0000","last_undegraded":"2024-02-02T13:29:05.880784+0000","last_fullsized":"2024-02-02T13:29:05.880784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:35:37.094490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.896734+0000","last_change":"2024-02-02T13:29:04.405333+0000","last_active":"2024-02-02T13:29:04.896734+0000","last_peered":"2024-02-02T13:29:04.896734+0000","last_clean":"2024-02-02T13:29:04.896734+0000","last_became_active":"2024-02-02T13:29:04.404992+0000","last_became_peered":"2024-02-02T13:29:04.404992+0000","last_unstale":"2024-02-02T13:29:04.896734+0000","last_undegraded":"2024-02-02T13:29:04.896734+0000","last_fullsized":"2024-02-02T13:29:04.896734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:52.420500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884057+0000","last_change":"2024-02-02T13:29:04.912565+0000","last_active":"2024-02-02T13:29:05.884057+0000","last_peered":"2024-02-02T13:29:05.884057+0000","last_clean":"2024-02-02T13:29:05.884057+0000","last_became_active":"2024-02-02T13:29:04.910842+0000","last_became_peered":"2024-02-02T13:29:04.910842+0000","last_unstale":"2024-02-02T13:29:05.884057+0000","last_undegraded":"2024-02-02T13:29:05.884057+0000","last_fullsized":"2024-02-02T13:29:05.884057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:54:03.744654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.916037+0000","last_change":"2024-02-02T13:29:04.406109+0000","last_active":"2024-02-02T13:29:04.916037+0000","last_peered":"2024-02-02T13:29:04.916037+0000","last_clean":"2024-02-02T13:29:04.916037+0000","last_became_active":"2024-02-02T13:29:04.405828+0000","last_became_peered":"2024-02-02T13:29:04.405828+0000","last_unstale":"2024-02-02T13:29:04.916037+0000","last_undegraded":"2024-02-02T13:29:04.916037+0000","last_fullsized":"2024-02-02T13:29:04.916037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:27:41.967706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.900474+0000","last_change":"2024-02-02T13:29:04.900434+0000","last_active":"2024-02-02T13:29:04.900474+0000","last_peered":"2024-02-02T13:29:04.900474+0000","last_clean":"2024-02-02T13:29:04.900474+0000","last_became_active":"2024-02-02T13:29:04.900270+0000","last_became_peered":"2024-02-02T13:29:04.900270+0000","last_unstale":"2024-02-02T13:29:04.900474+0000","last_undegraded":"2024-02-02T13:29:04.900474+0000","last_fullsized":"2024-02-02T13:29:04.900474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:21.935070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.877319+0000","last_change":"2024-02-02T13:29:04.394846+0000","last_active":"2024-02-02T13:29:04.877319+0000","last_peered":"2024-02-02T13:29:04.877319+0000","last_clean":"2024-02-02T13:29:04.877319+0000","last_became_active":"2024-02-02T13:29:04.394597+0000","last_became_peered":"2024-02-02T13:29:04.394597+0000","last_unstale":"2024-02-02T13:29:04.877319+0000","last_undegraded":"2024-02-02T13:29:04.877319+0000","last_fullsized":"2024-02-02T13:29:04.877319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:27:41.967706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.910400+0000","last_change":"2024-02-02T13:29:04.910328+0000","last_active":"2024-02-02T13:29:04.910400+0000","last_peered":"2024-02-02T13:29:04.910400+0000","last_clean":"2024-02-02T13:29:04.910400+0000","last_became_active":"2024-02-02T13:29:04.910076+0000","last_became_peered":"2024-02-02T13:29:04.910076+0000","last_unstale":"2024-02-02T13:29:04.910400+0000","last_undegraded":"2024-02-02T13:29:04.910400+0000","last_fullsized":"2024-02-02T13:29:04.910400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:07:25.371263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.878680+0000","last_change":"2024-02-02T13:29:04.401668+0000","last_active":"2024-02-02T13:29:04.878680+0000","last_peered":"2024-02-02T13:29:04.878680+0000","last_clean":"2024-02-02T13:29:04.878680+0000","last_became_active":"2024-02-02T13:29:04.401416+0000","last_became_peered":"2024-02-02T13:29:04.401416+0000","last_unstale":"2024-02-02T13:29:04.878680+0000","last_undegraded":"2024-02-02T13:29:04.878680+0000","last_fullsized":"2024-02-02T13:29:04.878680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:50:09.200253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881019+0000","last_change":"2024-02-02T13:29:04.912107+0000","last_active":"2024-02-02T13:29:05.881019+0000","last_peered":"2024-02-02T13:29:05.881019+0000","last_clean":"2024-02-02T13:29:05.881019+0000","last_became_active":"2024-02-02T13:29:04.911832+0000","last_became_peered":"2024-02-02T13:29:04.911832+0000","last_unstale":"2024-02-02T13:29:05.881019+0000","last_undegraded":"2024-02-02T13:29:05.881019+0000","last_fullsized":"2024-02-02T13:29:05.881019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:45.908362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884696+0000","last_change":"2024-02-02T13:29:04.407888+0000","last_active":"2024-02-02T13:29:05.884696+0000","last_peered":"2024-02-02T13:29:05.884696+0000","last_clean":"2024-02-02T13:29:05.884696+0000","last_became_active":"2024-02-02T13:29:04.406034+0000","last_became_peered":"2024-02-02T13:29:04.406034+0000","last_unstale":"2024-02-02T13:29:05.884696+0000","last_undegraded":"2024-02-02T13:29:05.884696+0000","last_fullsized":"2024-02-02T13:29:05.884696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:50:57.751135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882192+0000","last_change":"2024-02-02T13:29:04.916237+0000","last_active":"2024-02-02T13:29:05.882192+0000","last_peered":"2024-02-02T13:29:05.882192+0000","last_clean":"2024-02-02T13:29:05.882192+0000","last_became_active":"2024-02-02T13:29:04.915877+0000","last_became_peered":"2024-02-02T13:29:04.915877+0000","last_unstale":"2024-02-02T13:29:05.882192+0000","last_undegraded":"2024-02-02T13:29:05.882192+0000","last_fullsized":"2024-02-02T13:29:05.882192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:02:49.668441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898397+0000","last_change":"2024-02-02T13:29:04.406515+0000","last_active":"2024-02-02T13:29:04.898397+0000","last_peered":"2024-02-02T13:29:04.898397+0000","last_clean":"2024-02-02T13:29:04.898397+0000","last_became_active":"2024-02-02T13:29:04.406206+0000","last_became_peered":"2024-02-02T13:29:04.406206+0000","last_unstale":"2024-02-02T13:29:04.898397+0000","last_undegraded":"2024-02-02T13:29:04.898397+0000","last_fullsized":"2024-02-02T13:29:04.898397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:31.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.909011+0000","last_change":"2024-02-02T13:29:04.904697+0000","last_active":"2024-02-02T13:29:04.909011+0000","last_peered":"2024-02-02T13:29:04.909011+0000","last_clean":"2024-02-02T13:29:04.909011+0000","last_became_active":"2024-02-02T13:29:04.904455+0000","last_became_peered":"2024-02-02T13:29:04.904455+0000","last_unstale":"2024-02-02T13:29:04.909011+0000","last_undegraded":"2024-02-02T13:29:04.909011+0000","last_fullsized":"2024-02-02T13:29:04.909011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:12.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.895442+0000","last_change":"2024-02-02T13:29:04.387793+0000","last_active":"2024-02-02T13:29:04.895442+0000","last_peered":"2024-02-02T13:29:04.895442+0000","last_clean":"2024-02-02T13:29:04.895442+0000","last_became_active":"2024-02-02T13:29:04.387030+0000","last_became_peered":"2024-02-02T13:29:04.387030+0000","last_unstale":"2024-02-02T13:29:04.895442+0000","last_undegraded":"2024-02-02T13:29:04.895442+0000","last_fullsized":"2024-02-02T13:29:04.895442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:37.807873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886474+0000","last_change":"2024-02-02T13:29:04.885141+0000","last_active":"2024-02-02T13:29:04.886474+0000","last_peered":"2024-02-02T13:29:04.886474+0000","last_clean":"2024-02-02T13:29:04.886474+0000","last_became_active":"2024-02-02T13:29:04.884911+0000","last_became_peered":"2024-02-02T13:29:04.884911+0000","last_unstale":"2024-02-02T13:29:04.886474+0000","last_undegraded":"2024-02-02T13:29:04.886474+0000","last_fullsized":"2024-02-02T13:29:04.886474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:22.500925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898111+0000","last_change":"2024-02-02T13:29:04.400508+0000","last_active":"2024-02-02T13:29:04.898111+0000","last_peered":"2024-02-02T13:29:04.898111+0000","last_clean":"2024-02-02T13:29:04.898111+0000","last_became_active":"2024-02-02T13:29:04.399434+0000","last_became_peered":"2024-02-02T13:29:04.399434+0000","last_unstale":"2024-02-02T13:29:04.898111+0000","last_undegraded":"2024-02-02T13:29:04.898111+0000","last_fullsized":"2024-02-02T13:29:04.898111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:20.929409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882420+0000","last_change":"2024-02-02T13:29:04.911157+0000","last_active":"2024-02-02T13:29:05.882420+0000","last_peered":"2024-02-02T13:29:05.882420+0000","last_clean":"2024-02-02T13:29:05.882420+0000","last_became_active":"2024-02-02T13:29:04.909652+0000","last_became_peered":"2024-02-02T13:29:04.909652+0000","last_unstale":"2024-02-02T13:29:05.882420+0000","last_undegraded":"2024-02-02T13:29:05.882420+0000","last_fullsized":"2024-02-02T13:29:05.882420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:34:57.356543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.914698+0000","last_change":"2024-02-02T13:29:04.398976+0000","last_active":"2024-02-02T13:29:04.914698+0000","last_peered":"2024-02-02T13:29:04.914698+0000","last_clean":"2024-02-02T13:29:04.914698+0000","last_became_active":"2024-02-02T13:29:04.395566+0000","last_became_peered":"2024-02-02T13:29:04.395566+0000","last_unstale":"2024-02-02T13:29:04.914698+0000","last_undegraded":"2024-02-02T13:29:04.914698+0000","last_fullsized":"2024-02-02T13:29:04.914698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:17.180073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882818+0000","last_change":"2024-02-02T13:29:04.903240+0000","last_active":"2024-02-02T13:29:05.882818+0000","last_peered":"2024-02-02T13:29:05.882818+0000","last_clean":"2024-02-02T13:29:05.882818+0000","last_became_active":"2024-02-02T13:29:04.902752+0000","last_became_peered":"2024-02-02T13:29:04.902752+0000","last_unstale":"2024-02-02T13:29:05.882818+0000","last_undegraded":"2024-02-02T13:29:05.882818+0000","last_fullsized":"2024-02-02T13:29:05.882818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:50:02.616058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.904268+0000","last_change":"2024-02-02T13:29:04.409751+0000","last_active":"2024-02-02T13:29:04.904268+0000","last_peered":"2024-02-02T13:29:04.904268+0000","last_clean":"2024-02-02T13:29:04.904268+0000","last_became_active":"2024-02-02T13:29:04.409200+0000","last_became_peered":"2024-02-02T13:29:04.409200+0000","last_unstale":"2024-02-02T13:29:04.904268+0000","last_undegraded":"2024-02-02T13:29:04.904268+0000","last_fullsized":"2024-02-02T13:29:04.904268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:20.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.901174+0000","last_change":"2024-02-02T13:29:04.901084+0000","last_active":"2024-02-02T13:29:04.901174+0000","last_peered":"2024-02-02T13:29:04.901174+0000","last_clean":"2024-02-02T13:29:04.901174+0000","last_became_active":"2024-02-02T13:29:04.900725+0000","last_became_peered":"2024-02-02T13:29:04.900725+0000","last_unstale":"2024-02-02T13:29:04.901174+0000","last_undegraded":"2024-02-02T13:29:04.901174+0000","last_fullsized":"2024-02-02T13:29:04.901174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:53.541313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881488+0000","last_change":"2024-02-02T13:29:04.399510+0000","last_active":"2024-02-02T13:29:05.881488+0000","last_peered":"2024-02-02T13:29:05.881488+0000","last_clean":"2024-02-02T13:29:05.881488+0000","last_became_active":"2024-02-02T13:29:04.398902+0000","last_became_peered":"2024-02-02T13:29:04.398902+0000","last_unstale":"2024-02-02T13:29:05.881488+0000","last_undegraded":"2024-02-02T13:29:05.881488+0000","last_fullsized":"2024-02-02T13:29:05.881488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:11.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886681+0000","last_change":"2024-02-02T13:29:04.886599+0000","last_active":"2024-02-02T13:29:04.886681+0000","last_peered":"2024-02-02T13:29:04.886681+0000","last_clean":"2024-02-02T13:29:04.886681+0000","last_became_active":"2024-02-02T13:29:04.886260+0000","last_became_peered":"2024-02-02T13:29:04.886260+0000","last_unstale":"2024-02-02T13:29:04.886681+0000","last_undegraded":"2024-02-02T13:29:04.886681+0000","last_fullsized":"2024-02-02T13:29:04.886681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:57:44.798704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883848+0000","last_change":"2024-02-02T13:29:04.406001+0000","last_active":"2024-02-02T13:29:05.883848+0000","last_peered":"2024-02-02T13:29:05.883848+0000","last_clean":"2024-02-02T13:29:05.883848+0000","last_became_active":"2024-02-02T13:29:04.405782+0000","last_became_peered":"2024-02-02T13:29:04.405782+0000","last_unstale":"2024-02-02T13:29:05.883848+0000","last_undegraded":"2024-02-02T13:29:05.883848+0000","last_fullsized":"2024-02-02T13:29:05.883848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:14:49.868948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885007+0000","last_change":"2024-02-02T13:29:04.902592+0000","last_active":"2024-02-02T13:29:05.885007+0000","last_peered":"2024-02-02T13:29:05.885007+0000","last_clean":"2024-02-02T13:29:05.885007+0000","last_became_active":"2024-02-02T13:29:04.901491+0000","last_became_peered":"2024-02-02T13:29:04.901491+0000","last_unstale":"2024-02-02T13:29:05.885007+0000","last_undegraded":"2024-02-02T13:29:05.885007+0000","last_fullsized":"2024-02-02T13:29:05.885007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:21:10.301191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899943+0000","last_change":"2024-02-02T13:29:04.407899+0000","last_active":"2024-02-02T13:29:04.899943+0000","last_peered":"2024-02-02T13:29:04.899943+0000","last_clean":"2024-02-02T13:29:04.899943+0000","last_became_active":"2024-02-02T13:29:04.407242+0000","last_became_peered":"2024-02-02T13:29:04.407242+0000","last_unstale":"2024-02-02T13:29:04.899943+0000","last_undegraded":"2024-02-02T13:29:04.899943+0000","last_fullsized":"2024-02-02T13:29:04.899943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:50:57.751135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884829+0000","last_change":"2024-02-02T13:29:04.908733+0000","last_active":"2024-02-02T13:29:05.884829+0000","last_peered":"2024-02-02T13:29:05.884829+0000","last_clean":"2024-02-02T13:29:05.884829+0000","last_became_active":"2024-02-02T13:29:04.908364+0000","last_became_peered":"2024-02-02T13:29:04.908364+0000","last_unstale":"2024-02-02T13:29:05.884829+0000","last_undegraded":"2024-02-02T13:29:05.884829+0000","last_fullsized":"2024-02-02T13:29:05.884829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:02.835735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885336+0000","last_change":"2024-02-02T13:29:04.409834+0000","last_active":"2024-02-02T13:29:05.885336+0000","last_peered":"2024-02-02T13:29:05.885336+0000","last_clean":"2024-02-02T13:29:05.885336+0000","last_became_active":"2024-02-02T13:29:04.409063+0000","last_became_peered":"2024-02-02T13:29:04.409063+0000","last_unstale":"2024-02-02T13:29:05.885336+0000","last_undegraded":"2024-02-02T13:29:05.885336+0000","last_fullsized":"2024-02-02T13:29:05.885336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:33.577918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.882993+0000","last_change":"2024-02-02T13:29:04.881208+0000","last_active":"2024-02-02T13:29:04.882993+0000","last_peered":"2024-02-02T13:29:04.882993+0000","last_clean":"2024-02-02T13:29:04.882993+0000","last_became_active":"2024-02-02T13:29:04.880973+0000","last_became_peered":"2024-02-02T13:29:04.880973+0000","last_unstale":"2024-02-02T13:29:04.882993+0000","last_undegraded":"2024-02-02T13:29:04.882993+0000","last_fullsized":"2024-02-02T13:29:04.882993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:31:25.386243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915563+0000","last_change":"2024-02-02T13:29:04.409887+0000","last_active":"2024-02-02T13:29:04.915563+0000","last_peered":"2024-02-02T13:29:04.915563+0000","last_clean":"2024-02-02T13:29:04.915563+0000","last_became_active":"2024-02-02T13:29:04.409541+0000","last_became_peered":"2024-02-02T13:29:04.409541+0000","last_unstale":"2024-02-02T13:29:04.915563+0000","last_undegraded":"2024-02-02T13:29:04.915563+0000","last_fullsized":"2024-02-02T13:29:04.915563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:50.281988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.897261+0000","last_change":"2024-02-02T13:29:04.897199+0000","last_active":"2024-02-02T13:29:04.897261+0000","last_peered":"2024-02-02T13:29:04.897261+0000","last_clean":"2024-02-02T13:29:04.897261+0000","last_became_active":"2024-02-02T13:29:04.896967+0000","last_became_peered":"2024-02-02T13:29:04.896967+0000","last_unstale":"2024-02-02T13:29:04.897261+0000","last_undegraded":"2024-02-02T13:29:04.897261+0000","last_fullsized":"2024-02-02T13:29:04.897261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:51.287648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881632+0000","last_change":"2024-02-02T13:29:04.398676+0000","last_active":"2024-02-02T13:29:05.881632+0000","last_peered":"2024-02-02T13:29:05.881632+0000","last_clean":"2024-02-02T13:29:05.881632+0000","last_became_active":"2024-02-02T13:29:04.397930+0000","last_became_peered":"2024-02-02T13:29:04.397930+0000","last_unstale":"2024-02-02T13:29:05.881632+0000","last_undegraded":"2024-02-02T13:29:05.881632+0000","last_fullsized":"2024-02-02T13:29:05.881632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:45.700326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.888150+0000","last_change":"2024-02-02T13:29:04.888075+0000","last_active":"2024-02-02T13:29:04.888150+0000","last_peered":"2024-02-02T13:29:04.888150+0000","last_clean":"2024-02-02T13:29:04.888150+0000","last_became_active":"2024-02-02T13:29:04.887683+0000","last_became_peered":"2024-02-02T13:29:04.887683+0000","last_unstale":"2024-02-02T13:29:04.888150+0000","last_undegraded":"2024-02-02T13:29:04.888150+0000","last_fullsized":"2024-02-02T13:29:04.888150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:51:18.141044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.893201+0000","last_change":"2024-02-02T13:29:04.401796+0000","last_active":"2024-02-02T13:29:04.893201+0000","last_peered":"2024-02-02T13:29:04.893201+0000","last_clean":"2024-02-02T13:29:04.893201+0000","last_became_active":"2024-02-02T13:29:04.401292+0000","last_became_peered":"2024-02-02T13:29:04.401292+0000","last_unstale":"2024-02-02T13:29:04.893201+0000","last_undegraded":"2024-02-02T13:29:04.893201+0000","last_fullsized":"2024-02-02T13:29:04.893201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:17.180073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908829+0000","last_change":"2024-02-02T13:29:04.908777+0000","last_active":"2024-02-02T13:29:04.908829+0000","last_peered":"2024-02-02T13:29:04.908829+0000","last_clean":"2024-02-02T13:29:04.908829+0000","last_became_active":"2024-02-02T13:29:04.908613+0000","last_became_peered":"2024-02-02T13:29:04.908613+0000","last_unstale":"2024-02-02T13:29:04.908829+0000","last_undegraded":"2024-02-02T13:29:04.908829+0000","last_fullsized":"2024-02-02T13:29:04.908829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:09.586099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883015+0000","last_change":"2024-02-02T13:29:04.390122+0000","last_active":"2024-02-02T13:29:05.883015+0000","last_peered":"2024-02-02T13:29:05.883015+0000","last_clean":"2024-02-02T13:29:05.883015+0000","last_became_active":"2024-02-02T13:29:04.389480+0000","last_became_peered":"2024-02-02T13:29:04.389480+0000","last_unstale":"2024-02-02T13:29:05.883015+0000","last_undegraded":"2024-02-02T13:29:05.883015+0000","last_fullsized":"2024-02-02T13:29:05.883015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:30.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908031+0000","last_change":"2024-02-02T13:29:04.907972+0000","last_active":"2024-02-02T13:29:04.908031+0000","last_peered":"2024-02-02T13:29:04.908031+0000","last_clean":"2024-02-02T13:29:04.908031+0000","last_became_active":"2024-02-02T13:29:04.907742+0000","last_became_peered":"2024-02-02T13:29:04.907742+0000","last_unstale":"2024-02-02T13:29:04.908031+0000","last_undegraded":"2024-02-02T13:29:04.908031+0000","last_fullsized":"2024-02-02T13:29:04.908031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:00.475514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.883486+0000","last_change":"2024-02-02T13:29:04.399046+0000","last_active":"2024-02-02T13:29:04.883486+0000","last_peered":"2024-02-02T13:29:04.883486+0000","last_clean":"2024-02-02T13:29:04.883486+0000","last_became_active":"2024-02-02T13:29:04.398670+0000","last_became_peered":"2024-02-02T13:29:04.398670+0000","last_unstale":"2024-02-02T13:29:04.883486+0000","last_undegraded":"2024-02-02T13:29:04.883486+0000","last_fullsized":"2024-02-02T13:29:04.883486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:11.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890209+0000","last_change":"2024-02-02T13:29:04.890161+0000","last_active":"2024-02-02T13:29:04.890209+0000","last_peered":"2024-02-02T13:29:04.890209+0000","last_clean":"2024-02-02T13:29:04.890209+0000","last_became_active":"2024-02-02T13:29:04.889935+0000","last_became_peered":"2024-02-02T13:29:04.889935+0000","last_unstale":"2024-02-02T13:29:04.890209+0000","last_undegraded":"2024-02-02T13:29:04.890209+0000","last_fullsized":"2024-02-02T13:29:04.890209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:49:54.416602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915199+0000","last_change":"2024-02-02T13:29:04.409110+0000","last_active":"2024-02-02T13:29:04.915199+0000","last_peered":"2024-02-02T13:29:04.915199+0000","last_clean":"2024-02-02T13:29:04.915199+0000","last_became_active":"2024-02-02T13:29:04.408068+0000","last_became_peered":"2024-02-02T13:29:04.408068+0000","last_unstale":"2024-02-02T13:29:04.915199+0000","last_undegraded":"2024-02-02T13:29:04.915199+0000","last_fullsized":"2024-02-02T13:29:04.915199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:56.288505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881257+0000","last_change":"2024-02-02T13:29:04.886599+0000","last_active":"2024-02-02T13:29:05.881257+0000","last_peered":"2024-02-02T13:29:05.881257+0000","last_clean":"2024-02-02T13:29:05.881257+0000","last_became_active":"2024-02-02T13:29:04.885855+0000","last_became_peered":"2024-02-02T13:29:04.885855+0000","last_unstale":"2024-02-02T13:29:05.881257+0000","last_undegraded":"2024-02-02T13:29:05.881257+0000","last_fullsized":"2024-02-02T13:29:05.881257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:15.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881002+0000","last_change":"2024-02-02T13:29:04.401332+0000","last_active":"2024-02-02T13:29:05.881002+0000","last_peered":"2024-02-02T13:29:05.881002+0000","last_clean":"2024-02-02T13:29:05.881002+0000","last_became_active":"2024-02-02T13:29:04.401023+0000","last_became_peered":"2024-02-02T13:29:04.401023+0000","last_unstale":"2024-02-02T13:29:05.881002+0000","last_undegraded":"2024-02-02T13:29:05.881002+0000","last_fullsized":"2024-02-02T13:29:05.881002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:26.742938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882827+0000","last_change":"2024-02-02T13:29:04.914246+0000","last_active":"2024-02-02T13:29:05.882827+0000","last_peered":"2024-02-02T13:29:05.882827+0000","last_clean":"2024-02-02T13:29:05.882827+0000","last_became_active":"2024-02-02T13:29:04.913569+0000","last_became_peered":"2024-02-02T13:29:04.913569+0000","last_unstale":"2024-02-02T13:29:05.882827+0000","last_undegraded":"2024-02-02T13:29:05.882827+0000","last_fullsized":"2024-02-02T13:29:05.882827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:16:26.291379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.879142+0000","last_change":"2024-02-02T13:29:04.401423+0000","last_active":"2024-02-02T13:29:04.879142+0000","last_peered":"2024-02-02T13:29:04.879142+0000","last_clean":"2024-02-02T13:29:04.879142+0000","last_became_active":"2024-02-02T13:29:04.399922+0000","last_became_peered":"2024-02-02T13:29:04.399922+0000","last_unstale":"2024-02-02T13:29:04.879142+0000","last_undegraded":"2024-02-02T13:29:04.879142+0000","last_fullsized":"2024-02-02T13:29:04.879142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:18:31.933369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885046+0000","last_change":"2024-02-02T13:29:04.914774+0000","last_active":"2024-02-02T13:29:05.885046+0000","last_peered":"2024-02-02T13:29:05.885046+0000","last_clean":"2024-02-02T13:29:05.885046+0000","last_became_active":"2024-02-02T13:29:04.914371+0000","last_became_peered":"2024-02-02T13:29:04.914371+0000","last_unstale":"2024-02-02T13:29:05.885046+0000","last_undegraded":"2024-02-02T13:29:05.885046+0000","last_fullsized":"2024-02-02T13:29:05.885046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:01:22.123982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.884459+0000","last_change":"2024-02-02T13:29:04.406343+0000","last_active":"2024-02-02T13:29:04.884459+0000","last_peered":"2024-02-02T13:29:04.884459+0000","last_clean":"2024-02-02T13:29:04.884459+0000","last_became_active":"2024-02-02T13:29:04.405829+0000","last_became_peered":"2024-02-02T13:29:04.405829+0000","last_unstale":"2024-02-02T13:29:04.884459+0000","last_undegraded":"2024-02-02T13:29:04.884459+0000","last_fullsized":"2024-02-02T13:29:04.884459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:30.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.887290+0000","last_change":"2024-02-02T13:29:04.887120+0000","last_active":"2024-02-02T13:29:04.887290+0000","last_peered":"2024-02-02T13:29:04.887290+0000","last_clean":"2024-02-02T13:29:04.887290+0000","last_became_active":"2024-02-02T13:29:04.886857+0000","last_became_peered":"2024-02-02T13:29:04.886857+0000","last_unstale":"2024-02-02T13:29:04.887290+0000","last_undegraded":"2024-02-02T13:29:04.887290+0000","last_fullsized":"2024-02-02T13:29:04.887290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:28.794701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.879060+0000","last_change":"2024-02-02T13:29:04.399859+0000","last_active":"2024-02-02T13:29:04.879060+0000","last_peered":"2024-02-02T13:29:04.879060+0000","last_clean":"2024-02-02T13:29:04.879060+0000","last_became_active":"2024-02-02T13:29:04.397846+0000","last_became_peered":"2024-02-02T13:29:04.397846+0000","last_unstale":"2024-02-02T13:29:04.879060+0000","last_undegraded":"2024-02-02T13:29:04.879060+0000","last_fullsized":"2024-02-02T13:29:04.879060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:45:37.877511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.913942+0000","last_change":"2024-02-02T13:29:04.913527+0000","last_active":"2024-02-02T13:29:04.913942+0000","last_peered":"2024-02-02T13:29:04.913942+0000","last_clean":"2024-02-02T13:29:04.913942+0000","last_became_active":"2024-02-02T13:29:04.912030+0000","last_became_peered":"2024-02-02T13:29:04.912030+0000","last_unstale":"2024-02-02T13:29:04.913942+0000","last_undegraded":"2024-02-02T13:29:04.913942+0000","last_fullsized":"2024-02-02T13:29:04.913942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:53.821488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.876883+0000","last_change":"2024-02-02T13:29:04.406966+0000","last_active":"2024-02-02T13:29:04.876883+0000","last_peered":"2024-02-02T13:29:04.876883+0000","last_clean":"2024-02-02T13:29:04.876883+0000","last_became_active":"2024-02-02T13:29:04.406718+0000","last_became_peered":"2024-02-02T13:29:04.406718+0000","last_unstale":"2024-02-02T13:29:04.876883+0000","last_undegraded":"2024-02-02T13:29:04.876883+0000","last_fullsized":"2024-02-02T13:29:04.876883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:50.281988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.885903+0000","last_change":"2024-02-02T13:29:04.885614+0000","last_active":"2024-02-02T13:29:04.885903+0000","last_peered":"2024-02-02T13:29:04.885903+0000","last_clean":"2024-02-02T13:29:04.885903+0000","last_became_active":"2024-02-02T13:29:04.885402+0000","last_became_peered":"2024-02-02T13:29:04.885402+0000","last_unstale":"2024-02-02T13:29:04.885903+0000","last_undegraded":"2024-02-02T13:29:04.885903+0000","last_fullsized":"2024-02-02T13:29:04.885903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:12:04.591193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884058+0000","last_change":"2024-02-02T13:29:04.407855+0000","last_active":"2024-02-02T13:29:05.884058+0000","last_peered":"2024-02-02T13:29:05.884058+0000","last_clean":"2024-02-02T13:29:05.884058+0000","last_became_active":"2024-02-02T13:29:04.407303+0000","last_became_peered":"2024-02-02T13:29:04.407303+0000","last_unstale":"2024-02-02T13:29:05.884058+0000","last_undegraded":"2024-02-02T13:29:05.884058+0000","last_fullsized":"2024-02-02T13:29:05.884058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:31.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908081+0000","last_change":"2024-02-02T13:29:04.906628+0000","last_active":"2024-02-02T13:29:04.908081+0000","last_peered":"2024-02-02T13:29:04.908081+0000","last_clean":"2024-02-02T13:29:04.908081+0000","last_became_active":"2024-02-02T13:29:04.899186+0000","last_became_peered":"2024-02-02T13:29:04.899186+0000","last_unstale":"2024-02-02T13:29:04.908081+0000","last_undegraded":"2024-02-02T13:29:04.908081+0000","last_fullsized":"2024-02-02T13:29:04.908081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:31.600140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.892923+0000","last_change":"2024-02-02T13:29:04.407727+0000","last_active":"2024-02-02T13:29:04.892923+0000","last_peered":"2024-02-02T13:29:04.892923+0000","last_clean":"2024-02-02T13:29:04.892923+0000","last_became_active":"2024-02-02T13:29:04.406856+0000","last_became_peered":"2024-02-02T13:29:04.406856+0000","last_unstale":"2024-02-02T13:29:04.892923+0000","last_undegraded":"2024-02-02T13:29:04.892923+0000","last_fullsized":"2024-02-02T13:29:04.892923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:20.929409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898857+0000","last_change":"2024-02-02T13:29:04.898786+0000","last_active":"2024-02-02T13:29:04.898857+0000","last_peered":"2024-02-02T13:29:04.898857+0000","last_clean":"2024-02-02T13:29:04.898857+0000","last_became_active":"2024-02-02T13:29:04.898423+0000","last_became_peered":"2024-02-02T13:29:04.898423+0000","last_unstale":"2024-02-02T13:29:04.898857+0000","last_undegraded":"2024-02-02T13:29:04.898857+0000","last_fullsized":"2024-02-02T13:29:04.898857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:04:12.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915829+0000","last_change":"2024-02-02T13:29:04.406577+0000","last_active":"2024-02-02T13:29:04.915829+0000","last_peered":"2024-02-02T13:29:04.915829+0000","last_clean":"2024-02-02T13:29:04.915829+0000","last_became_active":"2024-02-02T13:29:04.406342+0000","last_became_peered":"2024-02-02T13:29:04.406342+0000","last_unstale":"2024-02-02T13:29:04.915829+0000","last_undegraded":"2024-02-02T13:29:04.915829+0000","last_fullsized":"2024-02-02T13:29:04.915829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:43:55.115001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.905952+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:29:04.905952+0000","last_peered":"2024-02-02T13:29:04.905952+0000","last_clean":"2024-02-02T13:29:04.905952+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:29:04.905952+0000","last_undegraded":"2024-02-02T13:29:04.905952+0000","last_fullsized":"2024-02-02T13:29:04.905952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1023,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194289,"statfs":{"total":200000000,"available":198952360,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.0840000000000001,"15min":1.0609999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6279999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.095},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1299999999999999,"15min":1.1319999999999999},"min":{"1min":0.80500000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.613,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.0389999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1000000000000001,"15min":1.0900000000000001},"min":{"1min":0.72799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8839999999999999,"5min":1.893,"15min":1.893},"last":0.878},{"interface":"front","average":{"1min":1.179,"5min":1.127,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5149999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.2110000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.226,"15min":1.2170000000000001},"min":{"1min":0.98999999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.909,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.1859999999999999,"15min":1.1899999999999999},"min":{"1min":0.89900000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.159}]},{"osd":3,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":1.03,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6339999999999999,"5min":1.819,"15min":1.819},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3180000000000001,"15min":1.3069999999999999},"min":{"1min":1.0609999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.133,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.2809999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.238,"15min":1.1870000000000001},"min":{"1min":0.75,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2909999999999999,"15min":1.3080000000000001},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1799999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.3500000000000001},"min":{"1min":0.95099999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.448,"15min":1.462},"min":{"1min":0.91300000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0750000000000002,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.3500000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5349999999999999,"15min":1.546},"min":{"1min":1.022,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.1869999999999998,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.96299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.8200000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.458}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":438,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194873,"statfs":{"total":200000000,"available":199550571,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.046,"15min":1.008},"min":{"1min":0.80600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5740000000000001,"5min":1.577,"15min":1.577},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.0369999999999999,"15min":1.008},"min":{"1min":0.89400000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0369999999999999,"15min":1.0149999999999999},"min":{"1min":0.85499999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":1.556,"15min":1.556},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.107,"15min":1.0640000000000001},"min":{"1min":0.89700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.508,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.343}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.127,"15min":1.0640000000000001},"min":{"1min":0.86299999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.498},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.169,"15min":1.153},"min":{"1min":0.871,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5820000000000001,"5min":1.845,"15min":1.845},"last":1.196}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2609999999999999,"15min":1.266},"min":{"1min":0.96999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.028},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1659999999999999,"15min":1.1339999999999999},"min":{"1min":0.93600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0820000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2609999999999999,"15min":1.2050000000000001},"min":{"1min":0.85899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.74,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.417,"15min":1.391},"min":{"1min":1.1519999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6719999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.444}]},{"osd":5,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.274,"15min":1.2709999999999999},"min":{"1min":0.81200000000000006,"5min":0.75,"15min":0.75},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.476},{"interface":"front","average":{"1min":1.419,"5min":1.391,"15min":1.345},"min":{"1min":1.0569999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.71,"5min":1.895,"15min":1.895},"last":1.6910000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":1.0920000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.9259999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.526},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4470000000000001,"15min":1.4370000000000001},"min":{"1min":1.1000000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.9470000000000001,"5min":1.954,"15min":1.954},"last":1.3959999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":446,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194866,"statfs":{"total":200000000,"available":199543254,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.032,"15min":1.022},"min":{"1min":0.75700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.401,"5min":1.431,"15min":1.431},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.04},"min":{"1min":0.79300000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3700000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.3300000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.212,"15min":1.28},"min":{"1min":0.88,"5min":0.62,"15min":0.62},"max":{"1min":1.6830000000000001,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.1259999999999999,"15min":1.157},"min":{"1min":0.83799999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.877,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.23,"15min":1.212},"min":{"1min":0.91200000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5740000000000001,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1830000000000001,"15min":1.173},"min":{"1min":0.83899999999999997,"5min":0.752,"15min":0.752},"max":{"1min":1.502,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.179,"15min":1.1830000000000001},"min":{"1min":0.93600000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.389,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.238},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2110000000000001,"15min":1.1619999999999999},"min":{"1min":0.98999999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.514,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.367}]},{"osd":4,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2010000000000001,"15min":1.224},"min":{"1min":0.85499999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5960000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.266,"15min":1.276},"min":{"1min":0.79500000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5920000000000001,"5min":1.833,"15min":1.833},"last":1.0720000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3,"15min":1.292},"min":{"1min":0.92600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.538,"5min":1.877,"15min":1.877},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.353,"15min":1.345},"min":{"1min":0.95399999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6879999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4019999999999999,"15min":1.3979999999999999},"min":{"1min":1.012,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.762,"5min":1.881,"15min":1.881},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.385,"15min":1.3600000000000001},"min":{"1min":1.036,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.645,"5min":1.806,"15min":1.806},"last":1.4450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":445,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194867,"statfs":{"total":200000000,"available":199544293,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.0600000000000001,"15min":0.98099999999999998},"min":{"1min":0.81200000000000006,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.536,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.78300000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.514}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.878,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5640000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.079,"15min":1.0269999999999999},"min":{"1min":0.81000000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.694,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.149,"15min":1.0860000000000001},"min":{"1min":0.78900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6100000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1140000000000001,"15min":1.056},"min":{"1min":0.86799999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.29}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2370000000000001,"15min":1.1699999999999999},"min":{"1min":0.90300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.663,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2769999999999999,"15min":1.228},"min":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6519999999999999,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.127}]},{"osd":5,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1659999999999999,"15min":1.1200000000000001},"min":{"1min":0.76700000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6040000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.2909999999999999,"15min":1.2589999999999999},"min":{"1min":0.91300000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7390000000000001,"5min":2.069,"15min":2.069},"last":1.6000000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3149999999999999,"15min":1.2749999999999999},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.788,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.353},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.403,"15min":1.3740000000000001},"min":{"1min":1,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9410000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.385}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.403,"15min":1.365},"min":{"1min":1.171,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.98,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4390000000000001,"15min":1.4319999999999999},"min":{"1min":0.97499999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7689999999999999,"5min":2.698,"15min":2.698},"last":1.486}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":439,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549532,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.266,"5min":1.302,"15min":1.302},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.89100000000000001,"15min":0.88300000000000001},"min":{"1min":0.73699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.619,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.92500000000000004,"15min":0.89400000000000002},"min":{"1min":0.81100000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3640000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.048,"15min":1.018},"min":{"1min":0.90200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6359999999999999,"5min":2.125,"15min":2.125},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0069999999999999,"15min":0.95299999999999996},"min":{"1min":0.84599999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.90400000000000003,"15min":0.88800000000000001},"min":{"1min":0.66500000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.206,"15min":1.1859999999999999},"min":{"1min":0.97599999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.669,"5min":3.585,"15min":3.585},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.244,"15min":1.2030000000000001},"min":{"1min":1.036,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.552,"5min":1.877,"15min":1.877},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.212,"15min":1.2210000000000001},"min":{"1min":0.94899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6579999999999999,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":1.075,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.7070000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.91700000000000004}]},{"osd":6,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.3120000000000001,"15min":1.256},"min":{"1min":1.2070000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.254,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.96999999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.55,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.2749999999999999,"15min":1.2609999999999999},"min":{"1min":1.153,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7190000000000001,"5min":3.391,"15min":3.391},"last":1.044},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3500000000000001,"15min":1.3120000000000001},"min":{"1min":1.2849999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7909999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.40799999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.143},{"interface":"front","average":{"1min":0.997,"5min":0.81899999999999995,"15min":0.76900000000000002},"min":{"1min":0.56599999999999995,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.123,"5min":0.91100000000000003,"15min":0.84399999999999997},"min":{"1min":0.73699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2849999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.85099999999999998,"15min":0.77700000000000002},"min":{"1min":0.61799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.087},{"interface":"front","average":{"1min":1.165,"5min":0.998,"15min":0.96999999999999997},"min":{"1min":0.68000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2050000000000001,"15min":1.165},"min":{"1min":1.018,"5min":0.748,"15min":0.748},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.1850000000000001,"15min":1.145},"min":{"1min":0.81799999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5620000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.401}]},{"osd":5,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.147,"15min":1.0700000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":1.1699999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.2070000000000001,"15min":1.161},"min":{"1min":0.89900000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.502},{"interface":"front","average":{"1min":1.329,"5min":1.2230000000000001,"15min":1.1870000000000001},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.421}]},{"osd":7,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.246,"15min":1.1850000000000001},"min":{"1min":0.92200000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.266,"15min":1.198},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.21}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548483,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91800000000000004,"15min":1.03},"min":{"1min":0.63,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.367,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.1579999999999999,"15min":1.4139999999999999},"min":{"1min":0.77600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.3120000000000001,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":1.296}]},{"osd":2,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.89500000000000002,"15min":0.91900000000000004},"min":{"1min":0.71899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.431,"5min":1.639,"15min":1.639},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.016,"15min":1.0960000000000001},"min":{"1min":0.79600000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.484,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":1.018}]},{"osd":3,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1439999999999999,"15min":1.2729999999999999},"min":{"1min":0.88300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.647,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":1.137},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.161,"15min":1.3129999999999999},"min":{"1min":0.89500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7030000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2310000000000001,"15min":1.343},"min":{"1min":0.99199999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.627,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.296,"15min":1.3740000000000001},"min":{"1min":1.0840000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.581,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.415}]},{"osd":5,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3700000000000001,"15min":1.474},"min":{"1min":1.119,"5min":0.753,"15min":0.753},"max":{"1min":1.6080000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.353,"15min":1.4530000000000001},"min":{"1min":1.1839999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6990000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.355}]},{"osd":6,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4370000000000001,"15min":1.603},"min":{"1min":1.0209999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.782,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.409,"15min":1.514},"min":{"1min":1.1060000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.806,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.226}]},{"osd":7,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.452,"15min":1.575},"min":{"1min":1.1040000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1469999999999998,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.329},{"interface":"front","average":{"1min":1.464,"5min":1.496,"15min":1.6439999999999999},"min":{"1min":1.23,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7350000000000001,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.389}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1015,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194297,"statfs":{"total":200000000,"available":198960295,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.98399999999999999,"15min":1.1699999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.472,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.2130000000000001,"15min":1.5489999999999999},"min":{"1min":0.45300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.343,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.425}]},{"osd":2,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0529999999999999,"15min":1.1919999999999999},"min":{"1min":0.48499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.401,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":1.244},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.236,"15min":1.5429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3129999999999999,"5min":10.781000000000001,"15min":10.781000000000001},"last":1.0049999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.093,"15min":1.274},"min":{"1min":0.64000000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.427,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.1890000000000001,"15min":1.345},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.5449999999999999,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2689999999999999,"15min":1.3640000000000001},"min":{"1min":0.996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.621,"5min":3.669,"15min":3.669},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3360000000000001,"15min":1.4239999999999999},"min":{"1min":0.78200000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.774,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.3340000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.579,"15min":1.891},"min":{"1min":0.83199999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6870000000000001,"5min":10.651,"15min":10.651},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.349,"15min":1.4510000000000001},"min":{"1min":1.0109999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5840000000000001,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.379}]},{"osd":6,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.395,"15min":1.494},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.8300000000000001,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.5980000000000001,"15min":1.9239999999999999},"min":{"1min":1.05,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6160000000000001,"5min":10.708,"15min":10.708},"last":1.129}]},{"osd":7,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.637,"15min":1.931},"min":{"1min":0.872,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5900000000000001,"5min":10.74,"15min":10.74},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3779999999999999,"15min":1.468},"min":{"1min":1.0920000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6539999999999999,"5min":3.863,"15min":3.863},"last":1.5149999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:29:09.254 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:29:09.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.501+0000 7fe160600640 1 Processor -- start 2024-02-02T13:29:09.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.502+0000 7fe160600640 1 -- start start 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15815d650 0x7fe1580cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe1580ce070 0x7fe1580ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1581610b0 0x7fe158161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cea30 con 0x7fe1580ce070 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cded0 con 0x7fe1581610b0 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe160600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580d12c0 con 0x7fe15815d650 2024-02-02T13:29:09.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe15d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe1580ce070 0x7fe1580ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe15d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe1580ce070 0x7fe1580ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51034/0 (socket says 172.21.15.91:51034) 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.503+0000 7fe15d200640 1 -- 172.21.15.91:0/3793808091 learned_addr learned my addr 172.21.15.91:0/3793808091 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.504+0000 7fe15d200640 1 -- 172.21.15.91:0/3793808091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15815d650 msgr2=0x7fe1580cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.504+0000 7fe15d200640 1 --2- 172.21.15.91:0/3793808091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15815d650 0x7fe1580cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.504+0000 7fe15d200640 1 -- 172.21.15.91:0/3793808091 >> v2:172.21.15.113:3300/0 conn(0x7fe1581610b0 msgr2=0x7fe158161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.504+0000 7fe15d200640 1 --2- 172.21.15.91:0/3793808091 >> v2:172.21.15.113:3300/0 conn(0x7fe1581610b0 0x7fe158161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.504+0000 7fe15d200640 1 -- 172.21.15.91:0/3793808091 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158071410 con 0x7fe1580ce070 2024-02-02T13:29:09.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.505+0000 7fe15d200640 1 --2- 172.21.15.91:0/3793808091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1580ce070 0x7fe1580ce4f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe1580b8d40 tx=0x7fe14002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894121f281353fee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.505+0000 7fe157e00640 1 -- 172.21.15.91:0/3793808091 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14002ca70 con 0x7fe1580ce070 2024-02-02T13:29:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.505+0000 7fe157e00640 1 -- 172.21.15.91:0/3793808091 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14002cbd0 con 0x7fe1580ce070 2024-02-02T13:29:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.505+0000 7fe157e00640 1 -- 172.21.15.91:0/3793808091 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400343f0 con 0x7fe1580ce070 2024-02-02T13:29:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 -- 172.21.15.91:0/3793808091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1580ce070 msgr2=0x7fe1580ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 --2- 172.21.15.91:0/3793808091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1580ce070 0x7fe1580ce4f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe1580b8d40 tx=0x7fe14002bed0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 -- 172.21.15.91:0/3793808091 shutdown_connections 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 --2- 172.21.15.91:0/3793808091 >> v2:172.21.15.113:3300/0 conn(0x7fe1581610b0 0x7fe158161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 --2- 172.21.15.91:0/3793808091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1580ce070 0x7fe1580ce4f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 --2- 172.21.15.91:0/3793808091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15815d650 0x7fe1580cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 -- 172.21.15.91:0/3793808091 >> 172.21.15.91:0/3793808091 conn(0x7fe1580d30a0 msgr2=0x7fe1580d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 -- 172.21.15.91:0/3793808091 shutdown_connections 2024-02-02T13:29:09.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.506+0000 7fe160600640 1 -- 172.21.15.91:0/3793808091 wait complete. 2024-02-02T13:29:09.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.507+0000 7fe160600640 1 Processor -- start 2024-02-02T13:29:09.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.508+0000 7fe160600640 1 -- start start 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.508+0000 7fe160600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 0x7fe1580c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe160600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe160600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1581610b0 0x7fe1581581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe160600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580ca170 con 0x7fe15815d650 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe160600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581603f0 con 0x7fe1580ce070 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe160600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581606d0 con 0x7fe1581610b0 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51048/0 (socket says 172.21.15.91:51048) 2024-02-02T13:29:09.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 -- 172.21.15.91:0/4034939393 learned_addr learned my addr 172.21.15.91:0/4034939393 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 -- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1581610b0 msgr2=0x7fe1581581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1581610b0 0x7fe1581581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15dc00640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 0x7fe1580c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 -- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 msgr2=0x7fe1580c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 0x7fe1580c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15d200640 1 -- 172.21.15.91:0/4034939393 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140009b30 con 0x7fe15815d650 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.509+0000 7fe15dc00640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 0x7fe1580c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:09.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe15d200640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe140000f80 tx=0x7fe140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14003e450 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14003e5f0 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1580ca410 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14003e9a0 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.510+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15815ec40 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.511+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15815edd0 con 0x7fe15815d650 2024-02-02T13:29:09.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.512+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe14003eb00 con 0x7fe15815d650 2024-02-02T13:29:09.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.513+0000 7fe155600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 0x7fe13006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:09.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.513+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1400ab5c0 con 0x7fe15815d650 2024-02-02T13:29:09.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.513+0000 7fe15dc00640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 0x7fe13006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:09.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.514+0000 7fe15dc00640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 0x7fe13006c920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe14800a870 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:09.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.518+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe140076bb0 con 0x7fe15815d650 2024-02-02T13:29:09.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.704+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1580d0c70 con 0x7fe13006a460 2024-02-02T13:29:09.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.725+0000 7fe155600640 1 -- 172.21.15.91:0/4034939393 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282245 (secure 0 0 0) 0x7fe14800c7d0 con 0x7fe13006a460 2024-02-02T13:29:09.730 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:09.732 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:29:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.730+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 msgr2=0x7fe13006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.730+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 0x7fe13006c920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe14800a870 tx=0x7fe148008040 comp rx=0 tx=0).stop 2024-02-02T13:29:09.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 msgr2=0x7fe15815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe140000f80 tx=0x7fe140004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 shutdown_connections 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe13006a460 0x7fe13006c920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1581610b0 0x7fe1581581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe15815d650 0x7fe15815f9a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 --2- 172.21.15.91:0/4034939393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1580ce070 0x7fe1580c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 >> 172.21.15.91:0/4034939393 conn(0x7fe1580d30a0 msgr2=0x7fe1580d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 shutdown_connections 2024-02-02T13:29:09.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:09.731+0000 7fe160600640 1 -- 172.21.15.91:0/4034939393 wait complete. 2024-02-02T13:29:09.750 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-02-02T13:29:08.323822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557809,"statfs":{"total":1600000000,"available":1595196868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.950463"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881546+0000","last_change":"2024-02-02T13:29:04.909851+0000","last_active":"2024-02-02T13:29:05.881546+0000","last_peered":"2024-02-02T13:29:05.881546+0000","last_clean":"2024-02-02T13:29:05.881546+0000","last_became_active":"2024-02-02T13:29:04.909111+0000","last_became_peered":"2024-02-02T13:29:04.909111+0000","last_unstale":"2024-02-02T13:29:05.881546+0000","last_undegraded":"2024-02-02T13:29:05.881546+0000","last_fullsized":"2024-02-02T13:29:05.881546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:33:58.933407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908056+0000","last_change":"2024-02-02T13:29:04.906480+0000","last_active":"2024-02-02T13:29:04.908056+0000","last_peered":"2024-02-02T13:29:04.908056+0000","last_clean":"2024-02-02T13:29:04.908056+0000","last_became_active":"2024-02-02T13:29:04.906133+0000","last_became_peered":"2024-02-02T13:29:04.906133+0000","last_unstale":"2024-02-02T13:29:04.908056+0000","last_undegraded":"2024-02-02T13:29:04.908056+0000","last_fullsized":"2024-02-02T13:29:04.908056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:45.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889086+0000","last_change":"2024-02-02T13:29:04.888505+0000","last_active":"2024-02-02T13:29:04.889086+0000","last_peered":"2024-02-02T13:29:04.889086+0000","last_clean":"2024-02-02T13:29:04.889086+0000","last_became_active":"2024-02-02T13:29:04.888296+0000","last_became_peered":"2024-02-02T13:29:04.888296+0000","last_unstale":"2024-02-02T13:29:04.889086+0000","last_undegraded":"2024-02-02T13:29:04.889086+0000","last_fullsized":"2024-02-02T13:29:04.889086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:10:32.798262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898352+0000","last_change":"2024-02-02T13:29:04.898282+0000","last_active":"2024-02-02T13:29:04.898352+0000","last_peered":"2024-02-02T13:29:04.898352+0000","last_clean":"2024-02-02T13:29:04.898352+0000","last_became_active":"2024-02-02T13:29:04.897892+0000","last_became_peered":"2024-02-02T13:29:04.897892+0000","last_unstale":"2024-02-02T13:29:04.898352+0000","last_undegraded":"2024-02-02T13:29:04.898352+0000","last_fullsized":"2024-02-02T13:29:04.898352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:55:29.138956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.909334+0000","last_change":"2024-02-02T13:29:04.909282+0000","last_active":"2024-02-02T13:29:04.909334+0000","last_peered":"2024-02-02T13:29:04.909334+0000","last_clean":"2024-02-02T13:29:04.909334+0000","last_became_active":"2024-02-02T13:29:04.909149+0000","last_became_peered":"2024-02-02T13:29:04.909149+0000","last_unstale":"2024-02-02T13:29:04.909334+0000","last_undegraded":"2024-02-02T13:29:04.909334+0000","last_fullsized":"2024-02-02T13:29:04.909334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:57.294165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908990+0000","last_change":"2024-02-02T13:29:04.908565+0000","last_active":"2024-02-02T13:29:04.908990+0000","last_peered":"2024-02-02T13:29:04.908990+0000","last_clean":"2024-02-02T13:29:04.908990+0000","last_became_active":"2024-02-02T13:29:04.908248+0000","last_became_peered":"2024-02-02T13:29:04.908248+0000","last_unstale":"2024-02-02T13:29:04.908990+0000","last_undegraded":"2024-02-02T13:29:04.908990+0000","last_fullsized":"2024-02-02T13:29:04.908990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:46.705986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.880772+0000","last_change":"2024-02-02T13:29:04.884957+0000","last_active":"2024-02-02T13:29:05.880772+0000","last_peered":"2024-02-02T13:29:05.880772+0000","last_clean":"2024-02-02T13:29:05.880772+0000","last_became_active":"2024-02-02T13:29:04.883916+0000","last_became_peered":"2024-02-02T13:29:04.883916+0000","last_unstale":"2024-02-02T13:29:05.880772+0000","last_undegraded":"2024-02-02T13:29:05.880772+0000","last_fullsized":"2024-02-02T13:29:05.880772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:08.230800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908119+0000","last_change":"2024-02-02T13:29:04.907789+0000","last_active":"2024-02-02T13:29:04.908119+0000","last_peered":"2024-02-02T13:29:04.908119+0000","last_clean":"2024-02-02T13:29:04.908119+0000","last_became_active":"2024-02-02T13:29:04.900053+0000","last_became_peered":"2024-02-02T13:29:04.900053+0000","last_unstale":"2024-02-02T13:29:04.908119+0000","last_undegraded":"2024-02-02T13:29:04.908119+0000","last_fullsized":"2024-02-02T13:29:04.908119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:21:10.301191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889387+0000","last_change":"2024-02-02T13:29:04.889001+0000","last_active":"2024-02-02T13:29:04.889387+0000","last_peered":"2024-02-02T13:29:04.889387+0000","last_clean":"2024-02-02T13:29:04.889387+0000","last_became_active":"2024-02-02T13:29:04.888734+0000","last_became_peered":"2024-02-02T13:29:04.888734+0000","last_unstale":"2024-02-02T13:29:04.889387+0000","last_undegraded":"2024-02-02T13:29:04.889387+0000","last_fullsized":"2024-02-02T13:29:04.889387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:05:57.256014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884627+0000","last_change":"2024-02-02T13:29:04.902839+0000","last_active":"2024-02-02T13:29:05.884627+0000","last_peered":"2024-02-02T13:29:05.884627+0000","last_clean":"2024-02-02T13:29:05.884627+0000","last_became_active":"2024-02-02T13:29:04.902177+0000","last_became_peered":"2024-02-02T13:29:04.902177+0000","last_unstale":"2024-02-02T13:29:05.884627+0000","last_undegraded":"2024-02-02T13:29:05.884627+0000","last_fullsized":"2024-02-02T13:29:05.884627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:27.748598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881965+0000","last_change":"2024-02-02T13:29:04.910007+0000","last_active":"2024-02-02T13:29:05.881965+0000","last_peered":"2024-02-02T13:29:05.881965+0000","last_clean":"2024-02-02T13:29:05.881965+0000","last_became_active":"2024-02-02T13:29:04.909376+0000","last_became_peered":"2024-02-02T13:29:04.909376+0000","last_unstale":"2024-02-02T13:29:05.881965+0000","last_undegraded":"2024-02-02T13:29:05.881965+0000","last_fullsized":"2024-02-02T13:29:05.881965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:53.426160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.883104+0000","last_change":"2024-02-02T13:29:04.882980+0000","last_active":"2024-02-02T13:29:04.883104+0000","last_peered":"2024-02-02T13:29:04.883104+0000","last_clean":"2024-02-02T13:29:04.883104+0000","last_became_active":"2024-02-02T13:29:04.882376+0000","last_became_peered":"2024-02-02T13:29:04.882376+0000","last_unstale":"2024-02-02T13:29:04.883104+0000","last_undegraded":"2024-02-02T13:29:04.883104+0000","last_fullsized":"2024-02-02T13:29:04.883104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:46:40.349335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.901257+0000","last_change":"2024-02-02T13:29:04.901202+0000","last_active":"2024-02-02T13:29:04.901257+0000","last_peered":"2024-02-02T13:29:04.901257+0000","last_clean":"2024-02-02T13:29:04.901257+0000","last_became_active":"2024-02-02T13:29:04.900927+0000","last_became_peered":"2024-02-02T13:29:04.900927+0000","last_unstale":"2024-02-02T13:29:04.901257+0000","last_undegraded":"2024-02-02T13:29:04.901257+0000","last_fullsized":"2024-02-02T13:29:04.901257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:45.908362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.882673+0000","last_change":"2024-02-02T13:29:04.882518+0000","last_active":"2024-02-02T13:29:04.882673+0000","last_peered":"2024-02-02T13:29:04.882673+0000","last_clean":"2024-02-02T13:29:04.882673+0000","last_became_active":"2024-02-02T13:29:04.882310+0000","last_became_peered":"2024-02-02T13:29:04.882310+0000","last_unstale":"2024-02-02T13:29:04.882673+0000","last_undegraded":"2024-02-02T13:29:04.882673+0000","last_fullsized":"2024-02-02T13:29:04.882673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:43:56.120661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890181+0000","last_change":"2024-02-02T13:29:04.890133+0000","last_active":"2024-02-02T13:29:04.890181+0000","last_peered":"2024-02-02T13:29:04.890181+0000","last_clean":"2024-02-02T13:29:04.890181+0000","last_became_active":"2024-02-02T13:29:04.889907+0000","last_became_peered":"2024-02-02T13:29:04.889907+0000","last_unstale":"2024-02-02T13:29:04.890181+0000","last_undegraded":"2024-02-02T13:29:04.890181+0000","last_fullsized":"2024-02-02T13:29:04.890181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:15.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884080+0000","last_change":"2024-02-02T13:29:04.912080+0000","last_active":"2024-02-02T13:29:05.884080+0000","last_peered":"2024-02-02T13:29:05.884080+0000","last_clean":"2024-02-02T13:29:05.884080+0000","last_became_active":"2024-02-02T13:29:04.911575+0000","last_became_peered":"2024-02-02T13:29:04.911575+0000","last_unstale":"2024-02-02T13:29:05.884080+0000","last_undegraded":"2024-02-02T13:29:05.884080+0000","last_fullsized":"2024-02-02T13:29:05.884080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:46.705986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886522+0000","last_change":"2024-02-02T13:29:04.886461+0000","last_active":"2024-02-02T13:29:04.886522+0000","last_peered":"2024-02-02T13:29:04.886522+0000","last_clean":"2024-02-02T13:29:04.886522+0000","last_became_active":"2024-02-02T13:29:04.885876+0000","last_became_peered":"2024-02-02T13:29:04.885876+0000","last_unstale":"2024-02-02T13:29:04.886522+0000","last_undegraded":"2024-02-02T13:29:04.886522+0000","last_fullsized":"2024-02-02T13:29:04.886522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:30.923374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.900075+0000","last_change":"2024-02-02T13:29:04.899976+0000","last_active":"2024-02-02T13:29:04.900075+0000","last_peered":"2024-02-02T13:29:04.900075+0000","last_clean":"2024-02-02T13:29:04.900075+0000","last_became_active":"2024-02-02T13:29:04.899744+0000","last_became_peered":"2024-02-02T13:29:04.899744+0000","last_unstale":"2024-02-02T13:29:04.900075+0000","last_undegraded":"2024-02-02T13:29:04.900075+0000","last_fullsized":"2024-02-02T13:29:04.900075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:09.878784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899762+0000","last_change":"2024-02-02T13:29:04.899628+0000","last_active":"2024-02-02T13:29:04.899762+0000","last_peered":"2024-02-02T13:29:04.899762+0000","last_clean":"2024-02-02T13:29:04.899762+0000","last_became_active":"2024-02-02T13:29:04.898887+0000","last_became_peered":"2024-02-02T13:29:04.898887+0000","last_unstale":"2024-02-02T13:29:04.899762+0000","last_undegraded":"2024-02-02T13:29:04.899762+0000","last_fullsized":"2024-02-02T13:29:04.899762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:47:26.089514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.893724+0000","last_change":"2024-02-02T13:29:04.893659+0000","last_active":"2024-02-02T13:29:04.893724+0000","last_peered":"2024-02-02T13:29:04.893724+0000","last_clean":"2024-02-02T13:29:04.893724+0000","last_became_active":"2024-02-02T13:29:04.893432+0000","last_became_peered":"2024-02-02T13:29:04.893432+0000","last_unstale":"2024-02-02T13:29:04.893724+0000","last_undegraded":"2024-02-02T13:29:04.893724+0000","last_fullsized":"2024-02-02T13:29:04.893724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:28:08.230800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884500+0000","last_change":"2024-02-02T13:29:04.895867+0000","last_active":"2024-02-02T13:29:05.884500+0000","last_peered":"2024-02-02T13:29:05.884500+0000","last_clean":"2024-02-02T13:29:05.884500+0000","last_became_active":"2024-02-02T13:29:04.894304+0000","last_became_peered":"2024-02-02T13:29:04.894304+0000","last_unstale":"2024-02-02T13:29:05.884500+0000","last_undegraded":"2024-02-02T13:29:05.884500+0000","last_fullsized":"2024-02-02T13:29:05.884500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:45.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883811+0000","last_change":"2024-02-02T13:29:04.901290+0000","last_active":"2024-02-02T13:29:05.883811+0000","last_peered":"2024-02-02T13:29:05.883811+0000","last_clean":"2024-02-02T13:29:05.883811+0000","last_became_active":"2024-02-02T13:29:04.900860+0000","last_became_peered":"2024-02-02T13:29:04.900860+0000","last_unstale":"2024-02-02T13:29:05.883811+0000","last_undegraded":"2024-02-02T13:29:05.883811+0000","last_fullsized":"2024-02-02T13:29:05.883811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:12.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881377+0000","last_change":"2024-02-02T13:29:04.909839+0000","last_active":"2024-02-02T13:29:05.881377+0000","last_peered":"2024-02-02T13:29:05.881377+0000","last_clean":"2024-02-02T13:29:05.881377+0000","last_became_active":"2024-02-02T13:29:04.909607+0000","last_became_peered":"2024-02-02T13:29:04.909607+0000","last_unstale":"2024-02-02T13:29:05.881377+0000","last_undegraded":"2024-02-02T13:29:05.881377+0000","last_fullsized":"2024-02-02T13:29:05.881377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:19:31.477807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884049+0000","last_change":"2024-02-02T13:29:04.914130+0000","last_active":"2024-02-02T13:29:05.884049+0000","last_peered":"2024-02-02T13:29:05.884049+0000","last_clean":"2024-02-02T13:29:05.884049+0000","last_became_active":"2024-02-02T13:29:04.913430+0000","last_became_peered":"2024-02-02T13:29:04.913430+0000","last_unstale":"2024-02-02T13:29:05.884049+0000","last_undegraded":"2024-02-02T13:29:05.884049+0000","last_fullsized":"2024-02-02T13:29:05.884049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:06.196541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884844+0000","last_change":"2024-02-02T13:29:04.916605+0000","last_active":"2024-02-02T13:29:05.884844+0000","last_peered":"2024-02-02T13:29:05.884844+0000","last_clean":"2024-02-02T13:29:05.884844+0000","last_became_active":"2024-02-02T13:29:04.916339+0000","last_became_peered":"2024-02-02T13:29:04.916339+0000","last_unstale":"2024-02-02T13:29:05.884844+0000","last_undegraded":"2024-02-02T13:29:05.884844+0000","last_fullsized":"2024-02-02T13:29:05.884844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:53.821488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881897+0000","last_change":"2024-02-02T13:29:04.910741+0000","last_active":"2024-02-02T13:29:05.881897+0000","last_peered":"2024-02-02T13:29:05.881897+0000","last_clean":"2024-02-02T13:29:05.881897+0000","last_became_active":"2024-02-02T13:29:04.910487+0000","last_became_peered":"2024-02-02T13:29:04.910487+0000","last_unstale":"2024-02-02T13:29:05.881897+0000","last_undegraded":"2024-02-02T13:29:05.881897+0000","last_fullsized":"2024-02-02T13:29:05.881897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:11.165171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.914530+0000","last_change":"2024-02-02T13:29:04.914450+0000","last_active":"2024-02-02T13:29:04.914530+0000","last_peered":"2024-02-02T13:29:04.914530+0000","last_clean":"2024-02-02T13:29:04.914530+0000","last_became_active":"2024-02-02T13:29:04.914063+0000","last_became_peered":"2024-02-02T13:29:04.914063+0000","last_unstale":"2024-02-02T13:29:04.914530+0000","last_undegraded":"2024-02-02T13:29:04.914530+0000","last_fullsized":"2024-02-02T13:29:04.914530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:38:37.104567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883617+0000","last_change":"2024-02-02T13:29:04.900779+0000","last_active":"2024-02-02T13:29:05.883617+0000","last_peered":"2024-02-02T13:29:05.883617+0000","last_clean":"2024-02-02T13:29:05.883617+0000","last_became_active":"2024-02-02T13:29:04.900259+0000","last_became_peered":"2024-02-02T13:29:04.900259+0000","last_unstale":"2024-02-02T13:29:05.883617+0000","last_undegraded":"2024-02-02T13:29:05.883617+0000","last_fullsized":"2024-02-02T13:29:05.883617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:31.600140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.889104+0000","last_change":"2024-02-02T13:29:04.888588+0000","last_active":"2024-02-02T13:29:04.889104+0000","last_peered":"2024-02-02T13:29:04.889104+0000","last_clean":"2024-02-02T13:29:04.889104+0000","last_became_active":"2024-02-02T13:29:04.888361+0000","last_became_peered":"2024-02-02T13:29:04.888361+0000","last_unstale":"2024-02-02T13:29:04.889104+0000","last_undegraded":"2024-02-02T13:29:04.889104+0000","last_fullsized":"2024-02-02T13:29:04.889104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:46:31.174891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882575+0000","last_change":"2024-02-02T13:29:04.908772+0000","last_active":"2024-02-02T13:29:05.882575+0000","last_peered":"2024-02-02T13:29:05.882575+0000","last_clean":"2024-02-02T13:29:05.882575+0000","last_became_active":"2024-02-02T13:29:04.908508+0000","last_became_peered":"2024-02-02T13:29:04.908508+0000","last_unstale":"2024-02-02T13:29:05.882575+0000","last_undegraded":"2024-02-02T13:29:05.882575+0000","last_fullsized":"2024-02-02T13:29:05.882575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:53.541313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882121+0000","last_change":"2024-02-02T13:29:04.886830+0000","last_active":"2024-02-02T13:29:05.882121+0000","last_peered":"2024-02-02T13:29:05.882121+0000","last_clean":"2024-02-02T13:29:05.882121+0000","last_became_active":"2024-02-02T13:29:04.885982+0000","last_became_peered":"2024-02-02T13:29:04.885982+0000","last_unstale":"2024-02-02T13:29:05.882121+0000","last_undegraded":"2024-02-02T13:29:05.882121+0000","last_fullsized":"2024-02-02T13:29:05.882121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:07:25.371263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899724+0000","last_change":"2024-02-02T13:29:04.899584+0000","last_active":"2024-02-02T13:29:04.899724+0000","last_peered":"2024-02-02T13:29:04.899724+0000","last_clean":"2024-02-02T13:29:04.899724+0000","last_became_active":"2024-02-02T13:29:04.899071+0000","last_became_peered":"2024-02-02T13:29:04.899071+0000","last_unstale":"2024-02-02T13:29:04.899724+0000","last_undegraded":"2024-02-02T13:29:04.899724+0000","last_fullsized":"2024-02-02T13:29:04.899724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:27.748598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884587+0000","last_change":"2024-02-02T13:29:04.914623+0000","last_active":"2024-02-02T13:29:05.884587+0000","last_peered":"2024-02-02T13:29:05.884587+0000","last_clean":"2024-02-02T13:29:05.884587+0000","last_became_active":"2024-02-02T13:29:04.913897+0000","last_became_peered":"2024-02-02T13:29:04.913897+0000","last_unstale":"2024-02-02T13:29:05.884587+0000","last_undegraded":"2024-02-02T13:29:05.884587+0000","last_fullsized":"2024-02-02T13:29:05.884587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:20.651424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881661+0000","last_change":"2024-02-02T13:29:04.401531+0000","last_active":"2024-02-02T13:29:05.881661+0000","last_peered":"2024-02-02T13:29:05.881661+0000","last_clean":"2024-02-02T13:29:05.881661+0000","last_became_active":"2024-02-02T13:29:04.400570+0000","last_became_peered":"2024-02-02T13:29:04.400570+0000","last_unstale":"2024-02-02T13:29:05.881661+0000","last_undegraded":"2024-02-02T13:29:05.881661+0000","last_fullsized":"2024-02-02T13:29:05.881661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:22:43.166181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.887521+0000","last_change":"2024-02-02T13:29:04.887446+0000","last_active":"2024-02-02T13:29:04.887521+0000","last_peered":"2024-02-02T13:29:04.887521+0000","last_clean":"2024-02-02T13:29:04.887521+0000","last_became_active":"2024-02-02T13:29:04.887160+0000","last_became_peered":"2024-02-02T13:29:04.887160+0000","last_unstale":"2024-02-02T13:29:04.887521+0000","last_undegraded":"2024-02-02T13:29:04.887521+0000","last_fullsized":"2024-02-02T13:29:04.887521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:36.516051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890307+0000","last_change":"2024-02-02T13:29:04.396861+0000","last_active":"2024-02-02T13:29:04.890307+0000","last_peered":"2024-02-02T13:29:04.890307+0000","last_clean":"2024-02-02T13:29:04.890307+0000","last_became_active":"2024-02-02T13:29:04.395937+0000","last_became_peered":"2024-02-02T13:29:04.395937+0000","last_unstale":"2024-02-02T13:29:04.890307+0000","last_undegraded":"2024-02-02T13:29:04.890307+0000","last_fullsized":"2024-02-02T13:29:04.890307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:44.872326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.913958+0000","last_change":"2024-02-02T13:29:04.913873+0000","last_active":"2024-02-02T13:29:04.913958+0000","last_peered":"2024-02-02T13:29:04.913958+0000","last_clean":"2024-02-02T13:29:04.913958+0000","last_became_active":"2024-02-02T13:29:04.912396+0000","last_became_peered":"2024-02-02T13:29:04.912396+0000","last_unstale":"2024-02-02T13:29:04.913958+0000","last_undegraded":"2024-02-02T13:29:04.913958+0000","last_fullsized":"2024-02-02T13:29:04.913958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:54:03.744654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885227+0000","last_change":"2024-02-02T13:29:04.409703+0000","last_active":"2024-02-02T13:29:05.885227+0000","last_peered":"2024-02-02T13:29:05.885227+0000","last_clean":"2024-02-02T13:29:05.885227+0000","last_became_active":"2024-02-02T13:29:04.408780+0000","last_became_peered":"2024-02-02T13:29:04.408780+0000","last_unstale":"2024-02-02T13:29:05.885227+0000","last_undegraded":"2024-02-02T13:29:05.885227+0000","last_fullsized":"2024-02-02T13:29:05.885227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:20.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882380+0000","last_change":"2024-02-02T13:29:04.913827+0000","last_active":"2024-02-02T13:29:05.882380+0000","last_peered":"2024-02-02T13:29:05.882380+0000","last_clean":"2024-02-02T13:29:05.882380+0000","last_became_active":"2024-02-02T13:29:04.913166+0000","last_became_peered":"2024-02-02T13:29:04.913166+0000","last_unstale":"2024-02-02T13:29:05.882380+0000","last_undegraded":"2024-02-02T13:29:05.882380+0000","last_fullsized":"2024-02-02T13:29:05.882380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:38.813534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.912678+0000","last_change":"2024-02-02T13:29:04.408968+0000","last_active":"2024-02-02T13:29:04.912678+0000","last_peered":"2024-02-02T13:29:04.912678+0000","last_clean":"2024-02-02T13:29:04.912678+0000","last_became_active":"2024-02-02T13:29:04.408182+0000","last_became_peered":"2024-02-02T13:29:04.408182+0000","last_unstale":"2024-02-02T13:29:04.912678+0000","last_undegraded":"2024-02-02T13:29:04.912678+0000","last_fullsized":"2024-02-02T13:29:04.912678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:33.577918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.880784+0000","last_change":"2024-02-02T13:29:04.889303+0000","last_active":"2024-02-02T13:29:05.880784+0000","last_peered":"2024-02-02T13:29:05.880784+0000","last_clean":"2024-02-02T13:29:05.880784+0000","last_became_active":"2024-02-02T13:29:04.888641+0000","last_became_peered":"2024-02-02T13:29:04.888641+0000","last_unstale":"2024-02-02T13:29:05.880784+0000","last_undegraded":"2024-02-02T13:29:05.880784+0000","last_fullsized":"2024-02-02T13:29:05.880784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:35:37.094490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.896734+0000","last_change":"2024-02-02T13:29:04.405333+0000","last_active":"2024-02-02T13:29:04.896734+0000","last_peered":"2024-02-02T13:29:04.896734+0000","last_clean":"2024-02-02T13:29:04.896734+0000","last_became_active":"2024-02-02T13:29:04.404992+0000","last_became_peered":"2024-02-02T13:29:04.404992+0000","last_unstale":"2024-02-02T13:29:04.896734+0000","last_undegraded":"2024-02-02T13:29:04.896734+0000","last_fullsized":"2024-02-02T13:29:04.896734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:52.420500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884057+0000","last_change":"2024-02-02T13:29:04.912565+0000","last_active":"2024-02-02T13:29:05.884057+0000","last_peered":"2024-02-02T13:29:05.884057+0000","last_clean":"2024-02-02T13:29:05.884057+0000","last_became_active":"2024-02-02T13:29:04.910842+0000","last_became_peered":"2024-02-02T13:29:04.910842+0000","last_unstale":"2024-02-02T13:29:05.884057+0000","last_undegraded":"2024-02-02T13:29:05.884057+0000","last_fullsized":"2024-02-02T13:29:05.884057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:54:03.744654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.916037+0000","last_change":"2024-02-02T13:29:04.406109+0000","last_active":"2024-02-02T13:29:04.916037+0000","last_peered":"2024-02-02T13:29:04.916037+0000","last_clean":"2024-02-02T13:29:04.916037+0000","last_became_active":"2024-02-02T13:29:04.405828+0000","last_became_peered":"2024-02-02T13:29:04.405828+0000","last_unstale":"2024-02-02T13:29:04.916037+0000","last_undegraded":"2024-02-02T13:29:04.916037+0000","last_fullsized":"2024-02-02T13:29:04.916037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:27:41.967706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.900474+0000","last_change":"2024-02-02T13:29:04.900434+0000","last_active":"2024-02-02T13:29:04.900474+0000","last_peered":"2024-02-02T13:29:04.900474+0000","last_clean":"2024-02-02T13:29:04.900474+0000","last_became_active":"2024-02-02T13:29:04.900270+0000","last_became_peered":"2024-02-02T13:29:04.900270+0000","last_unstale":"2024-02-02T13:29:04.900474+0000","last_undegraded":"2024-02-02T13:29:04.900474+0000","last_fullsized":"2024-02-02T13:29:04.900474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:21.935070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.877319+0000","last_change":"2024-02-02T13:29:04.394846+0000","last_active":"2024-02-02T13:29:04.877319+0000","last_peered":"2024-02-02T13:29:04.877319+0000","last_clean":"2024-02-02T13:29:04.877319+0000","last_became_active":"2024-02-02T13:29:04.394597+0000","last_became_peered":"2024-02-02T13:29:04.394597+0000","last_unstale":"2024-02-02T13:29:04.877319+0000","last_undegraded":"2024-02-02T13:29:04.877319+0000","last_fullsized":"2024-02-02T13:29:04.877319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:27:41.967706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.910400+0000","last_change":"2024-02-02T13:29:04.910328+0000","last_active":"2024-02-02T13:29:04.910400+0000","last_peered":"2024-02-02T13:29:04.910400+0000","last_clean":"2024-02-02T13:29:04.910400+0000","last_became_active":"2024-02-02T13:29:04.910076+0000","last_became_peered":"2024-02-02T13:29:04.910076+0000","last_unstale":"2024-02-02T13:29:04.910400+0000","last_undegraded":"2024-02-02T13:29:04.910400+0000","last_fullsized":"2024-02-02T13:29:04.910400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:07:25.371263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.878680+0000","last_change":"2024-02-02T13:29:04.401668+0000","last_active":"2024-02-02T13:29:04.878680+0000","last_peered":"2024-02-02T13:29:04.878680+0000","last_clean":"2024-02-02T13:29:04.878680+0000","last_became_active":"2024-02-02T13:29:04.401416+0000","last_became_peered":"2024-02-02T13:29:04.401416+0000","last_unstale":"2024-02-02T13:29:04.878680+0000","last_undegraded":"2024-02-02T13:29:04.878680+0000","last_fullsized":"2024-02-02T13:29:04.878680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:50:09.200253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881019+0000","last_change":"2024-02-02T13:29:04.912107+0000","last_active":"2024-02-02T13:29:05.881019+0000","last_peered":"2024-02-02T13:29:05.881019+0000","last_clean":"2024-02-02T13:29:05.881019+0000","last_became_active":"2024-02-02T13:29:04.911832+0000","last_became_peered":"2024-02-02T13:29:04.911832+0000","last_unstale":"2024-02-02T13:29:05.881019+0000","last_undegraded":"2024-02-02T13:29:05.881019+0000","last_fullsized":"2024-02-02T13:29:05.881019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:09:45.908362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884696+0000","last_change":"2024-02-02T13:29:04.407888+0000","last_active":"2024-02-02T13:29:05.884696+0000","last_peered":"2024-02-02T13:29:05.884696+0000","last_clean":"2024-02-02T13:29:05.884696+0000","last_became_active":"2024-02-02T13:29:04.406034+0000","last_became_peered":"2024-02-02T13:29:04.406034+0000","last_unstale":"2024-02-02T13:29:05.884696+0000","last_undegraded":"2024-02-02T13:29:05.884696+0000","last_fullsized":"2024-02-02T13:29:05.884696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:50:57.751135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882192+0000","last_change":"2024-02-02T13:29:04.916237+0000","last_active":"2024-02-02T13:29:05.882192+0000","last_peered":"2024-02-02T13:29:05.882192+0000","last_clean":"2024-02-02T13:29:05.882192+0000","last_became_active":"2024-02-02T13:29:04.915877+0000","last_became_peered":"2024-02-02T13:29:04.915877+0000","last_unstale":"2024-02-02T13:29:05.882192+0000","last_undegraded":"2024-02-02T13:29:05.882192+0000","last_fullsized":"2024-02-02T13:29:05.882192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:02:49.668441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898397+0000","last_change":"2024-02-02T13:29:04.406515+0000","last_active":"2024-02-02T13:29:04.898397+0000","last_peered":"2024-02-02T13:29:04.898397+0000","last_clean":"2024-02-02T13:29:04.898397+0000","last_became_active":"2024-02-02T13:29:04.406206+0000","last_became_peered":"2024-02-02T13:29:04.406206+0000","last_unstale":"2024-02-02T13:29:04.898397+0000","last_undegraded":"2024-02-02T13:29:04.898397+0000","last_fullsized":"2024-02-02T13:29:04.898397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:31.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.909011+0000","last_change":"2024-02-02T13:29:04.904697+0000","last_active":"2024-02-02T13:29:04.909011+0000","last_peered":"2024-02-02T13:29:04.909011+0000","last_clean":"2024-02-02T13:29:04.909011+0000","last_became_active":"2024-02-02T13:29:04.904455+0000","last_became_peered":"2024-02-02T13:29:04.904455+0000","last_unstale":"2024-02-02T13:29:04.909011+0000","last_undegraded":"2024-02-02T13:29:04.909011+0000","last_fullsized":"2024-02-02T13:29:04.909011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:12.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.895442+0000","last_change":"2024-02-02T13:29:04.387793+0000","last_active":"2024-02-02T13:29:04.895442+0000","last_peered":"2024-02-02T13:29:04.895442+0000","last_clean":"2024-02-02T13:29:04.895442+0000","last_became_active":"2024-02-02T13:29:04.387030+0000","last_became_peered":"2024-02-02T13:29:04.387030+0000","last_unstale":"2024-02-02T13:29:04.895442+0000","last_undegraded":"2024-02-02T13:29:04.895442+0000","last_fullsized":"2024-02-02T13:29:04.895442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:37.807873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886474+0000","last_change":"2024-02-02T13:29:04.885141+0000","last_active":"2024-02-02T13:29:04.886474+0000","last_peered":"2024-02-02T13:29:04.886474+0000","last_clean":"2024-02-02T13:29:04.886474+0000","last_became_active":"2024-02-02T13:29:04.884911+0000","last_became_peered":"2024-02-02T13:29:04.884911+0000","last_unstale":"2024-02-02T13:29:04.886474+0000","last_undegraded":"2024-02-02T13:29:04.886474+0000","last_fullsized":"2024-02-02T13:29:04.886474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:22.500925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898111+0000","last_change":"2024-02-02T13:29:04.400508+0000","last_active":"2024-02-02T13:29:04.898111+0000","last_peered":"2024-02-02T13:29:04.898111+0000","last_clean":"2024-02-02T13:29:04.898111+0000","last_became_active":"2024-02-02T13:29:04.399434+0000","last_became_peered":"2024-02-02T13:29:04.399434+0000","last_unstale":"2024-02-02T13:29:04.898111+0000","last_undegraded":"2024-02-02T13:29:04.898111+0000","last_fullsized":"2024-02-02T13:29:04.898111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:20.929409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882420+0000","last_change":"2024-02-02T13:29:04.911157+0000","last_active":"2024-02-02T13:29:05.882420+0000","last_peered":"2024-02-02T13:29:05.882420+0000","last_clean":"2024-02-02T13:29:05.882420+0000","last_became_active":"2024-02-02T13:29:04.909652+0000","last_became_peered":"2024-02-02T13:29:04.909652+0000","last_unstale":"2024-02-02T13:29:05.882420+0000","last_undegraded":"2024-02-02T13:29:05.882420+0000","last_fullsized":"2024-02-02T13:29:05.882420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:34:57.356543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.914698+0000","last_change":"2024-02-02T13:29:04.398976+0000","last_active":"2024-02-02T13:29:04.914698+0000","last_peered":"2024-02-02T13:29:04.914698+0000","last_clean":"2024-02-02T13:29:04.914698+0000","last_became_active":"2024-02-02T13:29:04.395566+0000","last_became_peered":"2024-02-02T13:29:04.395566+0000","last_unstale":"2024-02-02T13:29:04.914698+0000","last_undegraded":"2024-02-02T13:29:04.914698+0000","last_fullsized":"2024-02-02T13:29:04.914698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:17.180073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882818+0000","last_change":"2024-02-02T13:29:04.903240+0000","last_active":"2024-02-02T13:29:05.882818+0000","last_peered":"2024-02-02T13:29:05.882818+0000","last_clean":"2024-02-02T13:29:05.882818+0000","last_became_active":"2024-02-02T13:29:04.902752+0000","last_became_peered":"2024-02-02T13:29:04.902752+0000","last_unstale":"2024-02-02T13:29:05.882818+0000","last_undegraded":"2024-02-02T13:29:05.882818+0000","last_fullsized":"2024-02-02T13:29:05.882818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:50:02.616058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.904268+0000","last_change":"2024-02-02T13:29:04.409751+0000","last_active":"2024-02-02T13:29:04.904268+0000","last_peered":"2024-02-02T13:29:04.904268+0000","last_clean":"2024-02-02T13:29:04.904268+0000","last_became_active":"2024-02-02T13:29:04.409200+0000","last_became_peered":"2024-02-02T13:29:04.409200+0000","last_unstale":"2024-02-02T13:29:04.904268+0000","last_undegraded":"2024-02-02T13:29:04.904268+0000","last_fullsized":"2024-02-02T13:29:04.904268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:20.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.901174+0000","last_change":"2024-02-02T13:29:04.901084+0000","last_active":"2024-02-02T13:29:04.901174+0000","last_peered":"2024-02-02T13:29:04.901174+0000","last_clean":"2024-02-02T13:29:04.901174+0000","last_became_active":"2024-02-02T13:29:04.900725+0000","last_became_peered":"2024-02-02T13:29:04.900725+0000","last_unstale":"2024-02-02T13:29:04.901174+0000","last_undegraded":"2024-02-02T13:29:04.901174+0000","last_fullsized":"2024-02-02T13:29:04.901174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:53.541313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881488+0000","last_change":"2024-02-02T13:29:04.399510+0000","last_active":"2024-02-02T13:29:05.881488+0000","last_peered":"2024-02-02T13:29:05.881488+0000","last_clean":"2024-02-02T13:29:05.881488+0000","last_became_active":"2024-02-02T13:29:04.398902+0000","last_became_peered":"2024-02-02T13:29:04.398902+0000","last_unstale":"2024-02-02T13:29:05.881488+0000","last_undegraded":"2024-02-02T13:29:05.881488+0000","last_fullsized":"2024-02-02T13:29:05.881488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:11.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.886681+0000","last_change":"2024-02-02T13:29:04.886599+0000","last_active":"2024-02-02T13:29:04.886681+0000","last_peered":"2024-02-02T13:29:04.886681+0000","last_clean":"2024-02-02T13:29:04.886681+0000","last_became_active":"2024-02-02T13:29:04.886260+0000","last_became_peered":"2024-02-02T13:29:04.886260+0000","last_unstale":"2024-02-02T13:29:04.886681+0000","last_undegraded":"2024-02-02T13:29:04.886681+0000","last_fullsized":"2024-02-02T13:29:04.886681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:57:44.798704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883848+0000","last_change":"2024-02-02T13:29:04.406001+0000","last_active":"2024-02-02T13:29:05.883848+0000","last_peered":"2024-02-02T13:29:05.883848+0000","last_clean":"2024-02-02T13:29:05.883848+0000","last_became_active":"2024-02-02T13:29:04.405782+0000","last_became_peered":"2024-02-02T13:29:04.405782+0000","last_unstale":"2024-02-02T13:29:05.883848+0000","last_undegraded":"2024-02-02T13:29:05.883848+0000","last_fullsized":"2024-02-02T13:29:05.883848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:14:49.868948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885007+0000","last_change":"2024-02-02T13:29:04.902592+0000","last_active":"2024-02-02T13:29:05.885007+0000","last_peered":"2024-02-02T13:29:05.885007+0000","last_clean":"2024-02-02T13:29:05.885007+0000","last_became_active":"2024-02-02T13:29:04.901491+0000","last_became_peered":"2024-02-02T13:29:04.901491+0000","last_unstale":"2024-02-02T13:29:05.885007+0000","last_undegraded":"2024-02-02T13:29:05.885007+0000","last_fullsized":"2024-02-02T13:29:05.885007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:21:10.301191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.899943+0000","last_change":"2024-02-02T13:29:04.407899+0000","last_active":"2024-02-02T13:29:04.899943+0000","last_peered":"2024-02-02T13:29:04.899943+0000","last_clean":"2024-02-02T13:29:04.899943+0000","last_became_active":"2024-02-02T13:29:04.407242+0000","last_became_peered":"2024-02-02T13:29:04.407242+0000","last_unstale":"2024-02-02T13:29:04.899943+0000","last_undegraded":"2024-02-02T13:29:04.899943+0000","last_fullsized":"2024-02-02T13:29:04.899943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:50:57.751135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884829+0000","last_change":"2024-02-02T13:29:04.908733+0000","last_active":"2024-02-02T13:29:05.884829+0000","last_peered":"2024-02-02T13:29:05.884829+0000","last_clean":"2024-02-02T13:29:05.884829+0000","last_became_active":"2024-02-02T13:29:04.908364+0000","last_became_peered":"2024-02-02T13:29:04.908364+0000","last_unstale":"2024-02-02T13:29:05.884829+0000","last_undegraded":"2024-02-02T13:29:05.884829+0000","last_fullsized":"2024-02-02T13:29:05.884829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:02.835735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885336+0000","last_change":"2024-02-02T13:29:04.409834+0000","last_active":"2024-02-02T13:29:05.885336+0000","last_peered":"2024-02-02T13:29:05.885336+0000","last_clean":"2024-02-02T13:29:05.885336+0000","last_became_active":"2024-02-02T13:29:04.409063+0000","last_became_peered":"2024-02-02T13:29:04.409063+0000","last_unstale":"2024-02-02T13:29:05.885336+0000","last_undegraded":"2024-02-02T13:29:05.885336+0000","last_fullsized":"2024-02-02T13:29:05.885336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:33.577918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.882993+0000","last_change":"2024-02-02T13:29:04.881208+0000","last_active":"2024-02-02T13:29:04.882993+0000","last_peered":"2024-02-02T13:29:04.882993+0000","last_clean":"2024-02-02T13:29:04.882993+0000","last_became_active":"2024-02-02T13:29:04.880973+0000","last_became_peered":"2024-02-02T13:29:04.880973+0000","last_unstale":"2024-02-02T13:29:04.882993+0000","last_undegraded":"2024-02-02T13:29:04.882993+0000","last_fullsized":"2024-02-02T13:29:04.882993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:31:25.386243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915563+0000","last_change":"2024-02-02T13:29:04.409887+0000","last_active":"2024-02-02T13:29:04.915563+0000","last_peered":"2024-02-02T13:29:04.915563+0000","last_clean":"2024-02-02T13:29:04.915563+0000","last_became_active":"2024-02-02T13:29:04.409541+0000","last_became_peered":"2024-02-02T13:29:04.409541+0000","last_unstale":"2024-02-02T13:29:04.915563+0000","last_undegraded":"2024-02-02T13:29:04.915563+0000","last_fullsized":"2024-02-02T13:29:04.915563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:50.281988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.897261+0000","last_change":"2024-02-02T13:29:04.897199+0000","last_active":"2024-02-02T13:29:04.897261+0000","last_peered":"2024-02-02T13:29:04.897261+0000","last_clean":"2024-02-02T13:29:04.897261+0000","last_became_active":"2024-02-02T13:29:04.896967+0000","last_became_peered":"2024-02-02T13:29:04.896967+0000","last_unstale":"2024-02-02T13:29:04.897261+0000","last_undegraded":"2024-02-02T13:29:04.897261+0000","last_fullsized":"2024-02-02T13:29:04.897261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:51.287648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881632+0000","last_change":"2024-02-02T13:29:04.398676+0000","last_active":"2024-02-02T13:29:05.881632+0000","last_peered":"2024-02-02T13:29:05.881632+0000","last_clean":"2024-02-02T13:29:05.881632+0000","last_became_active":"2024-02-02T13:29:04.397930+0000","last_became_peered":"2024-02-02T13:29:04.397930+0000","last_unstale":"2024-02-02T13:29:05.881632+0000","last_undegraded":"2024-02-02T13:29:05.881632+0000","last_fullsized":"2024-02-02T13:29:05.881632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:57:45.700326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.888150+0000","last_change":"2024-02-02T13:29:04.888075+0000","last_active":"2024-02-02T13:29:04.888150+0000","last_peered":"2024-02-02T13:29:04.888150+0000","last_clean":"2024-02-02T13:29:04.888150+0000","last_became_active":"2024-02-02T13:29:04.887683+0000","last_became_peered":"2024-02-02T13:29:04.887683+0000","last_unstale":"2024-02-02T13:29:04.888150+0000","last_undegraded":"2024-02-02T13:29:04.888150+0000","last_fullsized":"2024-02-02T13:29:04.888150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:51:18.141044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.893201+0000","last_change":"2024-02-02T13:29:04.401796+0000","last_active":"2024-02-02T13:29:04.893201+0000","last_peered":"2024-02-02T13:29:04.893201+0000","last_clean":"2024-02-02T13:29:04.893201+0000","last_became_active":"2024-02-02T13:29:04.401292+0000","last_became_peered":"2024-02-02T13:29:04.401292+0000","last_unstale":"2024-02-02T13:29:04.893201+0000","last_undegraded":"2024-02-02T13:29:04.893201+0000","last_fullsized":"2024-02-02T13:29:04.893201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:17.180073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908829+0000","last_change":"2024-02-02T13:29:04.908777+0000","last_active":"2024-02-02T13:29:04.908829+0000","last_peered":"2024-02-02T13:29:04.908829+0000","last_clean":"2024-02-02T13:29:04.908829+0000","last_became_active":"2024-02-02T13:29:04.908613+0000","last_became_peered":"2024-02-02T13:29:04.908613+0000","last_unstale":"2024-02-02T13:29:04.908829+0000","last_undegraded":"2024-02-02T13:29:04.908829+0000","last_fullsized":"2024-02-02T13:29:04.908829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:09.586099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.883015+0000","last_change":"2024-02-02T13:29:04.390122+0000","last_active":"2024-02-02T13:29:05.883015+0000","last_peered":"2024-02-02T13:29:05.883015+0000","last_clean":"2024-02-02T13:29:05.883015+0000","last_became_active":"2024-02-02T13:29:04.389480+0000","last_became_peered":"2024-02-02T13:29:04.389480+0000","last_unstale":"2024-02-02T13:29:05.883015+0000","last_undegraded":"2024-02-02T13:29:05.883015+0000","last_fullsized":"2024-02-02T13:29:05.883015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:30.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908031+0000","last_change":"2024-02-02T13:29:04.907972+0000","last_active":"2024-02-02T13:29:04.908031+0000","last_peered":"2024-02-02T13:29:04.908031+0000","last_clean":"2024-02-02T13:29:04.908031+0000","last_became_active":"2024-02-02T13:29:04.907742+0000","last_became_peered":"2024-02-02T13:29:04.907742+0000","last_unstale":"2024-02-02T13:29:04.908031+0000","last_undegraded":"2024-02-02T13:29:04.908031+0000","last_fullsized":"2024-02-02T13:29:04.908031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:00.475514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.883486+0000","last_change":"2024-02-02T13:29:04.399046+0000","last_active":"2024-02-02T13:29:04.883486+0000","last_peered":"2024-02-02T13:29:04.883486+0000","last_clean":"2024-02-02T13:29:04.883486+0000","last_became_active":"2024-02-02T13:29:04.398670+0000","last_became_peered":"2024-02-02T13:29:04.398670+0000","last_unstale":"2024-02-02T13:29:04.883486+0000","last_undegraded":"2024-02-02T13:29:04.883486+0000","last_fullsized":"2024-02-02T13:29:04.883486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:11.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.890209+0000","last_change":"2024-02-02T13:29:04.890161+0000","last_active":"2024-02-02T13:29:04.890209+0000","last_peered":"2024-02-02T13:29:04.890209+0000","last_clean":"2024-02-02T13:29:04.890209+0000","last_became_active":"2024-02-02T13:29:04.889935+0000","last_became_peered":"2024-02-02T13:29:04.889935+0000","last_unstale":"2024-02-02T13:29:04.890209+0000","last_undegraded":"2024-02-02T13:29:04.890209+0000","last_fullsized":"2024-02-02T13:29:04.890209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:49:54.416602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915199+0000","last_change":"2024-02-02T13:29:04.409110+0000","last_active":"2024-02-02T13:29:04.915199+0000","last_peered":"2024-02-02T13:29:04.915199+0000","last_clean":"2024-02-02T13:29:04.915199+0000","last_became_active":"2024-02-02T13:29:04.408068+0000","last_became_peered":"2024-02-02T13:29:04.408068+0000","last_unstale":"2024-02-02T13:29:04.915199+0000","last_undegraded":"2024-02-02T13:29:04.915199+0000","last_fullsized":"2024-02-02T13:29:04.915199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:56.288505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881257+0000","last_change":"2024-02-02T13:29:04.886599+0000","last_active":"2024-02-02T13:29:05.881257+0000","last_peered":"2024-02-02T13:29:05.881257+0000","last_clean":"2024-02-02T13:29:05.881257+0000","last_became_active":"2024-02-02T13:29:04.885855+0000","last_became_peered":"2024-02-02T13:29:04.885855+0000","last_unstale":"2024-02-02T13:29:05.881257+0000","last_undegraded":"2024-02-02T13:29:05.881257+0000","last_fullsized":"2024-02-02T13:29:05.881257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:15.696320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.881002+0000","last_change":"2024-02-02T13:29:04.401332+0000","last_active":"2024-02-02T13:29:05.881002+0000","last_peered":"2024-02-02T13:29:05.881002+0000","last_clean":"2024-02-02T13:29:05.881002+0000","last_became_active":"2024-02-02T13:29:04.401023+0000","last_became_peered":"2024-02-02T13:29:04.401023+0000","last_unstale":"2024-02-02T13:29:05.881002+0000","last_undegraded":"2024-02-02T13:29:05.881002+0000","last_fullsized":"2024-02-02T13:29:05.881002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:15:26.742938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.882827+0000","last_change":"2024-02-02T13:29:04.914246+0000","last_active":"2024-02-02T13:29:05.882827+0000","last_peered":"2024-02-02T13:29:05.882827+0000","last_clean":"2024-02-02T13:29:05.882827+0000","last_became_active":"2024-02-02T13:29:04.913569+0000","last_became_peered":"2024-02-02T13:29:04.913569+0000","last_unstale":"2024-02-02T13:29:05.882827+0000","last_undegraded":"2024-02-02T13:29:05.882827+0000","last_fullsized":"2024-02-02T13:29:05.882827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:16:26.291379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.879142+0000","last_change":"2024-02-02T13:29:04.401423+0000","last_active":"2024-02-02T13:29:04.879142+0000","last_peered":"2024-02-02T13:29:04.879142+0000","last_clean":"2024-02-02T13:29:04.879142+0000","last_became_active":"2024-02-02T13:29:04.399922+0000","last_became_peered":"2024-02-02T13:29:04.399922+0000","last_unstale":"2024-02-02T13:29:04.879142+0000","last_undegraded":"2024-02-02T13:29:04.879142+0000","last_fullsized":"2024-02-02T13:29:04.879142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:18:31.933369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.885046+0000","last_change":"2024-02-02T13:29:04.914774+0000","last_active":"2024-02-02T13:29:05.885046+0000","last_peered":"2024-02-02T13:29:05.885046+0000","last_clean":"2024-02-02T13:29:05.885046+0000","last_became_active":"2024-02-02T13:29:04.914371+0000","last_became_peered":"2024-02-02T13:29:04.914371+0000","last_unstale":"2024-02-02T13:29:05.885046+0000","last_undegraded":"2024-02-02T13:29:05.885046+0000","last_fullsized":"2024-02-02T13:29:05.885046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:01:22.123982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.884459+0000","last_change":"2024-02-02T13:29:04.406343+0000","last_active":"2024-02-02T13:29:04.884459+0000","last_peered":"2024-02-02T13:29:04.884459+0000","last_clean":"2024-02-02T13:29:04.884459+0000","last_became_active":"2024-02-02T13:29:04.405829+0000","last_became_peered":"2024-02-02T13:29:04.405829+0000","last_unstale":"2024-02-02T13:29:04.884459+0000","last_undegraded":"2024-02-02T13:29:04.884459+0000","last_fullsized":"2024-02-02T13:29:04.884459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:30.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.887290+0000","last_change":"2024-02-02T13:29:04.887120+0000","last_active":"2024-02-02T13:29:04.887290+0000","last_peered":"2024-02-02T13:29:04.887290+0000","last_clean":"2024-02-02T13:29:04.887290+0000","last_became_active":"2024-02-02T13:29:04.886857+0000","last_became_peered":"2024-02-02T13:29:04.886857+0000","last_unstale":"2024-02-02T13:29:04.887290+0000","last_undegraded":"2024-02-02T13:29:04.887290+0000","last_fullsized":"2024-02-02T13:29:04.887290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:28.794701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.879060+0000","last_change":"2024-02-02T13:29:04.399859+0000","last_active":"2024-02-02T13:29:04.879060+0000","last_peered":"2024-02-02T13:29:04.879060+0000","last_clean":"2024-02-02T13:29:04.879060+0000","last_became_active":"2024-02-02T13:29:04.397846+0000","last_became_peered":"2024-02-02T13:29:04.397846+0000","last_unstale":"2024-02-02T13:29:04.879060+0000","last_undegraded":"2024-02-02T13:29:04.879060+0000","last_fullsized":"2024-02-02T13:29:04.879060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:45:37.877511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.913942+0000","last_change":"2024-02-02T13:29:04.913527+0000","last_active":"2024-02-02T13:29:04.913942+0000","last_peered":"2024-02-02T13:29:04.913942+0000","last_clean":"2024-02-02T13:29:04.913942+0000","last_became_active":"2024-02-02T13:29:04.912030+0000","last_became_peered":"2024-02-02T13:29:04.912030+0000","last_unstale":"2024-02-02T13:29:04.913942+0000","last_undegraded":"2024-02-02T13:29:04.913942+0000","last_fullsized":"2024-02-02T13:29:04.913942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:53.821488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.876883+0000","last_change":"2024-02-02T13:29:04.406966+0000","last_active":"2024-02-02T13:29:04.876883+0000","last_peered":"2024-02-02T13:29:04.876883+0000","last_clean":"2024-02-02T13:29:04.876883+0000","last_became_active":"2024-02-02T13:29:04.406718+0000","last_became_peered":"2024-02-02T13:29:04.406718+0000","last_unstale":"2024-02-02T13:29:04.876883+0000","last_undegraded":"2024-02-02T13:29:04.876883+0000","last_fullsized":"2024-02-02T13:29:04.876883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:23:50.281988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.885903+0000","last_change":"2024-02-02T13:29:04.885614+0000","last_active":"2024-02-02T13:29:04.885903+0000","last_peered":"2024-02-02T13:29:04.885903+0000","last_clean":"2024-02-02T13:29:04.885903+0000","last_became_active":"2024-02-02T13:29:04.885402+0000","last_became_peered":"2024-02-02T13:29:04.885402+0000","last_unstale":"2024-02-02T13:29:04.885903+0000","last_undegraded":"2024-02-02T13:29:04.885903+0000","last_fullsized":"2024-02-02T13:29:04.885903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:12:04.591193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T13:29:05.884058+0000","last_change":"2024-02-02T13:29:04.407855+0000","last_active":"2024-02-02T13:29:05.884058+0000","last_peered":"2024-02-02T13:29:05.884058+0000","last_clean":"2024-02-02T13:29:05.884058+0000","last_became_active":"2024-02-02T13:29:04.407303+0000","last_became_peered":"2024-02-02T13:29:04.407303+0000","last_unstale":"2024-02-02T13:29:05.884058+0000","last_undegraded":"2024-02-02T13:29:05.884058+0000","last_fullsized":"2024-02-02T13:29:05.884058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:58:31.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.908081+0000","last_change":"2024-02-02T13:29:04.906628+0000","last_active":"2024-02-02T13:29:04.908081+0000","last_peered":"2024-02-02T13:29:04.908081+0000","last_clean":"2024-02-02T13:29:04.908081+0000","last_became_active":"2024-02-02T13:29:04.899186+0000","last_became_peered":"2024-02-02T13:29:04.899186+0000","last_unstale":"2024-02-02T13:29:04.908081+0000","last_undegraded":"2024-02-02T13:29:04.908081+0000","last_fullsized":"2024-02-02T13:29:04.908081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:40:31.600140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.892923+0000","last_change":"2024-02-02T13:29:04.407727+0000","last_active":"2024-02-02T13:29:04.892923+0000","last_peered":"2024-02-02T13:29:04.892923+0000","last_clean":"2024-02-02T13:29:04.892923+0000","last_became_active":"2024-02-02T13:29:04.406856+0000","last_became_peered":"2024-02-02T13:29:04.406856+0000","last_unstale":"2024-02-02T13:29:04.892923+0000","last_undegraded":"2024-02-02T13:29:04.892923+0000","last_fullsized":"2024-02-02T13:29:04.892923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:20.929409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.898857+0000","last_change":"2024-02-02T13:29:04.898786+0000","last_active":"2024-02-02T13:29:04.898857+0000","last_peered":"2024-02-02T13:29:04.898857+0000","last_clean":"2024-02-02T13:29:04.898857+0000","last_became_active":"2024-02-02T13:29:04.898423+0000","last_became_peered":"2024-02-02T13:29:04.898423+0000","last_unstale":"2024-02-02T13:29:04.898857+0000","last_undegraded":"2024-02-02T13:29:04.898857+0000","last_fullsized":"2024-02-02T13:29:04.898857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:04.371516+0000","last_clean_scrub_stamp":"2024-02-02T13:29:04.371516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:04:12.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.915829+0000","last_change":"2024-02-02T13:29:04.406577+0000","last_active":"2024-02-02T13:29:04.915829+0000","last_peered":"2024-02-02T13:29:04.915829+0000","last_clean":"2024-02-02T13:29:04.915829+0000","last_became_active":"2024-02-02T13:29:04.406342+0000","last_became_peered":"2024-02-02T13:29:04.406342+0000","last_unstale":"2024-02-02T13:29:04.915829+0000","last_undegraded":"2024-02-02T13:29:04.915829+0000","last_fullsized":"2024-02-02T13:29:04.915829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:29:03.365856+0000","last_clean_scrub_stamp":"2024-02-02T13:29:03.365856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:43:55.115001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T13:29:04.905952+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:29:04.905952+0000","last_peered":"2024-02-02T13:29:04.905952+0000","last_clean":"2024-02-02T13:29:04.905952+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:29:04.905952+0000","last_undegraded":"2024-02-02T13:29:04.905952+0000","last_fullsized":"2024-02-02T13:29:04.905952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1023,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194289,"statfs":{"total":200000000,"available":198952360,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.0840000000000001,"15min":1.0609999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6279999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.095},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1299999999999999,"15min":1.1319999999999999},"min":{"1min":0.80500000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.613,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.0389999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1000000000000001,"15min":1.0900000000000001},"min":{"1min":0.72799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8839999999999999,"5min":1.893,"15min":1.893},"last":0.878},{"interface":"front","average":{"1min":1.179,"5min":1.127,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5149999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.2110000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.226,"15min":1.2170000000000001},"min":{"1min":0.98999999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.909,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.1859999999999999,"15min":1.1899999999999999},"min":{"1min":0.89900000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.159}]},{"osd":3,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":1.03,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6339999999999999,"5min":1.819,"15min":1.819},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3180000000000001,"15min":1.3069999999999999},"min":{"1min":1.0609999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.133,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.2809999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.238,"15min":1.1870000000000001},"min":{"1min":0.75,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2909999999999999,"15min":1.3080000000000001},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1799999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.3500000000000001},"min":{"1min":0.95099999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.448,"15min":1.462},"min":{"1min":0.91300000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0750000000000002,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.3500000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5349999999999999,"15min":1.546},"min":{"1min":1.022,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.1869999999999998,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.96299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.8200000000000001,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.458}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":438,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194873,"statfs":{"total":200000000,"available":199550571,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.046,"15min":1.008},"min":{"1min":0.80600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5740000000000001,"5min":1.577,"15min":1.577},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.0369999999999999,"15min":1.008},"min":{"1min":0.89400000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0369999999999999,"15min":1.0149999999999999},"min":{"1min":0.85499999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.484,"5min":1.556,"15min":1.556},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.107,"15min":1.0640000000000001},"min":{"1min":0.89700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.508,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.343}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.127,"15min":1.0640000000000001},"min":{"1min":0.86299999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.498},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.169,"15min":1.153},"min":{"1min":0.871,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5820000000000001,"5min":1.845,"15min":1.845},"last":1.196}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2609999999999999,"15min":1.266},"min":{"1min":0.96999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.028},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1659999999999999,"15min":1.1339999999999999},"min":{"1min":0.93600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0820000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2609999999999999,"15min":1.2050000000000001},"min":{"1min":0.85899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.74,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.417,"15min":1.391},"min":{"1min":1.1519999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6719999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.444}]},{"osd":5,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.274,"15min":1.2709999999999999},"min":{"1min":0.81200000000000006,"5min":0.75,"15min":0.75},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.476},{"interface":"front","average":{"1min":1.419,"5min":1.391,"15min":1.345},"min":{"1min":1.0569999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.71,"5min":1.895,"15min":1.895},"last":1.6910000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":1.0920000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.9259999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.526},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4470000000000001,"15min":1.4370000000000001},"min":{"1min":1.1000000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.9470000000000001,"5min":1.954,"15min":1.954},"last":1.3959999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":446,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194866,"statfs":{"total":200000000,"available":199543254,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.032,"15min":1.022},"min":{"1min":0.75700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.401,"5min":1.431,"15min":1.431},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.04},"min":{"1min":0.79300000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3700000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.3300000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.212,"15min":1.28},"min":{"1min":0.88,"5min":0.62,"15min":0.62},"max":{"1min":1.6830000000000001,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.1259999999999999,"15min":1.157},"min":{"1min":0.83799999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.877,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.23,"15min":1.212},"min":{"1min":0.91200000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5740000000000001,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1830000000000001,"15min":1.173},"min":{"1min":0.83899999999999997,"5min":0.752,"15min":0.752},"max":{"1min":1.502,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.179,"15min":1.1830000000000001},"min":{"1min":0.93600000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.389,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.238},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2110000000000001,"15min":1.1619999999999999},"min":{"1min":0.98999999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.514,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.367}]},{"osd":4,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2010000000000001,"15min":1.224},"min":{"1min":0.85499999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5960000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.266,"15min":1.276},"min":{"1min":0.79500000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5920000000000001,"5min":1.833,"15min":1.833},"last":1.0720000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3,"15min":1.292},"min":{"1min":0.92600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.538,"5min":1.877,"15min":1.877},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.353,"15min":1.345},"min":{"1min":0.95399999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6879999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4019999999999999,"15min":1.3979999999999999},"min":{"1min":1.012,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.762,"5min":1.881,"15min":1.881},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.385,"15min":1.3600000000000001},"min":{"1min":1.036,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.645,"5min":1.806,"15min":1.806},"last":1.4450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":445,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194867,"statfs":{"total":200000000,"available":199544293,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.0600000000000001,"15min":0.98099999999999998},"min":{"1min":0.81200000000000006,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.536,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.78300000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.514}]},{"osd":1,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.878,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5640000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.079,"15min":1.0269999999999999},"min":{"1min":0.81000000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.694,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.149,"15min":1.0860000000000001},"min":{"1min":0.78900000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6100000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1140000000000001,"15min":1.056},"min":{"1min":0.86799999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.29}]},{"osd":3,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2370000000000001,"15min":1.1699999999999999},"min":{"1min":0.90300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.663,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2769999999999999,"15min":1.228},"min":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6519999999999999,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.127}]},{"osd":5,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1659999999999999,"15min":1.1200000000000001},"min":{"1min":0.76700000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6040000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.2909999999999999,"15min":1.2589999999999999},"min":{"1min":0.91300000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7390000000000001,"5min":2.069,"15min":2.069},"last":1.6000000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3149999999999999,"15min":1.2749999999999999},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.788,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.353},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.403,"15min":1.3740000000000001},"min":{"1min":1,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9410000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.385}]},{"osd":7,"last update":"Fri Feb 2 13:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.403,"15min":1.365},"min":{"1min":1.171,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.98,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4390000000000001,"15min":1.4319999999999999},"min":{"1min":0.97499999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7689999999999999,"5min":2.698,"15min":2.698},"last":1.486}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":439,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549532,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.266,"5min":1.302,"15min":1.302},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.89100000000000001,"15min":0.88300000000000001},"min":{"1min":0.73699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.619,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.92500000000000004,"15min":0.89400000000000002},"min":{"1min":0.81100000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3640000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.048,"15min":1.018},"min":{"1min":0.90200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6359999999999999,"5min":2.125,"15min":2.125},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0069999999999999,"15min":0.95299999999999996},"min":{"1min":0.84599999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.90400000000000003,"15min":0.88800000000000001},"min":{"1min":0.66500000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.206,"15min":1.1859999999999999},"min":{"1min":0.97599999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.669,"5min":3.585,"15min":3.585},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.244,"15min":1.2030000000000001},"min":{"1min":1.036,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.552,"5min":1.877,"15min":1.877},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.212,"15min":1.2210000000000001},"min":{"1min":0.94899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6579999999999999,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":1.075,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.7070000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.91700000000000004}]},{"osd":6,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.3120000000000001,"15min":1.256},"min":{"1min":1.2070000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.254,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.96999999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.55,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri Feb 2 13:28:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.2749999999999999,"15min":1.2609999999999999},"min":{"1min":1.153,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7190000000000001,"5min":3.391,"15min":3.391},"last":1.044},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3500000000000001,"15min":1.3120000000000001},"min":{"1min":1.2849999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7909999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.40799999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.143},{"interface":"front","average":{"1min":0.997,"5min":0.81899999999999995,"15min":0.76900000000000002},"min":{"1min":0.56599999999999995,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.123,"5min":0.91100000000000003,"15min":0.84399999999999997},"min":{"1min":0.73699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2849999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.85099999999999998,"15min":0.77700000000000002},"min":{"1min":0.61799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.087},{"interface":"front","average":{"1min":1.165,"5min":0.998,"15min":0.96999999999999997},"min":{"1min":0.68000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2050000000000001,"15min":1.165},"min":{"1min":1.018,"5min":0.748,"15min":0.748},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.1850000000000001,"15min":1.145},"min":{"1min":0.81799999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5620000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.401}]},{"osd":5,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.147,"15min":1.0700000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":1.1699999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.2070000000000001,"15min":1.161},"min":{"1min":0.89900000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.502},{"interface":"front","average":{"1min":1.329,"5min":1.2230000000000001,"15min":1.1870000000000001},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.421}]},{"osd":7,"last update":"Fri Feb 2 13:28:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.246,"15min":1.1850000000000001},"min":{"1min":0.92200000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.266,"15min":1.198},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.21}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548483,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91800000000000004,"15min":1.03},"min":{"1min":0.63,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.367,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.1579999999999999,"15min":1.4139999999999999},"min":{"1min":0.77600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.3120000000000001,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":1.296}]},{"osd":2,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.89500000000000002,"15min":0.91900000000000004},"min":{"1min":0.71899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.431,"5min":1.639,"15min":1.639},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.016,"15min":1.0960000000000001},"min":{"1min":0.79600000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.484,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":1.018}]},{"osd":3,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1439999999999999,"15min":1.2729999999999999},"min":{"1min":0.88300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.647,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":1.137},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.161,"15min":1.3129999999999999},"min":{"1min":0.89500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7030000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2310000000000001,"15min":1.343},"min":{"1min":0.99199999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.627,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.296,"15min":1.3740000000000001},"min":{"1min":1.0840000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.581,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.415}]},{"osd":5,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3700000000000001,"15min":1.474},"min":{"1min":1.119,"5min":0.753,"15min":0.753},"max":{"1min":1.6080000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.353,"15min":1.4530000000000001},"min":{"1min":1.1839999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6990000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.355}]},{"osd":6,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4370000000000001,"15min":1.603},"min":{"1min":1.0209999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.782,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.409,"15min":1.514},"min":{"1min":1.1060000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.806,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.226}]},{"osd":7,"last update":"Fri Feb 2 13:28:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.452,"15min":1.575},"min":{"1min":1.1040000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1469999999999998,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.329},{"interface":"front","average":{"1min":1.464,"5min":1.496,"15min":1.6439999999999999},"min":{"1min":1.23,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7350000000000001,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.389}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1015,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194297,"statfs":{"total":200000000,"available":198960295,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.98399999999999999,"15min":1.1699999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.472,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.2130000000000001,"15min":1.5489999999999999},"min":{"1min":0.45300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.343,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.425}]},{"osd":2,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0529999999999999,"15min":1.1919999999999999},"min":{"1min":0.48499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.401,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":1.244},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.236,"15min":1.5429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3129999999999999,"5min":10.781000000000001,"15min":10.781000000000001},"last":1.0049999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.093,"15min":1.274},"min":{"1min":0.64000000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.427,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.1890000000000001,"15min":1.345},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.5449999999999999,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2689999999999999,"15min":1.3640000000000001},"min":{"1min":0.996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.621,"5min":3.669,"15min":3.669},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3360000000000001,"15min":1.4239999999999999},"min":{"1min":0.78200000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.774,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.3340000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.579,"15min":1.891},"min":{"1min":0.83199999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6870000000000001,"5min":10.651,"15min":10.651},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.349,"15min":1.4510000000000001},"min":{"1min":1.0109999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5840000000000001,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.379}]},{"osd":6,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.395,"15min":1.494},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.8300000000000001,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.5980000000000001,"15min":1.9239999999999999},"min":{"1min":1.05,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6160000000000001,"5min":10.708,"15min":10.708},"last":1.129}]},{"osd":7,"last update":"Fri Feb 2 13:28:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.637,"15min":1.931},"min":{"1min":0.872,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5900000000000001,"5min":10.74,"15min":10.74},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3779999999999999,"15min":1.468},"min":{"1min":1.0920000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6539999999999999,"5min":3.863,"15min":3.863},"last":1.5149999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:29:09.755 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:29:09.755 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:29:10.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.008+0000 7fe463000640 1 Processor -- start 2024-02-02T13:29:10.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.008+0000 7fe463000640 1 -- start start 2024-02-02T13:29:10.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.009+0000 7fe463000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe45c15d490 0x7fe45c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.009+0000 7fe463000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 0x7fe45c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.009+0000 7fe463000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45c15d870 con 0x7fe45c0b0700 2024-02-02T13:29:10.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.009+0000 7fe463000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c0ad270 con 0x7fe45c15d490 2024-02-02T13:29:10.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.009+0000 7fe463000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c06bce0 con 0x7fe45c0ac8d0 2024-02-02T13:29:10.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.010+0000 7fe45b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 0x7fe45c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.010+0000 7fe45be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe45c15d490 0x7fe45c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.010+0000 7fe45be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe45c15d490 0x7fe45c06c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:51676/0 (socket says 172.21.15.91:51676) 2024-02-02T13:29:10.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.010+0000 7fe45be00640 1 -- 172.21.15.91:0/1591539678 learned_addr learned my addr 172.21.15.91:0/1591539678 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:10.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.010+0000 7fe45b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 0x7fe45c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:58102/0 (socket says 172.21.15.91:58102) 2024-02-02T13:29:10.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.011+0000 7fe45be00640 1 -- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 msgr2=0x7fe45c0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:10.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.011+0000 7fe45be00640 1 --2- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 0x7fe45c0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.011+0000 7fe45be00640 1 -- 172.21.15.91:0/1591539678 >> v1:172.21.15.113:6789/0 conn(0x7fe45c0b0700 legacy=0x7fe45c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.011+0000 7fe45be00640 1 -- 172.21.15.91:0/1591539678 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c161220 con 0x7fe45c15d490 2024-02-02T13:29:10.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.013+0000 7fe45be00640 1 --2- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c15d490 0x7fe45c06c170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009870 tx=0x7fe44c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f468f0c9d990c53b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.013+0000 7fe45aa00640 1 -- 172.21.15.91:0/1591539678 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c030840 con 0x7fe45c15d490 2024-02-02T13:29:10.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.013+0000 7fe45aa00640 1 -- 172.21.15.91:0/1591539678 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c0309a0 con 0x7fe45c15d490 2024-02-02T13:29:10.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.013+0000 7fe45aa00640 1 -- 172.21.15.91:0/1591539678 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c030d30 con 0x7fe45c15d490 2024-02-02T13:29:10.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 -- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c15d490 msgr2=0x7fe45c06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 --2- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c15d490 0x7fe45c06c170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009870 tx=0x7fe44c02b980 comp rx=0 tx=0).stop 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 -- 172.21.15.91:0/1591539678 shutdown_connections 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 --2- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c0ac8d0 0x7fe45c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 --2- 172.21.15.91:0/1591539678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c15d490 0x7fe45c06c170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 -- 172.21.15.91:0/1591539678 >> 172.21.15.91:0/1591539678 conn(0x7fe45c0cec80 msgr2=0x7fe45c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 -- 172.21.15.91:0/1591539678 shutdown_connections 2024-02-02T13:29:10.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.014+0000 7fe463000640 1 -- 172.21.15.91:0/1591539678 wait complete. 2024-02-02T13:29:10.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.016+0000 7fe463000640 1 Processor -- start 2024-02-02T13:29:10.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.016+0000 7fe463000640 1 -- start start 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 0x7fe45c0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c15d490 0x7fe45c070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c15e310 con 0x7fe45c0b0700 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c2c60 con 0x7fe45c0ac8d0 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe463000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c15da00 con 0x7fe45c15d490 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe45b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe45b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58372/0 (socket says 172.21.15.91:58372) 2024-02-02T13:29:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.017+0000 7fe45b400640 1 -- 172.21.15.91:0/3442190642 learned_addr learned my addr 172.21.15.91:0/3442190642 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 -- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c15d490 msgr2=0x7fe45c070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c15d490 0x7fe45c070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45be00640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 0x7fe45c0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 -- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 msgr2=0x7fe45c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 0x7fe45c0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 -- 172.21.15.91:0/3442190642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0094d0 con 0x7fe45c0b0700 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45be00640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 0x7fe45c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe45b400640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe44400e870 tx=0x7fe44400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.018+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4440094f0 con 0x7fe45c0b0700 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.019+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44400c940 con 0x7fe45c0b0700 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.019+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44400ccf0 con 0x7fe45c0b0700 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.019+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c0c57e0 con 0x7fe45c0b0700 2024-02-02T13:29:10.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.019+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c15dc50 con 0x7fe45c0b0700 2024-02-02T13:29:10.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.020+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c073830 con 0x7fe45c0b0700 2024-02-02T13:29:10.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.021+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe444013470 con 0x7fe45c0b0700 2024-02-02T13:29:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.021+0000 7fe453e00640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 0x7fe43406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.022+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe444088a60 con 0x7fe45c0b0700 2024-02-02T13:29:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.022+0000 7fe45be00640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 0x7fe43406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.023+0000 7fe45be00640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 0x7fe43406c8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe44c036000 tx=0x7fe44c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.026+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe444053fc0 con 0x7fe45c0b0700 2024-02-02T13:29:10.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.283+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe45c0ad770 con 0x7fe45c0b0700 2024-02-02T13:29:10.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.291+0000 7fe453e00640 1 -- 172.21.15.91:0/3442190642 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe4440575e0 con 0x7fe45c0b0700 2024-02-02T13:29:10.293 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.293+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 msgr2=0x7fe43406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.293+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 0x7fe43406c8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe44c036000 tx=0x7fe44c03b040 comp rx=0 tx=0).stop 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 msgr2=0x7fe45c14e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe44400e870 tx=0x7fe44400ed30 comp rx=0 tx=0).stop 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 shutdown_connections 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe43406a430 0x7fe43406c8f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe45c15d490 0x7fe45c070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe45c0b0700 0x7fe45c14e080 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 --2- 172.21.15.91:0/3442190642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe45c0ac8d0 0x7fe45c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 >> 172.21.15.91:0/3442190642 conn(0x7fe45c0cec80 msgr2=0x7fe45c157910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:10.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 shutdown_connections 2024-02-02T13:29:10.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.294+0000 7fe463000640 1 -- 172.21.15.91:0/3442190642 wait complete. 2024-02-02T13:29:10.312 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:29:10.356 INFO:tasks.ceph.mds.c.smithi091.stderr:2024-02-02T13:29:10.355+0000 7f9202200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:29:10.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.515+0000 7f9164c00640 1 Processor -- start 2024-02-02T13:29:10.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.515+0000 7f9164c00640 1 -- start start 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.516+0000 7f9164c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f916006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.516+0000 7f9164c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91600ccdd0 con 0x7f91600ce270 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.516+0000 7f9164c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9160166020 con 0x7f91600b0700 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.516+0000 7f9164c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91600cc7c0 con 0x7f9160161660 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f91600ce270 0x7f9160165910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40612/0 (socket says 172.21.15.91:40612) 2024-02-02T13:29:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915e800640 1 -- 172.21.15.91:0/3120849507 learned_addr learned my addr 172.21.15.91:0/3120849507 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915de00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f916006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892432496 0 0) 0x7f91600ccdd0 con 0x7f91600ce270 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9140003520 con 0x7f91600ce270 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399926608 0 0) 0x7f9140003520 con 0x7f91600ce270 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 msgr2=0x7f916006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f916006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 >> v1:172.21.15.113:6789/0 conn(0x7f91600b0700 legacy=0x7f9160165200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916006c540 con 0x7f91600ce270 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.517+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f91580019f0 con 0x7f91600ce270 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 >> v1:172.21.15.91:6789/0 conn(0x7f91600ce270 legacy=0x7f9160165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91400039a0 0x7f9140003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f91400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9140004d00 0x7f9140005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600cc7c0 con 0x7f9140004d00 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914000bab0 con 0x7f91400039a0 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914000bc70 con 0x7f9160161660 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9140004d00 0x7f9140005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915de00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91400039a0 0x7f9140003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915d400640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f91400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 -- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 msgr2=0x7f91400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f91400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 -- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91400039a0 msgr2=0x7f9140003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91400039a0 0x7f9140003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915e800640 1 -- 172.21.15.91:0/3120849507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140003520 con 0x7f9140004d00 2024-02-02T13:29:10.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.518+0000 7f915d400640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9160161660 0x7f91400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:10.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.519+0000 7f915e800640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9140004d00 0x7f9140005180 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f915800a830 tx=0x7f91400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8303e5d35dbdb9ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580072d0 con 0x7f9140004d00 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915802f070 con 0x7f9140004d00 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f915ca00640 1 -- 172.21.15.91:0/3120849507 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580076b0 con 0x7f9140004d00 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 -- 172.21.15.91:0/3120849507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9140004d00 msgr2=0x7f9140005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 --2- 172.21.15.91:0/3120849507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9140004d00 0x7f9140005180 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f915800a830 tx=0x7f91400057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f915e800640 1 -- 172.21.15.91:0/3120849507 reap_dead start 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 -- 172.21.15.91:0/3120849507 shutdown_connections 2024-02-02T13:29:10.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 -- 172.21.15.91:0/3120849507 >> 172.21.15.91:0/3120849507 conn(0x7f91600cec80 msgr2=0x7f91600cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:10.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 -- 172.21.15.91:0/3120849507 shutdown_connections 2024-02-02T13:29:10.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.520+0000 7f9164c00640 1 -- 172.21.15.91:0/3120849507 wait complete. 2024-02-02T13:29:10.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.522+0000 7f9164c00640 1 Processor -- start 2024-02-02T13:29:10.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.522+0000 7f9164c00640 1 -- start start 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 0x7f9160155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 0x7f91600cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600cc0c0 con 0x7f91601fa720 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91601558d0 con 0x7f91600cb700 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f9164c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91600c5cc0 con 0x7f9140004d00 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f915d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f915d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58378/0 (socket says 172.21.15.91:58378) 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.523+0000 7f915d400640 1 -- 172.21.15.91:0/2130582188 learned_addr learned my addr 172.21.15.91:0/2130582188 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915de00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 0x7f9160155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915e800640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 0x7f91600cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 -- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 msgr2=0x7f9160155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 0x7f9160155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 -- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 msgr2=0x7f91600cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 0x7f91600cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 -- 172.21.15.91:0/2130582188 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91601fac90 con 0x7f91601fa720 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915de00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 0x7f9160155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915e800640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 0x7f91600cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f915d400640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f914800d7a0 tx=0x7f914800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.524+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148017070 con 0x7f91601fa720 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.525+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9148005070 con 0x7f91601fa720 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.525+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148005420 con 0x7f91601fa720 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.525+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91601faf70 con 0x7f91601fa720 2024-02-02T13:29:10.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.525+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91601fb530 con 0x7f91601fa720 2024-02-02T13:29:10.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.526+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91600c5e00 con 0x7f91601fa720 2024-02-02T13:29:10.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.527+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9148007030 con 0x7f91601fa720 2024-02-02T13:29:10.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.529+0000 7f9156000640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 0x7f913806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:10.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.529+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91480881d0 con 0x7f91601fa720 2024-02-02T13:29:10.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.529+0000 7f915de00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 0x7f913806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:10.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.529+0000 7f915de00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 0x7f913806c8f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9150009a00 tx=0x7f9150008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:10.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.533+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f91480537c0 con 0x7f91601fa720 2024-02-02T13:29:10.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.763+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9160066cf0 con 0x7f91601fa720 2024-02-02T13:29:10.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.764+0000 7f9156000640 1 -- 172.21.15.91:0/2130582188 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f9148056de0 con 0x7f91601fa720 2024-02-02T13:29:10.765 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:10.765 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:29:10.765 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:29:10.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.766+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 msgr2=0x7f913806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.766+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 0x7f913806c8f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9150009a00 tx=0x7f9150008040 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 msgr2=0x7f9160152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f914800d7a0 tx=0x7f914800dc60 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 shutdown_connections 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f913806a430 0x7f913806c8f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91600cb700 0x7f91600cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91601fa720 0x7f9160152150 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 --2- 172.21.15.91:0/2130582188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9140004d00 0x7f9160155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 >> 172.21.15.91:0/2130582188 conn(0x7f91600cec80 msgr2=0x7f91600b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 shutdown_connections 2024-02-02T13:29:10.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:10.767+0000 7f9164c00640 1 -- 172.21.15.91:0/2130582188 wait complete. 2024-02-02T13:29:11.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:29:04.868792+0000', 'modified': '2024-02-02T13:29:07.903616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6324}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6324': {'gid': 6324, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/3733831376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6836', 'nonce': 3733831376}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 3733831376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:29:11.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:29:11.188 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T13:29:11.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.403+0000 7f6e24800640 1 Processor -- start 2024-02-02T13:29:11.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.403+0000 7f6e24800640 1 -- start start 2024-02-02T13:29:11.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e24800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e1c070270 0x7f6e1c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e24800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e1c0b6bd0 0x7f6e1c0b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e24800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1c160d90 con 0x7f6e1c1609b0 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e24800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c070bf0 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e24800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c070fe0 con 0x7f6e1c070270 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e21e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6e1c1609b0 0x7f6e1c0cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40626/0 (socket says 172.21.15.91:40626) 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e21e00640 1 -- 172.21.15.91:0/916103221 learned_addr learned my addr 172.21.15.91:0/916103221 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:11.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.405+0000 7f6e22800640 1 --2- 172.21.15.91:0/916103221 >> v2:172.21.15.91:3300/0 conn(0x7f6e1c0b6bd0 0x7f6e1c0b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.406+0000 7f6e21400640 1 --2- 172.21.15.91:0/916103221 >> v2:172.21.15.113:3300/0 conn(0x7f6e1c070270 0x7f6e1c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.406+0000 7f6e22800640 1 -- 172.21.15.91:0/916103221 >> v2:172.21.15.113:3300/0 conn(0x7f6e1c070270 msgr2=0x7f6e1c0706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.406+0000 7f6e22800640 1 --2- 172.21.15.91:0/916103221 >> v2:172.21.15.113:3300/0 conn(0x7f6e1c070270 0x7f6e1c0706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.406+0000 7f6e22800640 1 -- 172.21.15.91:0/916103221 >> v1:172.21.15.91:6789/0 conn(0x7f6e1c1609b0 legacy=0x7f6e1c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.406+0000 7f6e22800640 1 -- 172.21.15.91:0/916103221 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0d2940 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.407+0000 7f6e22800640 1 --2- 172.21.15.91:0/916103221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b7020 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c067830 tx=0x7f6e1402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0a9a46fe03ad62 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.407+0000 7f6e20a00640 1 -- 172.21.15.91:0/916103221 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1402cb50 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.407+0000 7f6e20a00640 1 -- 172.21.15.91:0/916103221 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1402ccb0 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.407+0000 7f6e20a00640 1 -- 172.21.15.91:0/916103221 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14034660 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 -- 172.21.15.91:0/916103221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c0b6bd0 msgr2=0x7f6e1c0b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 --2- 172.21.15.91:0/916103221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b7020 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c067830 tx=0x7f6e1402bda0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 -- 172.21.15.91:0/916103221 shutdown_connections 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 --2- 172.21.15.91:0/916103221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b7020 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 --2- 172.21.15.91:0/916103221 >> v2:172.21.15.113:3300/0 conn(0x7f6e1c070270 0x7f6e1c0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 -- 172.21.15.91:0/916103221 >> 172.21.15.91:0/916103221 conn(0x7f6e1c0d3020 msgr2=0x7f6e1c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 -- 172.21.15.91:0/916103221 shutdown_connections 2024-02-02T13:29:11.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.408+0000 7f6e24800640 1 -- 172.21.15.91:0/916103221 wait complete. 2024-02-02T13:29:11.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.409+0000 7f6e24800640 1 Processor -- start 2024-02-02T13:29:11.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.409+0000 7f6e24800640 1 -- start start 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e1c070270 0x7f6e1c0a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c14de70 con 0x7f6e1c1609b0 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c14d260 con 0x7f6e1c0b6bd0 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.410+0000 7f6e24800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c14d540 con 0x7f6e1c070270 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58388/0 (socket says 172.21.15.91:58388) 2024-02-02T13:29:11.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 -- 172.21.15.91:0/1670352747 learned_addr learned my addr 172.21.15.91:0/1670352747 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e21400640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e21e00640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e1c070270 0x7f6e1c0a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 -- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e1c070270 msgr2=0x7f6e1c0a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e1c070270 0x7f6e1c0a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 -- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 msgr2=0x7f6e1c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 -- 172.21.15.91:0/1670352747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009a70 con 0x7f6e1c1609b0 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e21400640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.411+0000 7f6e22800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009e80 tx=0x7f6e14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.412+0000 7f6e1a000640 1 -- 172.21.15.91:0/1670352747 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14040650 con 0x7f6e1c1609b0 2024-02-02T13:29:11.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.412+0000 7f6e1a000640 1 -- 172.21.15.91:0/1670352747 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e140407d0 con 0x7f6e1c1609b0 2024-02-02T13:29:11.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.412+0000 7f6e1a000640 1 -- 172.21.15.91:0/1670352747 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14040b80 con 0x7f6e1c1609b0 2024-02-02T13:29:11.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.412+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c14d820 con 0x7f6e1c1609b0 2024-02-02T13:29:11.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.412+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c0b4a20 con 0x7f6e1c1609b0 2024-02-02T13:29:11.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.414+0000 7f6e1a000640 1 -- 172.21.15.91:0/1670352747 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6e14040ce0 con 0x7f6e1c1609b0 2024-02-02T13:29:11.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.415+0000 7f6e1a000640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.415+0000 7f6e1a000640 1 -- 172.21.15.91:0/1670352747 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e140ab8d0 con 0x7f6e1c1609b0 2024-02-02T13:29:11.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.415+0000 7f6e21e00640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.415+0000 7f6e27634740 1 Processor -- start 2024-02-02T13:29:11.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.415+0000 7f6e21e00640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 0x7f6df406c8f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005880 tx=0x7f6e0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 -- start start 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5469ed0 0x55ddc54971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5445650 con 0x55ddc5499f50 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc53da000 con 0x55ddc546dad0 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.416+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddc53d7790 con 0x55ddc5469ed0 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58392/0 (socket says 172.21.15.91:58392) 2024-02-02T13:29:11.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 -- 172.21.15.91:0/4164712338 learned_addr learned my addr 172.21.15.91:0/4164712338 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 -- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5469ed0 msgr2=0x55ddc54971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5469ed0 0x55ddc54971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 -- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 msgr2=0x55ddc5497700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 -- 172.21.15.91:0/4164712338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc539a710 con 0x55ddc5499f50 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e21e00640 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005c80 tx=0x7f6e0c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c71d7663a22786e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.417+0000 7f6e12a00640 1 -- 172.21.15.91:0/4164712338 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c007760 con 0x55ddc5499f50 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e12a00640 1 -- 172.21.15.91:0/4164712338 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00da70 con 0x55ddc5499f50 2024-02-02T13:29:11.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e12a00640 1 -- 172.21.15.91:0/4164712338 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00dde0 con 0x55ddc5499f50 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 -- 172.21.15.91:0/4164712338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 msgr2=0x55ddc549a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005c80 tx=0x7f6e0c010000 comp rx=0 tx=0).stop 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 -- 172.21.15.91:0/4164712338 shutdown_connections 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5469ed0 0x55ddc54971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 --2- 172.21.15.91:0/4164712338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc5499f50 0x55ddc549a330 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 -- 172.21.15.91:0/4164712338 >> 172.21.15.91:0/4164712338 conn(0x55ddc546c1e0 msgr2=0x55ddc54b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 -- 172.21.15.91:0/4164712338 shutdown_connections 2024-02-02T13:29:11.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.418+0000 7f6e27634740 1 -- 172.21.15.91:0/4164712338 wait complete. 2024-02-02T13:29:11.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.419+0000 7f6e27634740 1 -- start start 2024-02-02T13:29:11.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.419+0000 7f6e27634740 1 Processor -- start 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc5499f50 0x55ddc548e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc54b9c90 0x55ddc548ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc53d3080 con 0x55ddc546dad0 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5374850 con 0x55ddc5499f50 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddc53744f0 con 0x55ddc54b9c90 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e27634740 10 client.0 _ll_register_callbacks cb 0x55ddc53cc9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58402/0 (socket says 172.21.15.91:58402) 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.420+0000 7f6e21e00640 1 -- 172.21.15.91:0/2108182205 learned_addr learned my addr 172.21.15.91:0/2108182205 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 -- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc54b9c90 msgr2=0x55ddc548ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc54b9c90 0x55ddc548ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 -- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc5499f50 msgr2=0x55ddc548e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc5499f50 0x55ddc548e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 -- 172.21.15.91:0/2108182205 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c0059f0 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000f60 tx=0x7f6e0c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6078f9a3caefe31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e0b400640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c027040 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e0c02dae0 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddc53fd5d0 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e0c02dce0 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e27634740 10 client.6465 fetch_fsmap learned FSMap version 40 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.421+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55ddc53d1700 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddc53fd5d0 con 0x55ddc546dad0 2024-02-02T13:29:11.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6e0c0208e0 con 0x55ddc546dad0 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 10 client.6465 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 10 client.6465 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 20 client.6465 populate_metadata read hostname 'smithi091' 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc5494940 0x55ddc54c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 4 client.6465 mds_command: new command op to 6324 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 --> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddc53c58d0 con 0x55ddc5494940 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.422+0000 7f6e21400640 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc5494940 0x55ddc54c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.423+0000 7f6e21400640 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc5494940 0x55ddc54c7e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4a0c6e05c63f4b server_cookie=8622a2157ef62fd9 in_seq=0 out_seq=0 2024-02-02T13:29:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.423+0000 7f6e0b400640 10 client.6465 ms_handle_connect on v2:172.21.15.91:6836/3733831376 2024-02-02T13:29:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.423+0000 7f6e0b400640 1 -- 172.21.15.91:0/2108182205 <== mds.0 v2:172.21.15.91:6836/3733831376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ddc53c58d0 con 0x55ddc5494940 2024-02-02T13:29:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.423+0000 7f6e0b400640 10 client.6465 handle_command_reply: tid=0 2024-02-02T13:29:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.423+0000 7f6e27634740 1 client.6465 shutdown 2024-02-02T13:29:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 msgr2=0x55ddc54900b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000f60 tx=0x7f6e0c007000 comp rx=0 tx=0).stop 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 shutdown_connections 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc5494940 0x55ddc54c7e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc54b9c90 0x55ddc548ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc5499f50 0x55ddc548e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e0b400640 0 client.6465 ms_handle_reset on v2:172.21.15.91:6836/3733831376 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 --2- 172.21.15.91:0/2108182205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc54900b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 >> 172.21.15.91:0/2108182205 conn(0x55ddc5469ed0 msgr2=0x55ddc546e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 shutdown_connections 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 1 -- 172.21.15.91:0/2108182205 wait complete. 2024-02-02T13:29:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.424+0000 7f6e27634740 20 client.6465 trim_cache size 0 max 16384 2024-02-02T13:29:11.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.440+0000 7f6e27634740 1 Processor -- start 2024-02-02T13:29:11.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.440+0000 7f6e27634740 1 -- start start 2024-02-02T13:29:11.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc546a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 0x55ddc54693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc513af20 con 0x55ddc546dad0 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc54b60f0 con 0x55ddc54b9c90 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddc54b79a0 con 0x55ddc5499f50 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58418/0 (socket says 172.21.15.91:58418) 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21e00640 1 -- 172.21.15.91:0/2025510978 learned_addr learned my addr 172.21.15.91:0/2025510978 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e22800640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 0x55ddc54693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21400640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc546a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21e00640 1 -- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 msgr2=0x55ddc546a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.441+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc546a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e21e00640 1 -- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 msgr2=0x55ddc54693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 0x55ddc54693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e21e00640 1 -- 172.21.15.91:0/2025510978 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc5407280 con 0x55ddc546dad0 2024-02-02T13:29:11.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e22800640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 0x55ddc54693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e21e00640 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007e90 tx=0x7f6e0c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d025302735061f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e13400640 1 -- 172.21.15.91:0/2025510978 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c027070 con 0x55ddc546dad0 2024-02-02T13:29:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e13400640 1 -- 172.21.15.91:0/2025510978 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c0208e0 con 0x55ddc546dad0 2024-02-02T13:29:11.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e13400640 1 -- 172.21.15.91:0/2025510978 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c02d240 con 0x55ddc546dad0 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e27634740 1 -- 172.21.15.91:0/2025510978 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 msgr2=0x55ddc5497700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.442+0000 7f6e27634740 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007e90 tx=0x7f6e0c011000 comp rx=0 tx=0).stop 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 -- 172.21.15.91:0/2025510978 shutdown_connections 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc546a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc54b9c90 0x55ddc54693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 --2- 172.21.15.91:0/2025510978 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc546dad0 0x55ddc5497700 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 -- 172.21.15.91:0/2025510978 >> 172.21.15.91:0/2025510978 conn(0x55ddc5469ed0 msgr2=0x55ddc54b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 -- 172.21.15.91:0/2025510978 shutdown_connections 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 -- 172.21.15.91:0/2025510978 wait complete. 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.443+0000 7f6e27634740 1 -- start start 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.444+0000 7f6e27634740 1 Processor -- start 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.444+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc54918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.444+0000 7f6e27634740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc54691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.444+0000 7f6e27634740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e27634740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc551b340 con 0x55ddc54b9c90 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc54c6980 con 0x55ddc546dad0 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e27634740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddc54c6bc0 con 0x55ddc5499f50 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e27634740 10 client.0 _ll_register_callbacks cb 0x7f6e0c005b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58422/0 (socket says 172.21.15.91:58422) 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 -- 172.21.15.91:0/3929053939 learned_addr learned my addr 172.21.15.91:0/3929053939 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 -- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 msgr2=0x55ddc54691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc54691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 -- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 msgr2=0x55ddc54918e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc54918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 -- 172.21.15.91:0/3929053939 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddc53d1700 con 0x55ddc54b9c90 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e22800640 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e1402c3b0 tx=0x7f6e1407aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabad5fd1586671a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.445+0000 7f6e0b400640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:29:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e140a3ea0 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e1407ad40 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e0c00da70 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1407af20 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 10 client.6471 fetch_fsmap learned FSMap version 40 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55ddc5373a60 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00da70 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 40) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6e14050020 con 0x55ddc54b9c90 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 10 client.6471 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 10 client.6471 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 20 client.6471 populate_metadata read hostname 'smithi091' 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc551e860 0x55ddc551ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 4 client.6471 mds_command: new command op to 6324 tid=0[{"prefix": "status"}] 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 --> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ddc53cc100 con 0x55ddc551e860 2024-02-02T13:29:11.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.446+0000 7f6e21e00640 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc551e860 0x55ddc551ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e21e00640 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc551e860 0x55ddc551ec40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30da81d5efcc7f2 server_cookie=1d7da76952fdc0e6 in_seq=0 out_seq=0 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e0b400640 10 client.6471 ms_handle_connect on v2:172.21.15.91:6836/3733831376 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e0b400640 1 -- 172.21.15.91:0/3929053939 <== mds.0 v2:172.21.15.91:6836/3733831376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ddc53cc100 con 0x55ddc551e860 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e0b400640 10 client.6471 handle_command_reply: tid=0 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 client.6471 shutdown 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 msgr2=0x55ddc5469730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e1402c3b0 tx=0x7f6e1407aae0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 shutdown_connections 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:6836/3733831376,v1:172.21.15.91:6837/3733831376] conn(0x55ddc551e860 0x55ddc551ec40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ddc5499f50 0x55ddc54691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55ddc54b9c90 0x55ddc5469730 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 --2- 172.21.15.91:0/3929053939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddc546dad0 0x55ddc54918e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 >> 172.21.15.91:0/3929053939 conn(0x55ddc5469ed0 msgr2=0x55ddc54b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.447+0000 7f6e0b400640 0 client.6471 ms_handle_reset on v2:172.21.15.91:6836/3733831376 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 shutdown_connections 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e27634740 1 -- 172.21.15.91:0/3929053939 wait complete. 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e27634740 20 client.6471 trim_cache size 0 max 16384 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 6324, 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:29:11.452 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.5736654239999996, 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 40, 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 86, 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 86, 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 11.840763924999999 2024-02-02T13:29:11.453 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 msgr2=0x7f6df406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 0x7f6df406c8f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005880 tx=0x7f6e0c009040 comp rx=0 tx=0).stop 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 msgr2=0x7f6e1c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009e80 tx=0x7f6e14004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 shutdown_connections 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e1c1609b0 0x7f6e1c0cca10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1c0b6bd0 0x7f6e1c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 --2- 172.21.15.91:0/1670352747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e1c070270 0x7f6e1c0a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 >> 172.21.15.91:0/1670352747 conn(0x7f6e1c0d3020 msgr2=0x7f6e1c0b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 shutdown_connections 2024-02-02T13:29:11.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.448+0000 7f6e24800640 1 -- 172.21.15.91:0/1670352747 wait complete. 2024-02-02T13:29:11.463 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:29:11.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.653+0000 7f43e5a00640 1 Processor -- start 2024-02-02T13:29:11.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.653+0000 7f43e5a00640 1 -- start start 2024-02-02T13:29:11.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.654+0000 7f43e5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43e00cd690 0x7f43e00cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.654+0000 7f43e5a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43e00c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43e5a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e006b5f0 con 0x7f43e01597a0 2024-02-02T13:29:11.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43e5a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e006cff0 con 0x7f43e00cd690 2024-02-02T13:29:11.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43e5a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00c9bf0 con 0x7f43e00c9240 2024-02-02T13:29:11.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43dea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f43e01597a0 0x7f43e006c8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40636/0 (socket says 172.21.15.91:40636) 2024-02-02T13:29:11.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43dea00640 1 -- 172.21.15.91:0/4168055398 learned_addr learned my addr 172.21.15.91:0/4168055398 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:11.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512805963 0 0) 0x7f43e006b5f0 con 0x7f43e01597a0 2024-02-02T13:29:11.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c0003520 con 0x7f43e01597a0 2024-02-02T13:29:11.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43df400640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43e00c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43d5e00640 1 --2- 172.21.15.91:0/4168055398 >> v2:172.21.15.113:3300/0 conn(0x7f43e00cd690 0x7f43e00cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34813975 0 0) 0x7f43c0003520 con 0x7f43e01597a0 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 msgr2=0x7f43e00c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43e00c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 >> v2:172.21.15.113:3300/0 conn(0x7f43e00cd690 msgr2=0x7f43e00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.655+0000 7f43de000640 1 --2- 172.21.15.91:0/4168055398 >> v2:172.21.15.113:3300/0 conn(0x7f43e00cd690 0x7f43e00cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e006c450 con 0x7f43e01597a0 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f43d0001670 con 0x7f43e01597a0 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 >> v1:172.21.15.91:6789/0 conn(0x7f43e01597a0 legacy=0x7f43e006c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43c00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43c0003fd0 0x7f43c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0004d00 0x7f43c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e006cff0 con 0x7f43c0003fd0 2024-02-02T13:29:11.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00c9bf0 con 0x7f43c0004d00 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000dcc0 con 0x7f43e00c9240 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43c0003fd0 0x7f43c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43dea00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43c00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 -- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 msgr2=0x7f43c00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e00c9240 0x7f43c00039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 -- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0004d00 msgr2=0x7f43c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0004d00 0x7f43c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.656+0000 7f43d5e00640 1 -- 172.21.15.91:0/4168055398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0003520 con 0x7f43c0003fd0 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.657+0000 7f43d5e00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43c0003fd0 0x7f43c00047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ba30 tx=0x7f43c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca53407318dce2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8032680 con 0x7f43c0003fd0 2024-02-02T13:29:11.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80327e0 con 0x7f43c0003fd0 2024-02-02T13:29:11.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43de000640 1 -- 172.21.15.91:0/4168055398 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8032b70 con 0x7f43c0003fd0 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 -- 172.21.15.91:0/4168055398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43c0003fd0 msgr2=0x7f43c00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 --2- 172.21.15.91:0/4168055398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43c0003fd0 0x7f43c00047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ba30 tx=0x7f43c00057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43df400640 1 -- 172.21.15.91:0/4168055398 reap_dead start 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 -- 172.21.15.91:0/4168055398 shutdown_connections 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 -- 172.21.15.91:0/4168055398 >> 172.21.15.91:0/4168055398 conn(0x7f43e00cec80 msgr2=0x7f43e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 -- 172.21.15.91:0/4168055398 shutdown_connections 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.658+0000 7f43e5a00640 1 -- 172.21.15.91:0/4168055398 wait complete. 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.660+0000 7f43e5a00640 1 Processor -- start 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.660+0000 7f43e5a00640 1 -- start start 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 0x7f43e0146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e015b150 0x7f43e015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e015bb10 con 0x7f43e00c7510 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00c8ea0 con 0x7f43c0003fd0 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43e5a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00c8290 con 0x7f43e015b150 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.661+0000 7f43df400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58436/0 (socket says 172.21.15.91:58436) 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 -- 172.21.15.91:0/3245280673 learned_addr learned my addr 172.21.15.91:0/3245280673 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43dea00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 0x7f43e0146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43d5e00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e015b150 0x7f43e015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 -- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e015b150 msgr2=0x7f43e015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e015b150 0x7f43e015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 -- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 msgr2=0x7f43e0146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 0x7f43e0146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 -- 172.21.15.91:0/3245280673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00c8570 con 0x7f43e00c7510 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43dea00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 0x7f43e0146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43df400640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f43e0066830 tx=0x7f43d8007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.662+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8018070 con 0x7f43e00c7510 2024-02-02T13:29:11.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.663+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8003990 con 0x7f43e00c7510 2024-02-02T13:29:11.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.663+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8003d60 con 0x7f43e00c7510 2024-02-02T13:29:11.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.663+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e00c8850 con 0x7f43e00c7510 2024-02-02T13:29:11.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.663+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0146b90 con 0x7f43e00c7510 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.664+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e0066cf0 con 0x7f43e00c7510 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.665+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f43d800e9c0 con 0x7f43e00c7510 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.666+0000 7f43d7400640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 0x7f43b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.666+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43d80886f0 con 0x7f43e00c7510 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.666+0000 7f43dea00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 0x7f43b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:11.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.666+0000 7f43dea00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 0x7f43b806c8f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43d0001200 tx=0x7f43d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:11.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.670+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f43d8053c40 con 0x7f43e00c7510 2024-02-02T13:29:11.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.893+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43e00c8fe0 con 0x7f43e00c7510 2024-02-02T13:29:11.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.894+0000 7f43d7400640 1 -- 172.21.15.91:0/3245280673 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43d8057260 con 0x7f43e00c7510 2024-02-02T13:29:11.895 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 msgr2=0x7f43b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 0x7f43b806c8f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43d0001200 tx=0x7f43d0009040 comp rx=0 tx=0).stop 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 msgr2=0x7f43e00c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f43e0066830 tx=0x7f43d8007ec0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 shutdown_connections 2024-02-02T13:29:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43b806a430 0x7f43b806c8f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e015b150 0x7f43e015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e00c7510 0x7f43e00c7990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 --2- 172.21.15.91:0/3245280673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43c0003fd0 0x7f43e0146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.896+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 >> 172.21.15.91:0/3245280673 conn(0x7f43e00cec80 msgr2=0x7f43e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.897+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 shutdown_connections 2024-02-02T13:29:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:11.897+0000 7f43e5a00640 1 -- 172.21.15.91:0/3245280673 wait complete. 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:29:11.916 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:29:11.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:11.916 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:29:11.966 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:29:11.966 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:29:11.966 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:29:11.967 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42374sec preferred_lft 42374sec 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:29:11.968 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:29:11.969 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:29:11.969 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:11.970 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:29:12.100 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:29:12.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.103 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:29:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.154 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:29:12.154 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:29:12.154 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:29:12.155 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:29:12.155 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:29:12.155 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:29:12.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.334 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:29:12.385 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:29:12.385 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:29:12.386 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:29:12.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.386 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:29:12.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:29:12.466 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:29:12.517 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:29:12.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:29:12.517 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:29:12.612 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:29:12.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.613 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:12.749 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:12.749 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:29:12.780 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:12.781 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:29:12.781 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:29:12.781 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:29:12.781 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:29:12.783 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:29:12.783 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:29:12.783 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:29:12.783 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:29:12.783 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:29:12.787 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:29:12.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.787 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:29:12.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:12.873 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:29:12.939 INFO:teuthology.orchestra.run.smithi091.stdout:client4970 2024-02-02T13:29:12.939 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:29:12.939 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:12.991 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:29:13.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.246+0000 7f2bc9800640 1 Processor -- start 2024-02-02T13:29:13.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.246+0000 7f2bc9800640 1 -- start start 2024-02-02T13:29:13.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.247+0000 7f2bc9800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2bc40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.247+0000 7f2bc9800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc4070f80 con 0x7f2bc415f5a0 2024-02-02T13:29:13.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.247+0000 7f2bc9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc406c600 con 0x7f2bc4151f90 2024-02-02T13:29:13.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.247+0000 7f2bc9800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc415fa30 con 0x7f2bc40b0e10 2024-02-02T13:29:13.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.248+0000 7f2bbbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2bc415f5a0 0x7f2bc40b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40638/0 (socket says 172.21.15.91:40638) 2024-02-02T13:29:13.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.248+0000 7f2bbbe00640 1 -- 172.21.15.91:0/204559171 learned_addr learned my addr 172.21.15.91:0/204559171 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.248+0000 7f2bc3400640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2bc40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.248+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496871795 0 0) 0x7f2bc4070f80 con 0x7f2bc415f5a0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba4003520 con 0x7f2bc415f5a0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083226936 0 0) 0x7f2bc406c600 con 0x7f2bc4151f90 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc4070f80 con 0x7f2bc4151f90 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599450791 0 0) 0x7f2ba4003520 con 0x7f2bc415f5a0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 msgr2=0x7f2bc40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2bc40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 >> v1:172.21.15.113:6789/0 conn(0x7f2bc4151f90 legacy=0x7f2bc406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4152370 con 0x7f2bc415f5a0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2bac001670 con 0x7f2bc415f5a0 2024-02-02T13:29:13.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 >> v1:172.21.15.91:6789/0 conn(0x7f2bc415f5a0 legacy=0x7f2bc40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ba4003b30 0x7f2ba4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40047c0 0x7f2ba4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2ba4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc415fa30 con 0x7f2ba4003b30 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400ba30 con 0x7f2ba40047c0 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400bbf0 con 0x7f2bc40b0e10 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.249+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ba4003b30 0x7f2ba4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bbbe00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40047c0 0x7f2ba4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc3400640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2ba4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc2a00640 1 -- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 msgr2=0x7f2ba4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bc40b0e10 0x7f2ba4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc2a00640 1 -- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40047c0 msgr2=0x7f2ba4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:13.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40047c0 0x7f2ba4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bc2a00640 1 -- 172.21.15.91:0/204559171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4003520 con 0x7f2ba4003b30 2024-02-02T13:29:13.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.250+0000 7f2bbbe00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40047c0 0x7f2ba4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.251+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ba4003b30 0x7f2ba4003f30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018a0 tx=0x7f2ba40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeaaf47698c15b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.251+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40052d0 con 0x7f2ba4003b30 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.251+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4005430 con 0x7f2ba4003b30 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.251+0000 7f2bc2000640 1 -- 172.21.15.91:0/204559171 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4003810 con 0x7f2ba4003b30 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 -- 172.21.15.91:0/204559171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ba4003b30 msgr2=0x7f2ba4003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 --2- 172.21.15.91:0/204559171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ba4003b30 0x7f2ba4003f30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018a0 tx=0x7f2ba40057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc3400640 1 -- 172.21.15.91:0/204559171 reap_dead start 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 -- 172.21.15.91:0/204559171 shutdown_connections 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 -- 172.21.15.91:0/204559171 >> 172.21.15.91:0/204559171 conn(0x7f2bc40cec80 msgr2=0x7f2bc40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 -- 172.21.15.91:0/204559171 shutdown_connections 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.252+0000 7f2bc9800640 1 -- 172.21.15.91:0/204559171 wait complete. 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.254+0000 7f2bc9800640 1 Processor -- start 2024-02-02T13:29:13.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.254+0000 7f2bc9800640 1 -- start start 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba4003b30 0x7f2bc40c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bc415bc00 0x7f2bc4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40c5580 con 0x7f2bc4145a30 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc415e550 con 0x7f2bc415bc00 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc9800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc415d4b0 con 0x7f2ba4003b30 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58446/0 (socket says 172.21.15.91:58446) 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 -- 172.21.15.91:0/3196603952 learned_addr learned my addr 172.21.15.91:0/3196603952 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc3400640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bc415bc00 0x7f2bc4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba4003b30 0x7f2bc40c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 -- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba4003b30 msgr2=0x7f2bc40c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba4003b30 0x7f2bc40c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 -- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bc415bc00 msgr2=0x7f2bc4154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bc415bc00 0x7f2bc4154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.255+0000 7f2bbbe00640 1 -- 172.21.15.91:0/3196603952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc415d790 con 0x7f2bc4145a30 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bbbe00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d7f0 tx=0x7f2bac00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0074e0 con 0x7f2bc4145a30 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac007640 con 0x7f2bc4145a30 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0049e0 con 0x7f2bc4145a30 2024-02-02T13:29:13.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc415da70 con 0x7f2bc4145a30 2024-02-02T13:29:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.256+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc415e030 con 0x7f2bc4145a30 2024-02-02T13:29:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.257+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc415e1f0 con 0x7f2bc4145a30 2024-02-02T13:29:13.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.258+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2bac004b40 con 0x7f2bc4145a30 2024-02-02T13:29:13.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.259+0000 7f2bbaa00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 0x7f2b9c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:13.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.259+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bac088200 con 0x7f2bc4145a30 2024-02-02T13:29:13.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.259+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 0x7f2b9c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:13.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.260+0000 7f2bc2a00640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 0x7f2b9c06c920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018a0 tx=0x7f2bb4031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:13.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.263+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2bac0537f0 con 0x7f2bc4145a30 2024-02-02T13:29:13.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.493+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2bc4066cf0 con 0x7f2bc4145a30 2024-02-02T13:29:13.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.493+0000 7f2bbaa00640 1 -- 172.21.15.91:0/3196603952 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2bac019060 con 0x7f2bc4145a30 2024-02-02T13:29:13.495 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 msgr2=0x7f2b9c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 0x7f2b9c06c920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018a0 tx=0x7f2bb4031000 comp rx=0 tx=0).stop 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 msgr2=0x7f2bc415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d7f0 tx=0x7f2bac00dbf0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 shutdown_connections 2024-02-02T13:29:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b9c06a460 0x7f2b9c06c920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bc415bc00 0x7f2bc4154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bc4145a30 0x7f2bc415b6c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 --2- 172.21.15.91:0/3196603952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba4003b30 0x7f2bc40c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.496+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 >> 172.21.15.91:0/3196603952 conn(0x7f2bc40cec80 msgr2=0x7f2bc40cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.497+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 shutdown_connections 2024-02-02T13:29:13.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:13.497+0000 7f2bc9800640 1 -- 172.21.15.91:0/3196603952 wait complete. 2024-02-02T13:29:13.515 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:29:13.516 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:29:13.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.516 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:29:13.573 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:29:13.573 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:29:13.573 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:29:13.573 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-02-02T13:29:13.574 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:29:13.575 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:29:13.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.576 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:29:13.663 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:29:13.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.667 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:29:13.720 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:29:13.720 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:29:13.720 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:29:13.720 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:29:13.720 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:29:13.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:29:13.722 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:29:13.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.900 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:29:13.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:29:13.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:29:13.949 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:29:13.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:13.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:29:14.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:29:14.043 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:29:14.095 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:29:14.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:29:14.096 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:29:14.145 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:29:14.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:14.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:29:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:29:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:29:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:29:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:29:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:29:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:29:14.320 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:29:14.320 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:29:14.320 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:29:14.320 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:29:14.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:29:14.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:14.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:29:14.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:14.405 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:29:14.458 INFO:teuthology.orchestra.run.smithi113.stdout:client6486 2024-02-02T13:29:14.458 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:29:14.458 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:14.508 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:29:14.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.721+0000 7fedcec00640 1 Processor -- start 2024-02-02T13:29:14.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.722+0000 7fedcec00640 1 -- start start 2024-02-02T13:29:14.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.723+0000 7fedcec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.723+0000 7fedcec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fedc80b71f0 0x7fedc80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.723+0000 7fedcec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80b5520 con 0x7fedc80b4ba0 2024-02-02T13:29:14.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.723+0000 7fedcec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80b85e0 con 0x7fedc80b71f0 2024-02-02T13:29:14.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.723+0000 7fedcec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80b9140 con 0x7fedc8160a20 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.724+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.724+0000 7fedccc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fedc80b71f0 0x7fedc80b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.724+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:58242/0 (socket says 172.21.15.91:58242) 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.724+0000 7fedc7e00640 1 -- 172.21.15.91:0/423718792 learned_addr learned my addr 172.21.15.91:0/423718792 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.725+0000 7fedccc00640 1 -- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 msgr2=0x7fedc80b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:14.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.725+0000 7fedccc00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.725+0000 7fedccc00640 1 -- 172.21.15.91:0/423718792 >> v1:172.21.15.113:6789/0 conn(0x7fedc80b4ba0 legacy=0x7fedc80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.725+0000 7fedccc00640 1 -- 172.21.15.91:0/423718792 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80b7960 con 0x7fedc80b71f0 2024-02-02T13:29:14.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.725+0000 7fedc7e00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:29:14.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.726+0000 7fedccc00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc80b71f0 0x7fedc80b4fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009dc0 tx=0x7fedc002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=322ce432f9c0c682 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:14.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.727+0000 7fedc7400640 1 -- 172.21.15.91:0/423718792 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc002cbb0 con 0x7fedc80b71f0 2024-02-02T13:29:14.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.727+0000 7fedc7400640 1 -- 172.21.15.91:0/423718792 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc002cd10 con 0x7fedc80b71f0 2024-02-02T13:29:14.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.727+0000 7fedc7400640 1 -- 172.21.15.91:0/423718792 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0034660 con 0x7fedc80b71f0 2024-02-02T13:29:14.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.727+0000 7fedcec00640 1 -- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc80b71f0 msgr2=0x7fedc80b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.727+0000 7fedcec00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc80b71f0 0x7fedc80b4fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009dc0 tx=0x7fedc002be00 comp rx=0 tx=0).stop 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 -- 172.21.15.91:0/423718792 shutdown_connections 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc80b71f0 0x7fedc80b4fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 --2- 172.21.15.91:0/423718792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc8160a20 0x7fedc80b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 -- 172.21.15.91:0/423718792 >> 172.21.15.91:0/423718792 conn(0x7fedc8070480 msgr2=0x7fedc8070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 -- 172.21.15.91:0/423718792 shutdown_connections 2024-02-02T13:29:14.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.728+0000 7fedcec00640 1 -- 172.21.15.91:0/423718792 wait complete. 2024-02-02T13:29:14.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.729+0000 7fedcec00640 1 Processor -- start 2024-02-02T13:29:14.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 -- start start 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc80b71f0 0x7fedc80d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 0x7fedc80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80aa290 con 0x7fedc80b4ba0 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80ac5d0 con 0x7fedc8160a20 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.730+0000 7fedcec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80cc230 con 0x7fedc80b71f0 2024-02-02T13:29:14.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58460/0 (socket says 172.21.15.91:58460) 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 -- 172.21.15.91:0/284408975 learned_addr learned my addr 172.21.15.91:0/284408975 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedccc00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 0x7fedc80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedbf200640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc80b71f0 0x7fedc80d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 -- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc80b71f0 msgr2=0x7fedc80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc80b71f0 0x7fedc80d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 -- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 msgr2=0x7fedc80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 0x7fedc80d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedc7e00640 1 -- 172.21.15.91:0/284408975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80094d0 con 0x7fedc80b4ba0 2024-02-02T13:29:14.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.731+0000 7fedccc00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 0x7fedc80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:14.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedc7e00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fedb800e9b0 tx=0x7fedb800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800c9a0 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800cb00 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800ce70 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc0009a70 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.732+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc80a9bd0 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.733+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80aa500 con 0x7fedc80b4ba0 2024-02-02T13:29:14.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.734+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fedb8013470 con 0x7fedc80b4ba0 2024-02-02T13:29:14.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.735+0000 7fedc4c00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 0x7feda006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:14.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.735+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedb8088f90 con 0x7fedc80b4ba0 2024-02-02T13:29:14.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.735+0000 7fedbf200640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 0x7feda006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:14.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.736+0000 7fedbf200640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 0x7feda006c8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fedc81f5f30 tx=0x7fedb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:14.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.740+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fedb80544e0 con 0x7fedc80b4ba0 2024-02-02T13:29:14.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.965+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc8066cf0 con 0x7fedc80b4ba0 2024-02-02T13:29:14.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.966+0000 7fedc4c00640 1 -- 172.21.15.91:0/284408975 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fedb8057b00 con 0x7fedc80b4ba0 2024-02-02T13:29:14.968 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:14.969 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:29:14.969 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:29:14.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.969+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 msgr2=0x7feda006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.969+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 0x7feda006c8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fedc81f5f30 tx=0x7fedb0009040 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.969+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 msgr2=0x7fedc81f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.969+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fedb800e9b0 tx=0x7fedb800ee70 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.969+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 shutdown_connections 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feda006a430 0x7feda006c8f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fedc80b71f0 0x7fedc80d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedc8160a20 0x7fedc80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 --2- 172.21.15.91:0/284408975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc80b4ba0 0x7fedc81f22a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 >> 172.21.15.91:0/284408975 conn(0x7fedc8070480 msgr2=0x7fedc80d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 shutdown_connections 2024-02-02T13:29:14.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:14.970+0000 7fedcec00640 1 -- 172.21.15.91:0/284408975 wait complete. 2024-02-02T13:29:14.990 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-02-02T13:29:15.205 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":6324,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.315817936,"mdsmap_epoch":40,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":15.582928236000001} 2024-02-02T13:29:15.205 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 6324, "mdsmap_epoch": 40, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 10.315817936, "state": "up:active", "uptime": 15.582928236, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:29:15.206 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:29:15.429 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:29:15.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:29:15.430 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-02-02T13:29:15.638 INFO:teuthology.orchestra.run.smithi091.stdout:{"mon_tick_interval":"5"} 2024-02-02T13:29:15.638 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-02-02T13:29:15.638 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2024-02-02T13:29:15.638 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:29:15.638 DEBUG:teuthology.orchestra.run.smithi091:> import time 2024-02-02T13:29:15.638 DEBUG:teuthology.orchestra.run.smithi091:> import datetime 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> # Write some buffered data through before going full, all should be well 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> print("writing some data through which we expect to succeed") 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> bytes = 0 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> # buffered IO, the error should always only appear from fsync and not 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> # from write 2024-02-02T13:29:15.639 DEBUG:teuthology.orchestra.run.smithi091:> full = False 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(531 * 1.1)): 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote bytes via buffered write, moving on to fsync") 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> except OSError as e: 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> if False: 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> raise 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> full = True 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> break 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:29:15.640 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed successfully") 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> except OSError as e: 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> full = True 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> break 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> if n > 531 * 0.9: 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> # Be cautious in the last region where we expect to hit 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(15.0) 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> if not full: 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> # close() should not raise an error because we already caught it in 2024-02-02T13:29:15.641 DEBUG:teuthology.orchestra.run.smithi091:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> # since then because all IOs got cancelled on the full flag. 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> print("calling close") 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> print("close() did not raise error") 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-02-02T13:29:15.642 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:writing some data through which we expect to succeed 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 1.00 MB 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.560 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 2.00 MB 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 3.00 MB 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 4.00 MB 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 5.00 MB 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 6.00 MB 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.561 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 7.00 MB 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 8.00 MB 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 9.00 MB 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 10.00 MB 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 11.00 MB 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.562 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 12.00 MB 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 13.00 MB 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 14.00 MB 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 15.00 MB 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 16.00 MB 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 17.00 MB 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 18.00 MB 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 19.00 MB 2024-02-02T13:29:20.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 20.00 MB 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 21.00 MB 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 22.00 MB 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 23.00 MB 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 24.00 MB 2024-02-02T13:29:20.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 25.00 MB 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 26.00 MB 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 27.00 MB 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 28.00 MB 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.566 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 29.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 30.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 31.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 32.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 33.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 34.00 MB 2024-02-02T13:29:20.567 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 35.00 MB 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 36.00 MB 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 37.00 MB 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 38.00 MB 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.568 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 39.00 MB 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 40.00 MB 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 41.00 MB 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 42.00 MB 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 43.00 MB 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.569 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 44.00 MB 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 45.00 MB 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 46.00 MB 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 47.00 MB 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 48.00 MB 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.570 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 49.00 MB 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 50.00 MB 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 51.00 MB 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 52.00 MB 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 53.00 MB 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 54.00 MB 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 55.00 MB 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 56.00 MB 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 57.00 MB 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.572 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 58.00 MB 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 59.00 MB 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 60.00 MB 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 61.00 MB 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 62.00 MB 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 63.00 MB 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 64.00 MB 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 65.00 MB 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 66.00 MB 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 67.00 MB 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.574 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 68.00 MB 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 69.00 MB 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 70.00 MB 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 71.00 MB 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.575 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 72.00 MB 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 73.00 MB 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 74.00 MB 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 75.00 MB 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 76.00 MB 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.576 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 77.00 MB 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 78.00 MB 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 79.00 MB 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:20.577 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 80.00 MB 2024-02-02T13:29:21.374 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.374 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 81.00 MB 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 82.00 MB 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 83.00 MB 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 84.00 MB 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 85.00 MB 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.375 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 86.00 MB 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 87.00 MB 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 88.00 MB 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 89.00 MB 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 90.00 MB 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.376 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 91.00 MB 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 92.00 MB 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 93.00 MB 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 94.00 MB 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 95.00 MB 2024-02-02T13:29:21.377 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 96.00 MB 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 97.00 MB 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 98.00 MB 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 99.00 MB 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 100.00 MB 2024-02-02T13:29:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 101.00 MB 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 102.00 MB 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 103.00 MB 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 104.00 MB 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 105.00 MB 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.379 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 106.00 MB 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 107.00 MB 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 108.00 MB 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 109.00 MB 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.380 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 110.00 MB 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 111.00 MB 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 112.00 MB 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 113.00 MB 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.381 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 114.00 MB 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 115.00 MB 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 116.00 MB 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 117.00 MB 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 118.00 MB 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 119.00 MB 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 120.00 MB 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 121.00 MB 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 122.00 MB 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 123.00 MB 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.383 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 124.00 MB 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 125.00 MB 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 126.00 MB 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 127.00 MB 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.384 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 128.00 MB 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 129.00 MB 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 130.00 MB 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 131.00 MB 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.385 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 132.00 MB 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 133.00 MB 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 134.00 MB 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 135.00 MB 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 136.00 MB 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.386 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 137.00 MB 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 138.00 MB 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 139.00 MB 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 140.00 MB 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 141.00 MB 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.387 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 142.00 MB 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 143.00 MB 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 144.00 MB 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 145.00 MB 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 146.00 MB 2024-02-02T13:29:21.388 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 147.00 MB 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 148.00 MB 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 149.00 MB 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 150.00 MB 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.389 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 151.00 MB 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 152.00 MB 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 153.00 MB 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 154.00 MB 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 155.00 MB 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 156.00 MB 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 157.00 MB 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 158.00 MB 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 159.00 MB 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 160.00 MB 2024-02-02T13:29:21.391 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.142 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.142 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 161.00 MB 2024-02-02T13:29:22.142 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.142 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 162.00 MB 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 163.00 MB 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 164.00 MB 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 165.00 MB 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.143 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 166.00 MB 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 167.00 MB 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 168.00 MB 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 169.00 MB 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 170.00 MB 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.144 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 171.00 MB 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 172.00 MB 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 173.00 MB 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 174.00 MB 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.145 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 175.00 MB 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 176.00 MB 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 177.00 MB 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 178.00 MB 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 179.00 MB 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.146 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 180.00 MB 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 181.00 MB 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 182.00 MB 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 183.00 MB 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 184.00 MB 2024-02-02T13:29:22.147 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 185.00 MB 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 186.00 MB 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 187.00 MB 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 188.00 MB 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.148 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 189.00 MB 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 190.00 MB 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 191.00 MB 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 192.00 MB 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 193.00 MB 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.149 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 194.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 195.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 196.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 197.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 198.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 199.00 MB 2024-02-02T13:29:22.150 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 200.00 MB 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 201.00 MB 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 202.00 MB 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 203.00 MB 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.151 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 204.00 MB 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 205.00 MB 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 206.00 MB 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 207.00 MB 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 208.00 MB 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.152 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 209.00 MB 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 210.00 MB 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 211.00 MB 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 212.00 MB 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 213.00 MB 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.153 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 214.00 MB 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 215.00 MB 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 216.00 MB 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 217.00 MB 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 218.00 MB 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.154 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 219.00 MB 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 220.00 MB 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 221.00 MB 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 222.00 MB 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 223.00 MB 2024-02-02T13:29:22.155 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 224.00 MB 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 225.00 MB 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 226.00 MB 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 227.00 MB 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 228.00 MB 2024-02-02T13:29:22.156 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 229.00 MB 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 230.00 MB 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 231.00 MB 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 232.00 MB 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 233.00 MB 2024-02-02T13:29:22.157 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 234.00 MB 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 235.00 MB 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 236.00 MB 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 237.00 MB 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 238.00 MB 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 239.00 MB 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 240.00 MB 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.159 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.945 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 241.00 MB 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 242.00 MB 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 243.00 MB 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 244.00 MB 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 245.00 MB 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.946 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 246.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 247.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 248.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 249.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 250.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 251.00 MB 2024-02-02T13:29:22.947 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 252.00 MB 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 253.00 MB 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 254.00 MB 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 255.00 MB 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.948 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 256.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 257.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 258.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 259.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 260.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 261.00 MB 2024-02-02T13:29:22.949 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 262.00 MB 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 263.00 MB 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 264.00 MB 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 265.00 MB 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 266.00 MB 2024-02-02T13:29:22.950 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 267.00 MB 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 268.00 MB 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 269.00 MB 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 270.00 MB 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 271.00 MB 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.951 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 272.00 MB 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 273.00 MB 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 274.00 MB 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 275.00 MB 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 276.00 MB 2024-02-02T13:29:22.952 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 277.00 MB 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 278.00 MB 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 279.00 MB 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 280.00 MB 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 281.00 MB 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.953 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 282.00 MB 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 283.00 MB 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 284.00 MB 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 285.00 MB 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 286.00 MB 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.954 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 287.00 MB 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 288.00 MB 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 289.00 MB 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 290.00 MB 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 291.00 MB 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.955 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 292.00 MB 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 293.00 MB 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 294.00 MB 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 295.00 MB 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 296.00 MB 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.956 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 297.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 298.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 299.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 300.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 301.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 302.00 MB 2024-02-02T13:29:22.957 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 303.00 MB 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 304.00 MB 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 305.00 MB 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 306.00 MB 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 307.00 MB 2024-02-02T13:29:22.958 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 308.00 MB 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 309.00 MB 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 310.00 MB 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 311.00 MB 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 312.00 MB 2024-02-02T13:29:22.959 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 313.00 MB 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 314.00 MB 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 315.00 MB 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 316.00 MB 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 317.00 MB 2024-02-02T13:29:22.960 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 318.00 MB 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 319.00 MB 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 320.00 MB 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 321.00 MB 2024-02-02T13:29:23.733 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 322.00 MB 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 323.00 MB 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 324.00 MB 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 325.00 MB 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 326.00 MB 2024-02-02T13:29:23.734 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 327.00 MB 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 328.00 MB 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 329.00 MB 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 330.00 MB 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 331.00 MB 2024-02-02T13:29:23.735 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 332.00 MB 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 333.00 MB 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 334.00 MB 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 335.00 MB 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 336.00 MB 2024-02-02T13:29:23.736 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 337.00 MB 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 338.00 MB 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 339.00 MB 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 340.00 MB 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.737 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 341.00 MB 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 342.00 MB 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 343.00 MB 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 344.00 MB 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 345.00 MB 2024-02-02T13:29:23.738 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 346.00 MB 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 347.00 MB 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 348.00 MB 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 349.00 MB 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 350.00 MB 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.739 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 351.00 MB 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 352.00 MB 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 353.00 MB 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 354.00 MB 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 355.00 MB 2024-02-02T13:29:23.740 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 356.00 MB 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 357.00 MB 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 358.00 MB 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 359.00 MB 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 360.00 MB 2024-02-02T13:29:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 361.00 MB 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 362.00 MB 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 363.00 MB 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 364.00 MB 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 365.00 MB 2024-02-02T13:29:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 366.00 MB 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 367.00 MB 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 368.00 MB 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 369.00 MB 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 370.00 MB 2024-02-02T13:29:23.743 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 371.00 MB 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 372.00 MB 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 373.00 MB 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 374.00 MB 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.744 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 375.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 376.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 377.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 378.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 379.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 380.00 MB 2024-02-02T13:29:23.745 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 381.00 MB 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 382.00 MB 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 383.00 MB 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 384.00 MB 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 385.00 MB 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 386.00 MB 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 387.00 MB 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 388.00 MB 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 389.00 MB 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 390.00 MB 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.747 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 391.00 MB 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 392.00 MB 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 393.00 MB 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 394.00 MB 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 395.00 MB 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 396.00 MB 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 397.00 MB 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 398.00 MB 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 399.00 MB 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 400.00 MB 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:23.750 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 401.00 MB 2024-02-02T13:29:26.405 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:29:26.403+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-02-02T13:29:30.077 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:29:30.086 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:29:32.445 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:29:32.443+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2024-02-02T13:29:34.345 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:29:34.344+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-02-02T13:29:54.586 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.586 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 402.00 MB 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 403.00 MB 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 404.00 MB 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 405.00 MB 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 406.00 MB 2024-02-02T13:29:54.587 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 407.00 MB 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 408.00 MB 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 409.00 MB 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 410.00 MB 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 411.00 MB 2024-02-02T13:29:54.588 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 412.00 MB 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 413.00 MB 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 414.00 MB 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 415.00 MB 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 416.00 MB 2024-02-02T13:29:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 417.00 MB 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 418.00 MB 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 419.00 MB 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 420.00 MB 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.590 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 421.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 422.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 423.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 424.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 425.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 426.00 MB 2024-02-02T13:29:54.591 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 427.00 MB 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 428.00 MB 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 429.00 MB 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 430.00 MB 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.592 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 431.00 MB 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 432.00 MB 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 433.00 MB 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 434.00 MB 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 435.00 MB 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.593 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 436.00 MB 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 437.00 MB 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 438.00 MB 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 439.00 MB 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 440.00 MB 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.594 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 441.00 MB 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 442.00 MB 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 443.00 MB 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 444.00 MB 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 445.00 MB 2024-02-02T13:29:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 446.00 MB 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 447.00 MB 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 448.00 MB 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 449.00 MB 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 450.00 MB 2024-02-02T13:29:54.596 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 451.00 MB 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 452.00 MB 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 453.00 MB 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 454.00 MB 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 455.00 MB 2024-02-02T13:29:54.597 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 456.00 MB 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 457.00 MB 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 458.00 MB 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 459.00 MB 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 460.00 MB 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.598 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 461.00 MB 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 462.00 MB 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 463.00 MB 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 464.00 MB 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 465.00 MB 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.599 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 466.00 MB 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 467.00 MB 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 468.00 MB 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 469.00 MB 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 470.00 MB 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.600 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 471.00 MB 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 472.00 MB 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 473.00 MB 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 474.00 MB 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 475.00 MB 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.601 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 476.00 MB 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 477.00 MB 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 478.00 MB 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 479.00 MB 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 480.00 MB 2024-02-02T13:29:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:29:54.720 INFO:teuthology.orchestra.run.smithi091.stdout:Reached fullness after 480.00 MB 2024-02-02T13:29:54.720 INFO:teuthology.orchestra.run.smithi091.stdout:calling close 2024-02-02T13:29:54.720 INFO:teuthology.orchestra.run.smithi091.stdout:close() did not raise error 2024-02-02T13:29:54.833 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:29:54.833 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:29:54.956 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:29:55.074 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:29:55.074 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:29:55.074 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:55.133 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:29:55.133 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:55.187 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:55.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.394+0000 7f717c800640 1 Processor -- start 2024-02-02T13:29:55.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.394+0000 7f717c800640 1 -- start start 2024-02-02T13:29:55.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.395+0000 7f717c800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71740b0e10 0x7f71740c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717c800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71740cc7c0 con 0x7f71740ce130 2024-02-02T13:29:55.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717c800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71740ce930 con 0x7f717415a010 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717c800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71740cd640 con 0x7f71740b0e10 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71740b0e10 0x7f71740c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71740b0e10 0x7f71740c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54800/0 (socket says 172.21.15.91:54800) 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 -- 172.21.15.91:0/767469703 learned_addr learned my addr 172.21.15.91:0/767469703 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f7179400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f71740ce130 0x7f71740b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35488/0 (socket says 172.21.15.91:35488) 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 -- 172.21.15.91:0/767469703 >> v1:172.21.15.113:6789/0 conn(0x7f717415a010 legacy=0x7f71740ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 -- 172.21.15.91:0/767469703 >> v1:172.21.15.91:6789/0 conn(0x7f71740ce130 legacy=0x7f71740b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.396+0000 7f717a800640 1 -- 172.21.15.91:0/767469703 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717406c890 con 0x7f71740b0e10 2024-02-02T13:29:55.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.398+0000 7f717a800640 1 --2- 172.21.15.91:0/767469703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740b0e10 0x7f71740c9670 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f716c009e50 tx=0x7f716c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65208aae6260d045 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.398+0000 7f7178a00640 1 -- 172.21.15.91:0/767469703 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c02ca80 con 0x7f71740b0e10 2024-02-02T13:29:55.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.398+0000 7f7178a00640 1 -- 172.21.15.91:0/767469703 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c02cbe0 con 0x7f71740b0e10 2024-02-02T13:29:55.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.398+0000 7f7178a00640 1 -- 172.21.15.91:0/767469703 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0343f0 con 0x7f71740b0e10 2024-02-02T13:29:55.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 -- 172.21.15.91:0/767469703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740b0e10 msgr2=0x7f71740c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 --2- 172.21.15.91:0/767469703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740b0e10 0x7f71740c9670 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f716c009e50 tx=0x7f716c02bee0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 -- 172.21.15.91:0/767469703 shutdown_connections 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 --2- 172.21.15.91:0/767469703 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740b0e10 0x7f71740c9670 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 -- 172.21.15.91:0/767469703 >> 172.21.15.91:0/767469703 conn(0x7f71740cec80 msgr2=0x7f71740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 -- 172.21.15.91:0/767469703 shutdown_connections 2024-02-02T13:29:55.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.399+0000 7f717c800640 1 -- 172.21.15.91:0/767469703 wait complete. 2024-02-02T13:29:55.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.401+0000 7f717c800640 1 Processor -- start 2024-02-02T13:29:55.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.401+0000 7f717c800640 1 -- start start 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71740b0e10 0x7f717414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f717415a010 0x7f71741e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741590d0 con 0x7f71740ce130 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71740c2430 con 0x7f717415a010 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717c800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740c26d0 con 0x7f71740b0e10 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f7179400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f7179400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54810/0 (socket says 172.21.15.91:54810) 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f7179400640 1 -- 172.21.15.91:0/857519276 learned_addr learned my addr 172.21.15.91:0/857519276 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f7179e00640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71740b0e10 0x7f717414a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.402+0000 7f717a800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f717415a010 0x7f71741e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 -- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71740b0e10 msgr2=0x7f717414a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71740b0e10 0x7f717414a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 -- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f717415a010 msgr2=0x7f71741e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f717415a010 0x7f71741e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c009b30 con 0x7f71740ce130 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7179400640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c4d0 tx=0x7f715c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c007480 con 0x7f71740ce130 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c0075e0 con 0x7f71740ce130 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c0049e0 con 0x7f71740ce130 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71740c29b0 con 0x7f71740ce130 2024-02-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.403+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717414a610 con 0x7f71740ce130 2024-02-02T13:29:55.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.405+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71740c30b0 con 0x7f71740ce130 2024-02-02T13:29:55.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.405+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f715c004b40 con 0x7f71740ce130 2024-02-02T13:29:55.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.406+0000 7f7172000640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 0x7f714c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.406+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f715c087aa0 con 0x7f71740ce130 2024-02-02T13:29:55.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.407+0000 7f7179e00640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 0x7f714c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.407+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f714c072020 con 0x7f71740ce130 2024-02-02T13:29:55.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.407+0000 7f7179e00640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 0x7f714c06c8f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7164006fd0 tx=0x7f7164008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.411+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f715c018070 con 0x7f71740ce130 2024-02-02T13:29:55.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.604+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71740c2ed0 con 0x7f71740ce130 2024-02-02T13:29:55.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.606+0000 7f7172000640 1 -- 172.21.15.91:0/857519276 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16241 (secure 0 0 0) 0x7f71740c2ed0 con 0x7f71740ce130 2024-02-02T13:29:55.608 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:55.608 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":90,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:28.426395+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:55.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.608+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 msgr2=0x7f714c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.608+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 0x7f714c06c8f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7164006fd0 tx=0x7f7164008040 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.608+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 msgr2=0x7f71741e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.608+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c4d0 tx=0x7f715c00c990 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 shutdown_connections 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f714c06a430 0x7f714c06c8f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f717415a010 0x7f71741e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71740ce130 0x7f71741e6b60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 --2- 172.21.15.91:0/857519276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71740b0e10 0x7f717414a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 >> 172.21.15.91:0/857519276 conn(0x7f71740cec80 msgr2=0x7f717406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 shutdown_connections 2024-02-02T13:29:55.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.609+0000 7f717c800640 1 -- 172.21.15.91:0/857519276 wait complete. 2024-02-02T13:29:55.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:29:02.962375+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:29:03.885489+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:29:55.630 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:29:55.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.841+0000 7f3e9bc00640 1 Processor -- start 2024-02-02T13:29:55.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.841+0000 7f3e9bc00640 1 -- start start 2024-02-02T13:29:55.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e9bc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e9415a010 0x7f3e940c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e9bc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e940cc7c0 con 0x7f3e940b0700 2024-02-02T13:29:55.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e9bc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e940ce930 con 0x7f3e940ce130 2024-02-02T13:29:55.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e9bc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e940cd640 con 0x7f3e9415a010 2024-02-02T13:29:55.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e99c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3e940b0700 0x7f3e9406ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35498/0 (socket says 172.21.15.91:35498) 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.843+0000 7f3e99c00640 1 -- 172.21.15.91:0/1942733438 learned_addr learned my addr 172.21.15.91:0/1942733438 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e99200640 1 --2- 172.21.15.91:0/1942733438 >> v2:172.21.15.113:3300/0 conn(0x7f3e9415a010 0x7f3e940c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403887103 0 0) 0x7f3e940cc7c0 con 0x7f3e940b0700 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e74003520 con 0x7f3e940b0700 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3981674957 0 0) 0x7f3e74003520 con 0x7f3e940b0700 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 >> v2:172.21.15.113:3300/0 conn(0x7f3e9415a010 msgr2=0x7f3e940c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 --2- 172.21.15.91:0/1942733438 >> v2:172.21.15.113:3300/0 conn(0x7f3e9415a010 0x7f3e940c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 >> v1:172.21.15.113:6789/0 conn(0x7f3e940ce130 legacy=0x7f3e940ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9406c890 con 0x7f3e940b0700 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3e8c001670 con 0x7f3e940b0700 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 >> v1:172.21.15.91:6789/0 conn(0x7f3e940b0700 legacy=0x7f3e9406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e74003970 0x7f3e74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e74005180 0x7f3e7400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940cd640 con 0x7f3e740047c0 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400e230 con 0x7f3e74003970 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.844+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400e3f0 con 0x7f3e74005180 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e99200640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e74003970 0x7f3e74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 -- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e74005180 msgr2=0x7f3e7400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e74005180 0x7f3e7400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 -- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e74003970 msgr2=0x7f3e74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e74003970 0x7f3e74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e8bc00640 1 -- 172.21.15.91:0/1942733438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74003520 con 0x7f3e740047c0 2024-02-02T13:29:55.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.845+0000 7f3e99c00640 1 -- 172.21.15.91:0/1942733438 reap_dead start 2024-02-02T13:29:55.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.846+0000 7f3e8bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001750 tx=0x7f3e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a8f1dd14c1567 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.846+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c0052d0 con 0x7f3e740047c0 2024-02-02T13:29:55.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.846+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c02e050 con 0x7f3e740047c0 2024-02-02T13:29:55.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.846+0000 7f3e93e00640 1 -- 172.21.15.91:0/1942733438 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c003830 con 0x7f3e740047c0 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 -- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 msgr2=0x7f3e74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001750 tx=0x7f3e740057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 -- 172.21.15.91:0/1942733438 shutdown_connections 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/1942733438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 -- 172.21.15.91:0/1942733438 >> 172.21.15.91:0/1942733438 conn(0x7f3e940cec80 msgr2=0x7f3e940cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 -- 172.21.15.91:0/1942733438 shutdown_connections 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.847+0000 7f3e9bc00640 1 -- 172.21.15.91:0/1942733438 wait complete. 2024-02-02T13:29:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.849+0000 7f3e9bc00640 1 Processor -- start 2024-02-02T13:29:55.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.849+0000 7f3e9bc00640 1 -- start start 2024-02-02T13:29:55.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e740047c0 0x7f3e940b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e940ae950 0x7f3e941eda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e94148c00 con 0x7f3e940aedd0 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9414ca30 con 0x7f3e740047c0 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e9bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941ee500 con 0x7f3e940ae950 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e99c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e99c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54834/0 (socket says 172.21.15.91:54834) 2024-02-02T13:29:55.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.850+0000 7f3e99c00640 1 -- 172.21.15.91:0/2249265933 learned_addr learned my addr 172.21.15.91:0/2249265933 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:55.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e8bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e940ae950 0x7f3e941eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99200640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e740047c0 0x7f3e940b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 -- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e940ae950 msgr2=0x7f3e941eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e940ae950 0x7f3e941eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 -- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e740047c0 msgr2=0x7f3e940b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e740047c0 0x7f3e940b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94148e10 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e99c00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e940cdc50 tx=0x7f3e8c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0074e0 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c007640 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c004a20 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.851+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e941490f0 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.852+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e941496b0 con 0x7f3e940aedd0 2024-02-02T13:29:55.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.853+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e94066cf0 con 0x7f3e940aedd0 2024-02-02T13:29:55.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.854+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e8c004b80 con 0x7f3e940aedd0 2024-02-02T13:29:55.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.854+0000 7f3e91600640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 0x7f3e6c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:55.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.855+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3e8c087af0 con 0x7f3e940aedd0 2024-02-02T13:29:55.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.855+0000 7f3e99200640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 0x7f3e6c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:55.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.855+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f3e6c072050 con 0x7f3e940aedd0 2024-02-02T13:29:55.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.855+0000 7f3e99200640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 0x7f3e6c06c920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007520 tx=0x7f3e8400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:55.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:55.859+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e8c0530c0 con 0x7f3e940aedd0 2024-02-02T13:29:56.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.046+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e940b4c00 con 0x7f3e940aedd0 2024-02-02T13:29:56.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.047+0000 7f3e91600640 1 -- 172.21.15.91:0/2249265933 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e8c0566e0 con 0x7f3e940aedd0 2024-02-02T13:29:56.048 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:29:56.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 msgr2=0x7f3e6c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 0x7f3e6c06c920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007520 tx=0x7f3e8400b040 comp rx=0 tx=0).stop 2024-02-02T13:29:56.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 msgr2=0x7f3e941edfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e940cdc50 tx=0x7f3e8c00c7a0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 shutdown_connections 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e6c06a460 0x7f3e6c06c920 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e940ae950 0x7f3e941eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e940aedd0 0x7f3e941edfc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 --2- 172.21.15.91:0/2249265933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e740047c0 0x7f3e940b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.049+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 >> 172.21.15.91:0/2249265933 conn(0x7f3e940cec80 msgr2=0x7f3e9415e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.050+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 shutdown_connections 2024-02-02T13:29:56.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.050+0000 7f3e9bc00640 1 -- 172.21.15.91:0/2249265933 wait complete. 2024-02-02T13:29:56.070 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:29:56.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.286+0000 7ff497e00640 1 Processor -- start 2024-02-02T13:29:56.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.286+0000 7ff497e00640 1 -- start start 2024-02-02T13:29:56.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff497e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff4900acfe0 0x7ff49006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff497e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff497e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49015d870 con 0x7ff49015d490 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff497e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49006bce0 con 0x7ff4900acfe0 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff497e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff490161220 con 0x7ff4900b0700 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff494a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff4900acfe0 0x7ff49006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff495400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff49015d490 0x7ff4900ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35514/0 (socket says 172.21.15.91:35514) 2024-02-02T13:29:56.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.288+0000 7ff495400640 1 -- 172.21.15.91:0/1259644161 learned_addr learned my addr 172.21.15.91:0/1259644161 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:56.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.289+0000 7ff495e00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.289+0000 7ff494a00640 1 -- 172.21.15.91:0/1259644161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 msgr2=0x7ff4900b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:56.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.289+0000 7ff494a00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.289+0000 7ff494a00640 1 -- 172.21.15.91:0/1259644161 >> v1:172.21.15.91:6789/0 conn(0x7ff49015d490 legacy=0x7ff4900ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.289+0000 7ff494a00640 1 -- 172.21.15.91:0/1259644161 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900b1070 con 0x7ff4900acfe0 2024-02-02T13:29:56.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.290+0000 7ff494a00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff49006c170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff4780097a0 tx=0x7ff47802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a2ddb278c0eac server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.290+0000 7ff48fe00640 1 -- 172.21.15.91:0/1259644161 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47802d040 con 0x7ff4900acfe0 2024-02-02T13:29:56.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.290+0000 7ff48fe00640 1 -- 172.21.15.91:0/1259644161 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478031810 con 0x7ff4900acfe0 2024-02-02T13:29:56.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.290+0000 7ff48fe00640 1 -- 172.21.15.91:0/1259644161 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478031ba0 con 0x7ff4900acfe0 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 -- 172.21.15.91:0/1259644161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 msgr2=0x7ff49006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff49006c170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff4780097a0 tx=0x7ff47802b8b0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 -- 172.21.15.91:0/1259644161 shutdown_connections 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 --2- 172.21.15.91:0/1259644161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff49006c170 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 -- 172.21.15.91:0/1259644161 >> 172.21.15.91:0/1259644161 conn(0x7ff4900cec80 msgr2=0x7ff4900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 -- 172.21.15.91:0/1259644161 shutdown_connections 2024-02-02T13:29:56.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.291+0000 7ff497e00640 1 -- 172.21.15.91:0/1259644161 wait complete. 2024-02-02T13:29:56.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.293+0000 7ff497e00640 1 Processor -- start 2024-02-02T13:29:56.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.293+0000 7ff497e00640 1 -- start start 2024-02-02T13:29:56.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49015d490 0x7ff4900b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c3e80 con 0x7ff4900acfe0 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c4c80 con 0x7ff49015d490 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff497e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c4070 con 0x7ff4900b0700 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff495400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff495400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54852/0 (socket says 172.21.15.91:54852) 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.294+0000 7ff495400640 1 -- 172.21.15.91:0/527362873 learned_addr learned my addr 172.21.15.91:0/527362873 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 -- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 msgr2=0x7ff4900c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 -- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49015d490 msgr2=0x7ff4900b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49015d490 0x7ff4900b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478009400 con 0x7ff4900acfe0 2024-02-02T13:29:56.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff495400640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c4a0 tx=0x7ff48000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480005ce0 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480005e40 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480007270 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4900c4350 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.295+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4900c4910 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.296+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490066cf0 con 0x7ff4900acfe0 2024-02-02T13:29:56.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.297+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4800073d0 con 0x7ff4900acfe0 2024-02-02T13:29:56.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.298+0000 7ff48d600640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 0x7ff46806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.298+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff480087ea0 con 0x7ff4900acfe0 2024-02-02T13:29:56.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.299+0000 7ff494a00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 0x7ff46806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.299+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7ff468072050 con 0x7ff4900acfe0 2024-02-02T13:29:56.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.299+0000 7ff494a00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 0x7ff46806c920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff478036000 tx=0x7ff478030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.304+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff480017060 con 0x7ff4900acfe0 2024-02-02T13:29:56.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.493+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4900b4c00 con 0x7ff4900acfe0 2024-02-02T13:29:56.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.494+0000 7ff48d600640 1 -- 172.21.15.91:0/527362873 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff480056ef0 con 0x7ff4900acfe0 2024-02-02T13:29:56.496 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:29:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 msgr2=0x7ff46806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 0x7ff46806c920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff478036000 tx=0x7ff478030000 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 msgr2=0x7ff4901f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c4a0 tx=0x7ff48000c960 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 shutdown_connections 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46806a460 0x7ff46806c920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4900b0700 0x7ff4900c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49015d490 0x7ff4900b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 --2- 172.21.15.91:0/527362873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4900acfe0 0x7ff4901f6780 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 >> 172.21.15.91:0/527362873 conn(0x7ff4900cec80 msgr2=0x7ff4901579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 shutdown_connections 2024-02-02T13:29:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.497+0000 7ff497e00640 1 -- 172.21.15.91:0/527362873 wait complete. 2024-02-02T13:29:56.518 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:29:56.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.729+0000 7f9a7d400640 1 Processor -- start 2024-02-02T13:29:56.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.729+0000 7f9a7d400640 1 -- start start 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.730+0000 7f9a7d400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a7815d490 0x7f9a7806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.730+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a780ac8d0 0x7f9a780acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.730+0000 7f9a7d400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7815d870 con 0x7f9a780b0700 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.730+0000 7f9a7d400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780ad270 con 0x7f9a7815d490 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.730+0000 7f9a7d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7806bce0 con 0x7f9a780ac8d0 2024-02-02T13:29:56.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a7815d490 0x7f9a7806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a7815d490 0x7f9a7806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54862/0 (socket says 172.21.15.91:54862) 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 -- 172.21.15.91:0/2039862020 learned_addr learned my addr 172.21.15.91:0/2039862020 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a77200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9a780b0700 0x7f9a780c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35522/0 (socket says 172.21.15.91:35522) 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 -- 172.21.15.91:0/2039862020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a780ac8d0 msgr2=0x7f9a780acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 --2- 172.21.15.91:0/2039862020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a780ac8d0 0x7f9a780acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 -- 172.21.15.91:0/2039862020 >> v1:172.21.15.91:6789/0 conn(0x7f9a780b0700 legacy=0x7f9a780c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.731+0000 7f9a76800640 1 -- 172.21.15.91:0/2039862020 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78161220 con 0x7f9a7815d490 2024-02-02T13:29:56.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.732+0000 7f9a76800640 1 --2- 172.21.15.91:0/2039862020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a7815d490 0x7f9a7806c170 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009870 tx=0x7f9a6802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619148bb8c3ecca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.732+0000 7f9a75e00640 1 -- 172.21.15.91:0/2039862020 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68030840 con 0x7f9a7815d490 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.732+0000 7f9a75e00640 1 -- 172.21.15.91:0/2039862020 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a680309a0 con 0x7f9a7815d490 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.732+0000 7f9a75e00640 1 -- 172.21.15.91:0/2039862020 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68030d30 con 0x7f9a7815d490 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 -- 172.21.15.91:0/2039862020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a7815d490 msgr2=0x7f9a7806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 --2- 172.21.15.91:0/2039862020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a7815d490 0x7f9a7806c170 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009870 tx=0x7f9a6802b980 comp rx=0 tx=0).stop 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 -- 172.21.15.91:0/2039862020 shutdown_connections 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 --2- 172.21.15.91:0/2039862020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a780ac8d0 0x7f9a780acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 --2- 172.21.15.91:0/2039862020 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a7815d490 0x7f9a7806c170 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 -- 172.21.15.91:0/2039862020 >> 172.21.15.91:0/2039862020 conn(0x7f9a780cec80 msgr2=0x7f9a780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:56.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 -- 172.21.15.91:0/2039862020 shutdown_connections 2024-02-02T13:29:56.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.733+0000 7f9a7d400640 1 -- 172.21.15.91:0/2039862020 wait complete. 2024-02-02T13:29:56.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.735+0000 7f9a7d400640 1 Processor -- start 2024-02-02T13:29:56.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.735+0000 7f9a7d400640 1 -- start start 2024-02-02T13:29:56.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 0x7f9a7814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 0x7f9a780cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780c3d80 con 0x7f9a780b0700 2024-02-02T13:29:56.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7814e6a0 con 0x7f9a780ac8d0 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a7d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78071430 con 0x7f9a7815d490 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a6dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a6dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54878/0 (socket says 172.21.15.91:54878) 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.736+0000 7f9a6dc00640 1 -- 172.21.15.91:0/643081358 learned_addr learned my addr 172.21.15.91:0/643081358 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a77200640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 0x7f9a780cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a76800640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 0x7f9a7814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 -- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 msgr2=0x7f9a780cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 0x7f9a780cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 -- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 msgr2=0x7f9a7814e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 0x7f9a7814e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680094d0 con 0x7f9a780b0700 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a76800640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 0x7f9a7814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a77200640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 0x7f9a780cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:56.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6dc00640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b410 tx=0x7f9a6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011830 con 0x7f9a780b0700 2024-02-02T13:29:56.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600119b0 con 0x7f9a780b0700 2024-02-02T13:29:56.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011d60 con 0x7f9a780b0700 2024-02-02T13:29:56.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.737+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a780c3450 con 0x7f9a780b0700 2024-02-02T13:29:56.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.738+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780c3a10 con 0x7f9a780b0700 2024-02-02T13:29:56.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.739+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a780c3bd0 con 0x7f9a780b0700 2024-02-02T13:29:56.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.739+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a6001a400 con 0x7f9a780b0700 2024-02-02T13:29:56.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.740+0000 7f9a6f400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 0x7f9a5006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:56.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.740+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9a60088b30 con 0x7f9a780b0700 2024-02-02T13:29:56.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.741+0000 7f9a76800640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 0x7f9a5006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:56.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.741+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f9a50072050 con 0x7f9a780b0700 2024-02-02T13:29:56.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.741+0000 7f9a76800640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 0x7f9a5006c920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a6803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:56.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.745+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9a60015020 con 0x7f9a780b0700 2024-02-02T13:29:56.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.932+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a78066cf0 con 0x7f9a780b0700 2024-02-02T13:29:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.933+0000 7f9a6f400640 1 -- 172.21.15.91:0/643081358 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a60054040 con 0x7f9a780b0700 2024-02-02T13:29:56.934 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 msgr2=0x7f9a5006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 0x7f9a5006c920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a6803b040 comp rx=0 tx=0).stop 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 msgr2=0x7f9a780ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b410 tx=0x7f9a6000b8d0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 shutdown_connections 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a5006a460 0x7f9a5006c920 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a7815d490 0x7f9a780cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a780b0700 0x7f9a780ca880 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 --2- 172.21.15.91:0/643081358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780ac8d0 0x7f9a7814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:56.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.935+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 >> 172.21.15.91:0/643081358 conn(0x7f9a780cec80 msgr2=0x7f9a781613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:56.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.936+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 shutdown_connections 2024-02-02T13:29:56.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:56.936+0000 7f9a7d400640 1 -- 172.21.15.91:0/643081358 wait complete. 2024-02-02T13:29:56.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:29:57.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.167+0000 7f6d0f400640 1 Processor -- start 2024-02-02T13:29:57.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.168+0000 7f6d0f400640 1 -- start start 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6d080b8750 0x7f6d080cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d080b8bb0 0x7f6d080cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0f400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080cde20 con 0x7f6d0815d0d0 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0f400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080d2a20 con 0x7f6d080b8750 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0f400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0815d4b0 con 0x7f6d080b8bb0 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0e000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6d0815d0d0 0x7f6d080cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35532/0 (socket says 172.21.15.91:35532) 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0e000640 1 -- 172.21.15.91:0/515006504 learned_addr learned my addr 172.21.15.91:0/515006504 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:57.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.169+0000 7f6d0d600640 1 --2- 172.21.15.91:0/515006504 >> v2:172.21.15.91:3300/0 conn(0x7f6d080b8750 0x7f6d080cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0ea00640 1 --2- 172.21.15.91:0/515006504 >> v2:172.21.15.113:3300/0 conn(0x7f6d080b8bb0 0x7f6d080cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0d600640 1 -- 172.21.15.91:0/515006504 >> v2:172.21.15.113:3300/0 conn(0x7f6d080b8bb0 msgr2=0x7f6d080cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:57.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0d600640 1 --2- 172.21.15.91:0/515006504 >> v2:172.21.15.113:3300/0 conn(0x7f6d080b8bb0 0x7f6d080cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0d600640 1 -- 172.21.15.91:0/515006504 >> v1:172.21.15.91:6789/0 conn(0x7f6d0815d0d0 legacy=0x7f6d080cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0d600640 1 -- 172.21.15.91:0/515006504 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080d23f0 con 0x7f6d080b8750 2024-02-02T13:29:57.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.170+0000 7f6d0cc00640 1 -- 172.21.15.91:0/515006504 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922916622 0 0) 0x7f6d080cde20 con 0x7f6d0815d0d0 2024-02-02T13:29:57.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.171+0000 7f6d0d600640 1 --2- 172.21.15.91:0/515006504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d080cf810 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d0806a070 tx=0x7f6cf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adb1755404690f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.171+0000 7f6d0cc00640 1 -- 172.21.15.91:0/515006504 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0031830 con 0x7f6d080b8750 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.171+0000 7f6d0cc00640 1 -- 172.21.15.91:0/515006504 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0031990 con 0x7f6d080b8750 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.171+0000 7f6d0cc00640 1 -- 172.21.15.91:0/515006504 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0031d20 con 0x7f6d080b8750 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 -- 172.21.15.91:0/515006504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 msgr2=0x7f6d080cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 --2- 172.21.15.91:0/515006504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d080cf810 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d0806a070 tx=0x7f6cf002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 -- 172.21.15.91:0/515006504 shutdown_connections 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 --2- 172.21.15.91:0/515006504 >> v2:172.21.15.113:3300/0 conn(0x7f6d080b8bb0 0x7f6d080cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 --2- 172.21.15.91:0/515006504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d080cf810 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 -- 172.21.15.91:0/515006504 >> 172.21.15.91:0/515006504 conn(0x7f6d08065910 msgr2=0x7f6d08065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 -- 172.21.15.91:0/515006504 shutdown_connections 2024-02-02T13:29:57.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.172+0000 7f6d0f400640 1 -- 172.21.15.91:0/515006504 wait complete. 2024-02-02T13:29:57.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.174+0000 7f6d0f400640 1 Processor -- start 2024-02-02T13:29:57.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.174+0000 7f6d0f400640 1 -- start start 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 0x7f6d080c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d080c85d0 0x7f6d080c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080c7140 con 0x7f6d080b8750 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080c6ea0 con 0x7f6d0815d0d0 2024-02-02T13:29:57.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b45a0 con 0x7f6d080c85d0 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54888/0 (socket says 172.21.15.91:54888) 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.175+0000 7f6d0e000640 1 -- 172.21.15.91:0/1408767777 learned_addr learned my addr 172.21.15.91:0/1408767777 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0d600640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 0x7f6d080c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 -- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d080c85d0 msgr2=0x7f6d080c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0ea00640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d080c85d0 0x7f6d080c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d080c85d0 0x7f6d080c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 -- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 msgr2=0x7f6d080c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 0x7f6d080c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0009400 con 0x7f6d080b8750 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0d600640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 0x7f6d080c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:57.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0e000640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c4d0 tx=0x7f6d0000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000074e0 con 0x7f6d080b8750 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d00007660 con 0x7f6d080b8750 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004a20 con 0x7f6d080b8750 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080b4880 con 0x7f6d080b8750 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.176+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d080b4e40 con 0x7f6d080b8750 2024-02-02T13:29:57.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.177+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d080b5030 con 0x7f6d080b8750 2024-02-02T13:29:57.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.178+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6d00004b80 con 0x7f6d080b8750 2024-02-02T13:29:57.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.179+0000 7f6d06000640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 0x7f6ce006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.179+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6d00087c20 con 0x7f6d080b8750 2024-02-02T13:29:57.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.180+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f6ce0072020 con 0x7f6d080b8750 2024-02-02T13:29:57.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.180+0000 7f6d0d600640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 0x7f6ce006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.181+0000 7f6d0d600640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 0x7f6ce006c8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0036000 tx=0x7f6cf003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.184+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6d00018020 con 0x7f6d080b8750 2024-02-02T13:29:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.390+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d0804dff0 con 0x7f6d080b8750 2024-02-02T13:29:57.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.391+0000 7f6d06000640 1 -- 172.21.15.91:0/1408767777 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6d00053130 con 0x7f6d080b8750 2024-02-02T13:29:57.393 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:29:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.393+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 msgr2=0x7f6ce006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.393+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 0x7f6ce006c8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0036000 tx=0x7f6cf003c040 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.393+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 msgr2=0x7f6d0814c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.393+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c4d0 tx=0x7f6d0000c990 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 shutdown_connections 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6ce006a430 0x7f6ce006c8f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d080c85d0 0x7f6d080c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d0815d0d0 0x7f6d080c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 --2- 172.21.15.91:0/1408767777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6d080b8750 0x7f6d0814c680 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 >> 172.21.15.91:0/1408767777 conn(0x7f6d08065910 msgr2=0x7f6d080cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 shutdown_connections 2024-02-02T13:29:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.394+0000 7f6d0f400640 1 -- 172.21.15.91:0/1408767777 wait complete. 2024-02-02T13:29:57.414 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:29:57.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:57.415 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:29:57.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:29:57.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:57.516 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:29:57.566 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:29:57.566 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:57.622 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:29:57.622 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:29:57.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:29:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.884+0000 7f84b2200640 1 Processor -- start 2024-02-02T13:29:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.884+0000 7f84b2200640 1 -- start start 2024-02-02T13:29:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.885+0000 7f84b2200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84ac15d2c0 0x7f84ac06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.885+0000 7f84b2200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 0x7f84ac0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.885+0000 7f84b2200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac0ad2e0 con 0x7f84ac06c820 2024-02-02T13:29:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.885+0000 7f84b2200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ac0ad5c0 con 0x7f84ac15d2c0 2024-02-02T13:29:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.885+0000 7f84b2200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac06b4f0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.886+0000 7f84b0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84ac15d2c0 0x7f84ac06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.886+0000 7f84b1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 0x7f84ac0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.886+0000 7f84a3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f84ac06c820 0x7f84ac0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:38620/0 (socket says 172.21.15.91:38620) 2024-02-02T13:29:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.886+0000 7f84a3e00640 1 -- 172.21.15.91:0/2911470702 learned_addr learned my addr 172.21.15.91:0/2911470702 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.887+0000 7f84b1800640 1 -- 172.21.15.91:0/2911470702 >> v2:172.21.15.113:3300/0 conn(0x7f84ac15d2c0 msgr2=0x7f84ac06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.887+0000 7f84b1800640 1 --2- 172.21.15.91:0/2911470702 >> v2:172.21.15.113:3300/0 conn(0x7f84ac15d2c0 0x7f84ac06bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.887+0000 7f84b1800640 1 -- 172.21.15.91:0/2911470702 >> v1:172.21.15.113:6789/0 conn(0x7f84ac06c820 legacy=0x7f84ac0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.887+0000 7f84abe00640 1 -- 172.21.15.91:0/2911470702 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464291793 0 0) 0x7f84ac0ad2e0 con 0x7f84ac06c820 2024-02-02T13:29:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.887+0000 7f84b1800640 1 -- 172.21.15.91:0/2911470702 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac06b7d0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.889+0000 7f84b1800640 1 --2- 172.21.15.91:0/2911470702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 0x7f84ac0c90e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009d90 tx=0x7f84a402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76b31583f3f045b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.889+0000 7f84abe00640 1 -- 172.21.15.91:0/2911470702 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a402cc50 con 0x7f84ac0ccd70 2024-02-02T13:29:57.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.889+0000 7f84abe00640 1 -- 172.21.15.91:0/2911470702 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a402cdb0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.889+0000 7f84abe00640 1 -- 172.21.15.91:0/2911470702 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4034660 con 0x7f84ac0ccd70 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 -- 172.21.15.91:0/2911470702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 msgr2=0x7f84ac0c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 --2- 172.21.15.91:0/2911470702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 0x7f84ac0c90e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009d90 tx=0x7f84a402bea0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 -- 172.21.15.91:0/2911470702 shutdown_connections 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 --2- 172.21.15.91:0/2911470702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac0ccd70 0x7f84ac0c90e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 --2- 172.21.15.91:0/2911470702 >> v2:172.21.15.113:3300/0 conn(0x7f84ac15d2c0 0x7f84ac06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 -- 172.21.15.91:0/2911470702 >> 172.21.15.91:0/2911470702 conn(0x7f84ac0cea90 msgr2=0x7f84ac0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:57.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 -- 172.21.15.91:0/2911470702 shutdown_connections 2024-02-02T13:29:57.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.890+0000 7f84b2200640 1 -- 172.21.15.91:0/2911470702 wait complete. 2024-02-02T13:29:57.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.892+0000 7f84b2200640 1 Processor -- start 2024-02-02T13:29:57.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.892+0000 7f84b2200640 1 -- start start 2024-02-02T13:29:57.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac06c820 0x7f84ac0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac15d2c0 0x7f84ac157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac1f29f0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0c89d0 con 0x7f84ac06c820 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84b2200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0b2af0 con 0x7f84ac15d2c0 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54890/0 (socket says 172.21.15.91:54890) 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.893+0000 7f84a3e00640 1 -- 172.21.15.91:0/719189970 learned_addr learned my addr 172.21.15.91:0/719189970 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84b0e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac06c820 0x7f84ac0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84b1800640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac15d2c0 0x7f84ac157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 -- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac15d2c0 msgr2=0x7f84ac157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac15d2c0 0x7f84ac157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 -- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac06c820 msgr2=0x7f84ac0c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac06c820 0x7f84ac0c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009a70 con 0x7f84ac0ccd70 2024-02-02T13:29:57.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a3e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f849400c4d0 tx=0x7f849400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940074e0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8494007660 con 0x7f84ac0ccd70 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494004a20 con 0x7f84ac0ccd70 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.894+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac1f2bf0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.895+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac1f31b0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.896+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac1f33d0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.897+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8494004b80 con 0x7f84ac0ccd70 2024-02-02T13:29:57.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.897+0000 7f84a9600640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 0x7f848406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:57.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.898+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8494087bd0 con 0x7f84ac0ccd70 2024-02-02T13:29:57.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.898+0000 7f84b0e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 0x7f848406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:57.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.898+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f8484072020 con 0x7f84ac0ccd70 2024-02-02T13:29:57.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.898+0000 7f84b0e00640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 0x7f848406c8f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f84ac160dc0 tx=0x7f849c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:57.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:57.902+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8494018070 con 0x7f84ac0ccd70 2024-02-02T13:29:58.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.183+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84ac04dff0 con 0x7f84ac0ccd70 2024-02-02T13:29:58.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.184+0000 7f84a9600640 1 -- 172.21.15.91:0/719189970 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16241 (secure 0 0 0) 0x7f84940530e0 con 0x7f84ac0ccd70 2024-02-02T13:29:58.186 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:29:58.186 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":90,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:28.426395+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:29:58.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 msgr2=0x7f848406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 0x7f848406c8f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f84ac160dc0 tx=0x7f849c008040 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 msgr2=0x7f84ac156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f849400c4d0 tx=0x7f849400c990 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 shutdown_connections 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f848406a430 0x7f848406c8f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84ac15d2c0 0x7f84ac157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84ac0ccd70 0x7f84ac156ac0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 --2- 172.21.15.91:0/719189970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac06c820 0x7f84ac0c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 >> 172.21.15.91:0/719189970 conn(0x7f84ac0cea90 msgr2=0x7f84ac15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 shutdown_connections 2024-02-02T13:29:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.187+0000 7f84b2200640 1 -- 172.21.15.91:0/719189970 wait complete. 2024-02-02T13:29:58.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:29:02.962375+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:29:03.885489+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:29:58.207 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:29:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.389+0000 7fb63c400640 1 Processor -- start 2024-02-02T13:29:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.389+0000 7fb63c400640 1 -- start start 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb63c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb63c400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340cc7c0 con 0x7fb63415a010 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb63c400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340ce930 con 0x7fb6340b0700 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb63c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340cd640 con 0x7fb6340ce130 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb639a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb63415a010 0x7fb6340ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35548/0 (socket says 172.21.15.91:35548) 2024-02-02T13:29:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.390+0000 7fb639a00640 1 -- 172.21.15.91:0/3052796157 learned_addr learned my addr 172.21.15.91:0/3052796157 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164826096 0 0) 0x7fb6340cc7c0 con 0x7fb63415a010 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb614003520 con 0x7fb63415a010 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb639000640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3086817239 0 0) 0x7fb614003520 con 0x7fb63415a010 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 msgr2=0x7fb6340c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6340c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 >> v1:172.21.15.113:6789/0 conn(0x7fb6340b0700 legacy=0x7fb63406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63406c890 con 0x7fb63415a010 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb6240019f0 con 0x7fb63415a010 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 >> v1:172.21.15.91:6789/0 conn(0x7fb63415a010 legacy=0x7fb6340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.391+0000 7fb633e00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6140039a0 0x7fb614003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb633e00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb633e00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb614004d00 0x7fb614005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340cd640 con 0x7fb6140039a0 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61400bab0 con 0x7fb614004d00 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61400de80 con 0x7fb6340ce130 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6140039a0 0x7fb614003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639000640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb63a400640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb614004d00 0x7fb614005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 -- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 msgr2=0x7fb6140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ce130 0x7fb6140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 -- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb614004d00 msgr2=0x7fb614005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb614004d00 0x7fb614005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.392+0000 7fb639a00640 1 -- 172.21.15.91:0/3052796157 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614003520 con 0x7fb6140039a0 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.393+0000 7fb639a00640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6140039a0 0x7fb614003da0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb62400a830 tx=0x7fb6140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d843ff554f53fd8a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.393+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624034630 con 0x7fb6140039a0 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62402f070 con 0x7fb6140039a0 2024-02-02T13:29:58.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb633e00640 1 -- 172.21.15.91:0/3052796157 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240349f0 con 0x7fb6140039a0 2024-02-02T13:29:58.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 -- 172.21.15.91:0/3052796157 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6140039a0 msgr2=0x7fb614003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 --2- 172.21.15.91:0/3052796157 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6140039a0 0x7fb614003da0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb62400a830 tx=0x7fb6140057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63a400640 1 -- 172.21.15.91:0/3052796157 reap_dead start 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 -- 172.21.15.91:0/3052796157 shutdown_connections 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 -- 172.21.15.91:0/3052796157 >> 172.21.15.91:0/3052796157 conn(0x7fb6340cec80 msgr2=0x7fb6340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 -- 172.21.15.91:0/3052796157 shutdown_connections 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.394+0000 7fb63c400640 1 -- 172.21.15.91:0/3052796157 wait complete. 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.396+0000 7fb63c400640 1 Processor -- start 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.396+0000 7fb63c400640 1 -- start start 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6140039a0 0x7fb6341f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ad2e0 0x7fb6340ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63414f0a0 con 0x7fb6340c3af0 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340c44b0 con 0x7fb6140039a0 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340c2530 con 0x7fb6340ad2e0 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54908/0 (socket says 172.21.15.91:54908) 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.397+0000 7fb63a400640 1 -- 172.21.15.91:0/1537725183 learned_addr learned my addr 172.21.15.91:0/1537725183 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:58.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb639000640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ad2e0 0x7fb6340ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb639a00640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6140039a0 0x7fb6341f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 -- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ad2e0 msgr2=0x7fb6340ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ad2e0 0x7fb6340ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 -- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6140039a0 msgr2=0x7fb6341f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6140039a0 0x7fb6341f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340c2810 con 0x7fb6340c3af0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63a400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb6340c2320 tx=0x7fb62c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0125f0 con 0x7fb6340c3af0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c012750 con 0x7fb6340c3af0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6340c2af0 con 0x7fb6340c3af0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c012b00 con 0x7fb6340c3af0 2024-02-02T13:29:58.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.398+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6340c30b0 con 0x7fb6340c3af0 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.399+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634066cf0 con 0x7fb6340c3af0 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.400+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb62c012c60 con 0x7fb6340c3af0 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.401+0000 7fb631600640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 0x7fb60c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.401+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb62c0887e0 con 0x7fb6340c3af0 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.401+0000 7fb639a00640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 0x7fb60c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.402+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7fb60c072020 con 0x7fb6340c3af0 2024-02-02T13:29:58.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.402+0000 7fb639a00640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 0x7fb60c06c8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.406+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb62c053db0 con 0x7fb6340c3af0 2024-02-02T13:29:58.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.598+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6340b02d0 con 0x7fb6340c3af0 2024-02-02T13:29:58.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.599+0000 7fb631600640 1 -- 172.21.15.91:0/1537725183 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb62c016070 con 0x7fb6340c3af0 2024-02-02T13:29:58.601 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 msgr2=0x7fb60c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 0x7fb60c06c8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402f000 comp rx=0 tx=0).stop 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 msgr2=0x7fb6340c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb6340c2320 tx=0x7fb62c00d890 comp rx=0 tx=0).stop 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 shutdown_connections 2024-02-02T13:29:58.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb60c06a430 0x7fb60c06c8f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6340ad2e0 0x7fb6340ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6340c3af0 0x7fb6340c3f70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 --2- 172.21.15.91:0/1537725183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6140039a0 0x7fb6341f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 >> 172.21.15.91:0/1537725183 conn(0x7fb6340cec80 msgr2=0x7fb6340af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 shutdown_connections 2024-02-02T13:29:58.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.602+0000 7fb63c400640 1 -- 172.21.15.91:0/1537725183 wait complete. 2024-02-02T13:29:58.622 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:29:58.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.831+0000 7f0d6ea00640 1 Processor -- start 2024-02-02T13:29:58.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.831+0000 7f0d6ea00640 1 -- start start 2024-02-02T13:29:58.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d680b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d6ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d680b0c40 0x7f0d680b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d6ea00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6815fa00 con 0x7f0d68151f90 2024-02-02T13:29:58.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d6ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680b10d0 con 0x7f0d680b0c40 2024-02-02T13:29:58.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d6ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68070f80 con 0x7f0d6815f5a0 2024-02-02T13:29:58.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d67e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0d68151f90 0x7f0d6806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35550/0 (socket says 172.21.15.91:35550) 2024-02-02T13:29:58.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.833+0000 7f0d67e00640 1 -- 172.21.15.91:0/1278482055 learned_addr learned my addr 172.21.15.91:0/1278482055 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d67400640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d680b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d6ca00640 1 --2- 172.21.15.91:0/1278482055 >> v2:172.21.15.113:3300/0 conn(0x7f0d680b0c40 0x7f0d680b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037513471 0 0) 0x7f0d58001110 con 0x7f0d68151f90 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d48003520 con 0x7f0d68151f90 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023018749 0 0) 0x7f0d48003520 con 0x7f0d68151f90 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 msgr2=0x7f0d680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d680b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 >> v2:172.21.15.113:3300/0 conn(0x7f0d680b0c40 msgr2=0x7f0d680b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 --2- 172.21.15.91:0/1278482055 >> v2:172.21.15.113:3300/0 conn(0x7f0d680b0c40 0x7f0d680b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68152370 con 0x7f0d68151f90 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0d58001800 con 0x7f0d68151f90 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 >> v1:172.21.15.91:6789/0 conn(0x7f0d68151f90 legacy=0x7f0d6806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d48003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d48003ef0 0x7f0d480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d48004d00 0x7f0d48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680b10d0 con 0x7f0d48004d00 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68070f80 con 0x7f0d48003ef0 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.834+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4800b9d0 con 0x7f0d6815f5a0 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d48004d00 0x7f0d48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d67e00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d48003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d67400640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d48003ef0 0x7f0d480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 -- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 msgr2=0x7f0d48003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d48003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 -- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d48003ef0 msgr2=0x7f0d480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:58.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d48003ef0 0x7f0d480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d6ca00640 1 -- 172.21.15.91:0/1278482055 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48003520 con 0x7f0d48004d00 2024-02-02T13:29:58.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d67400640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d48003ef0 0x7f0d480047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:58.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.835+0000 7f0d67e00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815f5a0 0x7f0d48003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.837+0000 7f0d6ca00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d48004d00 0x7f0d48005180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0d60007360 tx=0x7f0d480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d543a24f431631 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.837+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6002ccf0 con 0x7f0d48004d00 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.837+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6002ce50 con 0x7f0d48004d00 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.837+0000 7f0d66a00640 1 -- 172.21.15.91:0/1278482055 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60034660 con 0x7f0d48004d00 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 -- 172.21.15.91:0/1278482055 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d48004d00 msgr2=0x7f0d48005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/1278482055 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d48004d00 0x7f0d48005180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0d60007360 tx=0x7f0d480057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ca00640 1 -- 172.21.15.91:0/1278482055 reap_dead start 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 -- 172.21.15.91:0/1278482055 shutdown_connections 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 -- 172.21.15.91:0/1278482055 >> 172.21.15.91:0/1278482055 conn(0x7f0d680cec80 msgr2=0x7f0d680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:58.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 -- 172.21.15.91:0/1278482055 shutdown_connections 2024-02-02T13:29:58.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.838+0000 7f0d6ea00640 1 -- 172.21.15.91:0/1278482055 wait complete. 2024-02-02T13:29:58.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.840+0000 7f0d6ea00640 1 Processor -- start 2024-02-02T13:29:58.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.840+0000 7f0d6ea00640 1 -- start start 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 0x7f0d68154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815ba50 0x7f0d6815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68154910 con 0x7f0d68155270 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6815bee0 con 0x7f0d681f22c0 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d6ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68153bf0 con 0x7f0d6815ba50 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d67400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d67400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54918/0 (socket says 172.21.15.91:54918) 2024-02-02T13:29:58.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d67400640 1 -- 172.21.15.91:0/3816474975 learned_addr learned my addr 172.21.15.91:0/3816474975 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d67e00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 0x7f0d68154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.841+0000 7f0d67400640 1 -- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815ba50 msgr2=0x7f0d6815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67400640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815ba50 0x7f0d6815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67400640 1 -- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 msgr2=0x7f0d68154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67400640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 0x7f0d68154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67400640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680c3a90 con 0x7f0d68155270 2024-02-02T13:29:58.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67e00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 0x7f0d68154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d67400640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d500108e0 tx=0x7f0d50010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000eb40 con 0x7f0d68155270 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000eca0 con 0x7f0d68155270 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680c3d70 con 0x7f0d68155270 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5001c400 con 0x7f0d68155270 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.842+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d68154530 con 0x7f0d68155270 2024-02-02T13:29:58.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.844+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68154a50 con 0x7f0d68155270 2024-02-02T13:29:58.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.845+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d5001c560 con 0x7f0d68155270 2024-02-02T13:29:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.845+0000 7f0d5fe00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 0x7f0d4006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.845+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0d50088f80 con 0x7f0d68155270 2024-02-02T13:29:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.846+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f0d40072020 con 0x7f0d68155270 2024-02-02T13:29:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.846+0000 7f0d67e00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 0x7f0d4006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.846+0000 7f0d67e00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 0x7f0d4006c8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d58001390 tx=0x7f0d58001320 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:58.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:58.850+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d500bbca0 con 0x7f0d68155270 2024-02-02T13:29:59.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.042+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d68066cf0 con 0x7f0d68155270 2024-02-02T13:29:59.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.043+0000 7f0d5fe00640 1 -- 172.21.15.91:0/3816474975 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d68154a50 con 0x7f0d68155270 2024-02-02T13:29:59.045 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 msgr2=0x7f0d4006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 0x7f0d4006c8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d58001390 tx=0x7f0d58001320 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 msgr2=0x7f0d6815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d500108e0 tx=0x7f0d50010da0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 shutdown_connections 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d4006a430 0x7f0d4006c8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d6815ba50 0x7f0d6815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d68155270 0x7f0d6815b510 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 --2- 172.21.15.91:0/3816474975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d681f22c0 0x7f0d68154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 >> 172.21.15.91:0/3816474975 conn(0x7f0d680cec80 msgr2=0x7f0d680cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 shutdown_connections 2024-02-02T13:29:59.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.046+0000 7f0d6ea00640 1 -- 172.21.15.91:0/3816474975 wait complete. 2024-02-02T13:29:59.066 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:29:59.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.275+0000 7f215c000640 1 Processor -- start 2024-02-02T13:29:59.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.275+0000 7f215c000640 1 -- start start 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215415d490 0x7f215406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215c000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f21540b0e10 0x7f21540acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215c000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215415d870 con 0x7f21540ac8d0 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215c000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21540ad270 con 0x7f21540b0e10 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215406bce0 con 0x7f215415d490 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f215a000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f21540b0e10 0x7f21540acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f2158c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f21540ac8d0 0x7f21540b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35560/0 (socket says 172.21.15.91:35560) 2024-02-02T13:29:59.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.277+0000 7f2158c00640 1 -- 172.21.15.91:0/133320837 learned_addr learned my addr 172.21.15.91:0/133320837 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f2159600640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215415d490 0x7f215406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f2153e00640 1 -- 172.21.15.91:0/133320837 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802300 0 0) 0x7f215415d870 con 0x7f21540ac8d0 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f2153e00640 1 -- 172.21.15.91:0/133320837 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2134003520 con 0x7f21540ac8d0 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f215a000640 1 -- 172.21.15.91:0/133320837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215415d490 msgr2=0x7f215406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f215a000640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215415d490 0x7f215406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f215a000640 1 -- 172.21.15.91:0/133320837 >> v1:172.21.15.91:6789/0 conn(0x7f21540ac8d0 legacy=0x7f21540b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.278+0000 7f215a000640 1 -- 172.21.15.91:0/133320837 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154161220 con 0x7f21540b0e10 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.279+0000 7f215a000640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21540b0e10 0x7f21540acd30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a92fe6466ad60a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.279+0000 7f2153e00640 1 -- 172.21.15.91:0/133320837 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c02cc50 con 0x7f21540b0e10 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.279+0000 7f2153e00640 1 -- 172.21.15.91:0/133320837 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c02cdb0 con 0x7f21540b0e10 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f2153e00640 1 -- 172.21.15.91:0/133320837 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c034660 con 0x7f21540b0e10 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 -- 172.21.15.91:0/133320837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21540b0e10 msgr2=0x7f21540acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21540b0e10 0x7f21540acd30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c02bea0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 -- 172.21.15.91:0/133320837 shutdown_connections 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21540b0e10 0x7f21540acd30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 --2- 172.21.15.91:0/133320837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215415d490 0x7f215406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 -- 172.21.15.91:0/133320837 >> 172.21.15.91:0/133320837 conn(0x7f21540cec80 msgr2=0x7f21540cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:59.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 -- 172.21.15.91:0/133320837 shutdown_connections 2024-02-02T13:29:59.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.280+0000 7f215c000640 1 -- 172.21.15.91:0/133320837 wait complete. 2024-02-02T13:29:59.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.282+0000 7f215c000640 1 Processor -- start 2024-02-02T13:29:59.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.282+0000 7f215c000640 1 -- start start 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21540ac8d0 0x7f2154156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21540b0e10 0x7f21540c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c70b0 con 0x7f215415d490 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154156cb0 con 0x7f21540b0e10 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c75b0 con 0x7f21540ac8d0 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54926/0 (socket says 172.21.15.91:54926) 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.283+0000 7f215a000640 1 -- 172.21.15.91:0/994423609 learned_addr learned my addr 172.21.15.91:0/994423609 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f2159600640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21540ac8d0 0x7f2154156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f2158c00640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21540b0e10 0x7f21540c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 -- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21540ac8d0 msgr2=0x7f2154156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21540ac8d0 0x7f2154156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 -- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21540b0e10 msgr2=0x7f21540c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21540b0e10 0x7f21540c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009a70 con 0x7f215415d490 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215a000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c040650 con 0x7f215415d490 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0407b0 con 0x7f215415d490 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21540c7890 con 0x7f215415d490 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.284+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c040b60 con 0x7f215415d490 2024-02-02T13:29:59.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.285+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21540c7e50 con 0x7f215415d490 2024-02-02T13:29:59.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.286+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540c8040 con 0x7f215415d490 2024-02-02T13:29:59.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.287+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f214c040cc0 con 0x7f215415d490 2024-02-02T13:29:59.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.287+0000 7f2151600640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 0x7f212c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.287+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f214c0ab5d0 con 0x7f215415d490 2024-02-02T13:29:59.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.288+0000 7f2159600640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 0x7f212c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.288+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f212c072020 con 0x7f215415d490 2024-02-02T13:29:59.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.288+0000 7f2159600640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 0x7f212c06c8f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2144009a10 tx=0x7f2144008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.292+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f214c076b70 con 0x7f215415d490 2024-02-02T13:29:59.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.481+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2154066cf0 con 0x7f215415d490 2024-02-02T13:29:59.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.482+0000 7f2151600640 1 -- 172.21.15.91:0/994423609 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f214c039020 con 0x7f215415d490 2024-02-02T13:29:59.484 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:29:59.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.484+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 msgr2=0x7f212c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.484+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 0x7f212c06c8f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2144009a10 tx=0x7f2144008040 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.484+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 msgr2=0x7f21540c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.484+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d90 tx=0x7f214c004a40 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 shutdown_connections 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f212c06a430 0x7f212c06c8f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f215415d490 0x7f21540c8370 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21540b0e10 0x7f21540c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 --2- 172.21.15.91:0/994423609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21540ac8d0 0x7f2154156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 >> 172.21.15.91:0/994423609 conn(0x7f21540cec80 msgr2=0x7f21540b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 shutdown_connections 2024-02-02T13:29:59.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.485+0000 7f215c000640 1 -- 172.21.15.91:0/994423609 wait complete. 2024-02-02T13:29:59.488 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:29:59.486+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-02-02T13:29:59.506 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:29:59.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.702+0000 7f5a77c00640 1 Processor -- start 2024-02-02T13:29:59.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.703+0000 7f5a77c00640 1 -- start start 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a77c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 0x7f5a700c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a77c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700d1db0 con 0x7f5a700cec60 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a77c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700b2650 con 0x7f5a7015ea40 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a77c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7015ee20 con 0x7f5a700cfb70 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a75c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5a700cec60 0x7f5a700c51f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:35570/0 (socket says 172.21.15.91:35570) 2024-02-02T13:29:59.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.704+0000 7f5a75c00640 1 -- 172.21.15.91:0/1665251192 learned_addr learned my addr 172.21.15.91:0/1665251192 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:29:59.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 0x7f5a700c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367423584 0 0) 0x7f5a700d1db0 con 0x7f5a700cec60 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a50003520 con 0x7f5a700cec60 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 580319959 0 0) 0x7f5a50003520 con 0x7f5a700cec60 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 msgr2=0x7f5a700c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 0x7f5a700c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 >> v1:172.21.15.113:6789/0 conn(0x7f5a7015ea40 legacy=0x7f5a700cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700d1ad0 con 0x7f5a700cec60 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5a68001670 con 0x7f5a700cec60 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 >> v1:172.21.15.91:6789/0 conn(0x7f5a700cec60 legacy=0x7f5a700c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 0x7f5a500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000bb10 con 0x7f5a50003f20 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000bc50 con 0x7f5a50004d00 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.705+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000be10 con 0x7f5a700cfb70 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 -- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 msgr2=0x7f5a500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700cfb70 0x7f5a500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a75c00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 -- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a50004d00 msgr2=0x7f5a50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a6fe00640 1 -- 172.21.15.91:0/1665251192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a50003520 con 0x7f5a50003f20 2024-02-02T13:29:59.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.706+0000 7f5a75c00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a50004d00 0x7f5a50005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.707+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a500047c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009fc0 tx=0x7f5a500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab831b53aae45dc1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.707+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5802cc60 con 0x7f5a50003f20 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.707+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5800f040 con 0x7f5a50003f20 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.707+0000 7f5a6f400640 1 -- 172.21.15.91:0/1665251192 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58034440 con 0x7f5a50003f20 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 -- 172.21.15.91:0/1665251192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 msgr2=0x7f5a500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 --2- 172.21.15.91:0/1665251192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a500047c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009fc0 tx=0x7f5a500057f0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a75c00640 1 -- 172.21.15.91:0/1665251192 reap_dead start 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 -- 172.21.15.91:0/1665251192 shutdown_connections 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 -- 172.21.15.91:0/1665251192 >> 172.21.15.91:0/1665251192 conn(0x7f5a700d0e80 msgr2=0x7f5a700d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 -- 172.21.15.91:0/1665251192 shutdown_connections 2024-02-02T13:29:59.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.708+0000 7f5a77c00640 1 -- 172.21.15.91:0/1665251192 wait complete. 2024-02-02T13:29:59.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.710+0000 7f5a77c00640 1 Processor -- start 2024-02-02T13:29:59.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.710+0000 7f5a77c00640 1 -- start start 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a700c9540 0x7f5a701f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700c99c0 0x7f5a701f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7014d3d0 con 0x7f5a50003f20 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a701f3150 con 0x7f5a700c9540 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a77c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700c9f90 con 0x7f5a700c99c0 2024-02-02T13:29:59.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54948/0 (socket says 172.21.15.91:54948) 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 -- 172.21.15.91:0/4151496514 learned_addr learned my addr 172.21.15.91:0/4151496514 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a700c9540 0x7f5a701f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700c99c0 0x7f5a701f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 -- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700c99c0 msgr2=0x7f5a701f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700c99c0 0x7f5a701f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 -- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a700c9540 msgr2=0x7f5a701f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a700c9540 0x7f5a701f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.711+0000 7f5a75200640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700ca270 con 0x7f5a50003f20 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a75200640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c500 tx=0x7f5a6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600052d0 con 0x7f5a50003f20 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60005430 con 0x7f5a50003f20 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a700ca550 con 0x7f5a50003f20 2024-02-02T13:29:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60012870 con 0x7f5a50003f20 2024-02-02T13:29:59.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.712+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a700cab10 con 0x7f5a50003f20 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.713+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a700cacd0 con 0x7f5a50003f20 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.714+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5a600129d0 con 0x7f5a50003f20 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.715+0000 7f5a6cc00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 0x7f5a4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.715+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5a600884c0 con 0x7f5a50003f20 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.715+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 0x7f5a4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.715+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f5a48072020 con 0x7f5a50003f20 2024-02-02T13:29:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.716+0000 7f5a6fe00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 0x7f5a4806c8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cb050 tx=0x7f5a5802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:29:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.720+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5a60053a60 con 0x7f5a50003f20 2024-02-02T13:29:59.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.920+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a70067400 con 0x7f5a50003f20 2024-02-02T13:29:59.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.921+0000 7f5a6cc00640 1 -- 172.21.15.91:0/4151496514 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5a60057080 con 0x7f5a50003f20 2024-02-02T13:29:59.923 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:29:59.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 msgr2=0x7f5a4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 0x7f5a4806c8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cb050 tx=0x7f5a5802e000 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 msgr2=0x7f5a7014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c500 tx=0x7f5a6000c9c0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 shutdown_connections 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5a4806a430 0x7f5a4806c8f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a700c99c0 0x7f5a701f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a700c9540 0x7f5a701f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 --2- 172.21.15.91:0/4151496514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a50003f20 0x7f5a7014ce90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 >> 172.21.15.91:0/4151496514 conn(0x7f5a700d0e80 msgr2=0x7f5a7015d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 shutdown_connections 2024-02-02T13:29:59.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:29:59.924+0000 7f5a77c00640 1 -- 172.21.15.91:0/4151496514 wait complete. 2024-02-02T13:29:59.942 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:29:59.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:29:59.943 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:00.000 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:29:59.999+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 3 full osd(s); Degraded data redundancy: 6/1008 objects degraded (0.595%), 1 pg degraded; Full OSDs blocking recovery: 1 pg recovery_toofull; 3 pool(s) full 2024-02-02T13:30:00.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:30:00.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:00.051 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:30:00.098 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:00.098 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:00.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:30:00.154 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:00.154 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:00.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:30:00.208 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:00.214 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:30:00.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:30:00.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.413+0000 7f1bbfc00640 1 Processor -- start 2024-02-02T13:30:00.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.413+0000 7f1bbfc00640 1 -- start start 2024-02-02T13:30:00.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbfc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1bb815f5a0 0x7f1bb80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbfc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bb80b0c40 0x7f1bb80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbfc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb80b1070 con 0x7f1bb8151f90 2024-02-02T13:30:00.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbfc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb815fa30 con 0x7f1bb80b0c40 2024-02-02T13:30:00.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbfc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb8070f80 con 0x7f1bb815f5a0 2024-02-02T13:30:00.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbdc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bb80b0c40 0x7f1bb80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbdc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bb80b0c40 0x7f1bb80b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36088/0 (socket says 172.21.15.91:36088) 2024-02-02T13:30:00.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bbdc00640 1 -- 172.21.15.91:0/4142503816 learned_addr learned my addr 172.21.15.91:0/4142503816 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:00.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.415+0000 7f1bafc00640 1 --2- 172.21.15.91:0/4142503816 >> v2:172.21.15.113:3300/0 conn(0x7f1bb815f5a0 0x7f1bb80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.416+0000 7f1bbdc00640 1 -- 172.21.15.91:0/4142503816 >> v2:172.21.15.113:3300/0 conn(0x7f1bb815f5a0 msgr2=0x7f1bb80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:00.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.416+0000 7f1bbdc00640 1 --2- 172.21.15.91:0/4142503816 >> v2:172.21.15.113:3300/0 conn(0x7f1bb815f5a0 0x7f1bb80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.416+0000 7f1bbdc00640 1 -- 172.21.15.91:0/4142503816 >> v1:172.21.15.113:6789/0 conn(0x7f1bb8151f90 legacy=0x7f1bb806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.416+0000 7f1bbdc00640 1 -- 172.21.15.91:0/4142503816 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8152370 con 0x7f1bb80b0c40 2024-02-02T13:30:00.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.417+0000 7f1bbdc00640 1 --2- 172.21.15.91:0/4142503816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb80b0c40 0x7f1bb80b3af0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8071a00 tx=0x7f1bb002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfad0fe9b72ecd91 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.417+0000 7f1bb7e00640 1 -- 172.21.15.91:0/4142503816 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb002cb20 con 0x7f1bb80b0c40 2024-02-02T13:30:00.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.417+0000 7f1bb7e00640 1 -- 172.21.15.91:0/4142503816 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cc80 con 0x7f1bb80b0c40 2024-02-02T13:30:00.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.417+0000 7f1bb7e00640 1 -- 172.21.15.91:0/4142503816 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00343f0 con 0x7f1bb80b0c40 2024-02-02T13:30:00.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 -- 172.21.15.91:0/4142503816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb80b0c40 msgr2=0x7f1bb80b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/4142503816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb80b0c40 0x7f1bb80b3af0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8071a00 tx=0x7f1bb002bf80 comp rx=0 tx=0).stop 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 -- 172.21.15.91:0/4142503816 shutdown_connections 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/4142503816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb80b0c40 0x7f1bb80b3af0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/4142503816 >> v2:172.21.15.113:3300/0 conn(0x7f1bb815f5a0 0x7f1bb80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 -- 172.21.15.91:0/4142503816 >> 172.21.15.91:0/4142503816 conn(0x7f1bb80cec80 msgr2=0x7f1bb80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 -- 172.21.15.91:0/4142503816 shutdown_connections 2024-02-02T13:30:00.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.418+0000 7f1bbfc00640 1 -- 172.21.15.91:0/4142503816 wait complete. 2024-02-02T13:30:00.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.420+0000 7f1bbfc00640 1 Processor -- start 2024-02-02T13:30:00.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.420+0000 7f1bbfc00640 1 -- start start 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bb80b0c40 0x7f1bb81f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bb8151f90 0x7f1bb8151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80c5cc0 con 0x7f1bb815f5a0 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb81f7130 con 0x7f1bb8151f90 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbfc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8151660 con 0x7f1bb80b0c40 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbdc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbdc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36096/0 (socket says 172.21.15.91:36096) 2024-02-02T13:30:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbdc00640 1 -- 172.21.15.91:0/534196895 learned_addr learned my addr 172.21.15.91:0/534196895 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bafc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bb8151f90 0x7f1bb8151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbd200640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bb80b0c40 0x7f1bb81f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbdc00640 1 -- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bb80b0c40 msgr2=0x7f1bb81f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.421+0000 7f1bbdc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bb80b0c40 0x7f1bb81f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbdc00640 1 -- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bb8151f90 msgr2=0x7f1bb8151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbdc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bb8151f90 0x7f1bb8151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbdc00640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0009b30 con 0x7f1bb815f5a0 2024-02-02T13:30:00.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbdc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009f40 tx=0x7f1bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0009510 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002c920 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb003e690 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb80c5f10 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.422+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb80c64d0 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.423+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8066cf0 con 0x7f1bb815f5a0 2024-02-02T13:30:00.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.424+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1bb003e7f0 con 0x7f1bb815f5a0 2024-02-02T13:30:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.425+0000 7f1bb5600640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 0x7f1b9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.425+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1bb00ab4a0 con 0x7f1bb815f5a0 2024-02-02T13:30:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.425+0000 7f1bbd200640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 0x7f1b9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.425+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f1b90072020 con 0x7f1bb815f5a0 2024-02-02T13:30:00.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.426+0000 7f1bbd200640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 0x7f1b9006c8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8000f60 tx=0x7f1ba8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.429+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1bb0076a40 con 0x7f1bb815f5a0 2024-02-02T13:30:00.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.660+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bb815b270 con 0x7f1bb815f5a0 2024-02-02T13:30:00.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.661+0000 7f1bb5600640 1 -- 172.21.15.91:0/534196895 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f1bb007a060 con 0x7f1bb815f5a0 2024-02-02T13:30:00.662 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:00.662 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:30:00.663 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:30:00.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 msgr2=0x7f1b9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 0x7f1b9006c8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8000f60 tx=0x7f1ba8009040 comp rx=0 tx=0).stop 2024-02-02T13:30:00.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 msgr2=0x7f1bb815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009f40 tx=0x7f1bb0004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:00.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 shutdown_connections 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1b9006a430 0x7f1b9006c8f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bb815f5a0 0x7f1bb815a1f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bb8151f90 0x7f1bb8151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 --2- 172.21.15.91:0/534196895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bb80b0c40 0x7f1bb81f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.663+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 >> 172.21.15.91:0/534196895 conn(0x7f1bb80cec80 msgr2=0x7f1bb806f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.664+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 shutdown_connections 2024-02-02T13:30:00.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.664+0000 7f1bbfc00640 1 -- 172.21.15.91:0/534196895 wait complete. 2024-02-02T13:30:00.683 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:00.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.933+0000 7f1a1c200640 1 Processor -- start 2024-02-02T13:30:00.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.933+0000 7f1a1c200640 1 -- start start 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.934+0000 7f1a1c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a1415f5a0 0x7f1a140b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.934+0000 7f1a1c200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140b1070 con 0x7f1a140b0c40 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.934+0000 7f1a1c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1406c600 con 0x7f1a14151f90 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.934+0000 7f1a1c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1415fa30 con 0x7f1a1415f5a0 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.934+0000 7f1a1a200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1a140b0c40 0x7f1a140b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39414/0 (socket says 172.21.15.91:39414) 2024-02-02T13:30:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a1a200640 1 -- 172.21.15.91:0/3681102466 learned_addr learned my addr 172.21.15.91:0/3681102466 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:00.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058251638 0 0) 0x7f1a140b1070 con 0x7f1a140b0c40 2024-02-02T13:30:00.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f4003520 con 0x7f1a140b0c40 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a18e00640 1 --2- 172.21.15.91:0/3681102466 >> v2:172.21.15.113:3300/0 conn(0x7f1a1415f5a0 0x7f1a140b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 684224229 0 0) 0x7f19f4003520 con 0x7f1a140b0c40 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 >> v2:172.21.15.113:3300/0 conn(0x7f1a1415f5a0 msgr2=0x7f1a140b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 --2- 172.21.15.91:0/3681102466 >> v2:172.21.15.113:3300/0 conn(0x7f1a1415f5a0 0x7f1a140b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 >> v1:172.21.15.113:6789/0 conn(0x7f1a14151f90 legacy=0x7f1a1406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14070f80 con 0x7f1a140b0c40 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1a0c0019f0 con 0x7f1a140b0c40 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 >> v1:172.21.15.91:6789/0 conn(0x7f1a140b0c40 legacy=0x7f1a140b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.935+0000 7f1a13e00640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 0x7f19f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a13e00640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19f40047c0 0x7f19f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a13e00640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40051d0 0x7f19f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1415fa30 con 0x7f19f40039c0 2024-02-02T13:30:00.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f400c120 con 0x7f19f40051d0 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f400c2e0 con 0x7f19f40047c0 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 0x7f19f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a18e00640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19f40047c0 0x7f19f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 -- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19f40047c0 msgr2=0x7f19f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19f40047c0 0x7f19f4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 -- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40051d0 msgr2=0x7f19f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40051d0 0x7f19f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a19800640 1 -- 172.21.15.91:0/3681102466 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4003520 con 0x7f19f40039c0 2024-02-02T13:30:00.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.936+0000 7f1a1a200640 1 -- 172.21.15.91:0/3681102466 reap_dead start 2024-02-02T13:30:00.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.937+0000 7f1a19800640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 0x7f19f4003de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001750 tx=0x7f19f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58ce83615918906 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040052d0 con 0x7f19f40039c0 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0402e050 con 0x7f19f40039c0 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a13e00640 1 -- 172.21.15.91:0/3681102466 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04003830 con 0x7f19f40039c0 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 -- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 msgr2=0x7f19f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 0x7f19f4003de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001750 tx=0x7f19f40057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 -- 172.21.15.91:0/3681102466 shutdown_connections 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 --2- 172.21.15.91:0/3681102466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19f40039c0 0x7f19f4003de0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 -- 172.21.15.91:0/3681102466 >> 172.21.15.91:0/3681102466 conn(0x7f1a140cec80 msgr2=0x7f1a140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 -- 172.21.15.91:0/3681102466 shutdown_connections 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.938+0000 7f1a1c200640 1 -- 172.21.15.91:0/3681102466 wait complete. 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.940+0000 7f1a1c200640 1 Processor -- start 2024-02-02T13:30:00.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.940+0000 7f1a1c200640 1 -- start start 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40039c0 0x7f1a141f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a1414ed70 0x7f1a14158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14150d10 con 0x7f1a14150320 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1413dc90 con 0x7f19f40039c0 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141f6f50 con 0x7f1a1414ed70 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36108/0 (socket says 172.21.15.91:36108) 2024-02-02T13:30:00.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.941+0000 7f1a1a200640 1 -- 172.21.15.91:0/1248770270 learned_addr learned my addr 172.21.15.91:0/1248770270 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a19800640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40039c0 0x7f1a141f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 -- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a1414ed70 msgr2=0x7f1a14158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a18e00640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a1414ed70 0x7f1a14158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a1414ed70 0x7f1a14158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 -- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40039c0 msgr2=0x7f1a141f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40039c0 0x7f1a141f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1413dee0 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1a200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1a140b1870 tx=0x7f1a0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00ce80 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00a5b0 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00a940 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.942+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1413e1c0 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.943+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1413e780 con 0x7f1a14150320 2024-02-02T13:30:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.944+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a14066cf0 con 0x7f1a14150320 2024-02-02T13:30:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.944+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1a0c00aaa0 con 0x7f1a14150320 2024-02-02T13:30:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.946+0000 7f1a11600640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 0x7f19ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.946+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a0c088630 con 0x7f1a14150320 2024-02-02T13:30:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.946+0000 7f1a19800640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 0x7f19ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.946+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f19ec072050 con 0x7f1a14150320 2024-02-02T13:30:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.947+0000 7f1a19800640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 0x7f19ec06c920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a04033000 tx=0x7f1a04009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:00.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:00.950+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1a0c053c70 con 0x7f1a14150320 2024-02-02T13:30:01.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.126+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a140c2480 con 0x7f1a14150320 2024-02-02T13:30:01.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.128+0000 7f1a11600640 1 -- 172.21.15.91:0/1248770270 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16234 (secure 0 0 0) 0x7f1a0c057290 con 0x7f1a14150320 2024-02-02T13:30:01.129 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:01.130 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":91,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:29:59.486880+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 msgr2=0x7f19ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 0x7f19ec06c920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a04033000 tx=0x7f1a04009030 comp rx=0 tx=0).stop 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 msgr2=0x7f1a141507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1a140b1870 tx=0x7f1a0c00cb70 comp rx=0 tx=0).stop 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 shutdown_connections 2024-02-02T13:30:01.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19ec06a460 0x7f19ec06c920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a1414ed70 0x7f1a14158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a14150320 0x7f1a141507d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 --2- 172.21.15.91:0/1248770270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f40039c0 0x7f1a141f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 >> 172.21.15.91:0/1248770270 conn(0x7f1a140cec80 msgr2=0x7f1a140cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 shutdown_connections 2024-02-02T13:30:01.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.130+0000 7f1a1c200640 1 -- 172.21.15.91:0/1248770270 wait complete. 2024-02-02T13:30:01.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:29:02.962375+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:29:03.885489+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:30:01.151 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:30:01.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.360+0000 7f7d19000640 1 Processor -- start 2024-02-02T13:30:01.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.360+0000 7f7d19000640 1 -- start start 2024-02-02T13:30:01.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d19000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d140d0070 0x7f7d140b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d19000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d19000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140cd930 con 0x7f7d140b6260 2024-02-02T13:30:01.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d19000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140b8f90 con 0x7f7d140d0070 2024-02-02T13:30:01.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d19000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140d1680 con 0x7f7d140b2260 2024-02-02T13:30:01.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d11e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d140d0070 0x7f7d140b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d11400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d11400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38764/0 (socket says 172.21.15.91:38764) 2024-02-02T13:30:01.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.362+0000 7f7d11400640 1 -- 172.21.15.91:0/3861809585 learned_addr learned my addr 172.21.15.91:0/3861809585 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:01.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.363+0000 7f7d11400640 1 -- 172.21.15.91:0/3861809585 >> v2:172.21.15.113:3300/0 conn(0x7f7d140d0070 msgr2=0x7f7d140b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:01.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.363+0000 7f7d11400640 1 --2- 172.21.15.91:0/3861809585 >> v2:172.21.15.113:3300/0 conn(0x7f7d140d0070 0x7f7d140b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.363+0000 7f7d11400640 1 -- 172.21.15.91:0/3861809585 >> v1:172.21.15.113:6789/0 conn(0x7f7d140b6260 legacy=0x7f7d140d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:01.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.363+0000 7f7d11400640 1 -- 172.21.15.91:0/3861809585 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140d1960 con 0x7f7d140b2260 2024-02-02T13:30:01.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.364+0000 7f7d11400640 1 --2- 172.21.15.91:0/3861809585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009e50 tx=0x7f7cfc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86854aa517bc16a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.365+0000 7f7d10a00640 1 -- 172.21.15.91:0/3861809585 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc02ca70 con 0x7f7d140b2260 2024-02-02T13:30:01.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.365+0000 7f7d10a00640 1 -- 172.21.15.91:0/3861809585 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc02cbd0 con 0x7f7d140b2260 2024-02-02T13:30:01.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.365+0000 7f7d10a00640 1 -- 172.21.15.91:0/3861809585 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0343f0 con 0x7f7d140b2260 2024-02-02T13:30:01.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 -- 172.21.15.91:0/3861809585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 msgr2=0x7f7d140d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 --2- 172.21.15.91:0/3861809585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009e50 tx=0x7f7cfc02bed0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 -- 172.21.15.91:0/3861809585 shutdown_connections 2024-02-02T13:30:01.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 --2- 172.21.15.91:0/3861809585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140b2260 0x7f7d140d0450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 --2- 172.21.15.91:0/3861809585 >> v2:172.21.15.113:3300/0 conn(0x7f7d140d0070 0x7f7d140b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 -- 172.21.15.91:0/3861809585 >> 172.21.15.91:0/3861809585 conn(0x7f7d14070660 msgr2=0x7f7d14070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:01.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 -- 172.21.15.91:0/3861809585 shutdown_connections 2024-02-02T13:30:01.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.366+0000 7f7d19000640 1 -- 172.21.15.91:0/3861809585 wait complete. 2024-02-02T13:30:01.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.368+0000 7f7d19000640 1 Processor -- start 2024-02-02T13:30:01.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.368+0000 7f7d19000640 1 -- start start 2024-02-02T13:30:01.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.368+0000 7f7d19000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d19000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 0x7f7d140ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d19000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140d0070 0x7f7d140b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d19000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140ac930 con 0x7f7d140b2260 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d19000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140ca2a0 con 0x7f7d140b6260 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d19000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140ca580 con 0x7f7d140d0070 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36116/0 (socket says 172.21.15.91:36116) 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 -- 172.21.15.91:0/3093568966 learned_addr learned my addr 172.21.15.91:0/3093568966 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11400640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 0x7f7d140ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d12800640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140d0070 0x7f7d140b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 -- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140d0070 msgr2=0x7f7d140b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140d0070 0x7f7d140b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 -- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 msgr2=0x7f7d140ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 0x7f7d140ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11e00640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009b30 con 0x7f7d140b2260 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.369+0000 7f7d11400640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 0x7f7d140ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d11e00640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9b0 tx=0x7f7d0400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04009670 con 0x7f7d140b2260 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400c9a0 con 0x7f7d140b2260 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d140ca860 con 0x7f7d140b2260 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400cd50 con 0x7f7d140b2260 2024-02-02T13:30:01.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.370+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d140b1dc0 con 0x7f7d140b2260 2024-02-02T13:30:01.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.371+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d14073830 con 0x7f7d140b2260 2024-02-02T13:30:01.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.372+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7d04013470 con 0x7f7d140b2260 2024-02-02T13:30:01.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.373+0000 7f7d0a000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 0x7f7cec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.373+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7d04088c30 con 0x7f7d140b2260 2024-02-02T13:30:01.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.373+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f7cec072020 con 0x7f7d140b2260 2024-02-02T13:30:01.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.373+0000 7f7d11400640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 0x7f7cec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.374+0000 7f7d11400640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 0x7f7cec06c8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc035000 tx=0x7f7cfc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.377+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7d04017060 con 0x7f7d140b2260 2024-02-02T13:30:01.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.514+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(92..92 src has 1..92) v4 ==== 296+0+0 (secure 0 0 0) 0x7f7d04058590 con 0x7f7d140b2260 2024-02-02T13:30:01.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.514+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f7cec0728d0 con 0x7f7d140b2260 2024-02-02T13:30:01.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.570+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d14066cf0 con 0x7f7d140b2260 2024-02-02T13:30:01.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.571+0000 7f7d0a000640 1 -- 172.21.15.91:0/3093568966 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d04021020 con 0x7f7d140b2260 2024-02-02T13:30:01.575 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:30:01.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 msgr2=0x7f7cec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 0x7f7cec06c8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc035000 tx=0x7f7cfc030000 comp rx=0 tx=0).stop 2024-02-02T13:30:01.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 msgr2=0x7f7d140a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9b0 tx=0x7f7d0400ee70 comp rx=0 tx=0).stop 2024-02-02T13:30:01.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 shutdown_connections 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cec06a430 0x7f7cec06c8f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d140d0070 0x7f7d140b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d140b6260 0x7f7d140ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 --2- 172.21.15.91:0/3093568966 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d140b2260 0x7f7d140a9a70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 >> 172.21.15.91:0/3093568966 conn(0x7f7d14070660 msgr2=0x7f7d140b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 shutdown_connections 2024-02-02T13:30:01.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.574+0000 7f7d19000640 1 -- 172.21.15.91:0/3093568966 wait complete. 2024-02-02T13:30:01.593 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:30:01.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.795+0000 7f291fc00640 1 Processor -- start 2024-02-02T13:30:01.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.795+0000 7f291fc00640 1 -- start start 2024-02-02T13:30:01.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291fc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f291815d490 0x7f291806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291fc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180b0e10 0x7f29180acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291fc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291815d870 con 0x7f29180ac8d0 2024-02-02T13:30:01.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291fc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180ad2a0 con 0x7f291815d490 2024-02-02T13:30:01.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291fc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291806bce0 con 0x7f29180b0e10 2024-02-02T13:30:01.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f291815d490 0x7f291806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.797+0000 7f291dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180b0e10 0x7f29180acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f291815d490 0x7f291806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:52540/0 (socket says 172.21.15.91:52540) 2024-02-02T13:30:01.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 -- 172.21.15.91:0/2997506680 learned_addr learned my addr 172.21.15.91:0/2997506680 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:01.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 -- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180b0e10 msgr2=0x7f29180acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:01.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 --2- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180b0e10 0x7f29180acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 -- 172.21.15.91:0/2997506680 >> v1:172.21.15.113:6789/0 conn(0x7f29180ac8d0 legacy=0x7f29180b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.798+0000 7f291d200640 1 -- 172.21.15.91:0/2997506680 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918161220 con 0x7f291815d490 2024-02-02T13:30:01.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.800+0000 7f291d200640 1 --2- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f291806c170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2908009870 tx=0x7f290802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2784d08d631d5d3d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.800+0000 7f2917400640 1 -- 172.21.15.91:0/2997506680 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908030840 con 0x7f291815d490 2024-02-02T13:30:01.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.800+0000 7f2917400640 1 -- 172.21.15.91:0/2997506680 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29080309a0 con 0x7f291815d490 2024-02-02T13:30:01.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.800+0000 7f2917400640 1 -- 172.21.15.91:0/2997506680 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908030d30 con 0x7f291815d490 2024-02-02T13:30:01.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 -- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 msgr2=0x7f291806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 --2- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f291806c170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2908009870 tx=0x7f290802b980 comp rx=0 tx=0).stop 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 -- 172.21.15.91:0/2997506680 shutdown_connections 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 --2- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180b0e10 0x7f29180acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 --2- 172.21.15.91:0/2997506680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f291806c170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 -- 172.21.15.91:0/2997506680 >> 172.21.15.91:0/2997506680 conn(0x7f29180cec80 msgr2=0x7f29180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 -- 172.21.15.91:0/2997506680 shutdown_connections 2024-02-02T13:30:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.801+0000 7f291fc00640 1 -- 172.21.15.91:0/2997506680 wait complete. 2024-02-02T13:30:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.803+0000 7f291fc00640 1 Processor -- start 2024-02-02T13:30:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.803+0000 7f291fc00640 1 -- start start 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180ac8d0 0x7f291814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f29180c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291815ece0 con 0x7f29180b0e10 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291815efc0 con 0x7f291815d490 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291fc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291815f2a0 con 0x7f29180ac8d0 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f2917e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f2917e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36126/0 (socket says 172.21.15.91:36126) 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f2917e00640 1 -- 172.21.15.91:0/358868453 learned_addr learned my addr 172.21.15.91:0/358868453 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291d200640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180ac8d0 0x7f291814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.804+0000 7f291dc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f29180c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 -- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180ac8d0 msgr2=0x7f291814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180ac8d0 0x7f291814e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 -- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 msgr2=0x7f29180c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:01.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f29180c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:01.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29080094d0 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2917e00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f290000c400 tx=0x7f290000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000052d0 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2900005430 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291815f580 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900003830 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.805+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291815e9d0 con 0x7f29180b0e10 2024-02-02T13:30:01.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.807+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291814ea70 con 0x7f29180b0e10 2024-02-02T13:30:01.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.807+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2900003990 con 0x7f29180b0e10 2024-02-02T13:30:01.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.808+0000 7f2914c00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 0x7f28f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:01.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.808+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2900087e60 con 0x7f29180b0e10 2024-02-02T13:30:01.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.809+0000 7f291d200640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 0x7f28f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:01.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.809+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f28f0072050 con 0x7f29180b0e10 2024-02-02T13:30:01.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.809+0000 7f291d200640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 0x7f28f006c920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2908036000 tx=0x7f2908008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:01.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:01.813+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f29000534a0 con 0x7f29180b0e10 2024-02-02T13:30:02.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.001+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2918066cf0 con 0x7f29180b0e10 2024-02-02T13:30:02.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.002+0000 7f2914c00640 1 -- 172.21.15.91:0/358868453 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2900017070 con 0x7f29180b0e10 2024-02-02T13:30:02.004 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:30:02.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 msgr2=0x7f28f006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 0x7f28f006c920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2908036000 tx=0x7f2908008030 comp rx=0 tx=0).stop 2024-02-02T13:30:02.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 msgr2=0x7f291815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f290000c400 tx=0x7f290000c8c0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 shutdown_connections 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f28f006a460 0x7f28f006c920 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291815d490 0x7f29180c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29180b0e10 0x7f291815e490 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 --2- 172.21.15.91:0/358868453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29180ac8d0 0x7f291814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 >> 172.21.15.91:0/358868453 conn(0x7f29180cec80 msgr2=0x7f2918070260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 shutdown_connections 2024-02-02T13:30:02.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.005+0000 7f291fc00640 1 -- 172.21.15.91:0/358868453 wait complete. 2024-02-02T13:30:02.024 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:30:02.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.236+0000 7fa5f8600640 1 Processor -- start 2024-02-02T13:30:02.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.237+0000 7fa5f8600640 1 -- start start 2024-02-02T13:30:02.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.238+0000 7fa5f8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.238+0000 7fa5f8600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5f00ce130 0x7fa5f00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.238+0000 7fa5f8600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f00cc7c0 con 0x7fa5f00cd470 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.238+0000 7fa5f8600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00ce930 con 0x7fa5f00ce130 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.238+0000 7fa5f8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f006c890 con 0x7fa5f015a010 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.239+0000 7fa5f5c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.239+0000 7fa5f5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5f00ce130 0x7fa5f00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.239+0000 7fa5f5c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38802/0 (socket says 172.21.15.91:38802) 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.239+0000 7fa5f5c00640 1 -- 172.21.15.91:0/1031877428 learned_addr learned my addr 172.21.15.91:0/1031877428 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.240+0000 7fa5f5c00640 1 -- 172.21.15.91:0/1031877428 >> v2:172.21.15.113:3300/0 conn(0x7fa5f00ce130 msgr2=0x7fa5f00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.240+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/1031877428 >> v2:172.21.15.113:3300/0 conn(0x7fa5f00ce130 0x7fa5f00ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.240+0000 7fa5f5c00640 1 -- 172.21.15.91:0/1031877428 >> v1:172.21.15.113:6789/0 conn(0x7fa5f00cd470 legacy=0x7fa5f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.240+0000 7fa5f5c00640 1 -- 172.21.15.91:0/1031877428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f006cb70 con 0x7fa5f015a010 2024-02-02T13:30:02.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.241+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/1031877428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e50 tx=0x7fa5e002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cae05bf5741141d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.241+0000 7fa5efe00640 1 -- 172.21.15.91:0/1031877428 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e002cb00 con 0x7fa5f015a010 2024-02-02T13:30:02.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.241+0000 7fa5efe00640 1 -- 172.21.15.91:0/1031877428 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e002cc60 con 0x7fa5f015a010 2024-02-02T13:30:02.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.241+0000 7fa5efe00640 1 -- 172.21.15.91:0/1031877428 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00343f0 con 0x7fa5f015a010 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 -- 172.21.15.91:0/1031877428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 msgr2=0x7fa5f00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 --2- 172.21.15.91:0/1031877428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e50 tx=0x7fa5e002bf60 comp rx=0 tx=0).stop 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 -- 172.21.15.91:0/1031877428 shutdown_connections 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 --2- 172.21.15.91:0/1031877428 >> v2:172.21.15.113:3300/0 conn(0x7fa5f00ce130 0x7fa5f00ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 --2- 172.21.15.91:0/1031877428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f015a010 0x7fa5f00c9670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 -- 172.21.15.91:0/1031877428 >> 172.21.15.91:0/1031877428 conn(0x7fa5f00cec80 msgr2=0x7fa5f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 -- 172.21.15.91:0/1031877428 shutdown_connections 2024-02-02T13:30:02.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.242+0000 7fa5f8600640 1 -- 172.21.15.91:0/1031877428 wait complete. 2024-02-02T13:30:02.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.244+0000 7fa5f8600640 1 Processor -- start 2024-02-02T13:30:02.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.244+0000 7fa5f8600640 1 -- start start 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 0x7fa5f014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00c2390 0x7fa5f00b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f015c690 con 0x7fa5f015a010 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00b4690 con 0x7fa5f00c2390 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f015ba80 con 0x7fa5f00ce130 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36142/0 (socket says 172.21.15.91:36142) 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.245+0000 7fa5f5200640 1 -- 172.21.15.91:0/3731382695 learned_addr learned my addr 172.21.15.91:0/3731382695 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:02.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f6600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00c2390 0x7fa5f00b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 0x7fa5f014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 -- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 msgr2=0x7fa5f014e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 0x7fa5f014e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 -- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00c2390 msgr2=0x7fa5f00b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00c2390 0x7fa5f00b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0009b30 con 0x7fa5f015a010 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 0x7fa5f014e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f5200640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9b0 tx=0x7fa5d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8009670 con 0x7fa5f015a010 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d800c9a0 con 0x7fa5f015a010 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800cd50 con 0x7fa5f015a010 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.246+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f015bd60 con 0x7fa5f015a010 2024-02-02T13:30:02.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.247+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f015c320 con 0x7fa5f015a010 2024-02-02T13:30:02.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.248+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f015c800 con 0x7fa5f015a010 2024-02-02T13:30:02.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.249+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa5d8013470 con 0x7fa5f015a010 2024-02-02T13:30:02.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.250+0000 7fa5ed600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 0x7fa5c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.250+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa5d8088c20 con 0x7fa5f015a010 2024-02-02T13:30:02.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.250+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 0x7fa5c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.250+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7fa5c8072020 con 0x7fa5f015a010 2024-02-02T13:30:02.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.251+0000 7fa5f5c00640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 0x7fa5c806c8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.254+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa5d8017060 con 0x7fa5f015a010 2024-02-02T13:30:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.440+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5f0066cf0 con 0x7fa5f015a010 2024-02-02T13:30:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.441+0000 7fa5ed600640 1 -- 172.21.15.91:0/3731382695 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5d8021070 con 0x7fa5f015a010 2024-02-02T13:30:02.443 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 msgr2=0x7fa5c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 0x7fa5c806c8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0030000 comp rx=0 tx=0).stop 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 msgr2=0x7fa5f00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9b0 tx=0x7fa5d800ee70 comp rx=0 tx=0).stop 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 shutdown_connections 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c806a430 0x7fa5c806c8f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00c2390 0x7fa5f00b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5f015a010 0x7fa5f00b4150 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 --2- 172.21.15.91:0/3731382695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00ce130 0x7fa5f014e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 >> 172.21.15.91:0/3731382695 conn(0x7fa5f00cec80 msgr2=0x7fa5f00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 shutdown_connections 2024-02-02T13:30:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.444+0000 7fa5f8600640 1 -- 172.21.15.91:0/3731382695 wait complete. 2024-02-02T13:30:02.466 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:02.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.668+0000 7f1253e00640 1 Processor -- start 2024-02-02T13:30:02.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.669+0000 7f1253e00640 1 -- start start 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540d2a80 0x7f12540b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f12540cff00 0x7f12540d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540d17b0 con 0x7f125415d110 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540d25f0 con 0x7f12540cff00 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b6fd0 con 0x7f12540d2a80 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f12540cff00 0x7f12540d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f12540cff00 0x7f12540d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36156/0 (socket says 172.21.15.91:36156) 2024-02-02T13:30:02.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.670+0000 7f1253400640 1 -- 172.21.15.91:0/744898063 learned_addr learned my addr 172.21.15.91:0/744898063 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:02.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.671+0000 7f1253400640 1 -- 172.21.15.91:0/744898063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540d2a80 msgr2=0x7f12540b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:02.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.671+0000 7f1253400640 1 --2- 172.21.15.91:0/744898063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540d2a80 0x7f12540b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.671+0000 7f1253400640 1 -- 172.21.15.91:0/744898063 >> v1:172.21.15.113:6789/0 conn(0x7f125415d110 legacy=0x7f12540cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.671+0000 7f1253400640 1 -- 172.21.15.91:0/744898063 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540d19e0 con 0x7f12540cff00 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.672+0000 7f1253400640 1 --2- 172.21.15.91:0/744898063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540cff00 0x7f12540d1270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f124c009870 tx=0x7f124c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368f47d3972cb8e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.672+0000 7f1251600640 1 -- 172.21.15.91:0/744898063 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c030840 con 0x7f12540cff00 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.672+0000 7f1251600640 1 -- 172.21.15.91:0/744898063 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c0309a0 con 0x7f12540cff00 2024-02-02T13:30:02.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.672+0000 7f1251600640 1 -- 172.21.15.91:0/744898063 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c030d30 con 0x7f12540cff00 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 -- 172.21.15.91:0/744898063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540cff00 msgr2=0x7f12540d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 --2- 172.21.15.91:0/744898063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540cff00 0x7f12540d1270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f124c009870 tx=0x7f124c02b980 comp rx=0 tx=0).stop 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 -- 172.21.15.91:0/744898063 shutdown_connections 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 --2- 172.21.15.91:0/744898063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540cff00 0x7f12540d1270 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 --2- 172.21.15.91:0/744898063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540d2a80 0x7f12540b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 -- 172.21.15.91:0/744898063 >> 172.21.15.91:0/744898063 conn(0x7f1254070690 msgr2=0x7f1254070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 -- 172.21.15.91:0/744898063 shutdown_connections 2024-02-02T13:30:02.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.673+0000 7f1253e00640 1 -- 172.21.15.91:0/744898063 wait complete. 2024-02-02T13:30:02.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.675+0000 7f1253e00640 1 Processor -- start 2024-02-02T13:30:02.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.675+0000 7f1253e00640 1 -- start start 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.675+0000 7f1253e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540cff00 0x7f12541f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1253e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1253e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f125415d110 0x7f12540cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1253e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541f25b0 con 0x7f12540d2a80 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1253e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125415e4c0 con 0x7f12540cff00 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1253e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125415d8b0 con 0x7f125415d110 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36166/0 (socket says 172.21.15.91:36166) 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 -- 172.21.15.91:0/2970573008 learned_addr learned my addr 172.21.15.91:0/2970573008 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 -- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f125415d110 msgr2=0x7f12540cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f125415d110 0x7f12540cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 -- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540cff00 msgr2=0x7f12541f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540cff00 0x7f12541f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 -- 172.21.15.91:0/2970573008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0094d0 con 0x7f12540d2a80 2024-02-02T13:30:02.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.676+0000 7f1252000640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b9c0 tx=0x7f123c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.677+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00cba0 con 0x7f12540d2a80 2024-02-02T13:30:02.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.677+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00cd20 con 0x7f12540d2a80 2024-02-02T13:30:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.677+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125415db90 con 0x7f12540d2a80 2024-02-02T13:30:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.677+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c014680 con 0x7f12540d2a80 2024-02-02T13:30:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.677+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125415e150 con 0x7f12540d2a80 2024-02-02T13:30:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.678+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125404dff0 con 0x7f12540d2a80 2024-02-02T13:30:02.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.679+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f123c01e440 con 0x7f12540d2a80 2024-02-02T13:30:02.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.680+0000 7f124aa00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 0x7f122c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:02.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.680+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f123c0888b0 con 0x7f12540d2a80 2024-02-02T13:30:02.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.680+0000 7f1252a00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 0x7f122c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:02.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.681+0000 7f1252a00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 0x7f122c06c920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1244009650 tx=0x7f1244006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:02.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.684+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f123c053e50 con 0x7f12540d2a80 2024-02-02T13:30:02.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.890+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12540cd130 con 0x7f12540d2a80 2024-02-02T13:30:02.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.891+0000 7f124aa00640 1 -- 172.21.15.91:0/2970573008 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f123c019020 con 0x7f12540d2a80 2024-02-02T13:30:02.892 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:02.892 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:30:02.893 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:30:02.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 msgr2=0x7f122c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 0x7f122c06c920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1244009650 tx=0x7f1244006ba0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 msgr2=0x7f1254151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b9c0 tx=0x7f123c00be80 comp rx=0 tx=0).stop 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 shutdown_connections 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f122c06a460 0x7f122c06c920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f125415d110 0x7f12540cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12540d2a80 0x7f1254151120 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.893+0000 7f1253e00640 1 --2- 172.21.15.91:0/2970573008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540cff00 0x7f12541f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.894+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 >> 172.21.15.91:0/2970573008 conn(0x7f1254070690 msgr2=0x7f125415b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.894+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 shutdown_connections 2024-02-02T13:30:02.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:02.894+0000 7f1253e00640 1 -- 172.21.15.91:0/2970573008 wait complete. 2024-02-02T13:30:02.912 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:03.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.111+0000 7f370c000640 1 Processor -- start 2024-02-02T13:30:03.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.111+0000 7f370c000640 1 -- start start 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370c000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f370415d4c0 0x7f370415e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370c000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37040cf3f0 0x7f370406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370c000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3704161890 con 0x7f370406df30 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370c000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370415d8a0 con 0x7f370415d4c0 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370c000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370415e940 con 0x7f37040cf3f0 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f3709600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f370415d4c0 0x7f370415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370a000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f370406df30 0x7f370406c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39420/0 (socket says 172.21.15.91:39420) 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f370a000640 1 -- 172.21.15.91:0/3839185057 learned_addr learned my addr 172.21.15.91:0/3839185057 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.113+0000 7f3708c00640 1 --2- 172.21.15.91:0/3839185057 >> v2:172.21.15.113:3300/0 conn(0x7f37040cf3f0 0x7f370406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.114+0000 7f3709600640 1 -- 172.21.15.91:0/3839185057 >> v2:172.21.15.113:3300/0 conn(0x7f37040cf3f0 msgr2=0x7f370406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.114+0000 7f3709600640 1 --2- 172.21.15.91:0/3839185057 >> v2:172.21.15.113:3300/0 conn(0x7f37040cf3f0 0x7f370406d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.114+0000 7f3709600640 1 -- 172.21.15.91:0/3839185057 >> v1:172.21.15.91:6789/0 conn(0x7f370406df30 legacy=0x7f370406c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.114+0000 7f3709600640 1 -- 172.21.15.91:0/3839185057 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370406e360 con 0x7f370415d4c0 2024-02-02T13:30:03.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.114+0000 7f3703e00640 1 -- 172.21.15.91:0/3839185057 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790762933 0 0) 0x7f3704161890 con 0x7f370406df30 2024-02-02T13:30:03.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.115+0000 7f3709600640 1 --2- 172.21.15.91:0/3839185057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415e400 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f370406ad70 tx=0x7f36ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28aeda713c7db31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.115+0000 7f3703e00640 1 -- 172.21.15.91:0/3839185057 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec02e070 con 0x7f370415d4c0 2024-02-02T13:30:03.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.115+0000 7f3703e00640 1 -- 172.21.15.91:0/3839185057 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec0315a0 con 0x7f370415d4c0 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.115+0000 7f3703e00640 1 -- 172.21.15.91:0/3839185057 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec031930 con 0x7f370415d4c0 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 -- 172.21.15.91:0/3839185057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 msgr2=0x7f370415e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 --2- 172.21.15.91:0/3839185057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415e400 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f370406ad70 tx=0x7f36ec02b840 comp rx=0 tx=0).stop 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 -- 172.21.15.91:0/3839185057 shutdown_connections 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 --2- 172.21.15.91:0/3839185057 >> v2:172.21.15.113:3300/0 conn(0x7f37040cf3f0 0x7f370406d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 --2- 172.21.15.91:0/3839185057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415e400 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 -- 172.21.15.91:0/3839185057 >> 172.21.15.91:0/3839185057 conn(0x7f37040b1940 msgr2=0x7f37040b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 -- 172.21.15.91:0/3839185057 shutdown_connections 2024-02-02T13:30:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.116+0000 7f370c000640 1 -- 172.21.15.91:0/3839185057 wait complete. 2024-02-02T13:30:03.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.117+0000 7f370c000640 1 Processor -- start 2024-02-02T13:30:03.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.118+0000 7f370c000640 1 -- start start 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.118+0000 7f370c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 0x7f370415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37040cf3f0 0x7f37040c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370c000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370c000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c7ed0 con 0x7f370415d4c0 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370415a940 con 0x7f370406df30 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37041f3c10 con 0x7f37040cf3f0 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36180/0 (socket says 172.21.15.91:36180) 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 -- 172.21.15.91:0/4018278749 learned_addr learned my addr 172.21.15.91:0/4018278749 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f3709600640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 0x7f370415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f3708c00640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37040cf3f0 0x7f37040c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 -- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37040cf3f0 msgr2=0x7f37040c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37040cf3f0 0x7f37040c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 -- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 msgr2=0x7f370415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 0x7f370415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f370a000640 1 -- 172.21.15.91:0/4018278749 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec009400 con 0x7f370415d4c0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.119+0000 7f3709600640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 0x7f370415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f370a000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c440 tx=0x7f36fc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0074e0 con 0x7f370415d4c0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc007640 con 0x7f370415d4c0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc004a20 con 0x7f370415d4c0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37041f3ef0 con 0x7f370415d4c0 2024-02-02T13:30:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.120+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37041f44b0 con 0x7f370415d4c0 2024-02-02T13:30:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.121+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37041f46d0 con 0x7f370415d4c0 2024-02-02T13:30:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.122+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f36fc004b80 con 0x7f370415d4c0 2024-02-02T13:30:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.123+0000 7f3701600640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 0x7f36dc06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.123+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f36fc088660 con 0x7f370415d4c0 2024-02-02T13:30:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.123+0000 7f3709600640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 0x7f36dc06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.124+0000 7f3709600640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 0x7f36dc06c940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f36ec036000 tx=0x7f36ec008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.127+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f36fc053c80 con 0x7f370415d4c0 2024-02-02T13:30:03.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.321+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3704066cf0 con 0x7f370415d4c0 2024-02-02T13:30:03.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.324+0000 7f3701600640 1 -- 172.21.15.91:0/4018278749 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16198 (secure 0 0 0) 0x7f36fc001060 con 0x7f370415d4c0 2024-02-02T13:30:03.325 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:03.326 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":93,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:02.525642+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 msgr2=0x7f36dc06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 0x7f36dc06c940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f36ec036000 tx=0x7f36ec008030 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 msgr2=0x7f370415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c440 tx=0x7f36fc00c900 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 shutdown_connections 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f36dc06a480 0x7f36dc06c940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37040cf3f0 0x7f37040c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f370415d4c0 0x7f370415b580 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 --2- 172.21.15.91:0/4018278749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f370406df30 0x7f370415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 >> 172.21.15.91:0/4018278749 conn(0x7f37040b1940 msgr2=0x7f37040af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:03.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 shutdown_connections 2024-02-02T13:30:03.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.327+0000 7f370c000640 1 -- 172.21.15.91:0/4018278749 wait complete. 2024-02-02T13:30:03.349 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:30:03.349 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:03.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.549+0000 7fe9c7200640 1 Processor -- start 2024-02-02T13:30:03.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.549+0000 7fe9c7200640 1 -- start start 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c7200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9c00d2a80 0x7fe9c00b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c7200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00d25f0 con 0x7fe9c015d110 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00d19e0 con 0x7fe9c00cff00 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c7200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c00d1cc0 con 0x7fe9c00d2a80 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c5e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe9c015d110 0x7fe9c00cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39430/0 (socket says 172.21.15.91:39430) 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c5e00640 1 -- 172.21.15.91:0/4213172553 learned_addr learned my addr 172.21.15.91:0/4213172553 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c5400640 1 --2- 172.21.15.91:0/4213172553 >> v2:172.21.15.113:3300/0 conn(0x7fe9c00d2a80 0x7fe9c00b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966854026 0 0) 0x7fe9c00d25f0 con 0x7fe9c015d110 2024-02-02T13:30:03.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.551+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a0003520 con 0x7fe9c015d110 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4276897201 0 0) 0x7fe9a0003520 con 0x7fe9c015d110 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 >> v2:172.21.15.113:3300/0 conn(0x7fe9c00d2a80 msgr2=0x7fe9c00b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 --2- 172.21.15.91:0/4213172553 >> v2:172.21.15.113:3300/0 conn(0x7fe9c00d2a80 0x7fe9c00b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 >> v1:172.21.15.113:6789/0 conn(0x7fe9c00cff00 legacy=0x7fe9c00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00d1fa0 con 0x7fe9c015d110 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe9b00019f0 con 0x7fe9c015d110 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 >> v1:172.21.15.91:6789/0 conn(0x7fe9c015d110 legacy=0x7fe9c00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9a0003970 0x7fe9a0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00047c0 0x7fe9a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 0x7fe9a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00d1cc0 con 0x7fe9a0005180 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a000c000 con 0x7fe9a00047c0 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a000c1c0 con 0x7fe9a0003970 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c6800640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 0x7fe9a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.552+0000 7fe9c5400640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00047c0 0x7fe9a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c5e00640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9a0003970 0x7fe9a0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c6800640 1 -- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9a0003970 msgr2=0x7fe9a0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c6800640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9a0003970 0x7fe9a0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c6800640 1 -- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00047c0 msgr2=0x7fe9a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c6800640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00047c0 0x7fe9a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c6800640 1 -- 172.21.15.91:0/4213172553 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0003520 con 0x7fe9a0005180 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.553+0000 7fe9c5e00640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9a0003970 0x7fe9a0003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c6800640 1 -- 172.21.15.91:0/4213172553 reap_dead start 2024-02-02T13:30:03.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c6800640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 0x7fe9a000b980 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00d0310 tx=0x7fe9a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112e014801f48a16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8007220 con 0x7fe9a0005180 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b802e050 con 0x7fe9a0005180 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c4a00640 1 -- 172.21.15.91:0/4213172553 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80075e0 con 0x7fe9a0005180 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c7200640 1 -- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 msgr2=0x7fe9a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.554+0000 7fe9c7200640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 0x7fe9a000b980 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00d0310 tx=0x7fe9a00057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.555+0000 7fe9c7200640 1 -- 172.21.15.91:0/4213172553 shutdown_connections 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.555+0000 7fe9c7200640 1 --2- 172.21.15.91:0/4213172553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9a0005180 0x7fe9a000b980 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.555+0000 7fe9c7200640 1 -- 172.21.15.91:0/4213172553 >> 172.21.15.91:0/4213172553 conn(0x7fe9c0070690 msgr2=0x7fe9c0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.555+0000 7fe9c7200640 1 -- 172.21.15.91:0/4213172553 shutdown_connections 2024-02-02T13:30:03.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.555+0000 7fe9c7200640 1 -- 172.21.15.91:0/4213172553 wait complete. 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.556+0000 7fe9c7200640 1 Processor -- start 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.556+0000 7fe9c7200640 1 -- start start 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a0005180 0x7fe9c014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9c015b270 0x7fe9c015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c014c890 con 0x7fe9c0159630 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c015bc30 con 0x7fe9a0005180 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.557+0000 7fe9c7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c014f960 con 0x7fe9c015b270 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36208/0 (socket says 172.21.15.91:36208) 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 -- 172.21.15.91:0/1316650668 learned_addr learned my addr 172.21.15.91:0/1316650668 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 -- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9c015b270 msgr2=0x7fe9c015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9c015b270 0x7fe9c015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 -- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a0005180 msgr2=0x7fe9c014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a0005180 0x7fe9c014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 -- 172.21.15.91:0/1316650668 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c015d670 con 0x7fe9c0159630 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c5400640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80115c0 tx=0x7fe9a8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8004220 con 0x7fe9c0159630 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8004380 con 0x7fe9c0159630 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8015930 con 0x7fe9c0159630 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.558+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c014e380 con 0x7fe9c0159630 2024-02-02T13:30:03.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.559+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c014e940 con 0x7fe9c0159630 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.560+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c004dff0 con 0x7fe9c0159630 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.561+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe9a8015a90 con 0x7fe9c0159630 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.561+0000 7fe9be000640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 0x7fe99806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.561+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe9a8088cf0 con 0x7fe9c0159630 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.562+0000 7fe9c5e00640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 0x7fe99806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:03.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.562+0000 7fe9c5e00640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 0x7fe99806c920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0002230 tx=0x7fe9b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:03.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.565+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe9a808d070 con 0x7fe9c0159630 2024-02-02T13:30:03.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.793+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c00d1410 con 0x7fe9c0159630 2024-02-02T13:30:03.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.794+0000 7fe9be000640 1 -- 172.21.15.91:0/1316650668 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fe9a8054330 con 0x7fe9c0159630 2024-02-02T13:30:03.796 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:03.796 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:30:03.796 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:30:03.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 msgr2=0x7fe99806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 0x7fe99806c920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0002230 tx=0x7fe9b0009040 comp rx=0 tx=0).stop 2024-02-02T13:30:03.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 msgr2=0x7fe9c014f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:03.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80115c0 tx=0x7fe9a8011a80 comp rx=0 tx=0).stop 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 shutdown_connections 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe99806a460 0x7fe99806c920 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9c015b270 0x7fe9c015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9c0159630 0x7fe9c014f420 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 --2- 172.21.15.91:0/1316650668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a0005180 0x7fe9c014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.796+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 >> 172.21.15.91:0/1316650668 conn(0x7fe9c0070690 msgr2=0x7fe9c015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.797+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 shutdown_connections 2024-02-02T13:30:03.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:03.797+0000 7fe9c7200640 1 -- 172.21.15.91:0/1316650668 wait complete. 2024-02-02T13:30:03.814 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:04.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.011+0000 7f74d6000640 1 Processor -- start 2024-02-02T13:30:04.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.012+0000 7f74d6000640 1 -- start start 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d015d110 0x7f74d00b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d6000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74d00d2a80 0x7f74d00d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d6000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d00d17b0 con 0x7f74d00cf7f0 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d6000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d00d25f0 con 0x7f74d00d2a80 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d6000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00b6fd0 con 0x7f74d015d110 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74cfe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74d00d2a80 0x7f74d00d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74cfe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74d00d2a80 0x7f74d00d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36214/0 (socket says 172.21.15.91:36214) 2024-02-02T13:30:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74cfe00640 1 -- 172.21.15.91:0/1885215048 learned_addr learned my addr 172.21.15.91:0/1885215048 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.013+0000 7f74d4c00640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d015d110 0x7f74d00b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.014+0000 7f74cfe00640 1 -- 172.21.15.91:0/1885215048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d015d110 msgr2=0x7f74d00b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.014+0000 7f74cfe00640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d015d110 0x7f74d00b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.014+0000 7f74cfe00640 1 -- 172.21.15.91:0/1885215048 >> v1:172.21.15.113:6789/0 conn(0x7f74d00cf7f0 legacy=0x7f74d00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.014+0000 7f74cfe00640 1 -- 172.21.15.91:0/1885215048 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00d19e0 con 0x7f74d00d2a80 2024-02-02T13:30:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.015+0000 7f74cfe00640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d00d2a80 0x7f74d00d1270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009870 tx=0x7f74b802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b5bc9c060d137c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.015+0000 7f74cf400640 1 -- 172.21.15.91:0/1885215048 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b802f840 con 0x7f74d00d2a80 2024-02-02T13:30:04.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.015+0000 7f74cf400640 1 -- 172.21.15.91:0/1885215048 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b802f9a0 con 0x7f74d00d2a80 2024-02-02T13:30:04.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.015+0000 7f74cf400640 1 -- 172.21.15.91:0/1885215048 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b802fd30 con 0x7f74d00d2a80 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 -- 172.21.15.91:0/1885215048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d00d2a80 msgr2=0x7f74d00d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d00d2a80 0x7f74d00d1270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009870 tx=0x7f74b802ba20 comp rx=0 tx=0).stop 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 -- 172.21.15.91:0/1885215048 shutdown_connections 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d00d2a80 0x7f74d00d1270 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 --2- 172.21.15.91:0/1885215048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d015d110 0x7f74d00b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 -- 172.21.15.91:0/1885215048 >> 172.21.15.91:0/1885215048 conn(0x7f74d0070690 msgr2=0x7f74d0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 -- 172.21.15.91:0/1885215048 shutdown_connections 2024-02-02T13:30:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.016+0000 7f74d6000640 1 -- 172.21.15.91:0/1885215048 wait complete. 2024-02-02T13:30:04.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.018+0000 7f74d6000640 1 Processor -- start 2024-02-02T13:30:04.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.018+0000 7f74d6000640 1 -- start start 2024-02-02T13:30:04.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d00cf7f0 0x7f74d0150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d00d2a80 0x7f74d00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0159290 con 0x7f74d015d110 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0157fa0 con 0x7f74d00cf7f0 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d6000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0158e00 con 0x7f74d00d2a80 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36224/0 (socket says 172.21.15.91:36224) 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 -- 172.21.15.91:0/418569931 learned_addr learned my addr 172.21.15.91:0/418569931 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d4c00640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d00cf7f0 0x7f74d0150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 -- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d00d2a80 msgr2=0x7f74d00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d00d2a80 0x7f74d00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 -- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d00cf7f0 msgr2=0x7f74d0150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d00cf7f0 0x7f74d0150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.019+0000 7f74d5600640 1 -- 172.21.15.91:0/418569931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80094d0 con 0x7f74d015d110 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74d5600640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c7d0 tx=0x7f74c800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8003b70 con 0x7f74d015d110 2024-02-02T13:30:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8003cf0 con 0x7f74d015d110 2024-02-02T13:30:04.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800aa90 con 0x7f74d015d110 2024-02-02T13:30:04.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d01581f0 con 0x7f74d015d110 2024-02-02T13:30:04.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.020+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d01587b0 con 0x7f74d015d110 2024-02-02T13:30:04.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.021+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d004dff0 con 0x7f74d015d110 2024-02-02T13:30:04.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.022+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f74c800abf0 con 0x7f74d015d110 2024-02-02T13:30:04.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.023+0000 7f74ccc00640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 0x7f74a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.023+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f74c8088790 con 0x7f74d015d110 2024-02-02T13:30:04.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.023+0000 7f74d4c00640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 0x7f74a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.023+0000 7f74d4c00640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 0x7f74a806c8f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006fd0 tx=0x7f74c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.027+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f74c8053d30 con 0x7f74d015d110 2024-02-02T13:30:04.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.233+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74d0158f40 con 0x7f74d015d110 2024-02-02T13:30:04.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.234+0000 7f74ccc00640 1 -- 172.21.15.91:0/418569931 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f74c8003e60 con 0x7f74d015d110 2024-02-02T13:30:04.235 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:04.235 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:29:04.868792+0000","modified":"2024-02-02T13:29:07.903616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3733831376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3733831376},{"type":"v1","addr":"172.21.15.91:6837","nonce":3733831376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-02T13:30:04.236 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 40 2024-02-02T13:30:04.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 msgr2=0x7f74a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 0x7f74a806c8f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006fd0 tx=0x7f74c0008040 comp rx=0 tx=0).stop 2024-02-02T13:30:04.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 msgr2=0x7f74d00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c7d0 tx=0x7f74c800cc90 comp rx=0 tx=0).stop 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 shutdown_connections 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f74a806a430 0x7f74a806c8f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74d00d2a80 0x7f74d00b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74d015d110 0x7f74d00b4610 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 --2- 172.21.15.91:0/418569931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d00cf7f0 0x7f74d0150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.236+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 >> 172.21.15.91:0/418569931 conn(0x7f74d0070690 msgr2=0x7f74d015c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.237+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 shutdown_connections 2024-02-02T13:30:04.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.237+0000 7f74d6000640 1 -- 172.21.15.91:0/418569931 wait complete. 2024-02-02T13:30:04.255 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.457+0000 7f467d000640 1 Processor -- start 2024-02-02T13:30:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.457+0000 7f467d000640 1 -- start start 2024-02-02T13:30:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f467d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f467806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f467d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f46780b0700 0x7f46780b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f467d000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467815d870 con 0x7f467815d490 2024-02-02T13:30:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f467d000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780b10b0 con 0x7f46780b0700 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f467d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467806bce0 con 0x7f46780acfe0 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f4675e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f467815d490 0x7f46780ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39436/0 (socket says 172.21.15.91:39436) 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f4675e00640 1 -- 172.21.15.91:0/2030384514 learned_addr learned my addr 172.21.15.91:0/2030384514 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.459+0000 7f4676800640 1 --2- 172.21.15.91:0/2030384514 >> v2:172.21.15.91:3300/0 conn(0x7f46780b0700 0x7f46780b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f466d200640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f467806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f4675400640 1 -- 172.21.15.91:0/2030384514 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135089320 0 0) 0x7f467815d870 con 0x7f467815d490 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f4676800640 1 -- 172.21.15.91:0/2030384514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 msgr2=0x7f467806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f4676800640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f467806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f4676800640 1 -- 172.21.15.91:0/2030384514 >> v1:172.21.15.91:6789/0 conn(0x7f467815d490 legacy=0x7f46780ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.460+0000 7f4676800640 1 -- 172.21.15.91:0/2030384514 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678161220 con 0x7f46780b0700 2024-02-02T13:30:04.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.461+0000 7f4676800640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46780b0b70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f467002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f0d6a70b794ea8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.461+0000 7f4675400640 1 -- 172.21.15.91:0/2030384514 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467002cc60 con 0x7f46780b0700 2024-02-02T13:30:04.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.461+0000 7f4675400640 1 -- 172.21.15.91:0/2030384514 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467002cdc0 con 0x7f46780b0700 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.461+0000 7f4675400640 1 -- 172.21.15.91:0/2030384514 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670034660 con 0x7f46780b0700 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 -- 172.21.15.91:0/2030384514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 msgr2=0x7f46780b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46780b0b70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f467002beb0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 -- 172.21.15.91:0/2030384514 shutdown_connections 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46780b0b70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 --2- 172.21.15.91:0/2030384514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f467806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 -- 172.21.15.91:0/2030384514 >> 172.21.15.91:0/2030384514 conn(0x7f46780cec80 msgr2=0x7f46780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 -- 172.21.15.91:0/2030384514 shutdown_connections 2024-02-02T13:30:04.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.462+0000 7f467d000640 1 -- 172.21.15.91:0/2030384514 wait complete. 2024-02-02T13:30:04.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.464+0000 7f467d000640 1 Processor -- start 2024-02-02T13:30:04.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.464+0000 7f467d000640 1 -- start start 2024-02-02T13:30:04.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f4678160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 0x7f46780c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b1e90 con 0x7f46780b0700 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b2170 con 0x7f467815d490 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f467d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b2450 con 0x7f46780acfe0 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f466d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f466d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36242/0 (socket says 172.21.15.91:36242) 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f466d200640 1 -- 172.21.15.91:0/3321504626 learned_addr learned my addr 172.21.15.91:0/3321504626 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f4675e00640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f4678160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.465+0000 7f4676800640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 0x7f46780c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 -- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 msgr2=0x7f4678160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f4678160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 -- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 msgr2=0x7f46780c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 0x7f46780c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 -- 172.21.15.91:0/3321504626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670009a70 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f4676800640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 0x7f46780c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466d200640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f466000d5c0 tx=0x7f466000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46600125f0 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660012750 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46780b2730 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660012b00 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.466+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781605c0 con 0x7f46780b0700 2024-02-02T13:30:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.467+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4678160780 con 0x7f46780b0700 2024-02-02T13:30:04.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.468+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4660012c60 con 0x7f46780b0700 2024-02-02T13:30:04.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.469+0000 7f466ea00640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 0x7f465006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.469+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4660088de0 con 0x7f46780b0700 2024-02-02T13:30:04.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.469+0000 7f4675e00640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 0x7f465006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.470+0000 7f4675e00640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 0x7f465006c8f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4668000f60 tx=0x7f4668007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.474+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4660054380 con 0x7f46780b0700 2024-02-02T13:30:04.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.668+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4678066cf0 con 0x7f46780b0700 2024-02-02T13:30:04.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.671+0000 7f466ea00640 1 -- 172.21.15.91:0/3321504626 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16198 (secure 0 0 0) 0x7f46600579a0 con 0x7f46780b0700 2024-02-02T13:30:04.673 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:04.673 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":93,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:02.525642+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:29:02.962375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-02-02T13:29:03.885489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 msgr2=0x7f465006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 0x7f465006c8f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4668000f60 tx=0x7f4668007b50 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 msgr2=0x7f46781f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f466000d5c0 tx=0x7f466000da80 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 shutdown_connections 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f465006a430 0x7f465006c8f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f467815d490 0x7f46780c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f46780b0700 0x7f46781f51f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 --2- 172.21.15.91:0/3321504626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46780acfe0 0x7f4678160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 >> 172.21.15.91:0/3321504626 conn(0x7f46780cec80 msgr2=0x7f4678157910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:04.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 shutdown_connections 2024-02-02T13:30:04.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.674+0000 7f467d000640 1 -- 172.21.15.91:0/3321504626 wait complete. 2024-02-02T13:30:04.695 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:30:04.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.898+0000 7f11e1800640 1 Processor -- start 2024-02-02T13:30:04.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.898+0000 7f11e1800640 1 -- start start 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.899+0000 7f11e1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.899+0000 7f11e1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11dc0ce4c0 0x7f11dc070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.899+0000 7f11e1800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11dc15d640 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.899+0000 7f11e1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11dc0d2530 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.899+0000 7f11e1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc0d2810 con 0x7f11dc15d260 2024-02-02T13:30:04.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.900+0000 7f11daa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.900+0000 7f11db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11dc0ce4c0 0x7f11dc070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.900+0000 7f11db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11dc0ce4c0 0x7f11dc070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:52650/0 (socket says 172.21.15.91:52650) 2024-02-02T13:30:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.900+0000 7f11db400640 1 -- 172.21.15.91:0/2956429038 learned_addr learned my addr 172.21.15.91:0/2956429038 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.901+0000 7f11db400640 1 -- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 msgr2=0x7f11dc0b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.901+0000 7f11db400640 1 --2- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc0b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.901+0000 7f11db400640 1 -- 172.21.15.91:0/2956429038 >> v1:172.21.15.113:6789/0 conn(0x7f11dc0d1dc0 legacy=0x7f11dc0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.901+0000 7f11db400640 1 -- 172.21.15.91:0/2956429038 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0d2af0 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.902+0000 7f11db400640 1 --2- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc070910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009870 tx=0x7f11d402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74d9369aedfaa73 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.902+0000 7f11d9600640 1 -- 172.21.15.91:0/2956429038 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4031640 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11d9600640 1 -- 172.21.15.91:0/2956429038 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d40317a0 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11d9600640 1 -- 172.21.15.91:0/2956429038 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4031b30 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 -- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 msgr2=0x7f11dc070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 --2- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc070910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009870 tx=0x7f11d402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 -- 172.21.15.91:0/2956429038 shutdown_connections 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 --2- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc070910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 --2- 172.21.15.91:0/2956429038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 -- 172.21.15.91:0/2956429038 >> 172.21.15.91:0/2956429038 conn(0x7f11dc0cfa60 msgr2=0x7f11dc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.903+0000 7f11e1800640 1 -- 172.21.15.91:0/2956429038 shutdown_connections 2024-02-02T13:30:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.904+0000 7f11e1800640 1 -- 172.21.15.91:0/2956429038 wait complete. 2024-02-02T13:30:04.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.906+0000 7f11e1800640 1 Processor -- start 2024-02-02T13:30:04.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.906+0000 7f11e1800640 1 -- start start 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc1e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc1e55e0 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc1e49d0 con 0x7f11dc0ce4c0 2024-02-02T13:30:04.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.907+0000 7f11e1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc1e4cb0 con 0x7f11dc15d260 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36254/0 (socket says 172.21.15.91:36254) 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 -- 172.21.15.91:0/2777837732 learned_addr learned my addr 172.21.15.91:0/2777837732 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11daa00640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11db400640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc1e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 -- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 msgr2=0x7f11dc1e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc1e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 -- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 msgr2=0x7f11dc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 -- 172.21.15.91:0/2777837732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40094d0 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11db400640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc1e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11da000640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c790 tx=0x7f11c400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.908+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400cea0 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.909+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4006d50 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.909+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11dc1e4f90 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.909+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4003bb0 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.909+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11dc15b010 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.910+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11dc0cf260 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.911+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f11c4003d10 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.911+0000 7f11d2a00640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 0x7f11b406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:04.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.912+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f11c4088630 con 0x7f11dc0d1dc0 2024-02-02T13:30:04.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.912+0000 7f11daa00640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 0x7f11b406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:04.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.912+0000 7f11daa00640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 0x7f11b406c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009a30 tx=0x7f11cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:04.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:04.916+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f11c40575d0 con 0x7f11dc0d1dc0 2024-02-02T13:30:05.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.123+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11dc067400 con 0x7f11dc0d1dc0 2024-02-02T13:30:05.126 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:30:05.125+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:30:05.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.141+0000 7f11d2a00640 1 -- 172.21.15.91:0/2777837732 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f11c4057160 con 0x7f11dc0d1dc0 2024-02-02T13:30:05.143 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:30:05.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 msgr2=0x7f11b406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 0x7f11b406c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009a30 tx=0x7f11cc008040 comp rx=0 tx=0).stop 2024-02-02T13:30:05.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 msgr2=0x7f11dc1e8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c790 tx=0x7f11c400cc50 comp rx=0 tx=0).stop 2024-02-02T13:30:05.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 shutdown_connections 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f11b406a460 0x7f11b406c920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11dc15d260 0x7f11dc1e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f11dc0d1dc0 0x7f11dc1e8fd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 --2- 172.21.15.91:0/2777837732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11dc0ce4c0 0x7f11dc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.144+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 >> 172.21.15.91:0/2777837732 conn(0x7f11dc0cfa60 msgr2=0x7f11dc0ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.145+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 shutdown_connections 2024-02-02T13:30:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.145+0000 7f11e1800640 1 -- 172.21.15.91:0/2777837732 wait complete. 2024-02-02T13:30:05.163 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:30:05.243 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:30:05.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.350+0000 7f0345800640 1 Processor -- start 2024-02-02T13:30:05.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.350+0000 7f0345800640 1 -- start start 2024-02-02T13:30:05.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.351+0000 7f0345800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f03400c9f90 0x7f03400b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.351+0000 7f0345800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f03400b0bd0 0x7f03400ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.351+0000 7f0345800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03400b1020 con 0x7f034015a1e0 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.351+0000 7f0345800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03400cca70 con 0x7f03400c9f90 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.351+0000 7f0345800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03400cc770 con 0x7f03400b0bd0 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f03400c9f90 0x7f03400b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f034015a1e0 0x7f0340070ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39456/0 (socket says 172.21.15.91:39456) 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033ea00640 1 -- 172.21.15.91:0/2783346366 learned_addr learned my addr 172.21.15.91:0/2783346366 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033e000640 1 -- 172.21.15.91:0/2783346366 >> v2:172.21.15.113:3300/0 conn(0x7f03400b0bd0 msgr2=0x7f03400ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033e000640 1 --2- 172.21.15.91:0/2783346366 >> v2:172.21.15.113:3300/0 conn(0x7f03400b0bd0 0x7f03400ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033e000640 1 -- 172.21.15.91:0/2783346366 >> v1:172.21.15.91:6789/0 conn(0x7f034015a1e0 legacy=0x7f0340070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033e000640 1 -- 172.21.15.91:0/2783346366 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400ca570 con 0x7f03400c9f90 2024-02-02T13:30:05.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.352+0000 7f033d600640 1 -- 172.21.15.91:0/2783346366 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007754655 0 0) 0x7f0330001110 con 0x7f034015a1e0 2024-02-02T13:30:05.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.353+0000 7f033e000640 1 --2- 172.21.15.91:0/2783346366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400c9f90 0x7f03400b0690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f03280097a0 tx=0x7f032802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e37656e5517fdc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:05.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f033d600640 1 -- 172.21.15.91:0/2783346366 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328031830 con 0x7f03400c9f90 2024-02-02T13:30:05.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f033d600640 1 -- 172.21.15.91:0/2783346366 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0328031990 con 0x7f03400c9f90 2024-02-02T13:30:05.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f033d600640 1 -- 172.21.15.91:0/2783346366 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328031d20 con 0x7f03400c9f90 2024-02-02T13:30:05.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 -- 172.21.15.91:0/2783346366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400c9f90 msgr2=0x7f03400b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 --2- 172.21.15.91:0/2783346366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400c9f90 0x7f03400b0690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f03280097a0 tx=0x7f032802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 -- 172.21.15.91:0/2783346366 shutdown_connections 2024-02-02T13:30:05.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 --2- 172.21.15.91:0/2783346366 >> v2:172.21.15.113:3300/0 conn(0x7f03400b0bd0 0x7f03400ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 --2- 172.21.15.91:0/2783346366 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400c9f90 0x7f03400b0690 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.354+0000 7f0345800640 1 -- 172.21.15.91:0/2783346366 >> 172.21.15.91:0/2783346366 conn(0x7f03400ceb90 msgr2=0x7f03400cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:05.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.355+0000 7f0345800640 1 -- 172.21.15.91:0/2783346366 shutdown_connections 2024-02-02T13:30:05.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.355+0000 7f0345800640 1 -- 172.21.15.91:0/2783346366 wait complete. 2024-02-02T13:30:05.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.356+0000 7f0345800640 1 Processor -- start 2024-02-02T13:30:05.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.357+0000 7f0345800640 1 -- start start 2024-02-02T13:30:05.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03400c9f90 0x7f03400a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f034015a1e0 0x7f03401f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03401f19e0 con 0x7f03400b0bd0 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400aae10 con 0x7f03400c9f90 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f0345800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03401f1420 con 0x7f034015a1e0 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36280/0 (socket says 172.21.15.91:36280) 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 -- 172.21.15.91:0/4248618200 learned_addr learned my addr 172.21.15.91:0/4248618200 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033e000640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03400c9f90 0x7f03400a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 -- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f034015a1e0 msgr2=0x7f03401f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f034015a1e0 0x7f03401f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 -- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03400c9f90 msgr2=0x7f03400a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03400c9f90 0x7f03400a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.358+0000 7f033ea00640 1 -- 172.21.15.91:0/4248618200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009400 con 0x7f03400b0bd0 2024-02-02T13:30:05.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f033ea00640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f033000c430 tx=0x7f033000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300074e0 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0330007660 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330004a20 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03401f1700 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.359+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03400a8940 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.360+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03400a84c0 con 0x7f03400b0bd0 2024-02-02T13:30:05.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.361+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0330004b80 con 0x7f03400b0bd0 2024-02-02T13:30:05.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.362+0000 7f0336a00640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.362+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f03300886e0 con 0x7f03400b0bd0 2024-02-02T13:30:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.362+0000 7f033e000640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.363+0000 7f033e000640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 0x7f031806c8f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0328036000 tx=0x7f0328008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:05.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.367+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f03400a84c0 con 0x7f03400b0bd0 2024-02-02T13:30:05.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:05.596+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f03400b1020 con 0x7f03400b0bd0 2024-02-02T13:30:06.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.147+0000 7f0336a00640 1 -- 172.21.15.91:0/4248618200 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f03400b1020 con 0x7f03400b0bd0 2024-02-02T13:30:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.150+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 msgr2=0x7f031806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.150+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 0x7f031806c8f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0328036000 tx=0x7f0328008030 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.150+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 msgr2=0x7f03400aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f033000c430 tx=0x7f033000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 shutdown_connections 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f034015a1e0 0x7f03401f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03400c9f90 0x7f03400a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 --2- 172.21.15.91:0/4248618200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03400b0bd0 0x7f03400aa8d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 >> 172.21.15.91:0/4248618200 conn(0x7f03400ceb90 msgr2=0x7f03400cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 shutdown_connections 2024-02-02T13:30:06.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.151+0000 7f0345800640 1 -- 172.21.15.91:0/4248618200 wait complete. 2024-02-02T13:30:06.172 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:30:06.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.380+0000 7fe9b9600640 1 Processor -- start 2024-02-02T13:30:06.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.380+0000 7fe9b9600640 1 -- start start 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.381+0000 7fe9b9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.381+0000 7fe9b9600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe9b40b0700 0x7fe9b40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.381+0000 7fe9b9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b40cc7c0 con 0x7fe9b40ce130 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.381+0000 7fe9b9600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40b10d0 con 0x7fe9b40b0700 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.381+0000 7fe9b9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40ce930 con 0x7fe9b415a010 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe9b40b0700 0x7fe9b40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe9b40b0700 0x7fe9b40b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36282/0 (socket says 172.21.15.91:36282) 2024-02-02T13:30:06.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 -- 172.21.15.91:0/1739989817 learned_addr learned my addr 172.21.15.91:0/1739989817 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:06.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b2a00640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:06.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 -- 172.21.15.91:0/1739989817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 msgr2=0x7fe9b40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:06.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 -- 172.21.15.91:0/1739989817 >> v1:172.21.15.113:6789/0 conn(0x7fe9b40ce130 legacy=0x7fe9b40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:06.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b3400640 1 -- 172.21.15.91:0/1739989817 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40cd640 con 0x7fe9b40b0700 2024-02-02T13:30:06.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.382+0000 7fe9b2a00640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.383+0000 7fe9b3400640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b40b0b90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e50 tx=0x7fe9ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac97ee6ba3202a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.383+0000 7fe9b2000640 1 -- 172.21.15.91:0/1739989817 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac02ca70 con 0x7fe9b40b0700 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b2000640 1 -- 172.21.15.91:0/1739989817 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac02cbd0 con 0x7fe9b40b0700 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b2000640 1 -- 172.21.15.91:0/1739989817 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0343f0 con 0x7fe9b40b0700 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 -- 172.21.15.91:0/1739989817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 msgr2=0x7fe9b40b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b40b0b90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e50 tx=0x7fe9ac02bed0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 -- 172.21.15.91:0/1739989817 shutdown_connections 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b40b0b90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 --2- 172.21.15.91:0/1739989817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 -- 172.21.15.91:0/1739989817 >> 172.21.15.91:0/1739989817 conn(0x7fe9b40cec80 msgr2=0x7fe9b40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 -- 172.21.15.91:0/1739989817 shutdown_connections 2024-02-02T13:30:06.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.384+0000 7fe9b9600640 1 -- 172.21.15.91:0/1739989817 wait complete. 2024-02-02T13:30:06.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.386+0000 7fe9b9600640 1 Processor -- start 2024-02-02T13:30:06.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.386+0000 7fe9b9600640 1 -- start start 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b40ce130 0x7fe9b4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b414f9f0 con 0x7fe9b40b0700 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4158e50 con 0x7fe9b40ce130 2024-02-02T13:30:06.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40c6f80 con 0x7fe9b415a010 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36298/0 (socket says 172.21.15.91:36298) 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.387+0000 7fe9b2a00640 1 -- 172.21.15.91:0/2198048168 learned_addr learned my addr 172.21.15.91:0/2198048168 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 -- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 msgr2=0x7fe9b40c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 -- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b40ce130 msgr2=0x7fe9b4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b40ce130 0x7fe9b4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 -- 172.21.15.91:0/2198048168 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac009b30 con 0x7fe9b40b0700 2024-02-02T13:30:06.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b2a00640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c7f0 tx=0x7fe9a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007820 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4007980 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007cf0 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40c7260 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.388+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b4159130 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.389+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b4066cf0 con 0x7fe9b40b0700 2024-02-02T13:30:06.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.390+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe9a40113f0 con 0x7fe9b40b0700 2024-02-02T13:30:06.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.391+0000 7fe9ab400640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 0x7fe98c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:06.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.391+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe9a4088d70 con 0x7fe9b40b0700 2024-02-02T13:30:06.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.391+0000 7fe9a9e00640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 0x7fe98c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:06.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.392+0000 7fe9a9e00640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 0x7fe98c06c8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c2320 tx=0x7fe99c0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:06.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.395+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe9a4054230 con 0x7fe9b40b0700 2024-02-02T13:30:06.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:06.587+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9b40ae410 con 0x7fe9b40b0700 2024-02-02T13:30:07.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.578+0000 7fe9ab400640 1 -- 172.21.15.91:0/2198048168 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe9a401f0d0 con 0x7fe9b40b0700 2024-02-02T13:30:07.583 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:30:07.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.584+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 msgr2=0x7fe98c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.584+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 0x7fe98c06c8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c2320 tx=0x7fe99c0079b0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.584+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 msgr2=0x7fe9b414f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.584+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c7f0 tx=0x7fe9a400ccb0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 shutdown_connections 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98c06a430 0x7fe98c06c8f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9b415a010 0x7fe9b40c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b40ce130 0x7fe9b4157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 --2- 172.21.15.91:0/2198048168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9b40b0700 0x7fe9b414f4b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 >> 172.21.15.91:0/2198048168 conn(0x7fe9b40cec80 msgr2=0x7fe9b40ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 shutdown_connections 2024-02-02T13:30:07.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.585+0000 7fe9b9600640 1 -- 172.21.15.91:0/2198048168 wait complete. 2024-02-02T13:30:07.605 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:30:07.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.803+0000 7f40b3c00640 1 Processor -- start 2024-02-02T13:30:07.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.803+0000 7f40b3c00640 1 -- start start 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b3c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f40ac06d180 0x7f40ac164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b3c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac0cace0 con 0x7f40ac159ea0 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b3c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac06aa70 con 0x7f40ac15a280 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b3c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0cf190 con 0x7f40ac06d180 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b1200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f40ac159ea0 0x7f40ac069c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39458/0 (socket says 172.21.15.91:39458) 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b1200640 1 -- 172.21.15.91:0/1187667193 learned_addr learned my addr 172.21.15.91:0/1187667193 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:07.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.805+0000 7f40b1c00640 1 --2- 172.21.15.91:0/1187667193 >> v2:172.21.15.91:3300/0 conn(0x7f40ac06d180 0x7f40ac164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:07.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.806+0000 7f40ab400640 1 -- 172.21.15.91:0/1187667193 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932664610 0 0) 0x7f40ac0cace0 con 0x7f40ac159ea0 2024-02-02T13:30:07.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.806+0000 7f40ab400640 1 -- 172.21.15.91:0/1187667193 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f408c005730 con 0x7f40ac159ea0 2024-02-02T13:30:07.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.806+0000 7f40b1c00640 1 -- 172.21.15.91:0/1187667193 >> v1:172.21.15.113:6789/0 conn(0x7f40ac15a280 legacy=0x7f40ac06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:07.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.806+0000 7f40b1c00640 1 -- 172.21.15.91:0/1187667193 >> v1:172.21.15.91:6789/0 conn(0x7f40ac159ea0 legacy=0x7f40ac069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.806+0000 7f40b1c00640 1 -- 172.21.15.91:0/1187667193 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0cc120 con 0x7f40ac06d180 2024-02-02T13:30:07.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.807+0000 7f40b1c00640 1 --2- 172.21.15.91:0/1187667193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac164c60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009dc0 tx=0x7f40a402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b37bd7204322c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:07.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.807+0000 7f40ab400640 1 -- 172.21.15.91:0/1187667193 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a402cc40 con 0x7f40ac06d180 2024-02-02T13:30:07.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.807+0000 7f40ab400640 1 -- 172.21.15.91:0/1187667193 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a402cda0 con 0x7f40ac06d180 2024-02-02T13:30:07.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.807+0000 7f40ab400640 1 -- 172.21.15.91:0/1187667193 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4034660 con 0x7f40ac06d180 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 -- 172.21.15.91:0/1187667193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 msgr2=0x7f40ac164c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 --2- 172.21.15.91:0/1187667193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac164c60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009dc0 tx=0x7f40a402be90 comp rx=0 tx=0).stop 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 -- 172.21.15.91:0/1187667193 shutdown_connections 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 --2- 172.21.15.91:0/1187667193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac164c60 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 -- 172.21.15.91:0/1187667193 >> 172.21.15.91:0/1187667193 conn(0x7f40ac0cfd40 msgr2=0x7f40ac0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 -- 172.21.15.91:0/1187667193 shutdown_connections 2024-02-02T13:30:07.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.808+0000 7f40b3c00640 1 -- 172.21.15.91:0/1187667193 wait complete. 2024-02-02T13:30:07.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.810+0000 7f40b3c00640 1 Processor -- start 2024-02-02T13:30:07.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.810+0000 7f40b3c00640 1 -- start start 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac159ea0 0x7f40ac1f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac15a280 0x7f40ac0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1f34f0 con 0x7f40ac06d180 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1f37d0 con 0x7f40ac15a280 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0c6fb0 con 0x7f40ac159ea0 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b1200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b1200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36322/0 (socket says 172.21.15.91:36322) 2024-02-02T13:30:07.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.811+0000 7f40b1200640 1 -- 172.21.15.91:0/3159340341 learned_addr learned my addr 172.21.15.91:0/3159340341 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:07.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 -- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac159ea0 msgr2=0x7f40ac1f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac15a280 0x7f40ac0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac159ea0 0x7f40ac1f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 -- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac15a280 msgr2=0x7f40ac0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac15a280 0x7f40ac0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 -- 172.21.15.91:0/3159340341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009a70 con 0x7f40ac06d180 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b1200640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c500 tx=0x7f409c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c005ce0 con 0x7f40ac06d180 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c005e40 con 0x7f40ac06d180 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0c7290 con 0x7f40ac06d180 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.812+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c007330 con 0x7f40ac06d180 2024-02-02T13:30:07.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.813+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac15c9f0 con 0x7f40ac06d180 2024-02-02T13:30:07.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.814+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac0c74d0 con 0x7f40ac06d180 2024-02-02T13:30:07.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.815+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f409c007550 con 0x7f40ac06d180 2024-02-02T13:30:07.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.816+0000 7f40a8c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 0x7f408406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:07.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.816+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f409c087e10 con 0x7f40ac06d180 2024-02-02T13:30:07.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.816+0000 7f40abe00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 0x7f408406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:07.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.817+0000 7f40abe00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 0x7f408406c8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:07.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:07.820+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f40ac0c74d0 con 0x7f40ac06d180 2024-02-02T13:30:08.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.008+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40ac0c74d0 con 0x7f40ac06d180 2024-02-02T13:30:08.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.589+0000 7f40a8c00640 1 -- 172.21.15.91:0/3159340341 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f40ac0c74d0 con 0x7f40ac06d180 2024-02-02T13:30:08.591 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 msgr2=0x7f408406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 0x7f408406c8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094006bc0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 msgr2=0x7f40ac1564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c500 tx=0x7f409c00c9c0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 shutdown_connections 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f408406a430 0x7f408406c8f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac159ea0 0x7f40ac1f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac15a280 0x7f40ac0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 --2- 172.21.15.91:0/3159340341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40ac06d180 0x7f40ac1564d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.593+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 >> 172.21.15.91:0/3159340341 conn(0x7f40ac0cfd40 msgr2=0x7f40ac0b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.594+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 shutdown_connections 2024-02-02T13:30:08.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.594+0000 7f40b3c00640 1 -- 172.21.15.91:0/3159340341 wait complete. 2024-02-02T13:30:08.615 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:30:08.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.824+0000 7f4532800640 1 Processor -- start 2024-02-02T13:30:08.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.824+0000 7f4532800640 1 -- start start 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.825+0000 7f4532800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f452c0b0b90 0x7f452c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.825+0000 7f4532800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c06b980 con 0x7f452c0b0700 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.825+0000 7f4532800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c06bc60 con 0x7f452c159840 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.825+0000 7f4532800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c06bf40 con 0x7f452c0b0b90 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f4522800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f452c0b0700 0x7f452c0c9b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39470/0 (socket says 172.21.15.91:39470) 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f452c0b0b90 0x7f452c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:08.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f4522800640 1 -- 172.21.15.91:0/1593233647 learned_addr learned my addr 172.21.15.91:0/1593233647 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452aa00640 1 -- 172.21.15.91:0/1593233647 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7006769 0 0) 0x7f452c06b980 con 0x7f452c0b0700 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452aa00640 1 -- 172.21.15.91:0/1593233647 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c003520 con 0x7f452c0b0700 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452be00640 1 -- 172.21.15.91:0/1593233647 >> v1:172.21.15.113:6789/0 conn(0x7f452c159840 legacy=0x7f452c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452be00640 1 -- 172.21.15.91:0/1593233647 >> v1:172.21.15.91:6789/0 conn(0x7f452c0b0700 legacy=0x7f452c0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.826+0000 7f452be00640 1 -- 172.21.15.91:0/1593233647 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c06c220 con 0x7f452c0b0b90 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.827+0000 7f452be00640 1 --2- 172.21.15.91:0/1593233647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c0b0b90 0x7f452c0ccbc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4524009dc0 tx=0x7f452402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7854877909c3d7a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:08.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.827+0000 7f452aa00640 1 -- 172.21.15.91:0/1593233647 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452402cc40 con 0x7f452c0b0b90 2024-02-02T13:30:08.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f452aa00640 1 -- 172.21.15.91:0/1593233647 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452402cda0 con 0x7f452c0b0b90 2024-02-02T13:30:08.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f452aa00640 1 -- 172.21.15.91:0/1593233647 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524034660 con 0x7f452c0b0b90 2024-02-02T13:30:08.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 -- 172.21.15.91:0/1593233647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c0b0b90 msgr2=0x7f452c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 --2- 172.21.15.91:0/1593233647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c0b0b90 0x7f452c0ccbc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4524009dc0 tx=0x7f452402be90 comp rx=0 tx=0).stop 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 -- 172.21.15.91:0/1593233647 shutdown_connections 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 --2- 172.21.15.91:0/1593233647 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c0b0b90 0x7f452c0ccbc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 -- 172.21.15.91:0/1593233647 >> 172.21.15.91:0/1593233647 conn(0x7f452c0ceb90 msgr2=0x7f452c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 -- 172.21.15.91:0/1593233647 shutdown_connections 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.828+0000 7f4532800640 1 -- 172.21.15.91:0/1593233647 wait complete. 2024-02-02T13:30:08.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.830+0000 7f4532800640 1 Processor -- start 2024-02-02T13:30:08.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.830+0000 7f4532800640 1 -- start start 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 0x7f452c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 0x7f452c13d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0c8ac0 con 0x7f452c159840 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c147300 con 0x7f452c0b0b90 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f4532800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c146900 con 0x7f452c0b0700 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f452be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f452be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36336/0 (socket says 172.21.15.91:36336) 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.831+0000 7f452be00640 1 -- 172.21.15.91:0/1285850371 learned_addr learned my addr 172.21.15.91:0/1285850371 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:08.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4522800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 0x7f452c13d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452b400640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 0x7f452c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 -- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 msgr2=0x7f452c0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 0x7f452c0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 -- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 msgr2=0x7f452c13d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 0x7f452c13d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 -- 172.21.15.91:0/1285850371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009a70 con 0x7f452c159840 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4522800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 0x7f452c13d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452b400640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 0x7f452c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f452be00640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4524009dc0 tx=0x7f4524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452403e610 con 0x7f452c159840 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452403e770 con 0x7f452c159840 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0c87e0 con 0x7f452c159840 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.832+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452403eb20 con 0x7f452c159840 2024-02-02T13:30:08.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.833+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c146f90 con 0x7f452c159840 2024-02-02T13:30:08.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.834+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c147490 con 0x7f452c159840 2024-02-02T13:30:08.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.834+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f452403b050 con 0x7f452c159840 2024-02-02T13:30:08.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.835+0000 7f4523e00640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 0x7f450406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:08.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.835+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f45240aaff0 con 0x7f452c159840 2024-02-02T13:30:08.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.836+0000 7f452b400640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 0x7f450406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:08.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.836+0000 7f452b400640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 0x7f450406c8f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f451c003eb0 tx=0x7f451c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:08.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:08.840+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4524076ac0 con 0x7f452c159840 2024-02-02T13:30:09.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.031+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f452c066cf0 con 0x7f452c159840 2024-02-02T13:30:09.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.037+0000 7f4523e00640 1 -- 172.21.15.91:0/1285850371 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4524009450 con 0x7f452c159840 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.039+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 msgr2=0x7f450406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.039+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 0x7f450406c8f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f451c003eb0 tx=0x7f451c009040 comp rx=0 tx=0).stop 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.039+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 msgr2=0x7f452c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.039+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4524009dc0 tx=0x7f4524004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 shutdown_connections 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f450406a430 0x7f450406c8f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f452c159840 0x7f452c0c4e30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f452c0b0b90 0x7f452c13d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 --2- 172.21.15.91:0/1285850371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f452c0b0700 0x7f452c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 >> 172.21.15.91:0/1285850371 conn(0x7f452c0ceb90 msgr2=0x7f452c14cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:09.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 shutdown_connections 2024-02-02T13:30:09.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.040+0000 7f4532800640 1 -- 172.21.15.91:0/1285850371 wait complete. 2024-02-02T13:30:09.060 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-02-02T13:30:09.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.260+0000 7f33cf000640 1 Processor -- start 2024-02-02T13:30:09.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.261+0000 7f33cf000640 1 -- start start 2024-02-02T13:30:09.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.262+0000 7f33cf000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.262+0000 7f33cf000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f33c80d2760 0x7f33c80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.262+0000 7f33cf000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80d12c0 con 0x7f33c80ce070 2024-02-02T13:30:09.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.262+0000 7f33cf000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80cea30 con 0x7f33c80d2760 2024-02-02T13:30:09.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.262+0000 7f33cf000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80cded0 con 0x7f33c815d650 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.263+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.263+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38978/0 (socket says 172.21.15.91:38978) 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.263+0000 7f33c7e00640 1 -- 172.21.15.91:0/1513915360 learned_addr learned my addr 172.21.15.91:0/1513915360 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.263+0000 7f33cd000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f33c80d2760 0x7f33c80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.264+0000 7f33c7e00640 1 -- 172.21.15.91:0/1513915360 >> v2:172.21.15.113:3300/0 conn(0x7f33c80d2760 msgr2=0x7f33c80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:09.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.264+0000 7f33c7e00640 1 --2- 172.21.15.91:0/1513915360 >> v2:172.21.15.113:3300/0 conn(0x7f33c80d2760 0x7f33c80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.264+0000 7f33c7e00640 1 -- 172.21.15.91:0/1513915360 >> v1:172.21.15.113:6789/0 conn(0x7f33c80ce070 legacy=0x7f33c81610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.264+0000 7f33c7e00640 1 -- 172.21.15.91:0/1513915360 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c8071410 con 0x7f33c815d650 2024-02-02T13:30:09.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.265+0000 7f33c7e00640 1 --2- 172.21.15.91:0/1513915360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009870 tx=0x7f33b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec4f777cf92a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.266+0000 7f33c6a00640 1 -- 172.21.15.91:0/1513915360 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8030840 con 0x7f33c815d650 2024-02-02T13:30:09.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.266+0000 7f33c6a00640 1 -- 172.21.15.91:0/1513915360 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b80309a0 con 0x7f33c815d650 2024-02-02T13:30:09.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.266+0000 7f33c6a00640 1 -- 172.21.15.91:0/1513915360 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8030d30 con 0x7f33c815d650 2024-02-02T13:30:09.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.266+0000 7f33cf000640 1 -- 172.21.15.91:0/1513915360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 msgr2=0x7f33c80cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.266+0000 7f33cf000640 1 --2- 172.21.15.91:0/1513915360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009870 tx=0x7f33b802b980 comp rx=0 tx=0).stop 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 -- 172.21.15.91:0/1513915360 shutdown_connections 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 --2- 172.21.15.91:0/1513915360 >> v2:172.21.15.113:3300/0 conn(0x7f33c80d2760 0x7f33c80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 --2- 172.21.15.91:0/1513915360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c815d650 0x7f33c80cd990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 -- 172.21.15.91:0/1513915360 >> 172.21.15.91:0/1513915360 conn(0x7f33c80d30a0 msgr2=0x7f33c80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 -- 172.21.15.91:0/1513915360 shutdown_connections 2024-02-02T13:30:09.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.267+0000 7f33cf000640 1 -- 172.21.15.91:0/1513915360 wait complete. 2024-02-02T13:30:09.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.269+0000 7f33cf000640 1 Processor -- start 2024-02-02T13:30:09.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.269+0000 7f33cf000640 1 -- start start 2024-02-02T13:30:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c80ce070 0x7f33c815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 0x7f33c80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c815ac90 con 0x7f33c80d2760 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b2ff0 con 0x7f33c815d650 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cf000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b3890 con 0x7f33c80ce070 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33c7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33c7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36342/0 (socket says 172.21.15.91:36342) 2024-02-02T13:30:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33c7400640 1 -- 172.21.15.91:0/2222065512 learned_addr learned my addr 172.21.15.91:0/2222065512 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:09.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33c7e00640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c80ce070 0x7f33c815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.270+0000 7f33cd000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 0x7f33c80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 -- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c80ce070 msgr2=0x7f33c815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c80ce070 0x7f33c815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 -- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 msgr2=0x7f33c80cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 0x7f33c80cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 -- 172.21.15.91:0/2222065512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80094d0 con 0x7f33c80d2760 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33cd000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 0x7f33c80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33c7400640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c3f0 tx=0x7f33b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00052d0 con 0x7f33c80d2760 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0005450 con 0x7f33c80d2760 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0007260 con 0x7f33c80d2760 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c80cb5d0 con 0x7f33c80d2760 2024-02-02T13:30:09.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.271+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c80b3520 con 0x7f33c80d2760 2024-02-02T13:30:09.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.273+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8071a30 con 0x7f33c80d2760 2024-02-02T13:30:09.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.273+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f33b00073c0 con 0x7f33c80d2760 2024-02-02T13:30:09.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.274+0000 7f33bfe00640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 0x7f33a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.274+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f33b0087a20 con 0x7f33c80d2760 2024-02-02T13:30:09.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.274+0000 7f33c7e00640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 0x7f33a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.275+0000 7f33c7e00640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 0x7f33a006c8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f33b8036000 tx=0x7f33b8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.278+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33b0053270 con 0x7f33c80d2760 2024-02-02T13:30:09.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.546+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f33c80b36e0 con 0x7f33c80d2760 2024-02-02T13:30:09.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.589+0000 7f33bfe00640 1 -- 172.21.15.91:0/2222065512 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v285) v1 ==== 133+0+0 (secure 0 0 0) 0x7f33b0056890 con 0x7f33c80d2760 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 msgr2=0x7f33a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 0x7f33a006c8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f33b8036000 tx=0x7f33b8008030 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 msgr2=0x7f33c81f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c3f0 tx=0x7f33b000c8b0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 shutdown_connections 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33a006a430 0x7f33a006c8f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c815d650 0x7f33c80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c80d2760 0x7f33c81f3660 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 --2- 172.21.15.91:0/2222065512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c80ce070 0x7f33c815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.592+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 >> 172.21.15.91:0/2222065512 conn(0x7f33c80d30a0 msgr2=0x7f33c80c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:09.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.593+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 shutdown_connections 2024-02-02T13:30:09.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.593+0000 7f33cf000640 1 -- 172.21.15.91:0/2222065512 wait complete. 2024-02-02T13:30:09.612 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-02-02T13:30:09.612 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-02-02T13:30:09.612 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-02-02T13:30:09.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.805+0000 7fc446000640 1 Processor -- start 2024-02-02T13:30:09.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.806+0000 7fc446000640 1 -- start start 2024-02-02T13:30:09.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc446000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc440151f90 0x7fc44015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc446000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 0x7fc4400b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc446000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc440070f80 con 0x7fc4400b0c40 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc446000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4400b10d0 con 0x7fc440151f90 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc446000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc440152370 con 0x7fc44006f4f0 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc43fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc4400b0c40 0x7fc4400b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39484/0 (socket says 172.21.15.91:39484) 2024-02-02T13:30:09.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.807+0000 7fc43fe00640 1 -- 172.21.15.91:0/860747165 learned_addr learned my addr 172.21.15.91:0/860747165 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084884243 0 0) 0x7fc440070f80 con 0x7fc4400b0c40 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc420003520 con 0x7fc4400b0c40 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43f400640 1 --2- 172.21.15.91:0/860747165 >> v2:172.21.15.113:3300/0 conn(0x7fc440151f90 0x7fc44015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc436800640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 0x7fc4400b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108498068 0 0) 0x7fc420003520 con 0x7fc4400b0c40 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 msgr2=0x7fc4400b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 0x7fc4400b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 >> v2:172.21.15.113:3300/0 conn(0x7fc440151f90 msgr2=0x7fc44015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 --2- 172.21.15.91:0/860747165 >> v2:172.21.15.113:3300/0 conn(0x7fc440151f90 0x7fc44015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400cb010 con 0x7fc4400b0c40 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc4380019f0 con 0x7fc4400b0c40 2024-02-02T13:30:09.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 >> v1:172.21.15.91:6789/0 conn(0x7fc4400b0c40 legacy=0x7fc4400b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420003970 0x7fc420003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 0x7fc4200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc420004d00 0x7fc420005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4400b10d0 con 0x7fc420004d00 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440152370 con 0x7fc420003970 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42000ba30 con 0x7fc44006f4f0 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.808+0000 7fc43f400640 1 --2- 172.21.15.91:0/860747165 >> v2:172.21.15.113:3300/0 conn(0x7fc440151f90 0x7fc44015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc420004d00 0x7fc420005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 -- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 msgr2=0x7fc4200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44006f4f0 0x7fc4200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 -- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420003970 msgr2=0x7fc420003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420003970 0x7fc420003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.809+0000 7fc43fe00640 1 -- 172.21.15.91:0/860747165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420003520 con 0x7fc420004d00 2024-02-02T13:30:09.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.810+0000 7fc43fe00640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc420004d00 0x7fc420005180 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc43800a790 tx=0x7fc4200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a227821917130d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.810+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438034630 con 0x7fc420004d00 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.810+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43802f070 con 0x7fc420004d00 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.810+0000 7fc43ea00640 1 -- 172.21.15.91:0/860747165 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380349f0 con 0x7fc420004d00 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 -- 172.21.15.91:0/860747165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc420004d00 msgr2=0x7fc420005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 --2- 172.21.15.91:0/860747165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc420004d00 0x7fc420005180 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc43800a790 tx=0x7fc4200057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc43fe00640 1 -- 172.21.15.91:0/860747165 reap_dead start 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 -- 172.21.15.91:0/860747165 shutdown_connections 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 -- 172.21.15.91:0/860747165 >> 172.21.15.91:0/860747165 conn(0x7fc4400cec80 msgr2=0x7fc4400cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 -- 172.21.15.91:0/860747165 shutdown_connections 2024-02-02T13:30:09.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.811+0000 7fc446000640 1 -- 172.21.15.91:0/860747165 wait complete. 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.813+0000 7fc446000640 1 Processor -- start 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.813+0000 7fc446000640 1 -- start start 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.813+0000 7fc446000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 0x7fc4400c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.813+0000 7fc446000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc440157c10 0x7fc44014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc446000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc446000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015d650 con 0x7fc44015d960 2024-02-02T13:30:09.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc446000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015c9d0 con 0x7fc420004d00 2024-02-02T13:30:09.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc446000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015ccb0 con 0x7fc440157c10 2024-02-02T13:30:09.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:36360/0 (socket says 172.21.15.91:36360) 2024-02-02T13:30:09.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 -- 172.21.15.91:0/1441971836 learned_addr learned my addr 172.21.15.91:0/1441971836 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:09.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc436800640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc440157c10 0x7fc44014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43f400640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 0x7fc4400c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 -- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc440157c10 msgr2=0x7fc44014e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc440157c10 0x7fc44014e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 -- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 msgr2=0x7fc4400c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 0x7fc4400c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43fe00640 1 -- 172.21.15.91:0/1441971836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44015cf90 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.814+0000 7fc43f400640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 0x7fc4400c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc43fe00640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc43802cad0 tx=0x7fc438001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438034640 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380347a0 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438034b10 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44015d270 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.815+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4400c8320 con 0x7fc44015d960 2024-02-02T13:30:09.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.816+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4400b1f80 con 0x7fc44015d960 2024-02-02T13:30:09.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.817+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc438031030 con 0x7fc44015d960 2024-02-02T13:30:09.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.818+0000 7fc437e00640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 0x7fc41806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:09.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.818+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc4380aa6d0 con 0x7fc44015d960 2024-02-02T13:30:09.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.818+0000 7fc43f400640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 0x7fc41806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:09.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.819+0000 7fc43f400640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 0x7fc41806c8f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc43000d940 tx=0x7fc430008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:09.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:09.823+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc438075fb0 con 0x7fc44015d960 2024-02-02T13:30:10.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.067+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7fc4400c8320 con 0x7fc44015d960 2024-02-02T13:30:10.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.599+0000 7fc437e00640 1 -- 172.21.15.91:0/1441971836 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v286) v1 ==== 140+0+0 (secure 0 0 0) 0x7fc4400c8320 con 0x7fc44015d960 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 msgr2=0x7fc41806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 0x7fc41806c8f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc43000d940 tx=0x7fc430008040 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 msgr2=0x7fc44014e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc43802cad0 tx=0x7fc438001cb0 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 shutdown_connections 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc41806a430 0x7fc41806c8f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc440157c10 0x7fc44014e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc44015d960 0x7fc44014e700 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 --2- 172.21.15.91:0/1441971836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc420004d00 0x7fc4400c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 >> 172.21.15.91:0/1441971836 conn(0x7fc4400cec80 msgr2=0x7fc44015a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 shutdown_connections 2024-02-02T13:30:10.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:10.603+0000 7fc446000640 1 -- 172.21.15.91:0/1441971836 wait complete. 2024-02-02T13:30:10.626 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-02-02T13:30:10.836 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_objectstore":"memstore"} 2024-02-02T13:30:10.836 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-02-02T13:30:10.837 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:30:11.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.046+0000 7f69b0600640 1 Processor -- start 2024-02-02T13:30:11.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.046+0000 7f69b0600640 1 -- start start 2024-02-02T13:30:11.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69b0600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69a815f5a0 0x7f69a80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69b0600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f69a80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69b0600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a8070f80 con 0x7f69a8151f90 2024-02-02T13:30:11.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69b0600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a815fa30 con 0x7f69a815f5a0 2024-02-02T13:30:11.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69b0600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a80b10b0 con 0x7f69a80b0c40 2024-02-02T13:30:11.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69adc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f69a8151f90 0x7f69a806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56846/0 (socket says 172.21.15.91:56846) 2024-02-02T13:30:11.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69adc00640 1 -- 172.21.15.91:0/4199243884 learned_addr learned my addr 172.21.15.91:0/4199243884 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69ad200640 1 --2- 172.21.15.91:0/4199243884 >> v2:172.21.15.113:3300/0 conn(0x7f69a815f5a0 0x7f69a80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452657659 0 0) 0x7f69a8070f80 con 0x7f69a8151f90 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6988003520 con 0x7f69a8151f90 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.048+0000 7f69ae600640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f69a80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4211262139 0 0) 0x7f6988003520 con 0x7f69a8151f90 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 msgr2=0x7f69a80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f69a80b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 >> v2:172.21.15.113:3300/0 conn(0x7f69a815f5a0 msgr2=0x7f69a80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 --2- 172.21.15.91:0/4199243884 >> v2:172.21.15.113:3300/0 conn(0x7f69a815f5a0 0x7f69a80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8152370 con 0x7f69a8151f90 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6998001670 con 0x7f69a8151f90 2024-02-02T13:30:11.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69ae600640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f69a80b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 >> v1:172.21.15.91:6789/0 conn(0x7f69a8151f90 legacy=0x7f69a806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6988003970 0x7f6988003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69880047c0 0x7f6988004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f6988005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a815fa30 con 0x7f6988003970 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a80b10b0 con 0x7f69880047c0 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698800b980 con 0x7f69a80b0c40 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.049+0000 7f69adc00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6988003970 0x7f6988003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69ad200640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69880047c0 0x7f6988004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69adc00640 1 -- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 msgr2=0x7f6988005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69adc00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a80b0c40 0x7f6988005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69adc00640 1 -- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69880047c0 msgr2=0x7f6988004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69adc00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69880047c0 0x7f6988004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.050+0000 7f69adc00640 1 -- 172.21.15.91:0/4199243884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988003520 con 0x7f6988003970 2024-02-02T13:30:11.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.051+0000 7f69adc00640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6988003970 0x7f6988003d90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f699800c270 tx=0x7f69880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599ebc042257535 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.051+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699802cae0 con 0x7f6988003970 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.051+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699802cc40 con 0x7f6988003970 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.051+0000 7f69a7e00640 1 -- 172.21.15.91:0/4199243884 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69980058d0 con 0x7f6988003970 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 -- 172.21.15.91:0/4199243884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6988003970 msgr2=0x7f6988003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 --2- 172.21.15.91:0/4199243884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6988003970 0x7f6988003d90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f699800c270 tx=0x7f69880057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69ae600640 1 -- 172.21.15.91:0/4199243884 reap_dead start 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 -- 172.21.15.91:0/4199243884 shutdown_connections 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 -- 172.21.15.91:0/4199243884 >> 172.21.15.91:0/4199243884 conn(0x7f69a80cec80 msgr2=0x7f69a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 -- 172.21.15.91:0/4199243884 shutdown_connections 2024-02-02T13:30:11.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.052+0000 7f69b0600640 1 -- 172.21.15.91:0/4199243884 wait complete. 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.053+0000 7f69b0600640 1 Processor -- start 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.054+0000 7f69b0600640 1 -- start start 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.054+0000 7f69b0600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003970 0x7f69a814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.054+0000 7f69b0600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69b0600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a80c6be0 0x7f69a80c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69b0600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a80c75a0 con 0x7f69a80cd880 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69b0600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a80ac8d0 con 0x7f69a80c6be0 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69b0600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a80cdd80 con 0x7f6988003970 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56136/0 (socket says 172.21.15.91:56136) 2024-02-02T13:30:11.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 -- 172.21.15.91:0/1933809367 learned_addr learned my addr 172.21.15.91:0/1933809367 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 -- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003970 msgr2=0x7f69a814d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003970 0x7f69a814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 -- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a80c6be0 msgr2=0x7f69a80c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a80c6be0 0x7f69a80c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 -- 172.21.15.91:0/1933809367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a80ce060 con 0x7f69a80cd880 2024-02-02T13:30:11.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.055+0000 7f69ad200640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f69900108e0 tx=0x7f6990010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.056+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000eb60 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.056+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699000ecc0 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.056+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699001c400 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.056+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a80ce340 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.056+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a814d640 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.057+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a8066cf0 con 0x7f69a80cd880 2024-02-02T13:30:11.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.058+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f699001c560 con 0x7f69a80cd880 2024-02-02T13:30:11.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.059+0000 7f69a5600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 0x7f698006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.059+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6990088ce0 con 0x7f69a80cd880 2024-02-02T13:30:11.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.059+0000 7f69adc00640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 0x7f698006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.060+0000 7f69adc00640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 0x7f698006c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6998034000 tx=0x7f699802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.063+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f69900545e0 con 0x7f69a80cd880 2024-02-02T13:30:11.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.245+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f69a8159890 con 0x7f69a80cd880 2024-02-02T13:30:11.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.251+0000 7f69a5600640 1 -- 172.21.15.91:0/1933809367 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6990057c00 con 0x7f69a80cd880 2024-02-02T13:30:11.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 msgr2=0x7f698006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 0x7f698006c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6998034000 tx=0x7f699802e000 comp rx=0 tx=0).stop 2024-02-02T13:30:11.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 msgr2=0x7f69a80c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f69900108e0 tx=0x7f6990010da0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 shutdown_connections 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f698006a430 0x7f698006c8f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a80c6be0 0x7f69a80c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69a80cd880 0x7f69a80c6300 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 --2- 172.21.15.91:0/1933809367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003970 0x7f69a814d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.253+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 >> 172.21.15.91:0/1933809367 conn(0x7f69a80cec80 msgr2=0x7f69a815a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.254+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 shutdown_connections 2024-02-02T13:30:11.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.254+0000 7f69b0600640 1 -- 172.21.15.91:0/1933809367 wait complete. 2024-02-02T13:30:11.270 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:30:11.271 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:30:11.374 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:30:11.492 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:30:11.493 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:11.493 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:11.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:30:11.549 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:11.549 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:11.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:30:11.606 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:11.606 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:11.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:30:11.662 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:11.662 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:11.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:30:11.719 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:30:11.719 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.934+0000 7f901ca00640 1 Processor -- start 2024-02-02T13:30:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.935+0000 7f901ca00640 1 -- start start 2024-02-02T13:30:11.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f901ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f901415a010 0x7f90140c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f901ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f90140ce130 0x7f90140ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f901ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140cc7c0 con 0x7f90140cd470 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f901ca00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90140ce930 con 0x7f90140ce130 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f901ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901406c890 con 0x7f901415a010 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f9019600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f90140ce130 0x7f90140ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f9019600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f90140ce130 0x7f90140ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56150/0 (socket says 172.21.15.91:56150) 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.936+0000 7f9019600640 1 -- 172.21.15.91:0/2499943520 learned_addr learned my addr 172.21.15.91:0/2499943520 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.937+0000 7f9019600640 1 -- 172.21.15.91:0/2499943520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f901415a010 msgr2=0x7f90140c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.937+0000 7f9019600640 1 --2- 172.21.15.91:0/2499943520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f901415a010 0x7f90140c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.937+0000 7f9019600640 1 -- 172.21.15.91:0/2499943520 >> v1:172.21.15.113:6789/0 conn(0x7f90140cd470 legacy=0x7f90140b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.937+0000 7f9019600640 1 -- 172.21.15.91:0/2499943520 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901406cb70 con 0x7f90140ce130 2024-02-02T13:30:11.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.938+0000 7f9019600640 1 --2- 172.21.15.91:0/2499943520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90140ce130 0x7f90140ccf30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009870 tx=0x7f8ffc02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56eb1d17efdb10b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.938+0000 7f9018c00640 1 -- 172.21.15.91:0/2499943520 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc02f8c0 con 0x7f90140ce130 2024-02-02T13:30:11.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.938+0000 7f9018c00640 1 -- 172.21.15.91:0/2499943520 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc02fa20 con 0x7f90140ce130 2024-02-02T13:30:11.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.938+0000 7f9018c00640 1 -- 172.21.15.91:0/2499943520 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc02fdb0 con 0x7f90140ce130 2024-02-02T13:30:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 -- 172.21.15.91:0/2499943520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90140ce130 msgr2=0x7f90140ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 --2- 172.21.15.91:0/2499943520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90140ce130 0x7f90140ccf30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009870 tx=0x7f8ffc02db00 comp rx=0 tx=0).stop 2024-02-02T13:30:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 -- 172.21.15.91:0/2499943520 shutdown_connections 2024-02-02T13:30:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 --2- 172.21.15.91:0/2499943520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90140ce130 0x7f90140ccf30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 --2- 172.21.15.91:0/2499943520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f901415a010 0x7f90140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 -- 172.21.15.91:0/2499943520 >> 172.21.15.91:0/2499943520 conn(0x7f90140cec80 msgr2=0x7f90140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:11.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 -- 172.21.15.91:0/2499943520 shutdown_connections 2024-02-02T13:30:11.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.939+0000 7f901ca00640 1 -- 172.21.15.91:0/2499943520 wait complete. 2024-02-02T13:30:11.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.941+0000 7f901ca00640 1 Processor -- start 2024-02-02T13:30:11.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.941+0000 7f901ca00640 1 -- start start 2024-02-02T13:30:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 0x7f90140c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 0x7f9014156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140ac8d0 con 0x7f901415a010 2024-02-02T13:30:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140b3b20 con 0x7f90140ce130 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140acb20 con 0x7f90141f6af0 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56160/0 (socket says 172.21.15.91:56160) 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 -- 172.21.15.91:0/784590803 learned_addr learned my addr 172.21.15.91:0/784590803 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 -- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 msgr2=0x7f9014156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901aa00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 0x7f9014156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 0x7f9014156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f9019600640 1 -- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 msgr2=0x7f90140c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.942+0000 7f901a000640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 0x7f90140c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9019600640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 0x7f90140c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9019600640 1 -- 172.21.15.91:0/784590803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0094d0 con 0x7f901415a010 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f901aa00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 0x7f9014156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:11.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f901a000640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 0x7f90140c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9019600640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc037930 con 0x7f901415a010 2024-02-02T13:30:11.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc037ab0 con 0x7f901415a010 2024-02-02T13:30:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc037e60 con 0x7f901415a010 2024-02-02T13:30:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90140ace00 con 0x7f901415a010 2024-02-02T13:30:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.943+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90140b37f0 con 0x7f901415a010 2024-02-02T13:30:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.945+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9014066cf0 con 0x7f901415a010 2024-02-02T13:30:11.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.945+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ffc042470 con 0x7f901415a010 2024-02-02T13:30:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.946+0000 7f9012000640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 0x7f8fec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.946+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ffc033070 con 0x7f901415a010 2024-02-02T13:30:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.946+0000 7f901a000640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 0x7f8fec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:11.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.947+0000 7f901a000640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 0x7f8fec06c920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9004007eb0 tx=0x7f90040073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:11.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:11.951+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ffc076cb0 con 0x7f901415a010 2024-02-02T13:30:12.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.163+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f901406ce20 con 0x7f901415a010 2024-02-02T13:30:12.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.165+0000 7f9012000640 1 -- 172.21.15.91:0/784590803 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f8ffc046de0 con 0x7f901415a010 2024-02-02T13:30:12.166 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:12.166 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1698040705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1698040705},{"type":"v1","addr":"172.21.15.113:6833","nonce":1698040705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1287450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1287450491},{"type":"v1","addr":"172.21.15.113:6835","nonce":1287450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/3043180991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3043180991},{"type":"v1","addr":"172.21.15.91:6835","nonce":3043180991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2379501282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":2379501282},{"type":"v1","addr":"172.21.15.91:6837","nonce":2379501282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2024-02-02T13:30:12.167 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 42 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 msgr2=0x7f8fec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 0x7f8fec06c920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9004007eb0 tx=0x7f90040073c0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 msgr2=0x7f90140b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc0046c0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 shutdown_connections 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fec06a460 0x7f8fec06c920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90141f6af0 0x7f9014156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f901415a010 0x7f90140b32b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 --2- 172.21.15.91:0/784590803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90140ce130 0x7f90140c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 >> 172.21.15.91:0/784590803 conn(0x7f90140cec80 msgr2=0x7f90141591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:12.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 shutdown_connections 2024-02-02T13:30:12.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.167+0000 7f901ca00640 1 -- 172.21.15.91:0/784590803 wait complete. 2024-02-02T13:30:12.186 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:30:12.186 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:30:12.186 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:30:12.186 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:30:12.187 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:30:12.187 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:30:12.187 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:30:12.187 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:30:12.187 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:30:12.187 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:30:12.187 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:30:12.188 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:30:12.188 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:30:12.188 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:30:12.188 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:30:12.188 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:30:12.202 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:30:12.202 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:30:12.204 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:30:12.204 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:30:12.212 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:30:12.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:30:12.214 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:30:12.214 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:30:12.251 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:30:12.251 DEBUG:teuthology.parallel:result is None 2024-02-02T13:30:12.258 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:30:12.258 DEBUG:teuthology.parallel:result is None 2024-02-02T13:30:12.260 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:30:12.261 DEBUG:teuthology.parallel:result is None 2024-02-02T13:30:12.267 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:30:12.267 DEBUG:teuthology.parallel:result is None 2024-02-02T13:30:12.267 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:30:12.406 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:30:12.406 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:30:12.429 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:30:12.431 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:30:12.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.452+0000 7f0c3ea00640 1 Processor -- start 2024-02-02T13:30:12.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.452+0000 7f0c3ea00640 1 -- start start 2024-02-02T13:30:12.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.453+0000 7f0c3ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c3815ad10 0x7f0c3806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c3ea00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3806d910 con 0x7f0c3815b0f0 2024-02-02T13:30:12.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c3ea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3806d730 con 0x7f0c3815ad10 2024-02-02T13:30:12.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c3ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380cbf50 con 0x7f0c380cb580 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0c3815b0f0 0x7f0c3806d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56848/0 (socket says 172.21.15.91:56848) 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37400640 1 -- 172.21.15.91:0/343289546 learned_addr learned my addr 172.21.15.91:0/343289546 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37e00640 1 --2- 172.21.15.91:0/343289546 >> v2:172.21.15.91:3300/0 conn(0x7f0c3815ad10 0x7f0c3806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c3ca00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37e00640 1 -- 172.21.15.91:0/343289546 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 msgr2=0x7f0c380cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:12.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37e00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37e00640 1 -- 172.21.15.91:0/343289546 >> v1:172.21.15.91:6789/0 conn(0x7f0c3815b0f0 legacy=0x7f0c3806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.454+0000 7f0c37e00640 1 -- 172.21.15.91:0/343289546 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380cc850 con 0x7f0c3815ad10 2024-02-02T13:30:12.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.455+0000 7f0c37e00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3806b7a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0c280097d0 tx=0x7f0c2802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08ba6703e8d146d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:12.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.456+0000 7f0c36a00640 1 -- 172.21.15.91:0/343289546 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28031780 con 0x7f0c3815ad10 2024-02-02T13:30:12.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.456+0000 7f0c36a00640 1 -- 172.21.15.91:0/343289546 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c280318e0 con 0x7f0c3815ad10 2024-02-02T13:30:12.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.456+0000 7f0c36a00640 1 -- 172.21.15.91:0/343289546 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28031c70 con 0x7f0c3815ad10 2024-02-02T13:30:12.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.456+0000 7f0c3ea00640 1 -- 172.21.15.91:0/343289546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 msgr2=0x7f0c3806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.456+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3806b7a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0c280097d0 tx=0x7f0c2802b820 comp rx=0 tx=0).stop 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 -- 172.21.15.91:0/343289546 shutdown_connections 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/343289546 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3806b7a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 -- 172.21.15.91:0/343289546 >> 172.21.15.91:0/343289546 conn(0x7f0c380b0700 msgr2=0x7f0c380b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 -- 172.21.15.91:0/343289546 shutdown_connections 2024-02-02T13:30:12.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.457+0000 7f0c3ea00640 1 -- 172.21.15.91:0/343289546 wait complete. 2024-02-02T13:30:12.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.459+0000 7f0c3ea00640 1 Processor -- start 2024-02-02T13:30:12.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.459+0000 7f0c3ea00640 1 -- start start 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 0x7f0c3814f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38151900 con 0x7f0c3815ad10 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381f6ae0 con 0x7f0c3815b0f0 2024-02-02T13:30:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.460+0000 7f0c3ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381f5d20 con 0x7f0c380cb580 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56198/0 (socket says 172.21.15.91:56198) 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 -- 172.21.15.91:0/2552601064 learned_addr learned my addr 172.21.15.91:0/2552601064 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c3ca00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 0x7f0c3814f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 -- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 msgr2=0x7f0c380c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 -- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 msgr2=0x7f0c3814f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 0x7f0c3814f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 -- 172.21.15.91:0/2552601064 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c28009400 con 0x7f0c3815ad10 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c3ca00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 0x7f0c3814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.461+0000 7f0c37400640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c4b0 tx=0x7f0c2000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.462+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20005ce0 con 0x7f0c3815ad10 2024-02-02T13:30:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.462+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c20005e40 con 0x7f0c3815ad10 2024-02-02T13:30:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.462+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c381f6000 con 0x7f0c3815ad10 2024-02-02T13:30:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.462+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20007270 con 0x7f0c3815ad10 2024-02-02T13:30:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.462+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c381f65c0 con 0x7f0c3815ad10 2024-02-02T13:30:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.463+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c381f67b0 con 0x7f0c3815ad10 2024-02-02T13:30:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.464+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c200073d0 con 0x7f0c3815ad10 2024-02-02T13:30:12.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.464+0000 7f0c2fe00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 0x7f0c1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:12.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.465+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0c20087a80 con 0x7f0c3815ad10 2024-02-02T13:30:12.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.465+0000 7f0c37e00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 0x7f0c1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:12.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.466+0000 7f0c37e00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 0x7f0c1006c8f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c28036000 tx=0x7f0c2803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:12.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.469+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c20053410 con 0x7f0c3815ad10 2024-02-02T13:30:12.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:12.678+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0c380ccde0 con 0x7f0c3815ad10 2024-02-02T13:30:13.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.626+0000 7f0c2fe00640 1 -- 172.21.15.91:0/2552601064 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0c20017080 con 0x7f0c3815ad10 2024-02-02T13:30:13.628 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 msgr2=0x7f0c1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 0x7f0c1006c8f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c28036000 tx=0x7f0c2803d040 comp rx=0 tx=0).stop 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 msgr2=0x7f0c3814ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c4b0 tx=0x7f0c2000c970 comp rx=0 tx=0).stop 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 shutdown_connections 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c1006a430 0x7f0c1006c8f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.629+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3815b0f0 0x7f0c3814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.630+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c3815ad10 0x7f0c3814ee10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.630+0000 7f0c3ea00640 1 --2- 172.21.15.91:0/2552601064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c380cb580 0x7f0c380c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.630+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 >> 172.21.15.91:0/2552601064 conn(0x7f0c380b0700 msgr2=0x7f0c380c2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:13.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.630+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 shutdown_connections 2024-02-02T13:30:13.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.630+0000 7f0c3ea00640 1 -- 172.21.15.91:0/2552601064 wait complete. 2024-02-02T13:30:13.651 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:30:13.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.853+0000 7f3190200640 1 Processor -- start 2024-02-02T13:30:13.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.853+0000 7f3190200640 1 -- start start 2024-02-02T13:30:13.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f3190200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31880cb580 0x7f31880ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:13.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f3190200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880c94a0 con 0x7f318815d3a0 2024-02-02T13:30:13.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f3190200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880ad6e0 con 0x7f31880ac960 2024-02-02T13:30:13.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f3190200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880ad500 con 0x7f31880cb580 2024-02-02T13:30:13.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f318d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31880cb580 0x7f31880ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f318ce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f318815d3a0 0x7f31880cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56864/0 (socket says 172.21.15.91:56864) 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.855+0000 7f318ce00640 1 -- 172.21.15.91:0/351672601 learned_addr learned my addr 172.21.15.91:0/351672601 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.856+0000 7f318d800640 1 -- 172.21.15.91:0/351672601 >> v1:172.21.15.113:6789/0 conn(0x7f31880ac960 legacy=0x7f31880acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.856+0000 7f318d800640 1 -- 172.21.15.91:0/351672601 >> v1:172.21.15.91:6789/0 conn(0x7f318815d3a0 legacy=0x7f31880cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.856+0000 7f318d800640 1 -- 172.21.15.91:0/351672601 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880c96f0 con 0x7f31880cb580 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.856+0000 7f3187e00640 1 -- 172.21.15.91:0/351672601 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490134365 0 0) 0x7f31880c94a0 con 0x7f318815d3a0 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.857+0000 7f318d800640 1 --2- 172.21.15.91:0/351672601 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31880cb580 0x7f31880ca1d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f31780097a0 tx=0x7f317802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc0ced71794d57d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.857+0000 7f3187e00640 1 -- 172.21.15.91:0/351672601 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178031830 con 0x7f31880cb580 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.857+0000 7f3187e00640 1 -- 172.21.15.91:0/351672601 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178031990 con 0x7f31880cb580 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.857+0000 7f3187e00640 1 -- 172.21.15.91:0/351672601 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178031d20 con 0x7f31880cb580 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 -- 172.21.15.91:0/351672601 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31880cb580 msgr2=0x7f31880ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 --2- 172.21.15.91:0/351672601 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31880cb580 0x7f31880ca1d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f31780097a0 tx=0x7f317802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 -- 172.21.15.91:0/351672601 shutdown_connections 2024-02-02T13:30:13.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 --2- 172.21.15.91:0/351672601 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31880cb580 0x7f31880ca1d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 -- 172.21.15.91:0/351672601 >> 172.21.15.91:0/351672601 conn(0x7f31880b0700 msgr2=0x7f31880b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:13.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 -- 172.21.15.91:0/351672601 shutdown_connections 2024-02-02T13:30:13.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.858+0000 7f3190200640 1 -- 172.21.15.91:0/351672601 wait complete. 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.859+0000 7f3190200640 1 Processor -- start 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 -- start start 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31880ac960 0x7f3188150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31880cb580 0x7f31880c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318815d7e0 con 0x7f318815d3a0 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31880b2390 con 0x7f31880ac960 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.860+0000 7f3190200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31880b2670 con 0x7f31880cb580 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56208/0 (socket says 172.21.15.91:56208) 2024-02-02T13:30:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 -- 172.21.15.91:0/874661192 learned_addr learned my addr 172.21.15.91:0/874661192 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318d800640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31880ac960 0x7f3188150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318ce00640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31880cb580 0x7f31880c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 -- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31880cb580 msgr2=0x7f31880c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31880cb580 0x7f31880c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 -- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31880ac960 msgr2=0x7f3188150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31880ac960 0x7f3188150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 -- 172.21.15.91:0/874661192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3178009400 con 0x7f318815d3a0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.861+0000 7f318e200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f318000c780 tx=0x7f318000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.862+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180004b60 con 0x7f318815d3a0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.862+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000ce90 con 0x7f318815d3a0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.862+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000a850 con 0x7f318815d3a0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.862+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31880b2950 con 0x7f318815d3a0 2024-02-02T13:30:13.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.862+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31880b2f10 con 0x7f318815d3a0 2024-02-02T13:30:13.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.863+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31880b3130 con 0x7f318815d3a0 2024-02-02T13:30:13.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.864+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f318000a9b0 con 0x7f318815d3a0 2024-02-02T13:30:13.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.865+0000 7f3185600640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 0x7f316006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:13.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.865+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f31800888b0 con 0x7f318815d3a0 2024-02-02T13:30:13.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.865+0000 7f318d800640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 0x7f316006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:13.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.866+0000 7f318d800640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 0x7f316006c8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3178036000 tx=0x7f317803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:13.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:13.870+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3180054150 con 0x7f318815d3a0 2024-02-02T13:30:14.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.129+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3188066cf0 con 0x7f318815d3a0 2024-02-02T13:30:14.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.130+0000 7f3185600640 1 -- 172.21.15.91:0/874661192 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3180057770 con 0x7f318815d3a0 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:30:14.132 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.133 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:30:14.134 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.136 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.137 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.138 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.139 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.140 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:30:14.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:30:14.143 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:14.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:14.145 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:14.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.132+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 msgr2=0x7f316006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.132+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 0x7f316006c8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3178036000 tx=0x7f317803c040 comp rx=0 tx=0).stop 2024-02-02T13:30:14.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.132+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 msgr2=0x7f31880c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.132+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f318000c780 tx=0x7f318000cc40 comp rx=0 tx=0).stop 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 shutdown_connections 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f316006a430 0x7f316006c8f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31880cb580 0x7f31880c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f318815d3a0 0x7f31880c2be0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 --2- 172.21.15.91:0/874661192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31880ac960 0x7f3188150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 >> 172.21.15.91:0/874661192 conn(0x7f31880b0700 msgr2=0x7f31880ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 shutdown_connections 2024-02-02T13:30:14.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.133+0000 7f3190200640 1 -- 172.21.15.91:0/874661192 wait complete. 2024-02-02T13:30:14.154 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:14.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.361+0000 7f4edd800640 1 Processor -- start 2024-02-02T13:30:14.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.362+0000 7f4edd800640 1 -- start start 2024-02-02T13:30:14.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4ed806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edd800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed80ad2e0 con 0x7f4ed80ccd70 2024-02-02T13:30:14.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edd800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed80ad5c0 con 0x7f4ed806c820 2024-02-02T13:30:14.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed806b4f0 con 0x7f4ed815d2c0 2024-02-02T13:30:14.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4ed80ccd70 0x7f4ed80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56876/0 (socket says 172.21.15.91:56876) 2024-02-02T13:30:14.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.363+0000 7f4edce00640 1 -- 172.21.15.91:0/905666426 learned_addr learned my addr 172.21.15.91:0/905666426 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7e00640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4ed806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038823417 0 0) 0x7f4ed80ad2e0 con 0x7f4ed80ccd70 2024-02-02T13:30:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb8003520 con 0x7f4ed80ccd70 2024-02-02T13:30:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113957501 0 0) 0x7f4eb8003520 con 0x7f4ed80ccd70 2024-02-02T13:30:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 msgr2=0x7f4ed806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4ed806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 >> v1:172.21.15.113:6789/0 conn(0x7f4ed806c820 legacy=0x7f4ed80cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed806b7d0 con 0x7f4ed80ccd70 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f4ed0001670 con 0x7f4ed80ccd70 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 >> v1:172.21.15.91:6789/0 conn(0x7f4ed80ccd70 legacy=0x7f4ed80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4eb80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.364+0000 7f4ed7400640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4eb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ed7400640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb8004d00 0x7f4eb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed806b4f0 con 0x7f4eb8003f70 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800bac0 con 0x7f4eb8004d00 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800bc80 con 0x7f4ed815d2c0 2024-02-02T13:30:14.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4eb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ed7e00640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4eb80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4edce00640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb8004d00 0x7f4eb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 -- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 msgr2=0x7f4eb80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed815d2c0 0x7f4eb80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 -- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb8004d00 msgr2=0x7f4eb8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb8004d00 0x7f4eb8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.365+0000 7f4ecf200640 1 -- 172.21.15.91:0/905666426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8003520 con 0x7f4eb8003f70 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.366+0000 7f4ecf200640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4eb80047c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0001750 tx=0x7f4eb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d1c3417d8f568d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00052d0 con 0x7f4eb8003f70 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec002e050 con 0x7f4eb8003f70 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4ed7400640 1 -- 172.21.15.91:0/905666426 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0003830 con 0x7f4eb8003f70 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 -- 172.21.15.91:0/905666426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 msgr2=0x7f4eb80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 --2- 172.21.15.91:0/905666426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4eb80047c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0001750 tx=0x7f4eb80057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edce00640 1 -- 172.21.15.91:0/905666426 reap_dead start 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 -- 172.21.15.91:0/905666426 shutdown_connections 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 -- 172.21.15.91:0/905666426 >> 172.21.15.91:0/905666426 conn(0x7f4ed80cea90 msgr2=0x7f4ed80ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 -- 172.21.15.91:0/905666426 shutdown_connections 2024-02-02T13:30:14.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.367+0000 7f4edd800640 1 -- 172.21.15.91:0/905666426 wait complete. 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.369+0000 7f4edd800640 1 Processor -- start 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.369+0000 7f4edd800640 1 -- start start 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 0x7f4ed8160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed80b0e00 0x7f4ed8161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80af6a0 con 0x7f4eb8003f70 2024-02-02T13:30:14.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80af980 con 0x7f4ed8156580 2024-02-02T13:30:14.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.370+0000 7f4edd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80afc60 con 0x7f4ed80b0e00 2024-02-02T13:30:14.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56232/0 (socket says 172.21.15.91:56232) 2024-02-02T13:30:14.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 -- 172.21.15.91:0/2362667922 learned_addr learned my addr 172.21.15.91:0/2362667922 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ecf200640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 0x7f4ed8160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 -- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed80b0e00 msgr2=0x7f4ed8161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed80b0e00 0x7f4ed8161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 -- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 msgr2=0x7f4ed8160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 0x7f4ed8160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 -- 172.21.15.91:0/2362667922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed815d820 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ecf200640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 0x7f4ed8160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.371+0000 7f4ed7e00640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800f350 tx=0x7f4ec800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.372+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8017600 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.372+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec8011040 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.372+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80179c0 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.372+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed80aff40 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.372+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed806cf60 con 0x7f4eb8003f70 2024-02-02T13:30:14.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.373+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed804dff0 con 0x7f4eb8003f70 2024-02-02T13:30:14.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.374+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4ec8017b20 con 0x7f4eb8003f70 2024-02-02T13:30:14.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.375+0000 7f4ed4c00640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 0x7f4eb006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.375+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4ec8088e00 con 0x7f4eb8003f70 2024-02-02T13:30:14.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.375+0000 7f4ecf200640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 0x7f4eb006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.376+0000 7f4ecf200640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 0x7f4eb006c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0033000 tx=0x7f4ec0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.380+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4ec80546d0 con 0x7f4eb8003f70 2024-02-02T13:30:14.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.572+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ed80b4a10 con 0x7f4eb8003f70 2024-02-02T13:30:14.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.574+0000 7f4ed4c00640 1 -- 172.21.15.91:0/2362667922 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+12391 (secure 0 0 0) 0x7f4ec801c070 con 0x7f4eb8003f70 2024-02-02T13:30:14.576 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:14.576 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":101,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:13.611588+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:14.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.576+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 msgr2=0x7f4eb006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.576+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 0x7f4eb006c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0033000 tx=0x7f4ec0009030 comp rx=0 tx=0).stop 2024-02-02T13:30:14.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.576+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 msgr2=0x7f4ed8155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.576+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800f350 tx=0x7f4ec800f810 comp rx=0 tx=0).stop 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 shutdown_connections 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4eb006a430 0x7f4eb006c8f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed80b0e00 0x7f4ed8161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed8156580 0x7f4ed8160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 --2- 172.21.15.91:0/2362667922 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4eb8003f70 0x7f4ed8155ce0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 >> 172.21.15.91:0/2362667922 conn(0x7f4ed80cea90 msgr2=0x7f4ed815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 shutdown_connections 2024-02-02T13:30:14.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.577+0000 7f4edd800640 1 -- 172.21.15.91:0/2362667922 wait complete. 2024-02-02T13:30:14.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:30:14.598 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:30:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.796+0000 7f4878800640 1 Processor -- start 2024-02-02T13:30:14.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.797+0000 7f4878800640 1 -- start start 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4878800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48700d0070 0x7f48700b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4878800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 0x7f48700d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4878800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48700cd930 con 0x7f48700b6260 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4878800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48700b8f90 con 0x7f48700d0070 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4878800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48700d1680 con 0x7f48700b2260 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4875e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48700d0070 0x7f48700b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4875e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48700d0070 0x7f48700b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56244/0 (socket says 172.21.15.91:56244) 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4875e00640 1 -- 172.21.15.91:0/1210848970 learned_addr learned my addr 172.21.15.91:0/1210848970 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:14.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4875400640 1 --2- 172.21.15.91:0/1210848970 >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 0x7f48700d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.798+0000 7f4875e00640 1 -- 172.21.15.91:0/1210848970 >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 msgr2=0x7f48700d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.799+0000 7f4875e00640 1 --2- 172.21.15.91:0/1210848970 >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 0x7f48700d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.799+0000 7f4875e00640 1 -- 172.21.15.91:0/1210848970 >> v1:172.21.15.113:6789/0 conn(0x7f48700b6260 legacy=0x7f48700d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.799+0000 7f4875e00640 1 -- 172.21.15.91:0/1210848970 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48700d1960 con 0x7f48700d0070 2024-02-02T13:30:14.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.799+0000 7f4875400640 1 --2- 172.21.15.91:0/1210848970 >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 0x7f48700d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4875e00640 1 --2- 172.21.15.91:0/1210848970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700d0070 0x7f48700b8a50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f48700b8810 tx=0x7f486002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d543b00248a59378 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4874a00640 1 -- 172.21.15.91:0/1210848970 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860031870 con 0x7f48700d0070 2024-02-02T13:30:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4874a00640 1 -- 172.21.15.91:0/1210848970 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48600319d0 con 0x7f48700d0070 2024-02-02T13:30:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4874a00640 1 -- 172.21.15.91:0/1210848970 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860031d60 con 0x7f48700d0070 2024-02-02T13:30:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4878800640 1 -- 172.21.15.91:0/1210848970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700d0070 msgr2=0x7f48700b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.800+0000 7f4878800640 1 --2- 172.21.15.91:0/1210848970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700d0070 0x7f48700b8a50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f48700b8810 tx=0x7f486002b910 comp rx=0 tx=0).stop 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 -- 172.21.15.91:0/1210848970 shutdown_connections 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 --2- 172.21.15.91:0/1210848970 >> v2:172.21.15.113:3300/0 conn(0x7f48700b2260 0x7f48700d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 --2- 172.21.15.91:0/1210848970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700d0070 0x7f48700b8a50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 -- 172.21.15.91:0/1210848970 >> 172.21.15.91:0/1210848970 conn(0x7f4870070660 msgr2=0x7f4870070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 -- 172.21.15.91:0/1210848970 shutdown_connections 2024-02-02T13:30:14.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.801+0000 7f4878800640 1 -- 172.21.15.91:0/1210848970 wait complete. 2024-02-02T13:30:14.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.802+0000 7f4878800640 1 Processor -- start 2024-02-02T13:30:14.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.802+0000 7f4878800640 1 -- start start 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700b6260 0x7f48700aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700d0070 0x7f48701f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48700ca200 con 0x7f48700b2260 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48700ca4e0 con 0x7f48700d0070 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.803+0000 7f4878800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48700ca7c0 con 0x7f48700b6260 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56252/0 (socket says 172.21.15.91:56252) 2024-02-02T13:30:14.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 -- 172.21.15.91:0/1658489372 learned_addr learned my addr 172.21.15.91:0/1658489372 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875400640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700b6260 0x7f48700aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 -- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700b6260 msgr2=0x7f48700aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700b6260 0x7f48700aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 -- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700d0070 msgr2=0x7f48701f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700d0070 0x7f48701f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 -- 172.21.15.91:0/1658489372 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48600094d0 con 0x7f48700b2260 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.804+0000 7f4875e00640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4860000c00 tx=0x7f48600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.805+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860031840 con 0x7f48700b2260 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.805+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48600443f0 con 0x7f48700b2260 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.805+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48600447a0 con 0x7f48700b2260 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.805+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48700caaa0 con 0x7f48700b2260 2024-02-02T13:30:14.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.805+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4870151930 con 0x7f48700b2260 2024-02-02T13:30:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.806+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48700cafc0 con 0x7f48700b2260 2024-02-02T13:30:14.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.807+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f486004d450 con 0x7f48700b2260 2024-02-02T13:30:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.808+0000 7f486e000640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 0x7f484806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.808+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f486002d040 con 0x7f48700b2260 2024-02-02T13:30:14.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.808+0000 7f4875400640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 0x7f484806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:14.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.809+0000 7f4875400640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 0x7f484806c8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f48700b86c0 tx=0x7f4858008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:14.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.812+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4860076500 con 0x7f48700b2260 2024-02-02T13:30:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.998+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48700ce0f0 con 0x7f48700b2260 2024-02-02T13:30:15.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:14.999+0000 7f486e000640 1 -- 172.21.15.91:0/1658489372 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48700ce0f0 con 0x7f48700b2260 2024-02-02T13:30:15.001 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 msgr2=0x7f484806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 0x7f484806c8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f48700b86c0 tx=0x7f4858008040 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 msgr2=0x7f48701513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4860000c00 tx=0x7f48600046c0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 shutdown_connections 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f484806a430 0x7f484806c8f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700b6260 0x7f48700aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700d0070 0x7f48701f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 --2- 172.21.15.91:0/1658489372 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700b2260 0x7f48701513f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 >> 172.21.15.91:0/1658489372 conn(0x7f4870070660 msgr2=0x7f487015b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 shutdown_connections 2024-02-02T13:30:15.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.002+0000 7f4878800640 1 -- 172.21.15.91:0/1658489372 wait complete. 2024-02-02T13:30:15.021 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:30:15.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.224+0000 7f15c3400640 1 Processor -- start 2024-02-02T13:30:15.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.225+0000 7f15c3400640 1 -- start start 2024-02-02T13:30:15.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 0x7f15bc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15bc15e340 0x7f15bc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c3400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0cbf20 con 0x7f15bc0cbac0 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c3400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc15e790 con 0x7f15bc15e340 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c3400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0ca0c0 con 0x7f15bc159de0 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15bc15e340 0x7f15bc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.226+0000 7f15c1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15bc15e340 0x7f15bc06fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56262/0 (socket says 172.21.15.91:56262) 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c1400640 1 -- 172.21.15.91:0/1383145296 learned_addr learned my addr 172.21.15.91:0/1383145296 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c0a00640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 0x7f15bc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c1400640 1 -- 172.21.15.91:0/1383145296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 msgr2=0x7f15bc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c1400640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 0x7f15bc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c1400640 1 -- 172.21.15.91:0/1383145296 >> v1:172.21.15.113:6789/0 conn(0x7f15bc0cbac0 legacy=0x7f15bc15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c1400640 1 -- 172.21.15.91:0/1383145296 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0ca380 con 0x7f15bc15e340 2024-02-02T13:30:15.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.227+0000 7f15c0a00640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 0x7f15bc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.228+0000 7f15c1400640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc15e340 0x7f15bc06fc90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009e50 tx=0x7f15b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8a95a93ea76daa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.228+0000 7f15bbe00640 1 -- 172.21.15.91:0/1383145296 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b402ca80 con 0x7f15bc15e340 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.228+0000 7f15bbe00640 1 -- 172.21.15.91:0/1383145296 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b402cbe0 con 0x7f15bc15e340 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.228+0000 7f15bbe00640 1 -- 172.21.15.91:0/1383145296 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40343f0 con 0x7f15bc15e340 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 -- 172.21.15.91:0/1383145296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc15e340 msgr2=0x7f15bc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc15e340 0x7f15bc06fc90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009e50 tx=0x7f15b402bee0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 -- 172.21.15.91:0/1383145296 shutdown_connections 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc15e340 0x7f15bc06fc90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 --2- 172.21.15.91:0/1383145296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc159de0 0x7f15bc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 -- 172.21.15.91:0/1383145296 >> 172.21.15.91:0/1383145296 conn(0x7f15bc0b0700 msgr2=0x7f15bc0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:15.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 -- 172.21.15.91:0/1383145296 shutdown_connections 2024-02-02T13:30:15.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.229+0000 7f15c3400640 1 -- 172.21.15.91:0/1383145296 wait complete. 2024-02-02T13:30:15.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.231+0000 7f15c3400640 1 Processor -- start 2024-02-02T13:30:15.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.231+0000 7f15c3400640 1 -- start start 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc0cbac0 0x7f15bc14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15bc15e340 0x7f15bc0c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0c69f0 con 0x7f15bc159de0 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc152f70 con 0x7f15bc15e340 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15c3400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0c5de0 con 0x7f15bc0cbac0 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15b3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15b3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56274/0 (socket says 172.21.15.91:56274) 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.232+0000 7f15b3c00640 1 -- 172.21.15.91:0/3109878651 learned_addr learned my addr 172.21.15.91:0/3109878651 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:15.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15c0a00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc0cbac0 0x7f15bc14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 -- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc0cbac0 msgr2=0x7f15bc14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc0cbac0 0x7f15bc14faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 -- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15bc15e340 msgr2=0x7f15bc0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15c1400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15bc15e340 0x7f15bc0c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15bc15e340 0x7f15bc0c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 -- 172.21.15.91:0/3109878651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b4009b30 con 0x7f15bc159de0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b3c00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c430 tx=0x7f15a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4011790 con 0x7f15bc159de0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a40118f0 con 0x7f15bc159de0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4011ce0 con 0x7f15bc159de0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc0c60c0 con 0x7f15bc159de0 2024-02-02T13:30:15.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.233+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc0c6680 con 0x7f15bc159de0 2024-02-02T13:30:15.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.234+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc0c6b60 con 0x7f15bc159de0 2024-02-02T13:30:15.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.235+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f15a401c460 con 0x7f15bc159de0 2024-02-02T13:30:15.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.236+0000 7f15b9600640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 0x7f159406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.237+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15a40885d0 con 0x7f15bc159de0 2024-02-02T13:30:15.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.237+0000 7f15c0a00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 0x7f159406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.237+0000 7f15c0a00640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 0x7f159406c8f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0c8c40 tx=0x7f15ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.241+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f15a4053e70 con 0x7f15bc159de0 2024-02-02T13:30:15.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.432+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f15bc066cf0 con 0x7f15bc159de0 2024-02-02T13:30:15.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.432+0000 7f15b9600640 1 -- 172.21.15.91:0/3109878651 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+65 (secure 0 0 0) 0x7f15a4057490 con 0x7f15bc159de0 2024-02-02T13:30:15.434 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:15.434 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:30:15.434 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:15.434 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:30:15.435 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:30:15.435 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:15.435 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:30:15.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 msgr2=0x7f159406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 0x7f159406c8f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0c8c40 tx=0x7f15ac008040 comp rx=0 tx=0).stop 2024-02-02T13:30:15.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 msgr2=0x7f15bc152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c430 tx=0x7f15a400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 shutdown_connections 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f159406a430 0x7f159406c8f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15bc15e340 0x7f15bc0c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15bc159de0 0x7f15bc152a30 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 --2- 172.21.15.91:0/3109878651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15bc0cbac0 0x7f15bc14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.435+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 >> 172.21.15.91:0/3109878651 conn(0x7f15bc0b0700 msgr2=0x7f15bc0afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.436+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 shutdown_connections 2024-02-02T13:30:15.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.436+0000 7f15c3400640 1 -- 172.21.15.91:0/3109878651 wait complete. 2024-02-02T13:30:15.454 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:30:15.455 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:30:15.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.659+0000 7ff03bc00640 1 Processor -- start 2024-02-02T13:30:15.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.660+0000 7ff03bc00640 1 -- start start 2024-02-02T13:30:15.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff03bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff03bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff0340b2260 0x7ff0340d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff03bc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340cd930 con 0x7ff0340b6260 2024-02-02T13:30:15.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff03bc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0340b8f90 con 0x7ff0340b2260 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff03bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340d1680 con 0x7ff0340d0070 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff033e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff0340b2260 0x7ff0340d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff033e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff0340b2260 0x7ff0340d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56288/0 (socket says 172.21.15.91:56288) 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff033e00640 1 -- 172.21.15.91:0/3929730585 learned_addr learned my addr 172.21.15.91:0/3929730585 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.661+0000 7ff039c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff0340b6260 0x7ff0340d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56886/0 (socket says 172.21.15.91:56886) 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff039200640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff033e00640 1 -- 172.21.15.91:0/3929730585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 msgr2=0x7ff0340b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff033e00640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff033e00640 1 -- 172.21.15.91:0/3929730585 >> v1:172.21.15.91:6789/0 conn(0x7ff0340b6260 legacy=0x7ff0340d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff033e00640 1 -- 172.21.15.91:0/3929730585 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340d1960 con 0x7ff0340b2260 2024-02-02T13:30:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.662+0000 7ff033400640 1 -- 172.21.15.91:0/3929730585 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050153707 0 0) 0x7ff02c001110 con 0x7ff0340b6260 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.663+0000 7ff033e00640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340d0450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009e50 tx=0x7ff01c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb379a52d24b0b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.663+0000 7ff033400640 1 -- 172.21.15.91:0/3929730585 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c02cb00 con 0x7ff0340b2260 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.663+0000 7ff033400640 1 -- 172.21.15.91:0/3929730585 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c02cc60 con 0x7ff0340b2260 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.663+0000 7ff033400640 1 -- 172.21.15.91:0/3929730585 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0343f0 con 0x7ff0340b2260 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 -- 172.21.15.91:0/3929730585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 msgr2=0x7ff0340d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340d0450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009e50 tx=0x7ff01c02bf60 comp rx=0 tx=0).stop 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 -- 172.21.15.91:0/3929730585 shutdown_connections 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340d0450 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3929730585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 -- 172.21.15.91:0/3929730585 >> 172.21.15.91:0/3929730585 conn(0x7ff034070660 msgr2=0x7ff034070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:15.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 -- 172.21.15.91:0/3929730585 shutdown_connections 2024-02-02T13:30:15.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.664+0000 7ff03bc00640 1 -- 172.21.15.91:0/3929730585 wait complete. 2024-02-02T13:30:15.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.665+0000 7ff03bc00640 1 Processor -- start 2024-02-02T13:30:15.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 -- start start 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0340b6260 0x7ff0341f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03415b810 con 0x7ff0340b2260 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341f35e0 con 0x7ff0340b6260 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.666+0000 7ff03bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340ca7e0 con 0x7ff0340d0070 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56296/0 (socket says 172.21.15.91:56296) 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 -- 172.21.15.91:0/3809118085 learned_addr learned my addr 172.21.15.91:0/3809118085 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 -- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 msgr2=0x7ff0340c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 -- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0340b6260 msgr2=0x7ff0341f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0340b6260 0x7ff0341f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 -- 172.21.15.91:0/3809118085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c009b30 con 0x7ff0340b2260 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff039200640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff02400d800 tx=0x7ff02400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.667+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400b6c0 con 0x7ff0340b2260 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.668+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02400b820 con 0x7ff0340b2260 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.668+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400bbb0 con 0x7ff0340b2260 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.668+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0340caac0 con 0x7ff0340b2260 2024-02-02T13:30:15.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.668+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0340cb080 con 0x7ff0340b2260 2024-02-02T13:30:15.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.669+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034066cf0 con 0x7ff0340b2260 2024-02-02T13:30:15.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.670+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff02400bd10 con 0x7ff0340b2260 2024-02-02T13:30:15.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.671+0000 7ff030c00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 0x7ff00c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:15.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.671+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff024088bb0 con 0x7ff0340b2260 2024-02-02T13:30:15.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.671+0000 7ff033e00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 0x7ff00c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:15.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.672+0000 7ff033e00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 0x7ff00c06c8f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff01c035000 tx=0x7ff01c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:15.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.675+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff024054450 con 0x7ff0340b2260 2024-02-02T13:30:15.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:15.849+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff0341607a0 con 0x7ff0340b2260 2024-02-02T13:30:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.655+0000 7ff030c00640 1 -- 172.21.15.91:0/3809118085 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff024057a70 con 0x7ff0340b2260 2024-02-02T13:30:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:30:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.660+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 msgr2=0x7ff00c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.660+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 0x7ff00c06c8f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff01c035000 tx=0x7ff01c02e000 comp rx=0 tx=0).stop 2024-02-02T13:30:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.660+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 msgr2=0x7ff0340ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.660+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff02400d800 tx=0x7ff02400dcc0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 shutdown_connections 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff00c06a430 0x7ff00c06c8f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0340d0070 0x7ff0340c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0340b6260 0x7ff0341f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 --2- 172.21.15.91:0/3809118085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0340b2260 0x7ff0340ca200 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.661+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 >> 172.21.15.91:0/3809118085 conn(0x7ff034070660 msgr2=0x7ff0340b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:16.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.662+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 shutdown_connections 2024-02-02T13:30:16.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.662+0000 7ff03bc00640 1 -- 172.21.15.91:0/3809118085 wait complete. 2024-02-02T13:30:16.686 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:30:16.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.898+0000 7f57d9c00640 1 Processor -- start 2024-02-02T13:30:16.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.899+0000 7f57d9c00640 1 -- start start 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.900+0000 7f57d9c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57d40d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.900+0000 7f57d9c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40cd930 con 0x7f57d40b2260 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.900+0000 7f57d9c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40b8ee0 con 0x7f57d40d0070 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.900+0000 7f57d9c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40d1680 con 0x7f57d40b8a50 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.901+0000 7f57d2000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f57d40b2260 0x7f57d40b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56900/0 (socket says 172.21.15.91:56900) 2024-02-02T13:30:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.901+0000 7f57d2000640 1 -- 172.21.15.91:0/2083939954 learned_addr learned my addr 172.21.15.91:0/2083939954 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:16.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.901+0000 7f57d3400640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57d40d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.901+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903895152 0 0) 0x7f57bc001110 con 0x7f57d40b2260 2024-02-02T13:30:16.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.901+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b4003520 con 0x7f57d40b2260 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1278872190 0 0) 0x7f57b4003520 con 0x7f57d40b2260 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 msgr2=0x7f57d40d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57d40d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 >> v1:172.21.15.113:6789/0 conn(0x7f57d40d0070 legacy=0x7f57d40d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40d1960 con 0x7f57d40b2260 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f57bc001800 con 0x7f57d40b2260 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 >> v1:172.21.15.91:6789/0 conn(0x7f57d40b2260 legacy=0x7f57d40b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40039a0 0x7f57b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40d1680 con 0x7f57b40039a0 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400ba30 con 0x7f57b4004d00 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400bbf0 con 0x7f57d40b8a50 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d2a00640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40039a0 0x7f57b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d3400640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.902+0000 7f57d2000640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2a00640 1 -- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 msgr2=0x7f57b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2a00640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2a00640 1 -- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b4004d00 msgr2=0x7f57b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2a00640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2a00640 1 -- 172.21.15.91:0/2083939954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4003520 con 0x7f57b40039a0 2024-02-02T13:30:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.903+0000 7f57d2000640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57d40b8a50 0x7f57b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.904+0000 7f57d2a00640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40039a0 0x7f57b4003dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f57c4001720 tx=0x7f57b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee65ca7d800e8773 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.904+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40052d0 con 0x7f57b40039a0 2024-02-02T13:30:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.904+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c402e050 con 0x7f57b40039a0 2024-02-02T13:30:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.904+0000 7f57d1600640 1 -- 172.21.15.91:0/2083939954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40035f0 con 0x7f57b40039a0 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 -- 172.21.15.91:0/2083939954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40039a0 msgr2=0x7f57b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 --2- 172.21.15.91:0/2083939954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40039a0 0x7f57b4003dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f57c4001720 tx=0x7f57b40057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d3400640 1 -- 172.21.15.91:0/2083939954 reap_dead start 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 -- 172.21.15.91:0/2083939954 shutdown_connections 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 -- 172.21.15.91:0/2083939954 >> 172.21.15.91:0/2083939954 conn(0x7f57d4070660 msgr2=0x7f57d4070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 -- 172.21.15.91:0/2083939954 shutdown_connections 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.905+0000 7f57d9c00640 1 -- 172.21.15.91:0/2083939954 wait complete. 2024-02-02T13:30:16.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.907+0000 7f57d9c00640 1 Processor -- start 2024-02-02T13:30:16.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.907+0000 7f57d9c00640 1 -- start start 2024-02-02T13:30:16.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40039a0 0x7f57d4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 0x7f57d40b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4156ab0 con 0x7f57d4159e70 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41f29c0 con 0x7f57d415ec60 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d9c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41f6400 con 0x7f57b40039a0 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56308/0 (socket says 172.21.15.91:56308) 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 -- 172.21.15.91:0/1763555462 learned_addr learned my addr 172.21.15.91:0/1763555462 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2a00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40039a0 0x7f57d4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d3400640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 0x7f57d40b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 -- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40039a0 msgr2=0x7f57d4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40039a0 0x7f57d4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 -- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 msgr2=0x7f57d40b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 0x7f57d40b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d2000640 1 -- 172.21.15.91:0/1763555462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d41f66e0 con 0x7f57d4159e70 2024-02-02T13:30:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.908+0000 7f57d3400640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 0x7f57d40b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57d2000640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f57bc001eb0 tx=0x7f57bc00dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00dea0 con 0x7f57d4159e70 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc0074e0 con 0x7f57d4159e70 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0049e0 con 0x7f57d4159e70 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d41f69c0 con 0x7f57d4159e70 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.909+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d415f110 con 0x7f57d4159e70 2024-02-02T13:30:16.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.910+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d4066cf0 con 0x7f57d4159e70 2024-02-02T13:30:16.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.911+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f57bc004b40 con 0x7f57d4159e70 2024-02-02T13:30:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.912+0000 7f57caa00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 0x7f57ac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.912+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f57bc087eb0 con 0x7f57d4159e70 2024-02-02T13:30:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.912+0000 7f57d2a00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 0x7f57ac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.913+0000 7f57d2a00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 0x7f57ac06c920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c4033000 tx=0x7f57c4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:16.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:16.917+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f57bc0571a0 con 0x7f57d4159e70 2024-02-02T13:30:17.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.106+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f57d40cd930 con 0x7f57d4159e70 2024-02-02T13:30:17.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.670+0000 7f57caa00640 1 -- 172.21.15.91:0/1763555462 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f57d40cd930 con 0x7f57d4159e70 2024-02-02T13:30:17.673 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:30:17.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.680+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 msgr2=0x7f57ac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.680+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 0x7f57ac06c920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c4033000 tx=0x7f57c4009030 comp rx=0 tx=0).stop 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.680+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 msgr2=0x7f57d4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.680+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f57bc001eb0 tx=0x7f57bc00dc50 comp rx=0 tx=0).stop 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 shutdown_connections 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f57ac06a460 0x7f57ac06c920 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57d415ec60 0x7f57d40b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57d4159e70 0x7f57d4156570 secure :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f57bc001eb0 tx=0x7f57bc00dc50 comp rx=0 tx=0).stop 2024-02-02T13:30:17.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 --2- 172.21.15.91:0/1763555462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40039a0 0x7f57d4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.681+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 >> 172.21.15.91:0/1763555462 conn(0x7f57d4070660 msgr2=0x7f57d40b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:17.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.682+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 shutdown_connections 2024-02-02T13:30:17.684 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.682+0000 7f57d9c00640 1 -- 172.21.15.91:0/1763555462 wait complete. 2024-02-02T13:30:17.710 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:30:17.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.894+0000 7f22e9200640 1 Processor -- start 2024-02-02T13:30:17.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.895+0000 7f22e9200640 1 -- start start 2024-02-02T13:30:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.896+0000 7f22e9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22e4161660 0x7f22e406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.896+0000 7f22e9200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40ccdd0 con 0x7f22e40ce270 2024-02-02T13:30:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.896+0000 7f22e9200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4166020 con 0x7f22e40b0700 2024-02-02T13:30:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.896+0000 7f22e9200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40cc7c0 con 0x7f22e4161660 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e2800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f22e40ce270 0x7f22e4165910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56908/0 (socket says 172.21.15.91:56908) 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e2800640 1 -- 172.21.15.91:0/1411063454 learned_addr learned my addr 172.21.15.91:0/1411063454 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e1e00640 1 --2- 172.21.15.91:0/1411063454 >> v2:172.21.15.113:3300/0 conn(0x7f22e4161660 0x7f22e406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307080485 0 0) 0x7f22e40ccdd0 con 0x7f22e40ce270 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c4003520 con 0x7f22e40ce270 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247055972 0 0) 0x7f22c4003520 con 0x7f22e40ce270 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 >> v2:172.21.15.113:3300/0 conn(0x7f22e4161660 msgr2=0x7f22e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 --2- 172.21.15.91:0/1411063454 >> v2:172.21.15.113:3300/0 conn(0x7f22e4161660 0x7f22e406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 >> v1:172.21.15.113:6789/0 conn(0x7f22e40b0700 legacy=0x7f22e4165200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e406c540 con 0x7f22e40ce270 2024-02-02T13:30:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.897+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f22dc0019f0 con 0x7f22e40ce270 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 >> v1:172.21.15.91:6789/0 conn(0x7f22e40ce270 legacy=0x7f22e4165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 0x7f22c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22c40047c0 0x7f22c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4005180 0x7f22c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40cc7c0 con 0x7f22c4003970 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400c120 con 0x7f22c40047c0 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400c2e0 con 0x7f22c4005180 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 0x7f22c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 -- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4005180 msgr2=0x7f22c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4005180 0x7f22c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1400640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22c40047c0 0x7f22c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 -- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22c40047c0 msgr2=0x7f22c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22c40047c0 0x7f22c4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e2800640 1 -- 172.21.15.91:0/1411063454 reap_dead start 2024-02-02T13:30:17.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.898+0000 7f22e1e00640 1 -- 172.21.15.91:0/1411063454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4003520 con 0x7f22c4003970 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e1e00640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 0x7f22c4003d90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ac0 tx=0x7f22c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b281731dd8a2be server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d402c6d0 con 0x7f22c4003970 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d402c830 con 0x7f22c4003970 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e0a00640 1 -- 172.21.15.91:0/1411063454 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d402cbc0 con 0x7f22c4003970 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e9200640 1 -- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 msgr2=0x7f22c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.900+0000 7f22e9200640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 0x7f22c4003d90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ac0 tx=0x7f22c40057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.901+0000 7f22e9200640 1 -- 172.21.15.91:0/1411063454 shutdown_connections 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.901+0000 7f22e9200640 1 --2- 172.21.15.91:0/1411063454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22c4003970 0x7f22c4003d90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.901+0000 7f22e9200640 1 -- 172.21.15.91:0/1411063454 >> 172.21.15.91:0/1411063454 conn(0x7f22e40cec80 msgr2=0x7f22e40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.901+0000 7f22e9200640 1 -- 172.21.15.91:0/1411063454 shutdown_connections 2024-02-02T13:30:17.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.901+0000 7f22e9200640 1 -- 172.21.15.91:0/1411063454 wait complete. 2024-02-02T13:30:17.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.902+0000 7f22e9200640 1 Processor -- start 2024-02-02T13:30:17.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.903+0000 7f22e9200640 1 -- start start 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.903+0000 7f22e9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4003970 0x7f22e40b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.903+0000 7f22e9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22e4162b10 0x7f22e4162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e9200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b1830 con 0x7f22e40c5cc0 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b1b10 con 0x7f22e4162b10 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b1df0 con 0x7f22c4003970 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56318/0 (socket says 172.21.15.91:56318) 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 -- 172.21.15.91:0/2237036142 learned_addr learned my addr 172.21.15.91:0/2237036142 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 -- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4003970 msgr2=0x7f22e40b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e2800640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22e4162b10 0x7f22e4162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1e00640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4003970 0x7f22e40b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4003970 0x7f22e40b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 -- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22e4162b10 msgr2=0x7f22e4162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:17.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22e4162b10 0x7f22e4162fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:17.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.904+0000 7f22e1400640 1 -- 172.21.15.91:0/2237036142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40b20d0 con 0x7f22e40c5cc0 2024-02-02T13:30:17.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22e1400640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00d470 tx=0x7f22cc00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc005480 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc003610 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e40b23b0 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc003980 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.905+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40b30f0 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.906+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e4066cf0 con 0x7f22e40c5cc0 2024-02-02T13:30:17.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.907+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f22cc026430 con 0x7f22e40c5cc0 2024-02-02T13:30:17.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.908+0000 7f22da000640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 0x7f22bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:17.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.908+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f22cc089120 con 0x7f22e40c5cc0 2024-02-02T13:30:17.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.908+0000 7f22e1e00640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 0x7f22bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:17.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.909+0000 7f22e1e00640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 0x7f22bc06c8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f22d4036000 tx=0x7f22d4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:17.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:17.912+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22cc0547e0 con 0x7f22e40c5cc0 2024-02-02T13:30:18.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.138+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f22e40b4c00 con 0x7f22e40c5cc0 2024-02-02T13:30:18.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.148+0000 7f22da000640 1 -- 172.21.15.91:0/2237036142 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f22cc057e00 con 0x7f22e40c5cc0 2024-02-02T13:30:18.152 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 12 and data pool 13 2024-02-02T13:30:18.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 msgr2=0x7f22bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 0x7f22bc06c8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f22d4036000 tx=0x7f22d4030000 comp rx=0 tx=0).stop 2024-02-02T13:30:18.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 msgr2=0x7f22e40c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00d470 tx=0x7f22cc00d930 comp rx=0 tx=0).stop 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 shutdown_connections 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f22bc06a430 0x7f22bc06c8f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22e4162b10 0x7f22e4162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22e40c5cc0 0x7f22e40c6120 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 --2- 172.21.15.91:0/2237036142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22c4003970 0x7f22e40b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 >> 172.21.15.91:0/2237036142 conn(0x7f22e40cec80 msgr2=0x7f22e406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 shutdown_connections 2024-02-02T13:30:18.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.154+0000 7f22e9200640 1 -- 172.21.15.91:0/2237036142 wait complete. 2024-02-02T13:30:18.183 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:18.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.351+0000 7f6730a00640 1 Processor -- start 2024-02-02T13:30:18.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.352+0000 7f6730a00640 1 -- start start 2024-02-02T13:30:18.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f6730a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f672815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f6730a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6728070f80 con 0x7f672806f4f0 2024-02-02T13:30:18.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f6730a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672806c600 con 0x7f67280b0e10 2024-02-02T13:30:18.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f6730a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6728152370 con 0x7f6728151f90 2024-02-02T13:30:18.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f672d600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f672806f4f0 0x7f67280b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56920/0 (socket says 172.21.15.91:56920) 2024-02-02T13:30:18.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.353+0000 7f672d600640 1 -- 172.21.15.91:0/3521207588 learned_addr learned my addr 172.21.15.91:0/3521207588 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:18.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131237487 0 0) 0x7f6728070f80 con 0x7f672806f4f0 2024-02-02T13:30:18.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6704003520 con 0x7f672806f4f0 2024-02-02T13:30:18.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672e000640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f672815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031919070 0 0) 0x7f6704003520 con 0x7f672806f4f0 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 msgr2=0x7f672815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f672815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 >> v1:172.21.15.113:6789/0 conn(0x7f67280b0e10 legacy=0x7f67280b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280cb010 con 0x7f672806f4f0 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f670c001670 con 0x7f672806f4f0 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 >> v1:172.21.15.91:6789/0 conn(0x7f672806f4f0 legacy=0x7f67280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67040039a0 0x7f6704003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f67040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6704004d00 0x7f6704005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728152370 con 0x7f6704004d00 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670400bb10 con 0x7f67040039a0 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670400bca0 con 0x7f6728151f90 2024-02-02T13:30:18.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.354+0000 7f672ea00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6704004d00 0x7f6704005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672d600640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f67040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672ea00640 1 -- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 msgr2=0x7f67040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672ea00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6728151f90 0x7f67040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672e000640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67040039a0 0x7f6704003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672ea00640 1 -- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67040039a0 msgr2=0x7f6704003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672ea00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67040039a0 0x7f6704003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.355+0000 7f672ea00640 1 -- 172.21.15.91:0/3521207588 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704003520 con 0x7f6704004d00 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.356+0000 7f672ea00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6704004d00 0x7f6704005180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6714001e20 tx=0x7f67040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09553ee5520c54c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.356+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714003b70 con 0x7f6704004d00 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.356+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714003cd0 con 0x7f6704004d00 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.356+0000 7f672cc00640 1 -- 172.21.15.91:0/3521207588 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714005310 con 0x7f6704004d00 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 -- 172.21.15.91:0/3521207588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6704004d00 msgr2=0x7f6704005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 --2- 172.21.15.91:0/3521207588 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6704004d00 0x7f6704005180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6714001e20 tx=0x7f67040057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f672ea00640 1 -- 172.21.15.91:0/3521207588 reap_dead start 2024-02-02T13:30:18.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 -- 172.21.15.91:0/3521207588 shutdown_connections 2024-02-02T13:30:18.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 -- 172.21.15.91:0/3521207588 >> 172.21.15.91:0/3521207588 conn(0x7f67280cec80 msgr2=0x7f67280cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:18.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 -- 172.21.15.91:0/3521207588 shutdown_connections 2024-02-02T13:30:18.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.357+0000 7f6730a00640 1 -- 172.21.15.91:0/3521207588 wait complete. 2024-02-02T13:30:18.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.358+0000 7f6730a00640 1 Processor -- start 2024-02-02T13:30:18.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.359+0000 7f6730a00640 1 -- start start 2024-02-02T13:30:18.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704004d00 0x7f67281503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 0x7f6728156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728155d70 con 0x7f67281561c0 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672815e5c0 con 0x7f6728156650 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f6730a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6728157350 con 0x7f6704004d00 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56334/0 (socket says 172.21.15.91:56334) 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 -- 172.21.15.91:0/2198839709 learned_addr learned my addr 172.21.15.91:0/2198839709 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672e000640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704004d00 0x7f67281503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672ea00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 0x7f6728156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 -- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704004d00 msgr2=0x7f67281503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704004d00 0x7f67281503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 -- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 msgr2=0x7f6728156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 0x7f6728156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672d600640 1 -- 172.21.15.91:0/2198839709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672815ba50 con 0x7f67281561c0 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.360+0000 7f672ea00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 0x7f6728156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:18.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f672d600640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f670c00d800 tx=0x7f670c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0074e0 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c007640 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0049e0 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672815e2e0 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.361+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6728157020 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.362+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6728155ee0 con 0x7f67281561c0 2024-02-02T13:30:18.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.363+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f670c004b40 con 0x7f67281561c0 2024-02-02T13:30:18.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.364+0000 7f6726000640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 0x7f66fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.364+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f670c088490 con 0x7f67281561c0 2024-02-02T13:30:18.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.364+0000 7f672e000640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 0x7f66fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.365+0000 7f672e000640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 0x7f66fc06c8f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f671c006fd0 tx=0x7f671c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.368+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6728155ee0 con 0x7f67281561c0 2024-02-02T13:30:18.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.543+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6728155ee0 con 0x7f67281561c0 2024-02-02T13:30:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.546+0000 7f6726000640 1 -- 172.21.15.91:0/2198839709 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f6728155ee0 con 0x7f67281561c0 2024-02-02T13:30:18.548 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:18.548 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":104,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:18.140608+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 msgr2=0x7f66fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 0x7f66fc06c8f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f671c006fd0 tx=0x7f671c008040 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 msgr2=0x7f672815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f670c00d800 tx=0x7f670c00dcc0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 shutdown_connections 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66fc06a430 0x7f66fc06c8f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6728156650 0x7f6728156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67281561c0 0x7f672815b510 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 --2- 172.21.15.91:0/2198839709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704004d00 0x7f67281503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 >> 172.21.15.91:0/2198839709 conn(0x7f67280cec80 msgr2=0x7f67280cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 shutdown_connections 2024-02-02T13:30:18.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.549+0000 7f6730a00640 1 -- 172.21.15.91:0/2198839709 wait complete. 2024-02-02T13:30:18.568 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:18.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.869+0000 7fb29e000640 1 Processor -- start 2024-02-02T13:30:18.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.869+0000 7fb29e000640 1 -- start start 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.870+0000 7fb29e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb29815a010 0x7fb2980c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.870+0000 7fb29e000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980b0700 0x7fb2980b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.870+0000 7fb29e000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980cc7c0 con 0x7fb2980ce130 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.870+0000 7fb29e000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980b10d0 con 0x7fb29815a010 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.870+0000 7fb29e000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980ce930 con 0x7fb2980b0700 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb29815a010 0x7fb2980c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb29815a010 0x7fb2980c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56350/0 (socket says 172.21.15.91:56350) 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 -- 172.21.15.91:0/3409542386 learned_addr learned my addr 172.21.15.91:0/3409542386 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb296a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb2980ce130 0x7fb2980ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56932/0 (socket says 172.21.15.91:56932) 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297e00640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980b0700 0x7fb2980b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 -- 172.21.15.91:0/3409542386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980b0700 msgr2=0x7fb2980b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980b0700 0x7fb2980b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 -- 172.21.15.91:0/3409542386 >> v1:172.21.15.91:6789/0 conn(0x7fb2980ce130 legacy=0x7fb2980ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.871+0000 7fb297400640 1 -- 172.21.15.91:0/3409542386 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980cd640 con 0x7fb29815a010 2024-02-02T13:30:18.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.872+0000 7fb297400640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb2980c9670 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb288009870 tx=0x7fb28802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ff7573a8f4b6fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.873+0000 7fb296000640 1 -- 172.21.15.91:0/3409542386 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288030840 con 0x7fb29815a010 2024-02-02T13:30:18.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.873+0000 7fb296000640 1 -- 172.21.15.91:0/3409542386 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2880309a0 con 0x7fb29815a010 2024-02-02T13:30:18.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.873+0000 7fb296000640 1 -- 172.21.15.91:0/3409542386 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288030d30 con 0x7fb29815a010 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.873+0000 7fb29e000640 1 -- 172.21.15.91:0/3409542386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 msgr2=0x7fb2980c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.873+0000 7fb29e000640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb2980c9670 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb288009870 tx=0x7fb28802b980 comp rx=0 tx=0).stop 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 -- 172.21.15.91:0/3409542386 shutdown_connections 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980b0700 0x7fb2980b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 --2- 172.21.15.91:0/3409542386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb2980c9670 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 -- 172.21.15.91:0/3409542386 >> 172.21.15.91:0/3409542386 conn(0x7fb2980cec80 msgr2=0x7fb2980cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:18.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 -- 172.21.15.91:0/3409542386 shutdown_connections 2024-02-02T13:30:18.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.874+0000 7fb29e000640 1 -- 172.21.15.91:0/3409542386 wait complete. 2024-02-02T13:30:18.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.876+0000 7fb29e000640 1 Processor -- start 2024-02-02T13:30:18.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.876+0000 7fb29e000640 1 -- start start 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2980b0700 0x7fb298156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 0x7fb2980c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c2b60 con 0x7fb29815a010 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c8790 con 0x7fb2980b0700 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb29e000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c8a70 con 0x7fb2980ce130 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.877+0000 7fb297e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56354/0 (socket says 172.21.15.91:56354) 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 -- 172.21.15.91:0/1813198584 learned_addr learned my addr 172.21.15.91:0/1813198584 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb296a00640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 0x7fb2980c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297400640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2980b0700 0x7fb298156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 -- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 msgr2=0x7fb2980c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 0x7fb2980c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 -- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2980b0700 msgr2=0x7fb298156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2980b0700 0x7fb298156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 -- 172.21.15.91:0/1813198584 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880094d0 con 0x7fb29815a010 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb296a00640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 0x7fb2980c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb297e00640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb29000cd20 tx=0x7fb29000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.878+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290003b90 con 0x7fb29815a010 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.879+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290003cf0 con 0x7fb29815a010 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.879+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290004240 con 0x7fb29815a010 2024-02-02T13:30:18.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.879+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2980c2db0 con 0x7fb29815a010 2024-02-02T13:30:18.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.879+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2981570e0 con 0x7fb29815a010 2024-02-02T13:30:18.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.880+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2980c32d0 con 0x7fb29815a010 2024-02-02T13:30:18.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.881+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb2900043a0 con 0x7fb29815a010 2024-02-02T13:30:18.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.882+0000 7fb28f400640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 0x7fb27006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:18.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.882+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb290012070 con 0x7fb29815a010 2024-02-02T13:30:18.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.882+0000 7fb297400640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 0x7fb27006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:18.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.883+0000 7fb297400640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 0x7fb27006c8f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb288036000 tx=0x7fb28803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:18.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:18.886+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb2980c32d0 con 0x7fb29815a010 2024-02-02T13:30:19.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.074+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2980c32d0 con 0x7fb29815a010 2024-02-02T13:30:19.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.076+0000 7fb28f400640 1 -- 172.21.15.91:0/1813198584 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fb2980c32d0 con 0x7fb29815a010 2024-02-02T13:30:19.078 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:19.078 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":104,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:18.140608+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 msgr2=0x7fb27006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 0x7fb27006c8f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb288036000 tx=0x7fb28803b040 comp rx=0 tx=0).stop 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 msgr2=0x7fb29815d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb29000cd20 tx=0x7fb29000a660 comp rx=0 tx=0).stop 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 shutdown_connections 2024-02-02T13:30:19.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb27006a430 0x7fb27006c8f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2980ce130 0x7fb2980c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb29815a010 0x7fb29815d940 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.079+0000 7fb29e000640 1 --2- 172.21.15.91:0/1813198584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2980b0700 0x7fb298156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.080+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 >> 172.21.15.91:0/1813198584 conn(0x7fb2980cec80 msgr2=0x7fb2980ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.080+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 shutdown_connections 2024-02-02T13:30:19.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.080+0000 7fb29e000640 1 -- 172.21.15.91:0/1813198584 wait complete. 2024-02-02T13:30:19.101 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:30:19.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.307+0000 7fe947600640 1 Processor -- start 2024-02-02T13:30:19.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.307+0000 7fe947600640 1 -- start start 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe94015b2f0 0x7fe9400b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 0x7fe9400b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe940071650 0x7fe9400b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94015fdc0 con 0x7fe940071650 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94015b6d0 con 0x7fe9400711f0 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe947600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015f1b0 con 0x7fe94015b2f0 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe945600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe940071650 0x7fe9400b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe945600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe940071650 0x7fe9400b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56358/0 (socket says 172.21.15.91:56358) 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.309+0000 7fe945600640 1 -- 172.21.15.91:0/1907567388 learned_addr learned my addr 172.21.15.91:0/1907567388 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe944c00640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe94015b2f0 0x7fe9400b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe93fe00640 1 --2- 172.21.15.91:0/1907567388 >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 0x7fe9400b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe945600640 1 -- 172.21.15.91:0/1907567388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe94015b2f0 msgr2=0x7fe9400b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe945600640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe94015b2f0 0x7fe9400b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe945600640 1 -- 172.21.15.91:0/1907567388 >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 msgr2=0x7fe9400b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe945600640 1 --2- 172.21.15.91:0/1907567388 >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 0x7fe9400b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe945600640 1 -- 172.21.15.91:0/1907567388 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94015f490 con 0x7fe940071650 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.310+0000 7fe93fe00640 1 --2- 172.21.15.91:0/1907567388 >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 0x7fe9400b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.311+0000 7fe945600640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe940071650 0x7fe9400b0bd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe94006d690 tx=0x7fe93802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8b37409b850045 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.311+0000 7fe93f400640 1 -- 172.21.15.91:0/1907567388 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93802c9e0 con 0x7fe940071650 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.311+0000 7fe93f400640 1 -- 172.21.15.91:0/1907567388 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93802cb40 con 0x7fe940071650 2024-02-02T13:30:19.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.311+0000 7fe93f400640 1 -- 172.21.15.91:0/1907567388 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380343f0 con 0x7fe940071650 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 -- 172.21.15.91:0/1907567388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe940071650 msgr2=0x7fe9400b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe940071650 0x7fe9400b0bd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe94006d690 tx=0x7fe93802be40 comp rx=0 tx=0).stop 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 -- 172.21.15.91:0/1907567388 shutdown_connections 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe940071650 0x7fe9400b0bd0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 --2- 172.21.15.91:0/1907567388 >> v2:172.21.15.113:3300/0 conn(0x7fe9400711f0 0x7fe9400b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 --2- 172.21.15.91:0/1907567388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe94015b2f0 0x7fe9400b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 -- 172.21.15.91:0/1907567388 >> 172.21.15.91:0/1907567388 conn(0x7fe9400ceb90 msgr2=0x7fe9400cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 -- 172.21.15.91:0/1907567388 shutdown_connections 2024-02-02T13:30:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.312+0000 7fe947600640 1 -- 172.21.15.91:0/1907567388 wait complete. 2024-02-02T13:30:19.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.314+0000 7fe947600640 1 Processor -- start 2024-02-02T13:30:19.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.314+0000 7fe947600640 1 -- start start 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9400711f0 0x7fe9400a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe940071650 0x7fe940141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940142460 con 0x7fe94015b2f0 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940158b50 con 0x7fe9400711f0 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe947600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe940158e30 con 0x7fe940071650 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:56360/0 (socket says 172.21.15.91:56360) 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 -- 172.21.15.91:0/563552843 learned_addr learned my addr 172.21.15.91:0/563552843 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe93fe00640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe940071650 0x7fe940141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe944c00640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9400711f0 0x7fe9400a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 -- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe940071650 msgr2=0x7fe940141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:19.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe940071650 0x7fe940141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 -- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9400711f0 msgr2=0x7fe9400a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:19.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9400711f0 0x7fe9400a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:19.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.315+0000 7fe945600640 1 -- 172.21.15.91:0/563552843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938009b30 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe945600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe938036040 tx=0x7fe938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938009510 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93803e450 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93803e7e0 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe940159110 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.316+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe940142050 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.317+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe940159300 con 0x7fe94015b2f0 2024-02-02T13:30:19.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.318+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe93803e940 con 0x7fe94015b2f0 2024-02-02T13:30:19.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.319+0000 7fe93cc00640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 0x7fe91806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:19.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.319+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9380ab680 con 0x7fe94015b2f0 2024-02-02T13:30:19.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.319+0000 7fe944c00640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 0x7fe91806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:19.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.320+0000 7fe944c00640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 0x7fe91806c8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:19.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.324+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe938076c70 con 0x7fe94015b2f0 2024-02-02T13:30:19.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:19.536+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe940142890 con 0x7fe94015b2f0 2024-02-02T13:30:20.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.369+0000 7fe93cc00640 1 -- 172.21.15.91:0/563552843 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe93807a290 con 0x7fe94015b2f0 2024-02-02T13:30:20.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.372+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 msgr2=0x7fe91806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.372+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 0x7fe91806c8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930008040 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.372+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 msgr2=0x7fe940158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.372+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe938036040 tx=0x7fe938004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 shutdown_connections 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe91806a430 0x7fe91806c8f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe940071650 0x7fe940141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe94015b2f0 0x7fe940158230 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 --2- 172.21.15.91:0/563552843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9400711f0 0x7fe9400a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 >> 172.21.15.91:0/563552843 conn(0x7fe9400ceb90 msgr2=0x7fe94006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 shutdown_connections 2024-02-02T13:30:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.373+0000 7fe947600640 1 -- 172.21.15.91:0/563552843 wait complete. 2024-02-02T13:30:20.396 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:30:20.396 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:30:20.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.605+0000 7f9317800640 1 Processor -- start 2024-02-02T13:30:20.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.605+0000 7f9317800640 1 -- start start 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.607+0000 7f9317800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f931015d3c0 0x7f9310070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.607+0000 7f9317800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f93100707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.607+0000 7f9317800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93100cd930 con 0x7f93100b1f60 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.607+0000 7f9317800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93100cde60 con 0x7f931015d3c0 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.607+0000 7f9317800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93100ce520 con 0x7f93100b24b0 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f9315800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f93100b1f60 0x7f9310160f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46834/0 (socket says 172.21.15.91:46834) 2024-02-02T13:30:20.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f9315800640 1 -- 172.21.15.91:0/1863067837 learned_addr learned my addr 172.21.15.91:0/1863067837 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f9314e00640 1 --2- 172.21.15.91:0/1863067837 >> v2:172.21.15.113:3300/0 conn(0x7f931015d3c0 0x7f9310070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f9307c00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f93100707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213362669 0 0) 0x7f93100cd930 con 0x7f93100b1f60 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92f0003520 con 0x7f93100b1f60 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006898119 0 0) 0x7f92f0003520 con 0x7f93100b1f60 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 msgr2=0x7f93100707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:20.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f93100707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 >> v2:172.21.15.113:3300/0 conn(0x7f931015d3c0 msgr2=0x7f9310070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 --2- 172.21.15.91:0/1863067837 >> v2:172.21.15.113:3300/0 conn(0x7f931015d3c0 0x7f9310070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.608+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100cdb80 con 0x7f93100b1f60 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f93080019f0 con 0x7f93100b1f60 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 >> v1:172.21.15.91:6789/0 conn(0x7f93100b1f60 legacy=0x7f9310160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f92f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0003ef0 0x7f92f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f0004d00 0x7f92f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100cde60 con 0x7f92f0004d00 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100ce520 con 0x7f92f0003ef0 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f000b9d0 con 0x7f93100b24b0 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f0004d00 0x7f92f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9307c00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0003ef0 0x7f92f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9314e00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f92f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 -- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 msgr2=0x7f92f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100b24b0 0x7f92f0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 -- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0003ef0 msgr2=0x7f92f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0003ef0 0x7f92f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9315800640 1 -- 172.21.15.91:0/1863067837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0003520 con 0x7f92f0004d00 2024-02-02T13:30:20.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.609+0000 7f9307c00640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0003ef0 0x7f92f00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.610+0000 7f9315800640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f0004d00 0x7f92f0005180 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f930800a890 tx=0x7f92f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c1427dbae72c88 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308034630 con 0x7f92f0004d00 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930802f070 con 0x7f92f0004d00 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f930fe00640 1 -- 172.21.15.91:0/1863067837 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308034a50 con 0x7f92f0004d00 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9317800640 1 -- 172.21.15.91:0/1863067837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f0004d00 msgr2=0x7f92f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9317800640 1 --2- 172.21.15.91:0/1863067837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92f0004d00 0x7f92f0005180 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f930800a890 tx=0x7f92f00057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9315800640 1 -- 172.21.15.91:0/1863067837 reap_dead start 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9317800640 1 -- 172.21.15.91:0/1863067837 shutdown_connections 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9317800640 1 -- 172.21.15.91:0/1863067837 >> 172.21.15.91:0/1863067837 conn(0x7f93100cfa30 msgr2=0x7f93100cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.611+0000 7f9317800640 1 -- 172.21.15.91:0/1863067837 shutdown_connections 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.612+0000 7f9317800640 1 -- 172.21.15.91:0/1863067837 wait complete. 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.613+0000 7f9317800640 1 Processor -- start 2024-02-02T13:30:20.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.613+0000 7f9317800640 1 -- start start 2024-02-02T13:30:20.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0004d00 0x7f9310148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9310156710 0x7f9310156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310158730 con 0x7f9310157d70 2024-02-02T13:30:20.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310152f00 con 0x7f92f0004d00 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.614+0000 7f9317800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9310148850 con 0x7f9310156710 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33350/0 (socket says 172.21.15.91:33350) 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 -- 172.21.15.91:0/930616698 learned_addr learned my addr 172.21.15.91:0/930616698 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 -- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9310156710 msgr2=0x7f9310156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9310156710 0x7f9310156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9314e00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0004d00 0x7f9310148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 -- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0004d00 msgr2=0x7f9310148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0004d00 0x7f9310148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 -- 172.21.15.91:0/930616698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310157490 con 0x7f9310157d70 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.615+0000 7f9307c00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f92f8010810 tx=0x7f92f8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.616+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800eb00 con 0x7f9310157d70 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.616+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800ec60 con 0x7f9310157d70 2024-02-02T13:30:20.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.616+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f801c400 con 0x7f9310157d70 2024-02-02T13:30:20.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.616+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9310157770 con 0x7f9310157d70 2024-02-02T13:30:20.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.616+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9310157a50 con 0x7f9310157d70 2024-02-02T13:30:20.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.617+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9310066cf0 con 0x7f9310157d70 2024-02-02T13:30:20.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.618+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92f801c560 con 0x7f9310157d70 2024-02-02T13:30:20.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.619+0000 7f930d600640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 0x7f92e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:20.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.619+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92f8088d90 con 0x7f9310157d70 2024-02-02T13:30:20.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.619+0000 7f9314e00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 0x7f92e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:20.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.620+0000 7f9314e00640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 0x7f92e806c920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:20.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.624+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f92f8054410 con 0x7f9310157d70 2024-02-02T13:30:20.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:20.851+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9310070f00 con 0x7f9310157d70 2024-02-02T13:30:21.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.681+0000 7f930d600640 1 -- 172.21.15.91:0/930616698 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92f8057a30 con 0x7f9310157d70 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 msgr2=0x7f92e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 0x7f92e806c920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300008040 comp rx=0 tx=0).stop 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 msgr2=0x7f93101581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f92f8010810 tx=0x7f92f8010cd0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 shutdown_connections 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92e806a460 0x7f92e806c920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9310156710 0x7f9310156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9310157d70 0x7f93101581f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 --2- 172.21.15.91:0/930616698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f0004d00 0x7f9310148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 >> 172.21.15.91:0/930616698 conn(0x7f93100cfa30 msgr2=0x7f93100d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:21.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 shutdown_connections 2024-02-02T13:30:21.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.685+0000 7f9317800640 1 -- 172.21.15.91:0/930616698 wait complete. 2024-02-02T13:30:21.707 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:30:21.708 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:21.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.907+0000 7f02b3c00640 1 Processor -- start 2024-02-02T13:30:21.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.907+0000 7f02b3c00640 1 -- start start 2024-02-02T13:30:21.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b3c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 0x7f02ac0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b3c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02ac0b0c40 0x7f02ac0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b3c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0b1070 con 0x7f02ac151f90 2024-02-02T13:30:21.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b3c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac15fa30 con 0x7f02ac0b0c40 2024-02-02T13:30:21.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b3c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac070f80 con 0x7f02ac15f5a0 2024-02-02T13:30:21.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02ac0b0c40 0x7f02ac0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f02ac151f90 0x7f02ac06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46850/0 (socket says 172.21.15.91:46850) 2024-02-02T13:30:21.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1200640 1 -- 172.21.15.91:0/3361666052 learned_addr learned my addr 172.21.15.91:0/3361666052 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:21.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02abe00640 1 --2- 172.21.15.91:0/3361666052 >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 0x7f02ac0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1c00640 1 -- 172.21.15.91:0/3361666052 >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 msgr2=0x7f02ac0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:21.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1c00640 1 --2- 172.21.15.91:0/3361666052 >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 0x7f02ac0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1c00640 1 -- 172.21.15.91:0/3361666052 >> v1:172.21.15.91:6789/0 conn(0x7f02ac151f90 legacy=0x7f02ac06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.909+0000 7f02b1c00640 1 -- 172.21.15.91:0/3361666052 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac152370 con 0x7f02ac0b0c40 2024-02-02T13:30:21.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.910+0000 7f02abe00640 1 --2- 172.21.15.91:0/3361666052 >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 0x7f02ac0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b1c00640 1 --2- 172.21.15.91:0/3361666052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0b3af0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f02ac071a00 tx=0x7f02a402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4525baec4bfac8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02ab400640 1 -- 172.21.15.91:0/3361666052 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a402cc70 con 0x7f02ac0b0c40 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02ab400640 1 -- 172.21.15.91:0/3361666052 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a402cdd0 con 0x7f02ac0b0c40 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02ab400640 1 -- 172.21.15.91:0/3361666052 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4034660 con 0x7f02ac0b0c40 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 -- 172.21.15.91:0/3361666052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac0b0c40 msgr2=0x7f02ac0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 --2- 172.21.15.91:0/3361666052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0b3af0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f02ac071a00 tx=0x7f02a402bec0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 -- 172.21.15.91:0/3361666052 shutdown_connections 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 --2- 172.21.15.91:0/3361666052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0b3af0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 --2- 172.21.15.91:0/3361666052 >> v2:172.21.15.113:3300/0 conn(0x7f02ac15f5a0 0x7f02ac0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.911+0000 7f02b3c00640 1 -- 172.21.15.91:0/3361666052 >> 172.21.15.91:0/3361666052 conn(0x7f02ac0cec80 msgr2=0x7f02ac0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:21.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.912+0000 7f02b3c00640 1 -- 172.21.15.91:0/3361666052 shutdown_connections 2024-02-02T13:30:21.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.912+0000 7f02b3c00640 1 -- 172.21.15.91:0/3361666052 wait complete. 2024-02-02T13:30:21.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.913+0000 7f02b3c00640 1 Processor -- start 2024-02-02T13:30:21.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.913+0000 7f02b3c00640 1 -- start start 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac151f90 0x7f02ac0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac15b690 con 0x7f02ac15f5a0 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0c4db0 con 0x7f02ac0b0c40 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0c5090 con 0x7f02ac151f90 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.914+0000 7f02b1c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33360/0 (socket says 172.21.15.91:33360) 2024-02-02T13:30:21.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 -- 172.21.15.91:0/2207638730 learned_addr learned my addr 172.21.15.91:0/2207638730 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:21.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1200640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 -- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac151f90 msgr2=0x7f02ac0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02abe00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac151f90 0x7f02ac0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac151f90 0x7f02ac0c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 -- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 msgr2=0x7f02ac0c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 -- 172.21.15.91:0/2207638730 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4009a70 con 0x7f02ac15f5a0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1200640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.915+0000 7f02b1c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009e80 tx=0x7f02a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.916+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4040650 con 0x7f02ac15f5a0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.916+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a40407b0 con 0x7f02ac15f5a0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.916+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac0c5370 con 0x7f02ac15f5a0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.916+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4040b60 con 0x7f02ac15f5a0 2024-02-02T13:30:21.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.916+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac14c810 con 0x7f02ac15f5a0 2024-02-02T13:30:21.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.917+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac15b8b0 con 0x7f02ac15f5a0 2024-02-02T13:30:21.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.918+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f02a4040cc0 con 0x7f02ac15f5a0 2024-02-02T13:30:21.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.919+0000 7f02a8c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 0x7f028406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:21.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.919+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02a40ab730 con 0x7f02ac15f5a0 2024-02-02T13:30:21.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.920+0000 7f02b1200640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 0x7f028406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:21.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.920+0000 7f02b1200640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 0x7f028406c8f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0294007fd0 tx=0x7f0294009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:21.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:21.924+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f02ac15b8b0 con 0x7f02ac15f5a0 2024-02-02T13:30:22.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.155+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02ac15b8b0 con 0x7f02ac15f5a0 2024-02-02T13:30:22.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.156+0000 7f02a8c00640 1 -- 172.21.15.91:0/2207638730 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f02ac15b8b0 con 0x7f02ac15f5a0 2024-02-02T13:30:22.158 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:22.158 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:30:22.158 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:30:22.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 msgr2=0x7f028406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 0x7f028406c8f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0294007fd0 tx=0x7f0294009040 comp rx=0 tx=0).stop 2024-02-02T13:30:22.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 msgr2=0x7f02ac0c2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009e80 tx=0x7f02a4004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 shutdown_connections 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f028406a430 0x7f028406c8f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac151f90 0x7f02ac0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02ac15f5a0 0x7f02ac0c2c60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 --2- 172.21.15.91:0/2207638730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac0b0c40 0x7f02ac0c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 >> 172.21.15.91:0/2207638730 conn(0x7f02ac0cec80 msgr2=0x7f02ac06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 shutdown_connections 2024-02-02T13:30:22.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.159+0000 7f02b3c00640 1 -- 172.21.15.91:0/2207638730 wait complete. 2024-02-02T13:30:22.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:22.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.372+0000 7f44b6800640 1 Processor -- start 2024-02-02T13:30:22.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.372+0000 7f44b6800640 1 -- start start 2024-02-02T13:30:22.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.373+0000 7f44b6800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f44b00b0e10 0x7f44b00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44b6800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b0070f80 con 0x7f44b015f5a0 2024-02-02T13:30:22.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44b6800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b006c600 con 0x7f44b0151f90 2024-02-02T13:30:22.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44b6800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b015fa30 con 0x7f44b00b0e10 2024-02-02T13:30:22.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44aea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f44b015f5a0 0x7f44b00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46858/0 (socket says 172.21.15.91:46858) 2024-02-02T13:30:22.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44aea00640 1 -- 172.21.15.91:0/3535520592 learned_addr learned my addr 172.21.15.91:0/3535520592 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:22.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44afe00640 1 --2- 172.21.15.91:0/3535520592 >> v2:172.21.15.91:3300/0 conn(0x7f44b00b0e10 0x7f44b00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44ae000640 1 -- 172.21.15.91:0/3535520592 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375689228 0 0) 0x7f44b0070f80 con 0x7f44b015f5a0 2024-02-02T13:30:22.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.374+0000 7f44ae000640 1 -- 172.21.15.91:0/3535520592 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4490003520 con 0x7f44b015f5a0 2024-02-02T13:30:22.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.375+0000 7f44afe00640 1 -- 172.21.15.91:0/3535520592 >> v1:172.21.15.113:6789/0 conn(0x7f44b0151f90 legacy=0x7f44b006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.375+0000 7f44afe00640 1 -- 172.21.15.91:0/3535520592 >> v1:172.21.15.91:6789/0 conn(0x7f44b015f5a0 legacy=0x7f44b00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.375+0000 7f44afe00640 1 -- 172.21.15.91:0/3535520592 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0152370 con 0x7f44b00b0e10 2024-02-02T13:30:22.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.376+0000 7f44afe00640 1 --2- 172.21.15.91:0/3535520592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b00b0e10 0x7f44b00b3af0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009d90 tx=0x7f44a802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf045d7b134d97f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.376+0000 7f44ae000640 1 -- 172.21.15.91:0/3535520592 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a802cd00 con 0x7f44b00b0e10 2024-02-02T13:30:22.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.376+0000 7f44ae000640 1 -- 172.21.15.91:0/3535520592 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a802ce60 con 0x7f44b00b0e10 2024-02-02T13:30:22.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.376+0000 7f44ae000640 1 -- 172.21.15.91:0/3535520592 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8034660 con 0x7f44b00b0e10 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 -- 172.21.15.91:0/3535520592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b00b0e10 msgr2=0x7f44b00b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 --2- 172.21.15.91:0/3535520592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b00b0e10 0x7f44b00b3af0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009d90 tx=0x7f44a802bf50 comp rx=0 tx=0).stop 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 -- 172.21.15.91:0/3535520592 shutdown_connections 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 --2- 172.21.15.91:0/3535520592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b00b0e10 0x7f44b00b3af0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 -- 172.21.15.91:0/3535520592 >> 172.21.15.91:0/3535520592 conn(0x7f44b00cec80 msgr2=0x7f44b00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 -- 172.21.15.91:0/3535520592 shutdown_connections 2024-02-02T13:30:22.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.377+0000 7f44b6800640 1 -- 172.21.15.91:0/3535520592 wait complete. 2024-02-02T13:30:22.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.379+0000 7f44b6800640 1 Processor -- start 2024-02-02T13:30:22.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.379+0000 7f44b6800640 1 -- start start 2024-02-02T13:30:22.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b00b0e10 0x7f44b00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44b015f5a0 0x7f44b01ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c2720 con 0x7f44b0151f90 2024-02-02T13:30:22.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c2ce0 con 0x7f44b00b0e10 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44b6800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b015b880 con 0x7f44b015f5a0 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44aea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44aea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33380/0 (socket says 172.21.15.91:33380) 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44aea00640 1 -- 172.21.15.91:0/49205999 learned_addr learned my addr 172.21.15.91:0/49205999 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44af400640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b00b0e10 0x7f44b00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44afe00640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44b015f5a0 0x7f44b01ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.380+0000 7f44aea00640 1 -- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44b015f5a0 msgr2=0x7f44b01ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44aea00640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44b015f5a0 0x7f44b01ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44aea00640 1 -- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b00b0e10 msgr2=0x7f44b00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44aea00640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b00b0e10 0x7f44b00c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44aea00640 1 -- 172.21.15.91:0/49205999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8009a70 con 0x7f44b0151f90 2024-02-02T13:30:22.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44aea00640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f449800c500 tx=0x7f449800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498005ce0 con 0x7f44b0151f90 2024-02-02T13:30:22.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498005e40 con 0x7f44b0151f90 2024-02-02T13:30:22.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498007330 con 0x7f44b0151f90 2024-02-02T13:30:22.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b015bb60 con 0x7f44b0151f90 2024-02-02T13:30:22.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.381+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b00c8a50 con 0x7f44b0151f90 2024-02-02T13:30:22.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.383+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b0155990 con 0x7f44b0151f90 2024-02-02T13:30:22.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.383+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4498007490 con 0x7f44b0151f90 2024-02-02T13:30:22.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.384+0000 7f44a7400640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 0x7f448806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.384+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4498087ed0 con 0x7f44b0151f90 2024-02-02T13:30:22.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.385+0000 7f44af400640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 0x7f448806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.385+0000 7f44af400640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 0x7f448806c920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000f60 tx=0x7f44a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.389+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4498053610 con 0x7f44b0151f90 2024-02-02T13:30:22.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.581+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44b00711d0 con 0x7f44b0151f90 2024-02-02T13:30:22.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.583+0000 7f44a7400640 1 -- 172.21.15.91:0/49205999 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16103 (secure 0 0 0) 0x7f4498017090 con 0x7f44b0151f90 2024-02-02T13:30:22.585 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:22.585 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":105,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:19.145606+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 msgr2=0x7f448806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 0x7f448806c920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000f60 tx=0x7f44a0009040 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 msgr2=0x7f44b00c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f449800c500 tx=0x7f449800c9c0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 shutdown_connections 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f448806a460 0x7f448806c920 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44b015f5a0 0x7f44b01ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44b0151f90 0x7f44b00c8510 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 --2- 172.21.15.91:0/49205999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b00b0e10 0x7f44b00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 >> 172.21.15.91:0/49205999 conn(0x7f44b00cec80 msgr2=0x7f44b00cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:22.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 shutdown_connections 2024-02-02T13:30:22.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.587+0000 7f44b6800640 1 -- 172.21.15.91:0/49205999 wait complete. 2024-02-02T13:30:22.608 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:30:22.608 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:30:22.627 INFO:tasks.ceph.mds.d.smithi113.stderr:2024-02-02T13:30:22.626+0000 7f35b6a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:30:22.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.815+0000 7f8043e00640 1 Processor -- start 2024-02-02T13:30:22.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.815+0000 7f8043e00640 1 -- start start 2024-02-02T13:30:22.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80440adf00 0x7f8044159b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440ae2e0 con 0x7f804406c3f0 2024-02-02T13:30:22.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804415a4b0 con 0x7f80440c9170 2024-02-02T13:30:22.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804415a790 con 0x7f80440adf00 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f804406c3f0 0x7f80440ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46868/0 (socket says 172.21.15.91:46868) 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.817+0000 7f8043400640 1 -- 172.21.15.91:0/1393790436 learned_addr learned my addr 172.21.15.91:0/1393790436 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042a00640 1 --2- 172.21.15.91:0/1393790436 >> v2:172.21.15.113:3300/0 conn(0x7f80440adf00 0x7f8044159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410134354 0 0) 0x7f80440ae2e0 con 0x7f804406c3f0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8024003520 con 0x7f804406c3f0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004710029 0 0) 0x7f8024003520 con 0x7f804406c3f0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 >> v2:172.21.15.113:3300/0 conn(0x7f80440adf00 msgr2=0x7f8044159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 --2- 172.21.15.91:0/1393790436 >> v2:172.21.15.113:3300/0 conn(0x7f80440adf00 0x7f8044159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 >> v1:172.21.15.113:6789/0 conn(0x7f80440c9170 legacy=0x7f804406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804415aa70 con 0x7f804406c3f0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f803c0019f0 con 0x7f804406c3f0 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 >> v1:172.21.15.91:6789/0 conn(0x7f804406c3f0 legacy=0x7f80440ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8024005180 0x7f802400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415a790 con 0x7f8024003970 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802400c020 con 0x7f80240047c0 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.818+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802400c1e0 con 0x7f8024005180 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 -- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8024005180 msgr2=0x7f802400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8024005180 0x7f802400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 -- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 msgr2=0x7f8024004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8039e00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8043400640 1 -- 172.21.15.91:0/1393790436 reap_dead start 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8042a00640 1 -- 172.21.15.91:0/1393790436 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024003520 con 0x7f8024003970 2024-02-02T13:30:22.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.819+0000 7f8039e00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:22.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.820+0000 7f8042a00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 0x7f8024003d90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f803400aac0 tx=0x7f80240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51df84cc8b926331 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034034600 con 0x7f8024003970 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034030050 con 0x7f8024003970 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8042000640 1 -- 172.21.15.91:0/1393790436 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034034980 con 0x7f8024003970 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 -- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 msgr2=0x7f8024003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 0x7f8024003d90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f803400aac0 tx=0x7f80240057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 -- 172.21.15.91:0/1393790436 shutdown_connections 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 --2- 172.21.15.91:0/1393790436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 -- 172.21.15.91:0/1393790436 >> 172.21.15.91:0/1393790436 conn(0x7f80440cea90 msgr2=0x7f80440ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 -- 172.21.15.91:0/1393790436 shutdown_connections 2024-02-02T13:30:22.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.821+0000 7f8043e00640 1 -- 172.21.15.91:0/1393790436 wait complete. 2024-02-02T13:30:22.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.823+0000 7f8043e00640 1 Processor -- start 2024-02-02T13:30:22.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.823+0000 7f8043e00640 1 -- start start 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8044152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 0x7f804415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044153000 con 0x7f80440c6ab0 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441532e0 con 0x7f8024003970 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.824+0000 7f8043e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441535c0 con 0x7f8044150ee0 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33398/0 (socket says 172.21.15.91:33398) 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 -- 172.21.15.91:0/4053721836 learned_addr learned my addr 172.21.15.91:0/4053721836 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8042a00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8044152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8043400640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 0x7f804415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 -- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 msgr2=0x7f804415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 0x7f804415bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 -- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 msgr2=0x7f8044152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:22.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8044152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:22.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 -- 172.21.15.91:0/4053721836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80441538a0 con 0x7f80440c6ab0 2024-02-02T13:30:22.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8043400640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 0x7f804415bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:22.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.825+0000 7f8039e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d430 tx=0x7f802c00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.826+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00b070 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.826+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c0054e0 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.826+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c003830 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.826+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8044153b80 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.826+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80440c6f30 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.827+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804404dff0 con 0x7f80440c6ab0 2024-02-02T13:30:22.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.828+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f802c003990 con 0x7f80440c6ab0 2024-02-02T13:30:22.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.829+0000 7f803b400640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 0x7f801c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:22.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.829+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f802c088160 con 0x7f80440c6ab0 2024-02-02T13:30:22.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.829+0000 7f8042a00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 0x7f801c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:22.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.830+0000 7f8042a00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 0x7f801c06c8f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8034037000 tx=0x7f8034032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:22.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:22.834+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f802c053760 con 0x7f80440c6ab0 2024-02-02T13:30:23.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.022+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80440cc9e0 con 0x7f801c06a430 2024-02-02T13:30:23.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.034+0000 7f803b400640 1 -- 172.21.15.91:0/4053721836 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282908 (secure 0 0 0) 0x7f80340043f0 con 0x7f801c06a430 2024-02-02T13:30:23.040 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:23.042 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:30:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 msgr2=0x7f801c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 0x7f801c06c8f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8034037000 tx=0x7f8034032000 comp rx=0 tx=0).stop 2024-02-02T13:30:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 msgr2=0x7f80441540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d430 tx=0x7f802c00d8f0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 shutdown_connections 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a430 0x7f801c06c8f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044150ee0 0x7f804415bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80440c6ab0 0x7f80441540a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 --2- 172.21.15.91:0/4053721836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8044152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 >> 172.21.15.91:0/4053721836 conn(0x7f80440cea90 msgr2=0x7f80440aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 shutdown_connections 2024-02-02T13:30:23.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.040+0000 7f8043e00640 1 -- 172.21.15.91:0/4053721836 wait complete. 2024-02-02T13:30:23.061 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-02-02T13:30:22.350175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5603,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556896,"statfs":{"total":1600000000,"available":1594262040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.699576"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372561+0000","last_change":"2024-02-02T13:30:18.168830+0000","last_active":"2024-02-02T13:30:19.372561+0000","last_peered":"2024-02-02T13:30:19.372561+0000","last_clean":"2024-02-02T13:30:19.372561+0000","last_became_active":"2024-02-02T13:30:18.168587+0000","last_became_peered":"2024-02-02T13:30:18.168587+0000","last_unstale":"2024-02-02T13:30:19.372561+0000","last_undegraded":"2024-02-02T13:30:19.372561+0000","last_fullsized":"2024-02-02T13:30:19.372561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:09.853880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368686+0000","last_change":"2024-02-02T13:30:18.172516+0000","last_active":"2024-02-02T13:30:19.368686+0000","last_peered":"2024-02-02T13:30:19.368686+0000","last_clean":"2024-02-02T13:30:19.368686+0000","last_became_active":"2024-02-02T13:30:18.172119+0000","last_became_peered":"2024-02-02T13:30:18.172119+0000","last_unstale":"2024-02-02T13:30:19.368686+0000","last_undegraded":"2024-02-02T13:30:19.368686+0000","last_fullsized":"2024-02-02T13:30:19.368686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:05:40.191518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371761+0000","last_change":"2024-02-02T13:30:18.171905+0000","last_active":"2024-02-02T13:30:19.371761+0000","last_peered":"2024-02-02T13:30:19.371761+0000","last_clean":"2024-02-02T13:30:19.371761+0000","last_became_active":"2024-02-02T13:30:18.171686+0000","last_became_peered":"2024-02-02T13:30:18.171686+0000","last_unstale":"2024-02-02T13:30:19.371761+0000","last_undegraded":"2024-02-02T13:30:19.371761+0000","last_fullsized":"2024-02-02T13:30:19.371761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:30:12.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156611+0000","last_change":"2024-02-02T13:30:18.166659+0000","last_active":"2024-02-02T13:30:19.156611+0000","last_peered":"2024-02-02T13:30:19.156611+0000","last_clean":"2024-02-02T13:30:19.156611+0000","last_became_active":"2024-02-02T13:30:18.166428+0000","last_became_peered":"2024-02-02T13:30:18.166428+0000","last_unstale":"2024-02-02T13:30:19.156611+0000","last_undegraded":"2024-02-02T13:30:19.156611+0000","last_fullsized":"2024-02-02T13:30:19.156611+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:10.177166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372163+0000","last_change":"2024-02-02T13:30:18.163024+0000","last_active":"2024-02-02T13:30:19.372163+0000","last_peered":"2024-02-02T13:30:19.372163+0000","last_clean":"2024-02-02T13:30:19.372163+0000","last_became_active":"2024-02-02T13:30:18.162787+0000","last_became_peered":"2024-02-02T13:30:18.162787+0000","last_unstale":"2024-02-02T13:30:19.372163+0000","last_undegraded":"2024-02-02T13:30:19.372163+0000","last_fullsized":"2024-02-02T13:30:19.372163+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:04.103118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371196+0000","last_change":"2024-02-02T13:30:18.162958+0000","last_active":"2024-02-02T13:30:19.371196+0000","last_peered":"2024-02-02T13:30:19.371196+0000","last_clean":"2024-02-02T13:30:19.371196+0000","last_became_active":"2024-02-02T13:30:18.162542+0000","last_became_peered":"2024-02-02T13:30:18.162542+0000","last_unstale":"2024-02-02T13:30:19.371196+0000","last_undegraded":"2024-02-02T13:30:19.371196+0000","last_fullsized":"2024-02-02T13:30:19.371196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:37.867876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156780+0000","last_change":"2024-02-02T13:30:18.173642+0000","last_active":"2024-02-02T13:30:19.156780+0000","last_peered":"2024-02-02T13:30:19.156780+0000","last_clean":"2024-02-02T13:30:19.156780+0000","last_became_active":"2024-02-02T13:30:18.173203+0000","last_became_peered":"2024-02-02T13:30:18.173203+0000","last_unstale":"2024-02-02T13:30:19.156780+0000","last_undegraded":"2024-02-02T13:30:19.156780+0000","last_fullsized":"2024-02-02T13:30:19.156780+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:15.473677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373213+0000","last_change":"2024-02-02T13:30:18.154310+0000","last_active":"2024-02-02T13:30:19.373213+0000","last_peered":"2024-02-02T13:30:19.373213+0000","last_clean":"2024-02-02T13:30:19.373213+0000","last_became_active":"2024-02-02T13:30:18.153381+0000","last_became_peered":"2024-02-02T13:30:18.153381+0000","last_unstale":"2024-02-02T13:30:19.373213+0000","last_undegraded":"2024-02-02T13:30:19.373213+0000","last_fullsized":"2024-02-02T13:30:19.373213+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:39:59.563638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371140+0000","last_change":"2024-02-02T13:30:18.179751+0000","last_active":"2024-02-02T13:30:19.371140+0000","last_peered":"2024-02-02T13:30:19.371140+0000","last_clean":"2024-02-02T13:30:19.371140+0000","last_became_active":"2024-02-02T13:30:18.178618+0000","last_became_peered":"2024-02-02T13:30:18.178618+0000","last_unstale":"2024-02-02T13:30:19.371140+0000","last_undegraded":"2024-02-02T13:30:19.371140+0000","last_fullsized":"2024-02-02T13:30:19.371140+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:49:17.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371716+0000","last_change":"2024-02-02T13:30:18.172938+0000","last_active":"2024-02-02T13:30:19.371716+0000","last_peered":"2024-02-02T13:30:19.371716+0000","last_clean":"2024-02-02T13:30:19.371716+0000","last_became_active":"2024-02-02T13:30:18.172170+0000","last_became_peered":"2024-02-02T13:30:18.172170+0000","last_unstale":"2024-02-02T13:30:19.371716+0000","last_undegraded":"2024-02-02T13:30:19.371716+0000","last_fullsized":"2024-02-02T13:30:19.371716+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:54:21.647131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369089+0000","last_change":"2024-02-02T13:30:18.162078+0000","last_active":"2024-02-02T13:30:19.369089+0000","last_peered":"2024-02-02T13:30:19.369089+0000","last_clean":"2024-02-02T13:30:19.369089+0000","last_became_active":"2024-02-02T13:30:18.161320+0000","last_became_peered":"2024-02-02T13:30:18.161320+0000","last_unstale":"2024-02-02T13:30:19.369089+0000","last_undegraded":"2024-02-02T13:30:19.369089+0000","last_fullsized":"2024-02-02T13:30:19.369089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:32.532884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373045+0000","last_change":"2024-02-02T13:30:18.175484+0000","last_active":"2024-02-02T13:30:19.373045+0000","last_peered":"2024-02-02T13:30:19.373045+0000","last_clean":"2024-02-02T13:30:19.373045+0000","last_became_active":"2024-02-02T13:30:18.174846+0000","last_became_peered":"2024-02-02T13:30:18.174846+0000","last_unstale":"2024-02-02T13:30:19.373045+0000","last_undegraded":"2024-02-02T13:30:19.373045+0000","last_fullsized":"2024-02-02T13:30:19.373045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:56.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371993+0000","last_change":"2024-02-02T13:30:18.166954+0000","last_active":"2024-02-02T13:30:19.371993+0000","last_peered":"2024-02-02T13:30:19.371993+0000","last_clean":"2024-02-02T13:30:19.371993+0000","last_became_active":"2024-02-02T13:30:18.166705+0000","last_became_peered":"2024-02-02T13:30:18.166705+0000","last_unstale":"2024-02-02T13:30:19.371993+0000","last_undegraded":"2024-02-02T13:30:19.371993+0000","last_fullsized":"2024-02-02T13:30:19.371993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:02:21.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156718+0000","last_change":"2024-02-02T13:30:18.164270+0000","last_active":"2024-02-02T13:30:19.156718+0000","last_peered":"2024-02-02T13:30:19.156718+0000","last_clean":"2024-02-02T13:30:19.156718+0000","last_became_active":"2024-02-02T13:30:18.163877+0000","last_became_peered":"2024-02-02T13:30:18.163877+0000","last_unstale":"2024-02-02T13:30:19.156718+0000","last_undegraded":"2024-02-02T13:30:19.156718+0000","last_fullsized":"2024-02-02T13:30:19.156718+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:35.636539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155542+0000","last_change":"2024-02-02T13:30:18.161364+0000","last_active":"2024-02-02T13:30:19.155542+0000","last_peered":"2024-02-02T13:30:19.155542+0000","last_clean":"2024-02-02T13:30:19.155542+0000","last_became_active":"2024-02-02T13:30:18.161140+0000","last_became_peered":"2024-02-02T13:30:18.161140+0000","last_unstale":"2024-02-02T13:30:19.155542+0000","last_undegraded":"2024-02-02T13:30:19.155542+0000","last_fullsized":"2024-02-02T13:30:19.155542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:30:12.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370919+0000","last_change":"2024-02-02T13:30:18.171599+0000","last_active":"2024-02-02T13:30:19.370919+0000","last_peered":"2024-02-02T13:30:19.370919+0000","last_clean":"2024-02-02T13:30:19.370919+0000","last_became_active":"2024-02-02T13:30:18.171341+0000","last_became_peered":"2024-02-02T13:30:18.171341+0000","last_unstale":"2024-02-02T13:30:19.370919+0000","last_undegraded":"2024-02-02T13:30:19.370919+0000","last_fullsized":"2024-02-02T13:30:19.370919+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:33.083769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372699+0000","last_change":"2024-02-02T13:30:18.169351+0000","last_active":"2024-02-02T13:30:19.372699+0000","last_peered":"2024-02-02T13:30:19.372699+0000","last_clean":"2024-02-02T13:30:19.372699+0000","last_became_active":"2024-02-02T13:30:18.168944+0000","last_became_peered":"2024-02-02T13:30:18.168944+0000","last_unstale":"2024-02-02T13:30:19.372699+0000","last_undegraded":"2024-02-02T13:30:19.372699+0000","last_fullsized":"2024-02-02T13:30:19.372699+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:39:59.563638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156997+0000","last_change":"2024-02-02T13:30:18.175763+0000","last_active":"2024-02-02T13:30:19.156997+0000","last_peered":"2024-02-02T13:30:19.156997+0000","last_clean":"2024-02-02T13:30:19.156997+0000","last_became_active":"2024-02-02T13:30:18.174920+0000","last_became_peered":"2024-02-02T13:30:18.174920+0000","last_unstale":"2024-02-02T13:30:19.156997+0000","last_undegraded":"2024-02-02T13:30:19.156997+0000","last_fullsized":"2024-02-02T13:30:19.156997+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:00:06.858544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158138+0000","last_change":"2024-02-02T13:30:18.177880+0000","last_active":"2024-02-02T13:30:19.158138+0000","last_peered":"2024-02-02T13:30:19.158138+0000","last_clean":"2024-02-02T13:30:19.158138+0000","last_became_active":"2024-02-02T13:30:18.176843+0000","last_became_peered":"2024-02-02T13:30:18.176843+0000","last_unstale":"2024-02-02T13:30:19.158138+0000","last_undegraded":"2024-02-02T13:30:19.158138+0000","last_fullsized":"2024-02-02T13:30:19.158138+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:49:17.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157042+0000","last_change":"2024-02-02T13:30:18.179286+0000","last_active":"2024-02-02T13:30:19.157042+0000","last_peered":"2024-02-02T13:30:19.157042+0000","last_clean":"2024-02-02T13:30:19.157042+0000","last_became_active":"2024-02-02T13:30:18.179045+0000","last_became_peered":"2024-02-02T13:30:18.179045+0000","last_unstale":"2024-02-02T13:30:19.157042+0000","last_undegraded":"2024-02-02T13:30:19.157042+0000","last_fullsized":"2024-02-02T13:30:19.157042+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:55:17.319782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157388+0000","last_change":"2024-02-02T13:30:18.179827+0000","last_active":"2024-02-02T13:30:19.157388+0000","last_peered":"2024-02-02T13:30:19.157388+0000","last_clean":"2024-02-02T13:30:19.157388+0000","last_became_active":"2024-02-02T13:30:18.177850+0000","last_became_peered":"2024-02-02T13:30:18.177850+0000","last_unstale":"2024-02-02T13:30:19.157388+0000","last_undegraded":"2024-02-02T13:30:19.157388+0000","last_fullsized":"2024-02-02T13:30:19.157388+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:41.922553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368909+0000","last_change":"2024-02-02T13:30:18.175659+0000","last_active":"2024-02-02T13:30:19.368909+0000","last_peered":"2024-02-02T13:30:19.368909+0000","last_clean":"2024-02-02T13:30:19.368909+0000","last_became_active":"2024-02-02T13:30:18.175451+0000","last_became_peered":"2024-02-02T13:30:18.175451+0000","last_unstale":"2024-02-02T13:30:19.368909+0000","last_undegraded":"2024-02-02T13:30:19.368909+0000","last_fullsized":"2024-02-02T13:30:19.368909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:14:47.703783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374103+0000","last_change":"2024-02-02T13:30:18.178031+0000","last_active":"2024-02-02T13:30:19.374103+0000","last_peered":"2024-02-02T13:30:19.374103+0000","last_clean":"2024-02-02T13:30:19.374103+0000","last_became_active":"2024-02-02T13:30:18.177494+0000","last_became_peered":"2024-02-02T13:30:18.177494+0000","last_unstale":"2024-02-02T13:30:19.374103+0000","last_undegraded":"2024-02-02T13:30:19.374103+0000","last_fullsized":"2024-02-02T13:30:19.374103+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156880+0000","last_change":"2024-02-02T13:30:18.180707+0000","last_active":"2024-02-02T13:30:19.156880+0000","last_peered":"2024-02-02T13:30:19.156880+0000","last_clean":"2024-02-02T13:30:19.156880+0000","last_became_active":"2024-02-02T13:30:18.178810+0000","last_became_peered":"2024-02-02T13:30:18.178810+0000","last_unstale":"2024-02-02T13:30:19.156880+0000","last_undegraded":"2024-02-02T13:30:19.156880+0000","last_fullsized":"2024-02-02T13:30:19.156880+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:04:09.568950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368746+0000","last_change":"2024-02-02T13:30:18.171852+0000","last_active":"2024-02-02T13:30:19.368746+0000","last_peered":"2024-02-02T13:30:19.368746+0000","last_clean":"2024-02-02T13:30:19.368746+0000","last_became_active":"2024-02-02T13:30:18.171560+0000","last_became_peered":"2024-02-02T13:30:18.171560+0000","last_unstale":"2024-02-02T13:30:19.368746+0000","last_undegraded":"2024-02-02T13:30:19.368746+0000","last_fullsized":"2024-02-02T13:30:19.368746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:02.316841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372659+0000","last_change":"2024-02-02T13:30:18.159264+0000","last_active":"2024-02-02T13:30:19.372659+0000","last_peered":"2024-02-02T13:30:19.372659+0000","last_clean":"2024-02-02T13:30:19.372659+0000","last_became_active":"2024-02-02T13:30:18.159017+0000","last_became_peered":"2024-02-02T13:30:18.159017+0000","last_unstale":"2024-02-02T13:30:19.372659+0000","last_undegraded":"2024-02-02T13:30:19.372659+0000","last_fullsized":"2024-02-02T13:30:19.372659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:01.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373350+0000","last_change":"2024-02-02T13:30:18.166076+0000","last_active":"2024-02-02T13:30:19.373350+0000","last_peered":"2024-02-02T13:30:19.373350+0000","last_clean":"2024-02-02T13:30:19.373350+0000","last_became_active":"2024-02-02T13:30:18.165808+0000","last_became_peered":"2024-02-02T13:30:18.165808+0000","last_unstale":"2024-02-02T13:30:19.373350+0000","last_undegraded":"2024-02-02T13:30:19.373350+0000","last_fullsized":"2024-02-02T13:30:19.373350+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:06.279475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373669+0000","last_change":"2024-02-02T13:30:18.173980+0000","last_active":"2024-02-02T13:30:19.373669+0000","last_peered":"2024-02-02T13:30:19.373669+0000","last_clean":"2024-02-02T13:30:19.373669+0000","last_became_active":"2024-02-02T13:30:18.173733+0000","last_became_peered":"2024-02-02T13:30:18.173733+0000","last_unstale":"2024-02-02T13:30:19.373669+0000","last_undegraded":"2024-02-02T13:30:19.373669+0000","last_fullsized":"2024-02-02T13:30:19.373669+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:03:45.072121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368615+0000","last_change":"2024-02-02T13:30:18.154020+0000","last_active":"2024-02-02T13:30:19.368615+0000","last_peered":"2024-02-02T13:30:19.368615+0000","last_clean":"2024-02-02T13:30:19.368615+0000","last_became_active":"2024-02-02T13:30:18.153724+0000","last_became_peered":"2024-02-02T13:30:18.153724+0000","last_unstale":"2024-02-02T13:30:19.368615+0000","last_undegraded":"2024-02-02T13:30:19.368615+0000","last_fullsized":"2024-02-02T13:30:19.368615+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:14:54.859623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368971+0000","last_change":"2024-02-02T13:30:18.162283+0000","last_active":"2024-02-02T13:30:19.368971+0000","last_peered":"2024-02-02T13:30:19.368971+0000","last_clean":"2024-02-02T13:30:19.368971+0000","last_became_active":"2024-02-02T13:30:18.162078+0000","last_became_peered":"2024-02-02T13:30:18.162078+0000","last_unstale":"2024-02-02T13:30:19.368971+0000","last_undegraded":"2024-02-02T13:30:19.368971+0000","last_fullsized":"2024-02-02T13:30:19.368971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:35:41.167599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371286+0000","last_change":"2024-02-02T13:30:18.176875+0000","last_active":"2024-02-02T13:30:19.371286+0000","last_peered":"2024-02-02T13:30:19.371286+0000","last_clean":"2024-02-02T13:30:19.371286+0000","last_became_active":"2024-02-02T13:30:18.175744+0000","last_became_peered":"2024-02-02T13:30:18.175744+0000","last_unstale":"2024-02-02T13:30:19.371286+0000","last_undegraded":"2024-02-02T13:30:19.371286+0000","last_fullsized":"2024-02-02T13:30:19.371286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:49.758240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373272+0000","last_change":"2024-02-02T13:30:18.172871+0000","last_active":"2024-02-02T13:30:19.373272+0000","last_peered":"2024-02-02T13:30:19.373272+0000","last_clean":"2024-02-02T13:30:19.373272+0000","last_became_active":"2024-02-02T13:30:18.172610+0000","last_became_peered":"2024-02-02T13:30:18.172610+0000","last_unstale":"2024-02-02T13:30:19.373272+0000","last_undegraded":"2024-02-02T13:30:19.373272+0000","last_fullsized":"2024-02-02T13:30:19.373272+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:33:54.971010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370059+0000","last_change":"2024-02-02T13:30:18.172097+0000","last_active":"2024-02-02T13:30:19.370059+0000","last_peered":"2024-02-02T13:30:19.370059+0000","last_clean":"2024-02-02T13:30:19.370059+0000","last_became_active":"2024-02-02T13:30:18.171870+0000","last_became_peered":"2024-02-02T13:30:18.171870+0000","last_unstale":"2024-02-02T13:30:19.370059+0000","last_undegraded":"2024-02-02T13:30:19.370059+0000","last_fullsized":"2024-02-02T13:30:19.370059+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"104'4","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374313+0000","last_change":"2024-02-02T13:30:17.696764+0000","last_active":"2024-02-02T13:30:19.374313+0000","last_peered":"2024-02-02T13:30:19.374313+0000","last_clean":"2024-02-02T13:30:19.374313+0000","last_became_active":"2024-02-02T13:30:17.692001+0000","last_became_peered":"2024-02-02T13:30:17.692001+0000","last_unstale":"2024-02-02T13:30:19.374313+0000","last_undegraded":"2024-02-02T13:30:19.374313+0000","last_fullsized":"2024-02-02T13:30:19.374313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:18:31.944480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373890+0000","last_change":"2024-02-02T13:30:18.178188+0000","last_active":"2024-02-02T13:30:19.373890+0000","last_peered":"2024-02-02T13:30:19.373890+0000","last_clean":"2024-02-02T13:30:19.373890+0000","last_became_active":"2024-02-02T13:30:18.177803+0000","last_became_peered":"2024-02-02T13:30:18.177803+0000","last_unstale":"2024-02-02T13:30:19.373890+0000","last_undegraded":"2024-02-02T13:30:19.373890+0000","last_fullsized":"2024-02-02T13:30:19.373890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:01.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370530+0000","last_change":"2024-02-02T13:30:17.692896+0000","last_active":"2024-02-02T13:30:19.370530+0000","last_peered":"2024-02-02T13:30:19.370530+0000","last_clean":"2024-02-02T13:30:19.370530+0000","last_became_active":"2024-02-02T13:30:17.692590+0000","last_became_peered":"2024-02-02T13:30:17.692590+0000","last_unstale":"2024-02-02T13:30:19.370530+0000","last_undegraded":"2024-02-02T13:30:19.370530+0000","last_fullsized":"2024-02-02T13:30:19.370530+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:35:40.153531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372812+0000","last_change":"2024-02-02T13:30:18.173330+0000","last_active":"2024-02-02T13:30:19.372812+0000","last_peered":"2024-02-02T13:30:19.372812+0000","last_clean":"2024-02-02T13:30:19.372812+0000","last_became_active":"2024-02-02T13:30:18.172818+0000","last_became_peered":"2024-02-02T13:30:18.172818+0000","last_unstale":"2024-02-02T13:30:19.372812+0000","last_undegraded":"2024-02-02T13:30:19.372812+0000","last_fullsized":"2024-02-02T13:30:19.372812+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:13:49.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158707+0000","last_change":"2024-02-02T13:30:17.692777+0000","last_active":"2024-02-02T13:30:19.158707+0000","last_peered":"2024-02-02T13:30:19.158707+0000","last_clean":"2024-02-02T13:30:19.158707+0000","last_became_active":"2024-02-02T13:30:17.691789+0000","last_became_peered":"2024-02-02T13:30:17.691789+0000","last_unstale":"2024-02-02T13:30:19.158707+0000","last_undegraded":"2024-02-02T13:30:19.158707+0000","last_fullsized":"2024-02-02T13:30:19.158707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:47.278881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372473+0000","last_change":"2024-02-02T13:30:18.162081+0000","last_active":"2024-02-02T13:30:19.372473+0000","last_peered":"2024-02-02T13:30:19.372473+0000","last_clean":"2024-02-02T13:30:19.372473+0000","last_became_active":"2024-02-02T13:30:18.161800+0000","last_became_peered":"2024-02-02T13:30:18.161800+0000","last_unstale":"2024-02-02T13:30:19.372473+0000","last_undegraded":"2024-02-02T13:30:19.372473+0000","last_fullsized":"2024-02-02T13:30:19.372473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:29:34.425654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158644+0000","last_change":"2024-02-02T13:30:17.688264+0000","last_active":"2024-02-02T13:30:19.158644+0000","last_peered":"2024-02-02T13:30:19.158644+0000","last_clean":"2024-02-02T13:30:19.158644+0000","last_became_active":"2024-02-02T13:30:17.687981+0000","last_became_peered":"2024-02-02T13:30:17.687981+0000","last_unstale":"2024-02-02T13:30:19.158644+0000","last_undegraded":"2024-02-02T13:30:19.158644+0000","last_fullsized":"2024-02-02T13:30:19.158644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:56.217020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368359+0000","last_change":"2024-02-02T13:30:18.162282+0000","last_active":"2024-02-02T13:30:19.368359+0000","last_peered":"2024-02-02T13:30:19.368359+0000","last_clean":"2024-02-02T13:30:19.368359+0000","last_became_active":"2024-02-02T13:30:18.162048+0000","last_became_peered":"2024-02-02T13:30:18.162048+0000","last_unstale":"2024-02-02T13:30:19.368359+0000","last_undegraded":"2024-02-02T13:30:19.368359+0000","last_fullsized":"2024-02-02T13:30:19.368359+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:57.356934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373129+0000","last_change":"2024-02-02T13:30:17.689984+0000","last_active":"2024-02-02T13:30:19.373129+0000","last_peered":"2024-02-02T13:30:19.373129+0000","last_clean":"2024-02-02T13:30:19.373129+0000","last_became_active":"2024-02-02T13:30:17.688925+0000","last_became_peered":"2024-02-02T13:30:17.688925+0000","last_unstale":"2024-02-02T13:30:19.373129+0000","last_undegraded":"2024-02-02T13:30:19.373129+0000","last_fullsized":"2024-02-02T13:30:19.373129+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:31.518816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372912+0000","last_change":"2024-02-02T13:30:18.169206+0000","last_active":"2024-02-02T13:30:19.372912+0000","last_peered":"2024-02-02T13:30:19.372912+0000","last_clean":"2024-02-02T13:30:19.372912+0000","last_became_active":"2024-02-02T13:30:18.168605+0000","last_became_peered":"2024-02-02T13:30:18.168605+0000","last_unstale":"2024-02-02T13:30:19.372912+0000","last_undegraded":"2024-02-02T13:30:19.372912+0000","last_fullsized":"2024-02-02T13:30:19.372912+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:08:38.966770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370396+0000","last_change":"2024-02-02T13:30:17.693364+0000","last_active":"2024-02-02T13:30:19.370396+0000","last_peered":"2024-02-02T13:30:19.370396+0000","last_clean":"2024-02-02T13:30:19.370396+0000","last_became_active":"2024-02-02T13:30:17.693145+0000","last_became_peered":"2024-02-02T13:30:17.693145+0000","last_unstale":"2024-02-02T13:30:19.370396+0000","last_undegraded":"2024-02-02T13:30:19.370396+0000","last_fullsized":"2024-02-02T13:30:19.370396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:34.744740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371990+0000","last_change":"2024-02-02T13:30:18.178201+0000","last_active":"2024-02-02T13:30:19.371990+0000","last_peered":"2024-02-02T13:30:19.371990+0000","last_clean":"2024-02-02T13:30:19.371990+0000","last_became_active":"2024-02-02T13:30:18.177946+0000","last_became_peered":"2024-02-02T13:30:18.177946+0000","last_unstale":"2024-02-02T13:30:19.371990+0000","last_undegraded":"2024-02-02T13:30:19.371990+0000","last_fullsized":"2024-02-02T13:30:19.371990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:55:33.625215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158302+0000","last_change":"2024-02-02T13:30:17.695679+0000","last_active":"2024-02-02T13:30:19.158302+0000","last_peered":"2024-02-02T13:30:19.158302+0000","last_clean":"2024-02-02T13:30:19.158302+0000","last_became_active":"2024-02-02T13:30:17.695344+0000","last_became_peered":"2024-02-02T13:30:17.695344+0000","last_unstale":"2024-02-02T13:30:19.158302+0000","last_undegraded":"2024-02-02T13:30:19.158302+0000","last_fullsized":"2024-02-02T13:30:19.158302+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:45:54.238347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155379+0000","last_change":"2024-02-02T13:30:18.159274+0000","last_active":"2024-02-02T13:30:19.155379+0000","last_peered":"2024-02-02T13:30:19.155379+0000","last_clean":"2024-02-02T13:30:19.155379+0000","last_became_active":"2024-02-02T13:30:18.159044+0000","last_became_peered":"2024-02-02T13:30:18.159044+0000","last_unstale":"2024-02-02T13:30:19.155379+0000","last_undegraded":"2024-02-02T13:30:19.155379+0000","last_fullsized":"2024-02-02T13:30:19.155379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:02:21.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369188+0000","last_change":"2024-02-02T13:30:17.690202+0000","last_active":"2024-02-02T13:30:19.369188+0000","last_peered":"2024-02-02T13:30:19.369188+0000","last_clean":"2024-02-02T13:30:19.369188+0000","last_became_active":"2024-02-02T13:30:17.689539+0000","last_became_peered":"2024-02-02T13:30:17.689539+0000","last_unstale":"2024-02-02T13:30:19.369188+0000","last_undegraded":"2024-02-02T13:30:19.369188+0000","last_fullsized":"2024-02-02T13:30:19.369188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:40.908485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155694+0000","last_change":"2024-02-02T13:30:18.153639+0000","last_active":"2024-02-02T13:30:19.155694+0000","last_peered":"2024-02-02T13:30:19.155694+0000","last_clean":"2024-02-02T13:30:19.155694+0000","last_became_active":"2024-02-02T13:30:18.153386+0000","last_became_peered":"2024-02-02T13:30:18.153386+0000","last_unstale":"2024-02-02T13:30:19.155694+0000","last_undegraded":"2024-02-02T13:30:19.155694+0000","last_fullsized":"2024-02-02T13:30:19.155694+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:29:34.425654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373038+0000","last_change":"2024-02-02T13:30:17.690131+0000","last_active":"2024-02-02T13:30:19.373038+0000","last_peered":"2024-02-02T13:30:19.373038+0000","last_clean":"2024-02-02T13:30:19.373038+0000","last_became_active":"2024-02-02T13:30:17.689752+0000","last_became_peered":"2024-02-02T13:30:17.689752+0000","last_unstale":"2024-02-02T13:30:19.373038+0000","last_undegraded":"2024-02-02T13:30:19.373038+0000","last_fullsized":"2024-02-02T13:30:19.373038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:22:05.189040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372139+0000","last_change":"2024-02-02T13:30:18.160638+0000","last_active":"2024-02-02T13:30:19.372139+0000","last_peered":"2024-02-02T13:30:19.372139+0000","last_clean":"2024-02-02T13:30:19.372139+0000","last_became_active":"2024-02-02T13:30:18.160319+0000","last_became_peered":"2024-02-02T13:30:18.160319+0000","last_unstale":"2024-02-02T13:30:19.372139+0000","last_undegraded":"2024-02-02T13:30:19.372139+0000","last_fullsized":"2024-02-02T13:30:19.372139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:08:38.966770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373495+0000","last_change":"2024-02-02T13:30:17.694587+0000","last_active":"2024-02-02T13:30:19.373495+0000","last_peered":"2024-02-02T13:30:19.373495+0000","last_clean":"2024-02-02T13:30:19.373495+0000","last_became_active":"2024-02-02T13:30:17.694295+0000","last_became_peered":"2024-02-02T13:30:17.694295+0000","last_unstale":"2024-02-02T13:30:19.373495+0000","last_undegraded":"2024-02-02T13:30:19.373495+0000","last_fullsized":"2024-02-02T13:30:19.373495+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:05:39.177449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157851+0000","last_change":"2024-02-02T13:30:18.178014+0000","last_active":"2024-02-02T13:30:19.157851+0000","last_peered":"2024-02-02T13:30:19.157851+0000","last_clean":"2024-02-02T13:30:19.157851+0000","last_became_active":"2024-02-02T13:30:18.177228+0000","last_became_peered":"2024-02-02T13:30:18.177228+0000","last_unstale":"2024-02-02T13:30:19.157851+0000","last_undegraded":"2024-02-02T13:30:19.157851+0000","last_fullsized":"2024-02-02T13:30:19.157851+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"104'3","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374770+0000","last_change":"2024-02-02T13:30:17.697490+0000","last_active":"2024-02-02T13:30:19.374770+0000","last_peered":"2024-02-02T13:30:19.374770+0000","last_clean":"2024-02-02T13:30:19.374770+0000","last_became_active":"2024-02-02T13:30:17.697038+0000","last_became_peered":"2024-02-02T13:30:17.697038+0000","last_unstale":"2024-02-02T13:30:19.374770+0000","last_undegraded":"2024-02-02T13:30:19.374770+0000","last_fullsized":"2024-02-02T13:30:19.374770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:06.872985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369910+0000","last_change":"2024-02-02T13:30:18.178121+0000","last_active":"2024-02-02T13:30:19.369910+0000","last_peered":"2024-02-02T13:30:19.369910+0000","last_clean":"2024-02-02T13:30:19.369910+0000","last_became_active":"2024-02-02T13:30:18.177507+0000","last_became_peered":"2024-02-02T13:30:18.177507+0000","last_unstale":"2024-02-02T13:30:19.369910+0000","last_undegraded":"2024-02-02T13:30:19.369910+0000","last_fullsized":"2024-02-02T13:30:19.369910+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:37.867876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"104'6","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371388+0000","last_change":"2024-02-02T13:30:17.681225+0000","last_active":"2024-02-02T13:30:19.371388+0000","last_peered":"2024-02-02T13:30:19.371388+0000","last_clean":"2024-02-02T13:30:19.371388+0000","last_became_active":"2024-02-02T13:30:17.680580+0000","last_became_peered":"2024-02-02T13:30:17.680580+0000","last_unstale":"2024-02-02T13:30:19.371388+0000","last_undegraded":"2024-02-02T13:30:19.371388+0000","last_fullsized":"2024-02-02T13:30:19.371388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:12:04.150942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158412+0000","last_change":"2024-02-02T13:30:18.177996+0000","last_active":"2024-02-02T13:30:19.158412+0000","last_peered":"2024-02-02T13:30:19.158412+0000","last_clean":"2024-02-02T13:30:19.158412+0000","last_became_active":"2024-02-02T13:30:18.177005+0000","last_became_peered":"2024-02-02T13:30:18.177005+0000","last_unstale":"2024-02-02T13:30:19.158412+0000","last_undegraded":"2024-02-02T13:30:19.158412+0000","last_fullsized":"2024-02-02T13:30:19.158412+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:33:54.971010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369394+0000","last_change":"2024-02-02T13:30:17.690366+0000","last_active":"2024-02-02T13:30:19.369394+0000","last_peered":"2024-02-02T13:30:19.369394+0000","last_clean":"2024-02-02T13:30:19.369394+0000","last_became_active":"2024-02-02T13:30:17.689898+0000","last_became_peered":"2024-02-02T13:30:17.689898+0000","last_unstale":"2024-02-02T13:30:19.369394+0000","last_undegraded":"2024-02-02T13:30:19.369394+0000","last_fullsized":"2024-02-02T13:30:19.369394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:34.622471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372212+0000","last_change":"2024-02-02T13:30:18.176917+0000","last_active":"2024-02-02T13:30:19.372212+0000","last_peered":"2024-02-02T13:30:19.372212+0000","last_clean":"2024-02-02T13:30:19.372212+0000","last_became_active":"2024-02-02T13:30:18.176687+0000","last_became_peered":"2024-02-02T13:30:18.176687+0000","last_unstale":"2024-02-02T13:30:19.372212+0000","last_undegraded":"2024-02-02T13:30:19.372212+0000","last_fullsized":"2024-02-02T13:30:19.372212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:20:34.227608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155376+0000","last_change":"2024-02-02T13:30:17.677888+0000","last_active":"2024-02-02T13:30:19.155376+0000","last_peered":"2024-02-02T13:30:19.155376+0000","last_clean":"2024-02-02T13:30:19.155376+0000","last_became_active":"2024-02-02T13:30:17.677654+0000","last_became_peered":"2024-02-02T13:30:17.677654+0000","last_unstale":"2024-02-02T13:30:19.155376+0000","last_undegraded":"2024-02-02T13:30:19.155376+0000","last_fullsized":"2024-02-02T13:30:19.155376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:00.207717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368256+0000","last_change":"2024-02-02T13:30:18.161843+0000","last_active":"2024-02-02T13:30:19.368256+0000","last_peered":"2024-02-02T13:30:19.368256+0000","last_clean":"2024-02-02T13:30:19.368256+0000","last_became_active":"2024-02-02T13:30:18.161632+0000","last_became_peered":"2024-02-02T13:30:18.161632+0000","last_unstale":"2024-02-02T13:30:19.368256+0000","last_undegraded":"2024-02-02T13:30:19.368256+0000","last_fullsized":"2024-02-02T13:30:19.368256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:42.765712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371595+0000","last_change":"2024-02-02T13:30:17.695003+0000","last_active":"2024-02-02T13:30:19.371595+0000","last_peered":"2024-02-02T13:30:19.371595+0000","last_clean":"2024-02-02T13:30:19.371595+0000","last_became_active":"2024-02-02T13:30:17.694578+0000","last_became_peered":"2024-02-02T13:30:17.694578+0000","last_unstale":"2024-02-02T13:30:19.371595+0000","last_undegraded":"2024-02-02T13:30:19.371595+0000","last_fullsized":"2024-02-02T13:30:19.371595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:04:55.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369156+0000","last_change":"2024-02-02T13:30:18.172367+0000","last_active":"2024-02-02T13:30:19.369156+0000","last_peered":"2024-02-02T13:30:19.369156+0000","last_clean":"2024-02-02T13:30:19.369156+0000","last_became_active":"2024-02-02T13:30:18.171745+0000","last_became_peered":"2024-02-02T13:30:18.171745+0000","last_unstale":"2024-02-02T13:30:19.369156+0000","last_undegraded":"2024-02-02T13:30:19.369156+0000","last_fullsized":"2024-02-02T13:30:19.369156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:07:01.500349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"104'8","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158371+0000","last_change":"2024-02-02T13:30:17.692366+0000","last_active":"2024-02-02T13:30:19.158371+0000","last_peered":"2024-02-02T13:30:19.158371+0000","last_clean":"2024-02-02T13:30:19.158371+0000","last_became_active":"2024-02-02T13:30:17.690527+0000","last_became_peered":"2024-02-02T13:30:17.690527+0000","last_unstale":"2024-02-02T13:30:19.158371+0000","last_undegraded":"2024-02-02T13:30:19.158371+0000","last_fullsized":"2024-02-02T13:30:19.158371+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:05.265407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372700+0000","last_change":"2024-02-02T13:30:18.165263+0000","last_active":"2024-02-02T13:30:19.372700+0000","last_peered":"2024-02-02T13:30:19.372700+0000","last_clean":"2024-02-02T13:30:19.372700+0000","last_became_active":"2024-02-02T13:30:18.165019+0000","last_became_peered":"2024-02-02T13:30:18.165019+0000","last_unstale":"2024-02-02T13:30:19.372700+0000","last_undegraded":"2024-02-02T13:30:19.372700+0000","last_fullsized":"2024-02-02T13:30:19.372700+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:29:44.188635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370537+0000","last_change":"2024-02-02T13:30:17.696174+0000","last_active":"2024-02-02T13:30:19.370537+0000","last_peered":"2024-02-02T13:30:19.370537+0000","last_clean":"2024-02-02T13:30:19.370537+0000","last_became_active":"2024-02-02T13:30:17.695873+0000","last_became_peered":"2024-02-02T13:30:17.695873+0000","last_unstale":"2024-02-02T13:30:19.370537+0000","last_undegraded":"2024-02-02T13:30:19.370537+0000","last_fullsized":"2024-02-02T13:30:19.370537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:04:55.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370161+0000","last_change":"2024-02-02T13:30:18.167798+0000","last_active":"2024-02-02T13:30:19.370161+0000","last_peered":"2024-02-02T13:30:19.370161+0000","last_clean":"2024-02-02T13:30:19.370161+0000","last_became_active":"2024-02-02T13:30:18.167582+0000","last_became_peered":"2024-02-02T13:30:18.167582+0000","last_unstale":"2024-02-02T13:30:19.370161+0000","last_undegraded":"2024-02-02T13:30:19.370161+0000","last_fullsized":"2024-02-02T13:30:19.370161+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:42:31.168036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158770+0000","last_change":"2024-02-02T13:30:17.690906+0000","last_active":"2024-02-02T13:30:19.158770+0000","last_peered":"2024-02-02T13:30:19.158770+0000","last_clean":"2024-02-02T13:30:19.158770+0000","last_became_active":"2024-02-02T13:30:17.690556+0000","last_became_peered":"2024-02-02T13:30:17.690556+0000","last_unstale":"2024-02-02T13:30:19.158770+0000","last_undegraded":"2024-02-02T13:30:19.158770+0000","last_fullsized":"2024-02-02T13:30:19.158770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:20:54.568579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371533+0000","last_change":"2024-02-02T13:30:18.167368+0000","last_active":"2024-02-02T13:30:19.371533+0000","last_peered":"2024-02-02T13:30:19.371533+0000","last_clean":"2024-02-02T13:30:19.371533+0000","last_became_active":"2024-02-02T13:30:18.167028+0000","last_became_peered":"2024-02-02T13:30:18.167028+0000","last_unstale":"2024-02-02T13:30:19.371533+0000","last_undegraded":"2024-02-02T13:30:19.371533+0000","last_fullsized":"2024-02-02T13:30:19.371533+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:45:55.252415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158085+0000","last_change":"2024-02-02T13:30:17.691999+0000","last_active":"2024-02-02T13:30:19.158085+0000","last_peered":"2024-02-02T13:30:19.158085+0000","last_clean":"2024-02-02T13:30:19.158085+0000","last_became_active":"2024-02-02T13:30:17.691758+0000","last_became_peered":"2024-02-02T13:30:17.691758+0000","last_unstale":"2024-02-02T13:30:19.158085+0000","last_undegraded":"2024-02-02T13:30:19.158085+0000","last_fullsized":"2024-02-02T13:30:19.158085+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:03.089050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156989+0000","last_change":"2024-02-02T13:30:18.164157+0000","last_active":"2024-02-02T13:30:19.156989+0000","last_peered":"2024-02-02T13:30:19.156989+0000","last_clean":"2024-02-02T13:30:19.156989+0000","last_became_active":"2024-02-02T13:30:18.162831+0000","last_became_peered":"2024-02-02T13:30:18.162831+0000","last_unstale":"2024-02-02T13:30:19.156989+0000","last_undegraded":"2024-02-02T13:30:19.156989+0000","last_fullsized":"2024-02-02T13:30:19.156989+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:45.411357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155469+0000","last_change":"2024-02-02T13:30:17.684246+0000","last_active":"2024-02-02T13:30:19.155469+0000","last_peered":"2024-02-02T13:30:19.155469+0000","last_clean":"2024-02-02T13:30:19.155469+0000","last_became_active":"2024-02-02T13:30:17.683993+0000","last_became_peered":"2024-02-02T13:30:17.683993+0000","last_unstale":"2024-02-02T13:30:19.155469+0000","last_undegraded":"2024-02-02T13:30:19.155469+0000","last_fullsized":"2024-02-02T13:30:19.155469+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:08.839812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373468+0000","last_change":"2024-02-02T13:30:18.154790+0000","last_active":"2024-02-02T13:30:19.373468+0000","last_peered":"2024-02-02T13:30:19.373468+0000","last_clean":"2024-02-02T13:30:19.373468+0000","last_became_active":"2024-02-02T13:30:18.154572+0000","last_became_peered":"2024-02-02T13:30:18.154572+0000","last_unstale":"2024-02-02T13:30:19.373468+0000","last_undegraded":"2024-02-02T13:30:19.373468+0000","last_fullsized":"2024-02-02T13:30:19.373468+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:33.083769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157100+0000","last_change":"2024-02-02T13:30:17.686586+0000","last_active":"2024-02-02T13:30:19.157100+0000","last_peered":"2024-02-02T13:30:19.157100+0000","last_clean":"2024-02-02T13:30:19.157100+0000","last_became_active":"2024-02-02T13:30:17.686305+0000","last_became_peered":"2024-02-02T13:30:17.686305+0000","last_unstale":"2024-02-02T13:30:19.157100+0000","last_undegraded":"2024-02-02T13:30:19.157100+0000","last_fullsized":"2024-02-02T13:30:19.157100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:18.234058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157443+0000","last_change":"2024-02-02T13:30:18.162886+0000","last_active":"2024-02-02T13:30:19.157443+0000","last_peered":"2024-02-02T13:30:19.157443+0000","last_clean":"2024-02-02T13:30:19.157443+0000","last_became_active":"2024-02-02T13:30:18.162607+0000","last_became_peered":"2024-02-02T13:30:18.162607+0000","last_unstale":"2024-02-02T13:30:19.157443+0000","last_undegraded":"2024-02-02T13:30:19.157443+0000","last_fullsized":"2024-02-02T13:30:19.157443+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:24:21.961257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157152+0000","last_change":"2024-02-02T13:30:17.693584+0000","last_active":"2024-02-02T13:30:19.157152+0000","last_peered":"2024-02-02T13:30:19.157152+0000","last_clean":"2024-02-02T13:30:19.157152+0000","last_became_active":"2024-02-02T13:30:17.693339+0000","last_became_peered":"2024-02-02T13:30:17.693339+0000","last_unstale":"2024-02-02T13:30:19.157152+0000","last_undegraded":"2024-02-02T13:30:19.157152+0000","last_fullsized":"2024-02-02T13:30:19.157152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:28:28.580941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371815+0000","last_change":"2024-02-02T13:30:18.176584+0000","last_active":"2024-02-02T13:30:19.371815+0000","last_peered":"2024-02-02T13:30:19.371815+0000","last_clean":"2024-02-02T13:30:19.371815+0000","last_became_active":"2024-02-02T13:30:18.176239+0000","last_became_peered":"2024-02-02T13:30:18.176239+0000","last_unstale":"2024-02-02T13:30:19.371815+0000","last_undegraded":"2024-02-02T13:30:19.371815+0000","last_fullsized":"2024-02-02T13:30:19.371815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155483+0000","last_change":"2024-02-02T13:30:17.684864+0000","last_active":"2024-02-02T13:30:19.155483+0000","last_peered":"2024-02-02T13:30:19.155483+0000","last_clean":"2024-02-02T13:30:19.155483+0000","last_became_active":"2024-02-02T13:30:17.684239+0000","last_became_peered":"2024-02-02T13:30:17.684239+0000","last_unstale":"2024-02-02T13:30:19.155483+0000","last_undegraded":"2024-02-02T13:30:19.155483+0000","last_fullsized":"2024-02-02T13:30:19.155483+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:54:20.633062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370160+0000","last_change":"2024-02-02T13:30:18.166635+0000","last_active":"2024-02-02T13:30:19.370160+0000","last_peered":"2024-02-02T13:30:19.370160+0000","last_clean":"2024-02-02T13:30:19.370160+0000","last_became_active":"2024-02-02T13:30:18.165704+0000","last_became_peered":"2024-02-02T13:30:18.165704+0000","last_unstale":"2024-02-02T13:30:19.370160+0000","last_undegraded":"2024-02-02T13:30:19.370160+0000","last_fullsized":"2024-02-02T13:30:19.370160+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:20:55.582647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369624+0000","last_change":"2024-02-02T13:30:17.688632+0000","last_active":"2024-02-02T13:30:19.369624+0000","last_peered":"2024-02-02T13:30:19.369624+0000","last_clean":"2024-02-02T13:30:19.369624+0000","last_became_active":"2024-02-02T13:30:17.688386+0000","last_became_peered":"2024-02-02T13:30:17.688386+0000","last_unstale":"2024-02-02T13:30:19.369624+0000","last_undegraded":"2024-02-02T13:30:19.369624+0000","last_fullsized":"2024-02-02T13:30:19.369624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:33:39.292239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371879+0000","last_change":"2024-02-02T13:30:18.171422+0000","last_active":"2024-02-02T13:30:19.371879+0000","last_peered":"2024-02-02T13:30:19.371879+0000","last_clean":"2024-02-02T13:30:19.371879+0000","last_became_active":"2024-02-02T13:30:18.171187+0000","last_became_peered":"2024-02-02T13:30:18.171187+0000","last_unstale":"2024-02-02T13:30:19.371879+0000","last_undegraded":"2024-02-02T13:30:19.371879+0000","last_fullsized":"2024-02-02T13:30:19.371879+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:17:49.734188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368446+0000","last_change":"2024-02-02T13:30:17.687826+0000","last_active":"2024-02-02T13:30:19.368446+0000","last_peered":"2024-02-02T13:30:19.368446+0000","last_clean":"2024-02-02T13:30:19.368446+0000","last_became_active":"2024-02-02T13:30:17.684775+0000","last_became_peered":"2024-02-02T13:30:17.684775+0000","last_unstale":"2024-02-02T13:30:19.368446+0000","last_undegraded":"2024-02-02T13:30:19.368446+0000","last_fullsized":"2024-02-02T13:30:19.368446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:44.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371943+0000","last_change":"2024-02-02T13:30:18.171492+0000","last_active":"2024-02-02T13:30:19.371943+0000","last_peered":"2024-02-02T13:30:19.371943+0000","last_clean":"2024-02-02T13:30:19.371943+0000","last_became_active":"2024-02-02T13:30:18.170925+0000","last_became_peered":"2024-02-02T13:30:18.170925+0000","last_unstale":"2024-02-02T13:30:19.371943+0000","last_undegraded":"2024-02-02T13:30:19.371943+0000","last_fullsized":"2024-02-02T13:30:19.371943+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:33.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370752+0000","last_change":"2024-02-02T13:30:17.691263+0000","last_active":"2024-02-02T13:30:19.370752+0000","last_peered":"2024-02-02T13:30:19.370752+0000","last_clean":"2024-02-02T13:30:19.370752+0000","last_became_active":"2024-02-02T13:30:17.691017+0000","last_became_peered":"2024-02-02T13:30:17.691017+0000","last_unstale":"2024-02-02T13:30:19.370752+0000","last_undegraded":"2024-02-02T13:30:19.370752+0000","last_fullsized":"2024-02-02T13:30:19.370752+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:56.342866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370290+0000","last_change":"2024-02-02T13:30:18.180187+0000","last_active":"2024-02-02T13:30:19.370290+0000","last_peered":"2024-02-02T13:30:19.370290+0000","last_clean":"2024-02-02T13:30:19.370290+0000","last_became_active":"2024-02-02T13:30:18.179671+0000","last_became_peered":"2024-02-02T13:30:18.179671+0000","last_unstale":"2024-02-02T13:30:19.370290+0000","last_undegraded":"2024-02-02T13:30:19.370290+0000","last_fullsized":"2024-02-02T13:30:19.370290+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:15:27.821324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372139+0000","last_change":"2024-02-02T13:30:17.696248+0000","last_active":"2024-02-02T13:30:19.372139+0000","last_peered":"2024-02-02T13:30:19.372139+0000","last_clean":"2024-02-02T13:30:19.372139+0000","last_became_active":"2024-02-02T13:30:17.695391+0000","last_became_peered":"2024-02-02T13:30:17.695391+0000","last_unstale":"2024-02-02T13:30:19.372139+0000","last_undegraded":"2024-02-02T13:30:19.372139+0000","last_fullsized":"2024-02-02T13:30:19.372139+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:12:04.150942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371347+0000","last_change":"2024-02-02T13:30:18.157250+0000","last_active":"2024-02-02T13:30:19.371347+0000","last_peered":"2024-02-02T13:30:19.371347+0000","last_clean":"2024-02-02T13:30:19.371347+0000","last_became_active":"2024-02-02T13:30:18.156796+0000","last_became_peered":"2024-02-02T13:30:18.156796+0000","last_unstale":"2024-02-02T13:30:19.371347+0000","last_undegraded":"2024-02-02T13:30:19.371347+0000","last_fullsized":"2024-02-02T13:30:19.371347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158750+0000","last_change":"2024-02-02T13:30:17.690926+0000","last_active":"2024-02-02T13:30:19.158750+0000","last_peered":"2024-02-02T13:30:19.158750+0000","last_clean":"2024-02-02T13:30:19.158750+0000","last_became_active":"2024-02-02T13:30:17.690702+0000","last_became_peered":"2024-02-02T13:30:17.690702+0000","last_unstale":"2024-02-02T13:30:19.158750+0000","last_undegraded":"2024-02-02T13:30:19.158750+0000","last_fullsized":"2024-02-02T13:30:19.158750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:48.744172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158537+0000","last_change":"2024-02-02T13:30:18.177220+0000","last_active":"2024-02-02T13:30:19.158537+0000","last_peered":"2024-02-02T13:30:19.158537+0000","last_clean":"2024-02-02T13:30:19.158537+0000","last_became_active":"2024-02-02T13:30:18.176714+0000","last_became_peered":"2024-02-02T13:30:18.176714+0000","last_unstale":"2024-02-02T13:30:19.158537+0000","last_undegraded":"2024-02-02T13:30:19.158537+0000","last_fullsized":"2024-02-02T13:30:19.158537+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:33.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374617+0000","last_change":"2024-02-02T13:30:17.692879+0000","last_active":"2024-02-02T13:30:19.374617+0000","last_peered":"2024-02-02T13:30:19.374617+0000","last_clean":"2024-02-02T13:30:19.374617+0000","last_became_active":"2024-02-02T13:30:17.692639+0000","last_became_peered":"2024-02-02T13:30:17.692639+0000","last_unstale":"2024-02-02T13:30:19.374617+0000","last_undegraded":"2024-02-02T13:30:19.374617+0000","last_fullsized":"2024-02-02T13:30:19.374617+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:14:53.845555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370114+0000","last_change":"2024-02-02T13:30:18.158545+0000","last_active":"2024-02-02T13:30:19.370114+0000","last_peered":"2024-02-02T13:30:19.370114+0000","last_clean":"2024-02-02T13:30:19.370114+0000","last_became_active":"2024-02-02T13:30:18.158044+0000","last_became_peered":"2024-02-02T13:30:18.158044+0000","last_unstale":"2024-02-02T13:30:19.370114+0000","last_undegraded":"2024-02-02T13:30:19.370114+0000","last_fullsized":"2024-02-02T13:30:19.370114+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:56.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374986+0000","last_change":"2024-02-02T13:30:17.697266+0000","last_active":"2024-02-02T13:30:19.374986+0000","last_peered":"2024-02-02T13:30:19.374986+0000","last_clean":"2024-02-02T13:30:19.374986+0000","last_became_active":"2024-02-02T13:30:17.693307+0000","last_became_peered":"2024-02-02T13:30:17.693307+0000","last_unstale":"2024-02-02T13:30:19.374986+0000","last_undegraded":"2024-02-02T13:30:19.374986+0000","last_fullsized":"2024-02-02T13:30:19.374986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:01.302773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374556+0000","last_change":"2024-02-02T13:30:18.175593+0000","last_active":"2024-02-02T13:30:19.374556+0000","last_peered":"2024-02-02T13:30:19.374556+0000","last_clean":"2024-02-02T13:30:19.374556+0000","last_became_active":"2024-02-02T13:30:18.175298+0000","last_became_peered":"2024-02-02T13:30:18.175298+0000","last_unstale":"2024-02-02T13:30:19.374556+0000","last_undegraded":"2024-02-02T13:30:19.374556+0000","last_fullsized":"2024-02-02T13:30:19.374556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370725+0000","last_change":"2024-02-02T13:30:17.694926+0000","last_active":"2024-02-02T13:30:19.370725+0000","last_peered":"2024-02-02T13:30:19.370725+0000","last_clean":"2024-02-02T13:30:19.370725+0000","last_became_active":"2024-02-02T13:30:17.694662+0000","last_became_peered":"2024-02-02T13:30:17.694662+0000","last_unstale":"2024-02-02T13:30:19.370725+0000","last_undegraded":"2024-02-02T13:30:19.370725+0000","last_fullsized":"2024-02-02T13:30:19.370725+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:14:46.689715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369516+0000","last_change":"2024-02-02T13:30:18.154968+0000","last_active":"2024-02-02T13:30:19.369516+0000","last_peered":"2024-02-02T13:30:19.369516+0000","last_clean":"2024-02-02T13:30:19.369516+0000","last_became_active":"2024-02-02T13:30:18.154665+0000","last_became_peered":"2024-02-02T13:30:18.154665+0000","last_unstale":"2024-02-02T13:30:19.369516+0000","last_undegraded":"2024-02-02T13:30:19.369516+0000","last_fullsized":"2024-02-02T13:30:19.369516+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:35.758808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372891+0000","last_change":"2024-02-02T13:30:17.687927+0000","last_active":"2024-02-02T13:30:19.372891+0000","last_peered":"2024-02-02T13:30:19.372891+0000","last_clean":"2024-02-02T13:30:19.372891+0000","last_became_active":"2024-02-02T13:30:17.685140+0000","last_became_peered":"2024-02-02T13:30:17.685140+0000","last_unstale":"2024-02-02T13:30:19.372891+0000","last_undegraded":"2024-02-02T13:30:19.372891+0000","last_fullsized":"2024-02-02T13:30:19.372891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:32.069701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-02-02T13:30:18.150325+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:30:18.150325+0000","last_peered":"2024-02-02T13:30:18.150325+0000","last_clean":"2024-02-02T13:30:18.150325+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:30:18.150325+0000","last_undegraded":"2024-02-02T13:30:18.150325+0000","last_fullsized":"2024-02-02T13:30:18.150325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1132,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194179,"statfs":{"total":200000000,"available":198839990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.097,"15min":1.0649999999999999},"min":{"1min":0.67600000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.492,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1160000000000001,"15min":1.1279999999999999},"min":{"1min":0.76500000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6040000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.3360000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.107,"15min":1.093},"min":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":1.893,"15min":1.893},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.159,"15min":1.117},"min":{"1min":0.87,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.599,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.532}]},{"osd":2,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2430000000000001,"15min":1.2230000000000001},"min":{"1min":1.002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5409999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.177,"15min":1.1870000000000001},"min":{"1min":0.80000000000000004,"5min":0.623,"15min":0.623},"max":{"1min":1.4510000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.4359999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.89200000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.633,"5min":1.819,"15min":1.819},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3340000000000001,"15min":1.3120000000000001},"min":{"1min":0.79100000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8660000000000001,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.268,"15min":1.1970000000000001},"min":{"1min":0.93600000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6559999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.222},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":1.056,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7929999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5780000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.373,"15min":1.351},"min":{"1min":0.80200000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.726,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.613},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.423,"15min":1.454},"min":{"1min":0.76000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7370000000000001,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.4970000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.51,"15min":1.538},"min":{"1min":0.88700000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.855,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.645},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.111,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.821,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.6879999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194756,"statfs":{"total":200000000,"available":199430753,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.077,"15min":1.018},"min":{"1min":0.86799999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.571,"5min":1.577,"15min":1.577},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.0700000000000001,"15min":1.0189999999999999},"min":{"1min":0.75800000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.002}]},{"osd":1,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0529999999999999,"15min":1.02},"min":{"1min":0.78900000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5229999999999999,"5min":1.556,"15min":1.556},"last":1.288},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.0740000000000001},"min":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.456,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.141}]},{"osd":2,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1519999999999999,"15min":1.0720000000000001},"min":{"1min":0.77600000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.498,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.169,"15min":1.153},"min":{"1min":0.873,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.639,"5min":1.845,"15min":1.845},"last":1.2110000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.26},"min":{"1min":0.749,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.482,"5min":1.792,"15min":1.792},"last":1.339},{"interface":"front","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.1359999999999999},"min":{"1min":0.83699999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5009999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.29,"15min":1.214},"min":{"1min":0.94499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6739999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.4379999999999999,"15min":1.3979999999999999},"min":{"1min":1.179,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5900000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.292,"15min":1.2769999999999999},"min":{"1min":0.97999999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.4239999999999999,"15min":1.3560000000000001},"min":{"1min":1.335,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":1.895,"15min":1.895},"last":1.323}]},{"osd":7,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.4850000000000001,"15min":1.4570000000000001},"min":{"1min":1.1020000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.752,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":1.1359999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":557,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194754,"statfs":{"total":200000000,"available":199428842,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0289999999999999,"15min":1.0209999999999999},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.49},{"interface":"front","average":{"1min":1.05,"5min":1.0489999999999999,"15min":1.0409999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.2290000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.202,"15min":1.2769999999999999},"min":{"1min":0.82599999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.607,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1140000000000001,"15min":1.153},"min":{"1min":0.746,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5209999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.014}]},{"osd":2,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.2689999999999999,"15min":1.2250000000000001},"min":{"1min":0.92900000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8899999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.204,"15min":1.1799999999999999},"min":{"1min":0.94499999999999995,"5min":0.752,"15min":0.752},"max":{"1min":1.706,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.171}]},{"osd":3,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1919999999999999,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.591,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.333},{"interface":"front","average":{"1min":1.268,"5min":1.2370000000000001,"15min":1.171},"min":{"1min":0.752,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.264}]},{"osd":4,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.194,"15min":1.222},"min":{"1min":0.86699999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2769999999999999,"15min":1.28},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8029999999999999,"5min":1.833,"15min":1.833},"last":1.421}]},{"osd":6,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.325,"15min":1.3},"min":{"1min":0.91600000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.456},{"interface":"front","average":{"1min":1.335,"5min":1.351,"15min":1.345},"min":{"1min":1.036,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7889999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.407}]},{"osd":7,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.4199999999999999,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4199999999999999,"15min":1.3720000000000001},"min":{"1min":1.161,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.3720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194757,"statfs":{"total":200000000,"available":199431447,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0960000000000001,"15min":0.99299999999999999},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.546,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.022},{"interface":"front","average":{"1min":1.157,"5min":1.107,"15min":1.0820000000000001},"min":{"1min":0.81100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.514,"5min":1.839,"15min":1.839},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":0.98699999999999999},"min":{"1min":0.71899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.292,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.097,"15min":1.0329999999999999},"min":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.177}]},{"osd":2,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1599999999999999,"15min":1.0900000000000001},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4930000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":1.071},{"interface":"front","average":{"1min":1.214,"5min":1.151,"15min":1.069},"min":{"1min":0.68000000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6459999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2529999999999999,"15min":1.175},"min":{"1min":0.83499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6000000000000001,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.3029999999999999,"15min":1.2370000000000001},"min":{"1min":0.91100000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.9670000000000001,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.3580000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.218,"15min":1.1379999999999999},"min":{"1min":0.63700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.958,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.3300000000000001,"15min":1.272},"min":{"1min":0.77900000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.004,"5min":2.069,"15min":2.069},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.3540000000000001,"15min":1.288},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2429999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.478},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.446,"15min":1.3879999999999999},"min":{"1min":0.75900000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1240000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4410000000000001,"15min":1.3779999999999999},"min":{"1min":0.878,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.889,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.629},{"interface":"front","average":{"1min":1.609,"5min":1.4750000000000001,"15min":1.444},"min":{"1min":0.74099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.181,"5min":2.698,"15min":2.698},"last":1.5469999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":557,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194754,"statfs":{"total":200000000,"available":199428864,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.70399999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.90000000000000002,"15min":0.88600000000000001},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4319999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4810000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0429999999999999,"15min":1.016},"min":{"1min":0.56599999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4419999999999999,"5min":2.125,"15min":2.125},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.028,"15min":0.95999999999999996},"min":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.484,"5min":1.611,"15min":1.611},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.008,"5min":0.92900000000000005,"15min":0.89600000000000002},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.4510000000000001,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.198,"15min":1.1830000000000001},"min":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.538,"5min":3.585,"15min":3.585},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.2070000000000001},"min":{"1min":0.68100000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.657,"5min":1.877,"15min":1.877},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.226},"min":{"1min":0.73699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":0.73399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.851,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.83799999999999997}]},{"osd":6,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.339,"15min":1.2649999999999999},"min":{"1min":0.72599999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.829,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3049999999999999,"15min":1.3069999999999999},"min":{"1min":0.77400000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.073}]},{"osd":7,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.282,"15min":1.2629999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.869,"5min":3.391,"15min":3.391},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.3160000000000001},"min":{"1min":0.753,"5min":0.625,"15min":0.625},"max":{"1min":1.9199999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.88800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":556,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194755,"statfs":{"total":200000000,"available":199429846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.6499999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.86199999999999999,"15min":0.78300000000000003},"min":{"1min":0.62,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.47,"5min":1.867,"15min":1.867},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.93899999999999995,"15min":0.90300000000000002},"min":{"1min":0.51700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.597,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.103,"5min":0.96999999999999997,"15min":0.86399999999999999},"min":{"1min":0.57599999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.6240000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.91500000000000004,"15min":0.79800000000000004},"min":{"1min":0.745,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.0529999999999999,"15min":0.98799999999999999},"min":{"1min":0.60099999999999998,"5min":0.432,"15min":0.432},"max":{"1min":2.0049999999999999,"5min":2.423,"15min":2.423},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.258,"15min":1.1830000000000001},"min":{"1min":0.87,"5min":0.748,"15min":0.748},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.353,"5min":1.2310000000000001,"15min":1.1599999999999999},"min":{"1min":1.048,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.202,"15min":1.0880000000000001},"min":{"1min":0.88500000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.806,"5min":2.246,"15min":2.246},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.92300000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.016}]},{"osd":6,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.2649999999999999,"15min":1.1799999999999999},"min":{"1min":0.81899999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.2689999999999999,"15min":1.202},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2330000000000001,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.3169999999999999,"15min":1.2090000000000001},"min":{"1min":0.80000000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.3149999999999999,"15min":1.2150000000000001},"min":{"1min":0.995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1589999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194757,"statfs":{"total":200000000,"available":199431253,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":1.0109999999999999},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.425,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.044,"15min":1.3759999999999999},"min":{"1min":0.51400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.296,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":1.2310000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88800000000000001,"15min":0.91600000000000004},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.639,"15min":1.639},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.006,"15min":1.0920000000000001},"min":{"1min":0.56499999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.823,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.101,"15min":1.2589999999999999},"min":{"1min":0.63200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.6200000000000001,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0880000000000001,"15min":1.288},"min":{"1min":0.501,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5109999999999999,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.117}]},{"osd":4,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1910000000000001,"15min":1.329},"min":{"1min":0.73699999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.837,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.351,"5min":1.284,"15min":1.3700000000000001},"min":{"1min":1.079,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.772,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.0249999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3360000000000001,"15min":1.462},"min":{"1min":0.94699999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.9970000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.327,"15min":1.4450000000000001},"min":{"1min":0.98299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7090000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.0629999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3660000000000001,"15min":1.579},"min":{"1min":0.97399999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.1539999999999999,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.46},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3720000000000001,"15min":1.502},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8280000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.409,"15min":1.5609999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.871,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.181},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4450000000000001,"15min":1.627},"min":{"1min":0.878,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1019999999999999,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194180,"statfs":{"total":200000000,"available":198841045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.88300000000000001,"15min":1.1359999999999999},"min":{"1min":0.42999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.8680000000000001,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":1.077},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0509999999999999,"15min":1.4950000000000001},"min":{"1min":0.45100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6870000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.995,"15min":1.173},"min":{"1min":0.53300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.0369999999999999,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.49},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.706,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.99099999999999999,"15min":1.24},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3819999999999999,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.117,"15min":1.321},"min":{"1min":0.51300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2170000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2170000000000001,"15min":1.347},"min":{"1min":0.753,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6759999999999999,"5min":3.669,"15min":3.669},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2989999999999999,"15min":1.4119999999999999},"min":{"1min":0.90000000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2719999999999998,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.2669999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.4159999999999999,"15min":1.8360000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9339999999999999,"5min":10.651,"15min":10.651},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.288,"15min":1.431},"min":{"1min":0.78600000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0009999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.224}]},{"osd":6,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3480000000000001,"15min":1.478},"min":{"1min":0.88300000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.319,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.4319999999999999,"15min":1.8680000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.774,"5min":10.708,"15min":10.708},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4850000000000001,"15min":1.881},"min":{"1min":0.96899999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8899999999999999,"5min":10.74,"15min":10.74},"last":1.466},{"interface":"front","average":{"1min":1.276,"5min":1.3300000000000001,"15min":1.452},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.764,"5min":3.863,"15min":3.863},"last":1.395}]}]}],"pool_statfs":[]}} 2024-02-02T13:30:23.066 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:30:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.308+0000 7f7d23c00640 1 Processor -- start 2024-02-02T13:30:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.309+0000 7f7d23c00640 1 -- start start 2024-02-02T13:30:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c15d650 0x7f7d1c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d1c1610b0 0x7f7d1c161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c0cea30 con 0x7f7d1c0ce070 2024-02-02T13:30:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c0cded0 con 0x7f7d1c1610b0 2024-02-02T13:30:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d23c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0d12c0 con 0x7f7d1c15d650 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d1be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d1be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33404/0 (socket says 172.21.15.91:33404) 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d1be00640 1 -- 172.21.15.91:0/1290490305 learned_addr learned my addr 172.21.15.91:0/1290490305 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d21200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c15d650 0x7f7d1c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.310+0000 7f7d21c00640 1 --2- 172.21.15.91:0/1290490305 >> v2:172.21.15.113:3300/0 conn(0x7f7d1c1610b0 0x7f7d1c161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.311+0000 7f7d1be00640 1 -- 172.21.15.91:0/1290490305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c15d650 msgr2=0x7f7d1c0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.311+0000 7f7d1be00640 1 --2- 172.21.15.91:0/1290490305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c15d650 0x7f7d1c0cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.311+0000 7f7d1be00640 1 -- 172.21.15.91:0/1290490305 >> v2:172.21.15.113:3300/0 conn(0x7f7d1c1610b0 msgr2=0x7f7d1c161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.311+0000 7f7d1be00640 1 --2- 172.21.15.91:0/1290490305 >> v2:172.21.15.113:3300/0 conn(0x7f7d1c1610b0 0x7f7d1c161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.311+0000 7f7d1be00640 1 -- 172.21.15.91:0/1290490305 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c071410 con 0x7f7d1c0ce070 2024-02-02T13:30:23.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.312+0000 7f7d1be00640 1 --2- 172.21.15.91:0/1290490305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0b8d40 tx=0x7f7d0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cfc6e516d973da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.312+0000 7f7d1b400640 1 -- 172.21.15.91:0/1290490305 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0402ca70 con 0x7f7d1c0ce070 2024-02-02T13:30:23.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.312+0000 7f7d1b400640 1 -- 172.21.15.91:0/1290490305 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0402cbd0 con 0x7f7d1c0ce070 2024-02-02T13:30:23.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.312+0000 7f7d1b400640 1 -- 172.21.15.91:0/1290490305 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040343f0 con 0x7f7d1c0ce070 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 -- 172.21.15.91:0/1290490305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c0ce070 msgr2=0x7f7d1c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 --2- 172.21.15.91:0/1290490305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0b8d40 tx=0x7f7d0402bed0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 -- 172.21.15.91:0/1290490305 shutdown_connections 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 --2- 172.21.15.91:0/1290490305 >> v2:172.21.15.113:3300/0 conn(0x7f7d1c1610b0 0x7f7d1c161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 --2- 172.21.15.91:0/1290490305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c0ce070 0x7f7d1c0ce4f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 --2- 172.21.15.91:0/1290490305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c15d650 0x7f7d1c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 -- 172.21.15.91:0/1290490305 >> 172.21.15.91:0/1290490305 conn(0x7f7d1c0d30a0 msgr2=0x7f7d1c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 -- 172.21.15.91:0/1290490305 shutdown_connections 2024-02-02T13:30:23.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.313+0000 7f7d23c00640 1 -- 172.21.15.91:0/1290490305 wait complete. 2024-02-02T13:30:23.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.315+0000 7f7d23c00640 1 Processor -- start 2024-02-02T13:30:23.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.315+0000 7f7d23c00640 1 -- start start 2024-02-02T13:30:23.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c0ce070 0x7f7d1c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 0x7f7d1c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1f6350 con 0x7f7d1c15d650 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1f6630 con 0x7f7d1c1610b0 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d23c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1f6910 con 0x7f7d1c0ce070 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33412/0 (socket says 172.21.15.91:33412) 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d1be00640 1 -- 172.21.15.91:0/4125455873 learned_addr learned my addr 172.21.15.91:0/4125455873 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d21200640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c0ce070 0x7f7d1c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.316+0000 7f7d21c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 0x7f7d1c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 -- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c0ce070 msgr2=0x7f7d1c1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c0ce070 0x7f7d1c1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 -- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 msgr2=0x7f7d1c0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 0x7f7d1c0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 -- 172.21.15.91:0/4125455873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04009b30 con 0x7f7d1c15d650 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d21c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 0x7f7d1c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:23.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d1be00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f80 tx=0x7f7d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04009510 con 0x7f7d1c15d650 2024-02-02T13:30:23.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0403e450 con 0x7f7d1c15d650 2024-02-02T13:30:23.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0403e800 con 0x7f7d1c15d650 2024-02-02T13:30:23.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c1f6bf0 con 0x7f7d1c15d650 2024-02-02T13:30:23.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.317+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c15da30 con 0x7f7d1c15d650 2024-02-02T13:30:23.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.319+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c071a30 con 0x7f7d1c15d650 2024-02-02T13:30:23.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.319+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7d0403e960 con 0x7f7d1c15d650 2024-02-02T13:30:23.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.320+0000 7f7d18c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 0x7f7cf406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.320+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d040ab760 con 0x7f7d1c15d650 2024-02-02T13:30:23.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.321+0000 7f7d21200640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 0x7f7cf406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.321+0000 7f7d21200640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 0x7f7cf406c8f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006fd0 tx=0x7f7d0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.325+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7d04076d40 con 0x7f7d1c15d650 2024-02-02T13:30:23.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.510+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d1c1f67a0 con 0x7f7cf406a430 2024-02-02T13:30:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.519+0000 7f7d18c00640 1 -- 172.21.15.91:0/4125455873 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282908 (secure 0 0 0) 0x7f7d1c1f67a0 con 0x7f7cf406a430 2024-02-02T13:30:23.525 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 msgr2=0x7f7cf406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 0x7f7cf406c8f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006fd0 tx=0x7f7d0c008040 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 msgr2=0x7f7d1c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f80 tx=0x7f7d04004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 shutdown_connections 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7cf406a430 0x7f7cf406c8f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d1c1610b0 0x7f7d1c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d1c15d650 0x7f7d1c0c8960 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 --2- 172.21.15.91:0/4125455873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d1c0ce070 0x7f7d1c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 >> 172.21.15.91:0/4125455873 conn(0x7f7d1c0d30a0 msgr2=0x7f7d1c15b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:23.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 shutdown_connections 2024-02-02T13:30:23.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.526+0000 7f7d23c00640 1 -- 172.21.15.91:0/4125455873 wait complete. 2024-02-02T13:30:23.545 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-02-02T13:30:22.350175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5603,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556896,"statfs":{"total":1600000000,"available":1594262040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.699576"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372561+0000","last_change":"2024-02-02T13:30:18.168830+0000","last_active":"2024-02-02T13:30:19.372561+0000","last_peered":"2024-02-02T13:30:19.372561+0000","last_clean":"2024-02-02T13:30:19.372561+0000","last_became_active":"2024-02-02T13:30:18.168587+0000","last_became_peered":"2024-02-02T13:30:18.168587+0000","last_unstale":"2024-02-02T13:30:19.372561+0000","last_undegraded":"2024-02-02T13:30:19.372561+0000","last_fullsized":"2024-02-02T13:30:19.372561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:09.853880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368686+0000","last_change":"2024-02-02T13:30:18.172516+0000","last_active":"2024-02-02T13:30:19.368686+0000","last_peered":"2024-02-02T13:30:19.368686+0000","last_clean":"2024-02-02T13:30:19.368686+0000","last_became_active":"2024-02-02T13:30:18.172119+0000","last_became_peered":"2024-02-02T13:30:18.172119+0000","last_unstale":"2024-02-02T13:30:19.368686+0000","last_undegraded":"2024-02-02T13:30:19.368686+0000","last_fullsized":"2024-02-02T13:30:19.368686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:05:40.191518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371761+0000","last_change":"2024-02-02T13:30:18.171905+0000","last_active":"2024-02-02T13:30:19.371761+0000","last_peered":"2024-02-02T13:30:19.371761+0000","last_clean":"2024-02-02T13:30:19.371761+0000","last_became_active":"2024-02-02T13:30:18.171686+0000","last_became_peered":"2024-02-02T13:30:18.171686+0000","last_unstale":"2024-02-02T13:30:19.371761+0000","last_undegraded":"2024-02-02T13:30:19.371761+0000","last_fullsized":"2024-02-02T13:30:19.371761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:30:12.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156611+0000","last_change":"2024-02-02T13:30:18.166659+0000","last_active":"2024-02-02T13:30:19.156611+0000","last_peered":"2024-02-02T13:30:19.156611+0000","last_clean":"2024-02-02T13:30:19.156611+0000","last_became_active":"2024-02-02T13:30:18.166428+0000","last_became_peered":"2024-02-02T13:30:18.166428+0000","last_unstale":"2024-02-02T13:30:19.156611+0000","last_undegraded":"2024-02-02T13:30:19.156611+0000","last_fullsized":"2024-02-02T13:30:19.156611+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:10.177166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372163+0000","last_change":"2024-02-02T13:30:18.163024+0000","last_active":"2024-02-02T13:30:19.372163+0000","last_peered":"2024-02-02T13:30:19.372163+0000","last_clean":"2024-02-02T13:30:19.372163+0000","last_became_active":"2024-02-02T13:30:18.162787+0000","last_became_peered":"2024-02-02T13:30:18.162787+0000","last_unstale":"2024-02-02T13:30:19.372163+0000","last_undegraded":"2024-02-02T13:30:19.372163+0000","last_fullsized":"2024-02-02T13:30:19.372163+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:04.103118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371196+0000","last_change":"2024-02-02T13:30:18.162958+0000","last_active":"2024-02-02T13:30:19.371196+0000","last_peered":"2024-02-02T13:30:19.371196+0000","last_clean":"2024-02-02T13:30:19.371196+0000","last_became_active":"2024-02-02T13:30:18.162542+0000","last_became_peered":"2024-02-02T13:30:18.162542+0000","last_unstale":"2024-02-02T13:30:19.371196+0000","last_undegraded":"2024-02-02T13:30:19.371196+0000","last_fullsized":"2024-02-02T13:30:19.371196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:37.867876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156780+0000","last_change":"2024-02-02T13:30:18.173642+0000","last_active":"2024-02-02T13:30:19.156780+0000","last_peered":"2024-02-02T13:30:19.156780+0000","last_clean":"2024-02-02T13:30:19.156780+0000","last_became_active":"2024-02-02T13:30:18.173203+0000","last_became_peered":"2024-02-02T13:30:18.173203+0000","last_unstale":"2024-02-02T13:30:19.156780+0000","last_undegraded":"2024-02-02T13:30:19.156780+0000","last_fullsized":"2024-02-02T13:30:19.156780+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:15.473677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373213+0000","last_change":"2024-02-02T13:30:18.154310+0000","last_active":"2024-02-02T13:30:19.373213+0000","last_peered":"2024-02-02T13:30:19.373213+0000","last_clean":"2024-02-02T13:30:19.373213+0000","last_became_active":"2024-02-02T13:30:18.153381+0000","last_became_peered":"2024-02-02T13:30:18.153381+0000","last_unstale":"2024-02-02T13:30:19.373213+0000","last_undegraded":"2024-02-02T13:30:19.373213+0000","last_fullsized":"2024-02-02T13:30:19.373213+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:39:59.563638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371140+0000","last_change":"2024-02-02T13:30:18.179751+0000","last_active":"2024-02-02T13:30:19.371140+0000","last_peered":"2024-02-02T13:30:19.371140+0000","last_clean":"2024-02-02T13:30:19.371140+0000","last_became_active":"2024-02-02T13:30:18.178618+0000","last_became_peered":"2024-02-02T13:30:18.178618+0000","last_unstale":"2024-02-02T13:30:19.371140+0000","last_undegraded":"2024-02-02T13:30:19.371140+0000","last_fullsized":"2024-02-02T13:30:19.371140+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:49:17.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371716+0000","last_change":"2024-02-02T13:30:18.172938+0000","last_active":"2024-02-02T13:30:19.371716+0000","last_peered":"2024-02-02T13:30:19.371716+0000","last_clean":"2024-02-02T13:30:19.371716+0000","last_became_active":"2024-02-02T13:30:18.172170+0000","last_became_peered":"2024-02-02T13:30:18.172170+0000","last_unstale":"2024-02-02T13:30:19.371716+0000","last_undegraded":"2024-02-02T13:30:19.371716+0000","last_fullsized":"2024-02-02T13:30:19.371716+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:54:21.647131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369089+0000","last_change":"2024-02-02T13:30:18.162078+0000","last_active":"2024-02-02T13:30:19.369089+0000","last_peered":"2024-02-02T13:30:19.369089+0000","last_clean":"2024-02-02T13:30:19.369089+0000","last_became_active":"2024-02-02T13:30:18.161320+0000","last_became_peered":"2024-02-02T13:30:18.161320+0000","last_unstale":"2024-02-02T13:30:19.369089+0000","last_undegraded":"2024-02-02T13:30:19.369089+0000","last_fullsized":"2024-02-02T13:30:19.369089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:32.532884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373045+0000","last_change":"2024-02-02T13:30:18.175484+0000","last_active":"2024-02-02T13:30:19.373045+0000","last_peered":"2024-02-02T13:30:19.373045+0000","last_clean":"2024-02-02T13:30:19.373045+0000","last_became_active":"2024-02-02T13:30:18.174846+0000","last_became_peered":"2024-02-02T13:30:18.174846+0000","last_unstale":"2024-02-02T13:30:19.373045+0000","last_undegraded":"2024-02-02T13:30:19.373045+0000","last_fullsized":"2024-02-02T13:30:19.373045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:56.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371993+0000","last_change":"2024-02-02T13:30:18.166954+0000","last_active":"2024-02-02T13:30:19.371993+0000","last_peered":"2024-02-02T13:30:19.371993+0000","last_clean":"2024-02-02T13:30:19.371993+0000","last_became_active":"2024-02-02T13:30:18.166705+0000","last_became_peered":"2024-02-02T13:30:18.166705+0000","last_unstale":"2024-02-02T13:30:19.371993+0000","last_undegraded":"2024-02-02T13:30:19.371993+0000","last_fullsized":"2024-02-02T13:30:19.371993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:02:21.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156718+0000","last_change":"2024-02-02T13:30:18.164270+0000","last_active":"2024-02-02T13:30:19.156718+0000","last_peered":"2024-02-02T13:30:19.156718+0000","last_clean":"2024-02-02T13:30:19.156718+0000","last_became_active":"2024-02-02T13:30:18.163877+0000","last_became_peered":"2024-02-02T13:30:18.163877+0000","last_unstale":"2024-02-02T13:30:19.156718+0000","last_undegraded":"2024-02-02T13:30:19.156718+0000","last_fullsized":"2024-02-02T13:30:19.156718+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:35.636539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155542+0000","last_change":"2024-02-02T13:30:18.161364+0000","last_active":"2024-02-02T13:30:19.155542+0000","last_peered":"2024-02-02T13:30:19.155542+0000","last_clean":"2024-02-02T13:30:19.155542+0000","last_became_active":"2024-02-02T13:30:18.161140+0000","last_became_peered":"2024-02-02T13:30:18.161140+0000","last_unstale":"2024-02-02T13:30:19.155542+0000","last_undegraded":"2024-02-02T13:30:19.155542+0000","last_fullsized":"2024-02-02T13:30:19.155542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:30:12.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370919+0000","last_change":"2024-02-02T13:30:18.171599+0000","last_active":"2024-02-02T13:30:19.370919+0000","last_peered":"2024-02-02T13:30:19.370919+0000","last_clean":"2024-02-02T13:30:19.370919+0000","last_became_active":"2024-02-02T13:30:18.171341+0000","last_became_peered":"2024-02-02T13:30:18.171341+0000","last_unstale":"2024-02-02T13:30:19.370919+0000","last_undegraded":"2024-02-02T13:30:19.370919+0000","last_fullsized":"2024-02-02T13:30:19.370919+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:33.083769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372699+0000","last_change":"2024-02-02T13:30:18.169351+0000","last_active":"2024-02-02T13:30:19.372699+0000","last_peered":"2024-02-02T13:30:19.372699+0000","last_clean":"2024-02-02T13:30:19.372699+0000","last_became_active":"2024-02-02T13:30:18.168944+0000","last_became_peered":"2024-02-02T13:30:18.168944+0000","last_unstale":"2024-02-02T13:30:19.372699+0000","last_undegraded":"2024-02-02T13:30:19.372699+0000","last_fullsized":"2024-02-02T13:30:19.372699+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:39:59.563638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156997+0000","last_change":"2024-02-02T13:30:18.175763+0000","last_active":"2024-02-02T13:30:19.156997+0000","last_peered":"2024-02-02T13:30:19.156997+0000","last_clean":"2024-02-02T13:30:19.156997+0000","last_became_active":"2024-02-02T13:30:18.174920+0000","last_became_peered":"2024-02-02T13:30:18.174920+0000","last_unstale":"2024-02-02T13:30:19.156997+0000","last_undegraded":"2024-02-02T13:30:19.156997+0000","last_fullsized":"2024-02-02T13:30:19.156997+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:00:06.858544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158138+0000","last_change":"2024-02-02T13:30:18.177880+0000","last_active":"2024-02-02T13:30:19.158138+0000","last_peered":"2024-02-02T13:30:19.158138+0000","last_clean":"2024-02-02T13:30:19.158138+0000","last_became_active":"2024-02-02T13:30:18.176843+0000","last_became_peered":"2024-02-02T13:30:18.176843+0000","last_unstale":"2024-02-02T13:30:19.158138+0000","last_undegraded":"2024-02-02T13:30:19.158138+0000","last_fullsized":"2024-02-02T13:30:19.158138+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:49:17.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157042+0000","last_change":"2024-02-02T13:30:18.179286+0000","last_active":"2024-02-02T13:30:19.157042+0000","last_peered":"2024-02-02T13:30:19.157042+0000","last_clean":"2024-02-02T13:30:19.157042+0000","last_became_active":"2024-02-02T13:30:18.179045+0000","last_became_peered":"2024-02-02T13:30:18.179045+0000","last_unstale":"2024-02-02T13:30:19.157042+0000","last_undegraded":"2024-02-02T13:30:19.157042+0000","last_fullsized":"2024-02-02T13:30:19.157042+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:55:17.319782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157388+0000","last_change":"2024-02-02T13:30:18.179827+0000","last_active":"2024-02-02T13:30:19.157388+0000","last_peered":"2024-02-02T13:30:19.157388+0000","last_clean":"2024-02-02T13:30:19.157388+0000","last_became_active":"2024-02-02T13:30:18.177850+0000","last_became_peered":"2024-02-02T13:30:18.177850+0000","last_unstale":"2024-02-02T13:30:19.157388+0000","last_undegraded":"2024-02-02T13:30:19.157388+0000","last_fullsized":"2024-02-02T13:30:19.157388+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:41.922553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368909+0000","last_change":"2024-02-02T13:30:18.175659+0000","last_active":"2024-02-02T13:30:19.368909+0000","last_peered":"2024-02-02T13:30:19.368909+0000","last_clean":"2024-02-02T13:30:19.368909+0000","last_became_active":"2024-02-02T13:30:18.175451+0000","last_became_peered":"2024-02-02T13:30:18.175451+0000","last_unstale":"2024-02-02T13:30:19.368909+0000","last_undegraded":"2024-02-02T13:30:19.368909+0000","last_fullsized":"2024-02-02T13:30:19.368909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:14:47.703783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374103+0000","last_change":"2024-02-02T13:30:18.178031+0000","last_active":"2024-02-02T13:30:19.374103+0000","last_peered":"2024-02-02T13:30:19.374103+0000","last_clean":"2024-02-02T13:30:19.374103+0000","last_became_active":"2024-02-02T13:30:18.177494+0000","last_became_peered":"2024-02-02T13:30:18.177494+0000","last_unstale":"2024-02-02T13:30:19.374103+0000","last_undegraded":"2024-02-02T13:30:19.374103+0000","last_fullsized":"2024-02-02T13:30:19.374103+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156880+0000","last_change":"2024-02-02T13:30:18.180707+0000","last_active":"2024-02-02T13:30:19.156880+0000","last_peered":"2024-02-02T13:30:19.156880+0000","last_clean":"2024-02-02T13:30:19.156880+0000","last_became_active":"2024-02-02T13:30:18.178810+0000","last_became_peered":"2024-02-02T13:30:18.178810+0000","last_unstale":"2024-02-02T13:30:19.156880+0000","last_undegraded":"2024-02-02T13:30:19.156880+0000","last_fullsized":"2024-02-02T13:30:19.156880+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:04:09.568950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368746+0000","last_change":"2024-02-02T13:30:18.171852+0000","last_active":"2024-02-02T13:30:19.368746+0000","last_peered":"2024-02-02T13:30:19.368746+0000","last_clean":"2024-02-02T13:30:19.368746+0000","last_became_active":"2024-02-02T13:30:18.171560+0000","last_became_peered":"2024-02-02T13:30:18.171560+0000","last_unstale":"2024-02-02T13:30:19.368746+0000","last_undegraded":"2024-02-02T13:30:19.368746+0000","last_fullsized":"2024-02-02T13:30:19.368746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:02.316841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372659+0000","last_change":"2024-02-02T13:30:18.159264+0000","last_active":"2024-02-02T13:30:19.372659+0000","last_peered":"2024-02-02T13:30:19.372659+0000","last_clean":"2024-02-02T13:30:19.372659+0000","last_became_active":"2024-02-02T13:30:18.159017+0000","last_became_peered":"2024-02-02T13:30:18.159017+0000","last_unstale":"2024-02-02T13:30:19.372659+0000","last_undegraded":"2024-02-02T13:30:19.372659+0000","last_fullsized":"2024-02-02T13:30:19.372659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:01.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373350+0000","last_change":"2024-02-02T13:30:18.166076+0000","last_active":"2024-02-02T13:30:19.373350+0000","last_peered":"2024-02-02T13:30:19.373350+0000","last_clean":"2024-02-02T13:30:19.373350+0000","last_became_active":"2024-02-02T13:30:18.165808+0000","last_became_peered":"2024-02-02T13:30:18.165808+0000","last_unstale":"2024-02-02T13:30:19.373350+0000","last_undegraded":"2024-02-02T13:30:19.373350+0000","last_fullsized":"2024-02-02T13:30:19.373350+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:06.279475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373669+0000","last_change":"2024-02-02T13:30:18.173980+0000","last_active":"2024-02-02T13:30:19.373669+0000","last_peered":"2024-02-02T13:30:19.373669+0000","last_clean":"2024-02-02T13:30:19.373669+0000","last_became_active":"2024-02-02T13:30:18.173733+0000","last_became_peered":"2024-02-02T13:30:18.173733+0000","last_unstale":"2024-02-02T13:30:19.373669+0000","last_undegraded":"2024-02-02T13:30:19.373669+0000","last_fullsized":"2024-02-02T13:30:19.373669+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:03:45.072121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368615+0000","last_change":"2024-02-02T13:30:18.154020+0000","last_active":"2024-02-02T13:30:19.368615+0000","last_peered":"2024-02-02T13:30:19.368615+0000","last_clean":"2024-02-02T13:30:19.368615+0000","last_became_active":"2024-02-02T13:30:18.153724+0000","last_became_peered":"2024-02-02T13:30:18.153724+0000","last_unstale":"2024-02-02T13:30:19.368615+0000","last_undegraded":"2024-02-02T13:30:19.368615+0000","last_fullsized":"2024-02-02T13:30:19.368615+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:14:54.859623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368971+0000","last_change":"2024-02-02T13:30:18.162283+0000","last_active":"2024-02-02T13:30:19.368971+0000","last_peered":"2024-02-02T13:30:19.368971+0000","last_clean":"2024-02-02T13:30:19.368971+0000","last_became_active":"2024-02-02T13:30:18.162078+0000","last_became_peered":"2024-02-02T13:30:18.162078+0000","last_unstale":"2024-02-02T13:30:19.368971+0000","last_undegraded":"2024-02-02T13:30:19.368971+0000","last_fullsized":"2024-02-02T13:30:19.368971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:35:41.167599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371286+0000","last_change":"2024-02-02T13:30:18.176875+0000","last_active":"2024-02-02T13:30:19.371286+0000","last_peered":"2024-02-02T13:30:19.371286+0000","last_clean":"2024-02-02T13:30:19.371286+0000","last_became_active":"2024-02-02T13:30:18.175744+0000","last_became_peered":"2024-02-02T13:30:18.175744+0000","last_unstale":"2024-02-02T13:30:19.371286+0000","last_undegraded":"2024-02-02T13:30:19.371286+0000","last_fullsized":"2024-02-02T13:30:19.371286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:49.758240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373272+0000","last_change":"2024-02-02T13:30:18.172871+0000","last_active":"2024-02-02T13:30:19.373272+0000","last_peered":"2024-02-02T13:30:19.373272+0000","last_clean":"2024-02-02T13:30:19.373272+0000","last_became_active":"2024-02-02T13:30:18.172610+0000","last_became_peered":"2024-02-02T13:30:18.172610+0000","last_unstale":"2024-02-02T13:30:19.373272+0000","last_undegraded":"2024-02-02T13:30:19.373272+0000","last_fullsized":"2024-02-02T13:30:19.373272+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:33:54.971010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370059+0000","last_change":"2024-02-02T13:30:18.172097+0000","last_active":"2024-02-02T13:30:19.370059+0000","last_peered":"2024-02-02T13:30:19.370059+0000","last_clean":"2024-02-02T13:30:19.370059+0000","last_became_active":"2024-02-02T13:30:18.171870+0000","last_became_peered":"2024-02-02T13:30:18.171870+0000","last_unstale":"2024-02-02T13:30:19.370059+0000","last_undegraded":"2024-02-02T13:30:19.370059+0000","last_fullsized":"2024-02-02T13:30:19.370059+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"104'4","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374313+0000","last_change":"2024-02-02T13:30:17.696764+0000","last_active":"2024-02-02T13:30:19.374313+0000","last_peered":"2024-02-02T13:30:19.374313+0000","last_clean":"2024-02-02T13:30:19.374313+0000","last_became_active":"2024-02-02T13:30:17.692001+0000","last_became_peered":"2024-02-02T13:30:17.692001+0000","last_unstale":"2024-02-02T13:30:19.374313+0000","last_undegraded":"2024-02-02T13:30:19.374313+0000","last_fullsized":"2024-02-02T13:30:19.374313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:18:31.944480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373890+0000","last_change":"2024-02-02T13:30:18.178188+0000","last_active":"2024-02-02T13:30:19.373890+0000","last_peered":"2024-02-02T13:30:19.373890+0000","last_clean":"2024-02-02T13:30:19.373890+0000","last_became_active":"2024-02-02T13:30:18.177803+0000","last_became_peered":"2024-02-02T13:30:18.177803+0000","last_unstale":"2024-02-02T13:30:19.373890+0000","last_undegraded":"2024-02-02T13:30:19.373890+0000","last_fullsized":"2024-02-02T13:30:19.373890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:01.221786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370530+0000","last_change":"2024-02-02T13:30:17.692896+0000","last_active":"2024-02-02T13:30:19.370530+0000","last_peered":"2024-02-02T13:30:19.370530+0000","last_clean":"2024-02-02T13:30:19.370530+0000","last_became_active":"2024-02-02T13:30:17.692590+0000","last_became_peered":"2024-02-02T13:30:17.692590+0000","last_unstale":"2024-02-02T13:30:19.370530+0000","last_undegraded":"2024-02-02T13:30:19.370530+0000","last_fullsized":"2024-02-02T13:30:19.370530+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:35:40.153531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372812+0000","last_change":"2024-02-02T13:30:18.173330+0000","last_active":"2024-02-02T13:30:19.372812+0000","last_peered":"2024-02-02T13:30:19.372812+0000","last_clean":"2024-02-02T13:30:19.372812+0000","last_became_active":"2024-02-02T13:30:18.172818+0000","last_became_peered":"2024-02-02T13:30:18.172818+0000","last_unstale":"2024-02-02T13:30:19.372812+0000","last_undegraded":"2024-02-02T13:30:19.372812+0000","last_fullsized":"2024-02-02T13:30:19.372812+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:13:49.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158707+0000","last_change":"2024-02-02T13:30:17.692777+0000","last_active":"2024-02-02T13:30:19.158707+0000","last_peered":"2024-02-02T13:30:19.158707+0000","last_clean":"2024-02-02T13:30:19.158707+0000","last_became_active":"2024-02-02T13:30:17.691789+0000","last_became_peered":"2024-02-02T13:30:17.691789+0000","last_unstale":"2024-02-02T13:30:19.158707+0000","last_undegraded":"2024-02-02T13:30:19.158707+0000","last_fullsized":"2024-02-02T13:30:19.158707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:47.278881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372473+0000","last_change":"2024-02-02T13:30:18.162081+0000","last_active":"2024-02-02T13:30:19.372473+0000","last_peered":"2024-02-02T13:30:19.372473+0000","last_clean":"2024-02-02T13:30:19.372473+0000","last_became_active":"2024-02-02T13:30:18.161800+0000","last_became_peered":"2024-02-02T13:30:18.161800+0000","last_unstale":"2024-02-02T13:30:19.372473+0000","last_undegraded":"2024-02-02T13:30:19.372473+0000","last_fullsized":"2024-02-02T13:30:19.372473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:29:34.425654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158644+0000","last_change":"2024-02-02T13:30:17.688264+0000","last_active":"2024-02-02T13:30:19.158644+0000","last_peered":"2024-02-02T13:30:19.158644+0000","last_clean":"2024-02-02T13:30:19.158644+0000","last_became_active":"2024-02-02T13:30:17.687981+0000","last_became_peered":"2024-02-02T13:30:17.687981+0000","last_unstale":"2024-02-02T13:30:19.158644+0000","last_undegraded":"2024-02-02T13:30:19.158644+0000","last_fullsized":"2024-02-02T13:30:19.158644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:56.217020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368359+0000","last_change":"2024-02-02T13:30:18.162282+0000","last_active":"2024-02-02T13:30:19.368359+0000","last_peered":"2024-02-02T13:30:19.368359+0000","last_clean":"2024-02-02T13:30:19.368359+0000","last_became_active":"2024-02-02T13:30:18.162048+0000","last_became_peered":"2024-02-02T13:30:18.162048+0000","last_unstale":"2024-02-02T13:30:19.368359+0000","last_undegraded":"2024-02-02T13:30:19.368359+0000","last_fullsized":"2024-02-02T13:30:19.368359+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:57.356934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373129+0000","last_change":"2024-02-02T13:30:17.689984+0000","last_active":"2024-02-02T13:30:19.373129+0000","last_peered":"2024-02-02T13:30:19.373129+0000","last_clean":"2024-02-02T13:30:19.373129+0000","last_became_active":"2024-02-02T13:30:17.688925+0000","last_became_peered":"2024-02-02T13:30:17.688925+0000","last_unstale":"2024-02-02T13:30:19.373129+0000","last_undegraded":"2024-02-02T13:30:19.373129+0000","last_fullsized":"2024-02-02T13:30:19.373129+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:57:31.518816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372912+0000","last_change":"2024-02-02T13:30:18.169206+0000","last_active":"2024-02-02T13:30:19.372912+0000","last_peered":"2024-02-02T13:30:19.372912+0000","last_clean":"2024-02-02T13:30:19.372912+0000","last_became_active":"2024-02-02T13:30:18.168605+0000","last_became_peered":"2024-02-02T13:30:18.168605+0000","last_unstale":"2024-02-02T13:30:19.372912+0000","last_undegraded":"2024-02-02T13:30:19.372912+0000","last_fullsized":"2024-02-02T13:30:19.372912+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:08:38.966770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370396+0000","last_change":"2024-02-02T13:30:17.693364+0000","last_active":"2024-02-02T13:30:19.370396+0000","last_peered":"2024-02-02T13:30:19.370396+0000","last_clean":"2024-02-02T13:30:19.370396+0000","last_became_active":"2024-02-02T13:30:17.693145+0000","last_became_peered":"2024-02-02T13:30:17.693145+0000","last_unstale":"2024-02-02T13:30:19.370396+0000","last_undegraded":"2024-02-02T13:30:19.370396+0000","last_fullsized":"2024-02-02T13:30:19.370396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:34.744740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371990+0000","last_change":"2024-02-02T13:30:18.178201+0000","last_active":"2024-02-02T13:30:19.371990+0000","last_peered":"2024-02-02T13:30:19.371990+0000","last_clean":"2024-02-02T13:30:19.371990+0000","last_became_active":"2024-02-02T13:30:18.177946+0000","last_became_peered":"2024-02-02T13:30:18.177946+0000","last_unstale":"2024-02-02T13:30:19.371990+0000","last_undegraded":"2024-02-02T13:30:19.371990+0000","last_fullsized":"2024-02-02T13:30:19.371990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:55:33.625215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158302+0000","last_change":"2024-02-02T13:30:17.695679+0000","last_active":"2024-02-02T13:30:19.158302+0000","last_peered":"2024-02-02T13:30:19.158302+0000","last_clean":"2024-02-02T13:30:19.158302+0000","last_became_active":"2024-02-02T13:30:17.695344+0000","last_became_peered":"2024-02-02T13:30:17.695344+0000","last_unstale":"2024-02-02T13:30:19.158302+0000","last_undegraded":"2024-02-02T13:30:19.158302+0000","last_fullsized":"2024-02-02T13:30:19.158302+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:45:54.238347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155379+0000","last_change":"2024-02-02T13:30:18.159274+0000","last_active":"2024-02-02T13:30:19.155379+0000","last_peered":"2024-02-02T13:30:19.155379+0000","last_clean":"2024-02-02T13:30:19.155379+0000","last_became_active":"2024-02-02T13:30:18.159044+0000","last_became_peered":"2024-02-02T13:30:18.159044+0000","last_unstale":"2024-02-02T13:30:19.155379+0000","last_undegraded":"2024-02-02T13:30:19.155379+0000","last_fullsized":"2024-02-02T13:30:19.155379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:02:21.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369188+0000","last_change":"2024-02-02T13:30:17.690202+0000","last_active":"2024-02-02T13:30:19.369188+0000","last_peered":"2024-02-02T13:30:19.369188+0000","last_clean":"2024-02-02T13:30:19.369188+0000","last_became_active":"2024-02-02T13:30:17.689539+0000","last_became_peered":"2024-02-02T13:30:17.689539+0000","last_unstale":"2024-02-02T13:30:19.369188+0000","last_undegraded":"2024-02-02T13:30:19.369188+0000","last_fullsized":"2024-02-02T13:30:19.369188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:40.908485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155694+0000","last_change":"2024-02-02T13:30:18.153639+0000","last_active":"2024-02-02T13:30:19.155694+0000","last_peered":"2024-02-02T13:30:19.155694+0000","last_clean":"2024-02-02T13:30:19.155694+0000","last_became_active":"2024-02-02T13:30:18.153386+0000","last_became_peered":"2024-02-02T13:30:18.153386+0000","last_unstale":"2024-02-02T13:30:19.155694+0000","last_undegraded":"2024-02-02T13:30:19.155694+0000","last_fullsized":"2024-02-02T13:30:19.155694+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:29:34.425654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373038+0000","last_change":"2024-02-02T13:30:17.690131+0000","last_active":"2024-02-02T13:30:19.373038+0000","last_peered":"2024-02-02T13:30:19.373038+0000","last_clean":"2024-02-02T13:30:19.373038+0000","last_became_active":"2024-02-02T13:30:17.689752+0000","last_became_peered":"2024-02-02T13:30:17.689752+0000","last_unstale":"2024-02-02T13:30:19.373038+0000","last_undegraded":"2024-02-02T13:30:19.373038+0000","last_fullsized":"2024-02-02T13:30:19.373038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:22:05.189040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372139+0000","last_change":"2024-02-02T13:30:18.160638+0000","last_active":"2024-02-02T13:30:19.372139+0000","last_peered":"2024-02-02T13:30:19.372139+0000","last_clean":"2024-02-02T13:30:19.372139+0000","last_became_active":"2024-02-02T13:30:18.160319+0000","last_became_peered":"2024-02-02T13:30:18.160319+0000","last_unstale":"2024-02-02T13:30:19.372139+0000","last_undegraded":"2024-02-02T13:30:19.372139+0000","last_fullsized":"2024-02-02T13:30:19.372139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:08:38.966770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373495+0000","last_change":"2024-02-02T13:30:17.694587+0000","last_active":"2024-02-02T13:30:19.373495+0000","last_peered":"2024-02-02T13:30:19.373495+0000","last_clean":"2024-02-02T13:30:19.373495+0000","last_became_active":"2024-02-02T13:30:17.694295+0000","last_became_peered":"2024-02-02T13:30:17.694295+0000","last_unstale":"2024-02-02T13:30:19.373495+0000","last_undegraded":"2024-02-02T13:30:19.373495+0000","last_fullsized":"2024-02-02T13:30:19.373495+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:05:39.177449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157851+0000","last_change":"2024-02-02T13:30:18.178014+0000","last_active":"2024-02-02T13:30:19.157851+0000","last_peered":"2024-02-02T13:30:19.157851+0000","last_clean":"2024-02-02T13:30:19.157851+0000","last_became_active":"2024-02-02T13:30:18.177228+0000","last_became_peered":"2024-02-02T13:30:18.177228+0000","last_unstale":"2024-02-02T13:30:19.157851+0000","last_undegraded":"2024-02-02T13:30:19.157851+0000","last_fullsized":"2024-02-02T13:30:19.157851+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"104'3","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374770+0000","last_change":"2024-02-02T13:30:17.697490+0000","last_active":"2024-02-02T13:30:19.374770+0000","last_peered":"2024-02-02T13:30:19.374770+0000","last_clean":"2024-02-02T13:30:19.374770+0000","last_became_active":"2024-02-02T13:30:17.697038+0000","last_became_peered":"2024-02-02T13:30:17.697038+0000","last_unstale":"2024-02-02T13:30:19.374770+0000","last_undegraded":"2024-02-02T13:30:19.374770+0000","last_fullsized":"2024-02-02T13:30:19.374770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:06.872985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369910+0000","last_change":"2024-02-02T13:30:18.178121+0000","last_active":"2024-02-02T13:30:19.369910+0000","last_peered":"2024-02-02T13:30:19.369910+0000","last_clean":"2024-02-02T13:30:19.369910+0000","last_became_active":"2024-02-02T13:30:18.177507+0000","last_became_peered":"2024-02-02T13:30:18.177507+0000","last_unstale":"2024-02-02T13:30:19.369910+0000","last_undegraded":"2024-02-02T13:30:19.369910+0000","last_fullsized":"2024-02-02T13:30:19.369910+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:26:37.867876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"104'6","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371388+0000","last_change":"2024-02-02T13:30:17.681225+0000","last_active":"2024-02-02T13:30:19.371388+0000","last_peered":"2024-02-02T13:30:19.371388+0000","last_clean":"2024-02-02T13:30:19.371388+0000","last_became_active":"2024-02-02T13:30:17.680580+0000","last_became_peered":"2024-02-02T13:30:17.680580+0000","last_unstale":"2024-02-02T13:30:19.371388+0000","last_undegraded":"2024-02-02T13:30:19.371388+0000","last_fullsized":"2024-02-02T13:30:19.371388+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:12:04.150942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158412+0000","last_change":"2024-02-02T13:30:18.177996+0000","last_active":"2024-02-02T13:30:19.158412+0000","last_peered":"2024-02-02T13:30:19.158412+0000","last_clean":"2024-02-02T13:30:19.158412+0000","last_became_active":"2024-02-02T13:30:18.177005+0000","last_became_peered":"2024-02-02T13:30:18.177005+0000","last_unstale":"2024-02-02T13:30:19.158412+0000","last_undegraded":"2024-02-02T13:30:19.158412+0000","last_fullsized":"2024-02-02T13:30:19.158412+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:33:54.971010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369394+0000","last_change":"2024-02-02T13:30:17.690366+0000","last_active":"2024-02-02T13:30:19.369394+0000","last_peered":"2024-02-02T13:30:19.369394+0000","last_clean":"2024-02-02T13:30:19.369394+0000","last_became_active":"2024-02-02T13:30:17.689898+0000","last_became_peered":"2024-02-02T13:30:17.689898+0000","last_unstale":"2024-02-02T13:30:19.369394+0000","last_undegraded":"2024-02-02T13:30:19.369394+0000","last_fullsized":"2024-02-02T13:30:19.369394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:34.622471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372212+0000","last_change":"2024-02-02T13:30:18.176917+0000","last_active":"2024-02-02T13:30:19.372212+0000","last_peered":"2024-02-02T13:30:19.372212+0000","last_clean":"2024-02-02T13:30:19.372212+0000","last_became_active":"2024-02-02T13:30:18.176687+0000","last_became_peered":"2024-02-02T13:30:18.176687+0000","last_unstale":"2024-02-02T13:30:19.372212+0000","last_undegraded":"2024-02-02T13:30:19.372212+0000","last_fullsized":"2024-02-02T13:30:19.372212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:20:34.227608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155376+0000","last_change":"2024-02-02T13:30:17.677888+0000","last_active":"2024-02-02T13:30:19.155376+0000","last_peered":"2024-02-02T13:30:19.155376+0000","last_clean":"2024-02-02T13:30:19.155376+0000","last_became_active":"2024-02-02T13:30:17.677654+0000","last_became_peered":"2024-02-02T13:30:17.677654+0000","last_unstale":"2024-02-02T13:30:19.155376+0000","last_undegraded":"2024-02-02T13:30:19.155376+0000","last_fullsized":"2024-02-02T13:30:19.155376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:00.207717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368256+0000","last_change":"2024-02-02T13:30:18.161843+0000","last_active":"2024-02-02T13:30:19.368256+0000","last_peered":"2024-02-02T13:30:19.368256+0000","last_clean":"2024-02-02T13:30:19.368256+0000","last_became_active":"2024-02-02T13:30:18.161632+0000","last_became_peered":"2024-02-02T13:30:18.161632+0000","last_unstale":"2024-02-02T13:30:19.368256+0000","last_undegraded":"2024-02-02T13:30:19.368256+0000","last_fullsized":"2024-02-02T13:30:19.368256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:42.765712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371595+0000","last_change":"2024-02-02T13:30:17.695003+0000","last_active":"2024-02-02T13:30:19.371595+0000","last_peered":"2024-02-02T13:30:19.371595+0000","last_clean":"2024-02-02T13:30:19.371595+0000","last_became_active":"2024-02-02T13:30:17.694578+0000","last_became_peered":"2024-02-02T13:30:17.694578+0000","last_unstale":"2024-02-02T13:30:19.371595+0000","last_undegraded":"2024-02-02T13:30:19.371595+0000","last_fullsized":"2024-02-02T13:30:19.371595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:04:55.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369156+0000","last_change":"2024-02-02T13:30:18.172367+0000","last_active":"2024-02-02T13:30:19.369156+0000","last_peered":"2024-02-02T13:30:19.369156+0000","last_clean":"2024-02-02T13:30:19.369156+0000","last_became_active":"2024-02-02T13:30:18.171745+0000","last_became_peered":"2024-02-02T13:30:18.171745+0000","last_unstale":"2024-02-02T13:30:19.369156+0000","last_undegraded":"2024-02-02T13:30:19.369156+0000","last_fullsized":"2024-02-02T13:30:19.369156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:07:01.500349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"104'8","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158371+0000","last_change":"2024-02-02T13:30:17.692366+0000","last_active":"2024-02-02T13:30:19.158371+0000","last_peered":"2024-02-02T13:30:19.158371+0000","last_clean":"2024-02-02T13:30:19.158371+0000","last_became_active":"2024-02-02T13:30:17.690527+0000","last_became_peered":"2024-02-02T13:30:17.690527+0000","last_unstale":"2024-02-02T13:30:19.158371+0000","last_undegraded":"2024-02-02T13:30:19.158371+0000","last_fullsized":"2024-02-02T13:30:19.158371+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:05.265407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372700+0000","last_change":"2024-02-02T13:30:18.165263+0000","last_active":"2024-02-02T13:30:19.372700+0000","last_peered":"2024-02-02T13:30:19.372700+0000","last_clean":"2024-02-02T13:30:19.372700+0000","last_became_active":"2024-02-02T13:30:18.165019+0000","last_became_peered":"2024-02-02T13:30:18.165019+0000","last_unstale":"2024-02-02T13:30:19.372700+0000","last_undegraded":"2024-02-02T13:30:19.372700+0000","last_fullsized":"2024-02-02T13:30:19.372700+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:29:44.188635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370537+0000","last_change":"2024-02-02T13:30:17.696174+0000","last_active":"2024-02-02T13:30:19.370537+0000","last_peered":"2024-02-02T13:30:19.370537+0000","last_clean":"2024-02-02T13:30:19.370537+0000","last_became_active":"2024-02-02T13:30:17.695873+0000","last_became_peered":"2024-02-02T13:30:17.695873+0000","last_unstale":"2024-02-02T13:30:19.370537+0000","last_undegraded":"2024-02-02T13:30:19.370537+0000","last_fullsized":"2024-02-02T13:30:19.370537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:04:55.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370161+0000","last_change":"2024-02-02T13:30:18.167798+0000","last_active":"2024-02-02T13:30:19.370161+0000","last_peered":"2024-02-02T13:30:19.370161+0000","last_clean":"2024-02-02T13:30:19.370161+0000","last_became_active":"2024-02-02T13:30:18.167582+0000","last_became_peered":"2024-02-02T13:30:18.167582+0000","last_unstale":"2024-02-02T13:30:19.370161+0000","last_undegraded":"2024-02-02T13:30:19.370161+0000","last_fullsized":"2024-02-02T13:30:19.370161+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:42:31.168036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158770+0000","last_change":"2024-02-02T13:30:17.690906+0000","last_active":"2024-02-02T13:30:19.158770+0000","last_peered":"2024-02-02T13:30:19.158770+0000","last_clean":"2024-02-02T13:30:19.158770+0000","last_became_active":"2024-02-02T13:30:17.690556+0000","last_became_peered":"2024-02-02T13:30:17.690556+0000","last_unstale":"2024-02-02T13:30:19.158770+0000","last_undegraded":"2024-02-02T13:30:19.158770+0000","last_fullsized":"2024-02-02T13:30:19.158770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:20:54.568579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371533+0000","last_change":"2024-02-02T13:30:18.167368+0000","last_active":"2024-02-02T13:30:19.371533+0000","last_peered":"2024-02-02T13:30:19.371533+0000","last_clean":"2024-02-02T13:30:19.371533+0000","last_became_active":"2024-02-02T13:30:18.167028+0000","last_became_peered":"2024-02-02T13:30:18.167028+0000","last_unstale":"2024-02-02T13:30:19.371533+0000","last_undegraded":"2024-02-02T13:30:19.371533+0000","last_fullsized":"2024-02-02T13:30:19.371533+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:45:55.252415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158085+0000","last_change":"2024-02-02T13:30:17.691999+0000","last_active":"2024-02-02T13:30:19.158085+0000","last_peered":"2024-02-02T13:30:19.158085+0000","last_clean":"2024-02-02T13:30:19.158085+0000","last_became_active":"2024-02-02T13:30:17.691758+0000","last_became_peered":"2024-02-02T13:30:17.691758+0000","last_unstale":"2024-02-02T13:30:19.158085+0000","last_undegraded":"2024-02-02T13:30:19.158085+0000","last_fullsized":"2024-02-02T13:30:19.158085+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:03.089050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.156989+0000","last_change":"2024-02-02T13:30:18.164157+0000","last_active":"2024-02-02T13:30:19.156989+0000","last_peered":"2024-02-02T13:30:19.156989+0000","last_clean":"2024-02-02T13:30:19.156989+0000","last_became_active":"2024-02-02T13:30:18.162831+0000","last_became_peered":"2024-02-02T13:30:18.162831+0000","last_unstale":"2024-02-02T13:30:19.156989+0000","last_undegraded":"2024-02-02T13:30:19.156989+0000","last_fullsized":"2024-02-02T13:30:19.156989+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:45.411357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155469+0000","last_change":"2024-02-02T13:30:17.684246+0000","last_active":"2024-02-02T13:30:19.155469+0000","last_peered":"2024-02-02T13:30:19.155469+0000","last_clean":"2024-02-02T13:30:19.155469+0000","last_became_active":"2024-02-02T13:30:17.683993+0000","last_became_peered":"2024-02-02T13:30:17.683993+0000","last_unstale":"2024-02-02T13:30:19.155469+0000","last_undegraded":"2024-02-02T13:30:19.155469+0000","last_fullsized":"2024-02-02T13:30:19.155469+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:01:08.839812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.373468+0000","last_change":"2024-02-02T13:30:18.154790+0000","last_active":"2024-02-02T13:30:19.373468+0000","last_peered":"2024-02-02T13:30:19.373468+0000","last_clean":"2024-02-02T13:30:19.373468+0000","last_became_active":"2024-02-02T13:30:18.154572+0000","last_became_peered":"2024-02-02T13:30:18.154572+0000","last_unstale":"2024-02-02T13:30:19.373468+0000","last_undegraded":"2024-02-02T13:30:19.373468+0000","last_fullsized":"2024-02-02T13:30:19.373468+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:33.083769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157100+0000","last_change":"2024-02-02T13:30:17.686586+0000","last_active":"2024-02-02T13:30:19.157100+0000","last_peered":"2024-02-02T13:30:19.157100+0000","last_clean":"2024-02-02T13:30:19.157100+0000","last_became_active":"2024-02-02T13:30:17.686305+0000","last_became_peered":"2024-02-02T13:30:17.686305+0000","last_unstale":"2024-02-02T13:30:19.157100+0000","last_undegraded":"2024-02-02T13:30:19.157100+0000","last_fullsized":"2024-02-02T13:30:19.157100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:18.234058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157443+0000","last_change":"2024-02-02T13:30:18.162886+0000","last_active":"2024-02-02T13:30:19.157443+0000","last_peered":"2024-02-02T13:30:19.157443+0000","last_clean":"2024-02-02T13:30:19.157443+0000","last_became_active":"2024-02-02T13:30:18.162607+0000","last_became_peered":"2024-02-02T13:30:18.162607+0000","last_unstale":"2024-02-02T13:30:19.157443+0000","last_undegraded":"2024-02-02T13:30:19.157443+0000","last_fullsized":"2024-02-02T13:30:19.157443+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:24:21.961257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.157152+0000","last_change":"2024-02-02T13:30:17.693584+0000","last_active":"2024-02-02T13:30:19.157152+0000","last_peered":"2024-02-02T13:30:19.157152+0000","last_clean":"2024-02-02T13:30:19.157152+0000","last_became_active":"2024-02-02T13:30:17.693339+0000","last_became_peered":"2024-02-02T13:30:17.693339+0000","last_unstale":"2024-02-02T13:30:19.157152+0000","last_undegraded":"2024-02-02T13:30:19.157152+0000","last_fullsized":"2024-02-02T13:30:19.157152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:28:28.580941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371815+0000","last_change":"2024-02-02T13:30:18.176584+0000","last_active":"2024-02-02T13:30:19.371815+0000","last_peered":"2024-02-02T13:30:19.371815+0000","last_clean":"2024-02-02T13:30:19.371815+0000","last_became_active":"2024-02-02T13:30:18.176239+0000","last_became_peered":"2024-02-02T13:30:18.176239+0000","last_unstale":"2024-02-02T13:30:19.371815+0000","last_undegraded":"2024-02-02T13:30:19.371815+0000","last_fullsized":"2024-02-02T13:30:19.371815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:30:13.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.155483+0000","last_change":"2024-02-02T13:30:17.684864+0000","last_active":"2024-02-02T13:30:19.155483+0000","last_peered":"2024-02-02T13:30:19.155483+0000","last_clean":"2024-02-02T13:30:19.155483+0000","last_became_active":"2024-02-02T13:30:17.684239+0000","last_became_peered":"2024-02-02T13:30:17.684239+0000","last_unstale":"2024-02-02T13:30:19.155483+0000","last_undegraded":"2024-02-02T13:30:19.155483+0000","last_fullsized":"2024-02-02T13:30:19.155483+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:54:20.633062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370160+0000","last_change":"2024-02-02T13:30:18.166635+0000","last_active":"2024-02-02T13:30:19.370160+0000","last_peered":"2024-02-02T13:30:19.370160+0000","last_clean":"2024-02-02T13:30:19.370160+0000","last_became_active":"2024-02-02T13:30:18.165704+0000","last_became_peered":"2024-02-02T13:30:18.165704+0000","last_unstale":"2024-02-02T13:30:19.370160+0000","last_undegraded":"2024-02-02T13:30:19.370160+0000","last_fullsized":"2024-02-02T13:30:19.370160+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:20:55.582647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369624+0000","last_change":"2024-02-02T13:30:17.688632+0000","last_active":"2024-02-02T13:30:19.369624+0000","last_peered":"2024-02-02T13:30:19.369624+0000","last_clean":"2024-02-02T13:30:19.369624+0000","last_became_active":"2024-02-02T13:30:17.688386+0000","last_became_peered":"2024-02-02T13:30:17.688386+0000","last_unstale":"2024-02-02T13:30:19.369624+0000","last_undegraded":"2024-02-02T13:30:19.369624+0000","last_fullsized":"2024-02-02T13:30:19.369624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:33:39.292239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371879+0000","last_change":"2024-02-02T13:30:18.171422+0000","last_active":"2024-02-02T13:30:19.371879+0000","last_peered":"2024-02-02T13:30:19.371879+0000","last_clean":"2024-02-02T13:30:19.371879+0000","last_became_active":"2024-02-02T13:30:18.171187+0000","last_became_peered":"2024-02-02T13:30:18.171187+0000","last_unstale":"2024-02-02T13:30:19.371879+0000","last_undegraded":"2024-02-02T13:30:19.371879+0000","last_fullsized":"2024-02-02T13:30:19.371879+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:17:49.734188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.368446+0000","last_change":"2024-02-02T13:30:17.687826+0000","last_active":"2024-02-02T13:30:19.368446+0000","last_peered":"2024-02-02T13:30:19.368446+0000","last_clean":"2024-02-02T13:30:19.368446+0000","last_became_active":"2024-02-02T13:30:17.684775+0000","last_became_peered":"2024-02-02T13:30:17.684775+0000","last_unstale":"2024-02-02T13:30:19.368446+0000","last_undegraded":"2024-02-02T13:30:19.368446+0000","last_fullsized":"2024-02-02T13:30:19.368446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:35:44.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371943+0000","last_change":"2024-02-02T13:30:18.171492+0000","last_active":"2024-02-02T13:30:19.371943+0000","last_peered":"2024-02-02T13:30:19.371943+0000","last_clean":"2024-02-02T13:30:19.371943+0000","last_became_active":"2024-02-02T13:30:18.170925+0000","last_became_peered":"2024-02-02T13:30:18.170925+0000","last_unstale":"2024-02-02T13:30:19.371943+0000","last_undegraded":"2024-02-02T13:30:19.371943+0000","last_fullsized":"2024-02-02T13:30:19.371943+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:33.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370752+0000","last_change":"2024-02-02T13:30:17.691263+0000","last_active":"2024-02-02T13:30:19.370752+0000","last_peered":"2024-02-02T13:30:19.370752+0000","last_clean":"2024-02-02T13:30:19.370752+0000","last_became_active":"2024-02-02T13:30:17.691017+0000","last_became_peered":"2024-02-02T13:30:17.691017+0000","last_unstale":"2024-02-02T13:30:19.370752+0000","last_undegraded":"2024-02-02T13:30:19.370752+0000","last_fullsized":"2024-02-02T13:30:19.370752+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:56.342866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370290+0000","last_change":"2024-02-02T13:30:18.180187+0000","last_active":"2024-02-02T13:30:19.370290+0000","last_peered":"2024-02-02T13:30:19.370290+0000","last_clean":"2024-02-02T13:30:19.370290+0000","last_became_active":"2024-02-02T13:30:18.179671+0000","last_became_peered":"2024-02-02T13:30:18.179671+0000","last_unstale":"2024-02-02T13:30:19.370290+0000","last_undegraded":"2024-02-02T13:30:19.370290+0000","last_fullsized":"2024-02-02T13:30:19.370290+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:15:27.821324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372139+0000","last_change":"2024-02-02T13:30:17.696248+0000","last_active":"2024-02-02T13:30:19.372139+0000","last_peered":"2024-02-02T13:30:19.372139+0000","last_clean":"2024-02-02T13:30:19.372139+0000","last_became_active":"2024-02-02T13:30:17.695391+0000","last_became_peered":"2024-02-02T13:30:17.695391+0000","last_unstale":"2024-02-02T13:30:19.372139+0000","last_undegraded":"2024-02-02T13:30:19.372139+0000","last_fullsized":"2024-02-02T13:30:19.372139+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:12:04.150942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.371347+0000","last_change":"2024-02-02T13:30:18.157250+0000","last_active":"2024-02-02T13:30:19.371347+0000","last_peered":"2024-02-02T13:30:19.371347+0000","last_clean":"2024-02-02T13:30:19.371347+0000","last_became_active":"2024-02-02T13:30:18.156796+0000","last_became_peered":"2024-02-02T13:30:18.156796+0000","last_unstale":"2024-02-02T13:30:19.371347+0000","last_undegraded":"2024-02-02T13:30:19.371347+0000","last_fullsized":"2024-02-02T13:30:19.371347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158750+0000","last_change":"2024-02-02T13:30:17.690926+0000","last_active":"2024-02-02T13:30:19.158750+0000","last_peered":"2024-02-02T13:30:19.158750+0000","last_clean":"2024-02-02T13:30:19.158750+0000","last_became_active":"2024-02-02T13:30:17.690702+0000","last_became_peered":"2024-02-02T13:30:17.690702+0000","last_unstale":"2024-02-02T13:30:19.158750+0000","last_undegraded":"2024-02-02T13:30:19.158750+0000","last_fullsized":"2024-02-02T13:30:19.158750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:38:48.744172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.158537+0000","last_change":"2024-02-02T13:30:18.177220+0000","last_active":"2024-02-02T13:30:19.158537+0000","last_peered":"2024-02-02T13:30:19.158537+0000","last_clean":"2024-02-02T13:30:19.158537+0000","last_became_active":"2024-02-02T13:30:18.176714+0000","last_became_peered":"2024-02-02T13:30:18.176714+0000","last_unstale":"2024-02-02T13:30:19.158537+0000","last_undegraded":"2024-02-02T13:30:19.158537+0000","last_fullsized":"2024-02-02T13:30:19.158537+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:33.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374617+0000","last_change":"2024-02-02T13:30:17.692879+0000","last_active":"2024-02-02T13:30:19.374617+0000","last_peered":"2024-02-02T13:30:19.374617+0000","last_clean":"2024-02-02T13:30:19.374617+0000","last_became_active":"2024-02-02T13:30:17.692639+0000","last_became_peered":"2024-02-02T13:30:17.692639+0000","last_unstale":"2024-02-02T13:30:19.374617+0000","last_undegraded":"2024-02-02T13:30:19.374617+0000","last_fullsized":"2024-02-02T13:30:19.374617+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:14:53.845555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370114+0000","last_change":"2024-02-02T13:30:18.158545+0000","last_active":"2024-02-02T13:30:19.370114+0000","last_peered":"2024-02-02T13:30:19.370114+0000","last_clean":"2024-02-02T13:30:19.370114+0000","last_became_active":"2024-02-02T13:30:18.158044+0000","last_became_peered":"2024-02-02T13:30:18.158044+0000","last_unstale":"2024-02-02T13:30:19.370114+0000","last_undegraded":"2024-02-02T13:30:19.370114+0000","last_fullsized":"2024-02-02T13:30:19.370114+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:56.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374986+0000","last_change":"2024-02-02T13:30:17.697266+0000","last_active":"2024-02-02T13:30:19.374986+0000","last_peered":"2024-02-02T13:30:19.374986+0000","last_clean":"2024-02-02T13:30:19.374986+0000","last_became_active":"2024-02-02T13:30:17.693307+0000","last_became_peered":"2024-02-02T13:30:17.693307+0000","last_unstale":"2024-02-02T13:30:19.374986+0000","last_undegraded":"2024-02-02T13:30:19.374986+0000","last_fullsized":"2024-02-02T13:30:19.374986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:01.302773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.374556+0000","last_change":"2024-02-02T13:30:18.175593+0000","last_active":"2024-02-02T13:30:19.374556+0000","last_peered":"2024-02-02T13:30:19.374556+0000","last_clean":"2024-02-02T13:30:19.374556+0000","last_became_active":"2024-02-02T13:30:18.175298+0000","last_became_peered":"2024-02-02T13:30:18.175298+0000","last_unstale":"2024-02-02T13:30:19.374556+0000","last_undegraded":"2024-02-02T13:30:19.374556+0000","last_fullsized":"2024-02-02T13:30:19.374556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:13.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.370725+0000","last_change":"2024-02-02T13:30:17.694926+0000","last_active":"2024-02-02T13:30:19.370725+0000","last_peered":"2024-02-02T13:30:19.370725+0000","last_clean":"2024-02-02T13:30:19.370725+0000","last_became_active":"2024-02-02T13:30:17.694662+0000","last_became_peered":"2024-02-02T13:30:17.694662+0000","last_unstale":"2024-02-02T13:30:19.370725+0000","last_undegraded":"2024-02-02T13:30:19.370725+0000","last_fullsized":"2024-02-02T13:30:19.370725+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:14:46.689715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.369516+0000","last_change":"2024-02-02T13:30:18.154968+0000","last_active":"2024-02-02T13:30:19.369516+0000","last_peered":"2024-02-02T13:30:19.369516+0000","last_clean":"2024-02-02T13:30:19.369516+0000","last_became_active":"2024-02-02T13:30:18.154665+0000","last_became_peered":"2024-02-02T13:30:18.154665+0000","last_unstale":"2024-02-02T13:30:19.369516+0000","last_undegraded":"2024-02-02T13:30:19.369516+0000","last_fullsized":"2024-02-02T13:30:19.369516+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:17.650014+0000","last_clean_scrub_stamp":"2024-02-02T13:30:17.650014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:35.758808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-02-02T13:30:19.372891+0000","last_change":"2024-02-02T13:30:17.687927+0000","last_active":"2024-02-02T13:30:19.372891+0000","last_peered":"2024-02-02T13:30:19.372891+0000","last_clean":"2024-02-02T13:30:19.372891+0000","last_became_active":"2024-02-02T13:30:17.685140+0000","last_became_peered":"2024-02-02T13:30:17.685140+0000","last_unstale":"2024-02-02T13:30:19.372891+0000","last_undegraded":"2024-02-02T13:30:19.372891+0000","last_fullsized":"2024-02-02T13:30:19.372891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:30:16.635945+0000","last_clean_scrub_stamp":"2024-02-02T13:30:16.635945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:32.069701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-02-02T13:30:18.150325+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:30:18.150325+0000","last_peered":"2024-02-02T13:30:18.150325+0000","last_clean":"2024-02-02T13:30:18.150325+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:30:18.150325+0000","last_undegraded":"2024-02-02T13:30:18.150325+0000","last_fullsized":"2024-02-02T13:30:18.150325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1132,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194179,"statfs":{"total":200000000,"available":198839990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.097,"15min":1.0649999999999999},"min":{"1min":0.67600000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.492,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1160000000000001,"15min":1.1279999999999999},"min":{"1min":0.76500000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6040000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":1.3360000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.107,"15min":1.093},"min":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":1.893,"15min":1.893},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.159,"15min":1.117},"min":{"1min":0.87,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.599,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.532}]},{"osd":2,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2430000000000001,"15min":1.2230000000000001},"min":{"1min":1.002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5409999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.177,"15min":1.1870000000000001},"min":{"1min":0.80000000000000004,"5min":0.623,"15min":0.623},"max":{"1min":1.4510000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.4359999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.89200000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.633,"5min":1.819,"15min":1.819},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3340000000000001,"15min":1.3120000000000001},"min":{"1min":0.79100000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8660000000000001,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.268,"15min":1.1970000000000001},"min":{"1min":0.93600000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6559999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.222},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":1.056,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7929999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5780000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.373,"15min":1.351},"min":{"1min":0.80200000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.726,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.613},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.423,"15min":1.454},"min":{"1min":0.76000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7370000000000001,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":1.4970000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:29:38 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.51,"15min":1.538},"min":{"1min":0.88700000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.855,"5min":3.6840000000000002,"15min":3.6840000000000002},"last":1.645},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.111,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.821,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.6879999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194756,"statfs":{"total":200000000,"available":199430753,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.077,"15min":1.018},"min":{"1min":0.86799999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.571,"5min":1.577,"15min":1.577},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.0700000000000001,"15min":1.0189999999999999},"min":{"1min":0.75800000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.002}]},{"osd":1,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0529999999999999,"15min":1.02},"min":{"1min":0.78900000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5229999999999999,"5min":1.556,"15min":1.556},"last":1.288},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.0740000000000001},"min":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.456,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.141}]},{"osd":2,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1519999999999999,"15min":1.0720000000000001},"min":{"1min":0.77600000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.498,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.169,"15min":1.153},"min":{"1min":0.873,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.639,"5min":1.845,"15min":1.845},"last":1.2110000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.26},"min":{"1min":0.749,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.482,"5min":1.792,"15min":1.792},"last":1.339},{"interface":"front","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.1359999999999999},"min":{"1min":0.83699999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5009999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.29,"15min":1.214},"min":{"1min":0.94499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6739999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.4379999999999999,"15min":1.3979999999999999},"min":{"1min":1.179,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5900000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.292,"15min":1.2769999999999999},"min":{"1min":0.97999999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.4239999999999999,"15min":1.3560000000000001},"min":{"1min":1.335,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":1.895,"15min":1.895},"last":1.323}]},{"osd":7,"last update":"Fri Feb 2 13:29:36 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.4850000000000001,"15min":1.4570000000000001},"min":{"1min":1.1020000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.752,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":1.1359999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":557,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194754,"statfs":{"total":200000000,"available":199428842,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0289999999999999,"15min":1.0209999999999999},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.49},{"interface":"front","average":{"1min":1.05,"5min":1.0489999999999999,"15min":1.0409999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5289999999999999,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.2290000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.202,"15min":1.2769999999999999},"min":{"1min":0.82599999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.607,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1140000000000001,"15min":1.153},"min":{"1min":0.746,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5209999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.014}]},{"osd":2,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.2689999999999999,"15min":1.2250000000000001},"min":{"1min":0.92900000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8899999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.204,"15min":1.1799999999999999},"min":{"1min":0.94499999999999995,"5min":0.752,"15min":0.752},"max":{"1min":1.706,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.171}]},{"osd":3,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1919999999999999,"15min":1.1870000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.591,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.333},{"interface":"front","average":{"1min":1.268,"5min":1.2370000000000001,"15min":1.171},"min":{"1min":0.752,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.264}]},{"osd":4,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.194,"15min":1.222},"min":{"1min":0.86699999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2769999999999999,"15min":1.28},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8029999999999999,"5min":1.833,"15min":1.833},"last":1.421}]},{"osd":6,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.325,"15min":1.3},"min":{"1min":0.91600000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.456},{"interface":"front","average":{"1min":1.335,"5min":1.351,"15min":1.345},"min":{"1min":1.036,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7889999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.407}]},{"osd":7,"last update":"Fri Feb 2 13:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.4199999999999999,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4199999999999999,"15min":1.3720000000000001},"min":{"1min":1.161,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.3720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194757,"statfs":{"total":200000000,"available":199431447,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0960000000000001,"15min":0.99299999999999999},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.546,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.022},{"interface":"front","average":{"1min":1.157,"5min":1.107,"15min":1.0820000000000001},"min":{"1min":0.81100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.514,"5min":1.839,"15min":1.839},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":0.98699999999999999},"min":{"1min":0.71899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.292,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.097,"15min":1.0329999999999999},"min":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.177}]},{"osd":2,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1599999999999999,"15min":1.0900000000000001},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4930000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":1.071},{"interface":"front","average":{"1min":1.214,"5min":1.151,"15min":1.069},"min":{"1min":0.68000000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6459999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2529999999999999,"15min":1.175},"min":{"1min":0.83499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6000000000000001,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.3029999999999999,"15min":1.2370000000000001},"min":{"1min":0.91100000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.9670000000000001,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.3580000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.218,"15min":1.1379999999999999},"min":{"1min":0.63700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.958,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.3300000000000001,"15min":1.272},"min":{"1min":0.77900000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.004,"5min":2.069,"15min":2.069},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.3540000000000001,"15min":1.288},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2429999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.478},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.446,"15min":1.3879999999999999},"min":{"1min":0.75900000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1240000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:29:34 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4410000000000001,"15min":1.3779999999999999},"min":{"1min":0.878,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.889,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.629},{"interface":"front","average":{"1min":1.609,"5min":1.4750000000000001,"15min":1.444},"min":{"1min":0.74099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.181,"5min":2.698,"15min":2.698},"last":1.5469999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":557,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194754,"statfs":{"total":200000000,"available":199428864,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.70399999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.90000000000000002,"15min":0.88600000000000001},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4319999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4810000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0429999999999999,"15min":1.016},"min":{"1min":0.56599999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4419999999999999,"5min":2.125,"15min":2.125},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.028,"15min":0.95999999999999996},"min":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.484,"5min":1.611,"15min":1.611},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.008,"5min":0.92900000000000005,"15min":0.89600000000000002},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.4510000000000001,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.198,"15min":1.1830000000000001},"min":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.538,"5min":3.585,"15min":3.585},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.2070000000000001},"min":{"1min":0.68100000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.657,"5min":1.877,"15min":1.877},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.228,"15min":1.226},"min":{"1min":0.73699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.276,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":0.73399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.851,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.83799999999999997}]},{"osd":6,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.339,"15min":1.2649999999999999},"min":{"1min":0.72599999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.829,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3049999999999999,"15min":1.3069999999999999},"min":{"1min":0.77400000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.073}]},{"osd":7,"last update":"Fri Feb 2 13:29:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.282,"15min":1.2629999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.869,"5min":3.391,"15min":3.391},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.3160000000000001},"min":{"1min":0.753,"5min":0.625,"15min":0.625},"max":{"1min":1.9199999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.88800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":556,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194755,"statfs":{"total":200000000,"available":199429846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.6499999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.86199999999999999,"15min":0.78300000000000003},"min":{"1min":0.62,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.47,"5min":1.867,"15min":1.867},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.93899999999999995,"15min":0.90300000000000002},"min":{"1min":0.51700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.597,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.103,"5min":0.96999999999999997,"15min":0.86399999999999999},"min":{"1min":0.57599999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.6240000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.91500000000000004,"15min":0.79800000000000004},"min":{"1min":0.745,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.0529999999999999,"15min":0.98799999999999999},"min":{"1min":0.60099999999999998,"5min":0.432,"15min":0.432},"max":{"1min":2.0049999999999999,"5min":2.423,"15min":2.423},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.258,"15min":1.1830000000000001},"min":{"1min":0.87,"5min":0.748,"15min":0.748},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.353,"5min":1.2310000000000001,"15min":1.1599999999999999},"min":{"1min":1.048,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.202,"15min":1.0880000000000001},"min":{"1min":0.88500000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.806,"5min":2.246,"15min":2.246},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.92300000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.016}]},{"osd":6,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.2649999999999999,"15min":1.1799999999999999},"min":{"1min":0.81899999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.2689999999999999,"15min":1.202},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2330000000000001,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.3169999999999999,"15min":1.2090000000000001},"min":{"1min":0.80000000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.3149999999999999,"15min":1.2150000000000001},"min":{"1min":0.995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1589999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":555,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194757,"statfs":{"total":200000000,"available":199431253,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":1.0109999999999999},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.425,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.044,"15min":1.3759999999999999},"min":{"1min":0.51400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.296,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":1.2310000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88800000000000001,"15min":0.91600000000000004},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.639,"15min":1.639},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.006,"15min":1.0920000000000001},"min":{"1min":0.56499999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.823,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.101,"15min":1.2589999999999999},"min":{"1min":0.63200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.6200000000000001,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0880000000000001,"15min":1.288},"min":{"1min":0.501,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5109999999999999,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.117}]},{"osd":4,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1910000000000001,"15min":1.329},"min":{"1min":0.73699999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.837,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.351,"5min":1.284,"15min":1.3700000000000001},"min":{"1min":1.079,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.772,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.0249999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3360000000000001,"15min":1.462},"min":{"1min":0.94699999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.9970000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.327,"15min":1.4450000000000001},"min":{"1min":0.98299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7090000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.0629999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3660000000000001,"15min":1.579},"min":{"1min":0.97399999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.1539999999999999,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.46},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3720000000000001,"15min":1.502},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8280000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Feb 2 13:29:30 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.409,"15min":1.5609999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.871,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.181},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4450000000000001,"15min":1.627},"min":{"1min":0.878,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1019999999999999,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":1.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194180,"statfs":{"total":200000000,"available":198841045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.88300000000000001,"15min":1.1359999999999999},"min":{"1min":0.42999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.8680000000000001,"5min":4.8070000000000004,"15min":4.8070000000000004},"last":1.077},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0509999999999999,"15min":1.4950000000000001},"min":{"1min":0.45100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6870000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.995,"15min":1.173},"min":{"1min":0.53300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.0369999999999999,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.49},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.706,"5min":10.781000000000001,"15min":10.781000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.99099999999999999,"15min":1.24},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3819999999999999,"5min":5.8410000000000002,"15min":5.8410000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.117,"15min":1.321},"min":{"1min":0.51300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2170000000000001,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2170000000000001,"15min":1.347},"min":{"1min":0.753,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6759999999999999,"5min":3.669,"15min":3.669},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2989999999999999,"15min":1.4119999999999999},"min":{"1min":0.90000000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2719999999999998,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.2669999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.4159999999999999,"15min":1.8360000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9339999999999999,"5min":10.651,"15min":10.651},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.288,"15min":1.431},"min":{"1min":0.78600000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0009999999999999,"5min":3.7869999999999999,"15min":3.7869999999999999},"last":1.224}]},{"osd":6,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3480000000000001,"15min":1.478},"min":{"1min":0.88300000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.319,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.4319999999999999,"15min":1.8680000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.774,"5min":10.708,"15min":10.708},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:29:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4850000000000001,"15min":1.881},"min":{"1min":0.96899999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.8899999999999999,"5min":10.74,"15min":10.74},"last":1.466},{"interface":"front","average":{"1min":1.276,"5min":1.3300000000000001,"15min":1.452},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.764,"5min":3.863,"15min":3.863},"last":1.395}]}]}],"pool_statfs":[]}} 2024-02-02T13:30:23.549 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:30:23.550 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:30:23.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.807+0000 7fa0efa00640 1 Processor -- start 2024-02-02T13:30:23.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.807+0000 7fa0efa00640 1 -- start start 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0efa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0e80acfe0 0x7fa0e806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0efa00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e815d870 con 0x7fa0e80b0700 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0efa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e806bce0 con 0x7fa0e815d490 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0efa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e8161220 con 0x7fa0e80acfe0 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0eda00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa0e80b0700 0x7fa0e80b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46878/0 (socket says 172.21.15.91:46878) 2024-02-02T13:30:23.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.809+0000 7fa0eda00640 1 -- 172.21.15.91:0/506889189 learned_addr learned my addr 172.21.15.91:0/506889189 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:23.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0dfe00640 1 --2- 172.21.15.91:0/506889189 >> v2:172.21.15.113:3300/0 conn(0x7fa0e80acfe0 0x7fa0e806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250727478 0 0) 0x7fa0e815d870 con 0x7fa0e80b0700 2024-02-02T13:30:23.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c8003520 con 0x7fa0e80b0700 2024-02-02T13:30:23.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1553054338 0 0) 0x7fa0c8003520 con 0x7fa0e80b0700 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 >> v2:172.21.15.113:3300/0 conn(0x7fa0e80acfe0 msgr2=0x7fa0e806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 --2- 172.21.15.91:0/506889189 >> v2:172.21.15.113:3300/0 conn(0x7fa0e80acfe0 0x7fa0e806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 >> v1:172.21.15.113:6789/0 conn(0x7fa0e815d490 legacy=0x7fa0e80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80ad490 con 0x7fa0e80b0700 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa0e0001a10 con 0x7fa0e80b0700 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 >> v1:172.21.15.91:6789/0 conn(0x7fa0e80b0700 legacy=0x7fa0e80b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0c80047c0 0x7fa0c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8005180 0x7fa0c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8161220 con 0x7fa0c8003970 2024-02-02T13:30:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.810+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c800c020 con 0x7fa0c8005180 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c800c1e0 con 0x7fa0c80047c0 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0eda00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8005180 0x7fa0c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 -- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0c80047c0 msgr2=0x7fa0c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0c80047c0 0x7fa0c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 -- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8005180 msgr2=0x7fa0c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8005180 0x7fa0c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0ed000640 1 -- 172.21.15.91:0/506889189 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8003520 con 0x7fa0c8003970 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0eda00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8005180 0x7fa0c800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:23.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.811+0000 7fa0eda00640 1 -- 172.21.15.91:0/506889189 reap_dead start 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.812+0000 7fa0ed000640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8001750 tx=0x7fa0c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e295edfd3822fcb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80052d0 con 0x7fa0c8003970 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d802e050 con 0x7fa0c8003970 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0e7e00640 1 -- 172.21.15.91:0/506889189 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003830 con 0x7fa0c8003970 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 -- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 msgr2=0x7fa0c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8001750 tx=0x7fa0c80057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 -- 172.21.15.91:0/506889189 shutdown_connections 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 --2- 172.21.15.91:0/506889189 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 -- 172.21.15.91:0/506889189 >> 172.21.15.91:0/506889189 conn(0x7fa0e80cec80 msgr2=0x7fa0e80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:23.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.813+0000 7fa0efa00640 1 -- 172.21.15.91:0/506889189 shutdown_connections 2024-02-02T13:30:23.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.814+0000 7fa0efa00640 1 -- 172.21.15.91:0/506889189 wait complete. 2024-02-02T13:30:23.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.815+0000 7fa0efa00640 1 Processor -- start 2024-02-02T13:30:23.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.815+0000 7fa0efa00640 1 -- start start 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8003970 0x7fa0e80c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0e80b35c0 0x7fa0e8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c9c70 con 0x7fa0e80b3140 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c8b90 con 0x7fa0c8003970 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0efa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c6c10 con 0x7fa0e80b35c0 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33432/0 (socket says 172.21.15.91:33432) 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.816+0000 7fa0dfe00640 1 -- 172.21.15.91:0/3039700561 learned_addr learned my addr 172.21.15.91:0/3039700561 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 -- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0e80b35c0 msgr2=0x7fa0e8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0e80b35c0 0x7fa0e8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0ed000640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8003970 0x7fa0e80c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 -- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8003970 msgr2=0x7fa0e80c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8003970 0x7fa0e80c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 -- 172.21.15.91:0/3039700561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80c6e20 con 0x7fa0e80b3140 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0dfe00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000e860 tx=0x7fa0d000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.817+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0009600 con 0x7fa0e80b3140 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.818+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d000c940 con 0x7fa0e80b3140 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.818+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000ccd0 con 0x7fa0e80b3140 2024-02-02T13:30:23.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.818+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e80c7100 con 0x7fa0e80b3140 2024-02-02T13:30:23.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.818+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e80c76c0 con 0x7fa0e80b3140 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.819+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e80c78e0 con 0x7fa0e80b3140 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.820+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0d0013470 con 0x7fa0e80b3140 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.821+0000 7fa0e5600640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 0x7fa0c006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.821+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0d00889f0 con 0x7fa0e80b3140 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.821+0000 7fa0ed000640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 0x7fa0c006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:23.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.822+0000 7fa0ed000640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 0x7fa0c006c920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8033000 tx=0x7fa0d8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:23.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:23.825+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0d0054070 con 0x7fa0e80b3140 2024-02-02T13:30:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.081+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa0e8066cf0 con 0x7fa0e80b3140 2024-02-02T13:30:24.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.088+0000 7fa0e5600640 1 -- 172.21.15.91:0/3039700561 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa0d0057690 con 0x7fa0e80b3140 2024-02-02T13:30:24.090 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:30:24.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.090+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 msgr2=0x7fa0c006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.090+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 0x7fa0c006c920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8033000 tx=0x7fa0d8009030 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 msgr2=0x7fa0e8158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000e860 tx=0x7fa0d000ed20 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 shutdown_connections 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa0c006a460 0x7fa0c006c920 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0e80b35c0 0x7fa0e8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0e80b3140 0x7fa0e8158ee0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 --2- 172.21.15.91:0/3039700561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0c8003970 0x7fa0e80c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 >> 172.21.15.91:0/3039700561 conn(0x7fa0e80cec80 msgr2=0x7fa0e806f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:24.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 shutdown_connections 2024-02-02T13:30:24.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.091+0000 7fa0efa00640 1 -- 172.21.15.91:0/3039700561 wait complete. 2024-02-02T13:30:24.111 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:24.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.322+0000 7f8ada000640 1 Processor -- start 2024-02-02T13:30:24.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.322+0000 7f8ada000640 1 -- start start 2024-02-02T13:30:24.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ada000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ad40c9450 0x7f8ad40c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ada000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40ce3b0 con 0x7f8ad40add80 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ada000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40ce8e0 con 0x7f8ad415d3a0 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ada000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40adbd0 con 0x7f8ad40c9450 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad3e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8ad40add80 0x7f8ad40706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46880/0 (socket says 172.21.15.91:46880) 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad3e00640 1 -- 172.21.15.91:0/2292381893 learned_addr learned my addr 172.21.15.91:0/2292381893 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad9600640 1 --2- 172.21.15.91:0/2292381893 >> v2:172.21.15.91:3300/0 conn(0x7f8ad40c9450 0x7f8ad40c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad3400640 1 -- 172.21.15.91:0/2292381893 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897205129 0 0) 0x7f8ad40ce3b0 con 0x7f8ad40add80 2024-02-02T13:30:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad3400640 1 -- 172.21.15.91:0/2292381893 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab4003520 con 0x7f8ad40add80 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad9600640 1 -- 172.21.15.91:0/2292381893 >> v1:172.21.15.113:6789/0 conn(0x7f8ad415d3a0 legacy=0x7f8ad406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.325+0000 7f8ad9600640 1 -- 172.21.15.91:0/2292381893 >> v1:172.21.15.91:6789/0 conn(0x7f8ad40add80 legacy=0x7f8ad40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.326+0000 7f8ad9600640 1 -- 172.21.15.91:0/2292381893 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4070db0 con 0x7f8ad40c9450 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.327+0000 7f8ad9600640 1 --2- 172.21.15.91:0/2292381893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40c9450 0x7f8ad40c98e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009d90 tx=0x7f8acc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4ffb36a8ae3d24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.327+0000 7f8ad3400640 1 -- 172.21.15.91:0/2292381893 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc02cbd0 con 0x7f8ad40c9450 2024-02-02T13:30:24.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.327+0000 7f8ad3400640 1 -- 172.21.15.91:0/2292381893 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc02cd30 con 0x7f8ad40c9450 2024-02-02T13:30:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.327+0000 7f8ad3400640 1 -- 172.21.15.91:0/2292381893 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc02cea0 con 0x7f8ad40c9450 2024-02-02T13:30:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 -- 172.21.15.91:0/2292381893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40c9450 msgr2=0x7f8ad40c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 --2- 172.21.15.91:0/2292381893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40c9450 0x7f8ad40c98e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009d90 tx=0x7f8acc02be20 comp rx=0 tx=0).stop 2024-02-02T13:30:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 -- 172.21.15.91:0/2292381893 shutdown_connections 2024-02-02T13:30:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 --2- 172.21.15.91:0/2292381893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40c9450 0x7f8ad40c98e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 -- 172.21.15.91:0/2292381893 >> 172.21.15.91:0/2292381893 conn(0x7f8ad40cea90 msgr2=0x7f8ad40ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 -- 172.21.15.91:0/2292381893 shutdown_connections 2024-02-02T13:30:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.328+0000 7f8ada000640 1 -- 172.21.15.91:0/2292381893 wait complete. 2024-02-02T13:30:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.330+0000 7f8ada000640 1 Processor -- start 2024-02-02T13:30:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 -- start start 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 0x7f8ad40acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad415d3a0 0x7f8ad41f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40cd770 con 0x7f8ad40add80 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40cced0 con 0x7f8ad40c9450 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.331+0000 7f8ada000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40cd1b0 con 0x7f8ad415d3a0 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33440/0 (socket says 172.21.15.91:33440) 2024-02-02T13:30:24.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 -- 172.21.15.91:0/1213517098 learned_addr learned my addr 172.21.15.91:0/1213517098 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad3e00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 0x7f8ad40acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 -- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad415d3a0 msgr2=0x7f8ad41f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad415d3a0 0x7f8ad41f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 -- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 msgr2=0x7f8ad40acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 0x7f8ad40acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad8c00640 1 -- 172.21.15.91:0/1213517098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009a70 con 0x7f8ad40add80 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.332+0000 7f8ad3e00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 0x7f8ad40acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ad8c00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c400 tx=0x7f8ac400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40052d0 con 0x7f8ad40add80 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4005430 con 0x7f8ad40add80 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4003830 con 0x7f8ad40add80 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40cd490 con 0x7f8ad40add80 2024-02-02T13:30:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.333+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad40ccaf0 con 0x7f8ad40add80 2024-02-02T13:30:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.334+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad404dff0 con 0x7f8ad40add80 2024-02-02T13:30:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.335+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ac4003990 con 0x7f8ad40add80 2024-02-02T13:30:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.336+0000 7f8ad0c00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 0x7f8aac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.336+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ac4087ec0 con 0x7f8ad40add80 2024-02-02T13:30:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.336+0000 7f8ad3e00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 0x7f8aac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.337+0000 7f8ad3e00640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 0x7f8aac06c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406b310 tx=0x7f8abc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:24.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.340+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ac40534a0 con 0x7f8ad40add80 2024-02-02T13:30:24.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.566+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ad40b4a10 con 0x7f8ad40add80 2024-02-02T13:30:24.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.567+0000 7f8ad0c00640 1 -- 172.21.15.91:0/1213517098 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f8ac4056ac0 con 0x7f8ad40add80 2024-02-02T13:30:24.569 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:24.569 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:30:24.570 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:30:24.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 msgr2=0x7f8aac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 0x7f8aac06c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406b310 tx=0x7f8abc009040 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 msgr2=0x7f8ad40ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c400 tx=0x7f8ac400c8c0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 shutdown_connections 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8aac06a430 0x7f8aac06c8f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad415d3a0 0x7f8ad41f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad40c9450 0x7f8ad40acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 --2- 172.21.15.91:0/1213517098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ad40add80 0x7f8ad40ae290 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:24.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 >> 172.21.15.91:0/1213517098 conn(0x7f8ad40cea90 msgr2=0x7f8ad4070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:24.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.570+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 shutdown_connections 2024-02-02T13:30:24.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:24.571+0000 7f8ada000640 1 -- 172.21.15.91:0/1213517098 wait complete. 2024-02-02T13:30:24.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:30:18.141444+0000', 'modified': '2024-02-02T13:30:21.674966+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6705}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6705': {'gid': 6705, 'name': 'd', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6835/2746977101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6833', 'nonce': 2746977101}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 2746977101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:30:24.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:30:24.994 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-02-02T13:30:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.213+0000 7fe4a2c00640 1 Processor -- start 2024-02-02T13:30:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.214+0000 7fe4a2c00640 1 -- start start 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe4a2c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe49c15e340 0x7fe49c164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe4a2c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c0ca0c0 con 0x7fe49c159de0 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe4a2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c0ca380 con 0x7fe49c0cbc90 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe4a2c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c070ca0 con 0x7fe49c15e340 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe49be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe49c159de0 0x7fe49c0cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46888/0 (socket says 172.21.15.91:46888) 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe49be00640 1 -- 172.21.15.91:0/3034578805 learned_addr learned my addr 172.21.15.91:0/3034578805 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:25.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.215+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/3034578805 >> v2:172.21.15.91:3300/0 conn(0x7fe49c15e340 0x7fe49c164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.216+0000 7fe49aa00640 1 -- 172.21.15.91:0/3034578805 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883576250 0 0) 0x7fe49c0ca0c0 con 0x7fe49c159de0 2024-02-02T13:30:25.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.216+0000 7fe49aa00640 1 -- 172.21.15.91:0/3034578805 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe47c005730 con 0x7fe49c159de0 2024-02-02T13:30:25.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.216+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3034578805 >> v1:172.21.15.113:6789/0 conn(0x7fe49c0cbc90 legacy=0x7fe49c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.216+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3034578805 >> v1:172.21.15.91:6789/0 conn(0x7fe49c159de0 legacy=0x7fe49c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.216+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3034578805 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0ca660 con 0x7fe49c15e340 2024-02-02T13:30:25.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.217+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/3034578805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c15e340 0x7fe49c164d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe49402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7521a04c10f81cd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.217+0000 7fe49aa00640 1 -- 172.21.15.91:0/3034578805 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494031830 con 0x7fe49c15e340 2024-02-02T13:30:25.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.217+0000 7fe49aa00640 1 -- 172.21.15.91:0/3034578805 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494031990 con 0x7fe49c15e340 2024-02-02T13:30:25.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.217+0000 7fe49aa00640 1 -- 172.21.15.91:0/3034578805 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494031d20 con 0x7fe49c15e340 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 -- 172.21.15.91:0/3034578805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c15e340 msgr2=0x7fe49c164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/3034578805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c15e340 0x7fe49c164d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe49402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 -- 172.21.15.91:0/3034578805 shutdown_connections 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/3034578805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c15e340 0x7fe49c164d90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 -- 172.21.15.91:0/3034578805 >> 172.21.15.91:0/3034578805 conn(0x7fe49c0b0700 msgr2=0x7fe49c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 -- 172.21.15.91:0/3034578805 shutdown_connections 2024-02-02T13:30:25.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.218+0000 7fe4a2c00640 1 -- 172.21.15.91:0/3034578805 wait complete. 2024-02-02T13:30:25.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.220+0000 7fe4a2c00640 1 Processor -- start 2024-02-02T13:30:25.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.220+0000 7fe4a2c00640 1 -- start start 2024-02-02T13:30:25.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe49c159de0 0x7fe49c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe49c15e340 0x7fe49c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0b05c0 con 0x7fe49c0cbc90 2024-02-02T13:30:25.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0adc30 con 0x7fe49c15e340 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe4a2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c15b6e0 con 0x7fe49c159de0 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe49be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe49be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33458/0 (socket says 172.21.15.91:33458) 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.221+0000 7fe49be00640 1 -- 172.21.15.91:0/883534785 learned_addr learned my addr 172.21.15.91:0/883534785 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 -- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe49c159de0 msgr2=0x7fe49c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe49c159de0 0x7fe49c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 -- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe49c15e340 msgr2=0x7fe49c14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe49c15e340 0x7fe49c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 -- 172.21.15.91:0/883534785 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009400 con 0x7fe49c0cbc90 2024-02-02T13:30:25.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe49be00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c500 tx=0x7fe48c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe493e00640 1 -- 172.21.15.91:0/883534785 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c005ce0 con 0x7fe49c0cbc90 2024-02-02T13:30:25.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe493e00640 1 -- 172.21.15.91:0/883534785 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c005e40 con 0x7fe49c0cbc90 2024-02-02T13:30:25.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe493e00640 1 -- 172.21.15.91:0/883534785 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c007330 con 0x7fe49c0cbc90 2024-02-02T13:30:25.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.222+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c15b9c0 con 0x7fe49c0cbc90 2024-02-02T13:30:25.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.223+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c15bf80 con 0x7fe49c0cbc90 2024-02-02T13:30:25.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.225+0000 7fe493e00640 1 -- 172.21.15.91:0/883534785 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe48c007550 con 0x7fe49c0cbc90 2024-02-02T13:30:25.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.225+0000 7fe493e00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 0x7fe47406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.226+0000 7fe493e00640 1 -- 172.21.15.91:0/883534785 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe48c087ff0 con 0x7fe49c0cbc90 2024-02-02T13:30:25.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.226+0000 7fe49b400640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 0x7fe47406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.226+0000 7fe4a55fd740 1 Processor -- start 2024-02-02T13:30:25.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.226+0000 7fe49b400640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 0x7fe47406c8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0cbc60 tx=0x7fe484009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 -- start start 2024-02-02T13:30:25.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c960f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95deed0 0x55b3c960c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c95ba650 con 0x55b3c95e2ad0 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c954f000 con 0x55b3c95deed0 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.227+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c954c790 con 0x55b3c960ef50 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33470/0 (socket says 172.21.15.91:33470) 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 -- 172.21.15.91:0/623447267 learned_addr learned my addr 172.21.15.91:0/623447267 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe49be00640 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c960f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 -- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 msgr2=0x55b3c960f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c960f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 -- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95deed0 msgr2=0x55b3c960c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe49b400640 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95deed0 0x55b3c960c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95deed0 0x55b3c960c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 -- 172.21.15.91:0/623447267 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c950f710 con 0x55b3c95e2ad0 2024-02-02T13:30:25.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.228+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe4940098c0 tx=0x7fe49402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b2595f694c08ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe490c00640 1 -- 172.21.15.91:0/623447267 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940316b0 con 0x55b3c95e2ad0 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe490c00640 1 -- 172.21.15.91:0/623447267 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494031b00 con 0x55b3c95e2ad0 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe490c00640 1 -- 172.21.15.91:0/623447267 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494031830 con 0x55b3c95e2ad0 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 -- 172.21.15.91:0/623447267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 msgr2=0x55b3c960c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe4940098c0 tx=0x7fe49402e000 comp rx=0 tx=0).stop 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 -- 172.21.15.91:0/623447267 shutdown_connections 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c960c700 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95deed0 0x55b3c960c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/623447267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c960f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 -- 172.21.15.91:0/623447267 >> 172.21.15.91:0/623447267 conn(0x55b3c95e11e0 msgr2=0x55b3c9629d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.229+0000 7fe4a55fd740 1 -- 172.21.15.91:0/623447267 shutdown_connections 2024-02-02T13:30:25.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.230+0000 7fe4a55fd740 1 -- 172.21.15.91:0/623447267 wait complete. 2024-02-02T13:30:25.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.230+0000 7fe4a55fd740 1 -- start start 2024-02-02T13:30:25.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.230+0000 7fe4a55fd740 1 Processor -- start 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95e2ad0 0x55b3c9613690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c95fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9548080 con 0x55b3c963cac0 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94e9850 con 0x55b3c95e2ad0 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94e94f0 con 0x55b3c960ef50 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.231+0000 7fe4a55fd740 10 client.0 _ll_register_callbacks cb 0x55b3c95419c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33474/0 (socket says 172.21.15.91:33474) 2024-02-02T13:30:25.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3311285058 learned_addr learned my addr 172.21.15.91:0/3311285058 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 msgr2=0x55b3c95fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c95fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95e2ad0 msgr2=0x55b3c9613690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95e2ad0 0x55b3c9613690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 -- 172.21.15.91:0/3311285058 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0ca660 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6b3ac96a15e4aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.232+0000 7fe489600640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494003610 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c95725d0 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe494003770 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe494003930 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 10 client.6849 fetch_fsmap learned FSMap version 48 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b3c9546700 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3c95725d0 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe494003a90 con 0x55b3c963cac0 2024-02-02T13:30:25.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 10 client.6849 fetch_fsmap finished waiting for FSMap version 48 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 10 client.6849 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 20 client.6849 populate_metadata read hostname 'smithi091' 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c960f890 0x55b3c960fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 4 client.6849 mds_command: new command op to 6705 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.233+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 --> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c953a8d0 con 0x55b3c960f890 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.234+0000 7fe49be00640 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c960f890 0x55b3c960fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.235+0000 7fe49be00640 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c960f890 0x55b3c960fc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7cc7c4824ebce7 server_cookie=7be012c4f8620341 in_seq=0 out_seq=0 2024-02-02T13:30:25.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.235+0000 7fe489600640 10 client.6849 ms_handle_connect on v2:172.21.15.113:6833/2746977101 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe489600640 1 -- 172.21.15.91:0/3311285058 <== mds.0 v2:172.21.15.113:6833/2746977101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b3c953a8d0 con 0x55b3c960f890 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe489600640 10 client.6849 handle_command_reply: tid=0 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 client.6849 shutdown 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 msgr2=0x55b3c96120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe494004730 comp rx=0 tx=0).stop 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 shutdown_connections 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c960f890 0x55b3c960fc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c95fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c963cac0 0x55b3c96120f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe489600640 0 client.6849 ms_handle_reset on v2:172.21.15.113:6833/2746977101 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/3311285058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c95e2ad0 0x55b3c9613690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 >> 172.21.15.91:0/3311285058 conn(0x55b3c95deed0 msgr2=0x55b3c9612ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 shutdown_connections 2024-02-02T13:30:25.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 1 -- 172.21.15.91:0/3311285058 wait complete. 2024-02-02T13:30:25.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.237+0000 7fe4a55fd740 20 client.6849 trim_cache size 0 max 16384 2024-02-02T13:30:25.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.253+0000 7fe4a55fd740 1 Processor -- start 2024-02-02T13:30:25.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.253+0000 7fe4a55fd740 1 -- start start 2024-02-02T13:30:25.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c95e2ad0 0x55b3c95dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c960c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c92aff20 con 0x55b3c960ef50 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c95dea40 con 0x55b3c963cac0 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.254+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c960cab0 con 0x55b3c95e2ad0 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33482/0 (socket says 172.21.15.91:33482) 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 -- 172.21.15.91:0/4000489995 learned_addr learned my addr 172.21.15.91:0/4000489995 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c960c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49be00640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c95e2ad0 0x55b3c95dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 -- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c95e2ad0 msgr2=0x55b3c95dfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c95e2ad0 0x55b3c95dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 -- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 msgr2=0x55b3c960c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c960c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 -- 172.21.15.91:0/4000489995 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c957c280 con 0x55b3c960ef50 2024-02-02T13:30:25.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c960c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:25.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.255+0000 7fe49b400640 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe484018610 tx=0x7fe48400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58095bf43cc41314 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe491600640 1 -- 172.21.15.91:0/4000489995 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484003b70 con 0x55b3c960ef50 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe491600640 1 -- 172.21.15.91:0/4000489995 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484003cd0 con 0x55b3c960ef50 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe491600640 1 -- 172.21.15.91:0/4000489995 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484009990 con 0x55b3c960ef50 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 -- 172.21.15.91:0/4000489995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 msgr2=0x55b3c9616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe484018610 tx=0x7fe48400e040 comp rx=0 tx=0).stop 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 -- 172.21.15.91:0/4000489995 shutdown_connections 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c960c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c960ef50 0x55b3c9616580 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/4000489995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c95e2ad0 0x55b3c95dfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 -- 172.21.15.91:0/4000489995 >> 172.21.15.91:0/4000489995 conn(0x55b3c95deed0 msgr2=0x55b3c9613690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 -- 172.21.15.91:0/4000489995 shutdown_connections 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.256+0000 7fe4a55fd740 1 -- 172.21.15.91:0/4000489995 wait complete. 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.257+0000 7fe4a55fd740 1 -- start start 2024-02-02T13:30:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.257+0000 7fe4a55fd740 1 Processor -- start 2024-02-02T13:30:25.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c9629a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c962d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9608470 con 0x55b3c95e2ad0 2024-02-02T13:30:25.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9629570 con 0x55b3c963cac0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9607fa0 con 0x55b3c960ef50 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe4a55fd740 10 client.0 _ll_register_callbacks cb 0x55b3c9612a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe49be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe49be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33490/0 (socket says 172.21.15.91:33490) 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.258+0000 7fe49be00640 1 -- 172.21.15.91:0/2884426810 learned_addr learned my addr 172.21.15.91:0/2884426810 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49b400640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c9629a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 -- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 msgr2=0x55b3c9629a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c9629a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe4a0c00640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c962d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 -- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 msgr2=0x55b3c962d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c962d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 -- 172.21.15.91:0/2884426810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe484012070 con 0x55b3c95e2ad0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe49be00640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe48c001eb0 tx=0x7fe48c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2dab6703ebac84 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe489600640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c055710 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe48c053a80 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.259+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe494031b00 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe48c053c40 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 10 client.6855 fetch_fsmap learned FSMap version 48 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b3c94e8a60 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c079080 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe48c0575d0 con 0x55b3c95e2ad0 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 10 client.6855 fetch_fsmap finished waiting for FSMap version 48 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 10 client.6855 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 20 client.6855 populate_metadata read hostname 'smithi091' 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c963d470 0x55b3c962d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 4 client.6855 mds_command: new command op to 6705 tid=0[{"prefix": "status"}] 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.260+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 --> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b3c9541100 con 0x55b3c963d470 2024-02-02T13:30:25.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.261+0000 7fe49b400640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c963d470 0x55b3c962d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.261+0000 7fe49b400640 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c963d470 0x55b3c962d7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59736b2c6b3622b7 server_cookie=a7c444ce6df44878 in_seq=0 out_seq=0 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.261+0000 7fe489600640 10 client.6855 ms_handle_connect on v2:172.21.15.113:6833/2746977101 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe489600640 1 -- 172.21.15.91:0/2884426810 <== mds.0 v2:172.21.15.113:6833/2746977101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b3c9541100 con 0x55b3c963d470 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe489600640 10 client.6855 handle_command_reply: tid=0 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 client.6855 shutdown 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 msgr2=0x55b3c9607a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe48c001eb0 tx=0x7fe48c007f40 comp rx=0 tx=0).stop 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 shutdown_connections 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:6833/2746977101,v1:172.21.15.113:6835/2746977101] conn(0x55b3c963d470 0x55b3c962d7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b3c960ef50 0x55b3c9629a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b3c963cac0 0x55b3c962d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe489600640 0 client.6855 ms_handle_reset on v2:172.21.15.113:6833/2746977101 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 --2- 172.21.15.91:0/2884426810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55b3c95e2ad0 0x55b3c9607a60 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.262+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 >> 172.21.15.91:0/2884426810 conn(0x55b3c95deed0 msgr2=0x55b3c95df940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 shutdown_connections 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a55fd740 1 -- 172.21.15.91:0/2884426810 wait complete. 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a55fd740 20 client.6855 trim_cache size 0 max 16384 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 6705, 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:30:25.267 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 7.1163384350000003, 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 48, 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 105, 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 105, 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 12.829362689 2024-02-02T13:30:25.268 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 msgr2=0x7fe47406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 0x7fe47406c8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0cbc60 tx=0x7fe484009230 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 msgr2=0x7fe49c159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c500 tx=0x7fe48c00c9c0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 shutdown_connections 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe47406a430 0x7fe47406c8f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe49c159de0 0x7fe49c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe49c15e340 0x7fe49c14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 --2- 172.21.15.91:0/883534785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe49c0cbc90 0x7fe49c159270 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.263+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 >> 172.21.15.91:0/883534785 conn(0x7fe49c0b0700 msgr2=0x7fe49c1561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.264+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 shutdown_connections 2024-02-02T13:30:25.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.264+0000 7fe4a2c00640 1 -- 172.21.15.91:0/883534785 wait complete. 2024-02-02T13:30:25.284 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:30:25.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.492+0000 7f7437200640 1 Processor -- start 2024-02-02T13:30:25.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.493+0000 7f7437200640 1 -- start start 2024-02-02T13:30:25.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.494+0000 7f7437200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f743815d3a0 0x7f743806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.494+0000 7f7437200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 0x7f74380706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.494+0000 7f7437200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380ce3b0 con 0x7f74380c9450 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.494+0000 7f7437200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380adbd0 con 0x7f743815d3a0 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.494+0000 7f7437200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380c59d0 con 0x7f74380add80 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f743815d3a0 0x7f743806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f742fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 0x7f74380706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f743815d3a0 0x7f743806b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:38500/0 (socket says 172.21.15.91:38500) 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 -- 172.21.15.91:0/1043034136 learned_addr learned my addr 172.21.15.91:0/1043034136 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 -- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 msgr2=0x7f74380706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 0x7f74380706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.495+0000 7f7435e00640 1 -- 172.21.15.91:0/1043034136 >> v1:172.21.15.113:6789/0 conn(0x7f74380c9450 legacy=0x7f74380c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.496+0000 7f7435e00640 1 -- 172.21.15.91:0/1043034136 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380ce8e0 con 0x7f743815d3a0 2024-02-02T13:30:25.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.496+0000 7f742fe00640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 0x7f74380706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:30:25.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.497+0000 7f7435e00640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743815d3a0 0x7f743806b450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7428009870 tx=0x7f742802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a88b725a1540f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.497+0000 7f7435400640 1 -- 172.21.15.91:0/1043034136 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280317c0 con 0x7f743815d3a0 2024-02-02T13:30:25.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.497+0000 7f7435400640 1 -- 172.21.15.91:0/1043034136 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428031920 con 0x7f743815d3a0 2024-02-02T13:30:25.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7435400640 1 -- 172.21.15.91:0/1043034136 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428031cb0 con 0x7f743815d3a0 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 -- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743815d3a0 msgr2=0x7f743806b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743815d3a0 0x7f743806b450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7428009870 tx=0x7f742802b860 comp rx=0 tx=0).stop 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 -- 172.21.15.91:0/1043034136 shutdown_connections 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74380add80 0x7f74380706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 --2- 172.21.15.91:0/1043034136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743815d3a0 0x7f743806b450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 -- 172.21.15.91:0/1043034136 >> 172.21.15.91:0/1043034136 conn(0x7f74380cea90 msgr2=0x7f74380ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.498+0000 7f7437200640 1 -- 172.21.15.91:0/1043034136 shutdown_connections 2024-02-02T13:30:25.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.499+0000 7f7437200640 1 -- 172.21.15.91:0/1043034136 wait complete. 2024-02-02T13:30:25.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.500+0000 7f7437200640 1 Processor -- start 2024-02-02T13:30:25.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.500+0000 7f7437200640 1 -- start start 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74380add80 0x7f7438151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 0x7f74381f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381f3be0 con 0x7f74380c9450 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381f7150 con 0x7f74380add80 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.501+0000 7f7437200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ae4a0 con 0x7f743815d3a0 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33502/0 (socket says 172.21.15.91:33502) 2024-02-02T13:30:25.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 -- 172.21.15.91:0/2357452419 learned_addr learned my addr 172.21.15.91:0/2357452419 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f7436800640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 0x7f74381f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f7435e00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74380add80 0x7f7438151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 -- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 msgr2=0x7f74381f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 0x7f74381f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 -- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74380add80 msgr2=0x7f7438151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74380add80 0x7f7438151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 -- 172.21.15.91:0/2357452419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009400 con 0x7f74380c9450 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f7436800640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 0x7f74381f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.502+0000 7f742fe00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f742000e9e0 tx=0x7f742000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.503+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000ca30 con 0x7f74380c9450 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.503+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000cb90 con 0x7f74380c9450 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.503+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420013470 con 0x7f74380c9450 2024-02-02T13:30:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.503+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280094d0 con 0x7f74380c9450 2024-02-02T13:30:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.503+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74381f6b90 con 0x7f74380c9450 2024-02-02T13:30:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.504+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74380ae270 con 0x7f74380c9450 2024-02-02T13:30:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.505+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f74200135d0 con 0x7f74380c9450 2024-02-02T13:30:25.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.506+0000 7f742e000640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 0x7f741006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:25.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.506+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7420088850 con 0x7f74380c9450 2024-02-02T13:30:25.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.506+0000 7f7435e00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 0x7f741006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:25.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.507+0000 7f7435e00640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 0x7f741006c8f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7428036000 tx=0x7f742803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:25.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.510+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f74380ae270 con 0x7f74380c9450 2024-02-02T13:30:25.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.740+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f74380ae270 con 0x7f74380c9450 2024-02-02T13:30:25.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.741+0000 7f742e000640 1 -- 172.21.15.91:0/2357452419 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7420017060 con 0x7f74380c9450 2024-02-02T13:30:25.743 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 msgr2=0x7f741006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 0x7f741006c8f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7428036000 tx=0x7f742803d040 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 msgr2=0x7f74381f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f742000e9e0 tx=0x7f742000eea0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 shutdown_connections 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f741006a430 0x7f741006c8f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f743815d3a0 0x7f74381f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74380c9450 0x7f74381f61b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 --2- 172.21.15.91:0/2357452419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74380add80 0x7f7438151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 >> 172.21.15.91:0/2357452419 conn(0x7f74380cea90 msgr2=0x7f74380c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 shutdown_connections 2024-02-02T13:30:25.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:25.744+0000 7f7437200640 1 -- 172.21.15.91:0/2357452419 wait complete. 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:30:25.765 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:30:25.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:25.766 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:30:25.814 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:30:25.815 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:30:25.816 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:25.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:25.818 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:30:25.897 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:30:25.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:25.902 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:30:25.955 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:30:25.955 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:30:25.955 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:30:25.955 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:30:25.955 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:30:25.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:30:25.956 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:30:26.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:26.141 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:30:26.190 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:30:26.190 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:30:26.191 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:30:26.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:26.191 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:30:26.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:30:26.277 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:30:26.330 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:30:26.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:30:26.330 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:30:26.378 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:30:26.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:26.378 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:26.505 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:26.506 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:30:26.538 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:26.538 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:30:26.538 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:30:26.538 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:30:26.538 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:30:26.541 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:30:26.541 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:30:26.541 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:30:26.541 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:30:26.541 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:30:26.544 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:30:26.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:26.545 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:30:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:26.620 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:30:26.685 INFO:teuthology.orchestra.run.smithi091.stdout:client6861 2024-02-02T13:30:26.685 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:26.686 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:26.739 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:30:26.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.947+0000 7fb817c00640 1 Processor -- start 2024-02-02T13:30:26.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.947+0000 7fb817c00640 1 -- start start 2024-02-02T13:30:26.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb817c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100cbec0 0x7fb8100cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb817c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb8100c9be0 0x7fb8100ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb817c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8100cc860 con 0x7fb81015e6d0 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb817c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8100cbca0 con 0x7fb8100c9be0 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb817c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81015ed50 con 0x7fb8100cbec0 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb815c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb8100c9be0 0x7fb8100ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb815200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb81015e6d0 0x7fb8100cb590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46900/0 (socket says 172.21.15.91:46900) 2024-02-02T13:30:26.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.949+0000 7fb815200640 1 -- 172.21.15.91:0/3914960764 learned_addr learned my addr 172.21.15.91:0/3914960764 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb807e00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100cbec0 0x7fb8100cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb80fe00640 1 -- 172.21.15.91:0/3914960764 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92943701 0 0) 0x7fb8100cc860 con 0x7fb81015e6d0 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb815c00640 1 -- 172.21.15.91:0/3914960764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100cbec0 msgr2=0x7fb8100cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb815c00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100cbec0 0x7fb8100cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb815c00640 1 -- 172.21.15.91:0/3914960764 >> v1:172.21.15.91:6789/0 conn(0x7fb81015e6d0 legacy=0x7fb8100cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.950+0000 7fb815c00640 1 -- 172.21.15.91:0/3914960764 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81015f030 con 0x7fb8100c9be0 2024-02-02T13:30:26.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.952+0000 7fb815c00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8100c9be0 0x7fb8100ca070 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb808009d90 tx=0x7fb80802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d766d617e38ec9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:26.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.952+0000 7fb80fe00640 1 -- 172.21.15.91:0/3914960764 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80802cb30 con 0x7fb8100c9be0 2024-02-02T13:30:26.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.952+0000 7fb80fe00640 1 -- 172.21.15.91:0/3914960764 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80802cc90 con 0x7fb8100c9be0 2024-02-02T13:30:26.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.952+0000 7fb80fe00640 1 -- 172.21.15.91:0/3914960764 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808034440 con 0x7fb8100c9be0 2024-02-02T13:30:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 -- 172.21.15.91:0/3914960764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8100c9be0 msgr2=0x7fb8100ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8100c9be0 0x7fb8100ca070 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb808009d90 tx=0x7fb80802bd80 comp rx=0 tx=0).stop 2024-02-02T13:30:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 -- 172.21.15.91:0/3914960764 shutdown_connections 2024-02-02T13:30:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8100c9be0 0x7fb8100ca070 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:26.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 --2- 172.21.15.91:0/3914960764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100cbec0 0x7fb8100cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:26.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 -- 172.21.15.91:0/3914960764 >> 172.21.15.91:0/3914960764 conn(0x7fb8100b0fc0 msgr2=0x7fb8100b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:26.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 -- 172.21.15.91:0/3914960764 shutdown_connections 2024-02-02T13:30:26.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.953+0000 7fb817c00640 1 -- 172.21.15.91:0/3914960764 wait complete. 2024-02-02T13:30:26.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.955+0000 7fb817c00640 1 Processor -- start 2024-02-02T13:30:26.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.955+0000 7fb817c00640 1 -- start start 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100c9be0 0x7fb8100ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8100cbec0 0x7fb8100c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100c7fb0 con 0x7fb81015e6d0 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ae570 con 0x7fb8100cbec0 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb817c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100c97d0 con 0x7fb8100c9be0 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb815c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb815c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33526/0 (socket says 172.21.15.91:33526) 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb815c00640 1 -- 172.21.15.91:0/2997607085 learned_addr learned my addr 172.21.15.91:0/2997607085 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:26.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb807e00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8100cbec0 0x7fb8100c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.956+0000 7fb815200640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100c9be0 0x7fb8100ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 -- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100c9be0 msgr2=0x7fb8100ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100c9be0 0x7fb8100ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 -- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8100cbec0 msgr2=0x7fb8100c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8100cbec0 0x7fb8100c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 -- 172.21.15.91:0/2997607085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808009a70 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb815c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb808009d90 tx=0x7fb808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80803e410 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80803e590 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80803e940 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8101f37c0 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.957+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8100ac8d0 con 0x7fb81015e6d0 2024-02-02T13:30:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.958+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb810067400 con 0x7fb81015e6d0 2024-02-02T13:30:26.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.960+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb80803b050 con 0x7fb81015e6d0 2024-02-02T13:30:26.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.961+0000 7fb80d600640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 0x7fb7e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:26.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.961+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb8080ab130 con 0x7fb81015e6d0 2024-02-02T13:30:26.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.961+0000 7fb815200640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 0x7fb7e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:26.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.962+0000 7fb815200640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 0x7fb7e806c8f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f60 tx=0x7fb800007b70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:26.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:26.966+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb808076820 con 0x7fb81015e6d0 2024-02-02T13:30:27.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.203+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb8100afbe0 con 0x7fb81015e6d0 2024-02-02T13:30:27.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.204+0000 7fb80d600640 1 -- 172.21.15.91:0/2997607085 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb808039060 con 0x7fb81015e6d0 2024-02-02T13:30:27.205 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:30:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 msgr2=0x7fb7e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 0x7fb7e806c8f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f60 tx=0x7fb800007b70 comp rx=0 tx=0).stop 2024-02-02T13:30:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 msgr2=0x7fb8100b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb808009d90 tx=0x7fb808004a40 comp rx=0 tx=0).stop 2024-02-02T13:30:27.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 shutdown_connections 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7e806a430 0x7fb7e806c8f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb81015e6d0 0x7fb8100b40e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8100cbec0 0x7fb8100c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 --2- 172.21.15.91:0/2997607085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8100c9be0 0x7fb8100ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.206+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 >> 172.21.15.91:0/2997607085 conn(0x7fb8100b0fc0 msgr2=0x7fb810071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.207+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 shutdown_connections 2024-02-02T13:30:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:27.207+0000 7fb817c00640 1 -- 172.21.15.91:0/2997607085 wait complete. 2024-02-02T13:30:27.226 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:30:27.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:30:27.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:30:27.226 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:30:27.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:27.227 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:30:27.227 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:30:27.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:30:27.227 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:30:27.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.227 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:30:27.284 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:30:27.284 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:30:27.284 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:30:27.284 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.284 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42339sec preferred_lft 42339sec 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:30:27.285 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:30:27.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:30:27.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.288 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:30:27.378 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:30:27.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.382 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:30:27.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:30:27.434 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:30:27.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.612 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:30:27.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:30:27.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:30:27.661 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:30:27.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:30:27.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:30:27.756 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:27.808 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:30:27.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:30:27.809 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:27.859 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:30:27.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:27.859 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:27.987 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:27.987 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:30:28.018 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:30:28.019 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:30:28.019 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:30:28.019 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:30:28.019 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:30:28.022 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:30:28.022 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:30:28.022 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:30:28.022 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:30:28.022 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:30:28.025 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:30:28.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:28.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:28.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:28.103 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:30:28.155 INFO:teuthology.orchestra.run.smithi113.stdout:client5464 2024-02-02T13:30:28.155 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:28.155 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:28.208 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:28.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.411+0000 7fc914400640 1 Processor -- start 2024-02-02T13:30:28.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.411+0000 7fc914400640 1 -- start start 2024-02-02T13:30:28.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc914400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc90c0ce130 0x7fc90c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:28.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc914400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0cc7c0 con 0x7fc90c15a010 2024-02-02T13:30:28.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc914400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0ce930 con 0x7fc90c0b0700 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc914400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c0cd640 con 0x7fc90c0ce130 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc911a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc90c15a010 0x7fc90c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46910/0 (socket says 172.21.15.91:46910) 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc911a00640 1 -- 172.21.15.91:0/413926717 learned_addr learned my addr 172.21.15.91:0/413926717 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc911000640 1 --2- 172.21.15.91:0/413926717 >> v2:172.21.15.91:3300/0 conn(0x7fc90c0ce130 0x7fc90c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc90be00640 1 -- 172.21.15.91:0/413926717 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660112270 0 0) 0x7fc90c0cc7c0 con 0x7fc90c15a010 2024-02-02T13:30:28.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.413+0000 7fc90be00640 1 -- 172.21.15.91:0/413926717 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ec003520 con 0x7fc90c15a010 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.414+0000 7fc911000640 1 -- 172.21.15.91:0/413926717 >> v1:172.21.15.113:6789/0 conn(0x7fc90c0b0700 legacy=0x7fc90c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.414+0000 7fc911000640 1 -- 172.21.15.91:0/413926717 >> v1:172.21.15.91:6789/0 conn(0x7fc90c15a010 legacy=0x7fc90c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.414+0000 7fc911000640 1 -- 172.21.15.91:0/413926717 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c06c890 con 0x7fc90c0ce130 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.415+0000 7fc911000640 1 --2- 172.21.15.91:0/413926717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0ce130 0x7fc90c0c9670 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40097a0 tx=0x7fc8f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c71e0f1bf1b9de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.415+0000 7fc90be00640 1 -- 172.21.15.91:0/413926717 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4030610 con 0x7fc90c0ce130 2024-02-02T13:30:28.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.415+0000 7fc90be00640 1 -- 172.21.15.91:0/413926717 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4030770 con 0x7fc90c0ce130 2024-02-02T13:30:28.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.415+0000 7fc90be00640 1 -- 172.21.15.91:0/413926717 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4030b00 con 0x7fc90c0ce130 2024-02-02T13:30:28.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 -- 172.21.15.91:0/413926717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0ce130 msgr2=0x7fc90c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 --2- 172.21.15.91:0/413926717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0ce130 0x7fc90c0c9670 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40097a0 tx=0x7fc8f402b960 comp rx=0 tx=0).stop 2024-02-02T13:30:28.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 -- 172.21.15.91:0/413926717 shutdown_connections 2024-02-02T13:30:28.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 --2- 172.21.15.91:0/413926717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0ce130 0x7fc90c0c9670 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 -- 172.21.15.91:0/413926717 >> 172.21.15.91:0/413926717 conn(0x7fc90c0cec80 msgr2=0x7fc90c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:28.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 -- 172.21.15.91:0/413926717 shutdown_connections 2024-02-02T13:30:28.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.416+0000 7fc914400640 1 -- 172.21.15.91:0/413926717 wait complete. 2024-02-02T13:30:28.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.417+0000 7fc914400640 1 Processor -- start 2024-02-02T13:30:28.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 -- start start 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc90c0ce130 0x7fc90c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc90c15a010 0x7fc90c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c070d00 con 0x7fc90c0b0700 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1f3b70 con 0x7fc90c0ce130 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.418+0000 7fc914400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c158010 con 0x7fc90c15a010 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:28.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33532/0 (socket says 172.21.15.91:33532) 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 -- 172.21.15.91:0/3357525056 learned_addr learned my addr 172.21.15.91:0/3357525056 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 -- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc90c15a010 msgr2=0x7fc90c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc90c15a010 0x7fc90c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 -- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc90c0ce130 msgr2=0x7fc90c0707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911000640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc90c0ce130 0x7fc90c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc90c0ce130 0x7fc90c0707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 -- 172.21.15.91:0/3357525056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4009400 con 0x7fc90c0b0700 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.419+0000 7fc911a00640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c410 tx=0x7fc8fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.420+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc005ce0 con 0x7fc90c0b0700 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.420+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc005e40 con 0x7fc90c0b0700 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.420+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0072f0 con 0x7fc90c0b0700 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.420+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c0b4980 con 0x7fc90c0b0700 2024-02-02T13:30:28.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.420+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c0b43c0 con 0x7fc90c0b0700 2024-02-02T13:30:28.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.421+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c066cf0 con 0x7fc90c0b0700 2024-02-02T13:30:28.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.422+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc8fc007510 con 0x7fc90c0b0700 2024-02-02T13:30:28.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.422+0000 7fc909600640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 0x7fc8e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:28.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.423+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc8fc087f10 con 0x7fc90c0b0700 2024-02-02T13:30:28.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.423+0000 7fc911000640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 0x7fc8e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:28.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.423+0000 7fc911000640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 0x7fc8e406c8f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4036000 tx=0x7fc8f403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:28.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.427+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8fc053630 con 0x7fc90c0b0700 2024-02-02T13:30:28.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.654+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc90c0b4c00 con 0x7fc90c0b0700 2024-02-02T13:30:28.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.655+0000 7fc909600640 1 -- 172.21.15.91:0/3357525056 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fc8fc05be60 con 0x7fc90c0b0700 2024-02-02T13:30:28.657 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:28.657 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:30:28.657 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 msgr2=0x7fc8e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 0x7fc8e406c8f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4036000 tx=0x7fc8f403b040 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 msgr2=0x7fc90c0b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c410 tx=0x7fc8fc00c8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 shutdown_connections 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8e406a430 0x7fc8e406c8f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc90c15a010 0x7fc90c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc90c0ce130 0x7fc90c0707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 --2- 172.21.15.91:0/3357525056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc90c0b0700 0x7fc90c0b3b00 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 >> 172.21.15.91:0/3357525056 conn(0x7fc90c0cec80 msgr2=0x7fc90c159200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 shutdown_connections 2024-02-02T13:30:28.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:28.658+0000 7fc914400640 1 -- 172.21.15.91:0/3357525056 wait complete. 2024-02-02T13:30:28.677 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-02-02T13:30:28.910 INFO:teuthology.orchestra.run.smithi113.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":6705,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.747732928,"mdsmap_epoch":48,"osdmap_epoch":107,"osdmap_epoch_barrier":105,"uptime":16.460761858000001} 2024-02-02T13:30:28.911 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 6705, "mdsmap_epoch": 48, "osdmap_epoch": 107, "osdmap_epoch_barrier": 105, "rank_uptime": 10.747732928, "state": "up:active", "uptime": 16.460761858, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:30:28.911 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:30:29.137 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:30:29.137 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:30:29.137 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-02-02T13:30:29.137 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:29.137 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:29.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:29.189 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-02-02T13:30:30.416 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:30:30.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:30:31.219 INFO:teuthology.orchestra.run.smithi091.stderr:265+0 records in 2024-02-02T13:30:31.219 INFO:teuthology.orchestra.run.smithi091.stderr:265+0 records out 2024-02-02T13:30:31.219 INFO:teuthology.orchestra.run.smithi091.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.92672 s, 144 MB/s 2024-02-02T13:30:31.220 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:31.220 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:31.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:31.260 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-02-02T13:30:33.378 INFO:teuthology.orchestra.run.smithi091.stderr:292+0 records in 2024-02-02T13:30:33.378 INFO:teuthology.orchestra.run.smithi091.stderr:292+0 records out 2024-02-02T13:30:33.378 INFO:teuthology.orchestra.run.smithi091.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.02544 s, 151 MB/s 2024-02-02T13:30:33.380 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-02-02T13:30:33.380 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:33.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.564+0000 7f1c7b200640 1 Processor -- start 2024-02-02T13:30:33.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.564+0000 7f1c7b200640 1 -- start start 2024-02-02T13:30:33.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.565+0000 7f1c7b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c7c0ce4d0 0x7f1c7c0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.565+0000 7f1c7b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.565+0000 7f1c7b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c06faa0 con 0x7f1c7c0c9a80 2024-02-02T13:30:33.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.565+0000 7f1c7b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c15e800 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.565+0000 7f1c7b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0c9640 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.566+0000 7f1c7a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.566+0000 7f1c7a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:52994/0 (socket says 172.21.15.91:52994) 2024-02-02T13:30:33.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.566+0000 7f1c7a800640 1 -- 172.21.15.91:0/2993653128 learned_addr learned my addr 172.21.15.91:0/2993653128 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:33.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.566+0000 7f1c73e00640 1 --1- 172.21.15.91:0/2993653128 >> v1:172.21.15.113:6789/0 conn(0x7f1c7c0c9a80 0x7f1c7c15d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:43826/0 (socket says 172.21.15.91:43826) 2024-02-02T13:30:33.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.566+0000 7f1c79e00640 1 --2- 172.21.15.91:0/2993653128 >> v2:172.21.15.113:3300/0 conn(0x7f1c7c0ce4d0 0x7f1c7c0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.567+0000 7f1c7a800640 1 -- 172.21.15.91:0/2993653128 >> v2:172.21.15.113:3300/0 conn(0x7f1c7c0ce4d0 msgr2=0x7f1c7c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:33.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.567+0000 7f1c7a800640 1 --2- 172.21.15.91:0/2993653128 >> v2:172.21.15.113:3300/0 conn(0x7f1c7c0ce4d0 0x7f1c7c0707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.567+0000 7f1c7a800640 1 -- 172.21.15.91:0/2993653128 >> v1:172.21.15.113:6789/0 conn(0x7f1c7c0c9a80 legacy=0x7f1c7c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.567+0000 7f1c7a800640 1 -- 172.21.15.91:0/2993653128 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0ca420 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.568+0000 7f1c7a800640 1 --2- 172.21.15.91:0/2993653128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009e50 tx=0x7f1c7402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610c8cf9f9c61869 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:33.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.569+0000 7f1c79400640 1 -- 172.21.15.91:0/2993653128 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7402cb00 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.569+0000 7f1c79400640 1 -- 172.21.15.91:0/2993653128 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7402cc60 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.569+0000 7f1c79400640 1 -- 172.21.15.91:0/2993653128 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740343f0 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.569+0000 7f1c7b200640 1 -- 172.21.15.91:0/2993653128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 msgr2=0x7f1c7c0c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.569+0000 7f1c7b200640 1 --2- 172.21.15.91:0/2993653128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009e50 tx=0x7f1c7402bf60 comp rx=0 tx=0).stop 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 -- 172.21.15.91:0/2993653128 shutdown_connections 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 --2- 172.21.15.91:0/2993653128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c15dfb0 0x7f1c7c0c9ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 --2- 172.21.15.91:0/2993653128 >> v2:172.21.15.113:3300/0 conn(0x7f1c7c0ce4d0 0x7f1c7c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 -- 172.21.15.91:0/2993653128 >> 172.21.15.91:0/2993653128 conn(0x7f1c7c0cb4a0 msgr2=0x7f1c7c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 -- 172.21.15.91:0/2993653128 shutdown_connections 2024-02-02T13:30:33.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.570+0000 7f1c7b200640 1 -- 172.21.15.91:0/2993653128 wait complete. 2024-02-02T13:30:33.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.571+0000 7f1c7b200640 1 Processor -- start 2024-02-02T13:30:33.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 -- start start 2024-02-02T13:30:33.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 0x7f1c7c0a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 0x7f1c7c14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0a84c0 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0a86a0 con 0x7f1c7c15dfb0 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.572+0000 7f1c7b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0a8980 con 0x7f1c7c0c9a80 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44908/0 (socket says 172.21.15.91:44908) 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 -- 172.21.15.91:0/1646166111 learned_addr learned my addr 172.21.15.91:0/1646166111 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c79e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 0x7f1c7c0a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c7a800640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 0x7f1c7c14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 -- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 msgr2=0x7f1c7c0a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 0x7f1c7c0a9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 -- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 msgr2=0x7f1c7c14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 0x7f1c7c14ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 -- 172.21.15.91:0/1646166111 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009b30 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c79e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 0x7f1c7c0a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c7a800640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 0x7f1c7c14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.573+0000 7f1c73e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c3e0 tx=0x7f1c6400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.574+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640074e0 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.574+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c64007640 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.574+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640049e0 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.574+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c0a8c60 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.574+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c0aa050 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.575+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c0a9180 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.576+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1c64004b40 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.577+0000 7f1c72000640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 0x7f1c5406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:33.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.577+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1c640882e0 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.577+0000 7f1c79e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 0x7f1c5406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:33.579 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.578+0000 7f1c79e00640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 0x7f1c5406c920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009a20 tx=0x7f1c6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:33.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.581+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1c7c0a9180 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.763+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1c7c0a9180 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.766+0000 7f1c72000640 1 -- 172.21.15.91:0/1646166111 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+33730 (secure 0 0 0) 0x7f1c7c0a9180 con 0x7f1c7c0ce4d0 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 107, 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:27.103014+0000", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:33.769 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:33.770 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:33.771 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:33.772 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "93", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:33.773 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:33.774 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:33.775 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:33.776 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:33.777 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:33.778 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "104", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:33.779 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:33.780 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:33.781 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.782 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:33.783 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:33.784 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "105", 2024-02-02T13:30:33.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:33.786 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:33.787 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.788 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.789 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.790 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.791 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.792 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.793 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.794 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.795 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.796 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.797 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:33.798 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:33.799 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.800 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.801 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.802 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:33.803 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.804 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.805 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.806 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.807 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.808 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.809 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.810 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.811 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.812 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.813 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.814 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.815 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.816 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.817 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.818 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.819 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.820 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:33.821 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.822 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.823 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.824 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.825 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:33.826 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:33.827 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.828 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.829 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.830 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:33.831 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:33.832 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.833 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:33.834 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.835 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.836 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:33.837 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:33.838 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:33.839 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:33.840 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 msgr2=0x7f1c5406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 0x7f1c5406c920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009a20 tx=0x7f1c6c008040 comp rx=0 tx=0).stop 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 msgr2=0x7f1c7c14faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c3e0 tx=0x7f1c6400c8a0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 shutdown_connections 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1c5406a460 0x7f1c5406c920 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c7c15dfb0 0x7f1c7c14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c7c0ce4d0 0x7f1c7c14faa0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 --2- 172.21.15.91:0/1646166111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c7c0c9a80 0x7f1c7c0a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:33.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 >> 172.21.15.91:0/1646166111 conn(0x7f1c7c0cb4a0 msgr2=0x7f1c7c0c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:33.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 shutdown_connections 2024-02-02T13:30:33.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:33.770+0000 7f1c7b200640 1 -- 172.21.15.91:0/1646166111 wait complete. 2024-02-02T13:30:33.843 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:30:35.387 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:30:35.386+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-02-02T13:30:38.844 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:39.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.061+0000 7f6147600640 1 Processor -- start 2024-02-02T13:30:39.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.061+0000 7f6147600640 1 -- start start 2024-02-02T13:30:39.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.062+0000 7f6147600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f614015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6147600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400ca0c0 con 0x7f6140159de0 2024-02-02T13:30:39.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6147600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400ca380 con 0x7f614015e170 2024-02-02T13:30:39.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6147600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ca660 con 0x7f61400cbc90 2024-02-02T13:30:39.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6144c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6140159de0 0x7f61400cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46154/0 (socket says 172.21.15.91:46154) 2024-02-02T13:30:39.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6144c00640 1 -- 172.21.15.91:0/3276837799 learned_addr learned my addr 172.21.15.91:0/3276837799 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948913448 0 0) 0x7f61400ca0c0 con 0x7f6140159de0 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6120005730 con 0x7f6140159de0 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.063+0000 7f6137e00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f614015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109601708 0 0) 0x7f6120005730 con 0x7f6140159de0 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 msgr2=0x7f614015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f614015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 >> v1:172.21.15.113:6789/0 conn(0x7f614015e170 legacy=0x7f614006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400ca940 con 0x7f6140159de0 2024-02-02T13:30:39.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6130001670 con 0x7f6140159de0 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 >> v1:172.21.15.91:6789/0 conn(0x7f6140159de0 legacy=0x7f61400cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f6120005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61200061e0 0x7f61200069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6120006f10 0x7f6120007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ca660 con 0x7f61200061e0 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612000dd50 con 0x7f6120006f10 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612000df10 con 0x7f61400cbc90 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f6137e00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61200061e0 0x7f61200069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.064+0000 7f6144c00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f6120005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6145600640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6120006f10 0x7f6120007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6137e00640 1 -- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 msgr2=0x7f6120005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:39.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6137e00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f6120005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6137e00640 1 -- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6120006f10 msgr2=0x7f6120007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:39.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6137e00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6120006f10 0x7f6120007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6137e00640 1 -- 172.21.15.91:0/3276837799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120005730 con 0x7f61200061e0 2024-02-02T13:30:39.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.065+0000 7f6144c00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cbc90 0x7f6120005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:30:39.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.066+0000 7f6137e00640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61200061e0 0x7f61200069d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f612800ba30 tx=0x7f6120007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cfb30aaea9dd85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:39.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.066+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128032640 con 0x7f61200061e0 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.066+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61280327a0 con 0x7f61200061e0 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.066+0000 7f613fe00640 1 -- 172.21.15.91:0/3276837799 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128032b30 con 0x7f61200061e0 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 -- 172.21.15.91:0/3276837799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61200061e0 msgr2=0x7f61200069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 --2- 172.21.15.91:0/3276837799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f61200061e0 0x7f61200069d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f612800ba30 tx=0x7f6120007a00 comp rx=0 tx=0).stop 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6145600640 1 -- 172.21.15.91:0/3276837799 reap_dead start 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 -- 172.21.15.91:0/3276837799 shutdown_connections 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 -- 172.21.15.91:0/3276837799 >> 172.21.15.91:0/3276837799 conn(0x7f61400b0700 msgr2=0x7f61400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 -- 172.21.15.91:0/3276837799 shutdown_connections 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.067+0000 7f6147600640 1 -- 172.21.15.91:0/3276837799 wait complete. 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.069+0000 7f6147600640 1 Processor -- start 2024-02-02T13:30:39.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.069+0000 7f6147600640 1 -- start start 2024-02-02T13:30:39.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 0x7f614014ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400c5060 0x7f614014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400c5f90 con 0x7f614014d050 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400c5c40 con 0x7f61200061e0 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6147600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400c5540 con 0x7f61400c5060 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6145600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6145600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44918/0 (socket says 172.21.15.91:44918) 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6145600640 1 -- 172.21.15.91:0/1914765967 learned_addr learned my addr 172.21.15.91:0/1914765967 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6144c00640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 0x7f614014ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.070+0000 7f6137e00640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400c5060 0x7f614014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 -- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400c5060 msgr2=0x7f614014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400c5060 0x7f614014c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 -- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 msgr2=0x7f614014ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 0x7f614014ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400c8610 con 0x7f614014d050 2024-02-02T13:30:39.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6144c00640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 0x7f614014ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6145600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f614015e550 tx=0x7f613800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800eed0 con 0x7f614014d050 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138003970 con 0x7f614014d050 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400c88f0 con 0x7f614014d050 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138003d20 con 0x7f614014d050 2024-02-02T13:30:39.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.071+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6140156110 con 0x7f614014d050 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.072+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61400c60d0 con 0x7f614014d050 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.073+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f61380050d0 con 0x7f614014d050 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.074+0000 7f613d600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 0x7f611806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.074+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f61380886e0 con 0x7f614014d050 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.074+0000 7f6144c00640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 0x7f611806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.075+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=112}) v3 -- 0x7f6118071fc0 con 0x7f614014d050 2024-02-02T13:30:39.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.075+0000 7f6144c00640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 0x7f611806c8f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f61300017a0 tx=0x7f6130001e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:39.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.079+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f61400c60d0 con 0x7f614014d050 2024-02-02T13:30:39.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.268+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f61400c60d0 con 0x7f614014d050 2024-02-02T13:30:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.271+0000 7f613d600640 1 -- 172.21.15.91:0/1914765967 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+33874 (secure 0 0 0) 0x7f61400c60d0 con 0x7f614014d050 2024-02-02T13:30:39.273 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:39.273 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:39.273 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 111, 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:38.426543+0000", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:39.274 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:39.275 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:39.276 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:39.277 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:39.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:39.279 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:39.280 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:39.281 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:39.282 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:39.283 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:39.284 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:39.285 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:39.286 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:39.287 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:39.288 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:39.289 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:39.290 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:39.291 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:39.292 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:39.293 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.294 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.295 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.296 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:39.297 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.298 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.299 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.300 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.301 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.302 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:39.303 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.304 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.305 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:39.306 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:39.307 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:39.308 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:39.309 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.312 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.313 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:39.314 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.315 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.316 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.317 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.318 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.319 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.320 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.321 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.322 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.323 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:39.324 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.325 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.326 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:39.327 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:39.328 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.329 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.330 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:39.331 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:39.332 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.333 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.334 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:39.335 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.336 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.337 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:39.338 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.339 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.340 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:39.341 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:39.342 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:39.343 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:39.344 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 msgr2=0x7f611806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 0x7f611806c8f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f61300017a0 tx=0x7f6130001e40 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 msgr2=0x7f61400c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f614015e550 tx=0x7f613800eb90 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 shutdown_connections 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f611806a430 0x7f611806c8f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400c5060 0x7f614014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f614014d050 0x7f61400c30b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 --2- 172.21.15.91:0/1914765967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61200061e0 0x7f614014ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 >> 172.21.15.91:0/1914765967 conn(0x7f61400b0700 msgr2=0x7f614006cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 shutdown_connections 2024-02-02T13:30:39.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.274+0000 7f6147600640 1 -- 172.21.15.91:0/1914765967 wait complete. 2024-02-02T13:30:39.347 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-02-02T13:30:39.347 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:39.347 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:39.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:39.401 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-02-02T13:30:39.790 INFO:teuthology.orchestra.run.smithi091.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-02-02T13:30:39.790 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:30:39.790 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:30:39.790 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.320178 s, 164 MB/s 2024-02-02T13:30:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:30:39.792 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:39.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.990+0000 7faa43600640 1 Processor -- start 2024-02-02T13:30:39.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.990+0000 7faa43600640 1 -- start start 2024-02-02T13:30:39.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa43600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa3c0cb580 0x7faa3c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa43600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa43600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c0c94a0 con 0x7faa3c15d3a0 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa43600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0ad6e0 con 0x7faa3c0cb580 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa43600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0c96f0 con 0x7faa3c0ac960 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa40c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa3c0cb580 0x7faa3c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa3be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faa3c15d3a0 0x7faa3c0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37138/0 (socket says 172.21.15.91:37138) 2024-02-02T13:30:39.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.992+0000 7faa3be00640 1 -- 172.21.15.91:0/3616127523 learned_addr learned my addr 172.21.15.91:0/3616127523 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:39.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.993+0000 7faa41600640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:39.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.993+0000 7faa40c00640 1 -- 172.21.15.91:0/3616127523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 msgr2=0x7faa3c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:39.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.993+0000 7faa40c00640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.993+0000 7faa40c00640 1 -- 172.21.15.91:0/3616127523 >> v1:172.21.15.91:6789/0 conn(0x7faa3c15d3a0 legacy=0x7faa3c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.993+0000 7faa40c00640 1 -- 172.21.15.91:0/3616127523 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0c99d0 con 0x7faa3c0cb580 2024-02-02T13:30:39.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.994+0000 7faa40c00640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c0cb580 0x7faa3c0ca1d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0097a0 tx=0x7faa2c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de68246b94d74da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:39.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.994+0000 7faa3b400640 1 -- 172.21.15.91:0/3616127523 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c02d040 con 0x7faa3c0cb580 2024-02-02T13:30:39.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.994+0000 7faa3b400640 1 -- 172.21.15.91:0/3616127523 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c031810 con 0x7faa3c0cb580 2024-02-02T13:30:39.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.994+0000 7faa3b400640 1 -- 172.21.15.91:0/3616127523 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c031ba0 con 0x7faa3c0cb580 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 -- 172.21.15.91:0/3616127523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c0cb580 msgr2=0x7faa3c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c0cb580 0x7faa3c0ca1d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0097a0 tx=0x7faa2c02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 -- 172.21.15.91:0/3616127523 shutdown_connections 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 --2- 172.21.15.91:0/3616127523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c0cb580 0x7faa3c0ca1d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 -- 172.21.15.91:0/3616127523 >> 172.21.15.91:0/3616127523 conn(0x7faa3c0b0700 msgr2=0x7faa3c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 -- 172.21.15.91:0/3616127523 shutdown_connections 2024-02-02T13:30:39.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.995+0000 7faa43600640 1 -- 172.21.15.91:0/3616127523 wait complete. 2024-02-02T13:30:39.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.997+0000 7faa43600640 1 Processor -- start 2024-02-02T13:30:39.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.997+0000 7faa43600640 1 -- start start 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa3c0cb580 0x7faa3c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0c4100 con 0x7faa3c15d3a0 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0af330 con 0x7faa3c0cb580 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa43600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0c42e0 con 0x7faa3c0ac960 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa41600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa41600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54770/0 (socket says 172.21.15.91:54770) 2024-02-02T13:30:40.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.998+0000 7faa41600640 1 -- 172.21.15.91:0/1781229787 learned_addr learned my addr 172.21.15.91:0/1781229787 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa40c00640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 -- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 msgr2=0x7faa3c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa3be00640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa3c0cb580 0x7faa3c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 -- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa3c0cb580 msgr2=0x7faa3c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa3c0cb580 0x7faa3c0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009400 con 0x7faa3c15d3a0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa41600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faa34007e40 tx=0x7faa3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34014620 con 0x7faa3c15d3a0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3400ee90 con 0x7faa3c15d3a0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340149e0 con 0x7faa3c15d3a0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c0ad4b0 con 0x7faa3c15d3a0 2024-02-02T13:30:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:39.999+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c0c48a0 con 0x7faa3c15d3a0 2024-02-02T13:30:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.001+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c066cf0 con 0x7faa3c15d3a0 2024-02-02T13:30:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.001+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faa34014b40 con 0x7faa3c15d3a0 2024-02-02T13:30:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.002+0000 7faa38c00640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 0x7faa1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.002+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5738+0+0 (secure 0 0 0) 0x7faa34089340 con 0x7faa3c15d3a0 2024-02-02T13:30:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.002+0000 7faa40c00640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 0x7faa1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.002+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=113}) v3 -- 0x7faa14071fc0 con 0x7faa3c15d3a0 2024-02-02T13:30:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.003+0000 7faa40c00640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 0x7faa1406c8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa2c036000 tx=0x7faa2c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:40.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.007+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faa34054870 con 0x7faa3c15d3a0 2024-02-02T13:30:40.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.191+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faa3c06a610 con 0x7faa3c15d3a0 2024-02-02T13:30:40.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.194+0000 7faa38c00640 1 -- 172.21.15.91:0/1781229787 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+33898 (secure 0 0 0) 0x7faa34057e90 con 0x7faa3c15d3a0 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 112, 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:40.196 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:39.436687+0000", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:40.197 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:40.198 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:40.199 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:40.200 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:40.201 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:40.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.203 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:40.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:40.206 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:40.207 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.208 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:40.209 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:40.210 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:40.211 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:40.212 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:40.213 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:40.214 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.215 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.216 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.217 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:40.218 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.219 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:40.220 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.221 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.222 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:40.224 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.225 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.226 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.227 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.228 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:40.230 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.231 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:40.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:40.235 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:40.236 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:40.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:40.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.244 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:40.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.249 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.250 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.251 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.252 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:40.254 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:40.255 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:40.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.258 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.259 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.260 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.261 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:40.262 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:40.265 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:40.266 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:40.267 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:40.268 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:40.269 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:40.269 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:40.269 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:40.269 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:40.269 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:40.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.196+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 msgr2=0x7faa1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.196+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 0x7faa1406c8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa2c036000 tx=0x7faa2c03c040 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.196+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 msgr2=0x7faa3c0b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.196+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faa34007e40 tx=0x7faa3400e4b0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 shutdown_connections 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faa1406a430 0x7faa1406c8f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa3c15d3a0 0x7faa3c0b2150 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa3c0cb580 0x7faa3c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 --2- 172.21.15.91:0/1781229787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3c0ac960 0x7faa3c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 >> 172.21.15.91:0/1781229787 conn(0x7faa3c0b0700 msgr2=0x7faa3c0af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 shutdown_connections 2024-02-02T13:30:40.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.197+0000 7faa43600640 1 -- 172.21.15.91:0/1781229787 wait complete. 2024-02-02T13:30:40.272 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:40.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.478+0000 7f9ebca00640 1 Processor -- start 2024-02-02T13:30:40.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.478+0000 7f9ebca00640 1 -- start start 2024-02-02T13:30:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.479+0000 7f9ebca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.479+0000 7f9ebca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9eb40b0c40 0x7f9eb40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.479+0000 7f9ebca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb415fa00 con 0x7f9eb4151f90 2024-02-02T13:30:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.479+0000 7f9ebca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb40b10d0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.479+0000 7f9ebca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4070f80 con 0x7f9eb415f5a0 2024-02-02T13:30:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.480+0000 7f9eb9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.480+0000 7f9eb9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:46374/0 (socket says 172.21.15.91:46374) 2024-02-02T13:30:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.480+0000 7f9eb9600640 1 -- 172.21.15.91:0/2274025763 learned_addr learned my addr 172.21.15.91:0/2274025763 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:40.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.480+0000 7f9ebaa00640 1 --2- 172.21.15.91:0/2274025763 >> v2:172.21.15.113:3300/0 conn(0x7f9eb40b0c40 0x7f9eb40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.481+0000 7f9eb9600640 1 -- 172.21.15.91:0/2274025763 >> v2:172.21.15.113:3300/0 conn(0x7f9eb40b0c40 msgr2=0x7f9eb40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.481+0000 7f9eb9600640 1 --2- 172.21.15.91:0/2274025763 >> v2:172.21.15.113:3300/0 conn(0x7f9eb40b0c40 0x7f9eb40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.481+0000 7f9eb9600640 1 -- 172.21.15.91:0/2274025763 >> v1:172.21.15.113:6789/0 conn(0x7f9eb4151f90 legacy=0x7f9eb406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.481+0000 7f9eb9600640 1 -- 172.21.15.91:0/2274025763 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4152370 con 0x7f9eb415f5a0 2024-02-02T13:30:40.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.482+0000 7f9eb9600640 1 --2- 172.21.15.91:0/2274025763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28dbfa3260c9275 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:40.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.483+0000 7f9eb8c00640 1 -- 172.21.15.91:0/2274025763 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4030840 con 0x7f9eb415f5a0 2024-02-02T13:30:40.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.483+0000 7f9eb8c00640 1 -- 172.21.15.91:0/2274025763 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea40309a0 con 0x7f9eb415f5a0 2024-02-02T13:30:40.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.483+0000 7f9eb8c00640 1 -- 172.21.15.91:0/2274025763 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4030d30 con 0x7f9eb415f5a0 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 -- 172.21.15.91:0/2274025763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 msgr2=0x7f9eb40b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 --2- 172.21.15.91:0/2274025763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea402b980 comp rx=0 tx=0).stop 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 -- 172.21.15.91:0/2274025763 shutdown_connections 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 --2- 172.21.15.91:0/2274025763 >> v2:172.21.15.113:3300/0 conn(0x7f9eb40b0c40 0x7f9eb40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 --2- 172.21.15.91:0/2274025763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb415f5a0 0x7f9eb40b0700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 -- 172.21.15.91:0/2274025763 >> 172.21.15.91:0/2274025763 conn(0x7f9eb40cec80 msgr2=0x7f9eb40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 -- 172.21.15.91:0/2274025763 shutdown_connections 2024-02-02T13:30:40.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.484+0000 7f9ebca00640 1 -- 172.21.15.91:0/2274025763 wait complete. 2024-02-02T13:30:40.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.486+0000 7f9ebca00640 1 Processor -- start 2024-02-02T13:30:40.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.486+0000 7f9ebca00640 1 -- start start 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb4151f90 0x7f9eb40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb415f5a0 0x7f9eb415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40c83b0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40c7730 con 0x7f9eb415f5a0 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.487+0000 7f9ebca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40c7a10 con 0x7f9eb4151f90 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54776/0 (socket says 172.21.15.91:54776) 2024-02-02T13:30:40.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 -- 172.21.15.91:0/138738836 learned_addr learned my addr 172.21.15.91:0/138738836 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 -- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb4151f90 msgr2=0x7f9eb40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb4151f90 0x7f9eb40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 -- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb415f5a0 msgr2=0x7f9eb415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb415f5a0 0x7f9eb415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40094d0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.488+0000 7f9eba000640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c400 tx=0x7f9e9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.489+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0052d0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.489+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c005430 con 0x7f9eb40b0c40 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.489+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c003830 con 0x7f9eb40b0c40 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.489+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb40c7cf0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.489+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb40b31e0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.490+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb40c84f0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.491+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9e9c003990 con 0x7f9eb40b0c40 2024-02-02T13:30:40.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.492+0000 7f9eb2000640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 0x7f9e8c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:40.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.493+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9e9c087d90 con 0x7f9eb40b0c40 2024-02-02T13:30:40.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.493+0000 7f9eb9600640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 0x7f9e8c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:40.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.493+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=113}) v3 -- 0x7f9e8c071fc0 con 0x7f9eb40b0c40 2024-02-02T13:30:40.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.493+0000 7f9eb9600640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 0x7f9e8c06c8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4036000 tx=0x7f9ea403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:40.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.497+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9e9c017060 con 0x7f9eb40b0c40 2024-02-02T13:30:40.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.723+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb415fa00 con 0x7f9eb40b0c40 2024-02-02T13:30:40.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.725+0000 7f9eb2000640 1 -- 172.21.15.91:0/138738836 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f9eb415fa00 con 0x7f9eb40b0c40 2024-02-02T13:30:40.726 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:40.726 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:30:40.727 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 msgr2=0x7f9e8c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 0x7f9e8c06c8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4036000 tx=0x7f9ea403b040 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 msgr2=0x7f9eb40c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c400 tx=0x7f9e9c00c8c0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 shutdown_connections 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e8c06a430 0x7f9e8c06c8f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb4151f90 0x7f9eb40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb415f5a0 0x7f9eb415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 --2- 172.21.15.91:0/138738836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9eb40b0c40 0x7f9eb40c7150 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:40.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 >> 172.21.15.91:0/138738836 conn(0x7f9eb40cec80 msgr2=0x7f9eb40c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:40.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 shutdown_connections 2024-02-02T13:30:40.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:40.727+0000 7f9ebca00640 1 -- 172.21.15.91:0/138738836 wait complete. 2024-02-02T13:30:40.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-02-02T13:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":6705,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.815361239000001,"mdsmap_epoch":48,"osdmap_epoch":112,"osdmap_epoch_barrier":112,"uptime":28.528380095999999} 2024-02-02T13:30:40.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 6705, "mdsmap_epoch": 48, "osdmap_epoch": 112, "osdmap_epoch_barrier": 112, "rank_uptime": 22.815361239, "state": "up:active", "uptime": 28.528380096, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:30:40.978 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:30:40.978 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:40.978 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:41.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:41.035 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-02-02T13:30:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:dd: failed to open 'small_file_1': Permission denied 2024-02-02T13:30:41.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:30:41.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:41.087 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-02-02T13:30:41.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:41.140 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-02-02T13:30:41.277 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:41.457 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:30:41.456+0000 7faab4600640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-02-02T13:30:41.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.474+0000 7f59e4800640 1 Processor -- start 2024-02-02T13:30:41.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.475+0000 7f59e4800640 1 -- start start 2024-02-02T13:30:41.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59dc0cf840 0x7f59dc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e4800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0cdb90 con 0x7f59dc159d50 2024-02-02T13:30:41.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e4800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0ccf80 con 0x7f59dc06b9c0 2024-02-02T13:30:41.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e4800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc15a8c0 con 0x7f59dc0cf840 2024-02-02T13:30:41.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e1400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f59dc159d50 0x7f59dc15a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37154/0 (socket says 172.21.15.91:37154) 2024-02-02T13:30:41.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.476+0000 7f59e1400640 1 -- 172.21.15.91:0/908570823 learned_addr learned my addr 172.21.15.91:0/908570823 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:41.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653134567 0 0) 0x7f59dc0cdb90 con 0x7f59dc159d50 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59bc003440 con 0x7f59dc159d50 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e1e00640 1 --2- 172.21.15.91:0/908570823 >> v2:172.21.15.113:3300/0 conn(0x7f59dc0cf840 0x7f59dc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967446792 0 0) 0x7f59bc003440 con 0x7f59dc159d50 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 >> v2:172.21.15.113:3300/0 conn(0x7f59dc0cf840 msgr2=0x7f59dc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 --2- 172.21.15.91:0/908570823 >> v2:172.21.15.113:3300/0 conn(0x7f59dc0cf840 0x7f59dc0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 >> v1:172.21.15.113:6789/0 conn(0x7f59dc06b9c0 legacy=0x7f59dc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc0cd260 con 0x7f59dc159d50 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f59cc001670 con 0x7f59dc159d50 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 >> v1:172.21.15.91:6789/0 conn(0x7f59dc159d50 legacy=0x7f59dc15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59bc003a60 0x7f59bc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59bc0088a0 0x7f59bc00ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59bc00d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00e280 con 0x7f59bc00b510 2024-02-02T13:30:41.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00e3f0 con 0x7f59bc0088a0 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.477+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00e5b0 con 0x7f59bc003a60 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59bc00d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 -- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59bc003a60 msgr2=0x7f59bc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59bc003a60 0x7f59bc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 -- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59bc0088a0 msgr2=0x7f59bc00ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59bc0088a0 0x7f59bc00ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.478+0000 7f59e2800640 1 -- 172.21.15.91:0/908570823 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc003440 con 0x7f59bc00b510 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.479+0000 7f59e2800640 1 -- 172.21.15.91:0/908570823 reap_dead start 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.479+0000 7f59e2800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59bc00d9d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f59d40044b0 tx=0x7f59bc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c384a498e494f69f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.479+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4006d50 con 0x7f59bc00b510 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.479+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4007020 con 0x7f59bc00b510 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.479+0000 7f59e0a00640 1 -- 172.21.15.91:0/908570823 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40073d0 con 0x7f59bc00b510 2024-02-02T13:30:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 -- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 msgr2=0x7f59bc00d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59bc00d9d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f59d40044b0 tx=0x7f59bc0036f0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 -- 172.21.15.91:0/908570823 shutdown_connections 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 --2- 172.21.15.91:0/908570823 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59bc00d9d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 -- 172.21.15.91:0/908570823 >> 172.21.15.91:0/908570823 conn(0x7f59dc0ced20 msgr2=0x7f59dc0cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 -- 172.21.15.91:0/908570823 shutdown_connections 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.480+0000 7f59e4800640 1 -- 172.21.15.91:0/908570823 wait complete. 2024-02-02T13:30:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.482+0000 7f59e4800640 1 Processor -- start 2024-02-02T13:30:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.482+0000 7f59e4800640 1 -- start start 2024-02-02T13:30:41.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59dc150990 0x7f59dc0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59dc139e20 0x7f59dc13a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0b47c0 con 0x7f59bc00b510 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc14b470 con 0x7f59dc139e20 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e4800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc153010 con 0x7f59dc150990 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e1e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54802/0 (socket says 172.21.15.91:54802) 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.483+0000 7f59e1e00640 1 -- 172.21.15.91:0/1967716884 learned_addr learned my addr 172.21.15.91:0/1967716884 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 -- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59dc150990 msgr2=0x7f59dc0b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59dc150990 0x7f59dc0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 -- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59dc139e20 msgr2=0x7f59dc13a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59dc139e20 0x7f59dc13a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc1532f0 con 0x7f59bc00b510 2024-02-02T13:30:41.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e1e00640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d800 tx=0x7f59c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400b6c0 con 0x7f59bc00b510 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c400b840 con 0x7f59bc00b510 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400bbf0 con 0x7f59bc00b510 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.484+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc1535d0 con 0x7f59bc00b510 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.485+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc13a810 con 0x7f59bc00b510 2024-02-02T13:30:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.486+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc066cf0 con 0x7f59bc00b510 2024-02-02T13:30:41.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.486+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f59c400bd50 con 0x7f59bc00b510 2024-02-02T13:30:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.487+0000 7f59da000640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 0x7f59b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.487+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f59c4088d20 con 0x7f59bc00b510 2024-02-02T13:30:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.488+0000 7f59e1400640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 0x7f59b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.488+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=114}) v3 -- 0x7f59b4071fc0 con 0x7f59bc00b510 2024-02-02T13:30:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.488+0000 7f59e1400640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 0x7f59b406c8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0011e0 tx=0x7f59cc0014e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.492+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f59c40542e0 con 0x7f59bc00b510 2024-02-02T13:30:41.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.687+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f59dc14bb30 con 0x7f59bc00b510 2024-02-02T13:30:41.693 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.692+0000 7f59da000640 1 -- 172.21.15.91:0/1967716884 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v113) v1 ==== 81+0+33906 (secure 0 0 0) 0x7f59c4018070 con 0x7f59bc00b510 2024-02-02T13:30:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:41.694 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 113, 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:41.456972+0000", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:41.711 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:41.712 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:41.713 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:41.714 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.715 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:41.716 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.717 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.718 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.719 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:41.720 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.721 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.722 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.723 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:41.724 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.726 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.728 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.732 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.734 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.735 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:41.737 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:41.739 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:41.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:41.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:41.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.743 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.746 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.748 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:41.749 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.753 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:41.755 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:41.756 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:41.760 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.761 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.762 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:41.764 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.765 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:41.767 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:41.768 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:41.770 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:41.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:41.771 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:41.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 msgr2=0x7f59b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 0x7f59b406c8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0011e0 tx=0x7f59cc0014e0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 msgr2=0x7f59dc0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d800 tx=0x7f59c400dcc0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 shutdown_connections 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f59b406a430 0x7f59b406c8f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59dc150990 0x7f59dc0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59dc139e20 0x7f59dc13a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 --2- 172.21.15.91:0/1967716884 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f59bc00b510 0x7f59dc0b3d40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.695+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 >> 172.21.15.91:0/1967716884 conn(0x7f59dc0ced20 msgr2=0x7f59dc15aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.696+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 shutdown_connections 2024-02-02T13:30:41.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:41.696+0000 7f59e4800640 1 -- 172.21.15.91:0/1967716884 wait complete. 2024-02-02T13:30:41.774 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:30:46.775 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:46.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.989+0000 7fd1d2600640 1 Processor -- start 2024-02-02T13:30:46.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.989+0000 7fd1d2600640 1 -- start start 2024-02-02T13:30:46.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 0x7fd1cc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc15e170 0x7fd1cc0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0cbf50 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ca0c0 con 0x7fd1cc159de0 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1d2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ca380 con 0x7fd1cc15e170 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1c2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1c2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54812/0 (socket says 172.21.15.91:54812) 2024-02-02T13:30:46.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1c2800640 1 -- 172.21.15.91:0/933969155 learned_addr learned my addr 172.21.15.91:0/933969155 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.991+0000 7fd1cbe00640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc15e170 0x7fd1cc0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1cb400640 1 --2- 172.21.15.91:0/933969155 >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 0x7fd1cc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1c2800640 1 -- 172.21.15.91:0/933969155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc15e170 msgr2=0x7fd1cc0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1c2800640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc15e170 0x7fd1cc0cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1c2800640 1 -- 172.21.15.91:0/933969155 >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 msgr2=0x7fd1cc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1c2800640 1 --2- 172.21.15.91:0/933969155 >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 0x7fd1cc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1c2800640 1 -- 172.21.15.91:0/933969155 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0ca660 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.992+0000 7fd1cb400640 1 --2- 172.21.15.91:0/933969155 >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 0x7fd1cc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:46.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.993+0000 7fd1c2800640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b1640 tx=0x7fd1b402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfa3085fd29fa74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:46.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.993+0000 7fd1caa00640 1 -- 172.21.15.91:0/933969155 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4031860 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.993+0000 7fd1caa00640 1 -- 172.21.15.91:0/933969155 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b40319c0 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.993+0000 7fd1caa00640 1 -- 172.21.15.91:0/933969155 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4031d50 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 -- 172.21.15.91:0/933969155 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 msgr2=0x7fd1cc15dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b1640 tx=0x7fd1b402b900 comp rx=0 tx=0).stop 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 -- 172.21.15.91:0/933969155 shutdown_connections 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc15e170 0x7fd1cc0cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 --2- 172.21.15.91:0/933969155 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc15dc30 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 --2- 172.21.15.91:0/933969155 >> v2:172.21.15.113:3300/0 conn(0x7fd1cc159de0 0x7fd1cc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 -- 172.21.15.91:0/933969155 >> 172.21.15.91:0/933969155 conn(0x7fd1cc0b0700 msgr2=0x7fd1cc0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 -- 172.21.15.91:0/933969155 shutdown_connections 2024-02-02T13:30:46.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.994+0000 7fd1d2600640 1 -- 172.21.15.91:0/933969155 wait complete. 2024-02-02T13:30:46.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.996+0000 7fd1d2600640 1 Processor -- start 2024-02-02T13:30:46.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.996+0000 7fd1d2600640 1 -- start start 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc159de0 0x7fd1cc153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc15e170 0x7fd1cc1f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc154580 con 0x7fd1cc0cbac0 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc154390 con 0x7fd1cc15e170 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1d2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc156ca0 con 0x7fd1cc159de0 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54828/0 (socket says 172.21.15.91:54828) 2024-02-02T13:30:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 -- 172.21.15.91:0/4235566586 learned_addr learned my addr 172.21.15.91:0/4235566586 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1c2800640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc159de0 0x7fd1cc153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 -- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc159de0 msgr2=0x7fd1cc153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc159de0 0x7fd1cc153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 -- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc15e170 msgr2=0x7fd1cc1f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cbe00640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc15e170 0x7fd1cc1f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc15e170 0x7fd1cc1f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.997+0000 7fd1cb400640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40094d0 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1cb400640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c7c0 tx=0x7fd1bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc007a30 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc007b90 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00ce70 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc156f80 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.998+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc0b1f60 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:46.999+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc066cf0 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.000+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd1bc0113f0 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.001+0000 7fd1c3e00640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 0x7fd1a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:47.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.002+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd1bc088af0 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.002+0000 7fd1c2800640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 0x7fd1a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:47.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.002+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=116}) v3 -- 0x7fd1a4071fc0 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.002+0000 7fd1c2800640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 0x7fd1a406c8f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4036000 tx=0x7fd1b403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:47.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.006+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd1bc01f060 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.195+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd1cc0c9a60 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.198+0000 7fd1c3e00640 1 -- 172.21.15.91:0/4235566586 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v115) v1 ==== 81+0+33818 (secure 0 0 0) 0x7fd1bc01f020 con 0x7fd1cc0cbac0 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 115, 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:45.497280+0000", 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:47.203 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:47.204 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:47.205 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:47.206 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:47.207 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.208 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:47.209 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:47.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:47.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:47.212 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:47.214 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:47.215 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 3, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full", 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:47.216 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.217 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "108", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:47.218 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:47.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:47.220 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:47.221 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.222 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.223 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.224 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.225 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:47.226 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.227 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.228 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:47.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:47.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.237 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.242 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.243 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.244 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.247 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.248 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.250 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:47.252 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.253 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.254 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.255 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.257 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "full", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:47.258 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:47.260 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:47.262 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.263 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.265 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.266 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.267 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:47.268 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:47.269 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:47.270 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:47.271 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:47.271 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:47.271 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:47.271 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:47.271 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:47.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 msgr2=0x7fd1a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:47.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 0x7fd1a406c8f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4036000 tx=0x7fd1b403c040 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 msgr2=0x7fd1cc1f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c7c0 tx=0x7fd1bc00cc80 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 shutdown_connections 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd1a406a430 0x7fd1a406c8f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc159de0 0x7fd1cc153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc15e170 0x7fd1cc1f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 --2- 172.21.15.91:0/4235566586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1cc0cbac0 0x7fd1cc1f21d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 >> 172.21.15.91:0/4235566586 conn(0x7fd1cc0b0700 msgr2=0x7fd1cc0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 shutdown_connections 2024-02-02T13:30:47.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:47.200+0000 7fd1d2600640 1 -- 172.21.15.91:0/4235566586 wait complete. 2024-02-02T13:30:47.274 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:30:52.275 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:52.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.492+0000 7f438ac00640 1 Processor -- start 2024-02-02T13:30:52.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.492+0000 7f438ac00640 1 -- start start 2024-02-02T13:30:52.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f438ac00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f438415ad10 0x7f438406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f438ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43840cb580 0x7f43840cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f438ac00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438406d910 con 0x7f438415b0f0 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f438ac00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438406d730 con 0x7f438415ad10 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f438ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43840cbf50 con 0x7f43840cb580 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f437b200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f438415b0f0 0x7f438406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45832/0 (socket says 172.21.15.91:45832) 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f437b200640 1 -- 172.21.15.91:0/3182518010 learned_addr learned my addr 172.21.15.91:0/3182518010 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:52.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f4383e00640 1 --2- 172.21.15.91:0/3182518010 >> v2:172.21.15.91:3300/0 conn(0x7f438415ad10 0x7f438406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.494+0000 7f4383400640 1 -- 172.21.15.91:0/3182518010 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378408433 0 0) 0x7f438406d910 con 0x7f438415b0f0 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4388c00640 1 --2- 172.21.15.91:0/3182518010 >> v2:172.21.15.113:3300/0 conn(0x7f43840cb580 0x7f43840cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4383400640 1 -- 172.21.15.91:0/3182518010 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4364003520 con 0x7f438415b0f0 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4383e00640 1 -- 172.21.15.91:0/3182518010 >> v2:172.21.15.113:3300/0 conn(0x7f43840cb580 msgr2=0x7f43840cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4383e00640 1 --2- 172.21.15.91:0/3182518010 >> v2:172.21.15.113:3300/0 conn(0x7f43840cb580 0x7f43840cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4383e00640 1 -- 172.21.15.91:0/3182518010 >> v1:172.21.15.91:6789/0 conn(0x7f438415b0f0 legacy=0x7f438406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.495+0000 7f4383e00640 1 -- 172.21.15.91:0/3182518010 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840cc850 con 0x7f438415ad10 2024-02-02T13:30:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.496+0000 7f4383e00640 1 --2- 172.21.15.91:0/3182518010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415ad10 0x7f438406b7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f43740097a0 tx=0x7f437402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128049e77ee7280c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.496+0000 7f4383400640 1 -- 172.21.15.91:0/3182518010 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437402e070 con 0x7f438415ad10 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.496+0000 7f4383400640 1 -- 172.21.15.91:0/3182518010 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43740315a0 con 0x7f438415ad10 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.496+0000 7f4383400640 1 -- 172.21.15.91:0/3182518010 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374031930 con 0x7f438415ad10 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 -- 172.21.15.91:0/3182518010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415ad10 msgr2=0x7f438406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 --2- 172.21.15.91:0/3182518010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415ad10 0x7f438406b7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f43740097a0 tx=0x7f437402b840 comp rx=0 tx=0).stop 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 -- 172.21.15.91:0/3182518010 shutdown_connections 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 --2- 172.21.15.91:0/3182518010 >> v2:172.21.15.113:3300/0 conn(0x7f43840cb580 0x7f43840cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 --2- 172.21.15.91:0/3182518010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415ad10 0x7f438406b7a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 -- 172.21.15.91:0/3182518010 >> 172.21.15.91:0/3182518010 conn(0x7f43840b0700 msgr2=0x7f43840b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 -- 172.21.15.91:0/3182518010 shutdown_connections 2024-02-02T13:30:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.497+0000 7f438ac00640 1 -- 172.21.15.91:0/3182518010 wait complete. 2024-02-02T13:30:52.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.499+0000 7f438ac00640 1 Processor -- start 2024-02-02T13:30:52.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.499+0000 7f438ac00640 1 -- start start 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43840cb580 0x7f43841502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438415ad10 0x7f43840c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438414da40 con 0x7f438415b0f0 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840ad9c0 con 0x7f438415ad10 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f438ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840adca0 con 0x7f43840cb580 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f4388c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f4388c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55486/0 (socket says 172.21.15.91:55486) 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.500+0000 7f4388c00640 1 -- 172.21.15.91:0/3922025279 learned_addr learned my addr 172.21.15.91:0/3922025279 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4383e00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43840cb580 0x7f43841502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 -- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43840cb580 msgr2=0x7f43841502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43840cb580 0x7f43841502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f437b200640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438415ad10 0x7f43840c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 -- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438415ad10 msgr2=0x7f43840c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438415ad10 0x7f43840c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 -- 172.21.15.91:0/3922025279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4374009400 con 0x7f438415b0f0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4388c00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f437c007e40 tx=0x7f437c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c014620 con 0x7f438415b0f0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c00ee90 con 0x7f438415b0f0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c0149e0 con 0x7f438415b0f0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43840adf80 con 0x7f438415b0f0 2024-02-02T13:30:52.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.501+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43840ae540 con 0x7f438415b0f0 2024-02-02T13:30:52.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.502+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384066cf0 con 0x7f438415b0f0 2024-02-02T13:30:52.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.503+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f437c014b40 con 0x7f438415b0f0 2024-02-02T13:30:52.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.505+0000 7f4380c00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 0x7f435c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.505+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f437c089380 con 0x7f438415b0f0 2024-02-02T13:30:52.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.505+0000 7f4383e00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 0x7f435c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.506+0000 7f4383e00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 0x7f435c06c8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4374036000 tx=0x7f437403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:52.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.510+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f437c0548b0 con 0x7f438415b0f0 2024-02-02T13:30:52.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.701+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f438406ff50 con 0x7f438415b0f0 2024-02-02T13:30:52.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.705+0000 7f4380c00640 1 -- 172.21.15.91:0/3922025279 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+33731 (secure 0 0 0) 0x7f437c057ed0 con 0x7f438415b0f0 2024-02-02T13:30:52.707 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:52.707 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:52.707 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 116, 2024-02-02T13:30:52.707 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:52.707 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:48.529769+0000", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:52.708 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:52.709 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:52.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:52.711 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:52.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:52.713 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:52.714 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:52.716 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:52.717 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:52.718 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:52.719 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:52.719 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:52.719 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:52.719 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:52.720 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:52.721 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:52.722 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:52.723 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:52.724 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:52.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.726 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:52.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.728 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:52.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.732 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.733 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:52.737 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.739 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.740 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.743 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.744 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:52.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:52.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.748 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.749 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.750 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:52.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:52.753 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.754 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.755 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.757 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:52.758 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.759 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.760 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.762 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.765 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.766 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:52.767 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:52.768 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:52.769 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.771 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.772 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.773 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.774 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:52.775 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:52.776 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:52.777 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:52.778 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:52.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.707+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 msgr2=0x7f435c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.707+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 0x7f435c06c8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4374036000 tx=0x7f437403d040 comp rx=0 tx=0).stop 2024-02-02T13:30:52.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 msgr2=0x7f43840c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:52.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f437c007e40 tx=0x7f437c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 shutdown_connections 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435c06a430 0x7f435c06c8f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f438415b0f0 0x7f43840c80b0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438415ad10 0x7f43840c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 --2- 172.21.15.91:0/3922025279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43840cb580 0x7f43841502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 >> 172.21.15.91:0/3922025279 conn(0x7f43840b0700 msgr2=0x7f438406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 shutdown_connections 2024-02-02T13:30:52.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.708+0000 7f438ac00640 1 -- 172.21.15.91:0/3922025279 wait complete. 2024-02-02T13:30:52.780 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-02-02T13:30:52.781 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:30:52.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.994+0000 7f780fe00640 1 Processor -- start 2024-02-02T13:30:52.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.994+0000 7f780fe00640 1 -- start start 2024-02-02T13:30:52.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f780815ad10 0x7f780806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 0x7f780806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780806d910 con 0x7f78080cb580 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080cd610 con 0x7f780815ad10 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080cc850 con 0x7f780815b0f0 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780d400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f780815ad10 0x7f780806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780d400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f780815ad10 0x7f780806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55502/0 (socket says 172.21.15.91:55502) 2024-02-02T13:30:52.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.996+0000 7f780d400640 1 -- 172.21.15.91:0/4082165294 learned_addr learned my addr 172.21.15.91:0/4082165294 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780d400640 1 -- 172.21.15.91:0/4082165294 >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 msgr2=0x7f780806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780ca00640 1 --2- 172.21.15.91:0/4082165294 >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 0x7f780806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780d400640 1 --2- 172.21.15.91:0/4082165294 >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 0x7f780806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780d400640 1 -- 172.21.15.91:0/4082165294 >> v1:172.21.15.113:6789/0 conn(0x7f78080cb580 legacy=0x7f780806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780d400640 1 -- 172.21.15.91:0/4082165294 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080ccb30 con 0x7f780815ad10 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.997+0000 7f780ca00640 1 --2- 172.21.15.91:0/4082165294 >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 0x7f780806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:52.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.998+0000 7f780d400640 1 --2- 172.21.15.91:0/4082165294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f780806b7a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009e50 tx=0x7f77f802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bbf707b5bf8fdf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f7807e00640 1 -- 172.21.15.91:0/4082165294 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f802c9f0 con 0x7f780815ad10 2024-02-02T13:30:53.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f7807e00640 1 -- 172.21.15.91:0/4082165294 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f802cb50 con 0x7f780815ad10 2024-02-02T13:30:53.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f7807e00640 1 -- 172.21.15.91:0/4082165294 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80343f0 con 0x7f780815ad10 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 -- 172.21.15.91:0/4082165294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 msgr2=0x7f780806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 --2- 172.21.15.91:0/4082165294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f780806b7a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009e50 tx=0x7f77f802be50 comp rx=0 tx=0).stop 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 -- 172.21.15.91:0/4082165294 shutdown_connections 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 --2- 172.21.15.91:0/4082165294 >> v2:172.21.15.113:3300/0 conn(0x7f780815b0f0 0x7f780806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 --2- 172.21.15.91:0/4082165294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f780806b7a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:52.999+0000 7f780fe00640 1 -- 172.21.15.91:0/4082165294 >> 172.21.15.91:0/4082165294 conn(0x7f78080b0700 msgr2=0x7f78080b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.000+0000 7f780fe00640 1 -- 172.21.15.91:0/4082165294 shutdown_connections 2024-02-02T13:30:53.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.000+0000 7f780fe00640 1 -- 172.21.15.91:0/4082165294 wait complete. 2024-02-02T13:30:53.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.001+0000 7f780fe00640 1 Processor -- start 2024-02-02T13:30:53.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.001+0000 7f780fe00640 1 -- start start 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080cb580 0x7f7808144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f780815b0f0 0x7f78081f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080afc50 con 0x7f780815ad10 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080af040 con 0x7f78080cb580 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.002+0000 7f780fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78080af320 con 0x7f780815b0f0 2024-02-02T13:30:53.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55512/0 (socket says 172.21.15.91:55512) 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 -- 172.21.15.91:0/2060374352 learned_addr learned my addr 172.21.15.91:0/2060374352 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 -- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f780815b0f0 msgr2=0x7f78081f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f780815b0f0 0x7f78081f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 -- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080cb580 msgr2=0x7f7808144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080cb580 0x7f7808144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 -- 172.21.15.91:0/2060374352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8009b30 con 0x7f780815ad10 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.003+0000 7f780ca00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c7f0 tx=0x7f77f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.004+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0007820 con 0x7f780815ad10 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.004+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0007980 con 0x7f780815ad10 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.004+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0007cf0 con 0x7f780815ad10 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.004+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78080af600 con 0x7f780815ad10 2024-02-02T13:30:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.004+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78080ace10 con 0x7f780815ad10 2024-02-02T13:30:53.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.005+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78080c5160 con 0x7f780815ad10 2024-02-02T13:30:53.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.006+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77f00113f0 con 0x7f780815ad10 2024-02-02T13:30:53.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.007+0000 7f7805600640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 0x7f77e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.007+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f77f0088c70 con 0x7f780815ad10 2024-02-02T13:30:53.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.007+0000 7f780d400640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 0x7f77e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.008+0000 7f780d400640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 0x7f77e006c8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f77f8035000 tx=0x7f77f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.011+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77f0054230 con 0x7f780815ad10 2024-02-02T13:30:53.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.204+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f78080ad030 con 0x7f780815ad10 2024-02-02T13:30:53.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.207+0000 7f7805600640 1 -- 172.21.15.91:0/2060374352 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+33731 (secure 0 0 0) 0x7f77f001f090 con 0x7f780815ad10 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 116, 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:30:48.529769+0000", 2024-02-02T13:30:53.209 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:30:53.210 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 13, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:53.211 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:53.212 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:53.213 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:53.214 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:53.215 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:53.216 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 12, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:15.902582+0000", 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:53.217 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:53.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:53.219 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:53.220 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:30:53.221 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:53.222 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 13, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:30:17.156657+0000", 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:30:53.223 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:30:53.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:30:53.225 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.226 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:30:53.227 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:30:53.228 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.230 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.231 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.232 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.233 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.234 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:30:53.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:30:53.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:30:53.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:30:53.238 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.239 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:30:53.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.245 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.246 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:30:53.249 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.251 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.252 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:30:53.254 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.255 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.256 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.257 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.258 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.260 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 103, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.263 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 106, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:30:53.266 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:30:53.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.268 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.269 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:30:53.270 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.271 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.272 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.273 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.274 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:30:53.275 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:30:53.276 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "13.26", 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 1, 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 3 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:30:53.277 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:30:53.278 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:30:53.279 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 msgr2=0x7f77e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 0x7f77e006c8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f77f8035000 tx=0x7f77f802e000 comp rx=0 tx=0).stop 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 msgr2=0x7f78080ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c7f0 tx=0x7f77f000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 shutdown_connections 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77e006a430 0x7f77e006c8f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f780815b0f0 0x7f78081f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f780815ad10 0x7f78080ac8d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 --2- 172.21.15.91:0/2060374352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080cb580 0x7f7808144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 >> 172.21.15.91:0/2060374352 conn(0x7f78080b0700 msgr2=0x7f780815dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:53.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 shutdown_connections 2024-02-02T13:30:53.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.210+0000 7f780fe00640 1 -- 172.21.15.91:0/2060374352 wait complete. 2024-02-02T13:30:53.282 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:53.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.497+0000 7f6051400640 1 Processor -- start 2024-02-02T13:30:53.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.497+0000 7f6051400640 1 -- start start 2024-02-02T13:30:53.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f604c151f90 0x7f604c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 0x7f604c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c0b0c40 0x7f604c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c0b10b0 con 0x7f604c151f90 2024-02-02T13:30:53.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c070f80 con 0x7f604c06f4f0 2024-02-02T13:30:53.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6051400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c152370 con 0x7f604c0b0c40 2024-02-02T13:30:53.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f604c151f90 0x7f604c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f604c151f90 0x7f604c15f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55524/0 (socket says 172.21.15.91:55524) 2024-02-02T13:30:53.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 -- 172.21.15.91:0/1988211425 learned_addr learned my addr 172.21.15.91:0/1988211425 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604b200640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c0b0c40 0x7f604c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6043e00640 1 --2- 172.21.15.91:0/1988211425 >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 0x7f604c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 -- 172.21.15.91:0/1988211425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c0b0c40 msgr2=0x7f604c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c0b0c40 0x7f604c0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 -- 172.21.15.91:0/1988211425 >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 msgr2=0x7f604c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 --2- 172.21.15.91:0/1988211425 >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 0x7f604c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f604a800640 1 -- 172.21.15.91:0/1988211425 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0cb010 con 0x7f604c151f90 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.499+0000 7f6043e00640 1 --2- 172.21.15.91:0/1988211425 >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 0x7f604c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:53.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f604a800640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c15f5a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f604c071a00 tx=0x7f603c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19713dd14d5e6ecf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6049e00640 1 -- 172.21.15.91:0/1988211425 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c031870 con 0x7f604c151f90 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6049e00640 1 -- 172.21.15.91:0/1988211425 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0319d0 con 0x7f604c151f90 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6049e00640 1 -- 172.21.15.91:0/1988211425 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c031d60 con 0x7f604c151f90 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6051400640 1 -- 172.21.15.91:0/1988211425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 msgr2=0x7f604c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6051400640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c15f5a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f604c071a00 tx=0x7f603c02b910 comp rx=0 tx=0).stop 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.501+0000 7f6051400640 1 -- 172.21.15.91:0/1988211425 shutdown_connections 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c0b0c40 0x7f604c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 --2- 172.21.15.91:0/1988211425 >> v2:172.21.15.113:3300/0 conn(0x7f604c06f4f0 0x7f604c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 --2- 172.21.15.91:0/1988211425 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c15f5a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 -- 172.21.15.91:0/1988211425 >> 172.21.15.91:0/1988211425 conn(0x7f604c0cec80 msgr2=0x7f604c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 -- 172.21.15.91:0/1988211425 shutdown_connections 2024-02-02T13:30:53.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.502+0000 7f6051400640 1 -- 172.21.15.91:0/1988211425 wait complete. 2024-02-02T13:30:53.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.503+0000 7f6051400640 1 Processor -- start 2024-02-02T13:30:53.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.503+0000 7f6051400640 1 -- start start 2024-02-02T13:30:53.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 0x7f604c0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c0b0c40 0x7f604c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0b3620 con 0x7f604c151f90 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c150b50 con 0x7f604c0b0c40 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.504+0000 7f6051400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0b38f0 con 0x7f604c06f4f0 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55536/0 (socket says 172.21.15.91:55536) 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 -- 172.21.15.91:0/4010985137 learned_addr learned my addr 172.21.15.91:0/4010985137 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604a800640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 0x7f604c0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f6043e00640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c0b0c40 0x7f604c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 -- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 msgr2=0x7f604c0b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 0x7f604c0b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 -- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c0b0c40 msgr2=0x7f604c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c0b0c40 0x7f604c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604b200640 1 -- 172.21.15.91:0/4010985137 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0094d0 con 0x7f604c151f90 2024-02-02T13:30:53.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.505+0000 7f604a800640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 0x7f604c0b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f604b200640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f604400cc80 tx=0x7f604400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400ce30 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6044003990 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044003d20 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c0c3c80 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.506+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c0c4240 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.507+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c066cf0 con 0x7f604c151f90 2024-02-02T13:30:53.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.508+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6044003e80 con 0x7f604c151f90 2024-02-02T13:30:53.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.509+0000 7f6042a00640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 0x7f602406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:53.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.509+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6044088b60 con 0x7f604c151f90 2024-02-02T13:30:53.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.509+0000 7f604a800640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 0x7f602406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:53.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.510+0000 7f604a800640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 0x7f602406c8f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f603c036000 tx=0x7f603c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:53.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.513+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f60440541c0 con 0x7f604c151f90 2024-02-02T13:30:53.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.740+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604c1516b0 con 0x7f604c151f90 2024-02-02T13:30:53.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.741+0000 7f6042a00640 1 -- 172.21.15.91:0/4010985137 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f604401d070 con 0x7f604c151f90 2024-02-02T13:30:53.744 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:53.745 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:30:53.745 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.743+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 msgr2=0x7f602406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.743+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 0x7f602406c8f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f603c036000 tx=0x7f603c03c040 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.743+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 msgr2=0x7f604c150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.743+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f604400cc80 tx=0x7f604400a5c0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 shutdown_connections 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f602406a430 0x7f602406c8f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f604c151f90 0x7f604c150610 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c0b0c40 0x7f604c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 --2- 172.21.15.91:0/4010985137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06f4f0 0x7f604c0b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 >> 172.21.15.91:0/4010985137 conn(0x7f604c0cec80 msgr2=0x7f604c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 shutdown_connections 2024-02-02T13:30:53.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:53.744+0000 7f6051400640 1 -- 172.21.15.91:0/4010985137 wait complete. 2024-02-02T13:30:53.765 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-02-02T13:30:54.002 INFO:teuthology.orchestra.run.smithi113.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":6705,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.838952265000003,"mdsmap_epoch":48,"osdmap_epoch":116,"osdmap_epoch_barrier":112,"uptime":41.551971627999997} 2024-02-02T13:30:54.003 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 6705, "mdsmap_epoch": 48, "osdmap_epoch": 116, "osdmap_epoch_barrier": 112, "rank_uptime": 35.838952265, "state": "up:active", "uptime": 41.551971628, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:30:54.003 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:30:54.003 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:54.003 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:54.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:54.059 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-02-02T13:30:54.486 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:30:54.486 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:30:54.487 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.373278 s, 140 MB/s 2024-02-02T13:30:54.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:30:54.488 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:30:54.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:30:54.706 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:30:54.706 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:54.706 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:54.764 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:54.764 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:54.817 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:55.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.065+0000 7f74a5e00640 1 Processor -- start 2024-02-02T13:30:55.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.065+0000 7f74a5e00640 1 -- start start 2024-02-02T13:30:55.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f74a00b1100 0x7f74a006c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a006be70 con 0x7f74a006d1f0 2024-02-02T13:30:55.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a015d480 con 0x7f74a015d0a0 2024-02-02T13:30:55.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a006b8f0 con 0x7f74a00b1100 2024-02-02T13:30:55.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f749fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f74a006d1f0 0x7f74a00b09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45836/0 (socket says 172.21.15.91:45836) 2024-02-02T13:30:55.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f749fe00640 1 -- 172.21.15.91:0/414296989 learned_addr learned my addr 172.21.15.91:0/414296989 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:55.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5400640 1 --2- 172.21.15.91:0/414296989 >> v2:172.21.15.91:3300/0 conn(0x7f74a00b1100 0x7f74a006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f749f400640 1 -- 172.21.15.91:0/414296989 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078531884 0 0) 0x7f74a006be70 con 0x7f74a006d1f0 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f749f400640 1 -- 172.21.15.91:0/414296989 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7480003520 con 0x7f74a006d1f0 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5400640 1 -- 172.21.15.91:0/414296989 >> v1:172.21.15.113:6789/0 conn(0x7f74a015d0a0 legacy=0x7f74a00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5400640 1 -- 172.21.15.91:0/414296989 >> v1:172.21.15.91:6789/0 conn(0x7f74a006d1f0 legacy=0x7f74a00b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.067+0000 7f74a5400640 1 -- 172.21.15.91:0/414296989 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00ce900 con 0x7f74a00b1100 2024-02-02T13:30:55.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5400640 1 --2- 172.21.15.91:0/414296989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a00b1100 0x7f74a006c300 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7498009d90 tx=0x7f749802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32295711278194aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f749f400640 1 -- 172.21.15.91:0/414296989 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749802cc60 con 0x7f74a00b1100 2024-02-02T13:30:55.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f749f400640 1 -- 172.21.15.91:0/414296989 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749802cdc0 con 0x7f74a00b1100 2024-02-02T13:30:55.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f749f400640 1 -- 172.21.15.91:0/414296989 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498034660 con 0x7f74a00b1100 2024-02-02T13:30:55.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5e00640 1 -- 172.21.15.91:0/414296989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a00b1100 msgr2=0x7f74a006c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5e00640 1 --2- 172.21.15.91:0/414296989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a00b1100 0x7f74a006c300 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7498009d90 tx=0x7f749802beb0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5e00640 1 -- 172.21.15.91:0/414296989 shutdown_connections 2024-02-02T13:30:55.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5e00640 1 --2- 172.21.15.91:0/414296989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a00b1100 0x7f74a006c300 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.069+0000 7f74a5e00640 1 -- 172.21.15.91:0/414296989 >> 172.21.15.91:0/414296989 conn(0x7f74a00cb9f0 msgr2=0x7f74a00cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:55.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.070+0000 7f74a5e00640 1 -- 172.21.15.91:0/414296989 shutdown_connections 2024-02-02T13:30:55.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.070+0000 7f74a5e00640 1 -- 172.21.15.91:0/414296989 wait complete. 2024-02-02T13:30:55.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.071+0000 7f74a5e00640 1 Processor -- start 2024-02-02T13:30:55.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.071+0000 7f74a5e00640 1 -- start start 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74a00b1100 0x7f74a015ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74a015d0a0 0x7f74a0159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a015ef90 con 0x7f74a006d1f0 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0159980 con 0x7f74a015d0a0 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.072+0000 7f74a5e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a015f1c0 con 0x7f74a00b1100 2024-02-02T13:30:55.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55566/0 (socket says 172.21.15.91:55566) 2024-02-02T13:30:55.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 -- 172.21.15.91:0/2646076506 learned_addr learned my addr 172.21.15.91:0/2646076506 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a5400640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74a015d0a0 0x7f74a0159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f749fe00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74a00b1100 0x7f74a015ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 -- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74a00b1100 msgr2=0x7f74a015ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74a00b1100 0x7f74a015ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 -- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74a015d0a0 msgr2=0x7f74a0159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74a015d0a0 0x7f74a0159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.073+0000 7f74a4a00640 1 -- 172.21.15.91:0/2646076506 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009a70 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f74a4a00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f749000b3f0 tx=0x7f749000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490011830 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490011990 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490011d40 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a015f4a0 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.074+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a015fa60 con 0x7f74a006d1f0 2024-02-02T13:30:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.075+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a004dff0 con 0x7f74a006d1f0 2024-02-02T13:30:55.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.076+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f749001a400 con 0x7f74a006d1f0 2024-02-02T13:30:55.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.077+0000 7f749cc00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 0x7f747806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.077+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7490088790 con 0x7f74a006d1f0 2024-02-02T13:30:55.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.077+0000 7f749fe00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 0x7f747806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.078+0000 7f749fe00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 0x7f747806c8f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f74a00adf70 tx=0x7f7488009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.082+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7490053d50 con 0x7f74a006d1f0 2024-02-02T13:30:55.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.274+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74a006cb80 con 0x7f74a006d1f0 2024-02-02T13:30:55.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.276+0000 7f749cc00640 1 -- 172.21.15.91:0/2646076506 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16154 (secure 0 0 0) 0x7f7490057370 con 0x7f74a006d1f0 2024-02-02T13:30:55.278 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:55.278 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":116,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:48.529769+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 msgr2=0x7f747806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 0x7f747806c8f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f74a00adf70 tx=0x7f7488009040 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 msgr2=0x7f74a015e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f749000b3f0 tx=0x7f749000b8b0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 shutdown_connections 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f747806a430 0x7f747806c8f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74a00b1100 0x7f74a015ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74a015d0a0 0x7f74a0159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 --2- 172.21.15.91:0/2646076506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74a006d1f0 0x7f74a015e220 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 >> 172.21.15.91:0/2646076506 conn(0x7f74a00cb9f0 msgr2=0x7f74a00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 shutdown_connections 2024-02-02T13:30:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.279+0000 7f74a5e00640 1 -- 172.21.15.91:0/2646076506 wait complete. 2024-02-02T13:30:55.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:30:15.902582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:30:17.156657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:30:55.300 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:30:55.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.502+0000 7ff055c00640 1 Processor -- start 2024-02-02T13:30:55.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.502+0000 7ff055c00640 1 -- start start 2024-02-02T13:30:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff05015aec0 0x7ff0500cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05006c1c0 con 0x7ff05006f3f0 2024-02-02T13:30:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05006b5f0 con 0x7ff05015b2a0 2024-02-02T13:30:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05006cd60 con 0x7ff05015aec0 2024-02-02T13:30:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff05006f3f0 0x7ff0500ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45840/0 (socket says 172.21.15.91:45840) 2024-02-02T13:30:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff055200640 1 -- 172.21.15.91:0/343742661 learned_addr learned my addr 172.21.15.91:0/343742661 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.504+0000 7ff04fe00640 1 --2- 172.21.15.91:0/343742661 >> v2:172.21.15.91:3300/0 conn(0x7ff05015aec0 0x7ff0500cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.505+0000 7ff04ea00640 1 -- 172.21.15.91:0/343742661 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499201391 0 0) 0x7ff05006c1c0 con 0x7ff05006f3f0 2024-02-02T13:30:55.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.505+0000 7ff04ea00640 1 -- 172.21.15.91:0/343742661 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff030003520 con 0x7ff05006f3f0 2024-02-02T13:30:55.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.505+0000 7ff04fe00640 1 -- 172.21.15.91:0/343742661 >> v1:172.21.15.113:6789/0 conn(0x7ff05015b2a0 legacy=0x7ff05006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.505+0000 7ff04fe00640 1 -- 172.21.15.91:0/343742661 >> v1:172.21.15.91:6789/0 conn(0x7ff05006f3f0 legacy=0x7ff0500ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.505+0000 7ff04fe00640 1 -- 172.21.15.91:0/343742661 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05006b890 con 0x7ff05015aec0 2024-02-02T13:30:55.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.506+0000 7ff04fe00640 1 --2- 172.21.15.91:0/343742661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015aec0 0x7ff0500cf5b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff0400097a0 tx=0x7ff04002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97ecae3807ce251 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.506+0000 7ff04ea00640 1 -- 172.21.15.91:0/343742661 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040030610 con 0x7ff05015aec0 2024-02-02T13:30:55.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.506+0000 7ff04ea00640 1 -- 172.21.15.91:0/343742661 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff040030770 con 0x7ff05015aec0 2024-02-02T13:30:55.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.506+0000 7ff04ea00640 1 -- 172.21.15.91:0/343742661 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040030b00 con 0x7ff05015aec0 2024-02-02T13:30:55.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 -- 172.21.15.91:0/343742661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015aec0 msgr2=0x7ff0500cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 --2- 172.21.15.91:0/343742661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015aec0 0x7ff0500cf5b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff0400097a0 tx=0x7ff04002b960 comp rx=0 tx=0).stop 2024-02-02T13:30:55.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 -- 172.21.15.91:0/343742661 shutdown_connections 2024-02-02T13:30:55.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 --2- 172.21.15.91:0/343742661 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015aec0 0x7ff0500cf5b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 -- 172.21.15.91:0/343742661 >> 172.21.15.91:0/343742661 conn(0x7ff0500cea90 msgr2=0x7ff0500ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:55.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 -- 172.21.15.91:0/343742661 shutdown_connections 2024-02-02T13:30:55.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.507+0000 7ff055c00640 1 -- 172.21.15.91:0/343742661 wait complete. 2024-02-02T13:30:55.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.509+0000 7ff055c00640 1 Processor -- start 2024-02-02T13:30:55.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.509+0000 7ff055c00640 1 -- start start 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05006f3f0 0x7ff0500c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05015aec0 0x7ff0500b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015e460 con 0x7ff05015b2a0 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500c45a0 con 0x7ff05015aec0 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501f68a0 con 0x7ff05006f3f0 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55574/0 (socket says 172.21.15.91:55574) 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff055200640 1 -- 172.21.15.91:0/3448584815 learned_addr learned my addr 172.21.15.91:0/3448584815 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:55.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff04fe00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05006f3f0 0x7ff0500c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.510+0000 7ff04f400640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05015aec0 0x7ff0500b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 -- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05006f3f0 msgr2=0x7ff0500c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05006f3f0 0x7ff0500c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 -- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05015aec0 msgr2=0x7ff0500b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05015aec0 0x7ff0500b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 -- 172.21.15.91:0/3448584815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009400 con 0x7ff05015b2a0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055200640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff04800caa0 tx=0x7ff04800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800ac00 con 0x7ff05015b2a0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800ad60 con 0x7ff05015b2a0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048012620 con 0x7ff05015b2a0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0501f6b80 con 0x7ff05015b2a0 2024-02-02T13:30:55.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.511+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0500c4a60 con 0x7ff05015b2a0 2024-02-02T13:30:55.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.512+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500c4c50 con 0x7ff05015b2a0 2024-02-02T13:30:55.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.513+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff048007070 con 0x7ff05015b2a0 2024-02-02T13:30:55.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.514+0000 7ff047e00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 0x7ff02806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.514+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff0480888a0 con 0x7ff05015b2a0 2024-02-02T13:30:55.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.514+0000 7ff04fe00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 0x7ff02806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.515+0000 7ff04fe00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 0x7ff02806c8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff040036000 tx=0x7ff040008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.518+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff048053f00 con 0x7ff05015b2a0 2024-02-02T13:30:55.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.705+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff05004dff0 con 0x7ff05015b2a0 2024-02-02T13:30:55.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.706+0000 7ff047e00640 1 -- 172.21.15.91:0/3448584815 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff048017020 con 0x7ff05015b2a0 2024-02-02T13:30:55.708 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 msgr2=0x7ff02806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 0x7ff02806c8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff040036000 tx=0x7ff040008030 comp rx=0 tx=0).stop 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 msgr2=0x7ff0501f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff04800caa0 tx=0x7ff04800cf60 comp rx=0 tx=0).stop 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 shutdown_connections 2024-02-02T13:30:55.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02806a430 0x7ff02806c8f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05015b2a0 0x7ff0501f5e70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05015aec0 0x7ff0500b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 --2- 172.21.15.91:0/3448584815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05006f3f0 0x7ff0500c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 >> 172.21.15.91:0/3448584815 conn(0x7ff0500cea90 msgr2=0x7ff0500c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 shutdown_connections 2024-02-02T13:30:55.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.709+0000 7ff055c00640 1 -- 172.21.15.91:0/3448584815 wait complete. 2024-02-02T13:30:55.729 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:30:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.933+0000 7f804ca00640 1 Processor -- start 2024-02-02T13:30:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.934+0000 7f804ca00640 1 -- start start 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f804ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f804415d330 0x7f80440b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f804ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f80440d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f804ca00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044161090 con 0x7f80440b2470 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f804ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8044161cd0 con 0x7f804415d330 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f804ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044161330 con 0x7f80440d3850 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f8049600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f80440b2470 0x7f80440d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45846/0 (socket says 172.21.15.91:45846) 2024-02-02T13:30:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.935+0000 7f8049600640 1 -- 172.21.15.91:0/720631465 learned_addr learned my addr 172.21.15.91:0/720631465 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:55.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f804a000640 1 --2- 172.21.15.91:0/720631465 >> v2:172.21.15.113:3300/0 conn(0x7f804415d330 0x7f80440b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f804aa00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f80440d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005272268 0 0) 0x7f8044161090 con 0x7f80440b2470 2024-02-02T13:30:55.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8024003520 con 0x7f80440b2470 2024-02-02T13:30:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018333183 0 0) 0x7f8024003520 con 0x7f80440b2470 2024-02-02T13:30:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 msgr2=0x7f80440d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f80440d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 >> v2:172.21.15.113:3300/0 conn(0x7f804415d330 msgr2=0x7f80440b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 --2- 172.21.15.91:0/720631465 >> v2:172.21.15.113:3300/0 conn(0x7f804415d330 0x7f80440b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044161610 con 0x7f80440b2470 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f802c001a10 con 0x7f80440b2470 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 >> v1:172.21.15.91:6789/0 conn(0x7f80440b2470 legacy=0x7f80440d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f8024005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044161cd0 con 0x7f80240047c0 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044161330 con 0x7f8024003970 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.936+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802400ba30 con 0x7f80440d3850 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80240047c0 0x7f8024004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f804a000640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 -- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 msgr2=0x7f8024005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f804aa00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f8024005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80440d3850 0x7f8024005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 -- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 msgr2=0x7f8024003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f8049600640 1 -- 172.21.15.91:0/720631465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024003520 con 0x7f80240047c0 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.937+0000 7f804a000640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8024003970 0x7f8024003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.938+0000 7f8049600640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80240047c0 0x7f8024004c40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f802c00a850 tx=0x7f80240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09497d474a917c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.938+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c034630 con 0x7f80240047c0 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.938+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c02f070 con 0x7f80240047c0 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.938+0000 7f8048c00640 1 -- 172.21.15.91:0/720631465 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0349d0 con 0x7f80240047c0 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 -- 172.21.15.91:0/720631465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80240047c0 msgr2=0x7f8024004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 --2- 172.21.15.91:0/720631465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80240047c0 0x7f8024004c40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f802c00a850 tx=0x7f80240057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804aa00640 1 -- 172.21.15.91:0/720631465 reap_dead start 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 -- 172.21.15.91:0/720631465 shutdown_connections 2024-02-02T13:30:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 -- 172.21.15.91:0/720631465 >> 172.21.15.91:0/720631465 conn(0x7f8044070660 msgr2=0x7f8044070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 -- 172.21.15.91:0/720631465 shutdown_connections 2024-02-02T13:30:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.939+0000 7f804ca00640 1 -- 172.21.15.91:0/720631465 wait complete. 2024-02-02T13:30:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.941+0000 7f804ca00640 1 Processor -- start 2024-02-02T13:30:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.941+0000 7f804ca00640 1 -- start start 2024-02-02T13:30:55.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f80440ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 0x7f8044151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044151c80 con 0x7f804415a470 2024-02-02T13:30:55.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440caa30 con 0x7f80240047c0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c6d40 con 0x7f8044159ff0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.942+0000 7f804aa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55602/0 (socket says 172.21.15.91:55602) 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 -- 172.21.15.91:0/4211481674 learned_addr learned my addr 172.21.15.91:0/4211481674 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804a000640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f80440ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f8049600640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 0x7f8044151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 -- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 msgr2=0x7f8044151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 0x7f8044151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 -- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 msgr2=0x7f80440ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f80440ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 -- 172.21.15.91:0/4211481674 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c7020 con 0x7f804415a470 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804a000640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f80440ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f8049600640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 0x7f8044151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f804aa00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80440b6100 tx=0x7f8034010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.943+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400e8a0 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.944+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400ea20 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.944+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80440c7300 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.944+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400edd0 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.944+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80440c78c0 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.945+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440c7a80 con 0x7f804415a470 2024-02-02T13:30:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.946+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8034020410 con 0x7f804415a470 2024-02-02T13:30:55.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.946+0000 7f8042000640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 0x7f801c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:55.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.947+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8034088150 con 0x7f804415a470 2024-02-02T13:30:55.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.947+0000 7f804a000640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 0x7f801c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:55.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.948+0000 7f804a000640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 0x7f801c06c920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f803c009b10 tx=0x7f803c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:55.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:55.951+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8034053680 con 0x7f804415a470 2024-02-02T13:30:56.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.140+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8044067400 con 0x7f804415a470 2024-02-02T13:30:56.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.140+0000 7f8042000640 1 -- 172.21.15.91:0/4211481674 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8034056ca0 con 0x7f804415a470 2024-02-02T13:30:56.142 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:30:56.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 msgr2=0x7f801c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 0x7f801c06c920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f803c009b10 tx=0x7f803c008040 comp rx=0 tx=0).stop 2024-02-02T13:30:56.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 msgr2=0x7f8044151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80440b6100 tx=0x7f8034010c20 comp rx=0 tx=0).stop 2024-02-02T13:30:56.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 shutdown_connections 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f801c06a460 0x7f801c06c920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8044159ff0 0x7f8044151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f804415a470 0x7f8044151740 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 --2- 172.21.15.91:0/4211481674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80240047c0 0x7f80440ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 >> 172.21.15.91:0/4211481674 conn(0x7f8044070660 msgr2=0x7f80440b7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:56.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 shutdown_connections 2024-02-02T13:30:56.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.143+0000 7f804ca00640 1 -- 172.21.15.91:0/4211481674 wait complete. 2024-02-02T13:30:56.164 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:30:56.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.371+0000 7f2065e00640 1 Processor -- start 2024-02-02T13:30:56.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.372+0000 7f2065e00640 1 -- start start 2024-02-02T13:30:56.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f2065e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20600ce130 0x7f20600c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f2065e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600cc7c0 con 0x7f20600b0700 2024-02-02T13:30:56.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f2065e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600ce930 con 0x7f206015a010 2024-02-02T13:30:56.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f2065e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600cd640 con 0x7f20600ce130 2024-02-02T13:30:56.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f205f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f20600b0700 0x7f206006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45862/0 (socket says 172.21.15.91:45862) 2024-02-02T13:30:56.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f205f400640 1 -- 172.21.15.91:0/3995747834 learned_addr learned my addr 172.21.15.91:0/3995747834 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148569547 0 0) 0x7f20600cc7c0 con 0x7f20600b0700 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2040005730 con 0x7f20600b0700 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.373+0000 7f2057e00640 1 --2- 172.21.15.91:0/3995747834 >> v2:172.21.15.113:3300/0 conn(0x7f20600ce130 0x7f20600c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3968557615 0 0) 0x7f2040005730 con 0x7f20600b0700 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 >> v2:172.21.15.113:3300/0 conn(0x7f20600ce130 msgr2=0x7f20600c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 --2- 172.21.15.91:0/3995747834 >> v2:172.21.15.113:3300/0 conn(0x7f20600ce130 0x7f20600c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 >> v1:172.21.15.113:6789/0 conn(0x7f206015a010 legacy=0x7f20600ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206006c890 con 0x7f20600b0700 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f20580019f0 con 0x7f20600b0700 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 >> v1:172.21.15.91:6789/0 conn(0x7f20600b0700 legacy=0x7f206006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2040005b80 0x7f2040005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 0x7f2040006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2040007390 0x7f204000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600cd640 con 0x7f20400069d0 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204000e2e0 con 0x7f2040005b80 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204000e4a0 con 0x7f2040007390 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.374+0000 7f2057e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 0x7f2040006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f2057e00640 1 -- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2040007390 msgr2=0x7f204000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f2057e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2040007390 0x7f204000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f2057e00640 1 -- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2040005b80 msgr2=0x7f2040005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f2057e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2040005b80 0x7f2040005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f2057e00640 1 -- 172.21.15.91:0/3995747834 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040005730 con 0x7f20400069d0 2024-02-02T13:30:56.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.375+0000 7f205f400640 1 -- 172.21.15.91:0/3995747834 reap_dead start 2024-02-02T13:30:56.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2057e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 0x7f2040006e50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f204800ba30 tx=0x7f2040007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698da68fff2f8a24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048032640 con 0x7f20400069d0 2024-02-02T13:30:56.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480327a0 con 0x7f20400069d0 2024-02-02T13:30:56.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f205e000640 1 -- 172.21.15.91:0/3995747834 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048032b30 con 0x7f20400069d0 2024-02-02T13:30:56.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2065e00640 1 -- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 msgr2=0x7f2040006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2065e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 0x7f2040006e50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f204800ba30 tx=0x7f2040007a00 comp rx=0 tx=0).stop 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2065e00640 1 -- 172.21.15.91:0/3995747834 shutdown_connections 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2065e00640 1 --2- 172.21.15.91:0/3995747834 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20400069d0 0x7f2040006e50 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.376+0000 7f2065e00640 1 -- 172.21.15.91:0/3995747834 >> 172.21.15.91:0/3995747834 conn(0x7f20600cec80 msgr2=0x7f20600cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.377+0000 7f2065e00640 1 -- 172.21.15.91:0/3995747834 shutdown_connections 2024-02-02T13:30:56.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.377+0000 7f2065e00640 1 -- 172.21.15.91:0/3995747834 wait complete. 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.378+0000 7f2065e00640 1 Processor -- start 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.378+0000 7f2065e00640 1 -- start start 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 0x7f20600c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 0x7f206015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206015e3a0 con 0x7f20600c8360 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206014efc0 con 0x7f206015d9b0 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2065e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206014df40 con 0x7f20400069d0 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2057e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2057e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55632/0 (socket says 172.21.15.91:55632) 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.379+0000 7f2057e00640 1 -- 172.21.15.91:0/681905989 learned_addr learned my addr 172.21.15.91:0/681905989 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f205ea00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 0x7f20600c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 -- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 msgr2=0x7f20600c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f205f400640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 0x7f206015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 0x7f20600c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 -- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 msgr2=0x7f206015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 0x7f206015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 -- 172.21.15.91:0/681905989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206014dc70 con 0x7f20600c8360 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f205ea00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 0x7f20600c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f205f400640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 0x7f206015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.380+0000 7f2057e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2048035040 tx=0x7f2048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.381+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204803baf0 con 0x7f20600c8360 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.381+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204803bc50 con 0x7f20600c8360 2024-02-02T13:30:56.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.381+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048030410 con 0x7f20600c8360 2024-02-02T13:30:56.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.381+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206014e200 con 0x7f20600c8360 2024-02-02T13:30:56.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.381+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206014e7c0 con 0x7f20600c8360 2024-02-02T13:30:56.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.382+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2060066cf0 con 0x7f20600c8360 2024-02-02T13:30:56.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.383+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2048030570 con 0x7f20600c8360 2024-02-02T13:30:56.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.384+0000 7f2057400640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 0x7f203806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.384+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f20480ab350 con 0x7f20600c8360 2024-02-02T13:30:56.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.384+0000 7f205ea00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 0x7f203806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.384+0000 7f205ea00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 0x7f203806c8f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f206004f8f0 tx=0x7f2050009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.388+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2048076920 con 0x7f20600c8360 2024-02-02T13:30:56.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.563+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20600b4c00 con 0x7f20600c8360 2024-02-02T13:30:56.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.564+0000 7f2057400640 1 -- 172.21.15.91:0/681905989 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2048079f40 con 0x7f20600c8360 2024-02-02T13:30:56.565 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 msgr2=0x7f203806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 0x7f203806c8f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f206004f8f0 tx=0x7f2050009040 comp rx=0 tx=0).stop 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 msgr2=0x7f20600c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2048035040 tx=0x7f2048004730 comp rx=0 tx=0).stop 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 shutdown_connections 2024-02-02T13:30:56.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f203806a430 0x7f203806c8f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f206015d9b0 0x7f206015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20600c8360 0x7f20600c87a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 --2- 172.21.15.91:0/681905989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20400069d0 0x7f20600c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.566+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 >> 172.21.15.91:0/681905989 conn(0x7f20600cec80 msgr2=0x7f20600c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.567+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 shutdown_connections 2024-02-02T13:30:56.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.567+0000 7f2065e00640 1 -- 172.21.15.91:0/681905989 wait complete. 2024-02-02T13:30:56.586 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:30:56.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.779+0000 7fc527200640 1 Processor -- start 2024-02-02T13:30:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.779+0000 7fc527200640 1 -- start start 2024-02-02T13:30:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc527200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc52815d0d0 0x7fc5280cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc527200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5280b8750 0x7fc5280cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc527200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280cde20 con 0x7fc5280b8bb0 2024-02-02T13:30:56.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc527200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280d2a20 con 0x7fc52815d0d0 2024-02-02T13:30:56.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc527200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52815d4b0 con 0x7fc5280b8750 2024-02-02T13:30:56.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc526800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc5280b8bb0 0x7fc5280d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45870/0 (socket says 172.21.15.91:45870) 2024-02-02T13:30:56.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc526800640 1 -- 172.21.15.91:0/164897380 learned_addr learned my addr 172.21.15.91:0/164897380 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.781+0000 7fc525e00640 1 --2- 172.21.15.91:0/164897380 >> v2:172.21.15.91:3300/0 conn(0x7fc52815d0d0 0x7fc5280cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc51d200640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5280b8750 0x7fc5280cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc525e00640 1 -- 172.21.15.91:0/164897380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5280b8750 msgr2=0x7fc5280cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc525e00640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5280b8750 0x7fc5280cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc525e00640 1 -- 172.21.15.91:0/164897380 >> v1:172.21.15.91:6789/0 conn(0x7fc5280b8bb0 legacy=0x7fc5280d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc525e00640 1 -- 172.21.15.91:0/164897380 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280d23f0 con 0x7fc52815d0d0 2024-02-02T13:30:56.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.782+0000 7fc525400640 1 -- 172.21.15.91:0/164897380 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583847118 0 0) 0x7fc5280cde20 con 0x7fc5280b8bb0 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.783+0000 7fc525e00640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc52815d0d0 0x7fc5280cd580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc5180097a0 tx=0x7fc51802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975a7cbd8dff6e12 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.783+0000 7fc525400640 1 -- 172.21.15.91:0/164897380 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518031830 con 0x7fc52815d0d0 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.783+0000 7fc525400640 1 -- 172.21.15.91:0/164897380 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518031990 con 0x7fc52815d0d0 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.783+0000 7fc525400640 1 -- 172.21.15.91:0/164897380 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518031d20 con 0x7fc52815d0d0 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 -- 172.21.15.91:0/164897380 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc52815d0d0 msgr2=0x7fc5280cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc52815d0d0 0x7fc5280cd580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc5180097a0 tx=0x7fc51802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 -- 172.21.15.91:0/164897380 shutdown_connections 2024-02-02T13:30:56.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5280b8750 0x7fc5280cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 --2- 172.21.15.91:0/164897380 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc52815d0d0 0x7fc5280cd580 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 -- 172.21.15.91:0/164897380 >> 172.21.15.91:0/164897380 conn(0x7fc528065910 msgr2=0x7fc528065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:56.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 -- 172.21.15.91:0/164897380 shutdown_connections 2024-02-02T13:30:56.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.784+0000 7fc527200640 1 -- 172.21.15.91:0/164897380 wait complete. 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.785+0000 7fc527200640 1 Processor -- start 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 -- start start 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5280b8750 0x7fc5281f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc52815d0d0 0x7fc5280c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280c82d0 con 0x7fc5280c7860 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280c7d10 con 0x7fc5280b8750 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.786+0000 7fc527200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280c7ff0 con 0x7fc52815d0d0 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55640/0 (socket says 172.21.15.91:55640) 2024-02-02T13:30:56.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 -- 172.21.15.91:0/2552094278 learned_addr learned my addr 172.21.15.91:0/2552094278 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc525e00640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5280b8750 0x7fc5281f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc51d200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc52815d0d0 0x7fc5280c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 -- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc52815d0d0 msgr2=0x7fc5280c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc52815d0d0 0x7fc5280c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 -- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5280b8750 msgr2=0x7fc5281f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5280b8750 0x7fc5281f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.787+0000 7fc526800640 1 -- 172.21.15.91:0/2552094278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518009400 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc526800640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c430 tx=0x7fc52000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5200074e0 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520007660 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5280b3a80 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520004a20 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.788+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281f6740 con 0x7fc5280c7860 2024-02-02T13:30:56.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.789+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5280b38e0 con 0x7fc5280c7860 2024-02-02T13:30:56.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.790+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc520004b80 con 0x7fc5280c7860 2024-02-02T13:30:56.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.791+0000 7fc51ea00640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 0x7fc50006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:56.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.791+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc5200888a0 con 0x7fc5280c7860 2024-02-02T13:30:56.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.791+0000 7fc525e00640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 0x7fc50006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:56.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.792+0000 7fc525e00640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 0x7fc50006c8f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc518036000 tx=0x7fc51803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:56.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.795+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc5280b38e0 con 0x7fc5280c7860 2024-02-02T13:30:56.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.980+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5280b3fa0 con 0x7fc5280c7860 2024-02-02T13:30:56.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.981+0000 7fc51ea00640 1 -- 172.21.15.91:0/2552094278 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc5280b3fa0 con 0x7fc5280c7860 2024-02-02T13:30:56.982 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 msgr2=0x7fc50006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 0x7fc50006c8f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc518036000 tx=0x7fc51803c040 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 msgr2=0x7fc52814ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c430 tx=0x7fc52000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 shutdown_connections 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc50006a430 0x7fc50006c8f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc52815d0d0 0x7fc5280c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc5280c7860 0x7fc52814ee60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 --2- 172.21.15.91:0/2552094278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5280b8750 0x7fc5281f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 >> 172.21.15.91:0/2552094278 conn(0x7fc528065910 msgr2=0x7fc5280b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:56.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.983+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 shutdown_connections 2024-02-02T13:30:56.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:56.984+0000 7fc527200640 1 -- 172.21.15.91:0/2552094278 wait complete. 2024-02-02T13:30:57.002 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:30:57.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:57.003 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:30:57.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:30:57.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:57.152 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:30:57.190 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:57.190 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:57.229 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:57.229 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:57.320 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:30:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.523+0000 7f065d200640 1 Processor -- start 2024-02-02T13:30:57.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.524+0000 7f065d200640 1 -- start start 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f065d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 0x7f065815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f065d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f065806f4f0 0x7f06580b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f065d200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06580b1070 con 0x7f06580b0c40 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f065d200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0658070f80 con 0x7f065806f4f0 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f065d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0658152370 con 0x7f0658151f90 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f064fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f065806f4f0 0x7f06580b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f064fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f065806f4f0 0x7f06580b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55648/0 (socket says 172.21.15.91:55648) 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f064fe00640 1 -- 172.21.15.91:0/3824641648 learned_addr learned my addr 172.21.15.91:0/3824641648 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.525+0000 7f0656800640 1 --1- 172.21.15.91:0/3824641648 >> v1:172.21.15.91:6789/0 conn(0x7f06580b0c40 0x7f06580b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45884/0 (socket says 172.21.15.91:45884) 2024-02-02T13:30:57.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f0655e00640 1 --2- 172.21.15.91:0/3824641648 >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 0x7f065815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f064fe00640 1 -- 172.21.15.91:0/3824641648 >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 msgr2=0x7f065815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f064fe00640 1 --2- 172.21.15.91:0/3824641648 >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 0x7f065815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f064fe00640 1 -- 172.21.15.91:0/3824641648 >> v1:172.21.15.91:6789/0 conn(0x7f06580b0c40 legacy=0x7f06580b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f064fe00640 1 -- 172.21.15.91:0/3824641648 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580cb010 con 0x7f065806f4f0 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f0655e00640 1 --2- 172.21.15.91:0/3824641648 >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 0x7f065815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.526+0000 7f0655400640 1 -- 172.21.15.91:0/3824641648 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542276482 0 0) 0x7f06580b1070 con 0x7f06580b0c40 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.527+0000 7f064fe00640 1 --2- 172.21.15.91:0/3824641648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f065806f4f0 0x7f06580b0700 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f06580cc770 tx=0x7f064002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8b17f6637be4e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.527+0000 7f0655400640 1 -- 172.21.15.91:0/3824641648 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640030870 con 0x7f065806f4f0 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.527+0000 7f0655400640 1 -- 172.21.15.91:0/3824641648 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06400309d0 con 0x7f065806f4f0 2024-02-02T13:30:57.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.527+0000 7f0655400640 1 -- 172.21.15.91:0/3824641648 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640030d60 con 0x7f065806f4f0 2024-02-02T13:30:57.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 -- 172.21.15.91:0/3824641648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f065806f4f0 msgr2=0x7f06580b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 --2- 172.21.15.91:0/3824641648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f065806f4f0 0x7f06580b0700 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f06580cc770 tx=0x7f064002b9b0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 -- 172.21.15.91:0/3824641648 shutdown_connections 2024-02-02T13:30:57.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 --2- 172.21.15.91:0/3824641648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f065806f4f0 0x7f06580b0700 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 --2- 172.21.15.91:0/3824641648 >> v2:172.21.15.113:3300/0 conn(0x7f0658151f90 0x7f065815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 -- 172.21.15.91:0/3824641648 >> 172.21.15.91:0/3824641648 conn(0x7f06580cec80 msgr2=0x7f06580cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:57.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 -- 172.21.15.91:0/3824641648 shutdown_connections 2024-02-02T13:30:57.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.528+0000 7f065d200640 1 -- 172.21.15.91:0/3824641648 wait complete. 2024-02-02T13:30:57.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.530+0000 7f065d200640 1 Processor -- start 2024-02-02T13:30:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.530+0000 7f065d200640 1 -- start start 2024-02-02T13:30:57.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 0x7f06581576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 0x7f06580c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580c70e0 con 0x7f06580b0c40 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580c6200 con 0x7f065806f4f0 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.532+0000 7f065d200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06580c64d0 con 0x7f0658151f90 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55650/0 (socket says 172.21.15.91:55650) 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 -- 172.21.15.91:0/1492848107 learned_addr learned my addr 172.21.15.91:0/1492848107 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f0655e00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 0x7f06581576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f0656800640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 0x7f06580c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 -- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 msgr2=0x7f06580c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 0x7f06580c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 -- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 msgr2=0x7f06581576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 0x7f06581576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f064fe00640 1 -- 172.21.15.91:0/1492848107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400094d0 con 0x7f06580b0c40 2024-02-02T13:30:57.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f0655e00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 0x7f06581576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.533+0000 7f0656800640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 0x7f06580c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f064fe00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f064002b9f0 tx=0x7f0640004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640030860 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640039d90 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06580c67b0 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640043650 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.534+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06580c6d70 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.535+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06580c7290 con 0x7f06580b0c40 2024-02-02T13:30:57.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.536+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f064002f3c0 con 0x7f06580b0c40 2024-02-02T13:30:57.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.537+0000 7f064e000640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 0x7f063006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.537+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f06400ab130 con 0x7f06580b0c40 2024-02-02T13:30:57.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.537+0000 7f0655e00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 0x7f063006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.537+0000 7f0655e00640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 0x7f063006c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f064800aa30 tx=0x7f0648008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.541+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0640076660 con 0x7f06580b0c40 2024-02-02T13:30:57.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.720+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f065806f980 con 0x7f06580b0c40 2024-02-02T13:30:57.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.721+0000 7f064e000640 1 -- 172.21.15.91:0/1492848107 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16154 (secure 0 0 0) 0x7f0640079c80 con 0x7f06580b0c40 2024-02-02T13:30:57.723 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:30:57.723 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":116,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:48.529769+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 msgr2=0x7f063006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 0x7f063006c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f064800aa30 tx=0x7f0648008040 comp rx=0 tx=0).stop 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 msgr2=0x7f06580c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f064002b9f0 tx=0x7f0640004730 comp rx=0 tx=0).stop 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 shutdown_connections 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f063006a430 0x7f063006c8f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0658151f90 0x7f06580c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06580b0c40 0x7f06580c5cc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 --2- 172.21.15.91:0/1492848107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f065806f4f0 0x7f06581576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 >> 172.21.15.91:0/1492848107 conn(0x7f06580cec80 msgr2=0x7f0658073ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:57.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.724+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 shutdown_connections 2024-02-02T13:30:57.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.725+0000 7f065d200640 1 -- 172.21.15.91:0/1492848107 wait complete. 2024-02-02T13:30:57.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:30:15.902582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:30:17.156657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:30:57.745 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:30:57.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.953+0000 7faddf200640 1 Processor -- start 2024-02-02T13:30:57.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.953+0000 7faddf200640 1 -- start start 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.954+0000 7faddf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fade00cdcc0 0x7fade00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.954+0000 7faddf200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fade015b1e0 0x7fade00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.954+0000 7faddf200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade00cc920 con 0x7fade015ae00 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.954+0000 7faddf200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00cd2f0 con 0x7fade015b1e0 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.954+0000 7faddf200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00cba70 con 0x7fade00cdcc0 2024-02-02T13:30:57.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fade015b1e0 0x7fade00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fade015b1e0 0x7fade00cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55662/0 (socket says 172.21.15.91:55662) 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 -- 172.21.15.91:0/3222796535 learned_addr learned my addr 172.21.15.91:0/3222796535 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 -- 172.21.15.91:0/3222796535 >> v2:172.21.15.113:3300/0 conn(0x7fade00cdcc0 msgr2=0x7fade00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadd7e00640 1 --2- 172.21.15.91:0/3222796535 >> v2:172.21.15.113:3300/0 conn(0x7fade00cdcc0 0x7fade00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 --2- 172.21.15.91:0/3222796535 >> v2:172.21.15.113:3300/0 conn(0x7fade00cdcc0 0x7fade00ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 -- 172.21.15.91:0/3222796535 >> v1:172.21.15.113:6789/0 conn(0x7fade015ae00 legacy=0x7fade00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.955+0000 7fadde800640 1 -- 172.21.15.91:0/3222796535 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00cbd10 con 0x7fade015b1e0 2024-02-02T13:30:57.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.956+0000 7fadde800640 1 --2- 172.21.15.91:0/3222796535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade015b1e0 0x7fade00cf470 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fade00ce200 tx=0x7fadd802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41370f0b069e43ac server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.957+0000 7faddd400640 1 -- 172.21.15.91:0/3222796535 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd802cbb0 con 0x7fade015b1e0 2024-02-02T13:30:57.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.957+0000 7faddd400640 1 -- 172.21.15.91:0/3222796535 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd802cd10 con 0x7fade015b1e0 2024-02-02T13:30:57.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.957+0000 7faddd400640 1 -- 172.21.15.91:0/3222796535 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8034610 con 0x7fade015b1e0 2024-02-02T13:30:57.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.957+0000 7faddf200640 1 -- 172.21.15.91:0/3222796535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade015b1e0 msgr2=0x7fade00cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.957+0000 7faddf200640 1 --2- 172.21.15.91:0/3222796535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade015b1e0 0x7fade00cf470 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fade00ce200 tx=0x7fadd802c010 comp rx=0 tx=0).stop 2024-02-02T13:30:57.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 -- 172.21.15.91:0/3222796535 shutdown_connections 2024-02-02T13:30:57.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 --2- 172.21.15.91:0/3222796535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade015b1e0 0x7fade00cf470 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 --2- 172.21.15.91:0/3222796535 >> v2:172.21.15.113:3300/0 conn(0x7fade00cdcc0 0x7fade00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 -- 172.21.15.91:0/3222796535 >> 172.21.15.91:0/3222796535 conn(0x7fade00d0c80 msgr2=0x7fade00d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:57.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 -- 172.21.15.91:0/3222796535 shutdown_connections 2024-02-02T13:30:57.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.958+0000 7faddf200640 1 -- 172.21.15.91:0/3222796535 wait complete. 2024-02-02T13:30:57.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.959+0000 7faddf200640 1 Processor -- start 2024-02-02T13:30:57.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.960+0000 7faddf200640 1 -- start start 2024-02-02T13:30:57.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.960+0000 7faddf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.960+0000 7faddf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 0x7fade00ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddf200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade015b1e0 0x7fade006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddf200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c9c90 con 0x7fade00cdcc0 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01f0480 con 0x7fade015ae00 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddf200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c8da0 con 0x7fade015b1e0 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55678/0 (socket says 172.21.15.91:55678) 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 -- 172.21.15.91:0/2427618197 learned_addr learned my addr 172.21.15.91:0/2427618197 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 -- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade015b1e0 msgr2=0x7fade006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade015b1e0 0x7fade006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 -- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 msgr2=0x7fade00ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7fadd7e00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 0x7fade00ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 0x7fade00ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7faddde00640 1 -- 172.21.15.91:0/2427618197 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8009b30 con 0x7fade00cdcc0 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.961+0000 7fadd7e00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 0x7fade00ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:57.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7faddde00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c450 tx=0x7fadc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80052d0 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8005430 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade00c9080 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8012870 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.962+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade00c9640 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.963+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade004dff0 con 0x7fade00cdcc0 2024-02-02T13:30:57.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.964+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fadc80129d0 con 0x7fade00cdcc0 2024-02-02T13:30:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.965+0000 7fadd6000640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.965+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fadc8088430 con 0x7fade00cdcc0 2024-02-02T13:30:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.965+0000 7fadd7e00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:57.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.966+0000 7fadd7e00640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fade0154240 tx=0x7fadd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:57.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:57.969+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fadc8053a00 con 0x7fade00cdcc0 2024-02-02T13:30:58.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.161+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fade015a720 con 0x7fade00cdcc0 2024-02-02T13:30:58.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.163+0000 7fadd6000640 1 -- 172.21.15.91:0/2427618197 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadc8057020 con 0x7fade00cdcc0 2024-02-02T13:30:58.165 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 msgr2=0x7fadb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fade0154240 tx=0x7fadd0008040 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 msgr2=0x7fade01eff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c450 tx=0x7fadc800c910 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 shutdown_connections 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadb806a430 0x7fadb806c8f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade015b1e0 0x7fade006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade015ae00 0x7fade00ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 --2- 172.21.15.91:0/2427618197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fade00cdcc0 0x7fade01eff40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 >> 172.21.15.91:0/2427618197 conn(0x7fade00d0c80 msgr2=0x7fade00d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 shutdown_connections 2024-02-02T13:30:58.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.166+0000 7faddf200640 1 -- 172.21.15.91:0/2427618197 wait complete. 2024-02-02T13:30:58.185 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:30:58.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.376+0000 7fabc8c00640 1 Processor -- start 2024-02-02T13:30:58.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.376+0000 7fabc8c00640 1 -- start start 2024-02-02T13:30:58.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabc40b2260 0x7fabc40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc8c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40cd930 con 0x7fabc40b6260 2024-02-02T13:30:58.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40d1680 con 0x7fabc40d0070 2024-02-02T13:30:58.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc40b8f90 con 0x7fabc40b2260 2024-02-02T13:30:58.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc2800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fabc40b6260 0x7fabc40d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45886/0 (socket says 172.21.15.91:45886) 2024-02-02T13:30:58.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc2800640 1 -- 172.21.15.91:0/511412684 learned_addr learned my addr 172.21.15.91:0/511412684 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:58.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc1400640 1 --2- 172.21.15.91:0/511412684 >> v2:172.21.15.113:3300/0 conn(0x7fabc40b2260 0x7fabc40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.378+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841404140 0 0) 0x7fabbc001110 con 0x7fabc40b6260 2024-02-02T13:30:58.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba4003520 con 0x7fabc40b6260 2024-02-02T13:30:58.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86679806 0 0) 0x7faba4003520 con 0x7fabc40b6260 2024-02-02T13:30:58.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 >> v2:172.21.15.113:3300/0 conn(0x7fabc40b2260 msgr2=0x7fabc40b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 --2- 172.21.15.91:0/511412684 >> v2:172.21.15.113:3300/0 conn(0x7fabc40b2260 0x7fabc40b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 >> v1:172.21.15.113:6789/0 conn(0x7fabc40d0070 legacy=0x7fabc40d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40d1960 con 0x7fabc40b6260 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fabbc001800 con 0x7fabc40b6260 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 >> v1:172.21.15.91:6789/0 conn(0x7fabc40b6260 legacy=0x7fabc40d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 0x7faba4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faba40047c0 0x7faba4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4005180 0x7faba400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40b8f90 con 0x7faba4003970 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba400c020 con 0x7faba40047c0 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba400c1e0 con 0x7faba4005180 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.379+0000 7fabc1e00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 0x7faba4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc2800640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4005180 0x7faba400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1400640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faba40047c0 0x7faba4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1e00640 1 -- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4005180 msgr2=0x7faba400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1e00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4005180 0x7faba400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1e00640 1 -- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faba40047c0 msgr2=0x7faba4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1e00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faba40047c0 0x7faba4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1e00640 1 -- 172.21.15.91:0/511412684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4003520 con 0x7faba4003970 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc1400640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faba40047c0 0x7faba4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc2800640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4005180 0x7faba400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:58.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.380+0000 7fabc2800640 1 -- 172.21.15.91:0/511412684 reap_dead start 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.381+0000 7fabc1e00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 0x7faba4003d90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001e50 tx=0x7faba40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624e92c91fd4f62f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4003b70 con 0x7faba4003970 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4003cd0 con 0x7faba4003970 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc0a00640 1 -- 172.21.15.91:0/511412684 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005310 con 0x7faba4003970 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc8c00640 1 -- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 msgr2=0x7faba4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc8c00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 0x7faba4003d90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001e50 tx=0x7faba40057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc8c00640 1 -- 172.21.15.91:0/511412684 shutdown_connections 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc8c00640 1 --2- 172.21.15.91:0/511412684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faba4003970 0x7faba4003d90 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.382+0000 7fabc8c00640 1 -- 172.21.15.91:0/511412684 >> 172.21.15.91:0/511412684 conn(0x7fabc4070660 msgr2=0x7fabc4070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.383+0000 7fabc8c00640 1 -- 172.21.15.91:0/511412684 shutdown_connections 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.383+0000 7fabc8c00640 1 -- 172.21.15.91:0/511412684 wait complete. 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.384+0000 7fabc8c00640 1 Processor -- start 2024-02-02T13:30:58.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.385+0000 7fabc8c00640 1 -- start start 2024-02-02T13:30:58.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.385+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4003970 0x7fabc4157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.385+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.385+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 0x7fabc40cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.385+0000 7fabc8c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415d840 con 0x7fabc40cb0e0 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc8c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40cb650 con 0x7fabc40c7a30 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc8c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415da90 con 0x7faba4003970 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55696/0 (socket says 172.21.15.91:55696) 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 -- 172.21.15.91:0/2227622462 learned_addr learned my addr 172.21.15.91:0/2227622462 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1e00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4003970 0x7fabc4157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 -- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4003970 msgr2=0x7fabc4157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc2800640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 0x7fabc40cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4003970 0x7fabc4157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 -- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 msgr2=0x7fabc40cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 0x7fabc40cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 -- 172.21.15.91:0/2227622462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc415dd70 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc2800640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 0x7fabc40cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.386+0000 7fabc1400640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fabac0109d0 tx=0x7fabac010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.387+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00ec20 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.387+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac00ed80 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.387+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac01c620 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.387+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc415e050 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.387+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41606e0 con 0x7fabc40cb0e0 2024-02-02T13:30:58.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.388+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4066cf0 con 0x7fabc40cb0e0 2024-02-02T13:30:58.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.389+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fabac01c780 con 0x7fabc40cb0e0 2024-02-02T13:30:58.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.390+0000 7fabba000640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 0x7fab9c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.390+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fabac088f10 con 0x7fabc40cb0e0 2024-02-02T13:30:58.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.390+0000 7fabc1e00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 0x7fab9c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.391+0000 7fabc1e00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 0x7fab9c06c8f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fabb4034000 tx=0x7fabb4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.394+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fabac054570 con 0x7fabc40cb0e0 2024-02-02T13:30:58.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.582+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabc4159ba0 con 0x7fabc40cb0e0 2024-02-02T13:30:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.583+0000 7fabba000640 1 -- 172.21.15.91:0/2227622462 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabac057b90 con 0x7fabc40cb0e0 2024-02-02T13:30:58.585 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 msgr2=0x7fab9c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 0x7fab9c06c8f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fabb4034000 tx=0x7fabb4009030 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 msgr2=0x7fabc41601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fabac0109d0 tx=0x7fabac010e90 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 shutdown_connections 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fab9c06a430 0x7fab9c06c8f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc40c7a30 0x7fabc40cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fabc40cb0e0 0x7fabc41601a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 --2- 172.21.15.91:0/2227622462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faba4003970 0x7fabc4157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 >> 172.21.15.91:0/2227622462 conn(0x7fabc4070660 msgr2=0x7fabc40b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:58.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 shutdown_connections 2024-02-02T13:30:58.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.586+0000 7fabc8c00640 1 -- 172.21.15.91:0/2227622462 wait complete. 2024-02-02T13:30:58.606 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:30:58.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.813+0000 7f7a72000640 1 Processor -- start 2024-02-02T13:30:58.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.813+0000 7f7a72000640 1 -- start start 2024-02-02T13:30:58.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a72000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a6c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a72000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c070f80 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a72000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c06c600 con 0x7f7a6c151f90 2024-02-02T13:30:58.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a72000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c15fa30 con 0x7f7a6c0b0e10 2024-02-02T13:30:58.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a6aa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7a6c15f5a0 0x7f7a6c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45902/0 (socket says 172.21.15.91:45902) 2024-02-02T13:30:58.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a6aa00640 1 -- 172.21.15.91:0/4258545012 learned_addr learned my addr 172.21.15.91:0/4258545012 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:58.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a6be00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a6c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.815+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338737924 0 0) 0x7f7a6c070f80 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c003520 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2700877257 0 0) 0x7f7a4c003520 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 msgr2=0x7f7a6c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a6c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 >> v1:172.21.15.113:6789/0 conn(0x7f7a6c151f90 legacy=0x7f7a6c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c152370 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f7a54001670 con 0x7f7a6c15f5a0 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 >> v1:172.21.15.91:6789/0 conn(0x7f7a6c15f5a0 legacy=0x7f7a6c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a4c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a4c003f20 0x7f7a4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c15fa30 con 0x7f7a4c004d00 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00ba60 con 0x7f7a4c003f20 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00bc20 con 0x7f7a6c0b0e10 2024-02-02T13:30:58.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.816+0000 7f7a6be00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6b400640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a4c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6aa00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a4c003f20 0x7f7a4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6be00640 1 -- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 msgr2=0x7f7a4c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6be00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c0b0e10 0x7f7a4c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6be00640 1 -- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a4c003f20 msgr2=0x7f7a4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6be00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a4c003f20 0x7f7a4c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6be00640 1 -- 172.21.15.91:0/4258545012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c003520 con 0x7f7a4c004d00 2024-02-02T13:30:58.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.817+0000 7f7a6aa00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a4c003f20 0x7f7a4c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:58.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.818+0000 7f7a6be00640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a4c005180 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0ca540 tx=0x7f7a4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2aab85e453bfc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.818+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6402c660 con 0x7f7a4c004d00 2024-02-02T13:30:58.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.818+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6402c7c0 con 0x7f7a4c004d00 2024-02-02T13:30:58.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.818+0000 7f7a6a000640 1 -- 172.21.15.91:0/4258545012 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6402cb70 con 0x7f7a4c004d00 2024-02-02T13:30:58.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 -- 172.21.15.91:0/4258545012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 msgr2=0x7f7a4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 --2- 172.21.15.91:0/4258545012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a4c005180 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0ca540 tx=0x7f7a4c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a6be00640 1 -- 172.21.15.91:0/4258545012 reap_dead start 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 -- 172.21.15.91:0/4258545012 shutdown_connections 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 -- 172.21.15.91:0/4258545012 >> 172.21.15.91:0/4258545012 conn(0x7f7a6c0cec80 msgr2=0x7f7a6c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 -- 172.21.15.91:0/4258545012 shutdown_connections 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.819+0000 7f7a72000640 1 -- 172.21.15.91:0/4258545012 wait complete. 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.821+0000 7f7a72000640 1 Processor -- start 2024-02-02T13:30:58.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.821+0000 7f7a72000640 1 -- start start 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c14da90 0x7f7a6c0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c0c7850 0x7f7a6c0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0c3540 con 0x7f7a4c004d00 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0c7cd0 con 0x7f7a6c0c7850 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a72000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0c28c0 con 0x7f7a6c14da90 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55702/0 (socket says 172.21.15.91:55702) 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 -- 172.21.15.91:0/1256437681 learned_addr learned my addr 172.21.15.91:0/1256437681 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6aa00640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c14da90 0x7f7a6c0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6be00640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c0c7850 0x7f7a6c0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 -- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c14da90 msgr2=0x7f7a6c0c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c14da90 0x7f7a6c0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 -- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c0c7850 msgr2=0x7f7a6c0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:58.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c0c7850 0x7f7a6c0c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.822+0000 7f7a6b400640 1 -- 172.21.15.91:0/1256437681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0c2ba0 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a6b400640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e8c0 tx=0x7f7a5c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c003b80 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c003ce0 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0052f0 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c0c2e80 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.823+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c14df10 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.824+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c1f28f0 con 0x7f7a4c004d00 2024-02-02T13:30:58.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.825+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a5c005450 con 0x7f7a4c004d00 2024-02-02T13:30:58.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.826+0000 7f7a63400640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 0x7f7a4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:58.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.826+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7a5c088ad0 con 0x7f7a4c004d00 2024-02-02T13:30:58.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.826+0000 7f7a6aa00640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 0x7f7a4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:58.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.827+0000 7f7a6aa00640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 0x7f7a4406c8f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7a54001eb0 tx=0x7f7a54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:58.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:58.830+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a5c054090 con 0x7f7a4c004d00 2024-02-02T13:30:59.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.017+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a6c0c2a00 con 0x7f7a4c004d00 2024-02-02T13:30:59.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.018+0000 7f7a63400640 1 -- 172.21.15.91:0/1256437681 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a6c0c2a00 con 0x7f7a4c004d00 2024-02-02T13:30:59.020 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 msgr2=0x7f7a4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 0x7f7a4406c8f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7a54001eb0 tx=0x7f7a54009040 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 msgr2=0x7f7a6c14e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e8c0 tx=0x7f7a5c00ed80 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 shutdown_connections 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7a4406a430 0x7f7a4406c8f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a6c14da90 0x7f7a6c0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c0c7850 0x7f7a6c0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 --2- 172.21.15.91:0/1256437681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a4c004d00 0x7f7a6c14e220 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 >> 172.21.15.91:0/1256437681 conn(0x7f7a6c0cec80 msgr2=0x7f7a6c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 shutdown_connections 2024-02-02T13:30:59.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.021+0000 7f7a72000640 1 -- 172.21.15.91:0/1256437681 wait complete. 2024-02-02T13:30:59.040 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:30:59.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.247+0000 7f8219600640 1 Processor -- start 2024-02-02T13:30:59.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.248+0000 7f8219600640 1 -- start start 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8219600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82140cb580 0x7f82140ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8219600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82140ac960 0x7f82140acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8219600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82140c94a0 con 0x7f821415d3a0 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8219600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140ad6e0 con 0x7f82140cb580 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8219600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140c96f0 con 0x7f82140ac960 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8212a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82140cb580 0x7f82140ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8209e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f821415d3a0 0x7f82140cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45908/0 (socket says 172.21.15.91:45908) 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8212a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82140cb580 0x7f82140ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55708/0 (socket says 172.21.15.91:55708) 2024-02-02T13:30:59.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.249+0000 7f8209e00640 1 -- 172.21.15.91:0/3295602491 learned_addr learned my addr 172.21.15.91:0/3295602491 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:59.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8213400640 1 --2- 172.21.15.91:0/3295602491 >> v2:172.21.15.113:3300/0 conn(0x7f82140ac960 0x7f82140acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8212a00640 1 -- 172.21.15.91:0/3295602491 >> v2:172.21.15.113:3300/0 conn(0x7f82140ac960 msgr2=0x7f82140acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:59.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8212a00640 1 --2- 172.21.15.91:0/3295602491 >> v2:172.21.15.113:3300/0 conn(0x7f82140ac960 0x7f82140acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8212a00640 1 -- 172.21.15.91:0/3295602491 >> v1:172.21.15.91:6789/0 conn(0x7f821415d3a0 legacy=0x7f82140cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8212a00640 1 -- 172.21.15.91:0/3295602491 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140c99d0 con 0x7f82140cb580 2024-02-02T13:30:59.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.250+0000 7f8212000640 1 -- 172.21.15.91:0/3295602491 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009987685 0 0) 0x7f82140c94a0 con 0x7f821415d3a0 2024-02-02T13:30:59.253 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.251+0000 7f8212a00640 1 --2- 172.21.15.91:0/3295602491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140cb580 0x7f82140ca1d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8214071a30 tx=0x7f820402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eb09b43deb9484 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.251+0000 7f8212000640 1 -- 172.21.15.91:0/3295602491 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204031830 con 0x7f82140cb580 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.251+0000 7f8212000640 1 -- 172.21.15.91:0/3295602491 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8204031990 con 0x7f82140cb580 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.251+0000 7f8212000640 1 -- 172.21.15.91:0/3295602491 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204031d20 con 0x7f82140cb580 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 -- 172.21.15.91:0/3295602491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140cb580 msgr2=0x7f82140ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 --2- 172.21.15.91:0/3295602491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140cb580 0x7f82140ca1d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8214071a30 tx=0x7f820402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 -- 172.21.15.91:0/3295602491 shutdown_connections 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 --2- 172.21.15.91:0/3295602491 >> v2:172.21.15.113:3300/0 conn(0x7f82140ac960 0x7f82140acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 --2- 172.21.15.91:0/3295602491 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140cb580 0x7f82140ca1d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 -- 172.21.15.91:0/3295602491 >> 172.21.15.91:0/3295602491 conn(0x7f82140b0700 msgr2=0x7f82140b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:59.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 -- 172.21.15.91:0/3295602491 shutdown_connections 2024-02-02T13:30:59.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.252+0000 7f8219600640 1 -- 172.21.15.91:0/3295602491 wait complete. 2024-02-02T13:30:59.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.254+0000 7f8219600640 1 Processor -- start 2024-02-02T13:30:59.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.254+0000 7f8219600640 1 -- start start 2024-02-02T13:30:59.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82140cb580 0x7f82141f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f821415d3a0 0x7f821415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214158dc0 con 0x7f82140ac960 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821415ed10 con 0x7f821415d3a0 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8219600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82141f3570 con 0x7f82140cb580 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.255+0000 7f8212a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:55710/0 (socket says 172.21.15.91:55710) 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 -- 172.21.15.91:0/4185711782 learned_addr learned my addr 172.21.15.91:0/4185711782 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8209e00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82140cb580 0x7f82141f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8213400640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f821415d3a0 0x7f821415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 -- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82140cb580 msgr2=0x7f82141f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82140cb580 0x7f82141f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 -- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f821415d3a0 msgr2=0x7f821415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f821415d3a0 0x7f821415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 -- 172.21.15.91:0/4185711782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009400 con 0x7f82140ac960 2024-02-02T13:30:59.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f8212a00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8204009890 tx=0x7f82040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.256+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040317d0 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.257+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820403ba70 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.257+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820403be20 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.257+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82140ad4b0 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.257+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821415e3e0 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.258+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82140ca5a0 con 0x7f82140ac960 2024-02-02T13:30:59.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.259+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f82040453f0 con 0x7f82140ac960 2024-02-02T13:30:59.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.259+0000 7f820b400640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 0x7f81ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.260+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f82040aad30 con 0x7f82140ac960 2024-02-02T13:30:59.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.260+0000 7f8209e00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 0x7f81ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.260+0000 7f8209e00640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 0x7f81ec06c8f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82140b3f90 tx=0x7f81fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.264+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8204076380 con 0x7f82140ac960 2024-02-02T13:30:59.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.467+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8214066cf0 con 0x7f82140ac960 2024-02-02T13:30:59.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.468+0000 7f820b400640 1 -- 172.21.15.91:0/4185711782 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) v1 ==== 80+0+0 (secure 0 0 0) 0x7f82040799a0 con 0x7f82140ac960 2024-02-02T13:30:59.470 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:30:59.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 msgr2=0x7f81ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 0x7f81ec06c8f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82140b3f90 tx=0x7f81fc009040 comp rx=0 tx=0).stop 2024-02-02T13:30:59.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 msgr2=0x7f8214158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8204009890 tx=0x7f82040046c0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 shutdown_connections 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f81ec06a430 0x7f81ec06c8f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82140cb580 0x7f82141f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f821415d3a0 0x7f821415f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 --2- 172.21.15.91:0/4185711782 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82140ac960 0x7f8214158880 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 >> 172.21.15.91:0/4185711782 conn(0x7f82140b0700 msgr2=0x7f82140b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 shutdown_connections 2024-02-02T13:30:59.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.471+0000 7f8219600640 1 -- 172.21.15.91:0/4185711782 wait complete. 2024-02-02T13:30:59.490 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:30:59.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:59.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:30:59.599 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:30:59.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:30:59.600 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:30:59.648 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:30:59.648 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:59.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:30:59.704 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:30:59.704 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:30:59.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:30:59.760 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:30:59.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.975+0000 7fe1b7200640 1 Processor -- start 2024-02-02T13:30:59.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.976+0000 7fe1b7200640 1 -- start start 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 0x7fe1b80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b7200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80b7d40 con 0x7fe1b80b6990 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80b81d0 con 0x7fe1b815cee0 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b7130 con 0x7fe1b80ce1d0 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b6800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe1b80b6990 0x7fe1b80d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39416/0 (socket says 172.21.15.91:39416) 2024-02-02T13:30:59.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.977+0000 7fe1b6800640 1 -- 172.21.15.91:0/1764499605 learned_addr learned my addr 172.21.15.91:0/1764499605 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b5400640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 0x7fe1b80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968786978 0 0) 0x7fe1b80b7d40 con 0x7fe1b80b6990 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe198003520 con 0x7fe1b80b6990 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172389638 0 0) 0x7fe198003520 con 0x7fe1b80b6990 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 msgr2=0x7fe1b80b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 0x7fe1b80b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 >> v1:172.21.15.113:6789/0 conn(0x7fe1b815cee0 legacy=0x7fe1b80cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80b7410 con 0x7fe1b80b6990 2024-02-02T13:30:59.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe1b0001670 con 0x7fe1b80b6990 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 >> v1:172.21.15.91:6789/0 conn(0x7fe1b80b6990 legacy=0x7fe1b80d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980039a0 0x7fe198003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1980047c0 0x7fe198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.978+0000 7fe1b4a00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 0x7fe1980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b7130 con 0x7fe1980047c0 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19800bab0 con 0x7fe1980039a0 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19800bc70 con 0x7fe1b80ce1d0 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1980047c0 0x7fe198004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5e00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980039a0 0x7fe198003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 -- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 msgr2=0x7fe1980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b80ce1d0 0x7fe1980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 -- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980039a0 msgr2=0x7fe198003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:30:59.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980039a0 0x7fe198003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5400640 1 -- 172.21.15.91:0/1764499605 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198003520 con 0x7fe1980047c0 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.979+0000 7fe1b5e00640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980039a0 0x7fe198003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.980+0000 7fe1b5400640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1980047c0 0x7fe198004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8007360 tx=0x7fe1980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0269468894418f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.980+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a802ccf0 con 0x7fe1980047c0 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.980+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a802ce50 con 0x7fe1980047c0 2024-02-02T13:30:59.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.980+0000 7fe1b4a00640 1 -- 172.21.15.91:0/1764499605 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8034660 con 0x7fe1980047c0 2024-02-02T13:30:59.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 -- 172.21.15.91:0/1764499605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1980047c0 msgr2=0x7fe198004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 --2- 172.21.15.91:0/1764499605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1980047c0 0x7fe198004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8007360 tx=0x7fe1980057f0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b6800640 1 -- 172.21.15.91:0/1764499605 reap_dead start 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 -- 172.21.15.91:0/1764499605 shutdown_connections 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 -- 172.21.15.91:0/1764499605 >> 172.21.15.91:0/1764499605 conn(0x7fe1b8070690 msgr2=0x7fe1b8070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 -- 172.21.15.91:0/1764499605 shutdown_connections 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.981+0000 7fe1b7200640 1 -- 172.21.15.91:0/1764499605 wait complete. 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.983+0000 7fe1b7200640 1 Processor -- start 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.983+0000 7fe1b7200640 1 -- start start 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980047c0 0x7fe1b81593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 0x7fe1b8160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80cc440 con 0x7fe1b80cba80 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81f6b80 con 0x7fe1980047c0 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b7200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80c9ec0 con 0x7fe1b815fba0 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43006/0 (socket says 172.21.15.91:43006) 2024-02-02T13:30:59.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b6800640 1 -- 172.21.15.91:0/2102827269 learned_addr learned my addr 172.21.15.91:0/2102827269 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b5e00640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980047c0 0x7fe1b81593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.984+0000 7fe1b5400640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 0x7fe1b8160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 -- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 msgr2=0x7fe1b8160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 0x7fe1b8160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 -- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980047c0 msgr2=0x7fe1b81593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:30:59.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980047c0 0x7fe1b81593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 -- 172.21.15.91:0/2102827269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8159910 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b5400640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 0x7fe1b8160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b6800640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80b5970 tx=0x7fe1b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0004cb0 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0004e10 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00073b0 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80ca110 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.985+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80ca6d0 con 0x7fe1b80cba80 2024-02-02T13:30:59.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.986+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b804dff0 con 0x7fe1b80cba80 2024-02-02T13:30:59.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.987+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe1b0007510 con 0x7fe1b80cba80 2024-02-02T13:30:59.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.988+0000 7fe1ae000640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 0x7fe19006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:30:59.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.988+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe1b0087c30 con 0x7fe1b80cba80 2024-02-02T13:30:59.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.988+0000 7fe1b5e00640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 0x7fe19006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:30:59.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.989+0000 7fe1b5e00640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 0x7fe19006c8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe1b815b670 tx=0x7fe1a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:30:59.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:30:59.992+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe1b00531f0 con 0x7fe1b80cba80 2024-02-02T13:31:00.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.217+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1b8066f00 con 0x7fe1b80cba80 2024-02-02T13:31:00.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.218+0000 7fe1ae000640 1 -- 172.21.15.91:0/2102827269 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fe1b0056810 con 0x7fe1b80cba80 2024-02-02T13:31:00.220 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:00.220 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:31:00.220 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:31:00.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.220+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 msgr2=0x7fe19006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 0x7fe19006c8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe1b815b670 tx=0x7fe1a0009040 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 msgr2=0x7fe1b80cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80b5970 tx=0x7fe1b000c890 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 shutdown_connections 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe19006a430 0x7fe19006c8f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1b815fba0 0x7fe1b8160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1b80cba80 0x7fe1b80cbf00 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 --2- 172.21.15.91:0/2102827269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1980047c0 0x7fe1b81593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 >> 172.21.15.91:0/2102827269 conn(0x7fe1b8070690 msgr2=0x7fe1b80c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 shutdown_connections 2024-02-02T13:31:00.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.221+0000 7fe1b7200640 1 -- 172.21.15.91:0/2102827269 wait complete. 2024-02-02T13:31:00.241 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:00.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.442+0000 7f5526400640 1 Processor -- start 2024-02-02T13:31:00.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.442+0000 7f5526400640 1 -- start start 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f5526400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f552015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f5526400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f55200b0e10 0x7f55200b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f5526400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5520070f80 con 0x7f552006f4f0 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f5526400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5520152370 con 0x7f55200b0e10 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f5526400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200cb010 con 0x7f5520151f90 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f551fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f55200b0e10 0x7f55200b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f551fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f55200b0e10 0x7f55200b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43008/0 (socket says 172.21.15.91:43008) 2024-02-02T13:31:00.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.443+0000 7f551fe00640 1 -- 172.21.15.91:0/2218197001 learned_addr learned my addr 172.21.15.91:0/2218197001 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:00.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.444+0000 7f551fe00640 1 -- 172.21.15.91:0/2218197001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 msgr2=0x7f552015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:00.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.444+0000 7f551fe00640 1 --2- 172.21.15.91:0/2218197001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f552015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.444+0000 7f551fe00640 1 -- 172.21.15.91:0/2218197001 >> v1:172.21.15.113:6789/0 conn(0x7f552006f4f0 legacy=0x7f55200b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:00.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.444+0000 7f551fe00640 1 -- 172.21.15.91:0/2218197001 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200cb2f0 con 0x7f55200b0e10 2024-02-02T13:31:00.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.445+0000 7f551fe00640 1 --2- 172.21.15.91:0/2218197001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55200b0e10 0x7f55200b3af0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44761b7cba35f44f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.445+0000 7f551e000640 1 -- 172.21.15.91:0/2218197001 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551802ca80 con 0x7f55200b0e10 2024-02-02T13:31:00.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.445+0000 7f551e000640 1 -- 172.21.15.91:0/2218197001 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551802cbe0 con 0x7f55200b0e10 2024-02-02T13:31:00.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.445+0000 7f551e000640 1 -- 172.21.15.91:0/2218197001 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180343f0 con 0x7f55200b0e10 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 -- 172.21.15.91:0/2218197001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55200b0e10 msgr2=0x7f55200b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 --2- 172.21.15.91:0/2218197001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55200b0e10 0x7f55200b3af0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802bee0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 -- 172.21.15.91:0/2218197001 shutdown_connections 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 --2- 172.21.15.91:0/2218197001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55200b0e10 0x7f55200b3af0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 --2- 172.21.15.91:0/2218197001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f552015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 -- 172.21.15.91:0/2218197001 >> 172.21.15.91:0/2218197001 conn(0x7f55200cec80 msgr2=0x7f55200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 -- 172.21.15.91:0/2218197001 shutdown_connections 2024-02-02T13:31:00.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.446+0000 7f5526400640 1 -- 172.21.15.91:0/2218197001 wait complete. 2024-02-02T13:31:00.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.448+0000 7f5526400640 1 Processor -- start 2024-02-02T13:31:00.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.448+0000 7f5526400640 1 -- start start 2024-02-02T13:31:00.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55200b0e10 0x7f55200c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f55200c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5520147da0 con 0x7f552006f4f0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200c4f30 con 0x7f55200b0e10 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f5526400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200c87c0 con 0x7f5520151f90 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43016/0 (socket says 172.21.15.91:43016) 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 -- 172.21.15.91:0/689348476 learned_addr learned my addr 172.21.15.91:0/689348476 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 -- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 msgr2=0x7f55200c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551fe00640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f55200c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551ea00640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55200b0e10 0x7f55200c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f55200c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 -- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55200b0e10 msgr2=0x7f55200c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55200b0e10 0x7f55200c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.449+0000 7f551f400640 1 -- 172.21.15.91:0/689348476 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009b30 con 0x7f552006f4f0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f551f400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f551000b4a0 tx=0x7f551000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510010610 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510010770 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510010b20 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55201525e0 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.450+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5520152ba0 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.451+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520066cf0 con 0x7f552006f4f0 2024-02-02T13:31:00.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.452+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5510010c80 con 0x7f552006f4f0 2024-02-02T13:31:00.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.452+0000 7f5517400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 0x7f54f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.453+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5510088660 con 0x7f552006f4f0 2024-02-02T13:31:00.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.453+0000 7f551ea00640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 0x7f54f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.453+0000 7f551ea00640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 0x7f54f806c8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f55200c4240 tx=0x7f5508008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.457+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5510053b90 con 0x7f552006f4f0 2024-02-02T13:31:00.566 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:31:00.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:31:00.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.647+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f552006c7b0 con 0x7f552006f4f0 2024-02-02T13:31:00.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.649+0000 7f5517400640 1 -- 172.21.15.91:0/689348476 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16154 (secure 0 0 0) 0x7f55100571b0 con 0x7f552006f4f0 2024-02-02T13:31:00.650 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:00.651 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":116,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:48.529769+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 msgr2=0x7f54f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 0x7f54f806c8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f55200c4240 tx=0x7f5508008040 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 msgr2=0x7f5520158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f551000b4a0 tx=0x7f551000b960 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 shutdown_connections 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f54f806a430 0x7f54f806c8f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5520151f90 0x7f55200c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55200b0e10 0x7f55200c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 --2- 172.21.15.91:0/689348476 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f552006f4f0 0x7f5520158510 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 >> 172.21.15.91:0/689348476 conn(0x7f55200cec80 msgr2=0x7f552015a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:00.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 shutdown_connections 2024-02-02T13:31:00.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.651+0000 7f5526400640 1 -- 172.21.15.91:0/689348476 wait complete. 2024-02-02T13:31:00.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:30:15.902582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:30:17.156657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:00.669 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:00.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.875+0000 7f853da00640 1 Processor -- start 2024-02-02T13:31:00.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.875+0000 7f853da00640 1 -- start start 2024-02-02T13:31:00.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f853806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853da00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f85380ccd70 0x7f85380c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853da00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85380ad2e0 con 0x7f853806c820 2024-02-02T13:31:00.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853da00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85380ad5c0 con 0x7f85380ccd70 2024-02-02T13:31:00.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853806b4f0 con 0x7f853815d2c0 2024-02-02T13:31:00.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f852f200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f853806c820 0x7f85380cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39418/0 (socket says 172.21.15.91:39418) 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f852f200640 1 -- 172.21.15.91:0/299846180 learned_addr learned my addr 172.21.15.91:0/299846180 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.877+0000 7f853d000640 1 --2- 172.21.15.91:0/299846180 >> v2:172.21.15.91:3300/0 conn(0x7f85380ccd70 0x7f85380c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f8537e00640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f853806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f8537400640 1 -- 172.21.15.91:0/299846180 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119663467 0 0) 0x7f85380ad2e0 con 0x7f853806c820 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f8537400640 1 -- 172.21.15.91:0/299846180 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8518003520 con 0x7f853806c820 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f853d000640 1 -- 172.21.15.91:0/299846180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 msgr2=0x7f853806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f853d000640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f853806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f853d000640 1 -- 172.21.15.91:0/299846180 >> v1:172.21.15.91:6789/0 conn(0x7f853806c820 legacy=0x7f85380cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.878+0000 7f853d000640 1 -- 172.21.15.91:0/299846180 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853806b7d0 con 0x7f85380ccd70 2024-02-02T13:31:00.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.879+0000 7f853d000640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85380ccd70 0x7f85380c90e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f853002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6c332e937b5eaa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.879+0000 7f8537400640 1 -- 172.21.15.91:0/299846180 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853002cc50 con 0x7f85380ccd70 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.879+0000 7f8537400640 1 -- 172.21.15.91:0/299846180 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853002cdb0 con 0x7f85380ccd70 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.879+0000 7f8537400640 1 -- 172.21.15.91:0/299846180 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530034660 con 0x7f85380ccd70 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 -- 172.21.15.91:0/299846180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85380ccd70 msgr2=0x7f85380c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85380ccd70 0x7f85380c90e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f853002bea0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 -- 172.21.15.91:0/299846180 shutdown_connections 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85380ccd70 0x7f85380c90e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 --2- 172.21.15.91:0/299846180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f853806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 -- 172.21.15.91:0/299846180 >> 172.21.15.91:0/299846180 conn(0x7f85380cea90 msgr2=0x7f85380ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 -- 172.21.15.91:0/299846180 shutdown_connections 2024-02-02T13:31:00.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.880+0000 7f853da00640 1 -- 172.21.15.91:0/299846180 wait complete. 2024-02-02T13:31:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.882+0000 7f853da00640 1 Processor -- start 2024-02-02T13:31:00.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.882+0000 7f853da00640 1 -- start start 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85380ccd70 0x7f8538150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f8538150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85380c6ae0 con 0x7f853806c820 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85380c5ed0 con 0x7f85380ccd70 2024-02-02T13:31:00.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380c61b0 con 0x7f853815d2c0 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f8537e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f8537e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43042/0 (socket says 172.21.15.91:43042) 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f8537e00640 1 -- 172.21.15.91:0/4110751845 learned_addr learned my addr 172.21.15.91:0/4110751845 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f8537e00640 1 -- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 msgr2=0x7f8538150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.883+0000 7f853d000640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f8538150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8537e00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f8538150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f852f200640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85380ccd70 0x7f8538150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8537e00640 1 -- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85380ccd70 msgr2=0x7f8538150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8537e00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85380ccd70 0x7f8538150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8537e00640 1 -- 172.21.15.91:0/4110751845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009a70 con 0x7f853806c820 2024-02-02T13:31:00.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8537e00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f852800d5f0 tx=0x7f852800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280125f0 con 0x7f853806c820 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528012750 con 0x7f853806c820 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528012b00 con 0x7f853806c820 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85380c6490 con 0x7f853806c820 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.884+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85380ae3f0 con 0x7f853806c820 2024-02-02T13:31:00.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.885+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853806d050 con 0x7f853806c820 2024-02-02T13:31:00.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.886+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8528012c60 con 0x7f853806c820 2024-02-02T13:31:00.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.887+0000 7f8534c00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 0x7f851006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:00.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.887+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8528088ea0 con 0x7f853806c820 2024-02-02T13:31:00.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.887+0000 7f852f200640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 0x7f851006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:00.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.888+0000 7f852f200640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 0x7f851006c8f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8538070b70 tx=0x7f8520009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:00.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:00.892+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f85280543d0 con 0x7f853806c820 2024-02-02T13:31:01.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.081+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f853804dff0 con 0x7f853806c820 2024-02-02T13:31:01.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.081+0000 7f8534c00640 1 -- 172.21.15.91:0/4110751845 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85280579f0 con 0x7f853806c820 2024-02-02T13:31:01.083 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 msgr2=0x7f851006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 0x7f851006c8f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8538070b70 tx=0x7f8520009040 comp rx=0 tx=0).stop 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 msgr2=0x7f85380b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f852800d5f0 tx=0x7f852800dab0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 shutdown_connections 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f851006a430 0x7f851006c8f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f853815d2c0 0x7f8538150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85380ccd70 0x7f8538150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 --2- 172.21.15.91:0/4110751845 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f853806c820 0x7f85380b3af0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 >> 172.21.15.91:0/4110751845 conn(0x7f85380cea90 msgr2=0x7f8538070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:01.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.084+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 shutdown_connections 2024-02-02T13:31:01.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.085+0000 7f853da00640 1 -- 172.21.15.91:0/4110751845 wait complete. 2024-02-02T13:31:01.104 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:31:01.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.314+0000 7f142ec00640 1 Processor -- start 2024-02-02T13:31:01.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.314+0000 7f142ec00640 1 -- start start 2024-02-02T13:31:01.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.315+0000 7f142ec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 0x7f142806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.315+0000 7f142ec00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f142806fc50 0x7f142806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.315+0000 7f142ec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1428070420 con 0x7f14281595a0 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.315+0000 7f142ec00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1428070fa0 con 0x7f142806fc50 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.315+0000 7f142ec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1428070700 con 0x7f14280c9c50 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f142806fc50 0x7f142806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f142806fc50 0x7f142806b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43046/0 (socket says 172.21.15.91:43046) 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 -- 172.21.15.91:0/78464165 learned_addr learned my addr 172.21.15.91:0/78464165 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f141f200640 1 --2- 172.21.15.91:0/78464165 >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 0x7f142806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 -- 172.21.15.91:0/78464165 >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 msgr2=0x7f142806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 --2- 172.21.15.91:0/78464165 >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 0x7f142806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 -- 172.21.15.91:0/78464165 >> v1:172.21.15.113:6789/0 conn(0x7f14281595a0 legacy=0x7f14280c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f142cc00640 1 -- 172.21.15.91:0/78464165 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280709e0 con 0x7f142806fc50 2024-02-02T13:31:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.316+0000 7f141f200640 1 --2- 172.21.15.91:0/78464165 >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 0x7f142806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.317+0000 7f142cc00640 1 --2- 172.21.15.91:0/78464165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142806fc50 0x7f142806b9d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f14280cb220 tx=0x7f142002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db6a9a38fb4f3d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.317+0000 7f1427400640 1 -- 172.21.15.91:0/78464165 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142002ca80 con 0x7f142806fc50 2024-02-02T13:31:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.317+0000 7f1427400640 1 -- 172.21.15.91:0/78464165 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142002cbe0 con 0x7f142806fc50 2024-02-02T13:31:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f1427400640 1 -- 172.21.15.91:0/78464165 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200343f0 con 0x7f142806fc50 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 -- 172.21.15.91:0/78464165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142806fc50 msgr2=0x7f142806b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 --2- 172.21.15.91:0/78464165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142806fc50 0x7f142806b9d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f14280cb220 tx=0x7f142002bee0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 -- 172.21.15.91:0/78464165 shutdown_connections 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 --2- 172.21.15.91:0/78464165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142806fc50 0x7f142806b9d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 --2- 172.21.15.91:0/78464165 >> v2:172.21.15.113:3300/0 conn(0x7f14280c9c50 0x7f142806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 -- 172.21.15.91:0/78464165 >> 172.21.15.91:0/78464165 conn(0x7f14280ceca0 msgr2=0x7f14280cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 -- 172.21.15.91:0/78464165 shutdown_connections 2024-02-02T13:31:01.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.318+0000 7f142ec00640 1 -- 172.21.15.91:0/78464165 wait complete. 2024-02-02T13:31:01.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.320+0000 7f142ec00640 1 Processor -- start 2024-02-02T13:31:01.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.320+0000 7f142ec00640 1 -- start start 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142806fc50 0x7f14280c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14281595a0 0x7f1428154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14281f6780 con 0x7f14280c9c50 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1428155950 con 0x7f142806fc50 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f142ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1428154860 con 0x7f14281595a0 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.321+0000 7f141f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43054/0 (socket says 172.21.15.91:43054) 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 -- 172.21.15.91:0/3182507414 learned_addr learned my addr 172.21.15.91:0/3182507414 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:01.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 -- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14281595a0 msgr2=0x7f1428154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14281595a0 0x7f1428154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f1427e00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142806fc50 0x7f14280c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 -- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142806fc50 msgr2=0x7f14280c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142806fc50 0x7f14280c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 -- 172.21.15.91:0/3182507414 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420009b30 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f141f200640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f141000c7f0 tx=0x7f141000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.322+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000cea0 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.323+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410007a30 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.323+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410007dc0 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.323+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14281f6a00 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.323+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14281f6fc0 con 0x7f14280c9c50 2024-02-02T13:31:01.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.324+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1428066cf0 con 0x7f14280c9c50 2024-02-02T13:31:01.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.325+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f14100113f0 con 0x7f14280c9c50 2024-02-02T13:31:01.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.325+0000 7f1424c00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 0x7f140006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.326+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1410088a90 con 0x7f14280c9c50 2024-02-02T13:31:01.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.326+0000 7f1427e00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 0x7f140006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.326+0000 7f1427e00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 0x7f140006c920 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14180015d0 tx=0x7f1418009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.330+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1410054060 con 0x7f14280c9c50 2024-02-02T13:31:01.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.518+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f142806cf30 con 0x7f14280c9c50 2024-02-02T13:31:01.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.519+0000 7f1424c00640 1 -- 172.21.15.91:0/3182507414 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1410057680 con 0x7f14280c9c50 2024-02-02T13:31:01.521 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:31:01.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 msgr2=0x7f140006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 0x7f140006c920 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14180015d0 tx=0x7f1418009040 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 msgr2=0x7f14281f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f141000c7f0 tx=0x7f141000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 shutdown_connections 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f140006a460 0x7f140006c920 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14281595a0 0x7f1428154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f14280c9c50 0x7f14281f6240 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 --2- 172.21.15.91:0/3182507414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142806fc50 0x7f14280c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 >> 172.21.15.91:0/3182507414 conn(0x7f14280ceca0 msgr2=0x7f14280b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 shutdown_connections 2024-02-02T13:31:01.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.522+0000 7f142ec00640 1 -- 172.21.15.91:0/3182507414 wait complete. 2024-02-02T13:31:01.541 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:31:01.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.753+0000 7f09b2a00640 1 Processor -- start 2024-02-02T13:31:01.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.753+0000 7f09b2a00640 1 -- start start 2024-02-02T13:31:01.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.754+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f09ac06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.754+0000 7f09b2a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac15d870 con 0x7f09ac0acfe0 2024-02-02T13:31:01.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09b2a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac06bce0 con 0x7f09ac15d490 2024-02-02T13:31:01.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09b2a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac161220 con 0x7f09ac0b0e10 2024-02-02T13:31:01.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09ab400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f09ac0acfe0 0x7f09ac0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39426/0 (socket says 172.21.15.91:39426) 2024-02-02T13:31:01.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09ab400640 1 -- 172.21.15.91:0/134664738 learned_addr learned my addr 172.21.15.91:0/134664738 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:01.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09b0a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f09ac06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884634482 0 0) 0x7f09ac15d870 con 0x7f09ac0acfe0 2024-02-02T13:31:01.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.755+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f098c003520 con 0x7f09ac0acfe0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922188242 0 0) 0x7f098c003520 con 0x7f09ac0acfe0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 msgr2=0x7f09ac06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f09ac06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 >> v1:172.21.15.113:6789/0 conn(0x7f09ac15d490 legacy=0x7f09ac0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0ad490 con 0x7f09ac0acfe0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0994001670 con 0x7f09ac0acfe0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 >> v1:172.21.15.91:6789/0 conn(0x7f09ac0acfe0 legacy=0x7f09ac0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f098c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098c003fd0 0x7f098c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f098c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac161220 con 0x7f098c004d00 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c00bb40 con 0x7f098c003fd0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c00bd00 con 0x7f09ac0b0e10 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.756+0000 7f09b0a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f098c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09abe00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f098c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09ab400640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098c003fd0 0x7f098c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09b0a00640 1 -- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 msgr2=0x7f098c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:01.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09b0a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0b0e10 0x7f098c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09b0a00640 1 -- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098c003fd0 msgr2=0x7f098c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:01.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09b0a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098c003fd0 0x7f098c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.757+0000 7f09b0a00640 1 -- 172.21.15.91:0/134664738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c003520 con 0x7f098c004d00 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09b0a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f098c005180 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f09a4007330 tx=0x7f098c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53127e38f9f10349 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a402cca0 con 0x7f098c004d00 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a402ce00 con 0x7f098c004d00 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09aaa00640 1 -- 172.21.15.91:0/134664738 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4034680 con 0x7f098c004d00 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09b2a00640 1 -- 172.21.15.91:0/134664738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 msgr2=0x7f098c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.758+0000 7f09b2a00640 1 --2- 172.21.15.91:0/134664738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f098c005180 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f09a4007330 tx=0x7f098c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.759+0000 7f09b0a00640 1 -- 172.21.15.91:0/134664738 reap_dead start 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.759+0000 7f09b2a00640 1 -- 172.21.15.91:0/134664738 shutdown_connections 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.759+0000 7f09b2a00640 1 -- 172.21.15.91:0/134664738 >> 172.21.15.91:0/134664738 conn(0x7f09ac0cec80 msgr2=0x7f09ac0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.759+0000 7f09b2a00640 1 -- 172.21.15.91:0/134664738 shutdown_connections 2024-02-02T13:31:01.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.759+0000 7f09b2a00640 1 -- 172.21.15.91:0/134664738 wait complete. 2024-02-02T13:31:01.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.760+0000 7f09b2a00640 1 Processor -- start 2024-02-02T13:31:01.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.761+0000 7f09b2a00640 1 -- start start 2024-02-02T13:31:01.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09ac0c4830 0x7f09ac0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0c4cb0 0x7f09ac0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0c3790 con 0x7f098c004d00 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0c3a70 con 0x7f09ac0c4830 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09b2a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0c3d50 con 0x7f09ac0c4cb0 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43062/0 (socket says 172.21.15.91:43062) 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 -- 172.21.15.91:0/4223613131 learned_addr learned my addr 172.21.15.91:0/4223613131 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09ab400640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09ac0c4830 0x7f09ac0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 -- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0c4cb0 msgr2=0x7f09ac0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0c4cb0 0x7f09ac0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 -- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09ac0c4830 msgr2=0x7f09ac0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09ac0c4830 0x7f09ac0c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.762+0000 7f09abe00640 1 -- 172.21.15.91:0/4223613131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0c4030 con 0x7f098c004d00 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09abe00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e550 tx=0x7f099c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0052e0 con 0x7f098c004d00 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c005440 con 0x7f098c004d00 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c003810 con 0x7f098c004d00 2024-02-02T13:31:01.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac0c4310 con 0x7f098c004d00 2024-02-02T13:31:01.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.763+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac0b1a50 con 0x7f098c004d00 2024-02-02T13:31:01.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.764+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac0ce460 con 0x7f098c004d00 2024-02-02T13:31:01.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.765+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f099c003970 con 0x7f098c004d00 2024-02-02T13:31:01.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.766+0000 7f09a3e00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 0x7f098406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:01.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.766+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f099c088680 con 0x7f098c004d00 2024-02-02T13:31:01.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.766+0000 7f09ab400640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 0x7f098406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:01.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.767+0000 7f09ab400640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 0x7f098406c920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0994001200 tx=0x7f0994001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:01.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.770+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f099c053bb0 con 0x7f098c004d00 2024-02-02T13:31:01.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.958+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09ac06b660 con 0x7f098c004d00 2024-02-02T13:31:01.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.959+0000 7f09a3e00640 1 -- 172.21.15.91:0/4223613131 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f099c0571d0 con 0x7f098c004d00 2024-02-02T13:31:01.961 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 msgr2=0x7f098406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 0x7f098406c920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0994001200 tx=0x7f0994001150 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 msgr2=0x7f09ac1f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e550 tx=0x7f099c00ea10 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 shutdown_connections 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f098406a460 0x7f098406c920 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09ac0c4cb0 0x7f09ac0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09ac0c4830 0x7f09ac0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 --2- 172.21.15.91:0/4223613131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f098c004d00 0x7f09ac1f2730 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 >> 172.21.15.91:0/4223613131 conn(0x7f09ac0cec80 msgr2=0x7f09ac071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 shutdown_connections 2024-02-02T13:31:01.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:01.962+0000 7f09b2a00640 1 -- 172.21.15.91:0/4223613131 wait complete. 2024-02-02T13:31:01.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:02.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.180+0000 7f90a8200640 1 Processor -- start 2024-02-02T13:31:02.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.181+0000 7f90a8200640 1 -- start start 2024-02-02T13:31:02.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90a00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a8200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a00cc7c0 con 0x7f90a00ce130 2024-02-02T13:31:02.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a00ce930 con 0x7f90a015a010 2024-02-02T13:31:02.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a00cd640 con 0x7f90a00b0e10 2024-02-02T13:31:02.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a4e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f90a00ce130 0x7f90a00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39434/0 (socket says 172.21.15.91:39434) 2024-02-02T13:31:02.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.182+0000 7f90a4e00640 1 -- 172.21.15.91:0/886785335 learned_addr learned my addr 172.21.15.91:0/886785335 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904162936 0 0) 0x7f90a00cc7c0 con 0x7f90a00ce130 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f90a6200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90a00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9080003520 con 0x7f90a00ce130 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006749427 0 0) 0x7f9080003520 con 0x7f90a00ce130 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 msgr2=0x7f90a00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90a00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 >> v1:172.21.15.113:6789/0 conn(0x7f90a015a010 legacy=0x7f90a00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a006c890 con 0x7f90a00ce130 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9088001670 con 0x7f90a00ce130 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 >> v1:172.21.15.91:6789/0 conn(0x7f90a00ce130 legacy=0x7f90a00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90800039a0 0x7f9080003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f9080005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a00cd640 con 0x7f9080004d00 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908000bb10 con 0x7f90800039a0 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.183+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908000bcd0 con 0x7f90a00b0e10 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f9080005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a4e00640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a5800640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90800039a0 0x7f9080003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 -- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 msgr2=0x7f90800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a00b0e10 0x7f90800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 -- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90800039a0 msgr2=0x7f9080003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90800039a0 0x7f9080003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.184+0000 7f90a6200640 1 -- 172.21.15.91:0/886785335 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080003520 con 0x7f9080004d00 2024-02-02T13:31:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.185+0000 7f90a6200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f9080005180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f909800ba00 tx=0x7f90800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f048d9933c68e619 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.185+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098032620 con 0x7f9080004d00 2024-02-02T13:31:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.185+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9098032780 con 0x7f9080004d00 2024-02-02T13:31:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.185+0000 7f909fe00640 1 -- 172.21.15.91:0/886785335 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098032b10 con 0x7f9080004d00 2024-02-02T13:31:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 -- 172.21.15.91:0/886785335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 msgr2=0x7f9080005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 --2- 172.21.15.91:0/886785335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f9080005180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f909800ba00 tx=0x7f90800057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a6200640 1 -- 172.21.15.91:0/886785335 reap_dead start 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 -- 172.21.15.91:0/886785335 shutdown_connections 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 -- 172.21.15.91:0/886785335 >> 172.21.15.91:0/886785335 conn(0x7f90a00cec80 msgr2=0x7f90a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 -- 172.21.15.91:0/886785335 shutdown_connections 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.186+0000 7f90a8200640 1 -- 172.21.15.91:0/886785335 wait complete. 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.188+0000 7f90a8200640 1 Processor -- start 2024-02-02T13:31:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.188+0000 7f90a8200640 1 -- start start 2024-02-02T13:31:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a014f4c0 0x7f90a014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90a00c5cc0 0x7f90a00c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a01507e0 con 0x7f9080004d00 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a014ff40 con 0x7f90a00c5cc0 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a0150220 con 0x7f90a014f4c0 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a5800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43072/0 (socket says 172.21.15.91:43072) 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.189+0000 7f90a5800640 1 -- 172.21.15.91:0/3236844773 learned_addr learned my addr 172.21.15.91:0/3236844773 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 -- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a014f4c0 msgr2=0x7f90a014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a014f4c0 0x7f90a014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 -- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90a00c5cc0 msgr2=0x7f90a00c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90a00c5cc0 0x7f90a00c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 -- 172.21.15.91:0/3236844773 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0150500 con 0x7f9080004d00 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a5800640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f909000e8c0 tx=0x7f909000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090003b80 con 0x7f9080004d00 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090003ce0 con 0x7f9080004d00 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900052f0 con 0x7f9080004d00 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a0156260 con 0x7f9080004d00 2024-02-02T13:31:02.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.190+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a015cce0 con 0x7f9080004d00 2024-02-02T13:31:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.191+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a0066cf0 con 0x7f9080004d00 2024-02-02T13:31:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.192+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9090005450 con 0x7f9080004d00 2024-02-02T13:31:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.193+0000 7f909d600640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 0x7f907806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.193+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9090088af0 con 0x7f9080004d00 2024-02-02T13:31:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.193+0000 7f90a4e00640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 0x7f907806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.194+0000 7f90a4e00640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 0x7f907806c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9088001eb0 tx=0x7f9088001d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.197+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9090054020 con 0x7f9080004d00 2024-02-02T13:31:02.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.423+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90a00b4c00 con 0x7f9080004d00 2024-02-02T13:31:02.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.425+0000 7f909d600640 1 -- 172.21.15.91:0/3236844773 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f909001a310 con 0x7f9080004d00 2024-02-02T13:31:02.427 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:02.427 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:31:02.427 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 msgr2=0x7f907806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 0x7f907806c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9088001eb0 tx=0x7f9088001d60 comp rx=0 tx=0).stop 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 msgr2=0x7f90a015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f909000e8c0 tx=0x7f909000ed80 comp rx=0 tx=0).stop 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 shutdown_connections 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f907806a430 0x7f907806c8f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90a014f4c0 0x7f90a014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90a00c5cc0 0x7f90a00c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 --2- 172.21.15.91:0/3236844773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9080004d00 0x7f90a015c7a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.427+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 >> 172.21.15.91:0/3236844773 conn(0x7f90a00cec80 msgr2=0x7f90a015b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.428+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 shutdown_connections 2024-02-02T13:31:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.428+0000 7f90a8200640 1 -- 172.21.15.91:0/3236844773 wait complete. 2024-02-02T13:31:02.447 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:02.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.651+0000 7f559ae00640 1 Processor -- start 2024-02-02T13:31:02.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.651+0000 7f559ae00640 1 -- start start 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f559ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f55940c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f559ae00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940cc7c0 con 0x7f559415a010 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f559ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940ce930 con 0x7f55940b0700 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f559ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55940cd640 con 0x7f55940ce130 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f5593e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f559415a010 0x7f55940ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39438/0 (socket says 172.21.15.91:39438) 2024-02-02T13:31:02.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.652+0000 7f5593e00640 1 -- 172.21.15.91:0/3551863456 learned_addr learned my addr 172.21.15.91:0/3551863456 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571226928 0 0) 0x7f55940cc7c0 con 0x7f559415a010 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5574003520 con 0x7f559415a010 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f558b200640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f55940c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2629014233 0 0) 0x7f5574003520 con 0x7f559415a010 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 msgr2=0x7f55940c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f55940c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 >> v1:172.21.15.113:6789/0 conn(0x7f55940b0700 legacy=0x7f559406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559406c890 con 0x7f559415a010 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5584001670 con 0x7f559415a010 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 >> v1:172.21.15.91:6789/0 conn(0x7f559415a010 legacy=0x7f55940ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.653+0000 7f5593400640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55740039a0 0x7f5574003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593400640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740047c0 0x7f5574004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593400640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f5574005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940cd640 con 0x7f55740039a0 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557400dd20 con 0x7f55740047c0 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557400dee0 con 0x7f55940ce130 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55740039a0 0x7f5574003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f558b200640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740047c0 0x7f5574004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 -- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 msgr2=0x7f5574005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5598e00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f5574005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55940ce130 0x7f5574005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 -- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740047c0 msgr2=0x7f5574004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:02.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740047c0 0x7f5574004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f5593e00640 1 -- 172.21.15.91:0/3551863456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574003520 con 0x7f55740039a0 2024-02-02T13:31:02.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.654+0000 7f558b200640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740047c0 0x7f5574004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.655+0000 7f5593e00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55740039a0 0x7f5574003dc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5584001050 tx=0x7f55740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5afd0687c7a7f43 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.655+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584005ad0 con 0x7f55740039a0 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584005c30 con 0x7f55740039a0 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f5593400640 1 -- 172.21.15.91:0/3551863456 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558402c7c0 con 0x7f55740039a0 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 -- 172.21.15.91:0/3551863456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55740039a0 msgr2=0x7f5574003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 --2- 172.21.15.91:0/3551863456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55740039a0 0x7f5574003dc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5584001050 tx=0x7f55740057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f5598e00640 1 -- 172.21.15.91:0/3551863456 reap_dead start 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 -- 172.21.15.91:0/3551863456 shutdown_connections 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 -- 172.21.15.91:0/3551863456 >> 172.21.15.91:0/3551863456 conn(0x7f55940cec80 msgr2=0x7f55940cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 -- 172.21.15.91:0/3551863456 shutdown_connections 2024-02-02T13:31:02.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.656+0000 7f559ae00640 1 -- 172.21.15.91:0/3551863456 wait complete. 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.658+0000 7f559ae00640 1 Processor -- start 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.658+0000 7f559ae00640 1 -- start start 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740039a0 0x7f5594152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 0x7f5594153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940aff90 con 0x7f55940af5d0 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941541e0 con 0x7f55740039a0 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f559ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55941544c0 con 0x7f55941534b0 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f558b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f558b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43088/0 (socket says 172.21.15.91:43088) 2024-02-02T13:31:02.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f558b200640 1 -- 172.21.15.91:0/553879538 learned_addr learned my addr 172.21.15.91:0/553879538 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.659+0000 7f5593e00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740039a0 0x7f5594152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f5598e00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 0x7f5594153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 -- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 msgr2=0x7f5594153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 0x7f5594153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 -- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740039a0 msgr2=0x7f5594152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740039a0 0x7f5594152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 -- 172.21.15.91:0/553879538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55941547a0 con 0x7f55940af5d0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f5598e00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 0x7f5594153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f558b200640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e7e0 tx=0x7f557c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00c6d0 con 0x7f55940af5d0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c00c830 con 0x7f55940af5d0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00cba0 con 0x7f55940af5d0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559415a3f0 con 0x7f55940af5d0 2024-02-02T13:31:02.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.660+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5594153e70 con 0x7f55940af5d0 2024-02-02T13:31:02.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.661+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5594066cf0 con 0x7f55940af5d0 2024-02-02T13:31:02.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.662+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f557c00cd00 con 0x7f55940af5d0 2024-02-02T13:31:02.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.663+0000 7f5590c00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 0x7f556c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:02.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.663+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f557c0885f0 con 0x7f55940af5d0 2024-02-02T13:31:02.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.663+0000 7f5593e00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 0x7f556c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:02.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.664+0000 7f5593e00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 0x7f556c06c8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5584001050 tx=0x7f5584007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:02.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.668+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f557c053bb0 con 0x7f55940af5d0 2024-02-02T13:31:02.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.858+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55940caa20 con 0x7f55940af5d0 2024-02-02T13:31:02.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.859+0000 7f5590c00640 1 -- 172.21.15.91:0/553879538 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16154 (secure 0 0 0) 0x7f557c009420 con 0x7f55940af5d0 2024-02-02T13:31:02.861 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:02.861 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":116,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:48.529769+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 msgr2=0x7f556c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 0x7f556c06c8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5584001050 tx=0x7f5584007000 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 msgr2=0x7f55940afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e7e0 tx=0x7f557c00eca0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 shutdown_connections 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f556c06a430 0x7f556c06c8f0 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5584001050 tx=0x7f5584007000 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55941534b0 0x7f5594153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55940af5d0 0x7f55940afa50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 --2- 172.21.15.91:0/553879538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55740039a0 0x7f5594152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 >> 172.21.15.91:0/553879538 conn(0x7f55940cec80 msgr2=0x7f55940ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:02.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 shutdown_connections 2024-02-02T13:31:02.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:02.862+0000 7f559ae00640 1 -- 172.21.15.91:0/553879538 wait complete. 2024-02-02T13:31:02.880 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:31:02.881 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:03.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.076+0000 7fe64e200640 1 Processor -- start 2024-02-02T13:31:03.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.076+0000 7fe64e200640 1 -- start start 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.077+0000 7fe64e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe64815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.077+0000 7fe64e200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe648070f80 con 0x7fe64806f4f0 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.077+0000 7fe64e200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64806c600 con 0x7fe6480b0e10 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.077+0000 7fe64e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe648152370 con 0x7fe648151f90 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.078+0000 7fe63e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe64806f4f0 0x7fe6480b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39450/0 (socket says 172.21.15.91:39450) 2024-02-02T13:31:03.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.078+0000 7fe63e800640 1 -- 172.21.15.91:0/3001614251 learned_addr learned my addr 172.21.15.91:0/3001614251 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:03.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.078+0000 7fe647400640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe64815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.078+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284236339 0 0) 0x7fe648070f80 con 0x7fe64806f4f0 2024-02-02T13:31:03.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.078+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe628003520 con 0x7fe64806f4f0 2024-02-02T13:31:03.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4139686609 0 0) 0x7fe628003520 con 0x7fe64806f4f0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 msgr2=0x7fe64815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe64815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 >> v1:172.21.15.113:6789/0 conn(0x7fe6480b0e10 legacy=0x7fe6480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480cb010 con 0x7fe64806f4f0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe630001670 con 0x7fe64806f4f0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 >> v1:172.21.15.91:6789/0 conn(0x7fe64806f4f0 legacy=0x7fe6480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe6280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628003ee0 0x7fe6280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe628004d00 0x7fe628005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe648152370 con 0x7fe628004d00 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62800b9e0 con 0x7fe628003ee0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62800bba0 con 0x7fe648151f90 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe647e00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe628004d00 0x7fe628005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe647400640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe6280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe63e800640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628003ee0 0x7fe6280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe647e00640 1 -- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 msgr2=0x7fe6280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:03.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.079+0000 7fe647e00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe648151f90 0x7fe6280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.080+0000 7fe647e00640 1 -- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628003ee0 msgr2=0x7fe6280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:03.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.080+0000 7fe647e00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628003ee0 0x7fe6280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.080+0000 7fe647e00640 1 -- 172.21.15.91:0/3001614251 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628003520 con 0x7fe628004d00 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe647e00640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe628004d00 0x7fe628005180 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe648071a20 tx=0x7fe6280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0016579d758291e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64002d6b0 con 0x7fe628004d00 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64002d830 con 0x7fe628004d00 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe646a00640 1 -- 172.21.15.91:0/3001614251 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64002dbe0 con 0x7fe628004d00 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe64e200640 1 -- 172.21.15.91:0/3001614251 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe628004d00 msgr2=0x7fe628005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe64e200640 1 --2- 172.21.15.91:0/3001614251 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe628004d00 0x7fe628005180 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe648071a20 tx=0x7fe6280057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe647e00640 1 -- 172.21.15.91:0/3001614251 reap_dead start 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe64e200640 1 -- 172.21.15.91:0/3001614251 shutdown_connections 2024-02-02T13:31:03.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe64e200640 1 -- 172.21.15.91:0/3001614251 >> 172.21.15.91:0/3001614251 conn(0x7fe6480cec80 msgr2=0x7fe6480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:03.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.081+0000 7fe64e200640 1 -- 172.21.15.91:0/3001614251 shutdown_connections 2024-02-02T13:31:03.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.082+0000 7fe64e200640 1 -- 172.21.15.91:0/3001614251 wait complete. 2024-02-02T13:31:03.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.083+0000 7fe64e200640 1 Processor -- start 2024-02-02T13:31:03.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.083+0000 7fe64e200640 1 -- start start 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628004d00 0x7fe6481f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6480aefc0 0x7fe64806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480c5290 con 0x7fe648140c30 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480af440 con 0x7fe628004d00 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe64e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481f41c0 con 0x7fe6480aefc0 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe63e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe63e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43116/0 (socket says 172.21.15.91:43116) 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.084+0000 7fe63e800640 1 -- 172.21.15.91:0/2072842211 learned_addr learned my addr 172.21.15.91:0/2072842211 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:03.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 -- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6480aefc0 msgr2=0x7fe64806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6480aefc0 0x7fe64806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 -- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628004d00 msgr2=0x7fe6481f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628004d00 0x7fe6481f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 -- 172.21.15.91:0/2072842211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480c4610 con 0x7fe648140c30 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63e800640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e840 tx=0x7fe63000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63000c850 con 0x7fe648140c30 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63000c9d0 con 0x7fe648140c30 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63000cd80 con 0x7fe648140c30 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6480c48f0 con 0x7fe648140c30 2024-02-02T13:31:03.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.085+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6480c4eb0 con 0x7fe648140c30 2024-02-02T13:31:03.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.086+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6480c53d0 con 0x7fe648140c30 2024-02-02T13:31:03.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.087+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe63001d460 con 0x7fe648140c30 2024-02-02T13:31:03.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.089+0000 7fe63fe00640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 0x7fe62006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.089+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe6300889a0 con 0x7fe648140c30 2024-02-02T13:31:03.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.089+0000 7fe647400640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 0x7fe62006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.090+0000 7fe647400640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 0x7fe62006c920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe638009910 tx=0x7fe638008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.093+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6480c53d0 con 0x7fe648140c30 2024-02-02T13:31:03.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.310+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6480c53d0 con 0x7fe648140c30 2024-02-02T13:31:03.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.311+0000 7fe63fe00640 1 -- 172.21.15.91:0/2072842211 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fe6480c53d0 con 0x7fe648140c30 2024-02-02T13:31:03.313 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:03.313 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:31:03.313 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:31:03.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 msgr2=0x7fe62006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 0x7fe62006c920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe638009910 tx=0x7fe638008040 comp rx=0 tx=0).stop 2024-02-02T13:31:03.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 msgr2=0x7fe6480aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e840 tx=0x7fe63000ed00 comp rx=0 tx=0).stop 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 shutdown_connections 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe62006a460 0x7fe62006c920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6480aefc0 0x7fe64806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe648140c30 0x7fe6480aea80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 --2- 172.21.15.91:0/2072842211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe628004d00 0x7fe6481f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 >> 172.21.15.91:0/2072842211 conn(0x7fe6480cec80 msgr2=0x7fe648159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 shutdown_connections 2024-02-02T13:31:03.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.314+0000 7fe64e200640 1 -- 172.21.15.91:0/2072842211 wait complete. 2024-02-02T13:31:03.332 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:03.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.532+0000 7ff5cbe00640 1 Processor -- start 2024-02-02T13:31:03.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.532+0000 7ff5cbe00640 1 -- start start 2024-02-02T13:31:03.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5cc15b2a0 0x7ff5cc06c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5cc06cb90 0x7ff5cc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc06c1c0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc06b5f0 con 0x7ff5cc06cb90 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5cbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc06b890 con 0x7ff5cc15aec0 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5cc15b2a0 0x7ff5cc06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5cc15b2a0 0x7ff5cc06c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43132/0 (socket says 172.21.15.91:43132) 2024-02-02T13:31:03.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 -- 172.21.15.91:0/2626135462 learned_addr learned my addr 172.21.15.91:0/2626135462 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 -- 172.21.15.91:0/2626135462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 msgr2=0x7ff5cc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 --2- 172.21.15.91:0/2626135462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 -- 172.21.15.91:0/2626135462 >> v2:172.21.15.113:3300/0 conn(0x7ff5cc06cb90 msgr2=0x7ff5cc0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 --2- 172.21.15.91:0/2626135462 >> v2:172.21.15.113:3300/0 conn(0x7ff5cc06cb90 0x7ff5cc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.534+0000 7ff5ca000640 1 -- 172.21.15.91:0/2626135462 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc06bb70 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.535+0000 7ff5ca000640 1 --2- 172.21.15.91:0/2626135462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc06c650 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc15eb40 tx=0x7ff5b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9251ab001659892 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5c9600640 1 -- 172.21.15.91:0/2626135462 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4030860 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5c9600640 1 -- 172.21.15.91:0/2626135462 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40309c0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5c9600640 1 -- 172.21.15.91:0/2626135462 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4030d50 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 -- 172.21.15.91:0/2626135462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 msgr2=0x7ff5cc06c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/2626135462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc06c650 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc15eb40 tx=0x7ff5b402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 -- 172.21.15.91:0/2626135462 shutdown_connections 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/2626135462 >> v2:172.21.15.113:3300/0 conn(0x7ff5cc06cb90 0x7ff5cc0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/2626135462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc06c650 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/2626135462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 -- 172.21.15.91:0/2626135462 >> 172.21.15.91:0/2626135462 conn(0x7ff5cc0cea90 msgr2=0x7ff5cc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.536+0000 7ff5cbe00640 1 -- 172.21.15.91:0/2626135462 shutdown_connections 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.537+0000 7ff5cbe00640 1 -- 172.21.15.91:0/2626135462 wait complete. 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.538+0000 7ff5cbe00640 1 Processor -- start 2024-02-02T13:31:03.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.538+0000 7ff5cbe00640 1 -- start start 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5cc06cb90 0x7ff5cc0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc155c60 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1492f0 con 0x7ff5cc06cb90 2024-02-02T13:31:03.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0ce630 con 0x7ff5cc15aec0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.539+0000 7ff5cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43140/0 (socket says 172.21.15.91:43140) 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 -- 172.21.15.91:0/1198795405 learned_addr learned my addr 172.21.15.91:0/1198795405 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5ca000640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5caa00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5cc06cb90 0x7ff5cc0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 -- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 msgr2=0x7ff5cc148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 -- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5cc06cb90 msgr2=0x7ff5cc0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5cc06cb90 0x7ff5cc0ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 -- 172.21.15.91:0/1198795405 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40094d0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5ca000640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5cb400640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b9e0 tx=0x7ff5c400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.540+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400caa0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.541+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400cc20 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.541+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc0cd050 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.541+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4014440 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.541+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc0cd610 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.542+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc148d90 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.543+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5c401d3b0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.544+0000 7ff5c2a00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 0x7ff5a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:03.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.544+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff5c40888e0 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.544+0000 7ff5caa00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 0x7ff5a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:03.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.545+0000 7ff5caa00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 0x7ff5a406c8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00a810 tx=0x7ff5bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:03.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.549+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5cc148d90 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.781+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5cc148d90 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.781+0000 7ff5c2a00640 1 -- 172.21.15.91:0/1198795405 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4717 (secure 0 0 0) 0x7ff5cc148d90 con 0x7ff5cc15b2a0 2024-02-02T13:31:03.783 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:03.783 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:30:18.141444+0000","modified":"2024-02-02T13:30:21.674966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6835/2746977101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2746977101},{"type":"v1","addr":"172.21.15.113:6835","nonce":2746977101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-02T13:31:03.783 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 48 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 msgr2=0x7ff5a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 0x7ff5a406c8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00a810 tx=0x7ff5bc008040 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 msgr2=0x7ff5cc157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b9e0 tx=0x7ff5c400bea0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 shutdown_connections 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff5a406a430 0x7ff5a406c8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5cc15aec0 0x7ff5cc148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5cc15b2a0 0x7ff5cc157030 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 --2- 172.21.15.91:0/1198795405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5cc06cb90 0x7ff5cc0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 >> 172.21.15.91:0/1198795405 conn(0x7ff5cc0cea90 msgr2=0x7ff5cc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 shutdown_connections 2024-02-02T13:31:03.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:03.784+0000 7ff5cbe00640 1 -- 172.21.15.91:0/1198795405 wait complete. 2024-02-02T13:31:03.804 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:04.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.005+0000 7fd66dc00640 1 Processor -- start 2024-02-02T13:31:04.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.005+0000 7fd66dc00640 1 -- start start 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.006+0000 7fd66dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 0x7fd6680cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.006+0000 7fd66dc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd66806f3f0 0x7fd6680ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.006+0000 7fd66dc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66806c1c0 con 0x7fd66815aec0 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.006+0000 7fd66dc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66806b5f0 con 0x7fd66806f3f0 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.006+0000 7fd66dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66806cd60 con 0x7fd66815b2a0 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd66806f3f0 0x7fd6680ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd66806f3f0 0x7fd6680ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43152/0 (socket says 172.21.15.91:43152) 2024-02-02T13:31:04.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 -- 172.21.15.91:0/972900006 learned_addr learned my addr 172.21.15.91:0/972900006 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd667400640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 0x7fd6680cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 -- 172.21.15.91:0/972900006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 msgr2=0x7fd6680cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 0x7fd6680cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 -- 172.21.15.91:0/972900006 >> v1:172.21.15.113:6789/0 conn(0x7fd66815aec0 legacy=0x7fd66806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd66d200640 1 -- 172.21.15.91:0/972900006 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66806b890 con 0x7fd66806f3f0 2024-02-02T13:31:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.007+0000 7fd667400640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 0x7fd6680cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:04.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.008+0000 7fd66d200640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66806f3f0 0x7fd6680ca620 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b6b423c300f702 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd666a00640 1 -- 172.21.15.91:0/972900006 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66002cb00 con 0x7fd66806f3f0 2024-02-02T13:31:04.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd666a00640 1 -- 172.21.15.91:0/972900006 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66002cc60 con 0x7fd66806f3f0 2024-02-02T13:31:04.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd666a00640 1 -- 172.21.15.91:0/972900006 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600343f0 con 0x7fd66806f3f0 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 -- 172.21.15.91:0/972900006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66806f3f0 msgr2=0x7fd6680ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66806f3f0 0x7fd6680ca620 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66002bf60 comp rx=0 tx=0).stop 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 -- 172.21.15.91:0/972900006 shutdown_connections 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66806f3f0 0x7fd6680ca620 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 --2- 172.21.15.91:0/972900006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66815b2a0 0x7fd6680cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 -- 172.21.15.91:0/972900006 >> 172.21.15.91:0/972900006 conn(0x7fd6680cea90 msgr2=0x7fd6680ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 -- 172.21.15.91:0/972900006 shutdown_connections 2024-02-02T13:31:04.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.009+0000 7fd66dc00640 1 -- 172.21.15.91:0/972900006 wait complete. 2024-02-02T13:31:04.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.011+0000 7fd66dc00640 1 Processor -- start 2024-02-02T13:31:04.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.011+0000 7fd66dc00640 1 -- start start 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66806f3f0 0x7fd6681f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd66815aec0 0x7fd6680b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681f4650 con 0x7fd66815b2a0 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681f3aa0 con 0x7fd66815aec0 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815a2e0 con 0x7fd66806f3f0 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43166/0 (socket says 172.21.15.91:43166) 2024-02-02T13:31:04.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.012+0000 7fd66d200640 1 -- 172.21.15.91:0/2394602233 learned_addr learned my addr 172.21.15.91:0/2394602233 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd667400640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd66815aec0 0x7fd6680b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd667e00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66806f3f0 0x7fd6681f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 -- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66806f3f0 msgr2=0x7fd6681f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66806f3f0 0x7fd6681f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 -- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd66815aec0 msgr2=0x7fd6680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd66815aec0 0x7fd6680b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 -- 172.21.15.91:0/2394602233 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009b30 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66d200640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660009510 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66003e450 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66003e800 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.013+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6681f3cb0 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.014+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6681f4270 con 0x7fd66815b2a0 2024-02-02T13:31:04.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.015+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66806bb50 con 0x7fd66815b2a0 2024-02-02T13:31:04.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.015+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd66003e960 con 0x7fd66815b2a0 2024-02-02T13:31:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.016+0000 7fd65fe00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 0x7fd64006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.016+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd6600ab6f0 con 0x7fd66815b2a0 2024-02-02T13:31:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.017+0000 7fd667e00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 0x7fd64006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.017+0000 7fd667e00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 0x7fd64006c8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd66815d1d0 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.021+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd66806bb50 con 0x7fd66815b2a0 2024-02-02T13:31:04.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.210+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6681f4790 con 0x7fd66815b2a0 2024-02-02T13:31:04.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.212+0000 7fd65fe00640 1 -- 172.21.15.91:0/2394602233 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16154 (secure 0 0 0) 0x7fd6681f4790 con 0x7fd66815b2a0 2024-02-02T13:31:04.214 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:04.214 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":116,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:30:48.529769+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:30:15.902582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-02T13:30:17.156657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.214+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 msgr2=0x7fd64006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.214+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 0x7fd64006c8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd66815d1d0 tx=0x7fd658009040 comp rx=0 tx=0).stop 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.214+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 msgr2=0x7fd6680b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.214+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd660004a40 comp rx=0 tx=0).stop 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 shutdown_connections 2024-02-02T13:31:04.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64006a430 0x7fd64006c8f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd66815b2a0 0x7fd6680b0c40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd66815aec0 0x7fd6680b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 --2- 172.21.15.91:0/2394602233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd66806f3f0 0x7fd6681f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 >> 172.21.15.91:0/2394602233 conn(0x7fd6680cea90 msgr2=0x7fd668070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 shutdown_connections 2024-02-02T13:31:04.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.215+0000 7fd66dc00640 1 -- 172.21.15.91:0/2394602233 wait complete. 2024-02-02T13:31:04.235 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:31:04.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.447+0000 7f2255800640 1 Processor -- start 2024-02-02T13:31:04.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.447+0000 7f2255800640 1 -- start start 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2255800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f22500adf20 0x7f22500b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2255800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500b0540 con 0x7f225006ce50 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2255800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225015b2d0 con 0x7f225015aef0 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2255800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500ca410 con 0x7f22500adf20 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2247200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f225006ce50 0x7f225006b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39466/0 (socket says 172.21.15.91:39466) 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2247200640 1 -- 172.21.15.91:0/626651672 learned_addr learned my addr 172.21.15.91:0/626651672 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:04.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.449+0000 7f2254e00640 1 --2- 172.21.15.91:0/626651672 >> v2:172.21.15.91:3300/0 conn(0x7f22500adf20 0x7f22500b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.450+0000 7f2254e00640 1 -- 172.21.15.91:0/626651672 >> v1:172.21.15.113:6789/0 conn(0x7f225015aef0 legacy=0x7f2250070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.450+0000 7f2254e00640 1 -- 172.21.15.91:0/626651672 >> v1:172.21.15.91:6789/0 conn(0x7f225006ce50 legacy=0x7f225006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.450+0000 7f2254e00640 1 -- 172.21.15.91:0/626651672 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500b07e0 con 0x7f22500adf20 2024-02-02T13:31:04.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.450+0000 7f224f400640 1 -- 172.21.15.91:0/626651672 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458542343 0 0) 0x7f22500b0540 con 0x7f225006ce50 2024-02-02T13:31:04.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.451+0000 7f2254e00640 1 --2- 172.21.15.91:0/626651672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f22500b0d00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2248009d90 tx=0x7f224802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd067bf595757d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.451+0000 7f224f400640 1 -- 172.21.15.91:0/626651672 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224802cc60 con 0x7f22500adf20 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.451+0000 7f224f400640 1 -- 172.21.15.91:0/626651672 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224802cdc0 con 0x7f22500adf20 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.451+0000 7f224f400640 1 -- 172.21.15.91:0/626651672 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248034660 con 0x7f22500adf20 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 -- 172.21.15.91:0/626651672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 msgr2=0x7f22500b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 --2- 172.21.15.91:0/626651672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f22500b0d00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2248009d90 tx=0x7f224802beb0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 -- 172.21.15.91:0/626651672 shutdown_connections 2024-02-02T13:31:04.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 --2- 172.21.15.91:0/626651672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f22500b0d00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 -- 172.21.15.91:0/626651672 >> 172.21.15.91:0/626651672 conn(0x7f22500cb4a0 msgr2=0x7f22500cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:04.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 -- 172.21.15.91:0/626651672 shutdown_connections 2024-02-02T13:31:04.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.452+0000 7f2255800640 1 -- 172.21.15.91:0/626651672 wait complete. 2024-02-02T13:31:04.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.453+0000 7f2255800640 1 Processor -- start 2024-02-02T13:31:04.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.454+0000 7f2255800640 1 -- start start 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.454+0000 7f2255800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f225006ce50 0x7f225014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.454+0000 7f2255800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.454+0000 7f2255800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 0x7f2250155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2255800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500ccc30 con 0x7f22500adf20 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2255800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500ccf10 con 0x7f225015aef0 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2255800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22501f29b0 con 0x7f225006ce50 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43178/0 (socket says 172.21.15.91:43178) 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 -- 172.21.15.91:0/2655103752 learned_addr learned my addr 172.21.15.91:0/2655103752 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2254e00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 0x7f2250155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 -- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f225006ce50 msgr2=0x7f225014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f224fe00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f225006ce50 0x7f225014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f225006ce50 0x7f225014dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 -- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 msgr2=0x7f2250155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 0x7f2250155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2247200640 1 -- 172.21.15.91:0/2655103752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2248009a70 con 0x7f22500adf20 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.455+0000 7f2254e00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 0x7f2250155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:04.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f2247200640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f223800c530 tx=0x7f223800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22380074e0 con 0x7f22500adf20 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238007660 con 0x7f22500adf20 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2250156510 con 0x7f22500adf20 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238004a20 con 0x7f22500adf20 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.456+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2250156ad0 con 0x7f22500adf20 2024-02-02T13:31:04.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.457+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225004dff0 con 0x7f22500adf20 2024-02-02T13:31:04.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.458+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2238004b80 con 0x7f22500adf20 2024-02-02T13:31:04.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.459+0000 7f224cc00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 0x7f222806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.459+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2238087e20 con 0x7f22500adf20 2024-02-02T13:31:04.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.459+0000 7f224fe00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 0x7f222806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.460+0000 7f224fe00640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 0x7f222806c8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2240006fd0 tx=0x7f2240008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.464+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22380533f0 con 0x7f22500adf20 2024-02-02T13:31:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.697+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f22500afba0 con 0x7f22500adf20 2024-02-02T13:31:04.699 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:31:04.698+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:31:04.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.718+0000 7f224cc00640 1 -- 172.21.15.91:0/2655103752 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2238018090 con 0x7f22500adf20 2024-02-02T13:31:04.720 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 msgr2=0x7f222806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 0x7f222806c8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2240006fd0 tx=0x7f2240008040 comp rx=0 tx=0).stop 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 msgr2=0x7f2250155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f223800c530 tx=0x7f223800c9f0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 shutdown_connections 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f222806a430 0x7f222806c8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225015aef0 0x7f2250155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f22500adf20 0x7f2250155930 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 --2- 172.21.15.91:0/2655103752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f225006ce50 0x7f225014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.721+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 >> 172.21.15.91:0/2655103752 conn(0x7f22500cb4a0 msgr2=0x7f2250158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:04.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.722+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 shutdown_connections 2024-02-02T13:31:04.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.722+0000 7f2255800640 1 -- 172.21.15.91:0/2655103752 wait complete. 2024-02-02T13:31:04.742 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:31:04.840 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:31:04.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.943+0000 7fe9b1c00640 1 Processor -- start 2024-02-02T13:31:04.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.943+0000 7fe9b1c00640 1 -- start start 2024-02-02T13:31:04.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9b1c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe9ac1612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9b1c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0cb4c0 con 0x7fe9ac15cae0 2024-02-02T13:31:04.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9b1c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0cb710 con 0x7fe9ac0b22f0 2024-02-02T13:31:04.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9b1c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac161810 con 0x7fe9ac160e50 2024-02-02T13:31:04.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9ab400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe9ac15cae0 0x7fe9ac15cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39476/0 (socket says 172.21.15.91:39476) 2024-02-02T13:31:04.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9ab400640 1 -- 172.21.15.91:0/3648938171 learned_addr learned my addr 172.21.15.91:0/3648938171 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.945+0000 7fe9a1e00640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe9ac1612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414237246 0 0) 0x7fe9ac0cb4c0 con 0x7fe9ac15cae0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe98c003520 con 0x7fe9ac15cae0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858713922 0 0) 0x7fe98c003520 con 0x7fe9ac15cae0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 msgr2=0x7fe9ac1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe9ac1612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 >> v1:172.21.15.113:6789/0 conn(0x7fe9ac0b22f0 legacy=0x7fe9ac0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0cb9f0 con 0x7fe9ac15cae0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe9a4001670 con 0x7fe9ac15cae0 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 >> v1:172.21.15.91:6789/0 conn(0x7fe9ac15cae0 legacy=0x7fe9ac15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe98c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe98c003f20 0x7fe98c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe98c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac161810 con 0x7fe98c004d00 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c00bb10 con 0x7fe98c003f20 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.946+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c00bcd0 con 0x7fe9ac160e50 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe98c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9aaa00640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe98c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9a1e00640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe98c003f20 0x7fe98c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 -- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 msgr2=0x7fe98c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac160e50 0x7fe98c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 -- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe98c003f20 msgr2=0x7fe98c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe98c003f20 0x7fe98c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9ab400640 1 -- 172.21.15.91:0/3648938171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c003520 con 0x7fe98c004d00 2024-02-02T13:31:04.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.947+0000 7fe9a1e00640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe98c003f20 0x7fe98c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:04.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.948+0000 7fe9ab400640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe98c005180 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c240 tx=0x7fe98c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4bf351df7b22af server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.948+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4005ad0 con 0x7fe98c004d00 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.948+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4005c30 con 0x7fe98c004d00 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.948+0000 7fe9aa000640 1 -- 172.21.15.91:0/3648938171 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a402c8b0 con 0x7fe98c004d00 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 -- 172.21.15.91:0/3648938171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 msgr2=0x7fe98c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/3648938171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe98c005180 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c240 tx=0x7fe98c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9ab400640 1 -- 172.21.15.91:0/3648938171 reap_dead start 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 -- 172.21.15.91:0/3648938171 shutdown_connections 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 -- 172.21.15.91:0/3648938171 >> 172.21.15.91:0/3648938171 conn(0x7fe9ac09a140 msgr2=0x7fe9ac0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 -- 172.21.15.91:0/3648938171 shutdown_connections 2024-02-02T13:31:04.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.949+0000 7fe9b1c00640 1 -- 172.21.15.91:0/3648938171 wait complete. 2024-02-02T13:31:04.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.951+0000 7fe9b1c00640 1 Processor -- start 2024-02-02T13:31:04.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.951+0000 7fe9b1c00640 1 -- start start 2024-02-02T13:31:04.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac15d380 0x7fe9ac15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac15a800 0x7fe9ac0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac15a620 con 0x7fe98c004d00 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c4950 con 0x7fe9ac15a800 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9b1c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c4c30 con 0x7fe9ac15d380 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43204/0 (socket says 172.21.15.91:43204) 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 -- 172.21.15.91:0/2446874449 learned_addr learned my addr 172.21.15.91:0/2446874449 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 -- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac15d380 msgr2=0x7fe9ac15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac15d380 0x7fe9ac15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 -- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac15a800 msgr2=0x7fe9ac0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.952+0000 7fe9aaa00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac15a800 0x7fe9ac0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9aaa00640 1 -- 172.21.15.91:0/2446874449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0c4f10 con 0x7fe98c004d00 2024-02-02T13:31:04.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9aaa00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e550 tx=0x7fe99c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0052e0 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c005440 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c003810 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac0c51f0 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.953+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac0c57b0 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.954+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac0c5970 con 0x7fe98c004d00 2024-02-02T13:31:04.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.955+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe99c003970 con 0x7fe98c004d00 2024-02-02T13:31:04.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.956+0000 7fe9a3400640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 0x7fe98406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:04.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.956+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe99c0885a0 con 0x7fe98c004d00 2024-02-02T13:31:04.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.956+0000 7fe9a1e00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 0x7fe98406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:04.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.957+0000 7fe9a1e00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 0x7fe98406c920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ccf70 tx=0x7fe994009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:04.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:04.960+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe99c053b30 con 0x7fe98c004d00 2024-02-02T13:31:05.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.186+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9ac0cbf80 con 0x7fe98c004d00 2024-02-02T13:31:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.722+0000 7fe9a3400640 1 -- 172.21.15.91:0/2446874449 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe99c057150 con 0x7fe98c004d00 2024-02-02T13:31:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.725+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 msgr2=0x7fe98406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.725+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 0x7fe98406c920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ccf70 tx=0x7fe994009750 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.725+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 msgr2=0x7fe9ac0b6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.725+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e550 tx=0x7fe99c00ea10 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 shutdown_connections 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe98406a460 0x7fe98406c920 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9ac15d380 0x7fe9ac15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac15a800 0x7fe9ac0cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 --2- 172.21.15.91:0/2446874449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe98c004d00 0x7fe9ac0b6550 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 >> 172.21.15.91:0/2446874449 conn(0x7fe9ac09a140 msgr2=0x7fe9ac0d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 shutdown_connections 2024-02-02T13:31:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.726+0000 7fe9b1c00640 1 -- 172.21.15.91:0/2446874449 wait complete. 2024-02-02T13:31:05.748 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:31:05.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.955+0000 7f0420c00640 1 Processor -- start 2024-02-02T13:31:05.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.955+0000 7f0420c00640 1 -- start start 2024-02-02T13:31:05.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.956+0000 7f0420c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f041c15a010 0x7f041c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.956+0000 7f0420c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 0x7f041c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0420c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0cc7c0 con 0x7f041c0cd470 2024-02-02T13:31:05.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0420c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0ce930 con 0x7f041c15a010 2024-02-02T13:31:05.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0420c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c06c890 con 0x7f041c0ce130 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f041c15a010 0x7f041c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f041c15a010 0x7f041c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43220/0 (socket says 172.21.15.91:43220) 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 -- 172.21.15.91:0/297132312 learned_addr learned my addr 172.21.15.91:0/297132312 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0413e00640 1 --2- 172.21.15.91:0/297132312 >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 0x7f041c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 -- 172.21.15.91:0/297132312 >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 msgr2=0x7f041c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 --2- 172.21.15.91:0/297132312 >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 0x7f041c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 -- 172.21.15.91:0/297132312 >> v1:172.21.15.113:6789/0 conn(0x7f041c0cd470 legacy=0x7f041c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:05.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0419e00640 1 -- 172.21.15.91:0/297132312 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c06cb70 con 0x7f041c15a010 2024-02-02T13:31:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.957+0000 7f0413e00640 1 --2- 172.21.15.91:0/297132312 >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 0x7f041c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.959+0000 7f0419e00640 1 --2- 172.21.15.91:0/297132312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c15a010 0x7f041c0c9670 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f041c0b1870 tx=0x7f040c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cd904b7e44bd87 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:05.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.959+0000 7f0419400640 1 -- 172.21.15.91:0/297132312 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c02ee00 con 0x7f041c15a010 2024-02-02T13:31:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.959+0000 7f0419400640 1 -- 172.21.15.91:0/297132312 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c036440 con 0x7f041c15a010 2024-02-02T13:31:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.959+0000 7f0419400640 1 -- 172.21.15.91:0/297132312 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0367d0 con 0x7f041c15a010 2024-02-02T13:31:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 -- 172.21.15.91:0/297132312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c15a010 msgr2=0x7f041c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 --2- 172.21.15.91:0/297132312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c15a010 0x7f041c0c9670 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f041c0b1870 tx=0x7f040c02e050 comp rx=0 tx=0).stop 2024-02-02T13:31:05.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 -- 172.21.15.91:0/297132312 shutdown_connections 2024-02-02T13:31:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 --2- 172.21.15.91:0/297132312 >> v2:172.21.15.113:3300/0 conn(0x7f041c0ce130 0x7f041c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 --2- 172.21.15.91:0/297132312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c15a010 0x7f041c0c9670 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 -- 172.21.15.91:0/297132312 >> 172.21.15.91:0/297132312 conn(0x7f041c0cec80 msgr2=0x7f041c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 -- 172.21.15.91:0/297132312 shutdown_connections 2024-02-02T13:31:05.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.960+0000 7f0420c00640 1 -- 172.21.15.91:0/297132312 wait complete. 2024-02-02T13:31:05.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.962+0000 7f0420c00640 1 Processor -- start 2024-02-02T13:31:05.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.962+0000 7f0420c00640 1 -- start start 2024-02-02T13:31:05.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15a010 0x7f041c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b42b0 0x7f041c156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c15ea70 con 0x7f041c0ce130 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c157bf0 con 0x7f041c15a010 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0420c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c157ed0 con 0x7f041c0b42b0 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43236/0 (socket says 172.21.15.91:43236) 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 -- 172.21.15.91:0/116131382 learned_addr learned my addr 172.21.15.91:0/116131382 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0413e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15a010 0x7f041c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 -- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b42b0 msgr2=0x7f041c156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b42b0 0x7f041c156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 -- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15a010 msgr2=0x7f041c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15a010 0x7f041c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:05.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.963+0000 7f0419e00640 1 -- 172.21.15.91:0/116131382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c00b770 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0419e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f040c02e090 tx=0x7f040c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c034b80 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c034d00 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c1581b0 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c041680 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.964+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1566b0 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.965+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c1586d0 con 0x7f041c0ce130 2024-02-02T13:31:05.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.966+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f040c0403e0 con 0x7f041c0ce130 2024-02-02T13:31:05.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.967+0000 7f0412000640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 0x7f03f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.967+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f040c0ab170 con 0x7f041c0ce130 2024-02-02T13:31:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.967+0000 7f0413e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 0x7f03f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.968+0000 7f0413e00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 0x7f03f406c920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ca880 tx=0x7f0404008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:05.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:05.971+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f041c1586d0 con 0x7f041c0ce130 2024-02-02T13:31:06.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.160+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f041c1586d0 con 0x7f041c0ce130 2024-02-02T13:31:06.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.738+0000 7f0412000640 1 -- 172.21.15.91:0/116131382 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) v1 ==== 181+0+0 (secure 0 0 0) 0x7f041c1586d0 con 0x7f041c0ce130 2024-02-02T13:31:06.740 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:31:06.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 msgr2=0x7f03f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 0x7f03f406c920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ca880 tx=0x7f0404008040 comp rx=0 tx=0).stop 2024-02-02T13:31:06.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 msgr2=0x7f041c1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f040c02e090 tx=0x7f040c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 shutdown_connections 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f03f406a460 0x7f03f406c920 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b42b0 0x7f041c156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15a010 0x7f041c15e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 --2- 172.21.15.91:0/116131382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f041c0ce130 0x7f041c1589b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.743+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 >> 172.21.15.91:0/116131382 conn(0x7f041c0cec80 msgr2=0x7f041c156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:06.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.744+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 shutdown_connections 2024-02-02T13:31:06.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.745+0000 7f0420c00640 1 -- 172.21.15.91:0/116131382 wait complete. 2024-02-02T13:31:06.767 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:31:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.968+0000 7fee05a00640 1 Processor -- start 2024-02-02T13:31:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.968+0000 7fee05a00640 1 -- start start 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.969+0000 7fee05a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fee000c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.969+0000 7fee05a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee000ce130 0x7fee000ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fee05a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee000cc7c0 con 0x7fee000cd470 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fee05a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee000ce930 con 0x7fee000ce130 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fee05a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0006c890 con 0x7fee0015a010 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedff400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fee000cd470 0x7fee000b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39492/0 (socket says 172.21.15.91:39492) 2024-02-02T13:31:06.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedff400640 1 -- 172.21.15.91:0/1378583452 learned_addr learned my addr 172.21.15.91:0/1378583452 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:06.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedfea00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fee000c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487529088 0 0) 0x7fee000cc7c0 con 0x7fee000cd470 2024-02-02T13:31:06.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> v2:172.21.15.113:3300/0 conn(0x7fee000ce130 0x7fee000ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.970+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede0005730 con 0x7fee000cd470 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891566402 0 0) 0x7fede0005730 con 0x7fee000cd470 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 msgr2=0x7fee000c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fee000c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 >> v2:172.21.15.113:3300/0 conn(0x7fee000ce130 msgr2=0x7fee000ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 --2- 172.21.15.91:0/1378583452 >> v2:172.21.15.113:3300/0 conn(0x7fee000ce130 0x7fee000ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0006cb70 con 0x7fee000cd470 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fedf80019f0 con 0x7fee000cd470 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> v2:172.21.15.113:3300/0 conn(0x7fee000ce130 0x7fee000ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 >> v1:172.21.15.91:6789/0 conn(0x7fee000cd470 legacy=0x7fee000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fede0005b80 0x7fede0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fede0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fede0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000ce930 con 0x7fede00069d0 2024-02-02T13:31:06.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0006c890 con 0x7fede0005b80 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede000dbe0 con 0x7fee0015a010 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.971+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fede0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedf7e00640 1 -- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 msgr2=0x7fede0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedff400640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fede0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fede0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedf7e00640 1 -- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fede0005b80 msgr2=0x7fede0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fede0005b80 0x7fede0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedf7e00640 1 -- 172.21.15.91:0/1378583452 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0005730 con 0x7fede00069d0 2024-02-02T13:31:06.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.972+0000 7fedff400640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee0015a010 0x7fede0007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fedf7e00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fede0006e50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fede80098a0 tx=0x7fede0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb920798f21d5169 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8004230 con 0x7fede00069d0 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8004390 con 0x7fede00069d0 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fedfe000640 1 -- 172.21.15.91:0/1378583452 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8003830 con 0x7fede00069d0 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fee05a00640 1 -- 172.21.15.91:0/1378583452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 msgr2=0x7fede0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fee05a00640 1 --2- 172.21.15.91:0/1378583452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fede0006e50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fede80098a0 tx=0x7fede0007a00 comp rx=0 tx=0).stop 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.973+0000 7fedff400640 1 -- 172.21.15.91:0/1378583452 reap_dead start 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.974+0000 7fee05a00640 1 -- 172.21.15.91:0/1378583452 shutdown_connections 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.974+0000 7fee05a00640 1 -- 172.21.15.91:0/1378583452 >> 172.21.15.91:0/1378583452 conn(0x7fee000cec80 msgr2=0x7fee000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.974+0000 7fee05a00640 1 -- 172.21.15.91:0/1378583452 shutdown_connections 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.974+0000 7fee05a00640 1 -- 172.21.15.91:0/1378583452 wait complete. 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.975+0000 7fee05a00640 1 Processor -- start 2024-02-02T13:31:06.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 -- start start 2024-02-02T13:31:06.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee000b1870 0x7fee0015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee000b1cf0 0x7fee0015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000c37c0 con 0x7fede00069d0 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000c3aa0 con 0x7fee000b1cf0 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.976+0000 7fee05a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee000c3d80 con 0x7fee000b1870 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43246/0 (socket says 172.21.15.91:43246) 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 -- 172.21.15.91:0/235240912 learned_addr learned my addr 172.21.15.91:0/235240912 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 -- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee000b1870 msgr2=0x7fee0015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee000b1870 0x7fee0015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 -- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee000b1cf0 msgr2=0x7fee0015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee000b1cf0 0x7fee0015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:06.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 -- 172.21.15.91:0/235240912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee000c4060 con 0x7fede00069d0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.977+0000 7fedfea00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e7a0 tx=0x7fedf000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.978+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000eeb0 con 0x7fede00069d0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.978+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf000c940 con 0x7fede00069d0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.978+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000cd10 con 0x7fede00069d0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.978+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee000c4340 con 0x7fede00069d0 2024-02-02T13:31:06.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.978+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee000c4860 con 0x7fede00069d0 2024-02-02T13:31:06.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.979+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee00066cf0 con 0x7fede00069d0 2024-02-02T13:31:06.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.980+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fedf0013470 con 0x7fede00069d0 2024-02-02T13:31:06.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.981+0000 7fedf6a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 0x7fedd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:06.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.981+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fedf00887b0 con 0x7fede00069d0 2024-02-02T13:31:06.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.981+0000 7fedf7e00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 0x7fedd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:06.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.982+0000 7fedf7e00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 0x7fedd806c8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fede8035000 tx=0x7fede8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:06.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:06.986+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fedf0053ee0 con 0x7fede00069d0 2024-02-02T13:31:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.175+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee000b4c00 con 0x7fede00069d0 2024-02-02T13:31:07.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.738+0000 7fedf6a00640 1 -- 172.21.15.91:0/235240912 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) v1 ==== 169+0+0 (secure 0 0 0) 0x7fedf0057500 con 0x7fede00069d0 2024-02-02T13:31:07.740 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 msgr2=0x7fedd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 0x7fedd806c8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fede8035000 tx=0x7fede8030000 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 msgr2=0x7fee0014f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e7a0 tx=0x7fedf000ec60 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 shutdown_connections 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fedd806a430 0x7fedd806c8f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee000b1870 0x7fee0015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee000b1cf0 0x7fee0015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 --2- 172.21.15.91:0/235240912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fede00069d0 0x7fee0014f900 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 >> 172.21.15.91:0/235240912 conn(0x7fee000cec80 msgr2=0x7fee000c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 shutdown_connections 2024-02-02T13:31:07.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.743+0000 7fee05a00640 1 -- 172.21.15.91:0/235240912 wait complete. 2024-02-02T13:31:07.767 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:31:07.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.982+0000 7f9333e00640 1 Processor -- start 2024-02-02T13:31:07.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.982+0000 7f9333e00640 1 -- start start 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.983+0000 7f9333e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f933415af00 0x7f933406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.983+0000 7f9333e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f933415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.983+0000 7f9333e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933415b2e0 con 0x7f93340ca9a0 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.983+0000 7f9333e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933406d140 con 0x7f933415af00 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.983+0000 7f9333e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340ca2a0 con 0x7f93340ad9d0 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9329e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f93340ca9a0 0x7f9334070bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39500/0 (socket says 172.21.15.91:39500) 2024-02-02T13:31:07.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9329e00640 1 -- 172.21.15.91:0/4233165423 learned_addr learned my addr 172.21.15.91:0/4233165423 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171949358 0 0) 0x7f933415b2e0 con 0x7f93340ca9a0 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9314003520 con 0x7f93340ca9a0 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9333400640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f933415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332a00640 1 --2- 172.21.15.91:0/4233165423 >> v2:172.21.15.113:3300/0 conn(0x7f933415af00 0x7f933406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180790634 0 0) 0x7f9314003520 con 0x7f93340ca9a0 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 msgr2=0x7f933415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f933415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 >> v2:172.21.15.113:3300/0 conn(0x7f933415af00 msgr2=0x7f933406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 --2- 172.21.15.91:0/4233165423 >> v2:172.21.15.113:3300/0 conn(0x7f933415af00 0x7f933406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340ca580 con 0x7f93340ca9a0 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f931c001670 con 0x7f93340ca9a0 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.984+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 >> v1:172.21.15.91:6789/0 conn(0x7f93340ca9a0 legacy=0x7f9334070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f9314003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f93140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9314004d00 0x7f9314005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933406d140 con 0x7f9314003970 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340ca2a0 con 0x7f9314004d00 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931400ba30 con 0x7f93340ad9d0 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f9314003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9329e00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f93140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 -- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 msgr2=0x7f93140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:07.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340ad9d0 0x7f93140047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 -- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9314004d00 msgr2=0x7f9314005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:07.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9314004d00 0x7f9314005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.985+0000 7f9332a00640 1 -- 172.21.15.91:0/4233165423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314003520 con 0x7f9314003970 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.986+0000 7f9332a00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f9314003d90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f932400ba60 tx=0x7f93140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e54a538753a59b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324032680 con 0x7f9314003970 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240327e0 con 0x7f9314003970 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9332000640 1 -- 172.21.15.91:0/4233165423 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324032b70 con 0x7f9314003970 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 -- 172.21.15.91:0/4233165423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 msgr2=0x7f9314003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 --2- 172.21.15.91:0/4233165423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f9314003d90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f932400ba60 tx=0x7f93140057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333400640 1 -- 172.21.15.91:0/4233165423 reap_dead start 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 -- 172.21.15.91:0/4233165423 shutdown_connections 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 -- 172.21.15.91:0/4233165423 >> 172.21.15.91:0/4233165423 conn(0x7f93340b0560 msgr2=0x7f93340b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 -- 172.21.15.91:0/4233165423 shutdown_connections 2024-02-02T13:31:07.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.987+0000 7f9333e00640 1 -- 172.21.15.91:0/4233165423 wait complete. 2024-02-02T13:31:07.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.989+0000 7f9333e00640 1 Processor -- start 2024-02-02T13:31:07.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.989+0000 7f9333e00640 1 -- start start 2024-02-02T13:31:07.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334159290 0x7f93340b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 0x7f93340ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340ac3b0 con 0x7f9314003970 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341f2b70 con 0x7f9334159290 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9333e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b3ef0 con 0x7f93340b3a10 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.990+0000 7f9332a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43262/0 (socket says 172.21.15.91:43262) 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 -- 172.21.15.91:0/132181635 learned_addr learned my addr 172.21.15.91:0/132181635 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9333400640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 0x7f93340ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9329e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334159290 0x7f93340b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 -- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 msgr2=0x7f93340ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 0x7f93340ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 -- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334159290 msgr2=0x7f93340b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334159290 0x7f93340b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 -- 172.21.15.91:0/132181635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340a88c0 con 0x7f9314003970 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9333400640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 0x7f93340ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:07.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.991+0000 7f9332a00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9324035040 tx=0x7f9324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:07.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.992+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324004220 con 0x7f9314003970 2024-02-02T13:31:07.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.992+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324004380 con 0x7f9314003970 2024-02-02T13:31:07.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.992+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93340a8ba0 con 0x7f9314003970 2024-02-02T13:31:07.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.992+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324030ac0 con 0x7f9314003970 2024-02-02T13:31:07.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.992+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340a9160 con 0x7f9314003970 2024-02-02T13:31:07.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.993+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93340a92f0 con 0x7f9314003970 2024-02-02T13:31:07.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.994+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9324030c20 con 0x7f9314003970 2024-02-02T13:31:07.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.994+0000 7f932be00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:07.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.995+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f93240ab0a0 con 0x7f9314003970 2024-02-02T13:31:07.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.995+0000 7f9329e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:07.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.995+0000 7f9329e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f931c000c00 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:07.999+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f93240768f0 con 0x7f9314003970 2024-02-02T13:31:08.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.187+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f93340a92f0 con 0x7f9314003970 2024-02-02T13:31:08.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.194+0000 7f932be00640 1 -- 172.21.15.91:0/132181635 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9324079f10 con 0x7f9314003970 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 msgr2=0x7f930c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f931c000c00 tx=0x7f931c009040 comp rx=0 tx=0).stop 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 msgr2=0x7f93341f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9324035040 tx=0x7f9324004730 comp rx=0 tx=0).stop 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 shutdown_connections 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f930c06a430 0x7f930c06c8f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93340b3a10 0x7f93340ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334159290 0x7f93340b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 --2- 172.21.15.91:0/132181635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9314003970 0x7f93341f2630 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 >> 172.21.15.91:0/132181635 conn(0x7f93340b0560 msgr2=0x7f93340cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:08.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 shutdown_connections 2024-02-02T13:31:08.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.196+0000 7f9333e00640 1 -- 172.21.15.91:0/132181635 wait complete. 2024-02-02T13:31:08.222 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-02-02T13:31:08.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.403+0000 7faf4a400640 1 Processor -- start 2024-02-02T13:31:08.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.403+0000 7faf4a400640 1 -- start start 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf4a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf4415a010 0x7faf440c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf4a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf440ce130 0x7faf440ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf4a400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf440cc7c0 con 0x7faf440cd470 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf4a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440ce930 con 0x7faf440ce130 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf4a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4406c890 con 0x7faf4415a010 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf42a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf440ce130 0x7faf440ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf43e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faf440cd470 0x7faf440b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39512/0 (socket says 172.21.15.91:39512) 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf43e00640 1 -- 172.21.15.91:0/1076576649 learned_addr learned my addr 172.21.15.91:0/1076576649 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf43400640 1 --2- 172.21.15.91:0/1076576649 >> v2:172.21.15.113:3300/0 conn(0x7faf4415a010 0x7faf440c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf42a00640 1 -- 172.21.15.91:0/1076576649 >> v2:172.21.15.113:3300/0 conn(0x7faf4415a010 msgr2=0x7faf440c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.405+0000 7faf42a00640 1 --2- 172.21.15.91:0/1076576649 >> v2:172.21.15.113:3300/0 conn(0x7faf4415a010 0x7faf440c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.406+0000 7faf42a00640 1 -- 172.21.15.91:0/1076576649 >> v1:172.21.15.91:6789/0 conn(0x7faf440cd470 legacy=0x7faf440b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.406+0000 7faf42a00640 1 -- 172.21.15.91:0/1076576649 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4406cb70 con 0x7faf440ce130 2024-02-02T13:31:08.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.406+0000 7faf42000640 1 -- 172.21.15.91:0/1076576649 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125707220 0 0) 0x7faf440cc7c0 con 0x7faf440cd470 2024-02-02T13:31:08.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.407+0000 7faf42a00640 1 --2- 172.21.15.91:0/1076576649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf440ccf30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7faf440cfaa0 tx=0x7faf2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2f27827adaec7d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.407+0000 7faf42000640 1 -- 172.21.15.91:0/1076576649 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c02e070 con 0x7faf440ce130 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.407+0000 7faf42000640 1 -- 172.21.15.91:0/1076576649 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c0315a0 con 0x7faf440ce130 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.407+0000 7faf42000640 1 -- 172.21.15.91:0/1076576649 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c031930 con 0x7faf440ce130 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 -- 172.21.15.91:0/1076576649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 msgr2=0x7faf440ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 --2- 172.21.15.91:0/1076576649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf440ccf30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7faf440cfaa0 tx=0x7faf2c02b840 comp rx=0 tx=0).stop 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 -- 172.21.15.91:0/1076576649 shutdown_connections 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 --2- 172.21.15.91:0/1076576649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf440ccf30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 --2- 172.21.15.91:0/1076576649 >> v2:172.21.15.113:3300/0 conn(0x7faf4415a010 0x7faf440c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 -- 172.21.15.91:0/1076576649 >> 172.21.15.91:0/1076576649 conn(0x7faf440cec80 msgr2=0x7faf440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 -- 172.21.15.91:0/1076576649 shutdown_connections 2024-02-02T13:31:08.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.408+0000 7faf4a400640 1 -- 172.21.15.91:0/1076576649 wait complete. 2024-02-02T13:31:08.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.409+0000 7faf4a400640 1 Processor -- start 2024-02-02T13:31:08.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.410+0000 7faf4a400640 1 -- start start 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.410+0000 7faf4a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.410+0000 7faf4a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf4415a010 0x7faf441588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.410+0000 7faf4a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 0x7faf44070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.410+0000 7faf4a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440b27d0 con 0x7faf440ce130 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf4a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440708d0 con 0x7faf4406fee0 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf4a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440c6520 con 0x7faf4415a010 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43274/0 (socket says 172.21.15.91:43274) 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 -- 172.21.15.91:0/3057234288 learned_addr learned my addr 172.21.15.91:0/3057234288 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf42a00640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf4415a010 0x7faf441588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 -- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf4415a010 msgr2=0x7faf441588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf4415a010 0x7faf441588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43e00640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 0x7faf44070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 -- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 msgr2=0x7faf44070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 0x7faf44070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43400640 1 -- 172.21.15.91:0/3057234288 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c009400 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.411+0000 7faf43e00640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 0x7faf44070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf43400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7faf3400d600 tx=0x7faf3400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340125f0 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf34012770 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf440b1870 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34012b20 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.412+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf440b1e30 con 0x7faf440ce130 2024-02-02T13:31:08.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.413+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf44066cf0 con 0x7faf440ce130 2024-02-02T13:31:08.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.414+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf34012c80 con 0x7faf440ce130 2024-02-02T13:31:08.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.415+0000 7faf3b400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.415+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faf34088b70 con 0x7faf440ce130 2024-02-02T13:31:08.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.415+0000 7faf42a00640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.416+0000 7faf42a00640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 0x7faf1c06c920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf2c036000 tx=0x7faf2c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.419+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf340543c0 con 0x7faf440ce130 2024-02-02T13:31:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.674+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7faf440b4c00 con 0x7faf440ce130 2024-02-02T13:31:08.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.738+0000 7faf3b400640 1 -- 172.21.15.91:0/3057234288 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v338) v1 ==== 137+0+0 (secure 0 0 0) 0x7faf340579e0 con 0x7faf440ce130 2024-02-02T13:31:08.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 msgr2=0x7faf1c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 0x7faf1c06c920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf2c036000 tx=0x7faf2c03d040 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 msgr2=0x7faf4414cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7faf3400d600 tx=0x7faf3400dac0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 shutdown_connections 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf4415a010 0x7faf441588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf4406fee0 0x7faf44070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 --2- 172.21.15.91:0/3057234288 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf440ce130 0x7faf4414cdb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 >> 172.21.15.91:0/3057234288 conn(0x7faf440cec80 msgr2=0x7faf44070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 shutdown_connections 2024-02-02T13:31:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.741+0000 7faf4a400640 1 -- 172.21.15.91:0/3057234288 wait complete. 2024-02-02T13:31:08.760 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-02-02T13:31:08.761 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-02-02T13:31:08.761 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-02-02T13:31:08.761 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-02-02T13:31:08.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.967+0000 7fe761800640 1 Processor -- start 2024-02-02T13:31:08.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.967+0000 7fe761800640 1 -- start start 2024-02-02T13:31:08.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.968+0000 7fe761800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe75c15d3a0 0x7fe75c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.968+0000 7fe761800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0c9450 0x7fe75c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.968+0000 7fe761800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c0ce3b0 con 0x7fe75c0add80 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.968+0000 7fe761800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c0adbd0 con 0x7fe75c15d3a0 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.968+0000 7fe761800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0c59d0 con 0x7fe75c0c9450 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe75c15d3a0 0x7fe75c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe75c15d3a0 0x7fe75c06b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43280/0 (socket says 172.21.15.91:43280) 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 -- 172.21.15.91:0/678279100 learned_addr learned my addr 172.21.15.91:0/678279100 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe760e00640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0c9450 0x7fe75c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 -- 172.21.15.91:0/678279100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0c9450 msgr2=0x7fe75c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0c9450 0x7fe75c0c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 -- 172.21.15.91:0/678279100 >> v1:172.21.15.113:6789/0 conn(0x7fe75c0add80 legacy=0x7fe75c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:08.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.969+0000 7fe75be00640 1 -- 172.21.15.91:0/678279100 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0ce8e0 con 0x7fe75c15d3a0 2024-02-02T13:31:08.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.970+0000 7fe75be00640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c06b450 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00bb10 tx=0x7fe74c02dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cebb313b8c6c82a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe75b400640 1 -- 172.21.15.91:0/678279100 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02ed50 con 0x7fe75c15d3a0 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe75b400640 1 -- 172.21.15.91:0/678279100 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c036440 con 0x7fe75c15d3a0 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe75b400640 1 -- 172.21.15.91:0/678279100 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0367d0 con 0x7fe75c15d3a0 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 -- 172.21.15.91:0/678279100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 msgr2=0x7fe75c06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c06b450 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00bb10 tx=0x7fe74c02dfa0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 -- 172.21.15.91:0/678279100 shutdown_connections 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0c9450 0x7fe75c0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 --2- 172.21.15.91:0/678279100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c06b450 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 -- 172.21.15.91:0/678279100 >> 172.21.15.91:0/678279100 conn(0x7fe75c0cea90 msgr2=0x7fe75c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:08.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.971+0000 7fe761800640 1 -- 172.21.15.91:0/678279100 shutdown_connections 2024-02-02T13:31:08.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.972+0000 7fe761800640 1 -- 172.21.15.91:0/678279100 wait complete. 2024-02-02T13:31:08.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.973+0000 7fe761800640 1 Processor -- start 2024-02-02T13:31:08.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.973+0000 7fe761800640 1 -- start start 2024-02-02T13:31:08.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0add80 0x7fe75c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c0c9450 0x7fe75c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0c8a30 con 0x7fe75c15d3a0 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0c5b60 con 0x7fe75c0c9450 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.974+0000 7fe761800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15b890 con 0x7fe75c0add80 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:43284/0 (socket says 172.21.15.91:43284) 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 -- 172.21.15.91:0/2824333250 learned_addr learned my addr 172.21.15.91:0/2824333250 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe75be00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0add80 0x7fe75c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 -- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0add80 msgr2=0x7fe75c15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0add80 0x7fe75c15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 -- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c0c9450 msgr2=0x7fe75c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:08.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c0c9450 0x7fe75c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 -- 172.21.15.91:0/2824333250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c00b770 con 0x7fe75c15d3a0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe760e00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cd20 tx=0x7fe75400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.975+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754003b90 con 0x7fe75c15d3a0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.976+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754003cf0 con 0x7fe75c15d3a0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.976+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754004240 con 0x7fe75c15d3a0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.976+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c15bb70 con 0x7fe75c15d3a0 2024-02-02T13:31:08.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.976+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c15c130 con 0x7fe75c15d3a0 2024-02-02T13:31:08.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.977+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c0c5d10 con 0x7fe75c15d3a0 2024-02-02T13:31:08.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.978+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe7540043a0 con 0x7fe75c15d3a0 2024-02-02T13:31:08.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.978+0000 7fe758c00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 0x7fe73406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:08.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.978+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe754012070 con 0x7fe75c15d3a0 2024-02-02T13:31:08.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.979+0000 7fe75be00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 0x7fe73406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:08.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.979+0000 7fe75be00640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 0x7fe73406c920 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe74c038000 tx=0x7fe74c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:08.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:08.983+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe75c0c5d10 con 0x7fe75c15d3a0 2024-02-02T13:31:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.240+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7fe75c0716f0 con 0x7fe75c15d3a0 2024-02-02T13:31:09.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.741+0000 7fe758c00640 1 -- 172.21.15.91:0/2824333250 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v339) v1 ==== 131+0+0 (secure 0 0 0) 0x7fe754054840 con 0x7fe75c15d3a0 2024-02-02T13:31:09.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.744+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 msgr2=0x7fe73406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.744+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 0x7fe73406c920 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe74c038000 tx=0x7fe74c008040 comp rx=0 tx=0).stop 2024-02-02T13:31:09.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.744+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 msgr2=0x7fe75c0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.744+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cd20 tx=0x7fe75400a660 comp rx=0 tx=0).stop 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 shutdown_connections 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe73406a460 0x7fe73406c920 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75c15d3a0 0x7fe75c0b3dd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c0c9450 0x7fe75c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 --2- 172.21.15.91:0/2824333250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75c0add80 0x7fe75c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 >> 172.21.15.91:0/2824333250 conn(0x7fe75c0cea90 msgr2=0x7fe75c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 shutdown_connections 2024-02-02T13:31:09.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.745+0000 7fe761800640 1 -- 172.21.15.91:0/2824333250 wait complete. 2024-02-02T13:31:09.767 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:31:09.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.971+0000 7fd434600640 1 Processor -- start 2024-02-02T13:31:09.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.971+0000 7fd434600640 1 -- start start 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.972+0000 7fd434600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd42c151f90 0x7fd42c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.972+0000 7fd434600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c070f80 con 0x7fd42c0b0e10 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd434600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c06c600 con 0x7fd42c06f4f0 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd434600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c152370 con 0x7fd42c151f90 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd432600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd42c0b0e10 0x7fd42c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40100/0 (socket says 172.21.15.91:40100) 2024-02-02T13:31:09.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd432600640 1 -- 172.21.15.91:0/3396534040 learned_addr learned my addr 172.21.15.91:0/3396534040 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14136935 0 0) 0x7fd42c070f80 con 0x7fd42c0b0e10 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd431c00640 1 --2- 172.21.15.91:0/3396534040 >> v2:172.21.15.113:3300/0 conn(0x7fd42c151f90 0x7fd42c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.973+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40c003520 con 0x7fd42c0b0e10 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018969338 0 0) 0x7fd40c003520 con 0x7fd42c0b0e10 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 >> v2:172.21.15.113:3300/0 conn(0x7fd42c151f90 msgr2=0x7fd42c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 --2- 172.21.15.91:0/3396534040 >> v2:172.21.15.113:3300/0 conn(0x7fd42c151f90 0x7fd42c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 >> v1:172.21.15.113:6789/0 conn(0x7fd42c06f4f0 legacy=0x7fd42c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0cb010 con 0x7fd42c0b0e10 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd4240019f0 con 0x7fd42c0b0e10 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 >> v1:172.21.15.91:6789/0 conn(0x7fd42c0b0e10 legacy=0x7fd42c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 0x7fd40c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c0047c0 0x7fd40c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd40c005180 0x7fd40c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c152370 con 0x7fd40c003970 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c00c0d0 con 0x7fd40c0047c0 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c00c290 con 0x7fd40c005180 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.974+0000 7fd431c00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 0x7fd40c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd432600640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd40c005180 0x7fd40c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431200640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c0047c0 0x7fd40c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431c00640 1 -- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd40c005180 msgr2=0x7fd40c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431c00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd40c005180 0x7fd40c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431c00640 1 -- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c0047c0 msgr2=0x7fd40c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431c00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c0047c0 0x7fd40c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431c00640 1 -- 172.21.15.91:0/3396534040 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c003520 con 0x7fd40c003970 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd431200640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c0047c0 0x7fd40c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:09.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.975+0000 7fd432600640 1 -- 172.21.15.91:0/3396534040 reap_dead start 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.976+0000 7fd431c00640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 0x7fd40c003d90 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00ba30 tx=0x7fd40c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b82078d7dab596 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.976+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c032630 con 0x7fd40c003970 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.976+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c032790 con 0x7fd40c003970 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.976+0000 7fd42be00640 1 -- 172.21.15.91:0/3396534040 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c032b20 con 0x7fd40c003970 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 -- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 msgr2=0x7fd40c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 0x7fd40c003d90 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00ba30 tx=0x7fd40c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 -- 172.21.15.91:0/3396534040 shutdown_connections 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 --2- 172.21.15.91:0/3396534040 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd40c003970 0x7fd40c003d90 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 -- 172.21.15.91:0/3396534040 >> 172.21.15.91:0/3396534040 conn(0x7fd42c0cec80 msgr2=0x7fd42c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 -- 172.21.15.91:0/3396534040 shutdown_connections 2024-02-02T13:31:09.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.977+0000 7fd434600640 1 -- 172.21.15.91:0/3396534040 wait complete. 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.979+0000 7fd434600640 1 Processor -- start 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.979+0000 7fd434600640 1 -- start start 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c003970 0x7fd42c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0ada40 0x7fd42c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c1f37e0 con 0x7fd42c0c7050 2024-02-02T13:31:09.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0ae400 con 0x7fd40c003970 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd434600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0c4c10 con 0x7fd42c0ada40 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd432600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd432600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44252/0 (socket says 172.21.15.91:44252) 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd432600640 1 -- 172.21.15.91:0/3475007768 learned_addr learned my addr 172.21.15.91:0/3475007768 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd431c00640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c003970 0x7fd42c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.980+0000 7fd431200640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0ada40 0x7fd42c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 -- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0ada40 msgr2=0x7fd42c0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0ada40 0x7fd42c0adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 -- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c003970 msgr2=0x7fd42c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c003970 0x7fd42c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 -- 172.21.15.91:0/3475007768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0c4320 con 0x7fd42c0c7050 2024-02-02T13:31:09.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd432600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0b1870 tx=0x7fd42400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400b6e0 con 0x7fd42c0c7050 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42400b880 con 0x7fd42c0c7050 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400bc30 con 0x7fd42c0c7050 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c1f3500 con 0x7fd42c0c7050 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.981+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c0c48a0 con 0x7fd42c0c7050 2024-02-02T13:31:09.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.982+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c066cf0 con 0x7fd42c0c7050 2024-02-02T13:31:09.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.983+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd42400bd90 con 0x7fd42c0c7050 2024-02-02T13:31:09.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.984+0000 7fd429600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 0x7fd40406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:09.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.984+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd4240883e0 con 0x7fd42c0c7050 2024-02-02T13:31:09.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.985+0000 7fd431c00640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 0x7fd40406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:09.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.985+0000 7fd431c00640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 0x7fd40406c8f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00ba00 tx=0x7fd41c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:09.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:09.990+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd424053ce0 con 0x7fd42c0c7050 2024-02-02T13:31:10.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.181+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd42c0c4d50 con 0x7fd42c0c7050 2024-02-02T13:31:10.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.186+0000 7fd429600640 1 -- 172.21.15.91:0/3475007768 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd424057300 con 0x7fd42c0c7050 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 msgr2=0x7fd40406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 0x7fd40406c8f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00ba00 tx=0x7fd41c03d040 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 msgr2=0x7fd42c0c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0b1870 tx=0x7fd42400de40 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 shutdown_connections 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd40406a430 0x7fd40406c8f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0ada40 0x7fd42c0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd42c0c7050 0x7fd42c0c7500 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 --2- 172.21.15.91:0/3475007768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd40c003970 0x7fd42c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 >> 172.21.15.91:0/3475007768 conn(0x7fd42c0cec80 msgr2=0x7fd42c0b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 shutdown_connections 2024-02-02T13:31:10.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.189+0000 7fd434600640 1 -- 172.21.15.91:0/3475007768 wait complete. 2024-02-02T13:31:10.208 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:31:10.208 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:10.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:10.446 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:31:10.446 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:10.446 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:10.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:31:10.504 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:10.504 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:10.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:31:10.561 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:10.561 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:10.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:31:10.617 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:10.618 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:10.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:31:10.674 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:31:10.674 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:10.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.887+0000 7f430d000640 1 Processor -- start 2024-02-02T13:31:10.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.888+0000 7f430d000640 1 -- start start 2024-02-02T13:31:10.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f430d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f430815b2a0 0x7f43080cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f430d000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430806c1c0 con 0x7f430815aec0 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f430d000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430806b5f0 con 0x7f430806f3f0 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f430d000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430806cd60 con 0x7f430815b2a0 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f4307400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f430815aec0 0x7f430806c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40108/0 (socket says 172.21.15.91:40108) 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f4307400640 1 -- 172.21.15.91:0/2959253063 learned_addr learned my addr 172.21.15.91:0/2959253063 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:10.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.889+0000 7f42fe800640 1 --2- 172.21.15.91:0/2959253063 >> v2:172.21.15.91:3300/0 conn(0x7f430815b2a0 0x7f43080cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:10.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.890+0000 7f4306a00640 1 -- 172.21.15.91:0/2959253063 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909934080 0 0) 0x7f430806c1c0 con 0x7f430815aec0 2024-02-02T13:31:10.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.890+0000 7f4306a00640 1 -- 172.21.15.91:0/2959253063 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e8003520 con 0x7f430815aec0 2024-02-02T13:31:10.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.890+0000 7f42fe800640 1 -- 172.21.15.91:0/2959253063 >> v1:172.21.15.113:6789/0 conn(0x7f430806f3f0 legacy=0x7f43080ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:10.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.890+0000 7f42fe800640 1 -- 172.21.15.91:0/2959253063 >> v1:172.21.15.91:6789/0 conn(0x7f430815aec0 legacy=0x7f430806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.890+0000 7f42fe800640 1 -- 172.21.15.91:0/2959253063 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430806b890 con 0x7f430815b2a0 2024-02-02T13:31:10.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.891+0000 7f42fe800640 1 --2- 172.21.15.91:0/2959253063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080cf5b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f42f00097a0 tx=0x7f42f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6089acf85b8bcfc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:10.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.891+0000 7f4306a00640 1 -- 172.21.15.91:0/2959253063 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0030610 con 0x7f430815b2a0 2024-02-02T13:31:10.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.891+0000 7f4306a00640 1 -- 172.21.15.91:0/2959253063 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0030770 con 0x7f430815b2a0 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.891+0000 7f4306a00640 1 -- 172.21.15.91:0/2959253063 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0030b00 con 0x7f430815b2a0 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 -- 172.21.15.91:0/2959253063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 msgr2=0x7f43080cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 --2- 172.21.15.91:0/2959253063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080cf5b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f42f00097a0 tx=0x7f42f002b960 comp rx=0 tx=0).stop 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 -- 172.21.15.91:0/2959253063 shutdown_connections 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 --2- 172.21.15.91:0/2959253063 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080cf5b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 -- 172.21.15.91:0/2959253063 >> 172.21.15.91:0/2959253063 conn(0x7f43080cea90 msgr2=0x7f43080ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 -- 172.21.15.91:0/2959253063 shutdown_connections 2024-02-02T13:31:10.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.892+0000 7f430d000640 1 -- 172.21.15.91:0/2959253063 wait complete. 2024-02-02T13:31:10.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.894+0000 7f430d000640 1 Processor -- start 2024-02-02T13:31:10.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.894+0000 7f430d000640 1 -- start start 2024-02-02T13:31:10.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 0x7f4308156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:10.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f430815aec0 0x7f43080cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081f2680 con 0x7f430815b2a0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081f2890 con 0x7f430815aec0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f430d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43081f2b70 con 0x7f430806f3f0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44276/0 (socket says 172.21.15.91:44276) 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 -- 172.21.15.91:0/1794258465 learned_addr learned my addr 172.21.15.91:0/1794258465 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 -- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 msgr2=0x7f4308156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307400640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 0x7f4308156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 0x7f4308156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 -- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f430815aec0 msgr2=0x7f43080cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f430815aec0 0x7f43080cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307e00640 1 -- 172.21.15.91:0/1794258465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f0009400 con 0x7f430815b2a0 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.895+0000 7f4307400640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 0x7f4308156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:10.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f4307e00640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f430000c410 tx=0x7f430000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300005230 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300005390 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43081f2e50 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000117d0 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.896+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43081f3410 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.897+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430804dff0 con 0x7f430815b2a0 2024-02-02T13:31:10.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.898+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4300011930 con 0x7f430815b2a0 2024-02-02T13:31:10.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.899+0000 7f42ffe00640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 0x7f42e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:10.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.900+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43000879d0 con 0x7f430815b2a0 2024-02-02T13:31:10.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.900+0000 7f4307400640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 0x7f42e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:10.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.900+0000 7f4307400640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 0x7f42e006c8f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42f8001560 tx=0x7f42f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:10.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:10.904+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f43000532b0 con 0x7f430815b2a0 2024-02-02T13:31:11.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.132+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43080cacd0 con 0x7f430815b2a0 2024-02-02T13:31:11.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.133+0000 7f42ffe00640 1 -- 172.21.15.91:0/1794258465 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f4300017090 con 0x7f430815b2a0 2024-02-02T13:31:11.134 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:11.134 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/1403219475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1403219475},{"type":"v1","addr":"172.21.15.113:6834","nonce":1403219475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1261173936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1261173936},{"type":"v1","addr":"172.21.15.113:6835","nonce":1261173936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3390275179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3390275179},{"type":"v1","addr":"172.21.15.91:6836","nonce":3390275179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2659365968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2659365968},{"type":"v1","addr":"172.21.15.91:6837","nonce":2659365968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2024-02-02T13:31:11.134 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 50 2024-02-02T13:31:11.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 msgr2=0x7f42e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 0x7f42e006c8f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42f8001560 tx=0x7f42f8009040 comp rx=0 tx=0).stop 2024-02-02T13:31:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 msgr2=0x7f43080c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f430000c410 tx=0x7f430000c8d0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 shutdown_connections 2024-02-02T13:31:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f42e006a430 0x7f42e006c8f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f430815b2a0 0x7f43080c9f70 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f430815aec0 0x7f43080cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 --2- 172.21.15.91:0/1794258465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f430806f3f0 0x7f4308156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 >> 172.21.15.91:0/1794258465 conn(0x7f43080cea90 msgr2=0x7f4308158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 shutdown_connections 2024-02-02T13:31:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.136+0000 7f430d000640 1 -- 172.21.15.91:0/1794258465 wait complete. 2024-02-02T13:31:11.156 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:31:11.156 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:31:11.156 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:31:11.156 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:11.156 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:31:11.156 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:31:11.157 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:31:11.157 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:11.157 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:31:11.157 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:31:11.157 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:31:11.157 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:11.157 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:31:11.158 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:31:11.158 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:31:11.158 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:11.176 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:31:11.177 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:31:11.177 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:31:11.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:31:11.179 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:31:11.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:31:11.182 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:31:11.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:31:11.226 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:31:11.226 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:11.229 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:31:11.229 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:11.232 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:31:11.232 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:11.235 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:31:11.235 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:11.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:31:11.381 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:31:11.381 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:31:11.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-02-02T13:31:11.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-02-02T13:31:11.390 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:31:11.390 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:31:11.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.422+0000 7f0cfba00640 1 Processor -- start 2024-02-02T13:31:11.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.422+0000 7f0cfba00640 1 -- start start 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cfba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cfba00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0cf415e410 0x7f0cf40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cfba00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf41615f0 con 0x7f0cf40c9b70 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cfba00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40ca4f0 con 0x7f0cf415e410 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cfba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415ebc0 con 0x7f0cf4161ca0 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cf3e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0cf40c9b70 0x7f0cf415dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40130/0 (socket says 172.21.15.91:40130) 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.424+0000 7f0cf3e00640 1 -- 172.21.15.91:0/2729065639 learned_addr learned my addr 172.21.15.91:0/2729065639 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9a00640 1 --2- 172.21.15.91:0/2729065639 >> v2:172.21.15.91:3300/0 conn(0x7f0cf415e410 0x7f0cf40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9000640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9a00640 1 -- 172.21.15.91:0/2729065639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 msgr2=0x7f0cf40cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:11.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9a00640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9a00640 1 -- 172.21.15.91:0/2729065639 >> v1:172.21.15.91:6789/0 conn(0x7f0cf40c9b70 legacy=0x7f0cf415dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.425+0000 7f0cf9a00640 1 -- 172.21.15.91:0/2729065639 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf40cc590 con 0x7f0cf415e410 2024-02-02T13:31:11.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.426+0000 7f0cf9a00640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40c9fb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009d90 tx=0x7f0cec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30934d596b29dd7f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.428+0000 7f0cf3400640 1 -- 172.21.15.91:0/2729065639 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec02cbd0 con 0x7f0cf415e410 2024-02-02T13:31:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.428+0000 7f0cf3400640 1 -- 172.21.15.91:0/2729065639 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec02cd30 con 0x7f0cf415e410 2024-02-02T13:31:11.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.428+0000 7f0cf3400640 1 -- 172.21.15.91:0/2729065639 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec02cea0 con 0x7f0cf415e410 2024-02-02T13:31:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 -- 172.21.15.91:0/2729065639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 msgr2=0x7f0cf40c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40c9fb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009d90 tx=0x7f0cec02be20 comp rx=0 tx=0).stop 2024-02-02T13:31:11.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 -- 172.21.15.91:0/2729065639 shutdown_connections 2024-02-02T13:31:11.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40c9fb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2729065639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 -- 172.21.15.91:0/2729065639 >> 172.21.15.91:0/2729065639 conn(0x7f0cf406bf70 msgr2=0x7f0cf406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:11.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 -- 172.21.15.91:0/2729065639 shutdown_connections 2024-02-02T13:31:11.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.429+0000 7f0cfba00640 1 -- 172.21.15.91:0/2729065639 wait complete. 2024-02-02T13:31:11.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.431+0000 7f0cfba00640 1 Processor -- start 2024-02-02T13:31:11.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.431+0000 7f0cfba00640 1 -- start start 2024-02-02T13:31:11.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40c9b70 0x7f0cf41f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40ceac0 con 0x7f0cf415e410 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40ce580 con 0x7f0cf40c9b70 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cfba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40ce7e0 con 0x7f0cf4161ca0 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44328/0 (socket says 172.21.15.91:44328) 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 -- 172.21.15.91:0/2392102232 learned_addr learned my addr 172.21.15.91:0/2392102232 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf9000640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40c9b70 0x7f0cf41f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf9a00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 -- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 msgr2=0x7f0cf40ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40ad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 -- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40c9b70 msgr2=0x7f0cf41f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40c9b70 0x7f0cf41f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.432+0000 7f0cf3e00640 1 -- 172.21.15.91:0/2392102232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009a70 con 0x7f0cf415e410 2024-02-02T13:31:11.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cf3e00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00c4a0 tx=0x7f0cdc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc005ce0 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc005e40 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf40c3160 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc007270 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.433+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf40c3720 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.434+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf40c3910 con 0x7f0cf415e410 2024-02-02T13:31:11.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.435+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0cdc007490 con 0x7f0cf415e410 2024-02-02T13:31:11.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.436+0000 7f0cf0c00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:11.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.436+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cdc087ac0 con 0x7f0cf415e410 2024-02-02T13:31:11.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.436+0000 7f0cf9000640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:11.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.437+0000 7f0cf9000640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009a30 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:11.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.441+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0cdc053450 con 0x7f0cf415e410 2024-02-02T13:31:11.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:11.645+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0cf4067400 con 0x7f0cf415e410 2024-02-02T13:31:12.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.437+0000 7f0cf0c00640 1 -- 172.21.15.91:0/2392102232 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v124) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0cdc017080 con 0x7f0cf415e410 2024-02-02T13:31:12.438 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 msgr2=0x7f0ccc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009a30 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 msgr2=0x7f0cf40ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00c4a0 tx=0x7f0cdc00c960 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 shutdown_connections 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0ccc06a430 0x7f0ccc06c8f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cf4161ca0 0x7f0cf40ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0cf415e410 0x7f0cf40ce040 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 --2- 172.21.15.91:0/2392102232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cf40c9b70 0x7f0cf41f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 >> 172.21.15.91:0/2392102232 conn(0x7f0cf406bf70 msgr2=0x7f0cf4156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 shutdown_connections 2024-02-02T13:31:12.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.441+0000 7f0cfba00640 1 -- 172.21.15.91:0/2392102232 wait complete. 2024-02-02T13:31:12.462 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:31:12.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.653+0000 7f8f97c00640 1 Processor -- start 2024-02-02T13:31:12.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.653+0000 7f8f97c00640 1 -- start start 2024-02-02T13:31:12.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f97c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f90159de0 0x7f8f900cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f97c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 0x7f8f9006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f97c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900ca0c0 con 0x7f8f900cbac0 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f97c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9015e790 con 0x7f8f90159de0 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f97c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900cbf50 con 0x7f8f9015e340 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f95200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f90159de0 0x7f8f900cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f95200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f90159de0 0x7f8f900cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44344/0 (socket says 172.21.15.91:44344) 2024-02-02T13:31:12.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.655+0000 7f8f95200640 1 -- 172.21.15.91:0/3577055897 learned_addr learned my addr 172.21.15.91:0/3577055897 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95200640 1 -- 172.21.15.91:0/3577055897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 msgr2=0x7f8f9006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95c00640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 0x7f8f9006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95200640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 0x7f8f9006fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95200640 1 -- 172.21.15.91:0/3577055897 >> v1:172.21.15.113:6789/0 conn(0x7f8f900cbac0 legacy=0x7f8f9015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95200640 1 -- 172.21.15.91:0/3577055897 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900ca380 con 0x7f8f90159de0 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.656+0000 7f8f95c00640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 0x7f8f9006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:12.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.657+0000 7f8f95200640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f90159de0 0x7f8f900cb580 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bb40 tx=0x7f8f8002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6de438ab8d2c203 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:12.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.657+0000 7f8f8fe00640 1 -- 172.21.15.91:0/3577055897 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8002ee70 con 0x7f8f90159de0 2024-02-02T13:31:12.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.657+0000 7f8f8fe00640 1 -- 172.21.15.91:0/3577055897 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80036440 con 0x7f8f90159de0 2024-02-02T13:31:12.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.657+0000 7f8f8fe00640 1 -- 172.21.15.91:0/3577055897 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800367d0 con 0x7f8f90159de0 2024-02-02T13:31:12.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 -- 172.21.15.91:0/3577055897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f90159de0 msgr2=0x7f8f900cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f90159de0 0x7f8f900cb580 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bb40 tx=0x7f8f8002e0c0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 -- 172.21.15.91:0/3577055897 shutdown_connections 2024-02-02T13:31:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f9015e340 0x7f8f9006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 --2- 172.21.15.91:0/3577055897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f90159de0 0x7f8f900cb580 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 -- 172.21.15.91:0/3577055897 >> 172.21.15.91:0/3577055897 conn(0x7f8f900b0700 msgr2=0x7f8f900b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:12.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 -- 172.21.15.91:0/3577055897 shutdown_connections 2024-02-02T13:31:12.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.658+0000 7f8f97c00640 1 -- 172.21.15.91:0/3577055897 wait complete. 2024-02-02T13:31:12.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.660+0000 7f8f97c00640 1 Processor -- start 2024-02-02T13:31:12.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.660+0000 7f8f97c00640 1 -- start start 2024-02-02T13:31:12.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f900cbac0 0x7f8f9014a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f90159de0 0x7f8f900c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9014a9c0 con 0x7f8f9015e340 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c40f0 con 0x7f8f90159de0 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.661+0000 7f8f97c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c4390 con 0x7f8f900cbac0 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44352/0 (socket says 172.21.15.91:44352) 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 -- 172.21.15.91:0/691683802 learned_addr learned my addr 172.21.15.91:0/691683802 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95200640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f900cbac0 0x7f8f9014a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 -- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f900cbac0 msgr2=0x7f8f9014a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f900cbac0 0x7f8f9014a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f87c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f90159de0 0x7f8f900c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 -- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f90159de0 msgr2=0x7f8f900c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f90159de0 0x7f8f900c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 -- 172.21.15.91:0/691683802 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8000b770 con 0x7f8f9015e340 2024-02-02T13:31:12.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.662+0000 7f8f95c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7c0 tx=0x7f8f8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.663+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004230 con 0x7f8f9015e340 2024-02-02T13:31:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.663+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88004390 con 0x7f8f9015e340 2024-02-02T13:31:12.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.663+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88007860 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.663+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f900c4670 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.663+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900c4cc0 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.664+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f900c26a0 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.665+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8f880079c0 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.665+0000 7f8f8d600640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 0x7f8f6806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.666+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f88088880 con 0x7f8f9015e340 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.666+0000 7f8f95200640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 0x7f8f6806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:12.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.666+0000 7f8f95200640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 0x7f8f6806c920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f80038000 tx=0x7f8f80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:12.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.670+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8f900c26a0 con 0x7f8f9015e340 2024-02-02T13:31:12.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.904+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8f900c26a0 con 0x7f8f9015e340 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.905+0000 7f8f8d600640 1 -- 172.21.15.91:0/691683802 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8f900c26a0 con 0x7f8f9015e340 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:12.907 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.909 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.911 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:31:12.912 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.913 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.914 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:12.915 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:31:12.916 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:31:12.917 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:12.918 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:31:12.919 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:31:12.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 msgr2=0x7f8f6806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 0x7f8f6806c920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f80038000 tx=0x7f8f80008040 comp rx=0 tx=0).stop 2024-02-02T13:31:12.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 msgr2=0x7f8f900c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:12.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7c0 tx=0x7f8f8800cc80 comp rx=0 tx=0).stop 2024-02-02T13:31:12.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 shutdown_connections 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8f6806a460 0x7f8f6806c920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f9015e340 0x7f8f900c84e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f90159de0 0x7f8f900c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 --2- 172.21.15.91:0/691683802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f900cbac0 0x7f8f9014a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 >> 172.21.15.91:0/691683802 conn(0x7f8f900b0700 msgr2=0x7f8f9006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 shutdown_connections 2024-02-02T13:31:12.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:12.908+0000 7f8f97c00640 1 -- 172.21.15.91:0/691683802 wait complete. 2024-02-02T13:31:12.927 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:13.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.128+0000 7f493c200640 1 Processor -- start 2024-02-02T13:31:13.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.128+0000 7f493c200640 1 -- start start 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f493c200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4934151f90 0x7f493415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f493c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f493406f4f0 0x7f49340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f493c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340b1070 con 0x7f49340b0c40 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f493c200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4934070f80 con 0x7f4934151f90 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f493c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4934152370 con 0x7f493406f4f0 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f4939800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4934151f90 0x7f493415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f4939800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4934151f90 0x7f493415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44358/0 (socket says 172.21.15.91:44358) 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f4939800640 1 -- 172.21.15.91:0/3832333580 learned_addr learned my addr 172.21.15.91:0/3832333580 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:13.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.130+0000 7f4938e00640 1 --2- 172.21.15.91:0/3832333580 >> v2:172.21.15.113:3300/0 conn(0x7f493406f4f0 0x7f49340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.131+0000 7f4939800640 1 -- 172.21.15.91:0/3832333580 >> v2:172.21.15.113:3300/0 conn(0x7f493406f4f0 msgr2=0x7f49340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:13.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.131+0000 7f4939800640 1 --2- 172.21.15.91:0/3832333580 >> v2:172.21.15.113:3300/0 conn(0x7f493406f4f0 0x7f49340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.131+0000 7f4939800640 1 -- 172.21.15.91:0/3832333580 >> v1:172.21.15.113:6789/0 conn(0x7f49340b0c40 legacy=0x7f49340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:13.133 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.131+0000 7f4939800640 1 -- 172.21.15.91:0/3832333580 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340cb010 con 0x7f4934151f90 2024-02-02T13:31:13.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.132+0000 7f4939800640 1 --2- 172.21.15.91:0/3832333580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4934151f90 0x7f493415f5a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4934071a00 tx=0x7f492402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e984d068770d7960 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.133+0000 7f4933e00640 1 -- 172.21.15.91:0/3832333580 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924031870 con 0x7f4934151f90 2024-02-02T13:31:13.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.133+0000 7f4933e00640 1 -- 172.21.15.91:0/3832333580 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49240319d0 con 0x7f4934151f90 2024-02-02T13:31:13.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.133+0000 7f4933e00640 1 -- 172.21.15.91:0/3832333580 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924031d60 con 0x7f4934151f90 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 -- 172.21.15.91:0/3832333580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4934151f90 msgr2=0x7f493415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 --2- 172.21.15.91:0/3832333580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4934151f90 0x7f493415f5a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4934071a00 tx=0x7f492402b910 comp rx=0 tx=0).stop 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 -- 172.21.15.91:0/3832333580 shutdown_connections 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 --2- 172.21.15.91:0/3832333580 >> v2:172.21.15.113:3300/0 conn(0x7f493406f4f0 0x7f49340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 --2- 172.21.15.91:0/3832333580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4934151f90 0x7f493415f5a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 -- 172.21.15.91:0/3832333580 >> 172.21.15.91:0/3832333580 conn(0x7f49340cec80 msgr2=0x7f49340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:13.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 -- 172.21.15.91:0/3832333580 shutdown_connections 2024-02-02T13:31:13.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.134+0000 7f493c200640 1 -- 172.21.15.91:0/3832333580 wait complete. 2024-02-02T13:31:13.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.136+0000 7f493c200640 1 Processor -- start 2024-02-02T13:31:13.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.136+0000 7f493c200640 1 -- start start 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 0x7f4934154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 0x7f49340c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340c5590 con 0x7f49340b0c40 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340c7360 con 0x7f4934151f90 2024-02-02T13:31:13.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.137+0000 7f493c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4934152890 con 0x7f493406f4f0 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44368/0 (socket says 172.21.15.91:44368) 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 -- 172.21.15.91:0/2449560116 learned_addr learned my addr 172.21.15.91:0/2449560116 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4939800640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 0x7f4934154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f493a200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 0x7f49340c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 -- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 msgr2=0x7f4934154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 0x7f4934154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 -- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 msgr2=0x7f49340c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 0x7f49340c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 -- 172.21.15.91:0/2449560116 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49240094d0 con 0x7f49340b0c40 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4939800640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 0x7f4934154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f493a200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 0x7f49340c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.138+0000 7f4938e00640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d5f0 tx=0x7f491c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.139+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0125f0 con 0x7f49340b0c40 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.139+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c012750 con 0x7f49340b0c40 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.139+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c012b00 con 0x7f49340b0c40 2024-02-02T13:31:13.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.139+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4934152b70 con 0x7f49340b0c40 2024-02-02T13:31:13.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.139+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49340c6ff0 con 0x7f49340b0c40 2024-02-02T13:31:13.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.140+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4934152d90 con 0x7f49340b0c40 2024-02-02T13:31:13.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.141+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f491c012c60 con 0x7f49340b0c40 2024-02-02T13:31:13.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.142+0000 7f4931600640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 0x7f490c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.142+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f491c088ab0 con 0x7f49340b0c40 2024-02-02T13:31:13.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.142+0000 7f4939800640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 0x7f490c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.143+0000 7f4939800640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 0x7f490c06c920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4924036000 tx=0x7f492403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.147+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f491c054350 con 0x7f49340b0c40 2024-02-02T13:31:13.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.342+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49340714b0 con 0x7f49340b0c40 2024-02-02T13:31:13.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.344+0000 7f4931600640 1 -- 172.21.15.91:0/2449560116 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f491c057970 con 0x7f49340b0c40 2024-02-02T13:31:13.345 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:13.346 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":124,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:12.426898+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 msgr2=0x7f490c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 0x7f490c06c920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4924036000 tx=0x7f492403c040 comp rx=0 tx=0).stop 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 msgr2=0x7f49340c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d5f0 tx=0x7f491c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 shutdown_connections 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f490c06a460 0x7f490c06c920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4934151f90 0x7f49340c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49340b0c40 0x7f49340c4b10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 --2- 172.21.15.91:0/2449560116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f493406f4f0 0x7f4934154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 >> 172.21.15.91:0/2449560116 conn(0x7f49340cec80 msgr2=0x7f49340cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:13.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 shutdown_connections 2024-02-02T13:31:13.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.347+0000 7f493c200640 1 -- 172.21.15.91:0/2449560116 wait complete. 2024-02-02T13:31:13.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:13.369 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:13.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.579+0000 7f09de000640 1 Processor -- start 2024-02-02T13:31:13.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.579+0000 7f09de000640 1 -- start start 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09d80d0070 0x7f09d80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 0x7f09d80d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f09d80b6260 0x7f09d80b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80b8f90 con 0x7f09d80d0070 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80cd930 con 0x7f09d80b6260 2024-02-02T13:31:13.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.580+0000 7f09de000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80d1680 con 0x7f09d80b2260 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09d80d0070 0x7f09d80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f09d80d0070 0x7f09d80b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44376/0 (socket says 172.21.15.91:44376) 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 -- 172.21.15.91:0/3170595824 learned_addr learned my addr 172.21.15.91:0/3170595824 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d6a00640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 0x7f09d80d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7e00640 1 --2- 172.21.15.91:0/3170595824 >> v2:172.21.15.113:3300/0 conn(0x7f09d80b6260 0x7f09d80b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 -- 172.21.15.91:0/3170595824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 msgr2=0x7f09d80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 0x7f09d80d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 -- 172.21.15.91:0/3170595824 >> v2:172.21.15.113:3300/0 conn(0x7f09d80b6260 msgr2=0x7f09d80b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:13.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 --2- 172.21.15.91:0/3170595824 >> v2:172.21.15.113:3300/0 conn(0x7f09d80b6260 0x7f09d80b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d7400640 1 -- 172.21.15.91:0/3170595824 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80d1960 con 0x7f09d80d0070 2024-02-02T13:31:13.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.581+0000 7f09d6a00640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 0x7f09d80d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:13.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.583+0000 7f09d7400640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b8a50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f09d80d0fc0 tx=0x7f09c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91ad265db6e209c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.583+0000 7f09d6000640 1 -- 172.21.15.91:0/3170595824 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8031870 con 0x7f09d80d0070 2024-02-02T13:31:13.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.583+0000 7f09d6000640 1 -- 172.21.15.91:0/3170595824 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c80319d0 con 0x7f09d80d0070 2024-02-02T13:31:13.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.583+0000 7f09d6000640 1 -- 172.21.15.91:0/3170595824 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8031d60 con 0x7f09d80d0070 2024-02-02T13:31:13.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 -- 172.21.15.91:0/3170595824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 msgr2=0x7f09d80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b8a50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f09d80d0fc0 tx=0x7f09c802b910 comp rx=0 tx=0).stop 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 -- 172.21.15.91:0/3170595824 shutdown_connections 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 --2- 172.21.15.91:0/3170595824 >> v2:172.21.15.113:3300/0 conn(0x7f09d80b6260 0x7f09d80b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b2260 0x7f09d80d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 --2- 172.21.15.91:0/3170595824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b8a50 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 -- 172.21.15.91:0/3170595824 >> 172.21.15.91:0/3170595824 conn(0x7f09d8070660 msgr2=0x7f09d8070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 -- 172.21.15.91:0/3170595824 shutdown_connections 2024-02-02T13:31:13.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.584+0000 7f09de000640 1 -- 172.21.15.91:0/3170595824 wait complete. 2024-02-02T13:31:13.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.586+0000 7f09de000640 1 Processor -- start 2024-02-02T13:31:13.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.587+0000 7f09de000640 1 -- start start 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d80b2260 0x7f09d81595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b6260 0x7f09d80aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f0490 con 0x7f09d80d0070 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f3a10 con 0x7f09d80b2260 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09de000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f3cf0 con 0x7f09d80b6260 2024-02-02T13:31:13.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09d7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44390/0 (socket says 172.21.15.91:44390) 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09d7e00640 1 -- 172.21.15.91:0/1383621515 learned_addr learned my addr 172.21.15.91:0/1383621515 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.588+0000 7f09d7400640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d80b2260 0x7f09d81595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d6a00640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b6260 0x7f09d80aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 -- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b6260 msgr2=0x7f09d80aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b6260 0x7f09d80aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 -- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d80b2260 msgr2=0x7f09d81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d80b2260 0x7f09d81595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 -- 172.21.15.91:0/1383621515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80094d0 con 0x7f09d80d0070 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09d7e00640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e7a0 tx=0x7f09d000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d000c730 con 0x7f09d80d0070 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d000c890 con 0x7f09d80d0070 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d000cc40 con 0x7f09d80d0070 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.589+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d81f3fd0 con 0x7f09d80d0070 2024-02-02T13:31:13.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.590+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d81f4590 con 0x7f09d80d0070 2024-02-02T13:31:13.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.591+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8066cf0 con 0x7f09d80d0070 2024-02-02T13:31:13.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.592+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f09d0013470 con 0x7f09d80d0070 2024-02-02T13:31:13.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.593+0000 7f09cf400640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 0x7f09b006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:13.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.593+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09d0088890 con 0x7f09d80d0070 2024-02-02T13:31:13.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.593+0000 7f09d7400640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 0x7f09b006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:13.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.594+0000 7f09d7400640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 0x7f09b006c920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:13.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.597+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f09d0054270 con 0x7f09d80d0070 2024-02-02T13:31:13.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.783+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09d80d1310 con 0x7f09d80d0070 2024-02-02T13:31:13.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.784+0000 7f09cf400640 1 -- 172.21.15.91:0/1383621515 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09d00170a0 con 0x7f09d80d0070 2024-02-02T13:31:13.786 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:13.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.786+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 msgr2=0x7f09b006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.786+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 0x7f09b006c920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c803c040 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 msgr2=0x7f09d80b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e7a0 tx=0x7f09d000ec60 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 shutdown_connections 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f09b006a460 0x7f09b006c920 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09d80b6260 0x7f09d80aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f09d80d0070 0x7f09d80b70f0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 --2- 172.21.15.91:0/1383621515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d80b2260 0x7f09d81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 >> 172.21.15.91:0/1383621515 conn(0x7f09d8070660 msgr2=0x7f09d815c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 shutdown_connections 2024-02-02T13:31:13.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:13.787+0000 7f09de000640 1 -- 172.21.15.91:0/1383621515 wait complete. 2024-02-02T13:31:13.805 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:31:14.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.004+0000 7f849f800640 1 Processor -- start 2024-02-02T13:31:14.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.005+0000 7f849f800640 1 -- start start 2024-02-02T13:31:14.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.006+0000 7f849f800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84980d1760 0x7f849806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.006+0000 7f849f800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849815db10 con 0x7f84980cb920 2024-02-02T13:31:14.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.006+0000 7f849f800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84980cb490 con 0x7f849815d730 2024-02-02T13:31:14.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.006+0000 7f849f800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980d1d40 con 0x7f84980d1760 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f849d800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f84980cb920 0x7f84980cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40144/0 (socket says 172.21.15.91:40144) 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f849d800640 1 -- 172.21.15.91:0/4155921212 learned_addr learned my addr 172.21.15.91:0/4155921212 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497e00640 1 --2- 172.21.15.91:0/4155921212 >> v2:172.21.15.113:3300/0 conn(0x7f84980d1760 0x7f849806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544048224 0 0) 0x7f849815db10 con 0x7f84980cb920 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8478003520 con 0x7f84980cb920 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3878711780 0 0) 0x7f8478003520 con 0x7f84980cb920 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 >> v2:172.21.15.113:3300/0 conn(0x7f84980d1760 msgr2=0x7f849806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 --2- 172.21.15.91:0/4155921212 >> v2:172.21.15.113:3300/0 conn(0x7f84980d1760 0x7f849806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 >> v1:172.21.15.113:6789/0 conn(0x7f849815d730 legacy=0x7f8498069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980cfe90 con 0x7f84980cb920 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.007+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8490001670 con 0x7f84980cb920 2024-02-02T13:31:14.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 >> v1:172.21.15.91:6789/0 conn(0x7f84980cb920 legacy=0x7f84980cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f8478003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980d1d40 con 0x7f8478003970 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847800bfd0 con 0x7f8478005180 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847800c190 con 0x7f84780047c0 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f8478003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849d800640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497e00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 -- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84780047c0 msgr2=0x7f8478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 -- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8478005180 msgr2=0x7f847800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849ce00640 1 -- 172.21.15.91:0/4155921212 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478003520 con 0x7f8478003970 2024-02-02T13:31:14.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849d800640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:14.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f849d800640 1 -- 172.21.15.91:0/4155921212 reap_dead start 2024-02-02T13:31:14.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.008+0000 7f8497e00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.009+0000 7f849ce00640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f8478003d50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8480001860 tx=0x7f84780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9303939f53ca809 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480033410 con 0x7f8478003970 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480033590 con 0x7f8478003970 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f8497400640 1 -- 172.21.15.91:0/4155921212 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480033940 con 0x7f8478003970 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f849f800640 1 -- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 msgr2=0x7f8478003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f849f800640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f8478003d50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8480001860 tx=0x7f84780057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.010+0000 7f849f800640 1 -- 172.21.15.91:0/4155921212 shutdown_connections 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.011+0000 7f849f800640 1 --2- 172.21.15.91:0/4155921212 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f8478003d50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.011+0000 7f849f800640 1 -- 172.21.15.91:0/4155921212 >> 172.21.15.91:0/4155921212 conn(0x7f849809a110 msgr2=0x7f84980d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.011+0000 7f849f800640 1 -- 172.21.15.91:0/4155921212 shutdown_connections 2024-02-02T13:31:14.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.011+0000 7f849f800640 1 -- 172.21.15.91:0/4155921212 wait complete. 2024-02-02T13:31:14.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.012+0000 7f849f800640 1 Processor -- start 2024-02-02T13:31:14.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.013+0000 7f849f800640 1 -- start start 2024-02-02T13:31:14.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.013+0000 7f849f800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.013+0000 7f849f800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84981f3640 0x7f84981583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849f800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84980b5710 0x7f84980b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849f800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980b6100 con 0x7f8478003970 2024-02-02T13:31:14.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849f800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849815e930 con 0x7f84981f3640 2024-02-02T13:31:14.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849f800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849815ec10 con 0x7f84980b5710 2024-02-02T13:31:14.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44410/0 (socket says 172.21.15.91:44410) 2024-02-02T13:31:14.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 -- 172.21.15.91:0/688800733 learned_addr learned my addr 172.21.15.91:0/688800733 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 -- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84980b5710 msgr2=0x7f84980b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84980b5710 0x7f84980b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 -- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84981f3640 msgr2=0x7f84981583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84981f3640 0x7f84981583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 -- 172.21.15.91:0/688800733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849815eef0 con 0x7f8478003970 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.014+0000 7f849ce00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f848002bd00 tx=0x7f8480001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.015+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480007320 con 0x7f8478003970 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.015+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480007480 con 0x7f8478003970 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.015+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800067d0 con 0x7f8478003970 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.015+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849815f1d0 con 0x7f8478003970 2024-02-02T13:31:14.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.015+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849815f4b0 con 0x7f8478003970 2024-02-02T13:31:14.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.016+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849815ed50 con 0x7f8478003970 2024-02-02T13:31:14.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.017+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f848003a070 con 0x7f8478003970 2024-02-02T13:31:14.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.018+0000 7f8494c00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 0x7f847006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.018+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84800a97a0 con 0x7f8478003970 2024-02-02T13:31:14.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.018+0000 7f8497e00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 0x7f847006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.019+0000 7f8497e00640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 0x7f847006c920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84980cb5f0 tx=0x7f8488009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.023+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f849815ed50 con 0x7f8478003970 2024-02-02T13:31:14.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.214+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f849815ed50 con 0x7f8478003970 2024-02-02T13:31:14.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.215+0000 7f8494c00640 1 -- 172.21.15.91:0/688800733 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v124) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8480041060 con 0x7f8478003970 2024-02-02T13:31:14.216 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:14.217 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 msgr2=0x7f847006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 0x7f847006c920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84980cb5f0 tx=0x7f8488009750 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 msgr2=0x7f849815e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f848002bd00 tx=0x7f8480001230 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 shutdown_connections 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f847006a460 0x7f847006c920 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84980b5710 0x7f84980b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84981f3640 0x7f84981583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 --2- 172.21.15.91:0/688800733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8478003970 0x7f849815e270 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 >> 172.21.15.91:0/688800733 conn(0x7f849809a110 msgr2=0x7f84980b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:14.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 shutdown_connections 2024-02-02T13:31:14.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.218+0000 7f849f800640 1 -- 172.21.15.91:0/688800733 wait complete. 2024-02-02T13:31:14.238 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:31:14.238 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:31:14.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.442+0000 7fe796a00640 1 Processor -- start 2024-02-02T13:31:14.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.442+0000 7fe796a00640 1 -- start start 2024-02-02T13:31:14.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe796a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe790151f90 0x7fe79015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe796a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe790070f80 con 0x7fe7900b0e10 2024-02-02T13:31:14.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe796a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79006c600 con 0x7fe79006f4f0 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe796a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe790152370 con 0x7fe790151f90 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe794a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe7900b0e10 0x7fe7900b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40160/0 (socket says 172.21.15.91:40160) 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.444+0000 7fe794a00640 1 -- 172.21.15.91:0/3948026615 learned_addr learned my addr 172.21.15.91:0/3948026615 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78fe00640 1 --2- 172.21.15.91:0/3948026615 >> v2:172.21.15.113:3300/0 conn(0x7fe790151f90 0x7fe79015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187427498 0 0) 0x7fe790070f80 con 0x7fe7900b0e10 2024-02-02T13:31:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe770003520 con 0x7fe7900b0e10 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1968103277 0 0) 0x7fe770003520 con 0x7fe7900b0e10 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 >> v2:172.21.15.113:3300/0 conn(0x7fe790151f90 msgr2=0x7fe79015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 --2- 172.21.15.91:0/3948026615 >> v2:172.21.15.113:3300/0 conn(0x7fe790151f90 0x7fe79015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 >> v1:172.21.15.113:6789/0 conn(0x7fe79006f4f0 legacy=0x7fe7900b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900cb010 con 0x7fe7900b0e10 2024-02-02T13:31:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe7880019f0 con 0x7fe7900b0e10 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 >> v1:172.21.15.91:6789/0 conn(0x7fe7900b0e10 legacy=0x7fe7900b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe770003970 0x7fe770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.445+0000 7fe78f400640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe78f400640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe770005180 0x7fe77000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe790152370 con 0x7fe7700047c0 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77000c020 con 0x7fe770003970 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77000c1e0 con 0x7fe770005180 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe78fe00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe770003970 0x7fe770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 -- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe770005180 msgr2=0x7fe77000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe794a00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe770005180 0x7fe77000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe770005180 0x7fe77000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 -- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe770003970 msgr2=0x7fe770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe770003970 0x7fe770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe787e00640 1 -- 172.21.15.91:0/3948026615 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770003520 con 0x7fe7700047c0 2024-02-02T13:31:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe794a00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe770005180 0x7fe77000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.446+0000 7fe794a00640 1 -- 172.21.15.91:0/3948026615 reap_dead start 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.447+0000 7fe787e00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe770004c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe778001750 tx=0x7fe7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee8407a28236f65 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780052d0 con 0x7fe7700047c0 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77802e050 con 0x7fe7700047c0 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe78f400640 1 -- 172.21.15.91:0/3948026615 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778003830 con 0x7fe7700047c0 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 -- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 msgr2=0x7fe770004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe770004c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe778001750 tx=0x7fe7700057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 -- 172.21.15.91:0/3948026615 shutdown_connections 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 --2- 172.21.15.91:0/3948026615 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe770004c40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 -- 172.21.15.91:0/3948026615 >> 172.21.15.91:0/3948026615 conn(0x7fe7900cec80 msgr2=0x7fe7900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 -- 172.21.15.91:0/3948026615 shutdown_connections 2024-02-02T13:31:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.448+0000 7fe796a00640 1 -- 172.21.15.91:0/3948026615 wait complete. 2024-02-02T13:31:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.450+0000 7fe796a00640 1 Processor -- start 2024-02-02T13:31:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.450+0000 7fe796a00640 1 -- start start 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7900c82f0 0x7fe7900c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7900c91f0 0x7fe7900c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900c9bb0 con 0x7fe7700047c0 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79015cfc0 con 0x7fe7900c82f0 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe796a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900c27c0 con 0x7fe7900c91f0 2024-02-02T13:31:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe78fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe78fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44420/0 (socket says 172.21.15.91:44420) 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.451+0000 7fe78fe00640 1 -- 172.21.15.91:0/1095347583 learned_addr learned my addr 172.21.15.91:0/1095347583 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe787e00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7900c82f0 0x7fe7900c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 -- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7900c91f0 msgr2=0x7fe7900c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7900c91f0 0x7fe7900c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 -- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7900c82f0 msgr2=0x7fe7900c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7900c82f0 0x7fe7900c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:14.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 -- 172.21.15.91:0/1095347583 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900c5210 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78fe00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe7800116d0 tx=0x7fe780011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000f9c0 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78000fb20 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79015d260 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.452+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000fe90 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.453+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900c2a60 con 0x7fe7700047c0 2024-02-02T13:31:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.453+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7900c2c20 con 0x7fe7700047c0 2024-02-02T13:31:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.455+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe780015460 con 0x7fe7700047c0 2024-02-02T13:31:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.456+0000 7fe78cc00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 0x7fe76806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.456+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe780088eb0 con 0x7fe7700047c0 2024-02-02T13:31:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.456+0000 7fe787e00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 0x7fe76806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.456+0000 7fe787e00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 0x7fe76806c920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe778033000 tx=0x7fe778009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:14.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.460+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe7800547f0 con 0x7fe7700047c0 2024-02-02T13:31:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:14.653+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe790066cf0 con 0x7fe7700047c0 2024-02-02T13:31:15.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.399+0000 7fe78cc00640 1 -- 172.21.15.91:0/1095347583 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v125) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe780057e10 con 0x7fe7700047c0 2024-02-02T13:31:15.402 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:31:15.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.405+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 msgr2=0x7fe76806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.405+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 0x7fe76806c920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe778033000 tx=0x7fe778009030 comp rx=0 tx=0).stop 2024-02-02T13:31:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.405+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 msgr2=0x7fe7900c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.405+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe7800116d0 tx=0x7fe780011b90 comp rx=0 tx=0).stop 2024-02-02T13:31:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 shutdown_connections 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe76806a460 0x7fe76806c920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7900c91f0 0x7fe7900c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7900c82f0 0x7fe7900c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 --2- 172.21.15.91:0/1095347583 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7700047c0 0x7fe7900c4cd0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 >> 172.21.15.91:0/1095347583 conn(0x7fe7900cec80 msgr2=0x7fe7900cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 shutdown_connections 2024-02-02T13:31:15.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.406+0000 7fe796a00640 1 -- 172.21.15.91:0/1095347583 wait complete. 2024-02-02T13:31:15.429 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:31:15.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.634+0000 7f2b3a600640 1 Processor -- start 2024-02-02T13:31:15.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.634+0000 7f2b3a600640 1 -- start start 2024-02-02T13:31:15.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b3a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b3415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b3a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b3406f4f0 0x7f2b340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b3a600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b34070f80 con 0x7f2b340b0c40 2024-02-02T13:31:15.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b3a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340b10d0 con 0x7f2b3406f4f0 2024-02-02T13:31:15.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b3a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b34152370 con 0x7f2b34151f90 2024-02-02T13:31:15.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b33e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2b340b0c40 0x7f2b340b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40172/0 (socket says 172.21.15.91:40172) 2024-02-02T13:31:15.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b33e00640 1 -- 172.21.15.91:0/1103760041 learned_addr learned my addr 172.21.15.91:0/1103760041 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1103760041 >> v2:172.21.15.113:3300/0 conn(0x7f2b3406f4f0 0x7f2b340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.636+0000 7f2b33400640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b3415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648892916 0 0) 0x7f2b34070f80 con 0x7f2b340b0c40 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b14003520 con 0x7f2b340b0c40 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105005409 0 0) 0x7f2b14003520 con 0x7f2b340b0c40 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 msgr2=0x7f2b3415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:15.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b3415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 >> v2:172.21.15.113:3300/0 conn(0x7f2b3406f4f0 msgr2=0x7f2b340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 --2- 172.21.15.91:0/1103760041 >> v2:172.21.15.113:3300/0 conn(0x7f2b3406f4f0 0x7f2b340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cb010 con 0x7f2b340b0c40 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2b2c001670 con 0x7f2b340b0c40 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 >> v1:172.21.15.91:6789/0 conn(0x7f2b340b0c40 legacy=0x7f2b340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b14003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b14003f40 0x7f2b140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b14004d00 0x7f2b14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b10d0 con 0x7f2b14004d00 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b34152370 con 0x7f2b14003f40 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.637+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1400b980 con 0x7f2b34151f90 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b14004d00 0x7f2b14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33400640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b14003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b14003f40 0x7f2b140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 -- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 msgr2=0x7f2b14003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b34151f90 0x7f2b14003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 -- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b14003f40 msgr2=0x7f2b140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b14003f40 0x7f2b140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b33e00640 1 -- 172.21.15.91:0/1103760041 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14003520 con 0x7f2b14004d00 2024-02-02T13:31:15.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.638+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b14003f40 0x7f2b140047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:15.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.639+0000 7f2b33e00640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b14004d00 0x7f2b14005180 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c140 tx=0x7f2b140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd2018d2b82fc57 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:15.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.639+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c005ad0 con 0x7f2b14004d00 2024-02-02T13:31:15.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.639+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c005c50 con 0x7f2b14004d00 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.639+0000 7f2b32a00640 1 -- 172.21.15.91:0/1103760041 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c02c810 con 0x7f2b14004d00 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 -- 172.21.15.91:0/1103760041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b14004d00 msgr2=0x7f2b14005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1103760041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b14004d00 0x7f2b14005180 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c140 tx=0x7f2b140057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b33e00640 1 -- 172.21.15.91:0/1103760041 reap_dead start 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 -- 172.21.15.91:0/1103760041 shutdown_connections 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 -- 172.21.15.91:0/1103760041 >> 172.21.15.91:0/1103760041 conn(0x7f2b340cec80 msgr2=0x7f2b340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 -- 172.21.15.91:0/1103760041 shutdown_connections 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.640+0000 7f2b3a600640 1 -- 172.21.15.91:0/1103760041 wait complete. 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.642+0000 7f2b3a600640 1 Processor -- start 2024-02-02T13:31:15.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.642+0000 7f2b3a600640 1 -- start start 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b14004d00 0x7f2b34154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b3406b7d0 0x7f2b340c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341553c0 con 0x7f2b34152f80 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340cd300 con 0x7f2b3406b7d0 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b3a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340cd5e0 con 0x7f2b14004d00 2024-02-02T13:31:15.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b2a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b2a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44442/0 (socket says 172.21.15.91:44442) 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b2a800640 1 -- 172.21.15.91:0/1251851758 learned_addr learned my addr 172.21.15.91:0/1251851758 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.643+0000 7f2b33400640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b14004d00 0x7f2b34154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 -- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b14004d00 msgr2=0x7f2b34154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b14004d00 0x7f2b34154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 -- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b3406b7d0 msgr2=0x7f2b340c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b3406b7d0 0x7f2b340c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 -- 172.21.15.91:0/1251851758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cd8c0 con 0x7f2b34152f80 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2a800640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010920 tx=0x7f2b1c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00eba0 con 0x7f2b34152f80 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00ed00 con 0x7f2b34152f80 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c01c640 con 0x7f2b34152f80 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340cdba0 con 0x7f2b34152f80 2024-02-02T13:31:15.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.644+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340ce600 con 0x7f2b34152f80 2024-02-02T13:31:15.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.645+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34066cf0 con 0x7f2b34152f80 2024-02-02T13:31:15.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.647+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2b1c01c7a0 con 0x7f2b34152f80 2024-02-02T13:31:15.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.647+0000 7f2b2be00640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 0x7f2b0c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:15.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.648+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2b1c088cf0 con 0x7f2b34152f80 2024-02-02T13:31:15.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.648+0000 7f2b33400640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 0x7f2b0c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:15.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.648+0000 7f2b33400640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 0x7f2b0c06c920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007380 tx=0x7f2b2400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:15.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.653+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2b1c054560 con 0x7f2b34152f80 2024-02-02T13:31:15.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:15.843+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b3415e7f0 con 0x7f2b34152f80 2024-02-02T13:31:16.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.415+0000 7f2b2be00640 1 -- 172.21.15.91:0/1251851758 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v126) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b1c057b80 con 0x7f2b34152f80 2024-02-02T13:31:16.417 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:31:16.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.420+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 msgr2=0x7f2b0c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.420+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 0x7f2b0c06c920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007380 tx=0x7f2b2400b040 comp rx=0 tx=0).stop 2024-02-02T13:31:16.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.420+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 msgr2=0x7f2b340ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.420+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010920 tx=0x7f2b1c010de0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 shutdown_connections 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b0c06a460 0x7f2b0c06c920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b3406b7d0 0x7f2b340c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b34152f80 0x7f2b340ce0c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 --2- 172.21.15.91:0/1251851758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b14004d00 0x7f2b34154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.424+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 >> 172.21.15.91:0/1251851758 conn(0x7f2b340cec80 msgr2=0x7f2b34159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:16.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.429+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 shutdown_connections 2024-02-02T13:31:16.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.430+0000 7f2b3a600640 1 -- 172.21.15.91:0/1251851758 wait complete. 2024-02-02T13:31:16.457 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:31:16.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.645+0000 7f1707400640 1 Processor -- start 2024-02-02T13:31:16.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.645+0000 7f1707400640 1 -- start start 2024-02-02T13:31:16.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1707400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f17000cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1707400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f170015b2a0 0x7f170006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1707400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170006c1c0 con 0x7f170006cb90 2024-02-02T13:31:16.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1707400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170006b5f0 con 0x7f170015b2a0 2024-02-02T13:31:16.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1707400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170006b890 con 0x7f170015aec0 2024-02-02T13:31:16.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1706a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f170006cb90 0x7f17000ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40178/0 (socket says 172.21.15.91:40178) 2024-02-02T13:31:16.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1706a00640 1 -- 172.21.15.91:0/3698638390 learned_addr learned my addr 172.21.15.91:0/3698638390 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:16.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1706000640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f17000cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.647+0000 7f1705600640 1 --2- 172.21.15.91:0/3698638390 >> v2:172.21.15.113:3300/0 conn(0x7f170015b2a0 0x7f170006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38479326 0 0) 0x7f170006c1c0 con 0x7f170006cb90 2024-02-02T13:31:16.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e0003520 con 0x7f170006cb90 2024-02-02T13:31:16.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930864006 0 0) 0x7f16e0003520 con 0x7f170006cb90 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 msgr2=0x7f17000cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f17000cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 >> v2:172.21.15.113:3300/0 conn(0x7f170015b2a0 msgr2=0x7f170006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 --2- 172.21.15.91:0/3698638390 >> v2:172.21.15.113:3300/0 conn(0x7f170015b2a0 0x7f170006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170006bb70 con 0x7f170006cb90 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f16f80019f0 con 0x7f170006cb90 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 >> v1:172.21.15.91:6789/0 conn(0x7f170006cb90 legacy=0x7f17000ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f16e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f16e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e0004d00 0x7f16e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170006b5f0 con 0x7f16e0003ef0 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170006b890 con 0x7f16e0004d00 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e000b9b0 con 0x7f170015aec0 2024-02-02T13:31:16.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.648+0000 7f1705600640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f16e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1706000640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f16e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1706a00640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e0004d00 0x7f16e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1705600640 1 -- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 msgr2=0x7f16e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1705600640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f170015aec0 0x7f16e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1705600640 1 -- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e0004d00 msgr2=0x7f16e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1705600640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16e0004d00 0x7f16e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.649+0000 7f1705600640 1 -- 172.21.15.91:0/3698638390 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e0003520 con 0x7f16e0003ef0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.650+0000 7f1705600640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f16e00047c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009ac0 tx=0x7f16e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab4740aef8d8600 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.650+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e802c710 con 0x7f16e0003ef0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.650+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e802c870 con 0x7f16e0003ef0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.650+0000 7f1704c00640 1 -- 172.21.15.91:0/3698638390 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e802cc00 con 0x7f16e0003ef0 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 -- 172.21.15.91:0/3698638390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 msgr2=0x7f16e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 --2- 172.21.15.91:0/3698638390 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f16e00047c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009ac0 tx=0x7f16e00057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1706a00640 1 -- 172.21.15.91:0/3698638390 reap_dead start 2024-02-02T13:31:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 -- 172.21.15.91:0/3698638390 shutdown_connections 2024-02-02T13:31:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 -- 172.21.15.91:0/3698638390 >> 172.21.15.91:0/3698638390 conn(0x7f17000cea90 msgr2=0x7f17000ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 -- 172.21.15.91:0/3698638390 shutdown_connections 2024-02-02T13:31:16.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.651+0000 7f1707400640 1 -- 172.21.15.91:0/3698638390 wait complete. 2024-02-02T13:31:16.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.653+0000 7f1707400640 1 Processor -- start 2024-02-02T13:31:16.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.653+0000 7f1707400640 1 -- start start 2024-02-02T13:31:16.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17001f2160 0x7f17001e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17001f25e0 0x7f17001e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700146650 con 0x7f16e0003ef0 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17001e12b0 con 0x7f17001f2160 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1707400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17001e1590 con 0x7f17001f25e0 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1706000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1706000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44458/0 (socket says 172.21.15.91:44458) 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.654+0000 7f1706000640 1 -- 172.21.15.91:0/3919666681 learned_addr learned my addr 172.21.15.91:0/3919666681 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706a00640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17001f25e0 0x7f17001e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 -- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17001f25e0 msgr2=0x7f17001e2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17001f25e0 0x7f17001e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 -- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17001f2160 msgr2=0x7f17001e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1705600640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17001f2160 0x7f17001e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17001f2160 0x7f17001e29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 -- 172.21.15.91:0/3919666681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17001e1870 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1706000640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f16f0011600 tx=0x7f16f0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f000f920 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f000fa80 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17001e1b50 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.655+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f000fdf0 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.656+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17001e0540 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.657+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170004dff0 con 0x7f16e0003ef0 2024-02-02T13:31:16.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.658+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f16f0015460 con 0x7f16e0003ef0 2024-02-02T13:31:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.659+0000 7f16fe000640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 0x7f16d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.659+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f16f0088fe0 con 0x7f16e0003ef0 2024-02-02T13:31:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.659+0000 7f1705600640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 0x7f16d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:16.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.660+0000 7f1705600640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 0x7f16d806c8f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f16e8036000 tx=0x7f16e8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:16.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.664+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f16f00546c0 con 0x7f16e0003ef0 2024-02-02T13:31:16.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.890+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f17000ca3d0 con 0x7f16e0003ef0 2024-02-02T13:31:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.903+0000 7f16fe000640 1 -- 172.21.15.91:0/3919666681 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7f16f0057ce0 con 0x7f16e0003ef0 2024-02-02T13:31:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 14 and data pool 15 2024-02-02T13:31:16.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.909+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 msgr2=0x7f16d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.909+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 0x7f16d806c8f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f16e8036000 tx=0x7f16e8030000 comp rx=0 tx=0).stop 2024-02-02T13:31:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.909+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 msgr2=0x7f17001e2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:16.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.909+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f16f0011600 tx=0x7f16f0011ac0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 shutdown_connections 2024-02-02T13:31:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f16d806a430 0x7f16d806c8f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17001f25e0 0x7f17001e2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17001f2160 0x7f17001e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 --2- 172.21.15.91:0/3919666681 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16e0003ef0 0x7f17001e2350 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:16.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.912+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 >> 172.21.15.91:0/3919666681 conn(0x7f17000cea90 msgr2=0x7f170015a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:16.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.913+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 shutdown_connections 2024-02-02T13:31:16.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:16.914+0000 7f1707400640 1 -- 172.21.15.91:0/3919666681 wait complete. 2024-02-02T13:31:16.939 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:17.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.112+0000 7fb7b9e00640 1 Processor -- start 2024-02-02T13:31:17.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.112+0000 7fb7b9e00640 1 -- start start 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb7b40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b9e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40b10a0 con 0x7fb7b40cd260 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b9e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40ccdd0 con 0x7fb7b4161660 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b9e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cc7c0 con 0x7fb7b40b0700 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b3400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb7b40cd260 0x7fb7b40cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40186/0 (socket says 172.21.15.91:40186) 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.114+0000 7fb7b3400640 1 -- 172.21.15.91:0/1070807673 learned_addr learned my addr 172.21.15.91:0/1070807673 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195472667 0 0) 0x7fb7b40b10a0 con 0x7fb7b40cd260 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7a9e00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb7b40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb794003520 con 0x7fb7b40cd260 2024-02-02T13:31:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628268569 0 0) 0x7fb794003520 con 0x7fb7b40cd260 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 msgr2=0x7fb7b40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb7b40b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 >> v1:172.21.15.113:6789/0 conn(0x7fb7b4161660 legacy=0x7fb7b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40c9570 con 0x7fb7b40cd260 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb7ac0019f0 con 0x7fb7b40cd260 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 >> v1:172.21.15.91:6789/0 conn(0x7fb7b40cd260 legacy=0x7fb7b40cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7940039a0 0x7fb794003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940047c0 0x7fb794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb79400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cc7c0 con 0x7fb7940039a0 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940054e0 con 0x7fb7940047c0 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.115+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79400bec0 con 0x7fb7b40b0700 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7940039a0 0x7fb794003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7a9e00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940047c0 0x7fb794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b3400640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb79400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 -- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 msgr2=0x7fb79400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb79400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 -- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940047c0 msgr2=0x7fb794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940047c0 0x7fb794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b2a00640 1 -- 172.21.15.91:0/1070807673 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794003520 con 0x7fb7940039a0 2024-02-02T13:31:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.116+0000 7fb7b3400640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b0700 0x7fb79400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:17.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.118+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7940039a0 0x7fb794003dc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001750 tx=0x7fb7940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e942faf0b3eb361 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.118+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40052d0 con 0x7fb7940039a0 2024-02-02T13:31:17.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.118+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a402e050 con 0x7fb7940039a0 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.118+0000 7fb7b2000640 1 -- 172.21.15.91:0/1070807673 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40035f0 con 0x7fb7940039a0 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 -- 172.21.15.91:0/1070807673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7940039a0 msgr2=0x7fb794003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/1070807673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7940039a0 0x7fb794003dc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001750 tx=0x7fb7940057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b3400640 1 -- 172.21.15.91:0/1070807673 reap_dead start 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 -- 172.21.15.91:0/1070807673 shutdown_connections 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 -- 172.21.15.91:0/1070807673 >> 172.21.15.91:0/1070807673 conn(0x7fb7b40cec80 msgr2=0x7fb7b40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 -- 172.21.15.91:0/1070807673 shutdown_connections 2024-02-02T13:31:17.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.119+0000 7fb7b9e00640 1 -- 172.21.15.91:0/1070807673 wait complete. 2024-02-02T13:31:17.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.121+0000 7fb7b9e00640 1 Processor -- start 2024-02-02T13:31:17.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.121+0000 7fb7b9e00640 1 -- start start 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 0x7fb7b40c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b1c60 0x7fb7b40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b21b0 con 0x7fb7b4159e10 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b415a7d0 con 0x7fb7940039a0 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b9e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40c5e60 con 0x7fb7b40b1c60 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44474/0 (socket says 172.21.15.91:44474) 2024-02-02T13:31:17.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b3400640 1 -- 172.21.15.91:0/3075194340 learned_addr learned my addr 172.21.15.91:0/3075194340 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 0x7fb7b40c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7a9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b1c60 0x7fb7b40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b3400640 1 -- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b1c60 msgr2=0x7fb7b40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.122+0000 7fb7b3400640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b1c60 0x7fb7b40b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b3400640 1 -- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 msgr2=0x7fb7b40c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b3400640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 0x7fb7b40c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b3400640 1 -- 172.21.15.91:0/3075194340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40c6140 con 0x7fb7b4159e10 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 0x7fb7b40c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b3400640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb7b406d770 tx=0x7fb7ac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00eb30 con 0x7fb7b4159e10 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00ec90 con 0x7fb7b4159e10 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac013420 con 0x7fb7b4159e10 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40c6420 con 0x7fb7b4159e10 2024-02-02T13:31:17.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.123+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40c6c20 con 0x7fb7b4159e10 2024-02-02T13:31:17.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.124+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b40c6940 con 0x7fb7b4159e10 2024-02-02T13:31:17.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.125+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb7ac013580 con 0x7fb7b4159e10 2024-02-02T13:31:17.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.126+0000 7fb7abe00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 0x7fb78c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.126+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7ac088d10 con 0x7fb7b4159e10 2024-02-02T13:31:17.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.126+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 0x7fb78c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.127+0000 7fb7b2a00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 0x7fb78c06c8f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4033000 tx=0x7fb7a4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.131+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb7b40c6940 con 0x7fb7b4159e10 2024-02-02T13:31:17.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.314+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7b40c6940 con 0x7fb7b4159e10 2024-02-02T13:31:17.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.316+0000 7fb7abe00640 1 -- 172.21.15.91:0/3075194340 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fb7b40c6940 con 0x7fb7b4159e10 2024-02-02T13:31:17.318 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:17.318 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":127,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:16.892576+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:17.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.318+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 msgr2=0x7fb78c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.318+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 0x7fb78c06c8f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4033000 tx=0x7fb7a4009030 comp rx=0 tx=0).stop 2024-02-02T13:31:17.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.318+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 msgr2=0x7fb7b415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.318+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb7b406d770 tx=0x7fb7ac00e530 comp rx=0 tx=0).stop 2024-02-02T13:31:17.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 shutdown_connections 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb78c06a430 0x7fb78c06c8f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7b40b1c60 0x7fb7b40b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7b4159e10 0x7fb7b415a290 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 --2- 172.21.15.91:0/3075194340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7940039a0 0x7fb7b40c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 >> 172.21.15.91:0/3075194340 conn(0x7fb7b40cec80 msgr2=0x7fb7b4160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 shutdown_connections 2024-02-02T13:31:17.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.319+0000 7fb7b9e00640 1 -- 172.21.15.91:0/3075194340 wait complete. 2024-02-02T13:31:17.338 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:17.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.529+0000 7f2623200640 1 Processor -- start 2024-02-02T13:31:17.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.529+0000 7f2623200640 1 -- start start 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.530+0000 7f2623200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f262415aec0 0x7f26240cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.530+0000 7f2623200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262406f3f0 0x7f26240ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.530+0000 7f2623200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262406c1c0 con 0x7f262415b2a0 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.530+0000 7f2623200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262406b5f0 con 0x7f262415aec0 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.530+0000 7f2623200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262406cd60 con 0x7f262406f3f0 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f262415aec0 0x7f26240cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f262415aec0 0x7f26240cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44488/0 (socket says 172.21.15.91:44488) 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 -- 172.21.15.91:0/3413919691 learned_addr learned my addr 172.21.15.91:0/3413919691 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2622800640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262406f3f0 0x7f26240ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 -- 172.21.15.91:0/3413919691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262406f3f0 msgr2=0x7f26240ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262406f3f0 0x7f26240ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 -- 172.21.15.91:0/3413919691 >> v1:172.21.15.113:6789/0 conn(0x7f262415b2a0 legacy=0x7f262406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.531+0000 7f2621e00640 1 -- 172.21.15.91:0/3413919691 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262406b890 con 0x7f262415aec0 2024-02-02T13:31:17.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2621e00640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415aec0 0x7f26240cf5b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d741d9126b20a423 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2621400640 1 -- 172.21.15.91:0/3413919691 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614030840 con 0x7f262415aec0 2024-02-02T13:31:17.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2621400640 1 -- 172.21.15.91:0/3413919691 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26140309a0 con 0x7f262415aec0 2024-02-02T13:31:17.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2621400640 1 -- 172.21.15.91:0/3413919691 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614030d30 con 0x7f262415aec0 2024-02-02T13:31:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2623200640 1 -- 172.21.15.91:0/3413919691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415aec0 msgr2=0x7f26240cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.533+0000 7f2623200640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415aec0 0x7f26240cf5b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b980 comp rx=0 tx=0).stop 2024-02-02T13:31:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 -- 172.21.15.91:0/3413919691 shutdown_connections 2024-02-02T13:31:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262406f3f0 0x7f26240ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 --2- 172.21.15.91:0/3413919691 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415aec0 0x7f26240cf5b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 -- 172.21.15.91:0/3413919691 >> 172.21.15.91:0/3413919691 conn(0x7f26240cea90 msgr2=0x7f26240ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:17.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 -- 172.21.15.91:0/3413919691 shutdown_connections 2024-02-02T13:31:17.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.534+0000 7f2623200640 1 -- 172.21.15.91:0/3413919691 wait complete. 2024-02-02T13:31:17.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.535+0000 7f2623200640 1 Processor -- start 2024-02-02T13:31:17.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.535+0000 7f2623200640 1 -- start start 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 0x7f26241586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262415aec0 0x7f26240ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240cc660 con 0x7f262415b2a0 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240cc8b0 con 0x7f262406f3f0 2024-02-02T13:31:17.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.536+0000 7f2623200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240ccb90 con 0x7f262415aec0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44502/0 (socket says 172.21.15.91:44502) 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 -- 172.21.15.91:0/1619985930 learned_addr learned my addr 172.21.15.91:0/1619985930 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2621e00640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 0x7f26241586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 -- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262415aec0 msgr2=0x7f26240ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f261be00640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262415aec0 0x7f26240ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262415aec0 0x7f26240ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 -- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 msgr2=0x7f26241586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 0x7f26241586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 -- 172.21.15.91:0/1619985930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140094d0 con 0x7f262415b2a0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2621e00640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 0x7f26241586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.537+0000 7f2622800640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f261c007d90 tx=0x7f261c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.538+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c014420 con 0x7f262415b2a0 2024-02-02T13:31:17.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.538+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0145a0 con 0x7f262415b2a0 2024-02-02T13:31:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.538+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26240cce70 con 0x7f262415b2a0 2024-02-02T13:31:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.538+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c014950 con 0x7f262415b2a0 2024-02-02T13:31:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.538+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624158c20 con 0x7f262415b2a0 2024-02-02T13:31:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.539+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26240cd390 con 0x7f262415b2a0 2024-02-02T13:31:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.540+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f261c014ab0 con 0x7f262415b2a0 2024-02-02T13:31:17.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.540+0000 7f261a000640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 0x7f25fc06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.541+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f261c089220 con 0x7f262415b2a0 2024-02-02T13:31:17.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.541+0000 7f2621e00640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 0x7f25fc06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.541+0000 7f2621e00640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 0x7f25fc06c940 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f261403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.545+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f26240cd390 con 0x7f262415b2a0 2024-02-02T13:31:17.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.720+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26240cd390 con 0x7f262415b2a0 2024-02-02T13:31:17.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.722+0000 7f261a000640 1 -- 172.21.15.91:0/1619985930 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f26240cd390 con 0x7f262415b2a0 2024-02-02T13:31:17.724 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:17.724 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":127,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:16.892576+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 msgr2=0x7f25fc06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 0x7f25fc06c940 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f261403b040 comp rx=0 tx=0).stop 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 msgr2=0x7f26240ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f261c007d90 tx=0x7f261c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 shutdown_connections 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f25fc06a480 0x7f25fc06c940 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f262415aec0 0x7f26240ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f262415b2a0 0x7f26240ace10 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 --2- 172.21.15.91:0/1619985930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262406f3f0 0x7f26241586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.724+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 >> 172.21.15.91:0/1619985930 conn(0x7f26240cea90 msgr2=0x7f2624070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:17.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.725+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 shutdown_connections 2024-02-02T13:31:17.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.725+0000 7f2623200640 1 -- 172.21.15.91:0/1619985930 wait complete. 2024-02-02T13:31:17.744 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:31:17.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.945+0000 7f71e6a00640 1 Processor -- start 2024-02-02T13:31:17.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.946+0000 7f71e6a00640 1 -- start start 2024-02-02T13:31:17.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71e015d170 0x7f71e00d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e00b6f00 0x7f71e00cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e6a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e00d11b0 con 0x7f71e00cf820 2024-02-02T13:31:17.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e6a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e00d01f0 con 0x7f71e015d170 2024-02-02T13:31:17.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e6a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e015d550 con 0x7f71e00b6f00 2024-02-02T13:31:17.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.947+0000 7f71e5600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71e015d170 0x7f71e00d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71e015d170 0x7f71e00d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44510/0 (socket says 172.21.15.91:44510) 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 -- 172.21.15.91:0/839436454 learned_addr learned my addr 172.21.15.91:0/839436454 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e6000640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e00b6f00 0x7f71e00cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 -- 172.21.15.91:0/839436454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e00b6f00 msgr2=0x7f71e00cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e00b6f00 0x7f71e00cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 -- 172.21.15.91:0/839436454 >> v1:172.21.15.113:6789/0 conn(0x7f71e00cf820 legacy=0x7f71e00b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.948+0000 7f71e5600640 1 -- 172.21.15.91:0/839436454 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e00d1490 con 0x7f71e015d170 2024-02-02T13:31:17.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.949+0000 7f71e5600640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e015d170 0x7f71e00d24c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098a0 tx=0x7f71d002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615fb1329d57bc39 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.949+0000 7f71dfe00640 1 -- 172.21.15.91:0/839436454 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d002f830 con 0x7f71e015d170 2024-02-02T13:31:17.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71dfe00640 1 -- 172.21.15.91:0/839436454 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d002f990 con 0x7f71e015d170 2024-02-02T13:31:17.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71dfe00640 1 -- 172.21.15.91:0/839436454 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d002fd20 con 0x7f71e015d170 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 -- 172.21.15.91:0/839436454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e015d170 msgr2=0x7f71e00d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e015d170 0x7f71e00d24c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098a0 tx=0x7f71d002ba10 comp rx=0 tx=0).stop 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 -- 172.21.15.91:0/839436454 shutdown_connections 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e00b6f00 0x7f71e00cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 --2- 172.21.15.91:0/839436454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e015d170 0x7f71e00d24c0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 -- 172.21.15.91:0/839436454 >> 172.21.15.91:0/839436454 conn(0x7f71e0065910 msgr2=0x7f71e0065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 -- 172.21.15.91:0/839436454 shutdown_connections 2024-02-02T13:31:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.950+0000 7f71e6a00640 1 -- 172.21.15.91:0/839436454 wait complete. 2024-02-02T13:31:17.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.952+0000 7f71e6a00640 1 Processor -- start 2024-02-02T13:31:17.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.952+0000 7f71e6a00640 1 -- start start 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 0x7f71e01f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e015d170 0x7f71e00b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00caf80 con 0x7f71e00b6f00 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00cb260 con 0x7f71e00cf820 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.953+0000 7f71e6a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00cb540 con 0x7f71e015d170 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44512/0 (socket says 172.21.15.91:44512) 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 -- 172.21.15.91:0/2094371333 learned_addr learned my addr 172.21.15.91:0/2094371333 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e4c00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 0x7f71e01f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e6000640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e015d170 0x7f71e00b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 -- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e015d170 msgr2=0x7f71e00b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e015d170 0x7f71e00b7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 -- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 msgr2=0x7f71e01f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 0x7f71e01f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e5600640 1 -- 172.21.15.91:0/2094371333 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00094d0 con 0x7f71e00b6f00 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.954+0000 7f71e4c00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 0x7f71e01f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:17.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71e5600640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098a0 tx=0x7f71d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0045070 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d00379c0 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037d70 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e00cb820 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.955+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e00cbde0 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.956+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e00a97e0 con 0x7f71e00b6f00 2024-02-02T13:31:17.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.957+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f71d004a410 con 0x7f71e00b6f00 2024-02-02T13:31:17.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.958+0000 7f71dd600640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 0x7f71b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:17.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.958+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71d003c070 con 0x7f71e00b6f00 2024-02-02T13:31:17.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.958+0000 7f71e4c00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 0x7f71b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:17.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.959+0000 7f71e4c00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 0x7f71b806c8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b1830 tx=0x7f71c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:17.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:17.962+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f71e00a97e0 con 0x7f71e00b6f00 2024-02-02T13:31:18.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-02-02T13:31:18.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-02-02T13:31:18.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.189+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f71e00a97e0 con 0x7f71e00b6f00 2024-02-02T13:31:18.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.932+0000 7f71dd600640 1 -- 172.21.15.91:0/2094371333 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f71d0076390 con 0x7f71e00b6f00 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 msgr2=0x7f71b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 0x7f71b806c8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b1830 tx=0x7f71c8009040 comp rx=0 tx=0).stop 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 msgr2=0x7f71e01f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098a0 tx=0x7f71d0004730 comp rx=0 tx=0).stop 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 shutdown_connections 2024-02-02T13:31:18.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f71b806a430 0x7f71b806c8f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71e015d170 0x7f71e00b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71e00cf820 0x7f71e01f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 --2- 172.21.15.91:0/2094371333 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71e00b6f00 0x7f71e01f2440 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.935+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 >> 172.21.15.91:0/2094371333 conn(0x7f71e0065910 msgr2=0x7f71e015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.936+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 shutdown_connections 2024-02-02T13:31:18.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:18.936+0000 7f71e6a00640 1 -- 172.21.15.91:0/2094371333 wait complete. 2024-02-02T13:31:18.957 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:31:18.957 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:31:19.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.160+0000 7fd862200640 1 Processor -- start 2024-02-02T13:31:19.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.160+0000 7fd862200640 1 -- start start 2024-02-02T13:31:19.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.161+0000 7fd862200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd85c15d420 0x7fd85c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.161+0000 7fd862200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.161+0000 7fd862200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd85c0d1b30 0x7fd85c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd862200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0d2500 con 0x7fd85c15d420 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd862200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0d27a0 con 0x7fd85c0d1b30 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd862200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0d2a80 con 0x7fd85c15d800 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd85c15d420 0x7fd85c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd85c15d420 0x7fd85c070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44514/0 (socket says 172.21.15.91:44514) 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 -- 172.21.15.91:0/1738519214 learned_addr learned my addr 172.21.15.91:0/1738519214 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85be00640 1 --2- 172.21.15.91:0/1738519214 >> v2:172.21.15.113:3300/0 conn(0x7fd85c0d1b30 0x7fd85c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd853e00640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 -- 172.21.15.91:0/1738519214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 msgr2=0x7fd85c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c0707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.162+0000 7fd85b400640 1 -- 172.21.15.91:0/1738519214 >> v2:172.21.15.113:3300/0 conn(0x7fd85c0d1b30 msgr2=0x7fd85c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.163+0000 7fd85b400640 1 --2- 172.21.15.91:0/1738519214 >> v2:172.21.15.113:3300/0 conn(0x7fd85c0d1b30 0x7fd85c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.163+0000 7fd85b400640 1 -- 172.21.15.91:0/1738519214 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0d2d60 con 0x7fd85c15d420 2024-02-02T13:31:19.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.164+0000 7fd85b400640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c070270 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b8ee0 tx=0x7fd84c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43786f92222cf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:19.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.164+0000 7fd85aa00640 1 -- 172.21.15.91:0/1738519214 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c031870 con 0x7fd85c15d420 2024-02-02T13:31:19.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.164+0000 7fd85aa00640 1 -- 172.21.15.91:0/1738519214 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c0319d0 con 0x7fd85c15d420 2024-02-02T13:31:19.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.164+0000 7fd85aa00640 1 -- 172.21.15.91:0/1738519214 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c031d60 con 0x7fd85c15d420 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 -- 172.21.15.91:0/1738519214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 msgr2=0x7fd85c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c070270 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b8ee0 tx=0x7fd84c02b910 comp rx=0 tx=0).stop 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 -- 172.21.15.91:0/1738519214 shutdown_connections 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 --2- 172.21.15.91:0/1738519214 >> v2:172.21.15.113:3300/0 conn(0x7fd85c0d1b30 0x7fd85c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 --2- 172.21.15.91:0/1738519214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c070270 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 -- 172.21.15.91:0/1738519214 >> 172.21.15.91:0/1738519214 conn(0x7fd85c0d3080 msgr2=0x7fd85c0d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 -- 172.21.15.91:0/1738519214 shutdown_connections 2024-02-02T13:31:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.165+0000 7fd862200640 1 -- 172.21.15.91:0/1738519214 wait complete. 2024-02-02T13:31:19.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.167+0000 7fd862200640 1 Processor -- start 2024-02-02T13:31:19.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.167+0000 7fd862200640 1 -- start start 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c0d1b30 0x7fd85c14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1f72d0 con 0x7fd85c15d420 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14e230 con 0x7fd85c0d1b30 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd862200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14b3e0 con 0x7fd85c15d800 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd853e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd853e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:44530/0 (socket says 172.21.15.91:44530) 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.168+0000 7fd853e00640 1 -- 172.21.15.91:0/1079424265 learned_addr learned my addr 172.21.15.91:0/1079424265 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd85b400640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c0d1b30 0x7fd85c14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 -- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 msgr2=0x7fd85c14a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:19.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd85be00640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c14a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 -- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c0d1b30 msgr2=0x7fd85c14d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c0d1b30 0x7fd85c14d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 -- 172.21.15.91:0/1079424265 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0094d0 con 0x7fd85c15d420 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853e00640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d620 tx=0x7fd84400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400b620 con 0x7fd85c15d420 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84400b780 con 0x7fd85c15d420 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400bb30 con 0x7fd85c15d420 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.169+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c14d990 con 0x7fd85c15d420 2024-02-02T13:31:19.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.170+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c14df50 con 0x7fd85c15d420 2024-02-02T13:31:19.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.171+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c066cf0 con 0x7fd85c15d420 2024-02-02T13:31:19.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.171+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd84400bc90 con 0x7fd85c15d420 2024-02-02T13:31:19.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.173+0000 7fd853400640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 0x7fd83406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:19.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.173+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd844088ce0 con 0x7fd85c15d420 2024-02-02T13:31:19.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.173+0000 7fd85b400640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 0x7fd83406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:19.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.173+0000 7fd85b400640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 0x7fd83406c8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd84c036000 tx=0x7fd84c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:19.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.177+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd844054360 con 0x7fd85c15d420 2024-02-02T13:31:19.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:19.408+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd85c0d0dc0 con 0x7fd85c15d420 2024-02-02T13:31:20.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.395+0000 7fd853400640 1 -- 172.21.15.91:0/1079424265 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd844057980 con 0x7fd85c15d420 2024-02-02T13:31:20.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 msgr2=0x7fd83406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 0x7fd83406c8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd84c036000 tx=0x7fd84c03c040 comp rx=0 tx=0).stop 2024-02-02T13:31:20.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 msgr2=0x7fd85c1f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d620 tx=0x7fd84400dae0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 shutdown_connections 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd83406a430 0x7fd83406c8f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c15d800 0x7fd85c14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd85c15d420 0x7fd85c1f6d90 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 --2- 172.21.15.91:0/1079424265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c0d1b30 0x7fd85c14d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.398+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 >> 172.21.15.91:0/1079424265 conn(0x7fd85c0d3080 msgr2=0x7fd85c160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.399+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 shutdown_connections 2024-02-02T13:31:20.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.399+0000 7fd862200640 1 -- 172.21.15.91:0/1079424265 wait complete. 2024-02-02T13:31:20.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:31:20.421 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:20.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.624+0000 7fc009400640 1 Processor -- start 2024-02-02T13:31:20.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.624+0000 7fc009400640 1 -- start start 2024-02-02T13:31:20.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fc009400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 0x7fc00406b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fc009400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc00406cad0 0x7fc0040b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fc009400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc004161140 con 0x7fc0040b0c60 2024-02-02T13:31:20.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fc009400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0040b10f0 con 0x7fc00406cad0 2024-02-02T13:31:20.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fc009400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc004161420 con 0x7fc00415ac90 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fbff9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc00406cad0 0x7fc0040b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fbff9c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc00406cad0 0x7fc0040b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49612/0 (socket says 172.21.15.91:49612) 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.626+0000 7fbff9c00640 1 -- 172.21.15.91:0/1301209763 learned_addr learned my addr 172.21.15.91:0/1301209763 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fc002800640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 0x7fc00406b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fbff9c00640 1 -- 172.21.15.91:0/1301209763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 msgr2=0x7fc00406b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:20.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fbff9c00640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 0x7fc00406b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fbff9c00640 1 -- 172.21.15.91:0/1301209763 >> v1:172.21.15.91:6789/0 conn(0x7fc0040b0c60 legacy=0x7fc00415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fbff9c00640 1 -- 172.21.15.91:0/1301209763 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004161700 con 0x7fc00406cad0 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.627+0000 7fc002800640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 0x7fc00406b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.628+0000 7fbff9c00640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc00406cad0 0x7fc0040b0720 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009870 tx=0x7fbfec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55567ed56de94859 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.628+0000 7fc001e00640 1 -- 172.21.15.91:0/1301209763 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec031640 con 0x7fc00406cad0 2024-02-02T13:31:20.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.628+0000 7fc001e00640 1 -- 172.21.15.91:0/1301209763 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0317a0 con 0x7fc00406cad0 2024-02-02T13:31:20.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.628+0000 7fc001e00640 1 -- 172.21.15.91:0/1301209763 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec031b30 con 0x7fc00406cad0 2024-02-02T13:31:20.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 -- 172.21.15.91:0/1301209763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc00406cad0 msgr2=0x7fc0040b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc00406cad0 0x7fc0040b0720 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009870 tx=0x7fbfec02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 -- 172.21.15.91:0/1301209763 shutdown_connections 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc00406cad0 0x7fc0040b0720 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 --2- 172.21.15.91:0/1301209763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00415ac90 0x7fc00406b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 -- 172.21.15.91:0/1301209763 >> 172.21.15.91:0/1301209763 conn(0x7fc0040ceca0 msgr2=0x7fc0040cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 -- 172.21.15.91:0/1301209763 shutdown_connections 2024-02-02T13:31:20.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.629+0000 7fc009400640 1 -- 172.21.15.91:0/1301209763 wait complete. 2024-02-02T13:31:20.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.631+0000 7fc009400640 1 Processor -- start 2024-02-02T13:31:20.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.631+0000 7fc009400640 1 -- start start 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 0x7fc00415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc00415ac90 0x7fc0040b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415da30 con 0x7fc0040b0c60 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040b07b0 con 0x7fc00415ac90 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc009400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040af610 con 0x7fc00406cad0 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49616/0 (socket says 172.21.15.91:49616) 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 -- 172.21.15.91:0/3610576917 learned_addr learned my addr 172.21.15.91:0/3610576917 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc002800640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 0x7fc00415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc003200640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc00415ac90 0x7fc0040b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 -- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 msgr2=0x7fc00415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 0x7fc00415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 -- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc00415ac90 msgr2=0x7fc0040b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc00415ac90 0x7fc0040b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fbff9c00640 1 -- 172.21.15.91:0/3610576917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec0094d0 con 0x7fc0040b0c60 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.632+0000 7fc002800640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 0x7fc00415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:20.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fbff9c00640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec03ac60 con 0x7fc0040b0c60 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec03ade0 con 0x7fc0040b0c60 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec044630 con 0x7fc0040b0c60 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040aed70 con 0x7fc0040b0c60 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.633+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040af330 con 0x7fc0040b0c60 2024-02-02T13:31:20.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.634+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00415d6a0 con 0x7fc0040b0c60 2024-02-02T13:31:20.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.635+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbfec02f050 con 0x7fc0040b0c60 2024-02-02T13:31:20.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.636+0000 7fbffb400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 0x7fbfdc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:20.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.636+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfec0aaac0 con 0x7fc0040b0c60 2024-02-02T13:31:20.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.636+0000 7fc002800640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 0x7fbfdc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:20.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.637+0000 7fc002800640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 0x7fbfdc06c8f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbff400d430 tx=0x7fbff4007530 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:20.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.641+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbfec0760d0 con 0x7fc0040b0c60 2024-02-02T13:31:20.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.868+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc004161c90 con 0x7fc0040b0c60 2024-02-02T13:31:20.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.869+0000 7fbffb400640 1 -- 172.21.15.91:0/3610576917 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fbfec0796f0 con 0x7fc0040b0c60 2024-02-02T13:31:20.871 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:20.871 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:20.871 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:20.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 msgr2=0x7fbfdc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 0x7fbfdc06c8f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbff400d430 tx=0x7fbff4007530 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 msgr2=0x7fc0040ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 shutdown_connections 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.871+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfdc06a430 0x7fbfdc06c8f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc00415ac90 0x7fc0040b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0040b0c60 0x7fc0040ae6e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 --2- 172.21.15.91:0/3610576917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc00406cad0 0x7fc00415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 >> 172.21.15.91:0/3610576917 conn(0x7fc0040ceca0 msgr2=0x7fc0040acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 shutdown_connections 2024-02-02T13:31:20.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:20.872+0000 7fc009400640 1 -- 172.21.15.91:0/3610576917 wait complete. 2024-02-02T13:31:20.891 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:21.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.099+0000 7f3d4d400640 1 Processor -- start 2024-02-02T13:31:21.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.099+0000 7f3d4d400640 1 -- start start 2024-02-02T13:31:21.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.100+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d480c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.100+0000 7f3d4d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d480b0700 0x7f3d480b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.100+0000 7f3d4d400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d480cc7c0 con 0x7f3d480ce130 2024-02-02T13:31:21.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.100+0000 7f3d4d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d480b10d0 con 0x7f3d480b0700 2024-02-02T13:31:21.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.100+0000 7f3d4d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480ce930 con 0x7f3d4815a010 2024-02-02T13:31:21.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d3dc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3d480ce130 0x7f3d480ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44044/0 (socket says 172.21.15.91:44044) 2024-02-02T13:31:21.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d3dc00640 1 -- 172.21.15.91:0/1898783580 learned_addr learned my addr 172.21.15.91:0/1898783580 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:21.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d47200640 1 --2- 172.21.15.91:0/1898783580 >> v2:172.21.15.113:3300/0 conn(0x7f3d480b0700 0x7f3d480b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d46800640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d480c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86614986 0 0) 0x7f3d480cc7c0 con 0x7f3d480ce130 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.101+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d28003520 con 0x7f3d480ce130 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3832199422 0 0) 0x7f3d28003520 con 0x7f3d480ce130 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 msgr2=0x7f3d480c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d480c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 >> v2:172.21.15.113:3300/0 conn(0x7f3d480b0700 msgr2=0x7f3d480b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 --2- 172.21.15.91:0/1898783580 >> v2:172.21.15.113:3300/0 conn(0x7f3d480b0700 0x7f3d480b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480cd640 con 0x7f3d480ce130 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f3d30001670 con 0x7f3d480ce130 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 >> v1:172.21.15.91:6789/0 conn(0x7f3d480ce130 legacy=0x7f3d480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d28003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d28003ef0 0x7f3d280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480b10d0 con 0x7f3d28004d00 2024-02-02T13:31:21.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480ce930 con 0x7f3d28003ef0 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2800b980 con 0x7f3d4815a010 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.102+0000 7f3d47200640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d46800640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d28003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d3dc00640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d28003ef0 0x7f3d280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d47200640 1 -- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 msgr2=0x7f3d28003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d47200640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d4815a010 0x7f3d28003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d47200640 1 -- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d28003ef0 msgr2=0x7f3d280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d47200640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d28003ef0 0x7f3d280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.103+0000 7f3d47200640 1 -- 172.21.15.91:0/1898783580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d28003520 con 0x7f3d28004d00 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.104+0000 7f3d47200640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d28005180 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009ac0 tx=0x7f3d280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f91f15faeb602ea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.104+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4002c6d0 con 0x7f3d28004d00 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.104+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4002c830 con 0x7f3d28004d00 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.104+0000 7f3d45e00640 1 -- 172.21.15.91:0/1898783580 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4002cbc0 con 0x7f3d28004d00 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 -- 172.21.15.91:0/1898783580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 msgr2=0x7f3d28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 --2- 172.21.15.91:0/1898783580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d28005180 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009ac0 tx=0x7f3d280057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d47200640 1 -- 172.21.15.91:0/1898783580 reap_dead start 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 -- 172.21.15.91:0/1898783580 shutdown_connections 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 -- 172.21.15.91:0/1898783580 >> 172.21.15.91:0/1898783580 conn(0x7f3d480cec80 msgr2=0x7f3d480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 -- 172.21.15.91:0/1898783580 shutdown_connections 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.105+0000 7f3d4d400640 1 -- 172.21.15.91:0/1898783580 wait complete. 2024-02-02T13:31:21.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.107+0000 7f3d4d400640 1 Processor -- start 2024-02-02T13:31:21.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.107+0000 7f3d4d400640 1 -- start start 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c6a70 0x7f3d481f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d480c4f00 0x7f3d480c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c25a0 con 0x7f3d28004d00 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c58c0 con 0x7f3d480c6a70 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d4d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c7280 con 0x7f3d480c4f00 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49632/0 (socket says 172.21.15.91:49632) 2024-02-02T13:31:21.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 -- 172.21.15.91:0/452918734 learned_addr learned my addr 172.21.15.91:0/452918734 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 -- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d480c4f00 msgr2=0x7f3d480c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d480c4f00 0x7f3d480c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 -- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c6a70 msgr2=0x7f3d481f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c6a70 0x7f3d481f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.109+0000 7f3d46800640 1 -- 172.21.15.91:0/452918734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480c7560 con 0x7f3d28004d00 2024-02-02T13:31:21.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d46800640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007f30 tx=0x7f3d38011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e030 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38016470 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38016820 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d480c7840 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.110+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d480c6ef0 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.111+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d480c7a30 con 0x7f3d28004d00 2024-02-02T13:31:21.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.112+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3d38016980 con 0x7f3d28004d00 2024-02-02T13:31:21.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.113+0000 7f3d3f400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 0x7f3d2006c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.114+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d3800f070 con 0x7f3d28004d00 2024-02-02T13:31:21.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.114+0000 7f3d3dc00640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 0x7f3d2006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.114+0000 7f3d3dc00640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 0x7f3d2006c910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d30000f60 tx=0x7f3d30000ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.118+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3d38054e00 con 0x7f3d28004d00 2024-02-02T13:31:21.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.309+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d48066cf0 con 0x7f3d28004d00 2024-02-02T13:31:21.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.311+0000 7f3d3f400640 1 -- 172.21.15.91:0/452918734 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f3d38058420 con 0x7f3d28004d00 2024-02-02T13:31:21.313 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:21.314 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":128,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:17.897216+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 msgr2=0x7f3d2006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 0x7f3d2006c910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d30000f60 tx=0x7f3d30000ef0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 msgr2=0x7f3d48151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007f30 tx=0x7f3d38011660 comp rx=0 tx=0).stop 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 shutdown_connections 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d2006a450 0x7f3d2006c910 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d480c4f00 0x7f3d480c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c6a70 0x7f3d481f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 --2- 172.21.15.91:0/452918734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d28004d00 0x7f3d48151f30 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 >> 172.21.15.91:0/452918734 conn(0x7f3d480cec80 msgr2=0x7f3d4806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:21.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.314+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 shutdown_connections 2024-02-02T13:31:21.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.315+0000 7f3d4d400640 1 -- 172.21.15.91:0/452918734 wait complete. 2024-02-02T13:31:21.335 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:31:21.335 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:31:21.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.534+0000 7f6a47e00640 1 Processor -- start 2024-02-02T13:31:21.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.535+0000 7f6a47e00640 1 -- start start 2024-02-02T13:31:21.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4815d3a0 0x7f6a4806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a480c9450 0x7f6a480c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480ce3b0 con 0x7f6a480add80 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a480adbd0 con 0x7f6a480c9450 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c59d0 con 0x7f6a4815d3a0 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.536+0000 7f6a47400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a480c9450 0x7f6a480c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6a480c9450 0x7f6a480c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49636/0 (socket says 172.21.15.91:49636) 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 -- 172.21.15.91:0/794233599 learned_addr learned my addr 172.21.15.91:0/794233599 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a46a00640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4815d3a0 0x7f6a4806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 -- 172.21.15.91:0/794233599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4815d3a0 msgr2=0x7f6a4806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4815d3a0 0x7f6a4806b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 -- 172.21.15.91:0/794233599 >> v1:172.21.15.113:6789/0 conn(0x7f6a480add80 legacy=0x7f6a480706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:21.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.537+0000 7f6a47400640 1 -- 172.21.15.91:0/794233599 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480ce8e0 con 0x7f6a480c9450 2024-02-02T13:31:21.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.538+0000 7f6a47400640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480c9450 0x7f6a480c98e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009e50 tx=0x7f6a4002dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f507357985c766 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.538+0000 7f6a46000640 1 -- 172.21.15.91:0/794233599 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4002eb60 con 0x7f6a480c9450 2024-02-02T13:31:21.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.538+0000 7f6a46000640 1 -- 172.21.15.91:0/794233599 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4002ecc0 con 0x7f6a480c9450 2024-02-02T13:31:21.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.538+0000 7f6a46000640 1 -- 172.21.15.91:0/794233599 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40036610 con 0x7f6a480c9450 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 -- 172.21.15.91:0/794233599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480c9450 msgr2=0x7f6a480c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480c9450 0x7f6a480c98e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009e50 tx=0x7f6a4002dfc0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 -- 172.21.15.91:0/794233599 shutdown_connections 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480c9450 0x7f6a480c98e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 --2- 172.21.15.91:0/794233599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4815d3a0 0x7f6a4806b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 -- 172.21.15.91:0/794233599 >> 172.21.15.91:0/794233599 conn(0x7f6a480cea90 msgr2=0x7f6a480ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 -- 172.21.15.91:0/794233599 shutdown_connections 2024-02-02T13:31:21.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.539+0000 7f6a47e00640 1 -- 172.21.15.91:0/794233599 wait complete. 2024-02-02T13:31:21.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.541+0000 7f6a47e00640 1 Processor -- start 2024-02-02T13:31:21.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.542+0000 7f6a47e00640 1 -- start start 2024-02-02T13:31:21.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a480c9450 0x7f6a48153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4815d3a0 0x7f6a480c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c3f50 con 0x7f6a480add80 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c3060 con 0x7f6a4815d3a0 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c3340 con 0x7f6a480c9450 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49646/0 (socket says 172.21.15.91:49646) 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 -- 172.21.15.91:0/2863927426 learned_addr learned my addr 172.21.15.91:0/2863927426 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a47400640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4815d3a0 0x7f6a480c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 -- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a480c9450 msgr2=0x7f6a48153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a480c9450 0x7f6a48153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 -- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4815d3a0 msgr2=0x7f6a480c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4815d3a0 0x7f6a480c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.543+0000 7f6a46a00640 1 -- 172.21.15.91:0/2863927426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40009b30 con 0x7f6a480add80 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a46a00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c7f0 tx=0x7f6a3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004220 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a380043a0 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480c3620 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38007860 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.544+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480c3be0 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.545+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4804dff0 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.546+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6a38007bd0 con 0x7f6a480add80 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.547+0000 7f6a3f400640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 0x7f6a2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:21.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.547+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a38088940 con 0x7f6a480add80 2024-02-02T13:31:21.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.547+0000 7f6a3de00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 0x7f6a2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:21.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.548+0000 7f6a3de00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 0x7f6a2006c8f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6a480cc720 tx=0x7f6a30006be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:21.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.552+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6a38053f20 con 0x7f6a480add80 2024-02-02T13:31:21.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.737+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a4815d780 con 0x7f6a2006a430 2024-02-02T13:31:21.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.749+0000 7f6a3f400640 1 -- 172.21.15.91:0/2863927426 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274195 (secure 0 0 0) 0x7f6a30012390 con 0x7f6a2006a430 2024-02-02T13:31:21.754 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:21.755 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 msgr2=0x7f6a2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 0x7f6a2006c8f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6a480cc720 tx=0x7f6a30006be0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 msgr2=0x7f6a4815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c7f0 tx=0x7f6a3800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 shutdown_connections 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6a2006a430 0x7f6a2006c8f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a480c9450 0x7f6a48153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4815d3a0 0x7f6a480c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 --2- 172.21.15.91:0/2863927426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a480add80 0x7f6a4815c890 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:21.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 >> 172.21.15.91:0/2863927426 conn(0x7f6a480cea90 msgr2=0x7f6a480cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:21.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 shutdown_connections 2024-02-02T13:31:21.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:21.755+0000 7f6a47e00640 1 -- 172.21.15.91:0/2863927426 wait complete. 2024-02-02T13:31:21.775 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-02-02T13:31:20.368895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555992,"statfs":{"total":1600000000,"available":1593336237,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.974478"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.401578+0000","last_change":"2024-02-02T13:31:15.401578+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.401578+0000","last_undegraded":"2024-02-02T13:31:15.401578+0000","last_fullsized":"2024-02-02T13:31:15.401578+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:16:12.721762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.407263+0000","last_change":"2024-02-02T13:31:15.407263+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.407263+0000","last_undegraded":"2024-02-02T13:31:15.407263+0000","last_fullsized":"2024-02-02T13:31:15.407263+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399421+0000","last_change":"2024-02-02T13:31:15.399421+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399421+0000","last_undegraded":"2024-02-02T13:31:15.399421+0000","last_fullsized":"2024-02-02T13:31:15.399421+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398278+0000","last_change":"2024-02-02T13:31:15.398278+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398278+0000","last_undegraded":"2024-02-02T13:31:15.398278+0000","last_fullsized":"2024-02-02T13:31:15.398278+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405447+0000","last_change":"2024-02-02T13:31:15.405447+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405447+0000","last_undegraded":"2024-02-02T13:31:15.405447+0000","last_fullsized":"2024-02-02T13:31:15.405447+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:54.011838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.400737+0000","last_change":"2024-02-02T13:31:15.400737+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.400737+0000","last_undegraded":"2024-02-02T13:31:15.400737+0000","last_fullsized":"2024-02-02T13:31:15.400737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404639+0000","last_change":"2024-02-02T13:31:15.404639+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404639+0000","last_undegraded":"2024-02-02T13:31:15.404639+0000","last_fullsized":"2024-02-02T13:31:15.404639+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:50:14.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404966+0000","last_change":"2024-02-02T13:31:15.404966+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404966+0000","last_undegraded":"2024-02-02T13:31:15.404966+0000","last_fullsized":"2024-02-02T13:31:15.404966+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:06.887358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404648+0000","last_change":"2024-02-02T13:31:15.404648+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404648+0000","last_undegraded":"2024-02-02T13:31:15.404648+0000","last_fullsized":"2024-02-02T13:31:15.404648+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:46.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404251+0000","last_change":"2024-02-02T13:31:15.404251+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404251+0000","last_undegraded":"2024-02-02T13:31:15.404251+0000","last_fullsized":"2024-02-02T13:31:15.404251+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.406293+0000","last_change":"2024-02-02T13:31:15.406293+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.406293+0000","last_undegraded":"2024-02-02T13:31:15.406293+0000","last_fullsized":"2024-02-02T13:31:15.406293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403212+0000","last_change":"2024-02-02T13:31:15.403212+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403212+0000","last_undegraded":"2024-02-02T13:31:15.403212+0000","last_fullsized":"2024-02-02T13:31:15.403212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:46.255117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.396179+0000","last_change":"2024-02-02T13:31:15.396179+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.396179+0000","last_undegraded":"2024-02-02T13:31:15.396179+0000","last_fullsized":"2024-02-02T13:31:15.396179+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:47.961252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395214+0000","last_change":"2024-02-02T13:31:15.395214+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395214+0000","last_undegraded":"2024-02-02T13:31:15.395214+0000","last_fullsized":"2024-02-02T13:31:15.395214+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:55.831956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403217+0000","last_change":"2024-02-02T13:31:15.403217+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403217+0000","last_undegraded":"2024-02-02T13:31:15.403217+0000","last_fullsized":"2024-02-02T13:31:15.403217+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403500+0000","last_change":"2024-02-02T13:31:15.403500+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403500+0000","last_undegraded":"2024-02-02T13:31:15.403500+0000","last_fullsized":"2024-02-02T13:31:15.403500+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:57.874029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403199+0000","last_change":"2024-02-02T13:31:15.403199+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403199+0000","last_undegraded":"2024-02-02T13:31:15.403199+0000","last_fullsized":"2024-02-02T13:31:15.403199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405066+0000","last_change":"2024-02-02T13:31:15.405066+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405066+0000","last_undegraded":"2024-02-02T13:31:15.405066+0000","last_fullsized":"2024-02-02T13:31:15.405066+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398582+0000","last_change":"2024-02-02T13:31:15.398582+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398582+0000","last_undegraded":"2024-02-02T13:31:15.398582+0000","last_fullsized":"2024-02-02T13:31:15.398582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:56.711812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399667+0000","last_change":"2024-02-02T13:31:15.399667+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399667+0000","last_undegraded":"2024-02-02T13:31:15.399667+0000","last_fullsized":"2024-02-02T13:31:15.399667+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:21.074291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395587+0000","last_change":"2024-02-02T13:31:15.395587+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395587+0000","last_undegraded":"2024-02-02T13:31:15.395587+0000","last_fullsized":"2024-02-02T13:31:15.395587+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:56.934997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395978+0000","last_change":"2024-02-02T13:31:15.395978+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395978+0000","last_undegraded":"2024-02-02T13:31:15.395978+0000","last_fullsized":"2024-02-02T13:31:15.395978+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.400249+0000","last_change":"2024-02-02T13:31:15.400249+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.400249+0000","last_undegraded":"2024-02-02T13:31:15.400249+0000","last_fullsized":"2024-02-02T13:31:15.400249+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.396037+0000","last_change":"2024-02-02T13:31:15.396037+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.396037+0000","last_undegraded":"2024-02-02T13:31:15.396037+0000","last_fullsized":"2024-02-02T13:31:15.396037+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398748+0000","last_change":"2024-02-02T13:31:15.398748+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398748+0000","last_undegraded":"2024-02-02T13:31:15.398748+0000","last_fullsized":"2024-02-02T13:31:15.398748+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:10.211668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.397968+0000","last_change":"2024-02-02T13:31:15.397968+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.397968+0000","last_undegraded":"2024-02-02T13:31:15.397968+0000","last_fullsized":"2024-02-02T13:31:15.397968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404841+0000","last_change":"2024-02-02T13:31:15.404841+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404841+0000","last_undegraded":"2024-02-02T13:31:15.404841+0000","last_fullsized":"2024-02-02T13:31:15.404841+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:58:45.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399149+0000","last_change":"2024-02-02T13:31:15.399149+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399149+0000","last_undegraded":"2024-02-02T13:31:15.399149+0000","last_fullsized":"2024-02-02T13:31:15.399149+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405057+0000","last_change":"2024-02-02T13:31:15.405057+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405057+0000","last_undegraded":"2024-02-02T13:31:15.405057+0000","last_fullsized":"2024-02-02T13:31:15.405057+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:57:01.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.402711+0000","last_change":"2024-02-02T13:31:15.402711+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.402711+0000","last_undegraded":"2024-02-02T13:31:15.402711+0000","last_fullsized":"2024-02-02T13:31:15.402711+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395212+0000","last_change":"2024-02-02T13:31:15.395212+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395212+0000","last_undegraded":"2024-02-02T13:31:15.395212+0000","last_fullsized":"2024-02-02T13:31:15.395212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:54.153974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.406222+0000","last_change":"2024-02-02T13:31:15.406222+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.406222+0000","last_undegraded":"2024-02-02T13:31:15.406222+0000","last_fullsized":"2024-02-02T13:31:15.406222+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":267,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-02-02T13:31:12.434245+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:31:12.434245+0000","last_peered":"2024-02-02T13:31:12.434245+0000","last_clean":"2024-02-02T13:31:12.434245+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:31:12.434245+0000","last_undegraded":"2024-02-02T13:31:12.434245+0000","last_fullsized":"2024-02-02T13:31:12.434245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194071,"statfs":{"total":200000000,"available":198728834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.0660000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.5149999999999999,"5min":1.6279999999999999,"15min":2.5310000000000001},"last":1.169},{"interface":"front","average":{"1min":1.083,"5min":1.105,"15min":1.1240000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4710000000000001,"5min":1.613,"15min":4.1609999999999996},"last":1.3240000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.0940000000000001},"min":{"1min":0.86299999999999999,"5min":0.66700000000000004,"15min":0.66300000000000003},"max":{"1min":1.4650000000000001,"5min":1.8839999999999999,"15min":1.893},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.595,"5min":1.758,"15min":2.5840000000000001},"last":1.4199999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.266,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.70099999999999996},"max":{"1min":1.7250000000000001,"5min":1.909,"15min":2.6669999999999998},"last":1.276},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.623},"max":{"1min":1.4359999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":1.4359999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.6339999999999999,"15min":1.819},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.752,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.825,"5min":2.133,"15min":2.4620000000000002},"last":1.4670000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.282,"15min":1.202},"min":{"1min":0.92500000000000004,"5min":0.60399999999999998,"15min":0.57599999999999996},"max":{"1min":1.6639999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.115},{"interface":"front","average":{"1min":1.333,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":0.88400000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.629,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.196}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.3520000000000001},"min":{"1min":0.95199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9239999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.397,"15min":1.4450000000000001},"min":{"1min":0.90900000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7190000000000001,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.258}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4850000000000001,"15min":1.5289999999999999},"min":{"1min":1.246,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.645,"5min":2.1869999999999998,"15min":3.6840000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.087,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.1439999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0960000000000001,"15min":1.024},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.403},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0820000000000001,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.3009999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0269999999999999},"min":{"1min":0.67600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.47,"5min":1.556,"15min":1.556},"last":1.171},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1659999999999999,"15min":1.0840000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.115}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1719999999999999,"15min":1.079},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.80300000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.2110000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.86299999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.89600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.093}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.4570000000000001,"15min":1.27},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":17.686,"5min":17.686,"15min":17.686},"last":1.3380000000000001},{"interface":"front","average":{"1min":2.1349999999999998,"5min":1.5900000000000001,"15min":1.4490000000000001},"min":{"1min":0.96899999999999997,"5min":0.89900000000000002,"15min":0.66000000000000003},"max":{"1min":17.388999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.1499999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.288,"15min":1.276},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":1.986,"15min":1.986},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4339999999999999,"15min":1.359},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.77400000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.2629999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.61199999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.101,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.74299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.294}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.26},"min":{"1min":0.76500000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.603,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.081,"15min":1.1419999999999999},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.337}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.2310000000000001},"min":{"1min":0.86399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.643,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1970000000000001,"15min":1.1779999999999999},"min":{"1min":0.79200000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.468,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.4350000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.202,"15min":1.1899999999999999},"min":{"1min":0.96799999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.409,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.1830000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6200000000000001,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.5349999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4570000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.85099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5569999999999999,"5min":1.833,"15min":1.833},"last":1.206}]},{"osd":6,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3380000000000001,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7010000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.351,"15min":1.345},"min":{"1min":0.98499999999999999,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.6220000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.0129999999999999,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":1.1279999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.75,"5min":1.851,"15min":1.851},"last":1.71}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.1080000000000001,"15min":0.997},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.6439999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4970000000000001,"5min":1.839,"15min":1.839},"last":1.3999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":0.995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1659999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.1180000000000001,"15min":1.04},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.097},"min":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9399999999999999,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1950000000000001,"15min":1.083},"min":{"1min":0.79100000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.077,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.04}]},{"osd":3,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.284,"15min":1.1859999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0470000000000002,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.339,"15min":1.2490000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1110000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2529999999999999,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6479999999999999,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3740000000000001,"15min":1.286},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4019999999999999,"15min":1.304},"min":{"1min":0.81499999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2869999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.466,"15min":1.395},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4750000000000001,"15min":1.389},"min":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.504,"15min":1.454},"min":{"1min":0.86099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4900000000000002,"5min":2.698,"15min":2.698},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.70999999999999996},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3480000000000001,"5min":1.619,"15min":2.6040000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94699999999999995,"15min":0.90100000000000002},"min":{"1min":0.497,"5min":0.46600000000000003,"15min":0.39700000000000002},"max":{"1min":1.4430000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.014,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.516,"5min":1.6359999999999999,"15min":2.125},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.034,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4079999999999999,"5min":1.611,"15min":1.611},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":0.89400000000000002},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.72699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.593,"5min":1.7110000000000001,"15min":3.585},"last":1.204},{"interface":"front","average":{"1min":1.165,"5min":1.2529999999999999,"15min":1.206},"min":{"1min":0.70499999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5249999999999999,"5min":1.6659999999999999,"15min":1.877},"last":1.234}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.202,"15min":1.218},"min":{"1min":0.64400000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3819999999999999,"5min":3.738,"15min":3.738},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.754,"5min":1.851,"15min":3.4950000000000001},"last":1.107}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.2689999999999999},"min":{"1min":0.81299999999999994,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.2989999999999999},"min":{"1min":0.80400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.462,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.3420000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2829999999999999,"15min":1.2629999999999999},"min":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6830000000000001,"5min":1.869,"15min":3.391},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3180000000000001},"min":{"1min":0.80900000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6379999999999999,"5min":1.9199999999999999,"15min":2.0409999999999999},"last":1.419}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4139999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.79900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.30299999999999999},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.91200000000000003},"min":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.441},"max":{"1min":2.153,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.139},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.002,"15min":0.874},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.5369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.81000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.9790000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.077,"15min":0.996},"min":{"1min":0.45500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.0189999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.296,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.748,"15min":0.748},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.26,"15min":1.1699999999999999},"min":{"1min":0.81899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.887,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2490000000000001,"15min":1.1040000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.056},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.327,"15min":1.28},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.421}]},{"osd":6,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.196},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3,"15min":1.212},"min":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.246}]},{"osd":7,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.367,"15min":1.2250000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.3779999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.98999999999999999},"min":{"1min":0.624,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.113,"5min":1.425,"15min":3.5680000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91600000000000004,"15min":1.333},"min":{"1min":0.54700000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.355,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.91000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.133,"5min":1.431,"15min":1.639},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":1.0820000000000001},"min":{"1min":0.59699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3999999999999999,"5min":1.823,"15min":3.0259999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.046,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.5549999999999999,"5min":1.647,"15min":3.1539999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.26},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.7030000000000001,"15min":4.8760000000000003},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1220000000000001,"15min":1.306},"min":{"1min":0.70499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4490000000000001,"5min":1.837,"15min":2.9449999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.3580000000000001},"min":{"1min":0.94399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.722,"5min":1.772,"15min":3.2970000000000002},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2929999999999999,"15min":1.448},"min":{"1min":1.085,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.589,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.278,"15min":1.4279999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5,"5min":1.8959999999999999,"15min":3.0249999999999999},"last":1.294}]},{"osd":6,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.282,"15min":1.5509999999999999},"min":{"1min":1.0069999999999999,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.304,"15min":1.4790000000000001},"min":{"1min":0.96899999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4039999999999999,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.2170000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3540000000000001,"15min":1.542},"min":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7889999999999999,"5min":2.1469999999999998,"15min":3.4689999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.363,"15min":1.6000000000000001},"min":{"1min":0.88700000000000001,"5min":0.86699999999999999,"15min":0.78000000000000003},"max":{"1min":1.6240000000000001,"5min":2.1019999999999999,"15min":4.3040000000000003},"last":1.3220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1246,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194066,"statfs":{"total":200000000,"available":198724036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77800000000000002,"15min":1.101},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2030000000000001,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89600000000000002,"15min":1.444},"min":{"1min":0.432,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.94999999999999996}]},{"osd":2,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92900000000000005,"15min":1.151},"min":{"1min":0.48999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4219999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92500000000000004,"15min":1.4390000000000001},"min":{"1min":0.47999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4079999999999999,"5min":1.706,"15min":10.781000000000001},"last":1.4279999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":1.2050000000000001},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1970000000000001,"5min":1.427,"15min":5.8410000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.292},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.413,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1719999999999999,"15min":1.3320000000000001},"min":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.669},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.397},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.895,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.375}]},{"osd":5,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.256,"15min":1.7829999999999999},"min":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.2350000000000001,"15min":1.413},"min":{"1min":0.92100000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6859999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.179}]},{"osd":6,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2969999999999999,"15min":1.4610000000000001},"min":{"1min":0.77000000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.2749999999999999,"15min":1.8160000000000001},"min":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":10.708},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3420000000000001,"15min":1.833},"min":{"1min":0.873,"5min":0.82499999999999996,"15min":0.70999999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":10.74},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2869999999999999,"15min":1.4379999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.869,"5min":1.869,"15min":3.863},"last":1.395}]}]}],"pool_statfs":[]}} 2024-02-02T13:31:21.780 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:31:22.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.025+0000 7fa0ab800640 1 Processor -- start 2024-02-02T13:31:22.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.025+0000 7fa0ab800640 1 -- start start 2024-02-02T13:31:22.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.026+0000 7fa0ab800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa0a4151f90 0x7fa0a415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0ab800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 0x7fa0a40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0ab800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a40b1070 con 0x7fa0a40b0c40 2024-02-02T13:31:22.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0ab800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a4070f80 con 0x7fa0a4151f90 2024-02-02T13:31:22.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0ab800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a4152370 con 0x7fa0a406f4f0 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa0a4151f90 0x7fa0a415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa0a4151f90 0x7fa0a415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49656/0 (socket says 172.21.15.91:49656) 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 -- 172.21.15.91:0/3580526824 learned_addr learned my addr 172.21.15.91:0/3580526824 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a3e00640 1 --2- 172.21.15.91:0/3580526824 >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 0x7fa0a40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 -- 172.21.15.91:0/3580526824 >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 msgr2=0x7fa0a40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 --2- 172.21.15.91:0/3580526824 >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 0x7fa0a40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 -- 172.21.15.91:0/3580526824 >> v1:172.21.15.113:6789/0 conn(0x7fa0a40b0c40 legacy=0x7fa0a40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a8e00640 1 -- 172.21.15.91:0/3580526824 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40cb010 con 0x7fa0a4151f90 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.027+0000 7fa0a3e00640 1 --2- 172.21.15.91:0/3580526824 >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 0x7fa0a40b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.028+0000 7fa0a8e00640 1 --2- 172.21.15.91:0/3580526824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a4151f90 0x7fa0a415f5a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4071a00 tx=0x7fa09402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf399fd263d918 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.029+0000 7fa0a3400640 1 -- 172.21.15.91:0/3580526824 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094031870 con 0x7fa0a4151f90 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.029+0000 7fa0a3400640 1 -- 172.21.15.91:0/3580526824 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0940319d0 con 0x7fa0a4151f90 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.029+0000 7fa0a3400640 1 -- 172.21.15.91:0/3580526824 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094031d60 con 0x7fa0a4151f90 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.029+0000 7fa0ab800640 1 -- 172.21.15.91:0/3580526824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a4151f90 msgr2=0x7fa0a415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.029+0000 7fa0ab800640 1 --2- 172.21.15.91:0/3580526824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a4151f90 0x7fa0a415f5a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4071a00 tx=0x7fa09402b910 comp rx=0 tx=0).stop 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 -- 172.21.15.91:0/3580526824 shutdown_connections 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 --2- 172.21.15.91:0/3580526824 >> v2:172.21.15.113:3300/0 conn(0x7fa0a406f4f0 0x7fa0a40b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 --2- 172.21.15.91:0/3580526824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a4151f90 0x7fa0a415f5a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 -- 172.21.15.91:0/3580526824 >> 172.21.15.91:0/3580526824 conn(0x7fa0a40cec80 msgr2=0x7fa0a40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 -- 172.21.15.91:0/3580526824 shutdown_connections 2024-02-02T13:31:22.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.030+0000 7fa0ab800640 1 -- 172.21.15.91:0/3580526824 wait complete. 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.031+0000 7fa0ab800640 1 Processor -- start 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 -- start start 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406f4f0 0x7fa0a415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0a4151f90 0x7fa0a4150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4154500 con 0x7fa0a40b0c40 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4153880 con 0x7fa0a406f4f0 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.032+0000 7fa0ab800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4153b60 con 0x7fa0a4151f90 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49666/0 (socket says 172.21.15.91:49666) 2024-02-02T13:31:22.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 -- 172.21.15.91:0/975081575 learned_addr learned my addr 172.21.15.91:0/975081575 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a8e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406f4f0 0x7fa0a415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a9800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0a4151f90 0x7fa0a4150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 -- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0a4151f90 msgr2=0x7fa0a4150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0a4151f90 0x7fa0a4150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 -- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406f4f0 msgr2=0x7fa0a415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406f4f0 0x7fa0a415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 -- 172.21.15.91:0/975081575 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940094d0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.033+0000 7fa0a3e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00c7f0 tx=0x7fa08c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.034+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c007820 con 0x7fa0a40b0c40 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.034+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c007980 con 0x7fa0a40b0c40 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.034+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c007cf0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.034+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a4153e40 con 0x7fa0a40b0c40 2024-02-02T13:31:22.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.034+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a415a8a0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.035+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a40b1ed0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.036+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa08c0113f0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.037+0000 7fa0a0c00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 0x7fa07c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.037+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa08c088bb0 con 0x7fa0a40b0c40 2024-02-02T13:31:22.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.037+0000 7fa0a8e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 0x7fa07c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.038+0000 7fa0a8e00640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 0x7fa07c06c8f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa094036000 tx=0x7fa09403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.042+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa08c054230 con 0x7fa0a40b0c40 2024-02-02T13:31:22.209 INFO:tasks.ceph.mds.a.smithi091.stderr:2024-02-02T13:31:22.207+0000 7fd1e3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:31:22.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.226+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0a4071200 con 0x7fa07c06a430 2024-02-02T13:31:22.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.235+0000 7fa0a0c00640 1 -- 172.21.15.91:0/975081575 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274195 (secure 0 0 0) 0x7fa094031810 con 0x7fa07c06a430 2024-02-02T13:31:22.241 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:22.242 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:31:22.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 msgr2=0x7fa07c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 0x7fa07c06c8f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa094036000 tx=0x7fa09403c040 comp rx=0 tx=0).stop 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 msgr2=0x7fa0a4152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00c7f0 tx=0x7fa08c00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 shutdown_connections 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa07c06a430 0x7fa07c06c8f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0a4151f90 0x7fa0a4150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa0a40b0c40 0x7fa0a4152ec0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 --2- 172.21.15.91:0/975081575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406f4f0 0x7fa0a415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 >> 172.21.15.91:0/975081575 conn(0x7fa0a40cec80 msgr2=0x7fa0a40adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:22.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 shutdown_connections 2024-02-02T13:31:22.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.240+0000 7fa0ab800640 1 -- 172.21.15.91:0/975081575 wait complete. 2024-02-02T13:31:22.260 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-02-02T13:31:20.368895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555992,"statfs":{"total":1600000000,"available":1593336237,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.974478"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.401578+0000","last_change":"2024-02-02T13:31:15.401578+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.401578+0000","last_undegraded":"2024-02-02T13:31:15.401578+0000","last_fullsized":"2024-02-02T13:31:15.401578+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:16:12.721762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.407263+0000","last_change":"2024-02-02T13:31:15.407263+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.407263+0000","last_undegraded":"2024-02-02T13:31:15.407263+0000","last_fullsized":"2024-02-02T13:31:15.407263+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399421+0000","last_change":"2024-02-02T13:31:15.399421+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399421+0000","last_undegraded":"2024-02-02T13:31:15.399421+0000","last_fullsized":"2024-02-02T13:31:15.399421+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398278+0000","last_change":"2024-02-02T13:31:15.398278+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398278+0000","last_undegraded":"2024-02-02T13:31:15.398278+0000","last_fullsized":"2024-02-02T13:31:15.398278+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405447+0000","last_change":"2024-02-02T13:31:15.405447+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405447+0000","last_undegraded":"2024-02-02T13:31:15.405447+0000","last_fullsized":"2024-02-02T13:31:15.405447+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:54.011838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.400737+0000","last_change":"2024-02-02T13:31:15.400737+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.400737+0000","last_undegraded":"2024-02-02T13:31:15.400737+0000","last_fullsized":"2024-02-02T13:31:15.400737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404639+0000","last_change":"2024-02-02T13:31:15.404639+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404639+0000","last_undegraded":"2024-02-02T13:31:15.404639+0000","last_fullsized":"2024-02-02T13:31:15.404639+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:50:14.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404966+0000","last_change":"2024-02-02T13:31:15.404966+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404966+0000","last_undegraded":"2024-02-02T13:31:15.404966+0000","last_fullsized":"2024-02-02T13:31:15.404966+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:06.887358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404648+0000","last_change":"2024-02-02T13:31:15.404648+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404648+0000","last_undegraded":"2024-02-02T13:31:15.404648+0000","last_fullsized":"2024-02-02T13:31:15.404648+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:46.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404251+0000","last_change":"2024-02-02T13:31:15.404251+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404251+0000","last_undegraded":"2024-02-02T13:31:15.404251+0000","last_fullsized":"2024-02-02T13:31:15.404251+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.406293+0000","last_change":"2024-02-02T13:31:15.406293+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.406293+0000","last_undegraded":"2024-02-02T13:31:15.406293+0000","last_fullsized":"2024-02-02T13:31:15.406293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403212+0000","last_change":"2024-02-02T13:31:15.403212+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403212+0000","last_undegraded":"2024-02-02T13:31:15.403212+0000","last_fullsized":"2024-02-02T13:31:15.403212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:46.255117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.396179+0000","last_change":"2024-02-02T13:31:15.396179+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.396179+0000","last_undegraded":"2024-02-02T13:31:15.396179+0000","last_fullsized":"2024-02-02T13:31:15.396179+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:47.961252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395214+0000","last_change":"2024-02-02T13:31:15.395214+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395214+0000","last_undegraded":"2024-02-02T13:31:15.395214+0000","last_fullsized":"2024-02-02T13:31:15.395214+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:55.831956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403217+0000","last_change":"2024-02-02T13:31:15.403217+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403217+0000","last_undegraded":"2024-02-02T13:31:15.403217+0000","last_fullsized":"2024-02-02T13:31:15.403217+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403500+0000","last_change":"2024-02-02T13:31:15.403500+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403500+0000","last_undegraded":"2024-02-02T13:31:15.403500+0000","last_fullsized":"2024-02-02T13:31:15.403500+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:57.874029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.403199+0000","last_change":"2024-02-02T13:31:15.403199+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.403199+0000","last_undegraded":"2024-02-02T13:31:15.403199+0000","last_fullsized":"2024-02-02T13:31:15.403199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405066+0000","last_change":"2024-02-02T13:31:15.405066+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405066+0000","last_undegraded":"2024-02-02T13:31:15.405066+0000","last_fullsized":"2024-02-02T13:31:15.405066+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398582+0000","last_change":"2024-02-02T13:31:15.398582+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398582+0000","last_undegraded":"2024-02-02T13:31:15.398582+0000","last_fullsized":"2024-02-02T13:31:15.398582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:56.711812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399667+0000","last_change":"2024-02-02T13:31:15.399667+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399667+0000","last_undegraded":"2024-02-02T13:31:15.399667+0000","last_fullsized":"2024-02-02T13:31:15.399667+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:21.074291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395587+0000","last_change":"2024-02-02T13:31:15.395587+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395587+0000","last_undegraded":"2024-02-02T13:31:15.395587+0000","last_fullsized":"2024-02-02T13:31:15.395587+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:56.934997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395978+0000","last_change":"2024-02-02T13:31:15.395978+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395978+0000","last_undegraded":"2024-02-02T13:31:15.395978+0000","last_fullsized":"2024-02-02T13:31:15.395978+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.400249+0000","last_change":"2024-02-02T13:31:15.400249+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.400249+0000","last_undegraded":"2024-02-02T13:31:15.400249+0000","last_fullsized":"2024-02-02T13:31:15.400249+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.396037+0000","last_change":"2024-02-02T13:31:15.396037+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.396037+0000","last_undegraded":"2024-02-02T13:31:15.396037+0000","last_fullsized":"2024-02-02T13:31:15.396037+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.398748+0000","last_change":"2024-02-02T13:31:15.398748+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.398748+0000","last_undegraded":"2024-02-02T13:31:15.398748+0000","last_fullsized":"2024-02-02T13:31:15.398748+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:10.211668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.397968+0000","last_change":"2024-02-02T13:31:15.397968+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.397968+0000","last_undegraded":"2024-02-02T13:31:15.397968+0000","last_fullsized":"2024-02-02T13:31:15.397968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.404841+0000","last_change":"2024-02-02T13:31:15.404841+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.404841+0000","last_undegraded":"2024-02-02T13:31:15.404841+0000","last_fullsized":"2024-02-02T13:31:15.404841+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:58:45.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.399149+0000","last_change":"2024-02-02T13:31:15.399149+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.399149+0000","last_undegraded":"2024-02-02T13:31:15.399149+0000","last_fullsized":"2024-02-02T13:31:15.399149+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.405057+0000","last_change":"2024-02-02T13:31:15.405057+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.405057+0000","last_undegraded":"2024-02-02T13:31:15.405057+0000","last_fullsized":"2024-02-02T13:31:15.405057+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:57:01.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.402711+0000","last_change":"2024-02-02T13:31:15.402711+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.402711+0000","last_undegraded":"2024-02-02T13:31:15.402711+0000","last_fullsized":"2024-02-02T13:31:15.402711+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.395212+0000","last_change":"2024-02-02T13:31:15.395212+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.395212+0000","last_undegraded":"2024-02-02T13:31:15.395212+0000","last_fullsized":"2024-02-02T13:31:15.395212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:54.153974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-02T13:31:16.392456+0000","last_change":"2024-02-02T13:31:16.392456+0000","last_active":"2024-02-02T13:31:16.392456+0000","last_peered":"2024-02-02T13:31:16.392456+0000","last_clean":"2024-02-02T13:31:16.392456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:16.392456+0000","last_undegraded":"2024-02-02T13:31:16.392456+0000","last_fullsized":"2024-02-02T13:31:16.392456+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":125,"state":"creating+peering","last_fresh":"2024-02-02T13:31:15.406222+0000","last_change":"2024-02-02T13:31:15.406222+0000","last_active":"2024-02-02T13:31:15.386838+0000","last_peered":"2024-02-02T13:31:15.386838+0000","last_clean":"2024-02-02T13:31:15.386838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-02T13:31:15.406222+0000","last_undegraded":"2024-02-02T13:31:15.406222+0000","last_fullsized":"2024-02-02T13:31:15.406222+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":267,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-02-02T13:31:12.434245+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:31:12.434245+0000","last_peered":"2024-02-02T13:31:12.434245+0000","last_clean":"2024-02-02T13:31:12.434245+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:31:12.434245+0000","last_undegraded":"2024-02-02T13:31:12.434245+0000","last_fullsized":"2024-02-02T13:31:12.434245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194071,"statfs":{"total":200000000,"available":198728834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.0660000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.5149999999999999,"5min":1.6279999999999999,"15min":2.5310000000000001},"last":1.169},{"interface":"front","average":{"1min":1.083,"5min":1.105,"15min":1.1240000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4710000000000001,"5min":1.613,"15min":4.1609999999999996},"last":1.3240000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.0940000000000001},"min":{"1min":0.86299999999999999,"5min":0.66700000000000004,"15min":0.66300000000000003},"max":{"1min":1.4650000000000001,"5min":1.8839999999999999,"15min":1.893},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.595,"5min":1.758,"15min":2.5840000000000001},"last":1.4199999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.266,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.70099999999999996},"max":{"1min":1.7250000000000001,"5min":1.909,"15min":2.6669999999999998},"last":1.276},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.623},"max":{"1min":1.4359999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":1.4359999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.6339999999999999,"15min":1.819},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.752,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.825,"5min":2.133,"15min":2.4620000000000002},"last":1.4670000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.282,"15min":1.202},"min":{"1min":0.92500000000000004,"5min":0.60399999999999998,"15min":0.57599999999999996},"max":{"1min":1.6639999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.115},{"interface":"front","average":{"1min":1.333,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":0.88400000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.629,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.196}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.3520000000000001},"min":{"1min":0.95199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9239999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.397,"15min":1.4450000000000001},"min":{"1min":0.90900000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7190000000000001,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.258}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4850000000000001,"15min":1.5289999999999999},"min":{"1min":1.246,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.645,"5min":2.1869999999999998,"15min":3.6840000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.087,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.1439999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0960000000000001,"15min":1.024},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.403},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0820000000000001,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.3009999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0269999999999999},"min":{"1min":0.67600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.47,"5min":1.556,"15min":1.556},"last":1.171},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1659999999999999,"15min":1.0840000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.115}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1719999999999999,"15min":1.079},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.80300000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.2110000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.86299999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.89600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.093}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.4570000000000001,"15min":1.27},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":17.686,"5min":17.686,"15min":17.686},"last":1.3380000000000001},{"interface":"front","average":{"1min":2.1349999999999998,"5min":1.5900000000000001,"15min":1.4490000000000001},"min":{"1min":0.96899999999999997,"5min":0.89900000000000002,"15min":0.66000000000000003},"max":{"1min":17.388999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.1499999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.288,"15min":1.276},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":1.986,"15min":1.986},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4339999999999999,"15min":1.359},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.77400000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.2629999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.61199999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.101,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.74299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.294}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.26},"min":{"1min":0.76500000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.603,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.081,"15min":1.1419999999999999},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.337}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.2310000000000001},"min":{"1min":0.86399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.643,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1970000000000001,"15min":1.1779999999999999},"min":{"1min":0.79200000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.468,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.4350000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.202,"15min":1.1899999999999999},"min":{"1min":0.96799999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.409,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.1830000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6200000000000001,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.5349999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4570000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.85099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5569999999999999,"5min":1.833,"15min":1.833},"last":1.206}]},{"osd":6,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3380000000000001,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7010000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.351,"15min":1.345},"min":{"1min":0.98499999999999999,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.6220000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.0129999999999999,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":1.1279999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.75,"5min":1.851,"15min":1.851},"last":1.71}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.1080000000000001,"15min":0.997},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.6439999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4970000000000001,"5min":1.839,"15min":1.839},"last":1.3999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":0.995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1659999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.1180000000000001,"15min":1.04},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.097},"min":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9399999999999999,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1950000000000001,"15min":1.083},"min":{"1min":0.79100000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.077,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.04}]},{"osd":3,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.284,"15min":1.1859999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0470000000000002,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.339,"15min":1.2490000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1110000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2529999999999999,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6479999999999999,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3740000000000001,"15min":1.286},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4019999999999999,"15min":1.304},"min":{"1min":0.81499999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2869999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.466,"15min":1.395},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4750000000000001,"15min":1.389},"min":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.504,"15min":1.454},"min":{"1min":0.86099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4900000000000002,"5min":2.698,"15min":2.698},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.70999999999999996},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3480000000000001,"5min":1.619,"15min":2.6040000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94699999999999995,"15min":0.90100000000000002},"min":{"1min":0.497,"5min":0.46600000000000003,"15min":0.39700000000000002},"max":{"1min":1.4430000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.014,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.516,"5min":1.6359999999999999,"15min":2.125},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.034,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4079999999999999,"5min":1.611,"15min":1.611},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":0.89400000000000002},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.72699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.593,"5min":1.7110000000000001,"15min":3.585},"last":1.204},{"interface":"front","average":{"1min":1.165,"5min":1.2529999999999999,"15min":1.206},"min":{"1min":0.70499999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5249999999999999,"5min":1.6659999999999999,"15min":1.877},"last":1.234}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.202,"15min":1.218},"min":{"1min":0.64400000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3819999999999999,"5min":3.738,"15min":3.738},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.754,"5min":1.851,"15min":3.4950000000000001},"last":1.107}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.2689999999999999},"min":{"1min":0.81299999999999994,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.2989999999999999},"min":{"1min":0.80400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.462,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.3420000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2829999999999999,"15min":1.2629999999999999},"min":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6830000000000001,"5min":1.869,"15min":3.391},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3180000000000001},"min":{"1min":0.80900000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6379999999999999,"5min":1.9199999999999999,"15min":2.0409999999999999},"last":1.419}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":670,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194641,"statfs":{"total":200000000,"available":199313385,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4139999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.79900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.30299999999999999},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.91200000000000003},"min":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.441},"max":{"1min":2.153,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.139},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.002,"15min":0.874},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.5369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.81000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.9790000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.077,"15min":0.996},"min":{"1min":0.45500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.0189999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.296,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.748,"15min":0.748},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.26,"15min":1.1699999999999999},"min":{"1min":0.81899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.887,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2490000000000001,"15min":1.1040000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.056},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.327,"15min":1.28},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.421}]},{"osd":6,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.196},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3,"15min":1.212},"min":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.246}]},{"osd":7,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.367,"15min":1.2250000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.3779999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.98999999999999999},"min":{"1min":0.624,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.113,"5min":1.425,"15min":3.5680000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91600000000000004,"15min":1.333},"min":{"1min":0.54700000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.355,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.91000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.133,"5min":1.431,"15min":1.639},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":1.0820000000000001},"min":{"1min":0.59699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3999999999999999,"5min":1.823,"15min":3.0259999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.046,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.5549999999999999,"5min":1.647,"15min":3.1539999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.26},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.7030000000000001,"15min":4.8760000000000003},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1220000000000001,"15min":1.306},"min":{"1min":0.70499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4490000000000001,"5min":1.837,"15min":2.9449999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.3580000000000001},"min":{"1min":0.94399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.722,"5min":1.772,"15min":3.2970000000000002},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2929999999999999,"15min":1.448},"min":{"1min":1.085,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.589,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.278,"15min":1.4279999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5,"5min":1.8959999999999999,"15min":3.0249999999999999},"last":1.294}]},{"osd":6,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.282,"15min":1.5509999999999999},"min":{"1min":1.0069999999999999,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.304,"15min":1.4790000000000001},"min":{"1min":0.96899999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4039999999999999,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.2170000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3540000000000001,"15min":1.542},"min":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7889999999999999,"5min":2.1469999999999998,"15min":3.4689999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.363,"15min":1.6000000000000001},"min":{"1min":0.88700000000000001,"5min":0.86699999999999999,"15min":0.78000000000000003},"max":{"1min":1.6240000000000001,"5min":2.1019999999999999,"15min":4.3040000000000003},"last":1.3220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1246,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194066,"statfs":{"total":200000000,"available":198724036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77800000000000002,"15min":1.101},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2030000000000001,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89600000000000002,"15min":1.444},"min":{"1min":0.432,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.94999999999999996}]},{"osd":2,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92900000000000005,"15min":1.151},"min":{"1min":0.48999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4219999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92500000000000004,"15min":1.4390000000000001},"min":{"1min":0.47999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4079999999999999,"5min":1.706,"15min":10.781000000000001},"last":1.4279999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":1.2050000000000001},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1970000000000001,"5min":1.427,"15min":5.8410000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.292},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.413,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1719999999999999,"15min":1.3320000000000001},"min":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.669},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.397},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.895,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.375}]},{"osd":5,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.256,"15min":1.7829999999999999},"min":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.2350000000000001,"15min":1.413},"min":{"1min":0.92100000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6859999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.179}]},{"osd":6,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2969999999999999,"15min":1.4610000000000001},"min":{"1min":0.77000000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.2749999999999999,"15min":1.8160000000000001},"min":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":10.708},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3420000000000001,"15min":1.833},"min":{"1min":0.873,"5min":0.82499999999999996,"15min":0.70999999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":10.74},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2869999999999999,"15min":1.4379999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.869,"5min":1.869,"15min":3.863},"last":1.395}]}]}],"pool_statfs":[]}} 2024-02-02T13:31:22.265 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:31:22.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.513+0000 7f8aa6000640 1 Processor -- start 2024-02-02T13:31:22.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.513+0000 7f8aa6000640 1 -- start start 2024-02-02T13:31:22.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8aa6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8aa00acfe0 0x7f8aa006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 0x7f8aa00b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8aa6000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa015d870 con 0x7f8aa015d490 2024-02-02T13:31:22.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8aa6000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa006bce0 con 0x7f8aa00acfe0 2024-02-02T13:31:22.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8aa6000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0161220 con 0x7f8aa00b0700 2024-02-02T13:31:22.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8a9f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8aa015d490 0x7f8aa00ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44056/0 (socket says 172.21.15.91:44056) 2024-02-02T13:31:22.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.515+0000 7f8a9f400640 1 -- 172.21.15.91:0/1036289347 learned_addr learned my addr 172.21.15.91:0/1036289347 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a96800640 1 --2- 172.21.15.91:0/1036289347 >> v2:172.21.15.113:3300/0 conn(0x7f8aa00acfe0 0x7f8aa006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9fe00640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 0x7f8aa00b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525497584 0 0) 0x7f8aa015d870 con 0x7f8aa015d490 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a80003520 con 0x7f8aa015d490 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281293676 0 0) 0x7f8a80003520 con 0x7f8aa015d490 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 msgr2=0x7f8aa00b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 0x7f8aa00b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 >> v2:172.21.15.113:3300/0 conn(0x7f8aa00acfe0 msgr2=0x7f8aa006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 --2- 172.21.15.91:0/1036289347 >> v2:172.21.15.113:3300/0 conn(0x7f8aa00acfe0 0x7f8aa006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00b1070 con 0x7f8aa015d490 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8a90001670 con 0x7f8aa015d490 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 >> v1:172.21.15.91:6789/0 conn(0x7f8aa015d490 legacy=0x7f8aa00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a80003970 0x7f8a80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8a80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 0x7f8a80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa006bce0 con 0x7f8a800047c0 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0161220 con 0x7f8a80003970 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.516+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8000b980 con 0x7f8aa00b0700 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8a80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 -- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 msgr2=0x7f8a80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b0700 0x7f8a80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a9f400640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a80003970 0x7f8a80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 -- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a80003970 msgr2=0x7f8a80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a80003970 0x7f8a80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a96800640 1 -- 172.21.15.91:0/1036289347 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80003520 con 0x7f8a800047c0 2024-02-02T13:31:22.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.517+0000 7f8a9f400640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a80003970 0x7f8a80003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:22.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.518+0000 7f8a96800640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8a80004c40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009ac0 tx=0x7f8a800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436c35a15cba2f4b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.518+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8802c710 con 0x7f8a800047c0 2024-02-02T13:31:22.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.518+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8802c870 con 0x7f8a800047c0 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.518+0000 7f8a9ea00640 1 -- 172.21.15.91:0/1036289347 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8802cc00 con 0x7f8a800047c0 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 -- 172.21.15.91:0/1036289347 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 msgr2=0x7f8a80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 --2- 172.21.15.91:0/1036289347 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8a80004c40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009ac0 tx=0x7f8a800057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8a9fe00640 1 -- 172.21.15.91:0/1036289347 reap_dead start 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 -- 172.21.15.91:0/1036289347 shutdown_connections 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 -- 172.21.15.91:0/1036289347 >> 172.21.15.91:0/1036289347 conn(0x7f8aa00cec80 msgr2=0x7f8aa00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 -- 172.21.15.91:0/1036289347 shutdown_connections 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.519+0000 7f8aa6000640 1 -- 172.21.15.91:0/1036289347 wait complete. 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.521+0000 7f8aa6000640 1 Processor -- start 2024-02-02T13:31:22.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.521+0000 7f8aa6000640 1 -- start start 2024-02-02T13:31:22.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8aa015e8c0 0x7f8aa015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b2860 0x7f8aa00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015f280 con 0x7f8a800047c0 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00cb540 con 0x7f8aa015e8c0 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8aa6000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00cb820 con 0x7f8aa00b2860 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8a9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8a9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49686/0 (socket says 172.21.15.91:49686) 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.522+0000 7f8a9f400640 1 -- 172.21.15.91:0/782215321 learned_addr learned my addr 172.21.15.91:0/782215321 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 -- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b2860 msgr2=0x7f8aa00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b2860 0x7f8aa00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 -- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8aa015e8c0 msgr2=0x7f8aa015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8aa015e8c0 0x7f8aa015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 -- 172.21.15.91:0/782215321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00cbb00 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a9f400640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000d860 tx=0x7f8a9000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900074e0 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90007640 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa00cbde0 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900049e0 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.523+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa015be40 con 0x7f8a800047c0 2024-02-02T13:31:22.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.525+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa006fc00 con 0x7f8a800047c0 2024-02-02T13:31:22.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.526+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8a90004b40 con 0x7f8a800047c0 2024-02-02T13:31:22.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.527+0000 7f8a97e00640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 0x7f8a7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:22.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.527+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a90088760 con 0x7f8a800047c0 2024-02-02T13:31:22.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.527+0000 7f8a96800640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 0x7f8a7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:22.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.528+0000 7f8a96800640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 0x7f8a7806c8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8a88036000 tx=0x7f8a8802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:22.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.531+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8a90053d40 con 0x7f8a800047c0 2024-02-02T13:31:22.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.720+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8aa00ad490 con 0x7f8a7806a430 2024-02-02T13:31:22.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.729+0000 7f8a97e00640 1 -- 172.21.15.91:0/782215321 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283307 (secure 0 0 0) 0x7f8a8802da70 con 0x7f8a7806a430 2024-02-02T13:31:22.735 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:22.736 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 msgr2=0x7f8a7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 0x7f8a7806c8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8a88036000 tx=0x7f8a8802e000 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 msgr2=0x7f8aa015b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000d860 tx=0x7f8a9000dd20 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 shutdown_connections 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8a7806a430 0x7f8a7806c8f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8aa00b2860 0x7f8aa00b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8aa015e8c0 0x7f8aa015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 --2- 172.21.15.91:0/782215321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a800047c0 0x7f8aa015b900 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 >> 172.21.15.91:0/782215321 conn(0x7f8aa00cec80 msgr2=0x7f8aa006f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 shutdown_connections 2024-02-02T13:31:22.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:22.736+0000 7f8aa6000640 1 -- 172.21.15.91:0/782215321 wait complete. 2024-02-02T13:31:22.758 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-02-02T13:31:22.369496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555830,"statfs":{"total":1600000000,"available":1593170896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001849"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902784+0000","last_change":"2024-02-02T13:31:16.918834+0000","last_active":"2024-02-02T13:31:17.902784+0000","last_peered":"2024-02-02T13:31:17.902784+0000","last_clean":"2024-02-02T13:31:17.902784+0000","last_became_active":"2024-02-02T13:31:16.918441+0000","last_became_peered":"2024-02-02T13:31:16.918441+0000","last_unstale":"2024-02-02T13:31:17.902784+0000","last_undegraded":"2024-02-02T13:31:17.902784+0000","last_fullsized":"2024-02-02T13:31:17.902784+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:49:00.816989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906309+0000","last_change":"2024-02-02T13:31:16.927818+0000","last_active":"2024-02-02T13:31:17.906309+0000","last_peered":"2024-02-02T13:31:17.906309+0000","last_clean":"2024-02-02T13:31:17.906309+0000","last_became_active":"2024-02-02T13:31:16.927553+0000","last_became_peered":"2024-02-02T13:31:16.927553+0000","last_unstale":"2024-02-02T13:31:17.906309+0000","last_undegraded":"2024-02-02T13:31:17.906309+0000","last_fullsized":"2024-02-02T13:31:17.906309+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902583+0000","last_change":"2024-02-02T13:31:16.925847+0000","last_active":"2024-02-02T13:31:17.902583+0000","last_peered":"2024-02-02T13:31:17.902583+0000","last_clean":"2024-02-02T13:31:17.902583+0000","last_became_active":"2024-02-02T13:31:16.925615+0000","last_became_peered":"2024-02-02T13:31:16.925615+0000","last_unstale":"2024-02-02T13:31:17.902583+0000","last_undegraded":"2024-02-02T13:31:17.902583+0000","last_fullsized":"2024-02-02T13:31:17.902583+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:42:48.173630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860390+0000","last_change":"2024-02-02T13:31:16.907328+0000","last_active":"2024-02-02T13:31:18.860390+0000","last_peered":"2024-02-02T13:31:18.860390+0000","last_clean":"2024-02-02T13:31:18.860390+0000","last_became_active":"2024-02-02T13:31:16.907025+0000","last_became_peered":"2024-02-02T13:31:16.907025+0000","last_unstale":"2024-02-02T13:31:18.860390+0000","last_undegraded":"2024-02-02T13:31:18.860390+0000","last_fullsized":"2024-02-02T13:31:18.860390+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411041+0000","last_change":"2024-02-02T13:31:16.911577+0000","last_active":"2024-02-02T13:31:18.411041+0000","last_peered":"2024-02-02T13:31:18.411041+0000","last_clean":"2024-02-02T13:31:18.411041+0000","last_became_active":"2024-02-02T13:31:16.911051+0000","last_became_peered":"2024-02-02T13:31:16.911051+0000","last_unstale":"2024-02-02T13:31:18.411041+0000","last_undegraded":"2024-02-02T13:31:18.411041+0000","last_fullsized":"2024-02-02T13:31:18.411041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678686+0000","last_change":"2024-02-02T13:31:16.933309+0000","last_active":"2024-02-02T13:31:18.678686+0000","last_peered":"2024-02-02T13:31:18.678686+0000","last_clean":"2024-02-02T13:31:18.678686+0000","last_became_active":"2024-02-02T13:31:16.932957+0000","last_became_peered":"2024-02-02T13:31:16.932957+0000","last_unstale":"2024-02-02T13:31:18.678686+0000","last_undegraded":"2024-02-02T13:31:18.678686+0000","last_fullsized":"2024-02-02T13:31:18.678686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678369+0000","last_change":"2024-02-02T13:31:16.929320+0000","last_active":"2024-02-02T13:31:18.678369+0000","last_peered":"2024-02-02T13:31:18.678369+0000","last_clean":"2024-02-02T13:31:18.678369+0000","last_became_active":"2024-02-02T13:31:16.929069+0000","last_became_peered":"2024-02-02T13:31:16.929069+0000","last_unstale":"2024-02-02T13:31:18.678369+0000","last_undegraded":"2024-02-02T13:31:18.678369+0000","last_fullsized":"2024-02-02T13:31:18.678369+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907186+0000","last_change":"2024-02-02T13:31:16.926583+0000","last_active":"2024-02-02T13:31:17.907186+0000","last_peered":"2024-02-02T13:31:17.907186+0000","last_clean":"2024-02-02T13:31:17.907186+0000","last_became_active":"2024-02-02T13:31:16.926431+0000","last_became_peered":"2024-02-02T13:31:16.926431+0000","last_unstale":"2024-02-02T13:31:17.907186+0000","last_undegraded":"2024-02-02T13:31:17.907186+0000","last_fullsized":"2024-02-02T13:31:17.907186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380086+0000","last_change":"2024-02-02T13:31:16.920257+0000","last_active":"2024-02-02T13:31:20.380086+0000","last_peered":"2024-02-02T13:31:20.380086+0000","last_clean":"2024-02-02T13:31:20.380086+0000","last_became_active":"2024-02-02T13:31:16.919808+0000","last_became_peered":"2024-02-02T13:31:16.919808+0000","last_unstale":"2024-02-02T13:31:20.380086+0000","last_undegraded":"2024-02-02T13:31:20.380086+0000","last_fullsized":"2024-02-02T13:31:20.380086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:20.515102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411221+0000","last_change":"2024-02-02T13:31:16.918453+0000","last_active":"2024-02-02T13:31:18.411221+0000","last_peered":"2024-02-02T13:31:18.411221+0000","last_clean":"2024-02-02T13:31:18.411221+0000","last_became_active":"2024-02-02T13:31:16.917549+0000","last_became_peered":"2024-02-02T13:31:16.917549+0000","last_unstale":"2024-02-02T13:31:18.411221+0000","last_undegraded":"2024-02-02T13:31:18.411221+0000","last_fullsized":"2024-02-02T13:31:18.411221+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678917+0000","last_change":"2024-02-02T13:31:16.925898+0000","last_active":"2024-02-02T13:31:18.678917+0000","last_peered":"2024-02-02T13:31:18.678917+0000","last_clean":"2024-02-02T13:31:18.678917+0000","last_became_active":"2024-02-02T13:31:16.925597+0000","last_became_peered":"2024-02-02T13:31:16.925597+0000","last_unstale":"2024-02-02T13:31:18.678917+0000","last_undegraded":"2024-02-02T13:31:18.678917+0000","last_fullsized":"2024-02-02T13:31:18.678917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380719+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:20.380719+0000","last_peered":"2024-02-02T13:31:20.380719+0000","last_clean":"2024-02-02T13:31:20.380719+0000","last_became_active":"2024-02-02T13:31:16.923735+0000","last_became_peered":"2024-02-02T13:31:16.923735+0000","last_unstale":"2024-02-02T13:31:20.380719+0000","last_undegraded":"2024-02-02T13:31:20.380719+0000","last_fullsized":"2024-02-02T13:31:20.380719+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:17.618621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.859985+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:18.859985+0000","last_peered":"2024-02-02T13:31:18.859985+0000","last_clean":"2024-02-02T13:31:18.859985+0000","last_became_active":"2024-02-02T13:31:16.924861+0000","last_became_peered":"2024-02-02T13:31:16.924861+0000","last_unstale":"2024-02-02T13:31:18.859985+0000","last_undegraded":"2024-02-02T13:31:18.859985+0000","last_fullsized":"2024-02-02T13:31:18.859985+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906357+0000","last_change":"2024-02-02T13:31:16.914250+0000","last_active":"2024-02-02T13:31:17.906357+0000","last_peered":"2024-02-02T13:31:17.906357+0000","last_clean":"2024-02-02T13:31:17.906357+0000","last_became_active":"2024-02-02T13:31:16.914003+0000","last_became_peered":"2024-02-02T13:31:16.914003+0000","last_unstale":"2024-02-02T13:31:17.906357+0000","last_undegraded":"2024-02-02T13:31:17.906357+0000","last_fullsized":"2024-02-02T13:31:17.906357+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:03:38.355640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860441+0000","last_change":"2024-02-02T13:31:16.906982+0000","last_active":"2024-02-02T13:31:18.860441+0000","last_peered":"2024-02-02T13:31:18.860441+0000","last_clean":"2024-02-02T13:31:18.860441+0000","last_became_active":"2024-02-02T13:31:16.906771+0000","last_became_peered":"2024-02-02T13:31:16.906771+0000","last_unstale":"2024-02-02T13:31:18.860441+0000","last_undegraded":"2024-02-02T13:31:18.860441+0000","last_fullsized":"2024-02-02T13:31:18.860441+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679128+0000","last_change":"2024-02-02T13:31:16.929883+0000","last_active":"2024-02-02T13:31:18.679128+0000","last_peered":"2024-02-02T13:31:18.679128+0000","last_clean":"2024-02-02T13:31:18.679128+0000","last_became_active":"2024-02-02T13:31:16.929203+0000","last_became_peered":"2024-02-02T13:31:16.929203+0000","last_unstale":"2024-02-02T13:31:18.679128+0000","last_undegraded":"2024-02-02T13:31:18.679128+0000","last_fullsized":"2024-02-02T13:31:18.679128+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906561+0000","last_change":"2024-02-02T13:31:16.920007+0000","last_active":"2024-02-02T13:31:17.906561+0000","last_peered":"2024-02-02T13:31:17.906561+0000","last_clean":"2024-02-02T13:31:17.906561+0000","last_became_active":"2024-02-02T13:31:16.919625+0000","last_became_peered":"2024-02-02T13:31:16.919625+0000","last_unstale":"2024-02-02T13:31:17.906561+0000","last_undegraded":"2024-02-02T13:31:17.906561+0000","last_fullsized":"2024-02-02T13:31:17.906561+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:19:35.465397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860275+0000","last_change":"2024-02-02T13:31:16.910385+0000","last_active":"2024-02-02T13:31:18.860275+0000","last_peered":"2024-02-02T13:31:18.860275+0000","last_clean":"2024-02-02T13:31:18.860275+0000","last_became_active":"2024-02-02T13:31:16.909793+0000","last_became_peered":"2024-02-02T13:31:16.909793+0000","last_unstale":"2024-02-02T13:31:18.860275+0000","last_undegraded":"2024-02-02T13:31:18.860275+0000","last_fullsized":"2024-02-02T13:31:18.860275+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906609+0000","last_change":"2024-02-02T13:31:16.909676+0000","last_active":"2024-02-02T13:31:17.906609+0000","last_peered":"2024-02-02T13:31:17.906609+0000","last_clean":"2024-02-02T13:31:17.906609+0000","last_became_active":"2024-02-02T13:31:16.909474+0000","last_became_peered":"2024-02-02T13:31:16.909474+0000","last_unstale":"2024-02-02T13:31:17.906609+0000","last_undegraded":"2024-02-02T13:31:17.906609+0000","last_fullsized":"2024-02-02T13:31:17.906609+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:58.879647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380271+0000","last_change":"2024-02-02T13:31:16.921784+0000","last_active":"2024-02-02T13:31:20.380271+0000","last_peered":"2024-02-02T13:31:20.380271+0000","last_clean":"2024-02-02T13:31:20.380271+0000","last_became_active":"2024-02-02T13:31:16.920919+0000","last_became_peered":"2024-02-02T13:31:16.920919+0000","last_unstale":"2024-02-02T13:31:20.380271+0000","last_undegraded":"2024-02-02T13:31:20.380271+0000","last_fullsized":"2024-02-02T13:31:20.380271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:31.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906361+0000","last_change":"2024-02-02T13:31:16.917770+0000","last_active":"2024-02-02T13:31:17.906361+0000","last_peered":"2024-02-02T13:31:17.906361+0000","last_clean":"2024-02-02T13:31:17.906361+0000","last_became_active":"2024-02-02T13:31:16.917542+0000","last_became_peered":"2024-02-02T13:31:16.917542+0000","last_unstale":"2024-02-02T13:31:17.906361+0000","last_undegraded":"2024-02-02T13:31:17.906361+0000","last_fullsized":"2024-02-02T13:31:17.906361+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.721834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906902+0000","last_change":"2024-02-02T13:31:16.927694+0000","last_active":"2024-02-02T13:31:17.906902+0000","last_peered":"2024-02-02T13:31:17.906902+0000","last_clean":"2024-02-02T13:31:17.906902+0000","last_became_active":"2024-02-02T13:31:16.927267+0000","last_became_peered":"2024-02-02T13:31:16.927267+0000","last_unstale":"2024-02-02T13:31:17.906902+0000","last_undegraded":"2024-02-02T13:31:17.906902+0000","last_fullsized":"2024-02-02T13:31:17.906902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907431+0000","last_change":"2024-02-02T13:31:16.931926+0000","last_active":"2024-02-02T13:31:17.907431+0000","last_peered":"2024-02-02T13:31:17.907431+0000","last_clean":"2024-02-02T13:31:17.907431+0000","last_became_active":"2024-02-02T13:31:16.931680+0000","last_became_peered":"2024-02-02T13:31:16.931680+0000","last_unstale":"2024-02-02T13:31:17.907431+0000","last_undegraded":"2024-02-02T13:31:17.907431+0000","last_fullsized":"2024-02-02T13:31:17.907431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678475+0000","last_change":"2024-02-02T13:31:16.920779+0000","last_active":"2024-02-02T13:31:18.678475+0000","last_peered":"2024-02-02T13:31:18.678475+0000","last_clean":"2024-02-02T13:31:18.678475+0000","last_became_active":"2024-02-02T13:31:16.918630+0000","last_became_peered":"2024-02-02T13:31:16.918630+0000","last_unstale":"2024-02-02T13:31:18.678475+0000","last_undegraded":"2024-02-02T13:31:18.678475+0000","last_fullsized":"2024-02-02T13:31:18.678475+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411470+0000","last_change":"2024-02-02T13:31:16.923782+0000","last_active":"2024-02-02T13:31:18.411470+0000","last_peered":"2024-02-02T13:31:18.411470+0000","last_clean":"2024-02-02T13:31:18.411470+0000","last_became_active":"2024-02-02T13:31:16.923387+0000","last_became_peered":"2024-02-02T13:31:16.923387+0000","last_unstale":"2024-02-02T13:31:18.411470+0000","last_undegraded":"2024-02-02T13:31:18.411470+0000","last_fullsized":"2024-02-02T13:31:18.411470+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907364+0000","last_change":"2024-02-02T13:31:16.920823+0000","last_active":"2024-02-02T13:31:17.907364+0000","last_peered":"2024-02-02T13:31:17.907364+0000","last_clean":"2024-02-02T13:31:17.907364+0000","last_became_active":"2024-02-02T13:31:16.920636+0000","last_became_peered":"2024-02-02T13:31:16.920636+0000","last_unstale":"2024-02-02T13:31:17.907364+0000","last_undegraded":"2024-02-02T13:31:17.907364+0000","last_fullsized":"2024-02-02T13:31:17.907364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678635+0000","last_change":"2024-02-02T13:31:16.922205+0000","last_active":"2024-02-02T13:31:18.678635+0000","last_peered":"2024-02-02T13:31:18.678635+0000","last_clean":"2024-02-02T13:31:18.678635+0000","last_became_active":"2024-02-02T13:31:16.921225+0000","last_became_peered":"2024-02-02T13:31:16.921225+0000","last_unstale":"2024-02-02T13:31:18.678635+0000","last_undegraded":"2024-02-02T13:31:18.678635+0000","last_fullsized":"2024-02-02T13:31:18.678635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410942+0000","last_change":"2024-02-02T13:31:16.919017+0000","last_active":"2024-02-02T13:31:18.410942+0000","last_peered":"2024-02-02T13:31:18.410942+0000","last_clean":"2024-02-02T13:31:18.410942+0000","last_became_active":"2024-02-02T13:31:16.913456+0000","last_became_peered":"2024-02-02T13:31:16.913456+0000","last_unstale":"2024-02-02T13:31:18.410942+0000","last_undegraded":"2024-02-02T13:31:18.410942+0000","last_fullsized":"2024-02-02T13:31:18.410942+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.905879+0000","last_change":"2024-02-02T13:31:16.920909+0000","last_active":"2024-02-02T13:31:17.905879+0000","last_peered":"2024-02-02T13:31:17.905879+0000","last_clean":"2024-02-02T13:31:17.905879+0000","last_became_active":"2024-02-02T13:31:16.920620+0000","last_became_peered":"2024-02-02T13:31:16.920620+0000","last_unstale":"2024-02-02T13:31:17.905879+0000","last_undegraded":"2024-02-02T13:31:17.905879+0000","last_fullsized":"2024-02-02T13:31:17.905879+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:47:29.170966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907939+0000","last_change":"2024-02-02T13:31:16.915569+0000","last_active":"2024-02-02T13:31:17.907939+0000","last_peered":"2024-02-02T13:31:17.907939+0000","last_clean":"2024-02-02T13:31:17.907939+0000","last_became_active":"2024-02-02T13:31:16.914947+0000","last_became_peered":"2024-02-02T13:31:16.914947+0000","last_unstale":"2024-02-02T13:31:17.907939+0000","last_undegraded":"2024-02-02T13:31:17.907939+0000","last_fullsized":"2024-02-02T13:31:17.907939+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:07.892976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860126+0000","last_change":"2024-02-02T13:31:16.908445+0000","last_active":"2024-02-02T13:31:18.860126+0000","last_peered":"2024-02-02T13:31:18.860126+0000","last_clean":"2024-02-02T13:31:18.860126+0000","last_became_active":"2024-02-02T13:31:16.908180+0000","last_became_peered":"2024-02-02T13:31:16.908180+0000","last_unstale":"2024-02-02T13:31:18.860126+0000","last_undegraded":"2024-02-02T13:31:18.860126+0000","last_fullsized":"2024-02-02T13:31:18.860126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902747+0000","last_change":"2024-02-02T13:31:16.921843+0000","last_active":"2024-02-02T13:31:17.902747+0000","last_peered":"2024-02-02T13:31:17.902747+0000","last_clean":"2024-02-02T13:31:17.902747+0000","last_became_active":"2024-02-02T13:31:16.921627+0000","last_became_peered":"2024-02-02T13:31:16.921627+0000","last_unstale":"2024-02-02T13:31:17.902747+0000","last_undegraded":"2024-02-02T13:31:17.902747+0000","last_fullsized":"2024-02-02T13:31:17.902747+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:58:17.957284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906836+0000","last_change":"2024-02-02T13:31:16.918012+0000","last_active":"2024-02-02T13:31:17.906836+0000","last_peered":"2024-02-02T13:31:17.906836+0000","last_clean":"2024-02-02T13:31:17.906836+0000","last_became_active":"2024-02-02T13:31:16.917805+0000","last_became_peered":"2024-02-02T13:31:16.917805+0000","last_unstale":"2024-02-02T13:31:17.906836+0000","last_undegraded":"2024-02-02T13:31:17.906836+0000","last_fullsized":"2024-02-02T13:31:17.906836+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:47.224031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"127'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925207+0000","last_change":"2024-02-02T13:31:16.433503+0000","last_active":"2024-02-02T13:31:16.925207+0000","last_peered":"2024-02-02T13:31:16.925207+0000","last_clean":"2024-02-02T13:31:16.925207+0000","last_became_active":"2024-02-02T13:31:16.433277+0000","last_became_peered":"2024-02-02T13:31:16.433277+0000","last_unstale":"2024-02-02T13:31:16.925207+0000","last_undegraded":"2024-02-02T13:31:16.925207+0000","last_fullsized":"2024-02-02T13:31:16.925207+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:16:12.721762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860526+0000","last_change":"2024-02-02T13:31:16.919617+0000","last_active":"2024-02-02T13:31:18.860526+0000","last_peered":"2024-02-02T13:31:18.860526+0000","last_clean":"2024-02-02T13:31:18.860526+0000","last_became_active":"2024-02-02T13:31:16.919275+0000","last_became_peered":"2024-02-02T13:31:16.919275+0000","last_unstale":"2024-02-02T13:31:18.860526+0000","last_undegraded":"2024-02-02T13:31:18.860526+0000","last_fullsized":"2024-02-02T13:31:18.860526+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931501+0000","last_change":"2024-02-02T13:31:16.424527+0000","last_active":"2024-02-02T13:31:16.931501+0000","last_peered":"2024-02-02T13:31:16.931501+0000","last_clean":"2024-02-02T13:31:16.931501+0000","last_became_active":"2024-02-02T13:31:16.423926+0000","last_became_peered":"2024-02-02T13:31:16.423926+0000","last_unstale":"2024-02-02T13:31:16.931501+0000","last_undegraded":"2024-02-02T13:31:16.931501+0000","last_fullsized":"2024-02-02T13:31:16.931501+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903041+0000","last_change":"2024-02-02T13:31:16.922269+0000","last_active":"2024-02-02T13:31:17.903041+0000","last_peered":"2024-02-02T13:31:17.903041+0000","last_clean":"2024-02-02T13:31:17.903041+0000","last_became_active":"2024-02-02T13:31:16.921999+0000","last_became_peered":"2024-02-02T13:31:16.921999+0000","last_unstale":"2024-02-02T13:31:17.903041+0000","last_undegraded":"2024-02-02T13:31:17.903041+0000","last_fullsized":"2024-02-02T13:31:17.903041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:26:34.152547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.921395+0000","last_change":"2024-02-02T13:31:16.434462+0000","last_active":"2024-02-02T13:31:16.921395+0000","last_peered":"2024-02-02T13:31:16.921395+0000","last_clean":"2024-02-02T13:31:16.921395+0000","last_became_active":"2024-02-02T13:31:16.434216+0000","last_became_peered":"2024-02-02T13:31:16.434216+0000","last_unstale":"2024-02-02T13:31:16.921395+0000","last_undegraded":"2024-02-02T13:31:16.921395+0000","last_fullsized":"2024-02-02T13:31:16.921395+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380530+0000","last_change":"2024-02-02T13:31:16.928736+0000","last_active":"2024-02-02T13:31:20.380530+0000","last_peered":"2024-02-02T13:31:20.380530+0000","last_clean":"2024-02-02T13:31:20.380530+0000","last_became_active":"2024-02-02T13:31:16.928489+0000","last_became_peered":"2024-02-02T13:31:16.928489+0000","last_unstale":"2024-02-02T13:31:20.380530+0000","last_undegraded":"2024-02-02T13:31:20.380530+0000","last_fullsized":"2024-02-02T13:31:20.380530+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:57.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.928318+0000","last_change":"2024-02-02T13:31:16.440048+0000","last_active":"2024-02-02T13:31:16.928318+0000","last_peered":"2024-02-02T13:31:16.928318+0000","last_clean":"2024-02-02T13:31:16.928318+0000","last_became_active":"2024-02-02T13:31:16.439503+0000","last_became_peered":"2024-02-02T13:31:16.439503+0000","last_unstale":"2024-02-02T13:31:16.928318+0000","last_undegraded":"2024-02-02T13:31:16.928318+0000","last_fullsized":"2024-02-02T13:31:16.928318+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.366981+0000","last_change":"2024-02-02T13:31:16.931809+0000","last_active":"2024-02-02T13:31:18.366981+0000","last_peered":"2024-02-02T13:31:18.366981+0000","last_clean":"2024-02-02T13:31:18.366981+0000","last_became_active":"2024-02-02T13:31:16.931497+0000","last_became_peered":"2024-02-02T13:31:16.931497+0000","last_unstale":"2024-02-02T13:31:18.366981+0000","last_undegraded":"2024-02-02T13:31:18.366981+0000","last_fullsized":"2024-02-02T13:31:18.366981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:55:21.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.922362+0000","last_change":"2024-02-02T13:31:16.418697+0000","last_active":"2024-02-02T13:31:16.922362+0000","last_peered":"2024-02-02T13:31:16.922362+0000","last_clean":"2024-02-02T13:31:16.922362+0000","last_became_active":"2024-02-02T13:31:16.417931+0000","last_became_peered":"2024-02-02T13:31:16.417931+0000","last_unstale":"2024-02-02T13:31:16.922362+0000","last_undegraded":"2024-02-02T13:31:16.922362+0000","last_fullsized":"2024-02-02T13:31:16.922362+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:54.011838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678787+0000","last_change":"2024-02-02T13:31:16.925253+0000","last_active":"2024-02-02T13:31:18.678787+0000","last_peered":"2024-02-02T13:31:18.678787+0000","last_clean":"2024-02-02T13:31:18.678787+0000","last_became_active":"2024-02-02T13:31:16.925007+0000","last_became_peered":"2024-02-02T13:31:16.925007+0000","last_unstale":"2024-02-02T13:31:18.678787+0000","last_undegraded":"2024-02-02T13:31:18.678787+0000","last_fullsized":"2024-02-02T13:31:18.678787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924884+0000","last_change":"2024-02-02T13:31:16.437360+0000","last_active":"2024-02-02T13:31:16.924884+0000","last_peered":"2024-02-02T13:31:16.924884+0000","last_clean":"2024-02-02T13:31:16.924884+0000","last_became_active":"2024-02-02T13:31:16.437067+0000","last_became_peered":"2024-02-02T13:31:16.437067+0000","last_unstale":"2024-02-02T13:31:16.924884+0000","last_undegraded":"2024-02-02T13:31:16.924884+0000","last_fullsized":"2024-02-02T13:31:16.924884+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860682+0000","last_change":"2024-02-02T13:31:16.907956+0000","last_active":"2024-02-02T13:31:18.860682+0000","last_peered":"2024-02-02T13:31:18.860682+0000","last_clean":"2024-02-02T13:31:18.860682+0000","last_became_active":"2024-02-02T13:31:16.907664+0000","last_became_peered":"2024-02-02T13:31:16.907664+0000","last_unstale":"2024-02-02T13:31:18.860682+0000","last_undegraded":"2024-02-02T13:31:18.860682+0000","last_fullsized":"2024-02-02T13:31:18.860682+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.909194+0000","last_change":"2024-02-02T13:31:16.435466+0000","last_active":"2024-02-02T13:31:16.909194+0000","last_peered":"2024-02-02T13:31:16.909194+0000","last_clean":"2024-02-02T13:31:16.909194+0000","last_became_active":"2024-02-02T13:31:16.434867+0000","last_became_peered":"2024-02-02T13:31:16.434867+0000","last_unstale":"2024-02-02T13:31:16.909194+0000","last_undegraded":"2024-02-02T13:31:16.909194+0000","last_fullsized":"2024-02-02T13:31:16.909194+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:50:14.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380345+0000","last_change":"2024-02-02T13:31:16.932808+0000","last_active":"2024-02-02T13:31:20.380345+0000","last_peered":"2024-02-02T13:31:20.380345+0000","last_clean":"2024-02-02T13:31:20.380345+0000","last_became_active":"2024-02-02T13:31:16.932559+0000","last_became_peered":"2024-02-02T13:31:16.932559+0000","last_unstale":"2024-02-02T13:31:20.380345+0000","last_undegraded":"2024-02-02T13:31:20.380345+0000","last_fullsized":"2024-02-02T13:31:20.380345+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:21.439554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902293+0000","last_change":"2024-02-02T13:31:16.432947+0000","last_active":"2024-02-02T13:31:16.902293+0000","last_peered":"2024-02-02T13:31:16.902293+0000","last_clean":"2024-02-02T13:31:16.902293+0000","last_became_active":"2024-02-02T13:31:16.432457+0000","last_became_peered":"2024-02-02T13:31:16.432457+0000","last_unstale":"2024-02-02T13:31:16.902293+0000","last_undegraded":"2024-02-02T13:31:16.902293+0000","last_fullsized":"2024-02-02T13:31:16.902293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:06.887358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906804+0000","last_change":"2024-02-02T13:31:16.928592+0000","last_active":"2024-02-02T13:31:17.906804+0000","last_peered":"2024-02-02T13:31:17.906804+0000","last_clean":"2024-02-02T13:31:17.906804+0000","last_became_active":"2024-02-02T13:31:16.928381+0000","last_became_peered":"2024-02-02T13:31:16.928381+0000","last_unstale":"2024-02-02T13:31:17.906804+0000","last_undegraded":"2024-02-02T13:31:17.906804+0000","last_fullsized":"2024-02-02T13:31:17.906804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.900135+0000","last_change":"2024-02-02T13:31:16.435253+0000","last_active":"2024-02-02T13:31:16.900135+0000","last_peered":"2024-02-02T13:31:16.900135+0000","last_clean":"2024-02-02T13:31:16.900135+0000","last_became_active":"2024-02-02T13:31:16.432671+0000","last_became_peered":"2024-02-02T13:31:16.432671+0000","last_unstale":"2024-02-02T13:31:16.900135+0000","last_undegraded":"2024-02-02T13:31:16.900135+0000","last_fullsized":"2024-02-02T13:31:16.900135+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:46.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380428+0000","last_change":"2024-02-02T13:31:16.924607+0000","last_active":"2024-02-02T13:31:20.380428+0000","last_peered":"2024-02-02T13:31:20.380428+0000","last_clean":"2024-02-02T13:31:20.380428+0000","last_became_active":"2024-02-02T13:31:16.923514+0000","last_became_peered":"2024-02-02T13:31:16.923514+0000","last_unstale":"2024-02-02T13:31:20.380428+0000","last_undegraded":"2024-02-02T13:31:20.380428+0000","last_fullsized":"2024-02-02T13:31:20.380428+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:11.217285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902541+0000","last_change":"2024-02-02T13:31:16.438628+0000","last_active":"2024-02-02T13:31:16.902541+0000","last_peered":"2024-02-02T13:31:16.902541+0000","last_clean":"2024-02-02T13:31:16.902541+0000","last_became_active":"2024-02-02T13:31:16.438393+0000","last_became_peered":"2024-02-02T13:31:16.438393+0000","last_unstale":"2024-02-02T13:31:16.902541+0000","last_undegraded":"2024-02-02T13:31:16.902541+0000","last_fullsized":"2024-02-02T13:31:16.902541+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410690+0000","last_change":"2024-02-02T13:31:16.922466+0000","last_active":"2024-02-02T13:31:18.410690+0000","last_peered":"2024-02-02T13:31:18.410690+0000","last_clean":"2024-02-02T13:31:18.410690+0000","last_became_active":"2024-02-02T13:31:16.922210+0000","last_became_peered":"2024-02-02T13:31:16.922210+0000","last_unstale":"2024-02-02T13:31:18.410690+0000","last_undegraded":"2024-02-02T13:31:18.410690+0000","last_fullsized":"2024-02-02T13:31:18.410690+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"127'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933710+0000","last_change":"2024-02-02T13:31:16.433800+0000","last_active":"2024-02-02T13:31:16.933710+0000","last_peered":"2024-02-02T13:31:16.933710+0000","last_clean":"2024-02-02T13:31:16.933710+0000","last_became_active":"2024-02-02T13:31:16.433510+0000","last_became_peered":"2024-02-02T13:31:16.433510+0000","last_unstale":"2024-02-02T13:31:16.933710+0000","last_undegraded":"2024-02-02T13:31:16.933710+0000","last_fullsized":"2024-02-02T13:31:16.933710+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678509+0000","last_change":"2024-02-02T13:31:16.930321+0000","last_active":"2024-02-02T13:31:18.678509+0000","last_peered":"2024-02-02T13:31:18.678509+0000","last_clean":"2024-02-02T13:31:18.678509+0000","last_became_active":"2024-02-02T13:31:16.929950+0000","last_became_peered":"2024-02-02T13:31:16.929950+0000","last_unstale":"2024-02-02T13:31:18.678509+0000","last_undegraded":"2024-02-02T13:31:18.678509+0000","last_fullsized":"2024-02-02T13:31:18.678509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"127'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.934404+0000","last_change":"2024-02-02T13:31:16.440711+0000","last_active":"2024-02-02T13:31:16.934404+0000","last_peered":"2024-02-02T13:31:16.934404+0000","last_clean":"2024-02-02T13:31:16.934404+0000","last_became_active":"2024-02-02T13:31:16.440503+0000","last_became_peered":"2024-02-02T13:31:16.440503+0000","last_unstale":"2024-02-02T13:31:16.934404+0000","last_undegraded":"2024-02-02T13:31:16.934404+0000","last_fullsized":"2024-02-02T13:31:16.934404+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:46.255117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860267+0000","last_change":"2024-02-02T13:31:16.925966+0000","last_active":"2024-02-02T13:31:18.860267+0000","last_peered":"2024-02-02T13:31:18.860267+0000","last_clean":"2024-02-02T13:31:18.860267+0000","last_became_active":"2024-02-02T13:31:16.925765+0000","last_became_peered":"2024-02-02T13:31:16.925765+0000","last_unstale":"2024-02-02T13:31:18.860267+0000","last_undegraded":"2024-02-02T13:31:18.860267+0000","last_fullsized":"2024-02-02T13:31:18.860267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925661+0000","last_change":"2024-02-02T13:31:16.435630+0000","last_active":"2024-02-02T13:31:16.925661+0000","last_peered":"2024-02-02T13:31:16.925661+0000","last_clean":"2024-02-02T13:31:16.925661+0000","last_became_active":"2024-02-02T13:31:16.435406+0000","last_became_peered":"2024-02-02T13:31:16.435406+0000","last_unstale":"2024-02-02T13:31:16.925661+0000","last_undegraded":"2024-02-02T13:31:16.925661+0000","last_fullsized":"2024-02-02T13:31:16.925661+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:47.961252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903675+0000","last_change":"2024-02-02T13:31:16.913272+0000","last_active":"2024-02-02T13:31:17.903675+0000","last_peered":"2024-02-02T13:31:17.903675+0000","last_clean":"2024-02-02T13:31:17.903675+0000","last_became_active":"2024-02-02T13:31:16.912436+0000","last_became_peered":"2024-02-02T13:31:16.912436+0000","last_unstale":"2024-02-02T13:31:17.903675+0000","last_undegraded":"2024-02-02T13:31:17.903675+0000","last_fullsized":"2024-02-02T13:31:17.903675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:58.823596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906345+0000","last_change":"2024-02-02T13:31:16.428888+0000","last_active":"2024-02-02T13:31:16.906345+0000","last_peered":"2024-02-02T13:31:16.906345+0000","last_clean":"2024-02-02T13:31:16.906345+0000","last_became_active":"2024-02-02T13:31:16.428511+0000","last_became_peered":"2024-02-02T13:31:16.428511+0000","last_unstale":"2024-02-02T13:31:16.906345+0000","last_undegraded":"2024-02-02T13:31:16.906345+0000","last_fullsized":"2024-02-02T13:31:16.906345+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:55.831956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380126+0000","last_change":"2024-02-02T13:31:16.929051+0000","last_active":"2024-02-02T13:31:20.380126+0000","last_peered":"2024-02-02T13:31:20.380126+0000","last_clean":"2024-02-02T13:31:20.380126+0000","last_became_active":"2024-02-02T13:31:16.928735+0000","last_became_peered":"2024-02-02T13:31:16.928735+0000","last_unstale":"2024-02-02T13:31:20.380126+0000","last_undegraded":"2024-02-02T13:31:20.380126+0000","last_fullsized":"2024-02-02T13:31:20.380126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:57.109278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.912976+0000","last_change":"2024-02-02T13:31:16.434208+0000","last_active":"2024-02-02T13:31:16.912976+0000","last_peered":"2024-02-02T13:31:16.912976+0000","last_clean":"2024-02-02T13:31:16.912976+0000","last_became_active":"2024-02-02T13:31:16.433931+0000","last_became_peered":"2024-02-02T13:31:16.433931+0000","last_unstale":"2024-02-02T13:31:16.912976+0000","last_undegraded":"2024-02-02T13:31:16.912976+0000","last_fullsized":"2024-02-02T13:31:16.912976+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906588+0000","last_change":"2024-02-02T13:31:16.925971+0000","last_active":"2024-02-02T13:31:17.906588+0000","last_peered":"2024-02-02T13:31:17.906588+0000","last_clean":"2024-02-02T13:31:17.906588+0000","last_became_active":"2024-02-02T13:31:16.925633+0000","last_became_peered":"2024-02-02T13:31:16.925633+0000","last_unstale":"2024-02-02T13:31:17.906588+0000","last_undegraded":"2024-02-02T13:31:17.906588+0000","last_fullsized":"2024-02-02T13:31:17.906588+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:18.174713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"127'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931113+0000","last_change":"2024-02-02T13:31:16.426042+0000","last_active":"2024-02-02T13:31:16.931113+0000","last_peered":"2024-02-02T13:31:16.931113+0000","last_clean":"2024-02-02T13:31:16.931113+0000","last_became_active":"2024-02-02T13:31:16.425745+0000","last_became_peered":"2024-02-02T13:31:16.425745+0000","last_unstale":"2024-02-02T13:31:16.931113+0000","last_undegraded":"2024-02-02T13:31:16.931113+0000","last_fullsized":"2024-02-02T13:31:16.931113+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:57.874029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410943+0000","last_change":"2024-02-02T13:31:16.927661+0000","last_active":"2024-02-02T13:31:18.410943+0000","last_peered":"2024-02-02T13:31:18.410943+0000","last_clean":"2024-02-02T13:31:18.410943+0000","last_became_active":"2024-02-02T13:31:16.927430+0000","last_became_peered":"2024-02-02T13:31:16.927430+0000","last_unstale":"2024-02-02T13:31:18.410943+0000","last_undegraded":"2024-02-02T13:31:18.410943+0000","last_fullsized":"2024-02-02T13:31:18.410943+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931470+0000","last_change":"2024-02-02T13:31:16.436067+0000","last_active":"2024-02-02T13:31:16.931470+0000","last_peered":"2024-02-02T13:31:16.931470+0000","last_clean":"2024-02-02T13:31:16.931470+0000","last_became_active":"2024-02-02T13:31:16.435779+0000","last_became_peered":"2024-02-02T13:31:16.435779+0000","last_unstale":"2024-02-02T13:31:16.931470+0000","last_undegraded":"2024-02-02T13:31:16.931470+0000","last_fullsized":"2024-02-02T13:31:16.931470+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.367030+0000","last_change":"2024-02-02T13:31:16.926917+0000","last_active":"2024-02-02T13:31:18.367030+0000","last_peered":"2024-02-02T13:31:18.367030+0000","last_clean":"2024-02-02T13:31:18.367030+0000","last_became_active":"2024-02-02T13:31:16.926662+0000","last_became_peered":"2024-02-02T13:31:16.926662+0000","last_unstale":"2024-02-02T13:31:18.367030+0000","last_undegraded":"2024-02-02T13:31:18.367030+0000","last_fullsized":"2024-02-02T13:31:18.367030+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:46:43.024952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931079+0000","last_change":"2024-02-02T13:31:16.431019+0000","last_active":"2024-02-02T13:31:16.931079+0000","last_peered":"2024-02-02T13:31:16.931079+0000","last_clean":"2024-02-02T13:31:16.931079+0000","last_became_active":"2024-02-02T13:31:16.430740+0000","last_became_peered":"2024-02-02T13:31:16.430740+0000","last_unstale":"2024-02-02T13:31:16.931079+0000","last_undegraded":"2024-02-02T13:31:16.931079+0000","last_fullsized":"2024-02-02T13:31:16.931079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679308+0000","last_change":"2024-02-02T13:31:16.930011+0000","last_active":"2024-02-02T13:31:18.679308+0000","last_peered":"2024-02-02T13:31:18.679308+0000","last_clean":"2024-02-02T13:31:18.679308+0000","last_became_active":"2024-02-02T13:31:16.929669+0000","last_became_peered":"2024-02-02T13:31:16.929669+0000","last_unstale":"2024-02-02T13:31:18.679308+0000","last_undegraded":"2024-02-02T13:31:18.679308+0000","last_fullsized":"2024-02-02T13:31:18.679308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916493+0000","last_change":"2024-02-02T13:31:16.436267+0000","last_active":"2024-02-02T13:31:16.916493+0000","last_peered":"2024-02-02T13:31:16.916493+0000","last_clean":"2024-02-02T13:31:16.916493+0000","last_became_active":"2024-02-02T13:31:16.434857+0000","last_became_peered":"2024-02-02T13:31:16.434857+0000","last_unstale":"2024-02-02T13:31:16.916493+0000","last_undegraded":"2024-02-02T13:31:16.916493+0000","last_fullsized":"2024-02-02T13:31:16.916493+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:56.711812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907106+0000","last_change":"2024-02-02T13:31:16.909236+0000","last_active":"2024-02-02T13:31:17.907106+0000","last_peered":"2024-02-02T13:31:17.907106+0000","last_clean":"2024-02-02T13:31:17.907106+0000","last_became_active":"2024-02-02T13:31:16.908955+0000","last_became_peered":"2024-02-02T13:31:16.908955+0000","last_unstale":"2024-02-02T13:31:17.907106+0000","last_undegraded":"2024-02-02T13:31:17.907106+0000","last_fullsized":"2024-02-02T13:31:17.907106+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:11:20.213389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924258+0000","last_change":"2024-02-02T13:31:16.434809+0000","last_active":"2024-02-02T13:31:16.924258+0000","last_peered":"2024-02-02T13:31:16.924258+0000","last_clean":"2024-02-02T13:31:16.924258+0000","last_became_active":"2024-02-02T13:31:16.434339+0000","last_became_peered":"2024-02-02T13:31:16.434339+0000","last_unstale":"2024-02-02T13:31:16.924258+0000","last_undegraded":"2024-02-02T13:31:16.924258+0000","last_fullsized":"2024-02-02T13:31:16.924258+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:21.074291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906252+0000","last_change":"2024-02-02T13:31:16.922167+0000","last_active":"2024-02-02T13:31:17.906252+0000","last_peered":"2024-02-02T13:31:17.906252+0000","last_clean":"2024-02-02T13:31:17.906252+0000","last_became_active":"2024-02-02T13:31:16.921940+0000","last_became_peered":"2024-02-02T13:31:16.921940+0000","last_unstale":"2024-02-02T13:31:17.906252+0000","last_undegraded":"2024-02-02T13:31:17.906252+0000","last_fullsized":"2024-02-02T13:31:17.906252+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:55.017456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.915787+0000","last_change":"2024-02-02T13:31:16.436209+0000","last_active":"2024-02-02T13:31:16.915787+0000","last_peered":"2024-02-02T13:31:16.915787+0000","last_clean":"2024-02-02T13:31:16.915787+0000","last_became_active":"2024-02-02T13:31:16.435952+0000","last_became_peered":"2024-02-02T13:31:16.435952+0000","last_unstale":"2024-02-02T13:31:16.915787+0000","last_undegraded":"2024-02-02T13:31:16.915787+0000","last_fullsized":"2024-02-02T13:31:16.915787+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:56.934997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906449+0000","last_change":"2024-02-02T13:31:16.908777+0000","last_active":"2024-02-02T13:31:17.906449+0000","last_peered":"2024-02-02T13:31:17.906449+0000","last_clean":"2024-02-02T13:31:17.906449+0000","last_became_active":"2024-02-02T13:31:16.908560+0000","last_became_peered":"2024-02-02T13:31:16.908560+0000","last_unstale":"2024-02-02T13:31:17.906449+0000","last_undegraded":"2024-02-02T13:31:17.906449+0000","last_fullsized":"2024-02-02T13:31:17.906449+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:44.264100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.920654+0000","last_change":"2024-02-02T13:31:16.430948+0000","last_active":"2024-02-02T13:31:16.920654+0000","last_peered":"2024-02-02T13:31:16.920654+0000","last_clean":"2024-02-02T13:31:16.920654+0000","last_became_active":"2024-02-02T13:31:16.430678+0000","last_became_peered":"2024-02-02T13:31:16.430678+0000","last_unstale":"2024-02-02T13:31:16.920654+0000","last_undegraded":"2024-02-02T13:31:16.920654+0000","last_fullsized":"2024-02-02T13:31:16.920654+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411647+0000","last_change":"2024-02-02T13:31:16.918149+0000","last_active":"2024-02-02T13:31:18.411647+0000","last_peered":"2024-02-02T13:31:18.411647+0000","last_clean":"2024-02-02T13:31:18.411647+0000","last_became_active":"2024-02-02T13:31:16.916696+0000","last_became_peered":"2024-02-02T13:31:16.916696+0000","last_unstale":"2024-02-02T13:31:18.411647+0000","last_undegraded":"2024-02-02T13:31:18.411647+0000","last_fullsized":"2024-02-02T13:31:18.411647+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:45.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916199+0000","last_change":"2024-02-02T13:31:16.432995+0000","last_active":"2024-02-02T13:31:16.916199+0000","last_peered":"2024-02-02T13:31:16.916199+0000","last_clean":"2024-02-02T13:31:16.916199+0000","last_became_active":"2024-02-02T13:31:16.432557+0000","last_became_peered":"2024-02-02T13:31:16.432557+0000","last_unstale":"2024-02-02T13:31:16.916199+0000","last_undegraded":"2024-02-02T13:31:16.916199+0000","last_fullsized":"2024-02-02T13:31:16.916199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907676+0000","last_change":"2024-02-02T13:31:16.931825+0000","last_active":"2024-02-02T13:31:17.907676+0000","last_peered":"2024-02-02T13:31:17.907676+0000","last_clean":"2024-02-02T13:31:17.907676+0000","last_became_active":"2024-02-02T13:31:16.931448+0000","last_became_peered":"2024-02-02T13:31:16.931448+0000","last_unstale":"2024-02-02T13:31:17.907676+0000","last_undegraded":"2024-02-02T13:31:17.907676+0000","last_fullsized":"2024-02-02T13:31:17.907676+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933817+0000","last_change":"2024-02-02T13:31:16.435712+0000","last_active":"2024-02-02T13:31:16.933817+0000","last_peered":"2024-02-02T13:31:16.933817+0000","last_clean":"2024-02-02T13:31:16.933817+0000","last_became_active":"2024-02-02T13:31:16.435452+0000","last_became_peered":"2024-02-02T13:31:16.435452+0000","last_unstale":"2024-02-02T13:31:16.933817+0000","last_undegraded":"2024-02-02T13:31:16.933817+0000","last_fullsized":"2024-02-02T13:31:16.933817+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902968+0000","last_change":"2024-02-02T13:31:16.908923+0000","last_active":"2024-02-02T13:31:17.902968+0000","last_peered":"2024-02-02T13:31:17.902968+0000","last_clean":"2024-02-02T13:31:17.902968+0000","last_became_active":"2024-02-02T13:31:16.908678+0000","last_became_peered":"2024-02-02T13:31:16.908678+0000","last_unstale":"2024-02-02T13:31:17.902968+0000","last_undegraded":"2024-02-02T13:31:17.902968+0000","last_fullsized":"2024-02-02T13:31:17.902968+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:39:39.190128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.917921+0000","last_change":"2024-02-02T13:31:16.434573+0000","last_active":"2024-02-02T13:31:16.917921+0000","last_peered":"2024-02-02T13:31:16.917921+0000","last_clean":"2024-02-02T13:31:16.917921+0000","last_became_active":"2024-02-02T13:31:16.433985+0000","last_became_peered":"2024-02-02T13:31:16.433985+0000","last_unstale":"2024-02-02T13:31:16.917921+0000","last_undegraded":"2024-02-02T13:31:16.917921+0000","last_fullsized":"2024-02-02T13:31:16.917921+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:10.211668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903757+0000","last_change":"2024-02-02T13:31:16.913433+0000","last_active":"2024-02-02T13:31:17.903757+0000","last_peered":"2024-02-02T13:31:17.903757+0000","last_clean":"2024-02-02T13:31:17.903757+0000","last_became_active":"2024-02-02T13:31:16.913071+0000","last_became_peered":"2024-02-02T13:31:16.913071+0000","last_unstale":"2024-02-02T13:31:17.903757+0000","last_undegraded":"2024-02-02T13:31:17.903757+0000","last_fullsized":"2024-02-02T13:31:17.903757+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:02:37.732207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916092+0000","last_change":"2024-02-02T13:31:16.437984+0000","last_active":"2024-02-02T13:31:16.916092+0000","last_peered":"2024-02-02T13:31:16.916092+0000","last_clean":"2024-02-02T13:31:16.916092+0000","last_became_active":"2024-02-02T13:31:16.437426+0000","last_became_peered":"2024-02-02T13:31:16.437426+0000","last_unstale":"2024-02-02T13:31:16.916092+0000","last_undegraded":"2024-02-02T13:31:16.916092+0000","last_fullsized":"2024-02-02T13:31:16.916092+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411212+0000","last_change":"2024-02-02T13:31:16.914410+0000","last_active":"2024-02-02T13:31:18.411212+0000","last_peered":"2024-02-02T13:31:18.411212+0000","last_clean":"2024-02-02T13:31:18.411212+0000","last_became_active":"2024-02-02T13:31:16.913957+0000","last_became_peered":"2024-02-02T13:31:16.913957+0000","last_unstale":"2024-02-02T13:31:18.411212+0000","last_undegraded":"2024-02-02T13:31:18.411212+0000","last_fullsized":"2024-02-02T13:31:18.411212+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.908331+0000","last_change":"2024-02-02T13:31:16.428377+0000","last_active":"2024-02-02T13:31:16.908331+0000","last_peered":"2024-02-02T13:31:16.908331+0000","last_clean":"2024-02-02T13:31:16.908331+0000","last_became_active":"2024-02-02T13:31:16.427907+0000","last_became_peered":"2024-02-02T13:31:16.427907+0000","last_unstale":"2024-02-02T13:31:16.908331+0000","last_undegraded":"2024-02-02T13:31:16.908331+0000","last_fullsized":"2024-02-02T13:31:16.908331+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:58:45.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906825+0000","last_change":"2024-02-02T13:31:16.925836+0000","last_active":"2024-02-02T13:31:17.906825+0000","last_peered":"2024-02-02T13:31:17.906825+0000","last_clean":"2024-02-02T13:31:17.906825+0000","last_became_active":"2024-02-02T13:31:16.925344+0000","last_became_peered":"2024-02-02T13:31:16.925344+0000","last_unstale":"2024-02-02T13:31:17.906825+0000","last_undegraded":"2024-02-02T13:31:17.906825+0000","last_fullsized":"2024-02-02T13:31:17.906825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:10:14.812842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916292+0000","last_change":"2024-02-02T13:31:16.440211+0000","last_active":"2024-02-02T13:31:16.916292+0000","last_peered":"2024-02-02T13:31:16.916292+0000","last_clean":"2024-02-02T13:31:16.916292+0000","last_became_active":"2024-02-02T13:31:16.439815+0000","last_became_peered":"2024-02-02T13:31:16.439815+0000","last_unstale":"2024-02-02T13:31:16.916292+0000","last_undegraded":"2024-02-02T13:31:16.916292+0000","last_fullsized":"2024-02-02T13:31:16.916292+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380628+0000","last_change":"2024-02-02T13:31:16.929773+0000","last_active":"2024-02-02T13:31:20.380628+0000","last_peered":"2024-02-02T13:31:20.380628+0000","last_clean":"2024-02-02T13:31:20.380628+0000","last_became_active":"2024-02-02T13:31:16.929288+0000","last_became_peered":"2024-02-02T13:31:16.929288+0000","last_unstale":"2024-02-02T13:31:20.380628+0000","last_undegraded":"2024-02-02T13:31:20.380628+0000","last_fullsized":"2024-02-02T13:31:20.380628+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:30.322652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.907605+0000","last_change":"2024-02-02T13:31:16.423671+0000","last_active":"2024-02-02T13:31:16.907605+0000","last_peered":"2024-02-02T13:31:16.907605+0000","last_clean":"2024-02-02T13:31:16.907605+0000","last_became_active":"2024-02-02T13:31:16.422898+0000","last_became_peered":"2024-02-02T13:31:16.422898+0000","last_unstale":"2024-02-02T13:31:16.907605+0000","last_undegraded":"2024-02-02T13:31:16.907605+0000","last_fullsized":"2024-02-02T13:31:16.907605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:57:01.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907355+0000","last_change":"2024-02-02T13:31:16.925478+0000","last_active":"2024-02-02T13:31:17.907355+0000","last_peered":"2024-02-02T13:31:17.907355+0000","last_clean":"2024-02-02T13:31:17.907355+0000","last_became_active":"2024-02-02T13:31:16.925257+0000","last_became_peered":"2024-02-02T13:31:16.925257+0000","last_unstale":"2024-02-02T13:31:17.907355+0000","last_undegraded":"2024-02-02T13:31:17.907355+0000","last_fullsized":"2024-02-02T13:31:17.907355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:28.646906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.903393+0000","last_change":"2024-02-02T13:31:16.439366+0000","last_active":"2024-02-02T13:31:16.903393+0000","last_peered":"2024-02-02T13:31:16.903393+0000","last_clean":"2024-02-02T13:31:16.903393+0000","last_became_active":"2024-02-02T13:31:16.438414+0000","last_became_peered":"2024-02-02T13:31:16.438414+0000","last_unstale":"2024-02-02T13:31:16.903393+0000","last_undegraded":"2024-02-02T13:31:16.903393+0000","last_fullsized":"2024-02-02T13:31:16.903393+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411280+0000","last_change":"2024-02-02T13:31:16.919515+0000","last_active":"2024-02-02T13:31:18.411280+0000","last_peered":"2024-02-02T13:31:18.411280+0000","last_clean":"2024-02-02T13:31:18.411280+0000","last_became_active":"2024-02-02T13:31:16.918683+0000","last_became_peered":"2024-02-02T13:31:16.918683+0000","last_unstale":"2024-02-02T13:31:18.411280+0000","last_undegraded":"2024-02-02T13:31:18.411280+0000","last_fullsized":"2024-02-02T13:31:18.411280+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906141+0000","last_change":"2024-02-02T13:31:16.433478+0000","last_active":"2024-02-02T13:31:16.906141+0000","last_peered":"2024-02-02T13:31:16.906141+0000","last_clean":"2024-02-02T13:31:16.906141+0000","last_became_active":"2024-02-02T13:31:16.433197+0000","last_became_peered":"2024-02-02T13:31:16.433197+0000","last_unstale":"2024-02-02T13:31:16.906141+0000","last_undegraded":"2024-02-02T13:31:16.906141+0000","last_fullsized":"2024-02-02T13:31:16.906141+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:54.153974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907538+0000","last_change":"2024-02-02T13:31:16.910890+0000","last_active":"2024-02-02T13:31:17.907538+0000","last_peered":"2024-02-02T13:31:17.907538+0000","last_clean":"2024-02-02T13:31:17.907538+0000","last_became_active":"2024-02-02T13:31:16.910665+0000","last_became_peered":"2024-02-02T13:31:16.910665+0000","last_unstale":"2024-02-02T13:31:17.907538+0000","last_undegraded":"2024-02-02T13:31:17.907538+0000","last_fullsized":"2024-02-02T13:31:17.907538+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:11:17.000587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.930725+0000","last_change":"2024-02-02T13:31:16.434228+0000","last_active":"2024-02-02T13:31:16.930725+0000","last_peered":"2024-02-02T13:31:16.930725+0000","last_clean":"2024-02-02T13:31:16.930725+0000","last_became_active":"2024-02-02T13:31:16.433955+0000","last_became_peered":"2024-02-02T13:31:16.433955+0000","last_unstale":"2024-02-02T13:31:16.930725+0000","last_undegraded":"2024-02-02T13:31:16.930725+0000","last_fullsized":"2024-02-02T13:31:16.930725+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.913104+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:31:16.913104+0000","last_peered":"2024-02-02T13:31:16.913104+0000","last_clean":"2024-02-02T13:31:16.913104+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:31:16.913104+0000","last_undegraded":"2024-02-02T13:31:16.913104+0000","last_fullsized":"2024-02-02T13:31:16.913104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198703555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.0660000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.5149999999999999,"5min":1.6279999999999999,"15min":2.5310000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.105,"15min":1.1240000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4710000000000001,"5min":1.613,"15min":4.1609999999999996},"last":1.091}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.0940000000000001},"min":{"1min":0.86299999999999999,"5min":0.66700000000000004,"15min":0.66300000000000003},"max":{"1min":1.4650000000000001,"5min":1.8839999999999999,"15min":1.893},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.595,"5min":1.758,"15min":2.5840000000000001},"last":1.351}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.266,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.70099999999999996},"max":{"1min":1.7250000000000001,"5min":1.909,"15min":2.6669999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.623},"max":{"1min":1.4359999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.6339999999999999,"15min":1.819},"last":1.202},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.752,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.825,"5min":2.133,"15min":2.4620000000000002},"last":1.254}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.282,"15min":1.202},"min":{"1min":0.92500000000000004,"5min":0.60399999999999998,"15min":0.57599999999999996},"max":{"1min":1.6639999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":0.88400000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.629,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.3520000000000001},"min":{"1min":0.95199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9239999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.454},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.397,"15min":1.4450000000000001},"min":{"1min":0.90900000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7190000000000001,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.8899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4850000000000001,"15min":1.5289999999999999},"min":{"1min":1.246,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.645,"5min":2.1869999999999998,"15min":3.6840000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.087,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0960000000000001,"15min":1.024},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0820000000000001,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.2729999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0269999999999999},"min":{"1min":0.67600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.47,"5min":1.556,"15min":1.556},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1659999999999999,"15min":1.0840000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.589}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1719999999999999,"15min":1.079},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.80300000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.86299999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.89600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.405}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.4570000000000001,"15min":1.27},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":17.686,"5min":17.686,"15min":17.686},"last":1.107},{"interface":"front","average":{"1min":2.1349999999999998,"5min":1.5900000000000001,"15min":1.4490000000000001},"min":{"1min":0.96899999999999997,"5min":0.89900000000000002,"15min":0.66000000000000003},"max":{"1min":17.388999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.498}]},{"osd":5,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.288,"15min":1.276},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":1.986,"15min":1.986},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4339999999999999,"15min":1.359},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.77400000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.462}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.61199999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.613},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.101,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199293965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.403},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.74299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.081}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.26},"min":{"1min":0.76500000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.603,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.284},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.081,"15min":1.1419999999999999},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.474}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.2310000000000001},"min":{"1min":0.86399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.643,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1970000000000001,"15min":1.1779999999999999},"min":{"1min":0.79200000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.468,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.5129999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.202,"15min":1.1899999999999999},"min":{"1min":0.96799999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.409,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.1830000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6200000000000001,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.169}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4570000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.615},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.85099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5569999999999999,"5min":1.833,"15min":1.833},"last":1.4379999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3380000000000001,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7010000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.329},{"interface":"front","average":{"1min":1.341,"5min":1.351,"15min":1.345},"min":{"1min":0.98499999999999999,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.6220000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.724}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.0129999999999999,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":1.1279999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.75,"5min":1.851,"15min":1.851},"last":1.5429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194622,"statfs":{"total":200000000,"available":199293366,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.1080000000000001,"15min":0.997},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.6439999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4970000000000001,"5min":1.839,"15min":1.839},"last":1.3999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":0.995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1659999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.1180000000000001,"15min":1.04},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.097},"min":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9399999999999999,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1950000000000001,"15min":1.083},"min":{"1min":0.79100000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.077,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.04}]},{"osd":3,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.284,"15min":1.1859999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0470000000000002,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.339,"15min":1.2490000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1110000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2529999999999999,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6479999999999999,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3740000000000001,"15min":1.286},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4019999999999999,"15min":1.304},"min":{"1min":0.81499999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2869999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.466,"15min":1.395},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4750000000000001,"15min":1.389},"min":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.504,"15min":1.454},"min":{"1min":0.86099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4900000000000002,"5min":2.698,"15min":2.698},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194621,"statfs":{"total":200000000,"available":199292449,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.70999999999999996},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3480000000000001,"5min":1.619,"15min":2.6040000000000001},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94699999999999995,"15min":0.90100000000000002},"min":{"1min":0.497,"5min":0.46600000000000003,"15min":0.39700000000000002},"max":{"1min":1.4430000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.014,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.516,"5min":1.6359999999999999,"15min":2.125},"last":1.5209999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.034,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4079999999999999,"5min":1.611,"15min":1.611},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":0.89400000000000002},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":1.353}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.72699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.593,"5min":1.7110000000000001,"15min":3.585},"last":1.204},{"interface":"front","average":{"1min":1.165,"5min":1.2529999999999999,"15min":1.206},"min":{"1min":0.70499999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5249999999999999,"5min":1.6659999999999999,"15min":1.877},"last":1.0309999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.202,"15min":1.218},"min":{"1min":0.64400000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3819999999999999,"5min":3.738,"15min":3.738},"last":1.242},{"interface":"front","average":{"1min":1.24,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.754,"5min":1.851,"15min":3.4950000000000001},"last":1.6299999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.2689999999999999},"min":{"1min":0.81299999999999994,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.2989999999999999},"min":{"1min":0.80400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.462,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.5629999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2829999999999999,"15min":1.2629999999999999},"min":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6830000000000001,"5min":1.869,"15min":3.391},"last":1.7},{"interface":"front","average":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3180000000000001},"min":{"1min":0.80900000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6379999999999999,"5min":1.9199999999999999,"15min":2.0409999999999999},"last":1.597}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294001,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4139999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.79900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.30299999999999999},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.91200000000000003},"min":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.441},"max":{"1min":2.153,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.002,"15min":0.874},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.5369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.81000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.9790000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.077,"15min":0.996},"min":{"1min":0.45500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.296,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.748,"15min":0.748},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.27,"5min":1.26,"15min":1.1699999999999999},"min":{"1min":0.81899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.887,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.503}]},{"osd":5,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2490000000000001,"15min":1.1040000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.327,"15min":1.28},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.627}]},{"osd":6,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.196},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3,"15min":1.212},"min":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.367,"15min":1.2250000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.98999999999999999},"min":{"1min":0.624,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.113,"5min":1.425,"15min":3.5680000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91600000000000004,"15min":1.333},"min":{"1min":0.54700000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.355,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.91000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.133,"5min":1.431,"15min":1.639},"last":0.88},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":1.0820000000000001},"min":{"1min":0.59699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3999999999999999,"5min":1.823,"15min":3.0259999999999998},"last":1.1399999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.046,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.5549999999999999,"5min":1.647,"15min":3.1539999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.26},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.7030000000000001,"15min":4.8760000000000003},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1220000000000001,"15min":1.306},"min":{"1min":0.70499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4490000000000001,"5min":1.837,"15min":2.9449999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.3580000000000001},"min":{"1min":0.94399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.722,"5min":1.772,"15min":3.2970000000000002},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2929999999999999,"15min":1.448},"min":{"1min":1.085,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.589,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.278,"15min":1.4279999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5,"5min":1.8959999999999999,"15min":3.0249999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.282,"15min":1.5509999999999999},"min":{"1min":1.0069999999999999,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.304,"15min":1.4790000000000001},"min":{"1min":0.96899999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4039999999999999,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.056}]},{"osd":7,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3540000000000001,"15min":1.542},"min":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7889999999999999,"5min":2.1469999999999998,"15min":3.4689999999999999},"last":1.248},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.363,"15min":1.6000000000000001},"min":{"1min":0.88700000000000001,"5min":0.86699999999999999,"15min":0.78000000000000003},"max":{"1min":1.6240000000000001,"5min":2.1019999999999999,"15min":4.3040000000000003},"last":1.2789999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1265,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198704104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77800000000000002,"15min":1.101},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2030000000000001,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89600000000000002,"15min":1.444},"min":{"1min":0.432,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92900000000000005,"15min":1.151},"min":{"1min":0.48999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4219999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92500000000000004,"15min":1.4390000000000001},"min":{"1min":0.47999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4079999999999999,"5min":1.706,"15min":10.781000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":1.2050000000000001},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1970000000000001,"5min":1.427,"15min":5.8410000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.292},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.413,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1719999999999999,"15min":1.3320000000000001},"min":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.669},"last":1.228},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.397},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.895,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.256,"15min":1.7829999999999999},"min":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.2350000000000001,"15min":1.413},"min":{"1min":0.92100000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6859999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2969999999999999,"15min":1.4610000000000001},"min":{"1min":0.77000000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.419},{"interface":"front","average":{"1min":1.302,"5min":1.2749999999999999,"15min":1.8160000000000001},"min":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":10.708},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3420000000000001,"15min":1.833},"min":{"1min":0.873,"5min":0.82499999999999996,"15min":0.70999999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":10.74},"last":1.518},{"interface":"front","average":{"1min":1.296,"5min":1.2869999999999999,"15min":1.4379999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.869,"5min":1.869,"15min":3.863},"last":1.278}]}]}],"pool_statfs":[]}} 2024-02-02T13:31:22.797 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-02-02T13:31:23.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.014+0000 7f324b800640 1 Processor -- start 2024-02-02T13:31:23.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.014+0000 7f324b800640 1 -- start start 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.016+0000 7f324b800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f324415d3a0 0x7f32440ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f324b800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32440ac960 0x7f32440acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f324b800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440c94a0 con 0x7f32440cb580 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f324b800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440ad6e0 con 0x7f324415d3a0 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f324b800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440c96f0 con 0x7f32440ac960 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f324415d3a0 0x7f32440ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f324415d3a0 0x7f32440ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49692/0 (socket says 172.21.15.91:49692) 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 -- 172.21.15.91:0/468274467 learned_addr learned my addr 172.21.15.91:0/468274467 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3249800640 1 --2- 172.21.15.91:0/468274467 >> v2:172.21.15.113:3300/0 conn(0x7f32440ac960 0x7f32440acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 -- 172.21.15.91:0/468274467 >> v2:172.21.15.113:3300/0 conn(0x7f32440ac960 msgr2=0x7f32440acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 --2- 172.21.15.91:0/468274467 >> v2:172.21.15.113:3300/0 conn(0x7f32440ac960 0x7f32440acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 -- 172.21.15.91:0/468274467 >> v1:172.21.15.113:6789/0 conn(0x7f32440cb580 legacy=0x7f32440cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:23.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.017+0000 7f3243e00640 1 -- 172.21.15.91:0/468274467 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440c99d0 con 0x7f324415d3a0 2024-02-02T13:31:23.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.018+0000 7f3243e00640 1 --2- 172.21.15.91:0/468274467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f324415d3a0 0x7f32440ca1d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3244071a30 tx=0x7f323402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9bf247e60fa455 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f3243400640 1 -- 172.21.15.91:0/468274467 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234031870 con 0x7f324415d3a0 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f3243400640 1 -- 172.21.15.91:0/468274467 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340319d0 con 0x7f324415d3a0 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f3243400640 1 -- 172.21.15.91:0/468274467 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234031d60 con 0x7f324415d3a0 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 -- 172.21.15.91:0/468274467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f324415d3a0 msgr2=0x7f32440ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 --2- 172.21.15.91:0/468274467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f324415d3a0 0x7f32440ca1d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3244071a30 tx=0x7f323402b910 comp rx=0 tx=0).stop 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 -- 172.21.15.91:0/468274467 shutdown_connections 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 --2- 172.21.15.91:0/468274467 >> v2:172.21.15.113:3300/0 conn(0x7f32440ac960 0x7f32440acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 --2- 172.21.15.91:0/468274467 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f324415d3a0 0x7f32440ca1d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.019+0000 7f324b800640 1 -- 172.21.15.91:0/468274467 >> 172.21.15.91:0/468274467 conn(0x7f32440b0700 msgr2=0x7f32440b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.020+0000 7f324b800640 1 -- 172.21.15.91:0/468274467 shutdown_connections 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.020+0000 7f324b800640 1 -- 172.21.15.91:0/468274467 wait complete. 2024-02-02T13:31:23.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.021+0000 7f324b800640 1 Processor -- start 2024-02-02T13:31:23.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.021+0000 7f324b800640 1 -- start start 2024-02-02T13:31:23.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 0x7f32441562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324415d3a0 0x7f32441554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c6da0 con 0x7f32440cb580 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c71c0 con 0x7f324415d3a0 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f324b800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c6120 con 0x7f32440ac960 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f3243e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f3243e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49698/0 (socket says 172.21.15.91:49698) 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.022+0000 7f3243e00640 1 -- 172.21.15.91:0/2151846928 learned_addr learned my addr 172.21.15.91:0/2151846928 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 -- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 msgr2=0x7f32441562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3248e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 0x7f32441562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3249800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324415d3a0 0x7f32441554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 0x7f32441562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 -- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324415d3a0 msgr2=0x7f32441554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324415d3a0 0x7f32441554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 -- 172.21.15.91:0/2151846928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32340094d0 con 0x7f32440cb580 2024-02-02T13:31:23.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3248e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 0x7f32441562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:23.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3243e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f323402b950 tx=0x7f32340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.023+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323403ac60 con 0x7f32440cb580 2024-02-02T13:31:23.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.024+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323403adc0 con 0x7f32440cb580 2024-02-02T13:31:23.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.024+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234044630 con 0x7f32440cb580 2024-02-02T13:31:23.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.024+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440ad4b0 con 0x7f32440cb580 2024-02-02T13:31:23.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.024+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32440c66e0 con 0x7f32440cb580 2024-02-02T13:31:23.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.025+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32440c6ee0 con 0x7f32440cb580 2024-02-02T13:31:23.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.026+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f323404d450 con 0x7f32440cb580 2024-02-02T13:31:23.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.026+0000 7f3240c00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 0x7f321c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.026+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f323402d040 con 0x7f32440cb580 2024-02-02T13:31:23.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.027+0000 7f3248e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 0x7f321c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.027+0000 7f3248e00640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 0x7f321c06c8f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f322c003eb0 tx=0x7f322c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.031+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f32440c6ee0 con 0x7f32440cb580 2024-02-02T13:31:23.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.291+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f32440c6ee0 con 0x7f32440cb580 2024-02-02T13:31:23.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.292+0000 7f3240c00640 1 -- 172.21.15.91:0/2151846928 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1168 (secure 0 0 0) 0x7f32440c6ee0 con 0x7f32440cb580 2024-02-02T13:31:23.294 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:23.294 INFO:teuthology.orchestra.run.smithi091.stdout:{"fsid":"cc006970-f900-4781-8947-81ed22cd959b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":424,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":128,"num_osds":8,"num_up_osds":8,"osd_up_since":1706880265,"num_in_osds":8,"osd_in_since":1706880262,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":6829104,"bytes_avail":1593170896,"bytes_total":1600000000,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":5578}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-02-02T13:30:02.344210+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 msgr2=0x7f321c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 0x7f321c06c8f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f322c003eb0 tx=0x7f322c009040 comp rx=0 tx=0).stop 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 msgr2=0x7f32440ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f323402b950 tx=0x7f32340046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 shutdown_connections 2024-02-02T13:31:23.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.295+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f321c06a430 0x7f321c06c8f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324415d3a0 0x7f32441554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32440cb580 0x7f32440ca650 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 --2- 172.21.15.91:0/2151846928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32440ac960 0x7f32441562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 >> 172.21.15.91:0/2151846928 conn(0x7f32440b0700 msgr2=0x7f32440ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 shutdown_connections 2024-02-02T13:31:23.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.296+0000 7f324b800640 1 -- 172.21.15.91:0/2151846928 wait complete. 2024-02-02T13:31:23.315 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-02-02T13:31:23.316 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:31:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.517+0000 7f723fa00640 1 Processor -- start 2024-02-02T13:31:23.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.517+0000 7f723fa00640 1 -- start start 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.518+0000 7f723fa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72381597a0 0x7f723806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.518+0000 7f723fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72380c9950 0x7f72380cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.518+0000 7f723fa00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723806b5f0 con 0x7f72380cd690 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.518+0000 7f723fa00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723806c450 con 0x7f72381597a0 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.518+0000 7f723fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7238159b80 con 0x7f72380c9950 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72381597a0 0x7f723806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72381597a0 0x7f723806c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49710/0 (socket says 172.21.15.91:49710) 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 -- 172.21.15.91:0/4090388788 learned_addr learned my addr 172.21.15.91:0/4090388788 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723da00640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72380c9950 0x7f72380cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 -- 172.21.15.91:0/4090388788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72380c9950 msgr2=0x7f72380cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72380c9950 0x7f72380cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 -- 172.21.15.91:0/4090388788 >> v1:172.21.15.91:6789/0 conn(0x7f72380cd690 legacy=0x7f72380c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.519+0000 7f723d000640 1 -- 172.21.15.91:0/4090388788 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723806b840 con 0x7f72381597a0 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.520+0000 7f723d000640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72381597a0 0x7f723806c8e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f7228009870 tx=0x7f722802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfea26a56b794219 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f7237400640 1 -- 172.21.15.91:0/4090388788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722802e850 con 0x7f72381597a0 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f7237400640 1 -- 172.21.15.91:0/4090388788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722802e9b0 con 0x7f72381597a0 2024-02-02T13:31:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f7237400640 1 -- 172.21.15.91:0/4090388788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722802ed40 con 0x7f72381597a0 2024-02-02T13:31:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 -- 172.21.15.91:0/4090388788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72381597a0 msgr2=0x7f723806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72381597a0 0x7f723806c8e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f7228009870 tx=0x7f722802ca90 comp rx=0 tx=0).stop 2024-02-02T13:31:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 -- 172.21.15.91:0/4090388788 shutdown_connections 2024-02-02T13:31:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72380c9950 0x7f72380cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 --2- 172.21.15.91:0/4090388788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72381597a0 0x7f723806c8e0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.521+0000 7f723fa00640 1 -- 172.21.15.91:0/4090388788 >> 172.21.15.91:0/4090388788 conn(0x7f72380cec80 msgr2=0x7f72380cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.522+0000 7f723fa00640 1 -- 172.21.15.91:0/4090388788 shutdown_connections 2024-02-02T13:31:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.522+0000 7f723fa00640 1 -- 172.21.15.91:0/4090388788 wait complete. 2024-02-02T13:31:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.523+0000 7f723fa00640 1 Processor -- start 2024-02-02T13:31:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.523+0000 7f723fa00640 1 -- start start 2024-02-02T13:31:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72380c9950 0x7f7238154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72381597a0 0x7f72381e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c6bd0 con 0x7f72380cd690 2024-02-02T13:31:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c4a20 con 0x7f72380c9950 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f723fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c4d00 con 0x7f72381597a0 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f7237e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f7237e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49718/0 (socket says 172.21.15.91:49718) 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.524+0000 7f7237e00640 1 -- 172.21.15.91:0/2227520744 learned_addr learned my addr 172.21.15.91:0/2227520744 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 -- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72381597a0 msgr2=0x7f72381e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72381597a0 0x7f72381e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 -- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72380c9950 msgr2=0x7f7238154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72380c9950 0x7f7238154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 -- 172.21.15.91:0/2227520744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72280094d0 con 0x7f72380cd690 2024-02-02T13:31:23.527 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7237e00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f722000c450 tx=0x7f722000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200052d0 con 0x7f72380cd690 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220005430 con 0x7f72380cd690 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220012870 con 0x7f72380cd690 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.525+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380c4fe0 con 0x7f72380cd690 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.526+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238155040 con 0x7f72380cd690 2024-02-02T13:31:23.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.527+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238066cf0 con 0x7f72380cd690 2024-02-02T13:31:23.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.527+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f72200129d0 con 0x7f72380cd690 2024-02-02T13:31:23.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.528+0000 7f7234c00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 0x7f721006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:23.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.528+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72200883f0 con 0x7f72380cd690 2024-02-02T13:31:23.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.529+0000 7f723d000640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 0x7f721006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:23.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.529+0000 7f723d000640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 0x7f721006c920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7228037000 tx=0x7f7228031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:23.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.533+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f72200539d0 con 0x7f72380cd690 2024-02-02T13:31:23.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.703+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f723806be00 con 0x7f721006a460 2024-02-02T13:31:23.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.724+0000 7f7234c00640 1 -- 172.21.15.91:0/2227520744 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283307 (secure 0 0 0) 0x7f722802e6d0 con 0x7f721006a460 2024-02-02T13:31:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:23.732 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.731+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 msgr2=0x7f721006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.731+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 0x7f721006c920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7228037000 tx=0x7f7228031000 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.731+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 msgr2=0x7f72381e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.731+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f722000c450 tx=0x7f722000c910 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 shutdown_connections 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f721006a460 0x7f721006c920 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72381597a0 0x7f72381e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72380cd690 0x7f72381e75f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 --2- 172.21.15.91:0/2227520744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72380c9950 0x7f7238154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 >> 172.21.15.91:0/2227520744 conn(0x7f72380cec80 msgr2=0x7f72381e3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:23.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 shutdown_connections 2024-02-02T13:31:23.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:23.732+0000 7f723fa00640 1 -- 172.21.15.91:0/2227520744 wait complete. 2024-02-02T13:31:23.750 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-02-02T13:31:22.369496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555830,"statfs":{"total":1600000000,"available":1593170896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001849"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902784+0000","last_change":"2024-02-02T13:31:16.918834+0000","last_active":"2024-02-02T13:31:17.902784+0000","last_peered":"2024-02-02T13:31:17.902784+0000","last_clean":"2024-02-02T13:31:17.902784+0000","last_became_active":"2024-02-02T13:31:16.918441+0000","last_became_peered":"2024-02-02T13:31:16.918441+0000","last_unstale":"2024-02-02T13:31:17.902784+0000","last_undegraded":"2024-02-02T13:31:17.902784+0000","last_fullsized":"2024-02-02T13:31:17.902784+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:49:00.816989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906309+0000","last_change":"2024-02-02T13:31:16.927818+0000","last_active":"2024-02-02T13:31:17.906309+0000","last_peered":"2024-02-02T13:31:17.906309+0000","last_clean":"2024-02-02T13:31:17.906309+0000","last_became_active":"2024-02-02T13:31:16.927553+0000","last_became_peered":"2024-02-02T13:31:16.927553+0000","last_unstale":"2024-02-02T13:31:17.906309+0000","last_undegraded":"2024-02-02T13:31:17.906309+0000","last_fullsized":"2024-02-02T13:31:17.906309+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902583+0000","last_change":"2024-02-02T13:31:16.925847+0000","last_active":"2024-02-02T13:31:17.902583+0000","last_peered":"2024-02-02T13:31:17.902583+0000","last_clean":"2024-02-02T13:31:17.902583+0000","last_became_active":"2024-02-02T13:31:16.925615+0000","last_became_peered":"2024-02-02T13:31:16.925615+0000","last_unstale":"2024-02-02T13:31:17.902583+0000","last_undegraded":"2024-02-02T13:31:17.902583+0000","last_fullsized":"2024-02-02T13:31:17.902583+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:42:48.173630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860390+0000","last_change":"2024-02-02T13:31:16.907328+0000","last_active":"2024-02-02T13:31:18.860390+0000","last_peered":"2024-02-02T13:31:18.860390+0000","last_clean":"2024-02-02T13:31:18.860390+0000","last_became_active":"2024-02-02T13:31:16.907025+0000","last_became_peered":"2024-02-02T13:31:16.907025+0000","last_unstale":"2024-02-02T13:31:18.860390+0000","last_undegraded":"2024-02-02T13:31:18.860390+0000","last_fullsized":"2024-02-02T13:31:18.860390+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411041+0000","last_change":"2024-02-02T13:31:16.911577+0000","last_active":"2024-02-02T13:31:18.411041+0000","last_peered":"2024-02-02T13:31:18.411041+0000","last_clean":"2024-02-02T13:31:18.411041+0000","last_became_active":"2024-02-02T13:31:16.911051+0000","last_became_peered":"2024-02-02T13:31:16.911051+0000","last_unstale":"2024-02-02T13:31:18.411041+0000","last_undegraded":"2024-02-02T13:31:18.411041+0000","last_fullsized":"2024-02-02T13:31:18.411041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678686+0000","last_change":"2024-02-02T13:31:16.933309+0000","last_active":"2024-02-02T13:31:18.678686+0000","last_peered":"2024-02-02T13:31:18.678686+0000","last_clean":"2024-02-02T13:31:18.678686+0000","last_became_active":"2024-02-02T13:31:16.932957+0000","last_became_peered":"2024-02-02T13:31:16.932957+0000","last_unstale":"2024-02-02T13:31:18.678686+0000","last_undegraded":"2024-02-02T13:31:18.678686+0000","last_fullsized":"2024-02-02T13:31:18.678686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678369+0000","last_change":"2024-02-02T13:31:16.929320+0000","last_active":"2024-02-02T13:31:18.678369+0000","last_peered":"2024-02-02T13:31:18.678369+0000","last_clean":"2024-02-02T13:31:18.678369+0000","last_became_active":"2024-02-02T13:31:16.929069+0000","last_became_peered":"2024-02-02T13:31:16.929069+0000","last_unstale":"2024-02-02T13:31:18.678369+0000","last_undegraded":"2024-02-02T13:31:18.678369+0000","last_fullsized":"2024-02-02T13:31:18.678369+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907186+0000","last_change":"2024-02-02T13:31:16.926583+0000","last_active":"2024-02-02T13:31:17.907186+0000","last_peered":"2024-02-02T13:31:17.907186+0000","last_clean":"2024-02-02T13:31:17.907186+0000","last_became_active":"2024-02-02T13:31:16.926431+0000","last_became_peered":"2024-02-02T13:31:16.926431+0000","last_unstale":"2024-02-02T13:31:17.907186+0000","last_undegraded":"2024-02-02T13:31:17.907186+0000","last_fullsized":"2024-02-02T13:31:17.907186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380086+0000","last_change":"2024-02-02T13:31:16.920257+0000","last_active":"2024-02-02T13:31:20.380086+0000","last_peered":"2024-02-02T13:31:20.380086+0000","last_clean":"2024-02-02T13:31:20.380086+0000","last_became_active":"2024-02-02T13:31:16.919808+0000","last_became_peered":"2024-02-02T13:31:16.919808+0000","last_unstale":"2024-02-02T13:31:20.380086+0000","last_undegraded":"2024-02-02T13:31:20.380086+0000","last_fullsized":"2024-02-02T13:31:20.380086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:20.515102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411221+0000","last_change":"2024-02-02T13:31:16.918453+0000","last_active":"2024-02-02T13:31:18.411221+0000","last_peered":"2024-02-02T13:31:18.411221+0000","last_clean":"2024-02-02T13:31:18.411221+0000","last_became_active":"2024-02-02T13:31:16.917549+0000","last_became_peered":"2024-02-02T13:31:16.917549+0000","last_unstale":"2024-02-02T13:31:18.411221+0000","last_undegraded":"2024-02-02T13:31:18.411221+0000","last_fullsized":"2024-02-02T13:31:18.411221+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678917+0000","last_change":"2024-02-02T13:31:16.925898+0000","last_active":"2024-02-02T13:31:18.678917+0000","last_peered":"2024-02-02T13:31:18.678917+0000","last_clean":"2024-02-02T13:31:18.678917+0000","last_became_active":"2024-02-02T13:31:16.925597+0000","last_became_peered":"2024-02-02T13:31:16.925597+0000","last_unstale":"2024-02-02T13:31:18.678917+0000","last_undegraded":"2024-02-02T13:31:18.678917+0000","last_fullsized":"2024-02-02T13:31:18.678917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380719+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:20.380719+0000","last_peered":"2024-02-02T13:31:20.380719+0000","last_clean":"2024-02-02T13:31:20.380719+0000","last_became_active":"2024-02-02T13:31:16.923735+0000","last_became_peered":"2024-02-02T13:31:16.923735+0000","last_unstale":"2024-02-02T13:31:20.380719+0000","last_undegraded":"2024-02-02T13:31:20.380719+0000","last_fullsized":"2024-02-02T13:31:20.380719+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:17.618621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.859985+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:18.859985+0000","last_peered":"2024-02-02T13:31:18.859985+0000","last_clean":"2024-02-02T13:31:18.859985+0000","last_became_active":"2024-02-02T13:31:16.924861+0000","last_became_peered":"2024-02-02T13:31:16.924861+0000","last_unstale":"2024-02-02T13:31:18.859985+0000","last_undegraded":"2024-02-02T13:31:18.859985+0000","last_fullsized":"2024-02-02T13:31:18.859985+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906357+0000","last_change":"2024-02-02T13:31:16.914250+0000","last_active":"2024-02-02T13:31:17.906357+0000","last_peered":"2024-02-02T13:31:17.906357+0000","last_clean":"2024-02-02T13:31:17.906357+0000","last_became_active":"2024-02-02T13:31:16.914003+0000","last_became_peered":"2024-02-02T13:31:16.914003+0000","last_unstale":"2024-02-02T13:31:17.906357+0000","last_undegraded":"2024-02-02T13:31:17.906357+0000","last_fullsized":"2024-02-02T13:31:17.906357+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:03:38.355640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860441+0000","last_change":"2024-02-02T13:31:16.906982+0000","last_active":"2024-02-02T13:31:18.860441+0000","last_peered":"2024-02-02T13:31:18.860441+0000","last_clean":"2024-02-02T13:31:18.860441+0000","last_became_active":"2024-02-02T13:31:16.906771+0000","last_became_peered":"2024-02-02T13:31:16.906771+0000","last_unstale":"2024-02-02T13:31:18.860441+0000","last_undegraded":"2024-02-02T13:31:18.860441+0000","last_fullsized":"2024-02-02T13:31:18.860441+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679128+0000","last_change":"2024-02-02T13:31:16.929883+0000","last_active":"2024-02-02T13:31:18.679128+0000","last_peered":"2024-02-02T13:31:18.679128+0000","last_clean":"2024-02-02T13:31:18.679128+0000","last_became_active":"2024-02-02T13:31:16.929203+0000","last_became_peered":"2024-02-02T13:31:16.929203+0000","last_unstale":"2024-02-02T13:31:18.679128+0000","last_undegraded":"2024-02-02T13:31:18.679128+0000","last_fullsized":"2024-02-02T13:31:18.679128+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906561+0000","last_change":"2024-02-02T13:31:16.920007+0000","last_active":"2024-02-02T13:31:17.906561+0000","last_peered":"2024-02-02T13:31:17.906561+0000","last_clean":"2024-02-02T13:31:17.906561+0000","last_became_active":"2024-02-02T13:31:16.919625+0000","last_became_peered":"2024-02-02T13:31:16.919625+0000","last_unstale":"2024-02-02T13:31:17.906561+0000","last_undegraded":"2024-02-02T13:31:17.906561+0000","last_fullsized":"2024-02-02T13:31:17.906561+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:19:35.465397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860275+0000","last_change":"2024-02-02T13:31:16.910385+0000","last_active":"2024-02-02T13:31:18.860275+0000","last_peered":"2024-02-02T13:31:18.860275+0000","last_clean":"2024-02-02T13:31:18.860275+0000","last_became_active":"2024-02-02T13:31:16.909793+0000","last_became_peered":"2024-02-02T13:31:16.909793+0000","last_unstale":"2024-02-02T13:31:18.860275+0000","last_undegraded":"2024-02-02T13:31:18.860275+0000","last_fullsized":"2024-02-02T13:31:18.860275+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906609+0000","last_change":"2024-02-02T13:31:16.909676+0000","last_active":"2024-02-02T13:31:17.906609+0000","last_peered":"2024-02-02T13:31:17.906609+0000","last_clean":"2024-02-02T13:31:17.906609+0000","last_became_active":"2024-02-02T13:31:16.909474+0000","last_became_peered":"2024-02-02T13:31:16.909474+0000","last_unstale":"2024-02-02T13:31:17.906609+0000","last_undegraded":"2024-02-02T13:31:17.906609+0000","last_fullsized":"2024-02-02T13:31:17.906609+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:58.879647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380271+0000","last_change":"2024-02-02T13:31:16.921784+0000","last_active":"2024-02-02T13:31:20.380271+0000","last_peered":"2024-02-02T13:31:20.380271+0000","last_clean":"2024-02-02T13:31:20.380271+0000","last_became_active":"2024-02-02T13:31:16.920919+0000","last_became_peered":"2024-02-02T13:31:16.920919+0000","last_unstale":"2024-02-02T13:31:20.380271+0000","last_undegraded":"2024-02-02T13:31:20.380271+0000","last_fullsized":"2024-02-02T13:31:20.380271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:31.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906361+0000","last_change":"2024-02-02T13:31:16.917770+0000","last_active":"2024-02-02T13:31:17.906361+0000","last_peered":"2024-02-02T13:31:17.906361+0000","last_clean":"2024-02-02T13:31:17.906361+0000","last_became_active":"2024-02-02T13:31:16.917542+0000","last_became_peered":"2024-02-02T13:31:16.917542+0000","last_unstale":"2024-02-02T13:31:17.906361+0000","last_undegraded":"2024-02-02T13:31:17.906361+0000","last_fullsized":"2024-02-02T13:31:17.906361+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.721834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906902+0000","last_change":"2024-02-02T13:31:16.927694+0000","last_active":"2024-02-02T13:31:17.906902+0000","last_peered":"2024-02-02T13:31:17.906902+0000","last_clean":"2024-02-02T13:31:17.906902+0000","last_became_active":"2024-02-02T13:31:16.927267+0000","last_became_peered":"2024-02-02T13:31:16.927267+0000","last_unstale":"2024-02-02T13:31:17.906902+0000","last_undegraded":"2024-02-02T13:31:17.906902+0000","last_fullsized":"2024-02-02T13:31:17.906902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907431+0000","last_change":"2024-02-02T13:31:16.931926+0000","last_active":"2024-02-02T13:31:17.907431+0000","last_peered":"2024-02-02T13:31:17.907431+0000","last_clean":"2024-02-02T13:31:17.907431+0000","last_became_active":"2024-02-02T13:31:16.931680+0000","last_became_peered":"2024-02-02T13:31:16.931680+0000","last_unstale":"2024-02-02T13:31:17.907431+0000","last_undegraded":"2024-02-02T13:31:17.907431+0000","last_fullsized":"2024-02-02T13:31:17.907431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678475+0000","last_change":"2024-02-02T13:31:16.920779+0000","last_active":"2024-02-02T13:31:18.678475+0000","last_peered":"2024-02-02T13:31:18.678475+0000","last_clean":"2024-02-02T13:31:18.678475+0000","last_became_active":"2024-02-02T13:31:16.918630+0000","last_became_peered":"2024-02-02T13:31:16.918630+0000","last_unstale":"2024-02-02T13:31:18.678475+0000","last_undegraded":"2024-02-02T13:31:18.678475+0000","last_fullsized":"2024-02-02T13:31:18.678475+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411470+0000","last_change":"2024-02-02T13:31:16.923782+0000","last_active":"2024-02-02T13:31:18.411470+0000","last_peered":"2024-02-02T13:31:18.411470+0000","last_clean":"2024-02-02T13:31:18.411470+0000","last_became_active":"2024-02-02T13:31:16.923387+0000","last_became_peered":"2024-02-02T13:31:16.923387+0000","last_unstale":"2024-02-02T13:31:18.411470+0000","last_undegraded":"2024-02-02T13:31:18.411470+0000","last_fullsized":"2024-02-02T13:31:18.411470+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907364+0000","last_change":"2024-02-02T13:31:16.920823+0000","last_active":"2024-02-02T13:31:17.907364+0000","last_peered":"2024-02-02T13:31:17.907364+0000","last_clean":"2024-02-02T13:31:17.907364+0000","last_became_active":"2024-02-02T13:31:16.920636+0000","last_became_peered":"2024-02-02T13:31:16.920636+0000","last_unstale":"2024-02-02T13:31:17.907364+0000","last_undegraded":"2024-02-02T13:31:17.907364+0000","last_fullsized":"2024-02-02T13:31:17.907364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678635+0000","last_change":"2024-02-02T13:31:16.922205+0000","last_active":"2024-02-02T13:31:18.678635+0000","last_peered":"2024-02-02T13:31:18.678635+0000","last_clean":"2024-02-02T13:31:18.678635+0000","last_became_active":"2024-02-02T13:31:16.921225+0000","last_became_peered":"2024-02-02T13:31:16.921225+0000","last_unstale":"2024-02-02T13:31:18.678635+0000","last_undegraded":"2024-02-02T13:31:18.678635+0000","last_fullsized":"2024-02-02T13:31:18.678635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410942+0000","last_change":"2024-02-02T13:31:16.919017+0000","last_active":"2024-02-02T13:31:18.410942+0000","last_peered":"2024-02-02T13:31:18.410942+0000","last_clean":"2024-02-02T13:31:18.410942+0000","last_became_active":"2024-02-02T13:31:16.913456+0000","last_became_peered":"2024-02-02T13:31:16.913456+0000","last_unstale":"2024-02-02T13:31:18.410942+0000","last_undegraded":"2024-02-02T13:31:18.410942+0000","last_fullsized":"2024-02-02T13:31:18.410942+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.905879+0000","last_change":"2024-02-02T13:31:16.920909+0000","last_active":"2024-02-02T13:31:17.905879+0000","last_peered":"2024-02-02T13:31:17.905879+0000","last_clean":"2024-02-02T13:31:17.905879+0000","last_became_active":"2024-02-02T13:31:16.920620+0000","last_became_peered":"2024-02-02T13:31:16.920620+0000","last_unstale":"2024-02-02T13:31:17.905879+0000","last_undegraded":"2024-02-02T13:31:17.905879+0000","last_fullsized":"2024-02-02T13:31:17.905879+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:47:29.170966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907939+0000","last_change":"2024-02-02T13:31:16.915569+0000","last_active":"2024-02-02T13:31:17.907939+0000","last_peered":"2024-02-02T13:31:17.907939+0000","last_clean":"2024-02-02T13:31:17.907939+0000","last_became_active":"2024-02-02T13:31:16.914947+0000","last_became_peered":"2024-02-02T13:31:16.914947+0000","last_unstale":"2024-02-02T13:31:17.907939+0000","last_undegraded":"2024-02-02T13:31:17.907939+0000","last_fullsized":"2024-02-02T13:31:17.907939+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:07.892976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860126+0000","last_change":"2024-02-02T13:31:16.908445+0000","last_active":"2024-02-02T13:31:18.860126+0000","last_peered":"2024-02-02T13:31:18.860126+0000","last_clean":"2024-02-02T13:31:18.860126+0000","last_became_active":"2024-02-02T13:31:16.908180+0000","last_became_peered":"2024-02-02T13:31:16.908180+0000","last_unstale":"2024-02-02T13:31:18.860126+0000","last_undegraded":"2024-02-02T13:31:18.860126+0000","last_fullsized":"2024-02-02T13:31:18.860126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902747+0000","last_change":"2024-02-02T13:31:16.921843+0000","last_active":"2024-02-02T13:31:17.902747+0000","last_peered":"2024-02-02T13:31:17.902747+0000","last_clean":"2024-02-02T13:31:17.902747+0000","last_became_active":"2024-02-02T13:31:16.921627+0000","last_became_peered":"2024-02-02T13:31:16.921627+0000","last_unstale":"2024-02-02T13:31:17.902747+0000","last_undegraded":"2024-02-02T13:31:17.902747+0000","last_fullsized":"2024-02-02T13:31:17.902747+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:58:17.957284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906836+0000","last_change":"2024-02-02T13:31:16.918012+0000","last_active":"2024-02-02T13:31:17.906836+0000","last_peered":"2024-02-02T13:31:17.906836+0000","last_clean":"2024-02-02T13:31:17.906836+0000","last_became_active":"2024-02-02T13:31:16.917805+0000","last_became_peered":"2024-02-02T13:31:16.917805+0000","last_unstale":"2024-02-02T13:31:17.906836+0000","last_undegraded":"2024-02-02T13:31:17.906836+0000","last_fullsized":"2024-02-02T13:31:17.906836+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:47.224031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"127'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925207+0000","last_change":"2024-02-02T13:31:16.433503+0000","last_active":"2024-02-02T13:31:16.925207+0000","last_peered":"2024-02-02T13:31:16.925207+0000","last_clean":"2024-02-02T13:31:16.925207+0000","last_became_active":"2024-02-02T13:31:16.433277+0000","last_became_peered":"2024-02-02T13:31:16.433277+0000","last_unstale":"2024-02-02T13:31:16.925207+0000","last_undegraded":"2024-02-02T13:31:16.925207+0000","last_fullsized":"2024-02-02T13:31:16.925207+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:16:12.721762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860526+0000","last_change":"2024-02-02T13:31:16.919617+0000","last_active":"2024-02-02T13:31:18.860526+0000","last_peered":"2024-02-02T13:31:18.860526+0000","last_clean":"2024-02-02T13:31:18.860526+0000","last_became_active":"2024-02-02T13:31:16.919275+0000","last_became_peered":"2024-02-02T13:31:16.919275+0000","last_unstale":"2024-02-02T13:31:18.860526+0000","last_undegraded":"2024-02-02T13:31:18.860526+0000","last_fullsized":"2024-02-02T13:31:18.860526+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931501+0000","last_change":"2024-02-02T13:31:16.424527+0000","last_active":"2024-02-02T13:31:16.931501+0000","last_peered":"2024-02-02T13:31:16.931501+0000","last_clean":"2024-02-02T13:31:16.931501+0000","last_became_active":"2024-02-02T13:31:16.423926+0000","last_became_peered":"2024-02-02T13:31:16.423926+0000","last_unstale":"2024-02-02T13:31:16.931501+0000","last_undegraded":"2024-02-02T13:31:16.931501+0000","last_fullsized":"2024-02-02T13:31:16.931501+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903041+0000","last_change":"2024-02-02T13:31:16.922269+0000","last_active":"2024-02-02T13:31:17.903041+0000","last_peered":"2024-02-02T13:31:17.903041+0000","last_clean":"2024-02-02T13:31:17.903041+0000","last_became_active":"2024-02-02T13:31:16.921999+0000","last_became_peered":"2024-02-02T13:31:16.921999+0000","last_unstale":"2024-02-02T13:31:17.903041+0000","last_undegraded":"2024-02-02T13:31:17.903041+0000","last_fullsized":"2024-02-02T13:31:17.903041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:26:34.152547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.921395+0000","last_change":"2024-02-02T13:31:16.434462+0000","last_active":"2024-02-02T13:31:16.921395+0000","last_peered":"2024-02-02T13:31:16.921395+0000","last_clean":"2024-02-02T13:31:16.921395+0000","last_became_active":"2024-02-02T13:31:16.434216+0000","last_became_peered":"2024-02-02T13:31:16.434216+0000","last_unstale":"2024-02-02T13:31:16.921395+0000","last_undegraded":"2024-02-02T13:31:16.921395+0000","last_fullsized":"2024-02-02T13:31:16.921395+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380530+0000","last_change":"2024-02-02T13:31:16.928736+0000","last_active":"2024-02-02T13:31:20.380530+0000","last_peered":"2024-02-02T13:31:20.380530+0000","last_clean":"2024-02-02T13:31:20.380530+0000","last_became_active":"2024-02-02T13:31:16.928489+0000","last_became_peered":"2024-02-02T13:31:16.928489+0000","last_unstale":"2024-02-02T13:31:20.380530+0000","last_undegraded":"2024-02-02T13:31:20.380530+0000","last_fullsized":"2024-02-02T13:31:20.380530+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:57.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.928318+0000","last_change":"2024-02-02T13:31:16.440048+0000","last_active":"2024-02-02T13:31:16.928318+0000","last_peered":"2024-02-02T13:31:16.928318+0000","last_clean":"2024-02-02T13:31:16.928318+0000","last_became_active":"2024-02-02T13:31:16.439503+0000","last_became_peered":"2024-02-02T13:31:16.439503+0000","last_unstale":"2024-02-02T13:31:16.928318+0000","last_undegraded":"2024-02-02T13:31:16.928318+0000","last_fullsized":"2024-02-02T13:31:16.928318+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.366981+0000","last_change":"2024-02-02T13:31:16.931809+0000","last_active":"2024-02-02T13:31:18.366981+0000","last_peered":"2024-02-02T13:31:18.366981+0000","last_clean":"2024-02-02T13:31:18.366981+0000","last_became_active":"2024-02-02T13:31:16.931497+0000","last_became_peered":"2024-02-02T13:31:16.931497+0000","last_unstale":"2024-02-02T13:31:18.366981+0000","last_undegraded":"2024-02-02T13:31:18.366981+0000","last_fullsized":"2024-02-02T13:31:18.366981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:55:21.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.922362+0000","last_change":"2024-02-02T13:31:16.418697+0000","last_active":"2024-02-02T13:31:16.922362+0000","last_peered":"2024-02-02T13:31:16.922362+0000","last_clean":"2024-02-02T13:31:16.922362+0000","last_became_active":"2024-02-02T13:31:16.417931+0000","last_became_peered":"2024-02-02T13:31:16.417931+0000","last_unstale":"2024-02-02T13:31:16.922362+0000","last_undegraded":"2024-02-02T13:31:16.922362+0000","last_fullsized":"2024-02-02T13:31:16.922362+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:54.011838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678787+0000","last_change":"2024-02-02T13:31:16.925253+0000","last_active":"2024-02-02T13:31:18.678787+0000","last_peered":"2024-02-02T13:31:18.678787+0000","last_clean":"2024-02-02T13:31:18.678787+0000","last_became_active":"2024-02-02T13:31:16.925007+0000","last_became_peered":"2024-02-02T13:31:16.925007+0000","last_unstale":"2024-02-02T13:31:18.678787+0000","last_undegraded":"2024-02-02T13:31:18.678787+0000","last_fullsized":"2024-02-02T13:31:18.678787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924884+0000","last_change":"2024-02-02T13:31:16.437360+0000","last_active":"2024-02-02T13:31:16.924884+0000","last_peered":"2024-02-02T13:31:16.924884+0000","last_clean":"2024-02-02T13:31:16.924884+0000","last_became_active":"2024-02-02T13:31:16.437067+0000","last_became_peered":"2024-02-02T13:31:16.437067+0000","last_unstale":"2024-02-02T13:31:16.924884+0000","last_undegraded":"2024-02-02T13:31:16.924884+0000","last_fullsized":"2024-02-02T13:31:16.924884+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860682+0000","last_change":"2024-02-02T13:31:16.907956+0000","last_active":"2024-02-02T13:31:18.860682+0000","last_peered":"2024-02-02T13:31:18.860682+0000","last_clean":"2024-02-02T13:31:18.860682+0000","last_became_active":"2024-02-02T13:31:16.907664+0000","last_became_peered":"2024-02-02T13:31:16.907664+0000","last_unstale":"2024-02-02T13:31:18.860682+0000","last_undegraded":"2024-02-02T13:31:18.860682+0000","last_fullsized":"2024-02-02T13:31:18.860682+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.909194+0000","last_change":"2024-02-02T13:31:16.435466+0000","last_active":"2024-02-02T13:31:16.909194+0000","last_peered":"2024-02-02T13:31:16.909194+0000","last_clean":"2024-02-02T13:31:16.909194+0000","last_became_active":"2024-02-02T13:31:16.434867+0000","last_became_peered":"2024-02-02T13:31:16.434867+0000","last_unstale":"2024-02-02T13:31:16.909194+0000","last_undegraded":"2024-02-02T13:31:16.909194+0000","last_fullsized":"2024-02-02T13:31:16.909194+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:50:14.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380345+0000","last_change":"2024-02-02T13:31:16.932808+0000","last_active":"2024-02-02T13:31:20.380345+0000","last_peered":"2024-02-02T13:31:20.380345+0000","last_clean":"2024-02-02T13:31:20.380345+0000","last_became_active":"2024-02-02T13:31:16.932559+0000","last_became_peered":"2024-02-02T13:31:16.932559+0000","last_unstale":"2024-02-02T13:31:20.380345+0000","last_undegraded":"2024-02-02T13:31:20.380345+0000","last_fullsized":"2024-02-02T13:31:20.380345+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:21.439554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902293+0000","last_change":"2024-02-02T13:31:16.432947+0000","last_active":"2024-02-02T13:31:16.902293+0000","last_peered":"2024-02-02T13:31:16.902293+0000","last_clean":"2024-02-02T13:31:16.902293+0000","last_became_active":"2024-02-02T13:31:16.432457+0000","last_became_peered":"2024-02-02T13:31:16.432457+0000","last_unstale":"2024-02-02T13:31:16.902293+0000","last_undegraded":"2024-02-02T13:31:16.902293+0000","last_fullsized":"2024-02-02T13:31:16.902293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:06.887358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906804+0000","last_change":"2024-02-02T13:31:16.928592+0000","last_active":"2024-02-02T13:31:17.906804+0000","last_peered":"2024-02-02T13:31:17.906804+0000","last_clean":"2024-02-02T13:31:17.906804+0000","last_became_active":"2024-02-02T13:31:16.928381+0000","last_became_peered":"2024-02-02T13:31:16.928381+0000","last_unstale":"2024-02-02T13:31:17.906804+0000","last_undegraded":"2024-02-02T13:31:17.906804+0000","last_fullsized":"2024-02-02T13:31:17.906804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.900135+0000","last_change":"2024-02-02T13:31:16.435253+0000","last_active":"2024-02-02T13:31:16.900135+0000","last_peered":"2024-02-02T13:31:16.900135+0000","last_clean":"2024-02-02T13:31:16.900135+0000","last_became_active":"2024-02-02T13:31:16.432671+0000","last_became_peered":"2024-02-02T13:31:16.432671+0000","last_unstale":"2024-02-02T13:31:16.900135+0000","last_undegraded":"2024-02-02T13:31:16.900135+0000","last_fullsized":"2024-02-02T13:31:16.900135+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:46.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380428+0000","last_change":"2024-02-02T13:31:16.924607+0000","last_active":"2024-02-02T13:31:20.380428+0000","last_peered":"2024-02-02T13:31:20.380428+0000","last_clean":"2024-02-02T13:31:20.380428+0000","last_became_active":"2024-02-02T13:31:16.923514+0000","last_became_peered":"2024-02-02T13:31:16.923514+0000","last_unstale":"2024-02-02T13:31:20.380428+0000","last_undegraded":"2024-02-02T13:31:20.380428+0000","last_fullsized":"2024-02-02T13:31:20.380428+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:11.217285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902541+0000","last_change":"2024-02-02T13:31:16.438628+0000","last_active":"2024-02-02T13:31:16.902541+0000","last_peered":"2024-02-02T13:31:16.902541+0000","last_clean":"2024-02-02T13:31:16.902541+0000","last_became_active":"2024-02-02T13:31:16.438393+0000","last_became_peered":"2024-02-02T13:31:16.438393+0000","last_unstale":"2024-02-02T13:31:16.902541+0000","last_undegraded":"2024-02-02T13:31:16.902541+0000","last_fullsized":"2024-02-02T13:31:16.902541+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410690+0000","last_change":"2024-02-02T13:31:16.922466+0000","last_active":"2024-02-02T13:31:18.410690+0000","last_peered":"2024-02-02T13:31:18.410690+0000","last_clean":"2024-02-02T13:31:18.410690+0000","last_became_active":"2024-02-02T13:31:16.922210+0000","last_became_peered":"2024-02-02T13:31:16.922210+0000","last_unstale":"2024-02-02T13:31:18.410690+0000","last_undegraded":"2024-02-02T13:31:18.410690+0000","last_fullsized":"2024-02-02T13:31:18.410690+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"127'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933710+0000","last_change":"2024-02-02T13:31:16.433800+0000","last_active":"2024-02-02T13:31:16.933710+0000","last_peered":"2024-02-02T13:31:16.933710+0000","last_clean":"2024-02-02T13:31:16.933710+0000","last_became_active":"2024-02-02T13:31:16.433510+0000","last_became_peered":"2024-02-02T13:31:16.433510+0000","last_unstale":"2024-02-02T13:31:16.933710+0000","last_undegraded":"2024-02-02T13:31:16.933710+0000","last_fullsized":"2024-02-02T13:31:16.933710+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678509+0000","last_change":"2024-02-02T13:31:16.930321+0000","last_active":"2024-02-02T13:31:18.678509+0000","last_peered":"2024-02-02T13:31:18.678509+0000","last_clean":"2024-02-02T13:31:18.678509+0000","last_became_active":"2024-02-02T13:31:16.929950+0000","last_became_peered":"2024-02-02T13:31:16.929950+0000","last_unstale":"2024-02-02T13:31:18.678509+0000","last_undegraded":"2024-02-02T13:31:18.678509+0000","last_fullsized":"2024-02-02T13:31:18.678509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"127'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.934404+0000","last_change":"2024-02-02T13:31:16.440711+0000","last_active":"2024-02-02T13:31:16.934404+0000","last_peered":"2024-02-02T13:31:16.934404+0000","last_clean":"2024-02-02T13:31:16.934404+0000","last_became_active":"2024-02-02T13:31:16.440503+0000","last_became_peered":"2024-02-02T13:31:16.440503+0000","last_unstale":"2024-02-02T13:31:16.934404+0000","last_undegraded":"2024-02-02T13:31:16.934404+0000","last_fullsized":"2024-02-02T13:31:16.934404+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:46.255117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860267+0000","last_change":"2024-02-02T13:31:16.925966+0000","last_active":"2024-02-02T13:31:18.860267+0000","last_peered":"2024-02-02T13:31:18.860267+0000","last_clean":"2024-02-02T13:31:18.860267+0000","last_became_active":"2024-02-02T13:31:16.925765+0000","last_became_peered":"2024-02-02T13:31:16.925765+0000","last_unstale":"2024-02-02T13:31:18.860267+0000","last_undegraded":"2024-02-02T13:31:18.860267+0000","last_fullsized":"2024-02-02T13:31:18.860267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925661+0000","last_change":"2024-02-02T13:31:16.435630+0000","last_active":"2024-02-02T13:31:16.925661+0000","last_peered":"2024-02-02T13:31:16.925661+0000","last_clean":"2024-02-02T13:31:16.925661+0000","last_became_active":"2024-02-02T13:31:16.435406+0000","last_became_peered":"2024-02-02T13:31:16.435406+0000","last_unstale":"2024-02-02T13:31:16.925661+0000","last_undegraded":"2024-02-02T13:31:16.925661+0000","last_fullsized":"2024-02-02T13:31:16.925661+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:47.961252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903675+0000","last_change":"2024-02-02T13:31:16.913272+0000","last_active":"2024-02-02T13:31:17.903675+0000","last_peered":"2024-02-02T13:31:17.903675+0000","last_clean":"2024-02-02T13:31:17.903675+0000","last_became_active":"2024-02-02T13:31:16.912436+0000","last_became_peered":"2024-02-02T13:31:16.912436+0000","last_unstale":"2024-02-02T13:31:17.903675+0000","last_undegraded":"2024-02-02T13:31:17.903675+0000","last_fullsized":"2024-02-02T13:31:17.903675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:58.823596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906345+0000","last_change":"2024-02-02T13:31:16.428888+0000","last_active":"2024-02-02T13:31:16.906345+0000","last_peered":"2024-02-02T13:31:16.906345+0000","last_clean":"2024-02-02T13:31:16.906345+0000","last_became_active":"2024-02-02T13:31:16.428511+0000","last_became_peered":"2024-02-02T13:31:16.428511+0000","last_unstale":"2024-02-02T13:31:16.906345+0000","last_undegraded":"2024-02-02T13:31:16.906345+0000","last_fullsized":"2024-02-02T13:31:16.906345+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:55.831956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380126+0000","last_change":"2024-02-02T13:31:16.929051+0000","last_active":"2024-02-02T13:31:20.380126+0000","last_peered":"2024-02-02T13:31:20.380126+0000","last_clean":"2024-02-02T13:31:20.380126+0000","last_became_active":"2024-02-02T13:31:16.928735+0000","last_became_peered":"2024-02-02T13:31:16.928735+0000","last_unstale":"2024-02-02T13:31:20.380126+0000","last_undegraded":"2024-02-02T13:31:20.380126+0000","last_fullsized":"2024-02-02T13:31:20.380126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:57.109278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.912976+0000","last_change":"2024-02-02T13:31:16.434208+0000","last_active":"2024-02-02T13:31:16.912976+0000","last_peered":"2024-02-02T13:31:16.912976+0000","last_clean":"2024-02-02T13:31:16.912976+0000","last_became_active":"2024-02-02T13:31:16.433931+0000","last_became_peered":"2024-02-02T13:31:16.433931+0000","last_unstale":"2024-02-02T13:31:16.912976+0000","last_undegraded":"2024-02-02T13:31:16.912976+0000","last_fullsized":"2024-02-02T13:31:16.912976+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906588+0000","last_change":"2024-02-02T13:31:16.925971+0000","last_active":"2024-02-02T13:31:17.906588+0000","last_peered":"2024-02-02T13:31:17.906588+0000","last_clean":"2024-02-02T13:31:17.906588+0000","last_became_active":"2024-02-02T13:31:16.925633+0000","last_became_peered":"2024-02-02T13:31:16.925633+0000","last_unstale":"2024-02-02T13:31:17.906588+0000","last_undegraded":"2024-02-02T13:31:17.906588+0000","last_fullsized":"2024-02-02T13:31:17.906588+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:18.174713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"127'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931113+0000","last_change":"2024-02-02T13:31:16.426042+0000","last_active":"2024-02-02T13:31:16.931113+0000","last_peered":"2024-02-02T13:31:16.931113+0000","last_clean":"2024-02-02T13:31:16.931113+0000","last_became_active":"2024-02-02T13:31:16.425745+0000","last_became_peered":"2024-02-02T13:31:16.425745+0000","last_unstale":"2024-02-02T13:31:16.931113+0000","last_undegraded":"2024-02-02T13:31:16.931113+0000","last_fullsized":"2024-02-02T13:31:16.931113+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:57.874029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410943+0000","last_change":"2024-02-02T13:31:16.927661+0000","last_active":"2024-02-02T13:31:18.410943+0000","last_peered":"2024-02-02T13:31:18.410943+0000","last_clean":"2024-02-02T13:31:18.410943+0000","last_became_active":"2024-02-02T13:31:16.927430+0000","last_became_peered":"2024-02-02T13:31:16.927430+0000","last_unstale":"2024-02-02T13:31:18.410943+0000","last_undegraded":"2024-02-02T13:31:18.410943+0000","last_fullsized":"2024-02-02T13:31:18.410943+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931470+0000","last_change":"2024-02-02T13:31:16.436067+0000","last_active":"2024-02-02T13:31:16.931470+0000","last_peered":"2024-02-02T13:31:16.931470+0000","last_clean":"2024-02-02T13:31:16.931470+0000","last_became_active":"2024-02-02T13:31:16.435779+0000","last_became_peered":"2024-02-02T13:31:16.435779+0000","last_unstale":"2024-02-02T13:31:16.931470+0000","last_undegraded":"2024-02-02T13:31:16.931470+0000","last_fullsized":"2024-02-02T13:31:16.931470+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.367030+0000","last_change":"2024-02-02T13:31:16.926917+0000","last_active":"2024-02-02T13:31:18.367030+0000","last_peered":"2024-02-02T13:31:18.367030+0000","last_clean":"2024-02-02T13:31:18.367030+0000","last_became_active":"2024-02-02T13:31:16.926662+0000","last_became_peered":"2024-02-02T13:31:16.926662+0000","last_unstale":"2024-02-02T13:31:18.367030+0000","last_undegraded":"2024-02-02T13:31:18.367030+0000","last_fullsized":"2024-02-02T13:31:18.367030+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:46:43.024952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931079+0000","last_change":"2024-02-02T13:31:16.431019+0000","last_active":"2024-02-02T13:31:16.931079+0000","last_peered":"2024-02-02T13:31:16.931079+0000","last_clean":"2024-02-02T13:31:16.931079+0000","last_became_active":"2024-02-02T13:31:16.430740+0000","last_became_peered":"2024-02-02T13:31:16.430740+0000","last_unstale":"2024-02-02T13:31:16.931079+0000","last_undegraded":"2024-02-02T13:31:16.931079+0000","last_fullsized":"2024-02-02T13:31:16.931079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679308+0000","last_change":"2024-02-02T13:31:16.930011+0000","last_active":"2024-02-02T13:31:18.679308+0000","last_peered":"2024-02-02T13:31:18.679308+0000","last_clean":"2024-02-02T13:31:18.679308+0000","last_became_active":"2024-02-02T13:31:16.929669+0000","last_became_peered":"2024-02-02T13:31:16.929669+0000","last_unstale":"2024-02-02T13:31:18.679308+0000","last_undegraded":"2024-02-02T13:31:18.679308+0000","last_fullsized":"2024-02-02T13:31:18.679308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916493+0000","last_change":"2024-02-02T13:31:16.436267+0000","last_active":"2024-02-02T13:31:16.916493+0000","last_peered":"2024-02-02T13:31:16.916493+0000","last_clean":"2024-02-02T13:31:16.916493+0000","last_became_active":"2024-02-02T13:31:16.434857+0000","last_became_peered":"2024-02-02T13:31:16.434857+0000","last_unstale":"2024-02-02T13:31:16.916493+0000","last_undegraded":"2024-02-02T13:31:16.916493+0000","last_fullsized":"2024-02-02T13:31:16.916493+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:56.711812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907106+0000","last_change":"2024-02-02T13:31:16.909236+0000","last_active":"2024-02-02T13:31:17.907106+0000","last_peered":"2024-02-02T13:31:17.907106+0000","last_clean":"2024-02-02T13:31:17.907106+0000","last_became_active":"2024-02-02T13:31:16.908955+0000","last_became_peered":"2024-02-02T13:31:16.908955+0000","last_unstale":"2024-02-02T13:31:17.907106+0000","last_undegraded":"2024-02-02T13:31:17.907106+0000","last_fullsized":"2024-02-02T13:31:17.907106+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:11:20.213389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924258+0000","last_change":"2024-02-02T13:31:16.434809+0000","last_active":"2024-02-02T13:31:16.924258+0000","last_peered":"2024-02-02T13:31:16.924258+0000","last_clean":"2024-02-02T13:31:16.924258+0000","last_became_active":"2024-02-02T13:31:16.434339+0000","last_became_peered":"2024-02-02T13:31:16.434339+0000","last_unstale":"2024-02-02T13:31:16.924258+0000","last_undegraded":"2024-02-02T13:31:16.924258+0000","last_fullsized":"2024-02-02T13:31:16.924258+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:21.074291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906252+0000","last_change":"2024-02-02T13:31:16.922167+0000","last_active":"2024-02-02T13:31:17.906252+0000","last_peered":"2024-02-02T13:31:17.906252+0000","last_clean":"2024-02-02T13:31:17.906252+0000","last_became_active":"2024-02-02T13:31:16.921940+0000","last_became_peered":"2024-02-02T13:31:16.921940+0000","last_unstale":"2024-02-02T13:31:17.906252+0000","last_undegraded":"2024-02-02T13:31:17.906252+0000","last_fullsized":"2024-02-02T13:31:17.906252+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:55.017456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.915787+0000","last_change":"2024-02-02T13:31:16.436209+0000","last_active":"2024-02-02T13:31:16.915787+0000","last_peered":"2024-02-02T13:31:16.915787+0000","last_clean":"2024-02-02T13:31:16.915787+0000","last_became_active":"2024-02-02T13:31:16.435952+0000","last_became_peered":"2024-02-02T13:31:16.435952+0000","last_unstale":"2024-02-02T13:31:16.915787+0000","last_undegraded":"2024-02-02T13:31:16.915787+0000","last_fullsized":"2024-02-02T13:31:16.915787+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:56.934997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906449+0000","last_change":"2024-02-02T13:31:16.908777+0000","last_active":"2024-02-02T13:31:17.906449+0000","last_peered":"2024-02-02T13:31:17.906449+0000","last_clean":"2024-02-02T13:31:17.906449+0000","last_became_active":"2024-02-02T13:31:16.908560+0000","last_became_peered":"2024-02-02T13:31:16.908560+0000","last_unstale":"2024-02-02T13:31:17.906449+0000","last_undegraded":"2024-02-02T13:31:17.906449+0000","last_fullsized":"2024-02-02T13:31:17.906449+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:44.264100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.920654+0000","last_change":"2024-02-02T13:31:16.430948+0000","last_active":"2024-02-02T13:31:16.920654+0000","last_peered":"2024-02-02T13:31:16.920654+0000","last_clean":"2024-02-02T13:31:16.920654+0000","last_became_active":"2024-02-02T13:31:16.430678+0000","last_became_peered":"2024-02-02T13:31:16.430678+0000","last_unstale":"2024-02-02T13:31:16.920654+0000","last_undegraded":"2024-02-02T13:31:16.920654+0000","last_fullsized":"2024-02-02T13:31:16.920654+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411647+0000","last_change":"2024-02-02T13:31:16.918149+0000","last_active":"2024-02-02T13:31:18.411647+0000","last_peered":"2024-02-02T13:31:18.411647+0000","last_clean":"2024-02-02T13:31:18.411647+0000","last_became_active":"2024-02-02T13:31:16.916696+0000","last_became_peered":"2024-02-02T13:31:16.916696+0000","last_unstale":"2024-02-02T13:31:18.411647+0000","last_undegraded":"2024-02-02T13:31:18.411647+0000","last_fullsized":"2024-02-02T13:31:18.411647+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:45.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916199+0000","last_change":"2024-02-02T13:31:16.432995+0000","last_active":"2024-02-02T13:31:16.916199+0000","last_peered":"2024-02-02T13:31:16.916199+0000","last_clean":"2024-02-02T13:31:16.916199+0000","last_became_active":"2024-02-02T13:31:16.432557+0000","last_became_peered":"2024-02-02T13:31:16.432557+0000","last_unstale":"2024-02-02T13:31:16.916199+0000","last_undegraded":"2024-02-02T13:31:16.916199+0000","last_fullsized":"2024-02-02T13:31:16.916199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907676+0000","last_change":"2024-02-02T13:31:16.931825+0000","last_active":"2024-02-02T13:31:17.907676+0000","last_peered":"2024-02-02T13:31:17.907676+0000","last_clean":"2024-02-02T13:31:17.907676+0000","last_became_active":"2024-02-02T13:31:16.931448+0000","last_became_peered":"2024-02-02T13:31:16.931448+0000","last_unstale":"2024-02-02T13:31:17.907676+0000","last_undegraded":"2024-02-02T13:31:17.907676+0000","last_fullsized":"2024-02-02T13:31:17.907676+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933817+0000","last_change":"2024-02-02T13:31:16.435712+0000","last_active":"2024-02-02T13:31:16.933817+0000","last_peered":"2024-02-02T13:31:16.933817+0000","last_clean":"2024-02-02T13:31:16.933817+0000","last_became_active":"2024-02-02T13:31:16.435452+0000","last_became_peered":"2024-02-02T13:31:16.435452+0000","last_unstale":"2024-02-02T13:31:16.933817+0000","last_undegraded":"2024-02-02T13:31:16.933817+0000","last_fullsized":"2024-02-02T13:31:16.933817+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902968+0000","last_change":"2024-02-02T13:31:16.908923+0000","last_active":"2024-02-02T13:31:17.902968+0000","last_peered":"2024-02-02T13:31:17.902968+0000","last_clean":"2024-02-02T13:31:17.902968+0000","last_became_active":"2024-02-02T13:31:16.908678+0000","last_became_peered":"2024-02-02T13:31:16.908678+0000","last_unstale":"2024-02-02T13:31:17.902968+0000","last_undegraded":"2024-02-02T13:31:17.902968+0000","last_fullsized":"2024-02-02T13:31:17.902968+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:39:39.190128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.917921+0000","last_change":"2024-02-02T13:31:16.434573+0000","last_active":"2024-02-02T13:31:16.917921+0000","last_peered":"2024-02-02T13:31:16.917921+0000","last_clean":"2024-02-02T13:31:16.917921+0000","last_became_active":"2024-02-02T13:31:16.433985+0000","last_became_peered":"2024-02-02T13:31:16.433985+0000","last_unstale":"2024-02-02T13:31:16.917921+0000","last_undegraded":"2024-02-02T13:31:16.917921+0000","last_fullsized":"2024-02-02T13:31:16.917921+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:10.211668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903757+0000","last_change":"2024-02-02T13:31:16.913433+0000","last_active":"2024-02-02T13:31:17.903757+0000","last_peered":"2024-02-02T13:31:17.903757+0000","last_clean":"2024-02-02T13:31:17.903757+0000","last_became_active":"2024-02-02T13:31:16.913071+0000","last_became_peered":"2024-02-02T13:31:16.913071+0000","last_unstale":"2024-02-02T13:31:17.903757+0000","last_undegraded":"2024-02-02T13:31:17.903757+0000","last_fullsized":"2024-02-02T13:31:17.903757+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:02:37.732207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916092+0000","last_change":"2024-02-02T13:31:16.437984+0000","last_active":"2024-02-02T13:31:16.916092+0000","last_peered":"2024-02-02T13:31:16.916092+0000","last_clean":"2024-02-02T13:31:16.916092+0000","last_became_active":"2024-02-02T13:31:16.437426+0000","last_became_peered":"2024-02-02T13:31:16.437426+0000","last_unstale":"2024-02-02T13:31:16.916092+0000","last_undegraded":"2024-02-02T13:31:16.916092+0000","last_fullsized":"2024-02-02T13:31:16.916092+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411212+0000","last_change":"2024-02-02T13:31:16.914410+0000","last_active":"2024-02-02T13:31:18.411212+0000","last_peered":"2024-02-02T13:31:18.411212+0000","last_clean":"2024-02-02T13:31:18.411212+0000","last_became_active":"2024-02-02T13:31:16.913957+0000","last_became_peered":"2024-02-02T13:31:16.913957+0000","last_unstale":"2024-02-02T13:31:18.411212+0000","last_undegraded":"2024-02-02T13:31:18.411212+0000","last_fullsized":"2024-02-02T13:31:18.411212+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.908331+0000","last_change":"2024-02-02T13:31:16.428377+0000","last_active":"2024-02-02T13:31:16.908331+0000","last_peered":"2024-02-02T13:31:16.908331+0000","last_clean":"2024-02-02T13:31:16.908331+0000","last_became_active":"2024-02-02T13:31:16.427907+0000","last_became_peered":"2024-02-02T13:31:16.427907+0000","last_unstale":"2024-02-02T13:31:16.908331+0000","last_undegraded":"2024-02-02T13:31:16.908331+0000","last_fullsized":"2024-02-02T13:31:16.908331+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:58:45.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906825+0000","last_change":"2024-02-02T13:31:16.925836+0000","last_active":"2024-02-02T13:31:17.906825+0000","last_peered":"2024-02-02T13:31:17.906825+0000","last_clean":"2024-02-02T13:31:17.906825+0000","last_became_active":"2024-02-02T13:31:16.925344+0000","last_became_peered":"2024-02-02T13:31:16.925344+0000","last_unstale":"2024-02-02T13:31:17.906825+0000","last_undegraded":"2024-02-02T13:31:17.906825+0000","last_fullsized":"2024-02-02T13:31:17.906825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:10:14.812842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916292+0000","last_change":"2024-02-02T13:31:16.440211+0000","last_active":"2024-02-02T13:31:16.916292+0000","last_peered":"2024-02-02T13:31:16.916292+0000","last_clean":"2024-02-02T13:31:16.916292+0000","last_became_active":"2024-02-02T13:31:16.439815+0000","last_became_peered":"2024-02-02T13:31:16.439815+0000","last_unstale":"2024-02-02T13:31:16.916292+0000","last_undegraded":"2024-02-02T13:31:16.916292+0000","last_fullsized":"2024-02-02T13:31:16.916292+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380628+0000","last_change":"2024-02-02T13:31:16.929773+0000","last_active":"2024-02-02T13:31:20.380628+0000","last_peered":"2024-02-02T13:31:20.380628+0000","last_clean":"2024-02-02T13:31:20.380628+0000","last_became_active":"2024-02-02T13:31:16.929288+0000","last_became_peered":"2024-02-02T13:31:16.929288+0000","last_unstale":"2024-02-02T13:31:20.380628+0000","last_undegraded":"2024-02-02T13:31:20.380628+0000","last_fullsized":"2024-02-02T13:31:20.380628+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:30.322652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.907605+0000","last_change":"2024-02-02T13:31:16.423671+0000","last_active":"2024-02-02T13:31:16.907605+0000","last_peered":"2024-02-02T13:31:16.907605+0000","last_clean":"2024-02-02T13:31:16.907605+0000","last_became_active":"2024-02-02T13:31:16.422898+0000","last_became_peered":"2024-02-02T13:31:16.422898+0000","last_unstale":"2024-02-02T13:31:16.907605+0000","last_undegraded":"2024-02-02T13:31:16.907605+0000","last_fullsized":"2024-02-02T13:31:16.907605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:57:01.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907355+0000","last_change":"2024-02-02T13:31:16.925478+0000","last_active":"2024-02-02T13:31:17.907355+0000","last_peered":"2024-02-02T13:31:17.907355+0000","last_clean":"2024-02-02T13:31:17.907355+0000","last_became_active":"2024-02-02T13:31:16.925257+0000","last_became_peered":"2024-02-02T13:31:16.925257+0000","last_unstale":"2024-02-02T13:31:17.907355+0000","last_undegraded":"2024-02-02T13:31:17.907355+0000","last_fullsized":"2024-02-02T13:31:17.907355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:28.646906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.903393+0000","last_change":"2024-02-02T13:31:16.439366+0000","last_active":"2024-02-02T13:31:16.903393+0000","last_peered":"2024-02-02T13:31:16.903393+0000","last_clean":"2024-02-02T13:31:16.903393+0000","last_became_active":"2024-02-02T13:31:16.438414+0000","last_became_peered":"2024-02-02T13:31:16.438414+0000","last_unstale":"2024-02-02T13:31:16.903393+0000","last_undegraded":"2024-02-02T13:31:16.903393+0000","last_fullsized":"2024-02-02T13:31:16.903393+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411280+0000","last_change":"2024-02-02T13:31:16.919515+0000","last_active":"2024-02-02T13:31:18.411280+0000","last_peered":"2024-02-02T13:31:18.411280+0000","last_clean":"2024-02-02T13:31:18.411280+0000","last_became_active":"2024-02-02T13:31:16.918683+0000","last_became_peered":"2024-02-02T13:31:16.918683+0000","last_unstale":"2024-02-02T13:31:18.411280+0000","last_undegraded":"2024-02-02T13:31:18.411280+0000","last_fullsized":"2024-02-02T13:31:18.411280+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906141+0000","last_change":"2024-02-02T13:31:16.433478+0000","last_active":"2024-02-02T13:31:16.906141+0000","last_peered":"2024-02-02T13:31:16.906141+0000","last_clean":"2024-02-02T13:31:16.906141+0000","last_became_active":"2024-02-02T13:31:16.433197+0000","last_became_peered":"2024-02-02T13:31:16.433197+0000","last_unstale":"2024-02-02T13:31:16.906141+0000","last_undegraded":"2024-02-02T13:31:16.906141+0000","last_fullsized":"2024-02-02T13:31:16.906141+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:54.153974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907538+0000","last_change":"2024-02-02T13:31:16.910890+0000","last_active":"2024-02-02T13:31:17.907538+0000","last_peered":"2024-02-02T13:31:17.907538+0000","last_clean":"2024-02-02T13:31:17.907538+0000","last_became_active":"2024-02-02T13:31:16.910665+0000","last_became_peered":"2024-02-02T13:31:16.910665+0000","last_unstale":"2024-02-02T13:31:17.907538+0000","last_undegraded":"2024-02-02T13:31:17.907538+0000","last_fullsized":"2024-02-02T13:31:17.907538+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:11:17.000587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.930725+0000","last_change":"2024-02-02T13:31:16.434228+0000","last_active":"2024-02-02T13:31:16.930725+0000","last_peered":"2024-02-02T13:31:16.930725+0000","last_clean":"2024-02-02T13:31:16.930725+0000","last_became_active":"2024-02-02T13:31:16.433955+0000","last_became_peered":"2024-02-02T13:31:16.433955+0000","last_unstale":"2024-02-02T13:31:16.930725+0000","last_undegraded":"2024-02-02T13:31:16.930725+0000","last_fullsized":"2024-02-02T13:31:16.930725+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.913104+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:31:16.913104+0000","last_peered":"2024-02-02T13:31:16.913104+0000","last_clean":"2024-02-02T13:31:16.913104+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:31:16.913104+0000","last_undegraded":"2024-02-02T13:31:16.913104+0000","last_fullsized":"2024-02-02T13:31:16.913104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198703555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.0660000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.5149999999999999,"5min":1.6279999999999999,"15min":2.5310000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.105,"15min":1.1240000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4710000000000001,"5min":1.613,"15min":4.1609999999999996},"last":1.091}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.0940000000000001},"min":{"1min":0.86299999999999999,"5min":0.66700000000000004,"15min":0.66300000000000003},"max":{"1min":1.4650000000000001,"5min":1.8839999999999999,"15min":1.893},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.595,"5min":1.758,"15min":2.5840000000000001},"last":1.351}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.266,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.70099999999999996},"max":{"1min":1.7250000000000001,"5min":1.909,"15min":2.6669999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.623},"max":{"1min":1.4359999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.6339999999999999,"15min":1.819},"last":1.202},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.752,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.825,"5min":2.133,"15min":2.4620000000000002},"last":1.254}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.282,"15min":1.202},"min":{"1min":0.92500000000000004,"5min":0.60399999999999998,"15min":0.57599999999999996},"max":{"1min":1.6639999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":0.88400000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.629,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.3520000000000001},"min":{"1min":0.95199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9239999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.454},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.397,"15min":1.4450000000000001},"min":{"1min":0.90900000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7190000000000001,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.8899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4850000000000001,"15min":1.5289999999999999},"min":{"1min":1.246,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.645,"5min":2.1869999999999998,"15min":3.6840000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.087,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0960000000000001,"15min":1.024},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0820000000000001,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.2729999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0269999999999999},"min":{"1min":0.67600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.47,"5min":1.556,"15min":1.556},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1659999999999999,"15min":1.0840000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.589}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1719999999999999,"15min":1.079},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.80300000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.86299999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.89600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.405}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.4570000000000001,"15min":1.27},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":17.686,"5min":17.686,"15min":17.686},"last":1.107},{"interface":"front","average":{"1min":2.1349999999999998,"5min":1.5900000000000001,"15min":1.4490000000000001},"min":{"1min":0.96899999999999997,"5min":0.89900000000000002,"15min":0.66000000000000003},"max":{"1min":17.388999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.498}]},{"osd":5,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.288,"15min":1.276},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":1.986,"15min":1.986},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4339999999999999,"15min":1.359},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.77400000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.462}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.61199999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.613},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.101,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199293965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.403},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.74299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.081}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.26},"min":{"1min":0.76500000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.603,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.284},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.081,"15min":1.1419999999999999},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.474}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.2310000000000001},"min":{"1min":0.86399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.643,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1970000000000001,"15min":1.1779999999999999},"min":{"1min":0.79200000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.468,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.5129999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.202,"15min":1.1899999999999999},"min":{"1min":0.96799999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.409,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.1830000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6200000000000001,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.169}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4570000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.615},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.85099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5569999999999999,"5min":1.833,"15min":1.833},"last":1.4379999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3380000000000001,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7010000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.329},{"interface":"front","average":{"1min":1.341,"5min":1.351,"15min":1.345},"min":{"1min":0.98499999999999999,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.6220000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.724}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.0129999999999999,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":1.1279999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.75,"5min":1.851,"15min":1.851},"last":1.5429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194622,"statfs":{"total":200000000,"available":199293366,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.1080000000000001,"15min":0.997},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.6439999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4970000000000001,"5min":1.839,"15min":1.839},"last":1.3999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":0.995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1659999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.1180000000000001,"15min":1.04},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.097},"min":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9399999999999999,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1950000000000001,"15min":1.083},"min":{"1min":0.79100000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.077,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.04}]},{"osd":3,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.284,"15min":1.1859999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0470000000000002,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.339,"15min":1.2490000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1110000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2529999999999999,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6479999999999999,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3740000000000001,"15min":1.286},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4019999999999999,"15min":1.304},"min":{"1min":0.81499999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2869999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.466,"15min":1.395},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4750000000000001,"15min":1.389},"min":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.504,"15min":1.454},"min":{"1min":0.86099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4900000000000002,"5min":2.698,"15min":2.698},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194621,"statfs":{"total":200000000,"available":199292449,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.70999999999999996},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3480000000000001,"5min":1.619,"15min":2.6040000000000001},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94699999999999995,"15min":0.90100000000000002},"min":{"1min":0.497,"5min":0.46600000000000003,"15min":0.39700000000000002},"max":{"1min":1.4430000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.014,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.516,"5min":1.6359999999999999,"15min":2.125},"last":1.5209999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.034,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4079999999999999,"5min":1.611,"15min":1.611},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":0.89400000000000002},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":1.353}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.72699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.593,"5min":1.7110000000000001,"15min":3.585},"last":1.204},{"interface":"front","average":{"1min":1.165,"5min":1.2529999999999999,"15min":1.206},"min":{"1min":0.70499999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5249999999999999,"5min":1.6659999999999999,"15min":1.877},"last":1.0309999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.202,"15min":1.218},"min":{"1min":0.64400000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3819999999999999,"5min":3.738,"15min":3.738},"last":1.242},{"interface":"front","average":{"1min":1.24,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.754,"5min":1.851,"15min":3.4950000000000001},"last":1.6299999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.2689999999999999},"min":{"1min":0.81299999999999994,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.2989999999999999},"min":{"1min":0.80400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.462,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.5629999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2829999999999999,"15min":1.2629999999999999},"min":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6830000000000001,"5min":1.869,"15min":3.391},"last":1.7},{"interface":"front","average":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3180000000000001},"min":{"1min":0.80900000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6379999999999999,"5min":1.9199999999999999,"15min":2.0409999999999999},"last":1.597}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294001,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4139999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.79900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.30299999999999999},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.91200000000000003},"min":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.441},"max":{"1min":2.153,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.002,"15min":0.874},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.5369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.81000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.9790000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.077,"15min":0.996},"min":{"1min":0.45500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.296,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.748,"15min":0.748},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.27,"5min":1.26,"15min":1.1699999999999999},"min":{"1min":0.81899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.887,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.503}]},{"osd":5,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2490000000000001,"15min":1.1040000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.327,"15min":1.28},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.627}]},{"osd":6,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.196},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3,"15min":1.212},"min":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.367,"15min":1.2250000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.98999999999999999},"min":{"1min":0.624,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.113,"5min":1.425,"15min":3.5680000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91600000000000004,"15min":1.333},"min":{"1min":0.54700000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.355,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.91000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.133,"5min":1.431,"15min":1.639},"last":0.88},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":1.0820000000000001},"min":{"1min":0.59699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3999999999999999,"5min":1.823,"15min":3.0259999999999998},"last":1.1399999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.046,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.5549999999999999,"5min":1.647,"15min":3.1539999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.26},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.7030000000000001,"15min":4.8760000000000003},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1220000000000001,"15min":1.306},"min":{"1min":0.70499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4490000000000001,"5min":1.837,"15min":2.9449999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.3580000000000001},"min":{"1min":0.94399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.722,"5min":1.772,"15min":3.2970000000000002},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2929999999999999,"15min":1.448},"min":{"1min":1.085,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.589,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.278,"15min":1.4279999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5,"5min":1.8959999999999999,"15min":3.0249999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.282,"15min":1.5509999999999999},"min":{"1min":1.0069999999999999,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.304,"15min":1.4790000000000001},"min":{"1min":0.96899999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4039999999999999,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.056}]},{"osd":7,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3540000000000001,"15min":1.542},"min":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7889999999999999,"5min":2.1469999999999998,"15min":3.4689999999999999},"last":1.248},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.363,"15min":1.6000000000000001},"min":{"1min":0.88700000000000001,"5min":0.86699999999999999,"15min":0.78000000000000003},"max":{"1min":1.6240000000000001,"5min":2.1019999999999999,"15min":4.3040000000000003},"last":1.2789999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1265,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198704104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77800000000000002,"15min":1.101},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2030000000000001,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89600000000000002,"15min":1.444},"min":{"1min":0.432,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92900000000000005,"15min":1.151},"min":{"1min":0.48999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4219999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92500000000000004,"15min":1.4390000000000001},"min":{"1min":0.47999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4079999999999999,"5min":1.706,"15min":10.781000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":1.2050000000000001},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1970000000000001,"5min":1.427,"15min":5.8410000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.292},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.413,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1719999999999999,"15min":1.3320000000000001},"min":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.669},"last":1.228},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.397},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.895,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.256,"15min":1.7829999999999999},"min":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.2350000000000001,"15min":1.413},"min":{"1min":0.92100000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6859999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2969999999999999,"15min":1.4610000000000001},"min":{"1min":0.77000000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.419},{"interface":"front","average":{"1min":1.302,"5min":1.2749999999999999,"15min":1.8160000000000001},"min":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":10.708},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3420000000000001,"15min":1.833},"min":{"1min":0.873,"5min":0.82499999999999996,"15min":0.70999999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":10.74},"last":1.518},{"interface":"front","average":{"1min":1.296,"5min":1.2869999999999999,"15min":1.4379999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.869,"5min":1.869,"15min":3.863},"last":1.278}]}]}],"pool_statfs":[]}} 2024-02-02T13:31:26.757 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:31:26.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.976+0000 7f45d2c00640 1 Processor -- start 2024-02-02T13:31:26.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.976+0000 7f45d2c00640 1 -- start start 2024-02-02T13:31:26.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45d2c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45cc070e30 0x7f45cc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45d2c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45cc0cfb90 con 0x7f45cc066020 2024-02-02T13:31:26.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45d2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45cc0af2a0 con 0x7f45cc15d2a0 2024-02-02T13:31:26.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45d2c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc0af580 con 0x7f45cc070e30 2024-02-02T13:31:26.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45cb400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f45cc066020 0x7f45cc06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44070/0 (socket says 172.21.15.91:44070) 2024-02-02T13:31:26.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45cb400640 1 -- 172.21.15.91:0/2789755708 learned_addr learned my addr 172.21.15.91:0/2789755708 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.978+0000 7f45d0c00640 1 --2- 172.21.15.91:0/2789755708 >> v2:172.21.15.113:3300/0 conn(0x7f45cc070e30 0x7f45cc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325826565 0 0) 0x7f45cc0cfb90 con 0x7f45cc066020 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003520 con 0x7f45cc066020 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520449344 0 0) 0x7f45ac003520 con 0x7f45cc066020 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 >> v2:172.21.15.113:3300/0 conn(0x7f45cc070e30 msgr2=0x7f45cc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 --2- 172.21.15.91:0/2789755708 >> v2:172.21.15.113:3300/0 conn(0x7f45cc070e30 0x7f45cc0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 >> v1:172.21.15.113:6789/0 conn(0x7f45cc15d2a0 legacy=0x7f45cc065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0cf8b0 con 0x7f45cc066020 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f45b4001670 con 0x7f45cc066020 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 >> v1:172.21.15.91:6789/0 conn(0x7f45cc066020 legacy=0x7f45cc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45ac0047c0 0x7f45ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45ac0052a0 0x7f45ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0af580 con 0x7f45ac003970 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac00c170 con 0x7f45ac0047c0 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac00c330 con 0x7f45ac0052a0 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.979+0000 7f45cbe00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cb400640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45ac0047c0 0x7f45ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cbe00640 1 -- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45ac0052a0 msgr2=0x7f45ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45d0c00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45ac0052a0 0x7f45ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cbe00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45ac0052a0 0x7f45ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cbe00640 1 -- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45ac0047c0 msgr2=0x7f45ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cbe00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45ac0047c0 0x7f45ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45cbe00640 1 -- 172.21.15.91:0/2789755708 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac003520 con 0x7f45ac003970 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45d0c00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45ac0052a0 0x7f45ac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:26.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.980+0000 7f45d0c00640 1 -- 172.21.15.91:0/2789755708 reap_dead start 2024-02-02T13:31:26.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.981+0000 7f45cbe00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45ac003d90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f45bc001e20 tx=0x7f45ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2218f0e79e7cbfb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.981+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc003b70 con 0x7f45ac003970 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.981+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc003cd0 con 0x7f45ac003970 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.981+0000 7f45caa00640 1 -- 172.21.15.91:0/2789755708 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc005310 con 0x7f45ac003970 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 -- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 msgr2=0x7f45ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45ac003d90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f45bc001e20 tx=0x7f45ac0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 -- 172.21.15.91:0/2789755708 shutdown_connections 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 --2- 172.21.15.91:0/2789755708 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45ac003d90 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 -- 172.21.15.91:0/2789755708 >> 172.21.15.91:0/2789755708 conn(0x7f45cc0cfda0 msgr2=0x7f45cc0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 -- 172.21.15.91:0/2789755708 shutdown_connections 2024-02-02T13:31:26.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.982+0000 7f45d2c00640 1 -- 172.21.15.91:0/2789755708 wait complete. 2024-02-02T13:31:26.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.984+0000 7f45d2c00640 1 Processor -- start 2024-02-02T13:31:26.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.984+0000 7f45d2c00640 1 -- start start 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc148050 0x7f45cc1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45cc14af80 0x7f45cc14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc148a10 con 0x7f45ac003970 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0c7550 con 0x7f45cc148050 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45d2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc148d20 con 0x7f45cc14af80 2024-02-02T13:31:26.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49728/0 (socket says 172.21.15.91:49728) 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.985+0000 7f45cbe00640 1 -- 172.21.15.91:0/118414623 learned_addr learned my addr 172.21.15.91:0/118414623 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cb400640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc148050 0x7f45cc1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45d0c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45cc14af80 0x7f45cc14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 -- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45cc14af80 msgr2=0x7f45cc14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45cc14af80 0x7f45cc14b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 -- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc148050 msgr2=0x7f45cc1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc148050 0x7f45cc1484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:26.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 -- 172.21.15.91:0/118414623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc149000 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45cbe00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005040 tx=0x7f45bc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc006d50 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc007030 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0073e0 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.986+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc1492e0 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.987+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc14c150 con 0x7f45ac003970 2024-02-02T13:31:26.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.988+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc066cf0 con 0x7f45ac003970 2024-02-02T13:31:26.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.989+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f45bc041020 con 0x7f45ac003970 2024-02-02T13:31:26.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.989+0000 7f45c3e00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 0x7f45a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:26.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.989+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45bc0aac90 con 0x7f45ac003970 2024-02-02T13:31:26.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.990+0000 7f45cb400640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 0x7f45a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:26.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.990+0000 7f45cb400640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 0x7f45a406c8f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45b4001120 tx=0x7f45b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:26.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:26.994+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f45bc0ddca0 con 0x7f45ac003970 2024-02-02T13:31:27.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.179+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45cc148a10 con 0x7f45a406a430 2024-02-02T13:31:27.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.188+0000 7f45c3e00640 1 -- 172.21.15.91:0/118414623 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283108 (secure 0 0 0) 0x7f45cc148a10 con 0x7f45a406a430 2024-02-02T13:31:27.193 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:27.194 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.192+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 msgr2=0x7f45a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.192+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 0x7f45a406c8f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45b4001120 tx=0x7f45b4009040 comp rx=0 tx=0).stop 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.192+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 msgr2=0x7f45cc0c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.192+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005040 tx=0x7f45bc0014a0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 shutdown_connections 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f45a406a430 0x7f45a406c8f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45cc14af80 0x7f45cc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc148050 0x7f45cc1484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 --2- 172.21.15.91:0/118414623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45ac003970 0x7f45cc0c6e70 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 >> 172.21.15.91:0/118414623 conn(0x7f45cc0cfda0 msgr2=0x7f45cc0ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 shutdown_connections 2024-02-02T13:31:27.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.193+0000 7f45d2c00640 1 -- 172.21.15.91:0/118414623 wait complete. 2024-02-02T13:31:27.213 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-02-02T13:31:26.370876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555830,"statfs":{"total":1600000000,"available":1593170896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.472574"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902784+0000","last_change":"2024-02-02T13:31:16.918834+0000","last_active":"2024-02-02T13:31:17.902784+0000","last_peered":"2024-02-02T13:31:17.902784+0000","last_clean":"2024-02-02T13:31:17.902784+0000","last_became_active":"2024-02-02T13:31:16.918441+0000","last_became_peered":"2024-02-02T13:31:16.918441+0000","last_unstale":"2024-02-02T13:31:17.902784+0000","last_undegraded":"2024-02-02T13:31:17.902784+0000","last_fullsized":"2024-02-02T13:31:17.902784+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:49:00.816989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906309+0000","last_change":"2024-02-02T13:31:16.927818+0000","last_active":"2024-02-02T13:31:17.906309+0000","last_peered":"2024-02-02T13:31:17.906309+0000","last_clean":"2024-02-02T13:31:17.906309+0000","last_became_active":"2024-02-02T13:31:16.927553+0000","last_became_peered":"2024-02-02T13:31:16.927553+0000","last_unstale":"2024-02-02T13:31:17.906309+0000","last_undegraded":"2024-02-02T13:31:17.906309+0000","last_fullsized":"2024-02-02T13:31:17.906309+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902583+0000","last_change":"2024-02-02T13:31:16.925847+0000","last_active":"2024-02-02T13:31:17.902583+0000","last_peered":"2024-02-02T13:31:17.902583+0000","last_clean":"2024-02-02T13:31:17.902583+0000","last_became_active":"2024-02-02T13:31:16.925615+0000","last_became_peered":"2024-02-02T13:31:16.925615+0000","last_unstale":"2024-02-02T13:31:17.902583+0000","last_undegraded":"2024-02-02T13:31:17.902583+0000","last_fullsized":"2024-02-02T13:31:17.902583+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:42:48.173630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860390+0000","last_change":"2024-02-02T13:31:16.907328+0000","last_active":"2024-02-02T13:31:18.860390+0000","last_peered":"2024-02-02T13:31:18.860390+0000","last_clean":"2024-02-02T13:31:18.860390+0000","last_became_active":"2024-02-02T13:31:16.907025+0000","last_became_peered":"2024-02-02T13:31:16.907025+0000","last_unstale":"2024-02-02T13:31:18.860390+0000","last_undegraded":"2024-02-02T13:31:18.860390+0000","last_fullsized":"2024-02-02T13:31:18.860390+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411041+0000","last_change":"2024-02-02T13:31:16.911577+0000","last_active":"2024-02-02T13:31:18.411041+0000","last_peered":"2024-02-02T13:31:18.411041+0000","last_clean":"2024-02-02T13:31:18.411041+0000","last_became_active":"2024-02-02T13:31:16.911051+0000","last_became_peered":"2024-02-02T13:31:16.911051+0000","last_unstale":"2024-02-02T13:31:18.411041+0000","last_undegraded":"2024-02-02T13:31:18.411041+0000","last_fullsized":"2024-02-02T13:31:18.411041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678686+0000","last_change":"2024-02-02T13:31:16.933309+0000","last_active":"2024-02-02T13:31:18.678686+0000","last_peered":"2024-02-02T13:31:18.678686+0000","last_clean":"2024-02-02T13:31:18.678686+0000","last_became_active":"2024-02-02T13:31:16.932957+0000","last_became_peered":"2024-02-02T13:31:16.932957+0000","last_unstale":"2024-02-02T13:31:18.678686+0000","last_undegraded":"2024-02-02T13:31:18.678686+0000","last_fullsized":"2024-02-02T13:31:18.678686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678369+0000","last_change":"2024-02-02T13:31:16.929320+0000","last_active":"2024-02-02T13:31:18.678369+0000","last_peered":"2024-02-02T13:31:18.678369+0000","last_clean":"2024-02-02T13:31:18.678369+0000","last_became_active":"2024-02-02T13:31:16.929069+0000","last_became_peered":"2024-02-02T13:31:16.929069+0000","last_unstale":"2024-02-02T13:31:18.678369+0000","last_undegraded":"2024-02-02T13:31:18.678369+0000","last_fullsized":"2024-02-02T13:31:18.678369+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907186+0000","last_change":"2024-02-02T13:31:16.926583+0000","last_active":"2024-02-02T13:31:17.907186+0000","last_peered":"2024-02-02T13:31:17.907186+0000","last_clean":"2024-02-02T13:31:17.907186+0000","last_became_active":"2024-02-02T13:31:16.926431+0000","last_became_peered":"2024-02-02T13:31:16.926431+0000","last_unstale":"2024-02-02T13:31:17.907186+0000","last_undegraded":"2024-02-02T13:31:17.907186+0000","last_fullsized":"2024-02-02T13:31:17.907186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:27:48.226607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380086+0000","last_change":"2024-02-02T13:31:16.920257+0000","last_active":"2024-02-02T13:31:20.380086+0000","last_peered":"2024-02-02T13:31:20.380086+0000","last_clean":"2024-02-02T13:31:20.380086+0000","last_became_active":"2024-02-02T13:31:16.919808+0000","last_became_peered":"2024-02-02T13:31:16.919808+0000","last_unstale":"2024-02-02T13:31:20.380086+0000","last_undegraded":"2024-02-02T13:31:20.380086+0000","last_fullsized":"2024-02-02T13:31:20.380086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:20.515102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411221+0000","last_change":"2024-02-02T13:31:16.918453+0000","last_active":"2024-02-02T13:31:18.411221+0000","last_peered":"2024-02-02T13:31:18.411221+0000","last_clean":"2024-02-02T13:31:18.411221+0000","last_became_active":"2024-02-02T13:31:16.917549+0000","last_became_peered":"2024-02-02T13:31:16.917549+0000","last_unstale":"2024-02-02T13:31:18.411221+0000","last_undegraded":"2024-02-02T13:31:18.411221+0000","last_fullsized":"2024-02-02T13:31:18.411221+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678917+0000","last_change":"2024-02-02T13:31:16.925898+0000","last_active":"2024-02-02T13:31:18.678917+0000","last_peered":"2024-02-02T13:31:18.678917+0000","last_clean":"2024-02-02T13:31:18.678917+0000","last_became_active":"2024-02-02T13:31:16.925597+0000","last_became_peered":"2024-02-02T13:31:16.925597+0000","last_unstale":"2024-02-02T13:31:18.678917+0000","last_undegraded":"2024-02-02T13:31:18.678917+0000","last_fullsized":"2024-02-02T13:31:18.678917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:52:05.077952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380719+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:20.380719+0000","last_peered":"2024-02-02T13:31:20.380719+0000","last_clean":"2024-02-02T13:31:20.380719+0000","last_became_active":"2024-02-02T13:31:16.923735+0000","last_became_peered":"2024-02-02T13:31:16.923735+0000","last_unstale":"2024-02-02T13:31:20.380719+0000","last_undegraded":"2024-02-02T13:31:20.380719+0000","last_fullsized":"2024-02-02T13:31:20.380719+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:17.618621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.859985+0000","last_change":"2024-02-02T13:31:16.925160+0000","last_active":"2024-02-02T13:31:18.859985+0000","last_peered":"2024-02-02T13:31:18.859985+0000","last_clean":"2024-02-02T13:31:18.859985+0000","last_became_active":"2024-02-02T13:31:16.924861+0000","last_became_peered":"2024-02-02T13:31:16.924861+0000","last_unstale":"2024-02-02T13:31:18.859985+0000","last_undegraded":"2024-02-02T13:31:18.859985+0000","last_fullsized":"2024-02-02T13:31:18.859985+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:14:24.537963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906357+0000","last_change":"2024-02-02T13:31:16.914250+0000","last_active":"2024-02-02T13:31:17.906357+0000","last_peered":"2024-02-02T13:31:17.906357+0000","last_clean":"2024-02-02T13:31:17.906357+0000","last_became_active":"2024-02-02T13:31:16.914003+0000","last_became_peered":"2024-02-02T13:31:16.914003+0000","last_unstale":"2024-02-02T13:31:17.906357+0000","last_undegraded":"2024-02-02T13:31:17.906357+0000","last_fullsized":"2024-02-02T13:31:17.906357+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:03:38.355640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860441+0000","last_change":"2024-02-02T13:31:16.906982+0000","last_active":"2024-02-02T13:31:18.860441+0000","last_peered":"2024-02-02T13:31:18.860441+0000","last_clean":"2024-02-02T13:31:18.860441+0000","last_became_active":"2024-02-02T13:31:16.906771+0000","last_became_peered":"2024-02-02T13:31:16.906771+0000","last_unstale":"2024-02-02T13:31:18.860441+0000","last_undegraded":"2024-02-02T13:31:18.860441+0000","last_fullsized":"2024-02-02T13:31:18.860441+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679128+0000","last_change":"2024-02-02T13:31:16.929883+0000","last_active":"2024-02-02T13:31:18.679128+0000","last_peered":"2024-02-02T13:31:18.679128+0000","last_clean":"2024-02-02T13:31:18.679128+0000","last_became_active":"2024-02-02T13:31:16.929203+0000","last_became_peered":"2024-02-02T13:31:16.929203+0000","last_unstale":"2024-02-02T13:31:18.679128+0000","last_undegraded":"2024-02-02T13:31:18.679128+0000","last_fullsized":"2024-02-02T13:31:18.679128+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906561+0000","last_change":"2024-02-02T13:31:16.920007+0000","last_active":"2024-02-02T13:31:17.906561+0000","last_peered":"2024-02-02T13:31:17.906561+0000","last_clean":"2024-02-02T13:31:17.906561+0000","last_became_active":"2024-02-02T13:31:16.919625+0000","last_became_peered":"2024-02-02T13:31:16.919625+0000","last_unstale":"2024-02-02T13:31:17.906561+0000","last_undegraded":"2024-02-02T13:31:17.906561+0000","last_fullsized":"2024-02-02T13:31:17.906561+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:19:35.465397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860275+0000","last_change":"2024-02-02T13:31:16.910385+0000","last_active":"2024-02-02T13:31:18.860275+0000","last_peered":"2024-02-02T13:31:18.860275+0000","last_clean":"2024-02-02T13:31:18.860275+0000","last_became_active":"2024-02-02T13:31:16.909793+0000","last_became_peered":"2024-02-02T13:31:16.909793+0000","last_unstale":"2024-02-02T13:31:18.860275+0000","last_undegraded":"2024-02-02T13:31:18.860275+0000","last_fullsized":"2024-02-02T13:31:18.860275+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906609+0000","last_change":"2024-02-02T13:31:16.909676+0000","last_active":"2024-02-02T13:31:17.906609+0000","last_peered":"2024-02-02T13:31:17.906609+0000","last_clean":"2024-02-02T13:31:17.906609+0000","last_became_active":"2024-02-02T13:31:16.909474+0000","last_became_peered":"2024-02-02T13:31:16.909474+0000","last_unstale":"2024-02-02T13:31:17.906609+0000","last_undegraded":"2024-02-02T13:31:17.906609+0000","last_fullsized":"2024-02-02T13:31:17.906609+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:58.879647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380271+0000","last_change":"2024-02-02T13:31:16.921784+0000","last_active":"2024-02-02T13:31:20.380271+0000","last_peered":"2024-02-02T13:31:20.380271+0000","last_clean":"2024-02-02T13:31:20.380271+0000","last_became_active":"2024-02-02T13:31:16.920919+0000","last_became_peered":"2024-02-02T13:31:16.920919+0000","last_unstale":"2024-02-02T13:31:20.380271+0000","last_undegraded":"2024-02-02T13:31:20.380271+0000","last_fullsized":"2024-02-02T13:31:20.380271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:31.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906361+0000","last_change":"2024-02-02T13:31:16.917770+0000","last_active":"2024-02-02T13:31:17.906361+0000","last_peered":"2024-02-02T13:31:17.906361+0000","last_clean":"2024-02-02T13:31:17.906361+0000","last_became_active":"2024-02-02T13:31:16.917542+0000","last_became_peered":"2024-02-02T13:31:16.917542+0000","last_unstale":"2024-02-02T13:31:17.906361+0000","last_undegraded":"2024-02-02T13:31:17.906361+0000","last_fullsized":"2024-02-02T13:31:17.906361+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.721834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906902+0000","last_change":"2024-02-02T13:31:16.927694+0000","last_active":"2024-02-02T13:31:17.906902+0000","last_peered":"2024-02-02T13:31:17.906902+0000","last_clean":"2024-02-02T13:31:17.906902+0000","last_became_active":"2024-02-02T13:31:16.927267+0000","last_became_peered":"2024-02-02T13:31:16.927267+0000","last_unstale":"2024-02-02T13:31:17.906902+0000","last_undegraded":"2024-02-02T13:31:17.906902+0000","last_fullsized":"2024-02-02T13:31:17.906902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907431+0000","last_change":"2024-02-02T13:31:16.931926+0000","last_active":"2024-02-02T13:31:17.907431+0000","last_peered":"2024-02-02T13:31:17.907431+0000","last_clean":"2024-02-02T13:31:17.907431+0000","last_became_active":"2024-02-02T13:31:16.931680+0000","last_became_peered":"2024-02-02T13:31:16.931680+0000","last_unstale":"2024-02-02T13:31:17.907431+0000","last_undegraded":"2024-02-02T13:31:17.907431+0000","last_fullsized":"2024-02-02T13:31:17.907431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:28:29.727079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678475+0000","last_change":"2024-02-02T13:31:16.920779+0000","last_active":"2024-02-02T13:31:18.678475+0000","last_peered":"2024-02-02T13:31:18.678475+0000","last_clean":"2024-02-02T13:31:18.678475+0000","last_became_active":"2024-02-02T13:31:16.918630+0000","last_became_peered":"2024-02-02T13:31:16.918630+0000","last_unstale":"2024-02-02T13:31:18.678475+0000","last_undegraded":"2024-02-02T13:31:18.678475+0000","last_fullsized":"2024-02-02T13:31:18.678475+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411470+0000","last_change":"2024-02-02T13:31:16.923782+0000","last_active":"2024-02-02T13:31:18.411470+0000","last_peered":"2024-02-02T13:31:18.411470+0000","last_clean":"2024-02-02T13:31:18.411470+0000","last_became_active":"2024-02-02T13:31:16.923387+0000","last_became_peered":"2024-02-02T13:31:16.923387+0000","last_unstale":"2024-02-02T13:31:18.411470+0000","last_undegraded":"2024-02-02T13:31:18.411470+0000","last_fullsized":"2024-02-02T13:31:18.411470+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907364+0000","last_change":"2024-02-02T13:31:16.920823+0000","last_active":"2024-02-02T13:31:17.907364+0000","last_peered":"2024-02-02T13:31:17.907364+0000","last_clean":"2024-02-02T13:31:17.907364+0000","last_became_active":"2024-02-02T13:31:16.920636+0000","last_became_peered":"2024-02-02T13:31:16.920636+0000","last_unstale":"2024-02-02T13:31:17.907364+0000","last_undegraded":"2024-02-02T13:31:17.907364+0000","last_fullsized":"2024-02-02T13:31:17.907364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678635+0000","last_change":"2024-02-02T13:31:16.922205+0000","last_active":"2024-02-02T13:31:18.678635+0000","last_peered":"2024-02-02T13:31:18.678635+0000","last_clean":"2024-02-02T13:31:18.678635+0000","last_became_active":"2024-02-02T13:31:16.921225+0000","last_became_peered":"2024-02-02T13:31:16.921225+0000","last_unstale":"2024-02-02T13:31:18.678635+0000","last_undegraded":"2024-02-02T13:31:18.678635+0000","last_fullsized":"2024-02-02T13:31:18.678635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410942+0000","last_change":"2024-02-02T13:31:16.919017+0000","last_active":"2024-02-02T13:31:18.410942+0000","last_peered":"2024-02-02T13:31:18.410942+0000","last_clean":"2024-02-02T13:31:18.410942+0000","last_became_active":"2024-02-02T13:31:16.913456+0000","last_became_peered":"2024-02-02T13:31:16.913456+0000","last_unstale":"2024-02-02T13:31:18.410942+0000","last_undegraded":"2024-02-02T13:31:18.410942+0000","last_fullsized":"2024-02-02T13:31:18.410942+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:50:20.312483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.905879+0000","last_change":"2024-02-02T13:31:16.920909+0000","last_active":"2024-02-02T13:31:17.905879+0000","last_peered":"2024-02-02T13:31:17.905879+0000","last_clean":"2024-02-02T13:31:17.905879+0000","last_became_active":"2024-02-02T13:31:16.920620+0000","last_became_peered":"2024-02-02T13:31:16.920620+0000","last_unstale":"2024-02-02T13:31:17.905879+0000","last_undegraded":"2024-02-02T13:31:17.905879+0000","last_fullsized":"2024-02-02T13:31:17.905879+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:47:29.170966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907939+0000","last_change":"2024-02-02T13:31:16.915569+0000","last_active":"2024-02-02T13:31:17.907939+0000","last_peered":"2024-02-02T13:31:17.907939+0000","last_clean":"2024-02-02T13:31:17.907939+0000","last_became_active":"2024-02-02T13:31:16.914947+0000","last_became_peered":"2024-02-02T13:31:16.914947+0000","last_unstale":"2024-02-02T13:31:17.907939+0000","last_undegraded":"2024-02-02T13:31:17.907939+0000","last_fullsized":"2024-02-02T13:31:17.907939+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:07.892976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860126+0000","last_change":"2024-02-02T13:31:16.908445+0000","last_active":"2024-02-02T13:31:18.860126+0000","last_peered":"2024-02-02T13:31:18.860126+0000","last_clean":"2024-02-02T13:31:18.860126+0000","last_became_active":"2024-02-02T13:31:16.908180+0000","last_became_peered":"2024-02-02T13:31:16.908180+0000","last_unstale":"2024-02-02T13:31:18.860126+0000","last_undegraded":"2024-02-02T13:31:18.860126+0000","last_fullsized":"2024-02-02T13:31:18.860126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902747+0000","last_change":"2024-02-02T13:31:16.921843+0000","last_active":"2024-02-02T13:31:17.902747+0000","last_peered":"2024-02-02T13:31:17.902747+0000","last_clean":"2024-02-02T13:31:17.902747+0000","last_became_active":"2024-02-02T13:31:16.921627+0000","last_became_peered":"2024-02-02T13:31:16.921627+0000","last_unstale":"2024-02-02T13:31:17.902747+0000","last_undegraded":"2024-02-02T13:31:17.902747+0000","last_fullsized":"2024-02-02T13:31:17.902747+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:58:17.957284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906836+0000","last_change":"2024-02-02T13:31:16.918012+0000","last_active":"2024-02-02T13:31:17.906836+0000","last_peered":"2024-02-02T13:31:17.906836+0000","last_clean":"2024-02-02T13:31:17.906836+0000","last_became_active":"2024-02-02T13:31:16.917805+0000","last_became_peered":"2024-02-02T13:31:16.917805+0000","last_unstale":"2024-02-02T13:31:17.906836+0000","last_undegraded":"2024-02-02T13:31:17.906836+0000","last_fullsized":"2024-02-02T13:31:17.906836+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:47.224031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"127'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925207+0000","last_change":"2024-02-02T13:31:16.433503+0000","last_active":"2024-02-02T13:31:16.925207+0000","last_peered":"2024-02-02T13:31:16.925207+0000","last_clean":"2024-02-02T13:31:16.925207+0000","last_became_active":"2024-02-02T13:31:16.433277+0000","last_became_peered":"2024-02-02T13:31:16.433277+0000","last_unstale":"2024-02-02T13:31:16.925207+0000","last_undegraded":"2024-02-02T13:31:16.925207+0000","last_fullsized":"2024-02-02T13:31:16.925207+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:16:12.721762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860526+0000","last_change":"2024-02-02T13:31:16.919617+0000","last_active":"2024-02-02T13:31:18.860526+0000","last_peered":"2024-02-02T13:31:18.860526+0000","last_clean":"2024-02-02T13:31:18.860526+0000","last_became_active":"2024-02-02T13:31:16.919275+0000","last_became_peered":"2024-02-02T13:31:16.919275+0000","last_unstale":"2024-02-02T13:31:18.860526+0000","last_undegraded":"2024-02-02T13:31:18.860526+0000","last_fullsized":"2024-02-02T13:31:18.860526+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931501+0000","last_change":"2024-02-02T13:31:16.424527+0000","last_active":"2024-02-02T13:31:16.931501+0000","last_peered":"2024-02-02T13:31:16.931501+0000","last_clean":"2024-02-02T13:31:16.931501+0000","last_became_active":"2024-02-02T13:31:16.423926+0000","last_became_peered":"2024-02-02T13:31:16.423926+0000","last_unstale":"2024-02-02T13:31:16.931501+0000","last_undegraded":"2024-02-02T13:31:16.931501+0000","last_fullsized":"2024-02-02T13:31:16.931501+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903041+0000","last_change":"2024-02-02T13:31:16.922269+0000","last_active":"2024-02-02T13:31:17.903041+0000","last_peered":"2024-02-02T13:31:17.903041+0000","last_clean":"2024-02-02T13:31:17.903041+0000","last_became_active":"2024-02-02T13:31:16.921999+0000","last_became_peered":"2024-02-02T13:31:16.921999+0000","last_unstale":"2024-02-02T13:31:17.903041+0000","last_undegraded":"2024-02-02T13:31:17.903041+0000","last_fullsized":"2024-02-02T13:31:17.903041+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:26:34.152547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.921395+0000","last_change":"2024-02-02T13:31:16.434462+0000","last_active":"2024-02-02T13:31:16.921395+0000","last_peered":"2024-02-02T13:31:16.921395+0000","last_clean":"2024-02-02T13:31:16.921395+0000","last_became_active":"2024-02-02T13:31:16.434216+0000","last_became_peered":"2024-02-02T13:31:16.434216+0000","last_unstale":"2024-02-02T13:31:16.921395+0000","last_undegraded":"2024-02-02T13:31:16.921395+0000","last_fullsized":"2024-02-02T13:31:16.921395+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380530+0000","last_change":"2024-02-02T13:31:16.928736+0000","last_active":"2024-02-02T13:31:20.380530+0000","last_peered":"2024-02-02T13:31:20.380530+0000","last_clean":"2024-02-02T13:31:20.380530+0000","last_became_active":"2024-02-02T13:31:16.928489+0000","last_became_peered":"2024-02-02T13:31:16.928489+0000","last_unstale":"2024-02-02T13:31:20.380530+0000","last_undegraded":"2024-02-02T13:31:20.380530+0000","last_fullsized":"2024-02-02T13:31:20.380530+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:57.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.928318+0000","last_change":"2024-02-02T13:31:16.440048+0000","last_active":"2024-02-02T13:31:16.928318+0000","last_peered":"2024-02-02T13:31:16.928318+0000","last_clean":"2024-02-02T13:31:16.928318+0000","last_became_active":"2024-02-02T13:31:16.439503+0000","last_became_peered":"2024-02-02T13:31:16.439503+0000","last_unstale":"2024-02-02T13:31:16.928318+0000","last_undegraded":"2024-02-02T13:31:16.928318+0000","last_fullsized":"2024-02-02T13:31:16.928318+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.366981+0000","last_change":"2024-02-02T13:31:16.931809+0000","last_active":"2024-02-02T13:31:18.366981+0000","last_peered":"2024-02-02T13:31:18.366981+0000","last_clean":"2024-02-02T13:31:18.366981+0000","last_became_active":"2024-02-02T13:31:16.931497+0000","last_became_peered":"2024-02-02T13:31:16.931497+0000","last_unstale":"2024-02-02T13:31:18.366981+0000","last_undegraded":"2024-02-02T13:31:18.366981+0000","last_fullsized":"2024-02-02T13:31:18.366981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:55:21.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.922362+0000","last_change":"2024-02-02T13:31:16.418697+0000","last_active":"2024-02-02T13:31:16.922362+0000","last_peered":"2024-02-02T13:31:16.922362+0000","last_clean":"2024-02-02T13:31:16.922362+0000","last_became_active":"2024-02-02T13:31:16.417931+0000","last_became_peered":"2024-02-02T13:31:16.417931+0000","last_unstale":"2024-02-02T13:31:16.922362+0000","last_undegraded":"2024-02-02T13:31:16.922362+0000","last_fullsized":"2024-02-02T13:31:16.922362+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:54.011838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678787+0000","last_change":"2024-02-02T13:31:16.925253+0000","last_active":"2024-02-02T13:31:18.678787+0000","last_peered":"2024-02-02T13:31:18.678787+0000","last_clean":"2024-02-02T13:31:18.678787+0000","last_became_active":"2024-02-02T13:31:16.925007+0000","last_became_peered":"2024-02-02T13:31:16.925007+0000","last_unstale":"2024-02-02T13:31:18.678787+0000","last_undegraded":"2024-02-02T13:31:18.678787+0000","last_fullsized":"2024-02-02T13:31:18.678787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924884+0000","last_change":"2024-02-02T13:31:16.437360+0000","last_active":"2024-02-02T13:31:16.924884+0000","last_peered":"2024-02-02T13:31:16.924884+0000","last_clean":"2024-02-02T13:31:16.924884+0000","last_became_active":"2024-02-02T13:31:16.437067+0000","last_became_peered":"2024-02-02T13:31:16.437067+0000","last_unstale":"2024-02-02T13:31:16.924884+0000","last_undegraded":"2024-02-02T13:31:16.924884+0000","last_fullsized":"2024-02-02T13:31:16.924884+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860682+0000","last_change":"2024-02-02T13:31:16.907956+0000","last_active":"2024-02-02T13:31:18.860682+0000","last_peered":"2024-02-02T13:31:18.860682+0000","last_clean":"2024-02-02T13:31:18.860682+0000","last_became_active":"2024-02-02T13:31:16.907664+0000","last_became_peered":"2024-02-02T13:31:16.907664+0000","last_unstale":"2024-02-02T13:31:18.860682+0000","last_undegraded":"2024-02-02T13:31:18.860682+0000","last_fullsized":"2024-02-02T13:31:18.860682+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:22:59.196697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.909194+0000","last_change":"2024-02-02T13:31:16.435466+0000","last_active":"2024-02-02T13:31:16.909194+0000","last_peered":"2024-02-02T13:31:16.909194+0000","last_clean":"2024-02-02T13:31:16.909194+0000","last_became_active":"2024-02-02T13:31:16.434867+0000","last_became_peered":"2024-02-02T13:31:16.434867+0000","last_unstale":"2024-02-02T13:31:16.909194+0000","last_undegraded":"2024-02-02T13:31:16.909194+0000","last_fullsized":"2024-02-02T13:31:16.909194+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:50:14.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380345+0000","last_change":"2024-02-02T13:31:16.932808+0000","last_active":"2024-02-02T13:31:20.380345+0000","last_peered":"2024-02-02T13:31:20.380345+0000","last_clean":"2024-02-02T13:31:20.380345+0000","last_became_active":"2024-02-02T13:31:16.932559+0000","last_became_peered":"2024-02-02T13:31:16.932559+0000","last_unstale":"2024-02-02T13:31:20.380345+0000","last_undegraded":"2024-02-02T13:31:20.380345+0000","last_fullsized":"2024-02-02T13:31:20.380345+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:21.439554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902293+0000","last_change":"2024-02-02T13:31:16.432947+0000","last_active":"2024-02-02T13:31:16.902293+0000","last_peered":"2024-02-02T13:31:16.902293+0000","last_clean":"2024-02-02T13:31:16.902293+0000","last_became_active":"2024-02-02T13:31:16.432457+0000","last_became_peered":"2024-02-02T13:31:16.432457+0000","last_unstale":"2024-02-02T13:31:16.902293+0000","last_undegraded":"2024-02-02T13:31:16.902293+0000","last_fullsized":"2024-02-02T13:31:16.902293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:02:06.887358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906804+0000","last_change":"2024-02-02T13:31:16.928592+0000","last_active":"2024-02-02T13:31:17.906804+0000","last_peered":"2024-02-02T13:31:17.906804+0000","last_clean":"2024-02-02T13:31:17.906804+0000","last_became_active":"2024-02-02T13:31:16.928381+0000","last_became_peered":"2024-02-02T13:31:16.928381+0000","last_unstale":"2024-02-02T13:31:17.906804+0000","last_undegraded":"2024-02-02T13:31:17.906804+0000","last_fullsized":"2024-02-02T13:31:17.906804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:41.015747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.900135+0000","last_change":"2024-02-02T13:31:16.435253+0000","last_active":"2024-02-02T13:31:16.900135+0000","last_peered":"2024-02-02T13:31:16.900135+0000","last_clean":"2024-02-02T13:31:16.900135+0000","last_became_active":"2024-02-02T13:31:16.432671+0000","last_became_peered":"2024-02-02T13:31:16.432671+0000","last_unstale":"2024-02-02T13:31:16.900135+0000","last_undegraded":"2024-02-02T13:31:16.900135+0000","last_fullsized":"2024-02-02T13:31:16.900135+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:46.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380428+0000","last_change":"2024-02-02T13:31:16.924607+0000","last_active":"2024-02-02T13:31:20.380428+0000","last_peered":"2024-02-02T13:31:20.380428+0000","last_clean":"2024-02-02T13:31:20.380428+0000","last_became_active":"2024-02-02T13:31:16.923514+0000","last_became_peered":"2024-02-02T13:31:16.923514+0000","last_unstale":"2024-02-02T13:31:20.380428+0000","last_undegraded":"2024-02-02T13:31:20.380428+0000","last_fullsized":"2024-02-02T13:31:20.380428+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:11.217285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.902541+0000","last_change":"2024-02-02T13:31:16.438628+0000","last_active":"2024-02-02T13:31:16.902541+0000","last_peered":"2024-02-02T13:31:16.902541+0000","last_clean":"2024-02-02T13:31:16.902541+0000","last_became_active":"2024-02-02T13:31:16.438393+0000","last_became_peered":"2024-02-02T13:31:16.438393+0000","last_unstale":"2024-02-02T13:31:16.902541+0000","last_undegraded":"2024-02-02T13:31:16.902541+0000","last_fullsized":"2024-02-02T13:31:16.902541+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:23:20.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410690+0000","last_change":"2024-02-02T13:31:16.922466+0000","last_active":"2024-02-02T13:31:18.410690+0000","last_peered":"2024-02-02T13:31:18.410690+0000","last_clean":"2024-02-02T13:31:18.410690+0000","last_became_active":"2024-02-02T13:31:16.922210+0000","last_became_peered":"2024-02-02T13:31:16.922210+0000","last_unstale":"2024-02-02T13:31:18.410690+0000","last_undegraded":"2024-02-02T13:31:18.410690+0000","last_fullsized":"2024-02-02T13:31:18.410690+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:36:38.723936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"127'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933710+0000","last_change":"2024-02-02T13:31:16.433800+0000","last_active":"2024-02-02T13:31:16.933710+0000","last_peered":"2024-02-02T13:31:16.933710+0000","last_clean":"2024-02-02T13:31:16.933710+0000","last_became_active":"2024-02-02T13:31:16.433510+0000","last_became_peered":"2024-02-02T13:31:16.433510+0000","last_unstale":"2024-02-02T13:31:16.933710+0000","last_undegraded":"2024-02-02T13:31:16.933710+0000","last_fullsized":"2024-02-02T13:31:16.933710+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:42:19.509484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.678509+0000","last_change":"2024-02-02T13:31:16.930321+0000","last_active":"2024-02-02T13:31:18.678509+0000","last_peered":"2024-02-02T13:31:18.678509+0000","last_clean":"2024-02-02T13:31:18.678509+0000","last_became_active":"2024-02-02T13:31:16.929950+0000","last_became_peered":"2024-02-02T13:31:16.929950+0000","last_unstale":"2024-02-02T13:31:18.678509+0000","last_undegraded":"2024-02-02T13:31:18.678509+0000","last_fullsized":"2024-02-02T13:31:18.678509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:08.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"127'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.934404+0000","last_change":"2024-02-02T13:31:16.440711+0000","last_active":"2024-02-02T13:31:16.934404+0000","last_peered":"2024-02-02T13:31:16.934404+0000","last_clean":"2024-02-02T13:31:16.934404+0000","last_became_active":"2024-02-02T13:31:16.440503+0000","last_became_peered":"2024-02-02T13:31:16.440503+0000","last_unstale":"2024-02-02T13:31:16.934404+0000","last_undegraded":"2024-02-02T13:31:16.934404+0000","last_fullsized":"2024-02-02T13:31:16.934404+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:46.255117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.860267+0000","last_change":"2024-02-02T13:31:16.925966+0000","last_active":"2024-02-02T13:31:18.860267+0000","last_peered":"2024-02-02T13:31:18.860267+0000","last_clean":"2024-02-02T13:31:18.860267+0000","last_became_active":"2024-02-02T13:31:16.925765+0000","last_became_peered":"2024-02-02T13:31:16.925765+0000","last_unstale":"2024-02-02T13:31:18.860267+0000","last_undegraded":"2024-02-02T13:31:18.860267+0000","last_fullsized":"2024-02-02T13:31:18.860267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:06.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.925661+0000","last_change":"2024-02-02T13:31:16.435630+0000","last_active":"2024-02-02T13:31:16.925661+0000","last_peered":"2024-02-02T13:31:16.925661+0000","last_clean":"2024-02-02T13:31:16.925661+0000","last_became_active":"2024-02-02T13:31:16.435406+0000","last_became_peered":"2024-02-02T13:31:16.435406+0000","last_unstale":"2024-02-02T13:31:16.925661+0000","last_undegraded":"2024-02-02T13:31:16.925661+0000","last_fullsized":"2024-02-02T13:31:16.925661+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:41:47.961252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903675+0000","last_change":"2024-02-02T13:31:16.913272+0000","last_active":"2024-02-02T13:31:17.903675+0000","last_peered":"2024-02-02T13:31:17.903675+0000","last_clean":"2024-02-02T13:31:17.903675+0000","last_became_active":"2024-02-02T13:31:16.912436+0000","last_became_peered":"2024-02-02T13:31:16.912436+0000","last_unstale":"2024-02-02T13:31:17.903675+0000","last_undegraded":"2024-02-02T13:31:17.903675+0000","last_fullsized":"2024-02-02T13:31:17.903675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:58.823596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906345+0000","last_change":"2024-02-02T13:31:16.428888+0000","last_active":"2024-02-02T13:31:16.906345+0000","last_peered":"2024-02-02T13:31:16.906345+0000","last_clean":"2024-02-02T13:31:16.906345+0000","last_became_active":"2024-02-02T13:31:16.428511+0000","last_became_peered":"2024-02-02T13:31:16.428511+0000","last_unstale":"2024-02-02T13:31:16.906345+0000","last_undegraded":"2024-02-02T13:31:16.906345+0000","last_fullsized":"2024-02-02T13:31:16.906345+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:55.831956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380126+0000","last_change":"2024-02-02T13:31:16.929051+0000","last_active":"2024-02-02T13:31:20.380126+0000","last_peered":"2024-02-02T13:31:20.380126+0000","last_clean":"2024-02-02T13:31:20.380126+0000","last_became_active":"2024-02-02T13:31:16.928735+0000","last_became_peered":"2024-02-02T13:31:16.928735+0000","last_unstale":"2024-02-02T13:31:20.380126+0000","last_undegraded":"2024-02-02T13:31:20.380126+0000","last_fullsized":"2024-02-02T13:31:20.380126+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:57.109278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.912976+0000","last_change":"2024-02-02T13:31:16.434208+0000","last_active":"2024-02-02T13:31:16.912976+0000","last_peered":"2024-02-02T13:31:16.912976+0000","last_clean":"2024-02-02T13:31:16.912976+0000","last_became_active":"2024-02-02T13:31:16.433931+0000","last_became_peered":"2024-02-02T13:31:16.433931+0000","last_unstale":"2024-02-02T13:31:16.912976+0000","last_undegraded":"2024-02-02T13:31:16.912976+0000","last_fullsized":"2024-02-02T13:31:16.912976+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906588+0000","last_change":"2024-02-02T13:31:16.925971+0000","last_active":"2024-02-02T13:31:17.906588+0000","last_peered":"2024-02-02T13:31:17.906588+0000","last_clean":"2024-02-02T13:31:17.906588+0000","last_became_active":"2024-02-02T13:31:16.925633+0000","last_became_peered":"2024-02-02T13:31:16.925633+0000","last_unstale":"2024-02-02T13:31:17.906588+0000","last_undegraded":"2024-02-02T13:31:17.906588+0000","last_fullsized":"2024-02-02T13:31:17.906588+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:18.174713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"127'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931113+0000","last_change":"2024-02-02T13:31:16.426042+0000","last_active":"2024-02-02T13:31:16.931113+0000","last_peered":"2024-02-02T13:31:16.931113+0000","last_clean":"2024-02-02T13:31:16.931113+0000","last_became_active":"2024-02-02T13:31:16.425745+0000","last_became_peered":"2024-02-02T13:31:16.425745+0000","last_unstale":"2024-02-02T13:31:16.931113+0000","last_undegraded":"2024-02-02T13:31:16.931113+0000","last_fullsized":"2024-02-02T13:31:16.931113+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:57.874029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.410943+0000","last_change":"2024-02-02T13:31:16.927661+0000","last_active":"2024-02-02T13:31:18.410943+0000","last_peered":"2024-02-02T13:31:18.410943+0000","last_clean":"2024-02-02T13:31:18.410943+0000","last_became_active":"2024-02-02T13:31:16.927430+0000","last_became_peered":"2024-02-02T13:31:16.927430+0000","last_unstale":"2024-02-02T13:31:18.410943+0000","last_undegraded":"2024-02-02T13:31:18.410943+0000","last_fullsized":"2024-02-02T13:31:18.410943+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:48.052580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931470+0000","last_change":"2024-02-02T13:31:16.436067+0000","last_active":"2024-02-02T13:31:16.931470+0000","last_peered":"2024-02-02T13:31:16.931470+0000","last_clean":"2024-02-02T13:31:16.931470+0000","last_became_active":"2024-02-02T13:31:16.435779+0000","last_became_peered":"2024-02-02T13:31:16.435779+0000","last_unstale":"2024-02-02T13:31:16.931470+0000","last_undegraded":"2024-02-02T13:31:16.931470+0000","last_fullsized":"2024-02-02T13:31:16.931470+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.367030+0000","last_change":"2024-02-02T13:31:16.926917+0000","last_active":"2024-02-02T13:31:18.367030+0000","last_peered":"2024-02-02T13:31:18.367030+0000","last_clean":"2024-02-02T13:31:18.367030+0000","last_became_active":"2024-02-02T13:31:16.926662+0000","last_became_peered":"2024-02-02T13:31:16.926662+0000","last_unstale":"2024-02-02T13:31:18.367030+0000","last_undegraded":"2024-02-02T13:31:18.367030+0000","last_fullsized":"2024-02-02T13:31:18.367030+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:46:43.024952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.931079+0000","last_change":"2024-02-02T13:31:16.431019+0000","last_active":"2024-02-02T13:31:16.931079+0000","last_peered":"2024-02-02T13:31:16.931079+0000","last_clean":"2024-02-02T13:31:16.931079+0000","last_became_active":"2024-02-02T13:31:16.430740+0000","last_became_peered":"2024-02-02T13:31:16.430740+0000","last_unstale":"2024-02-02T13:31:16.931079+0000","last_undegraded":"2024-02-02T13:31:16.931079+0000","last_fullsized":"2024-02-02T13:31:16.931079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.679308+0000","last_change":"2024-02-02T13:31:16.930011+0000","last_active":"2024-02-02T13:31:18.679308+0000","last_peered":"2024-02-02T13:31:18.679308+0000","last_clean":"2024-02-02T13:31:18.679308+0000","last_became_active":"2024-02-02T13:31:16.929669+0000","last_became_peered":"2024-02-02T13:31:16.929669+0000","last_unstale":"2024-02-02T13:31:18.679308+0000","last_undegraded":"2024-02-02T13:31:18.679308+0000","last_fullsized":"2024-02-02T13:31:18.679308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:05:46.944958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916493+0000","last_change":"2024-02-02T13:31:16.436267+0000","last_active":"2024-02-02T13:31:16.916493+0000","last_peered":"2024-02-02T13:31:16.916493+0000","last_clean":"2024-02-02T13:31:16.916493+0000","last_became_active":"2024-02-02T13:31:16.434857+0000","last_became_peered":"2024-02-02T13:31:16.434857+0000","last_unstale":"2024-02-02T13:31:16.916493+0000","last_undegraded":"2024-02-02T13:31:16.916493+0000","last_fullsized":"2024-02-02T13:31:16.916493+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:56.711812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907106+0000","last_change":"2024-02-02T13:31:16.909236+0000","last_active":"2024-02-02T13:31:17.907106+0000","last_peered":"2024-02-02T13:31:17.907106+0000","last_clean":"2024-02-02T13:31:17.907106+0000","last_became_active":"2024-02-02T13:31:16.908955+0000","last_became_peered":"2024-02-02T13:31:16.908955+0000","last_unstale":"2024-02-02T13:31:17.907106+0000","last_undegraded":"2024-02-02T13:31:17.907106+0000","last_fullsized":"2024-02-02T13:31:17.907106+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:11:20.213389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.924258+0000","last_change":"2024-02-02T13:31:16.434809+0000","last_active":"2024-02-02T13:31:16.924258+0000","last_peered":"2024-02-02T13:31:16.924258+0000","last_clean":"2024-02-02T13:31:16.924258+0000","last_became_active":"2024-02-02T13:31:16.434339+0000","last_became_peered":"2024-02-02T13:31:16.434339+0000","last_unstale":"2024-02-02T13:31:16.924258+0000","last_undegraded":"2024-02-02T13:31:16.924258+0000","last_fullsized":"2024-02-02T13:31:16.924258+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:09:21.074291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906252+0000","last_change":"2024-02-02T13:31:16.922167+0000","last_active":"2024-02-02T13:31:17.906252+0000","last_peered":"2024-02-02T13:31:17.906252+0000","last_clean":"2024-02-02T13:31:17.906252+0000","last_became_active":"2024-02-02T13:31:16.921940+0000","last_became_peered":"2024-02-02T13:31:16.921940+0000","last_unstale":"2024-02-02T13:31:17.906252+0000","last_undegraded":"2024-02-02T13:31:17.906252+0000","last_fullsized":"2024-02-02T13:31:17.906252+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:55.017456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.915787+0000","last_change":"2024-02-02T13:31:16.436209+0000","last_active":"2024-02-02T13:31:16.915787+0000","last_peered":"2024-02-02T13:31:16.915787+0000","last_clean":"2024-02-02T13:31:16.915787+0000","last_became_active":"2024-02-02T13:31:16.435952+0000","last_became_peered":"2024-02-02T13:31:16.435952+0000","last_unstale":"2024-02-02T13:31:16.915787+0000","last_undegraded":"2024-02-02T13:31:16.915787+0000","last_fullsized":"2024-02-02T13:31:16.915787+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:56.934997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906449+0000","last_change":"2024-02-02T13:31:16.908777+0000","last_active":"2024-02-02T13:31:17.906449+0000","last_peered":"2024-02-02T13:31:17.906449+0000","last_clean":"2024-02-02T13:31:17.906449+0000","last_became_active":"2024-02-02T13:31:16.908560+0000","last_became_peered":"2024-02-02T13:31:16.908560+0000","last_unstale":"2024-02-02T13:31:17.906449+0000","last_undegraded":"2024-02-02T13:31:17.906449+0000","last_fullsized":"2024-02-02T13:31:17.906449+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:44.264100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.920654+0000","last_change":"2024-02-02T13:31:16.430948+0000","last_active":"2024-02-02T13:31:16.920654+0000","last_peered":"2024-02-02T13:31:16.920654+0000","last_clean":"2024-02-02T13:31:16.920654+0000","last_became_active":"2024-02-02T13:31:16.430678+0000","last_became_peered":"2024-02-02T13:31:16.430678+0000","last_unstale":"2024-02-02T13:31:16.920654+0000","last_undegraded":"2024-02-02T13:31:16.920654+0000","last_fullsized":"2024-02-02T13:31:16.920654+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:30:41.701718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411647+0000","last_change":"2024-02-02T13:31:16.918149+0000","last_active":"2024-02-02T13:31:18.411647+0000","last_peered":"2024-02-02T13:31:18.411647+0000","last_clean":"2024-02-02T13:31:18.411647+0000","last_became_active":"2024-02-02T13:31:16.916696+0000","last_became_peered":"2024-02-02T13:31:16.916696+0000","last_unstale":"2024-02-02T13:31:18.411647+0000","last_undegraded":"2024-02-02T13:31:18.411647+0000","last_fullsized":"2024-02-02T13:31:18.411647+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:27:45.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916199+0000","last_change":"2024-02-02T13:31:16.432995+0000","last_active":"2024-02-02T13:31:16.916199+0000","last_peered":"2024-02-02T13:31:16.916199+0000","last_clean":"2024-02-02T13:31:16.916199+0000","last_became_active":"2024-02-02T13:31:16.432557+0000","last_became_peered":"2024-02-02T13:31:16.432557+0000","last_unstale":"2024-02-02T13:31:16.916199+0000","last_undegraded":"2024-02-02T13:31:16.916199+0000","last_fullsized":"2024-02-02T13:31:16.916199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:31.551419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907676+0000","last_change":"2024-02-02T13:31:16.931825+0000","last_active":"2024-02-02T13:31:17.907676+0000","last_peered":"2024-02-02T13:31:17.907676+0000","last_clean":"2024-02-02T13:31:17.907676+0000","last_became_active":"2024-02-02T13:31:16.931448+0000","last_became_peered":"2024-02-02T13:31:16.931448+0000","last_unstale":"2024-02-02T13:31:17.907676+0000","last_undegraded":"2024-02-02T13:31:17.907676+0000","last_fullsized":"2024-02-02T13:31:17.907676+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:53.377582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"127'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.933817+0000","last_change":"2024-02-02T13:31:16.435712+0000","last_active":"2024-02-02T13:31:16.933817+0000","last_peered":"2024-02-02T13:31:16.933817+0000","last_clean":"2024-02-02T13:31:16.933817+0000","last_became_active":"2024-02-02T13:31:16.435452+0000","last_became_peered":"2024-02-02T13:31:16.435452+0000","last_unstale":"2024-02-02T13:31:16.933817+0000","last_undegraded":"2024-02-02T13:31:16.933817+0000","last_fullsized":"2024-02-02T13:31:16.933817+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:22:40.728105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.902968+0000","last_change":"2024-02-02T13:31:16.908923+0000","last_active":"2024-02-02T13:31:17.902968+0000","last_peered":"2024-02-02T13:31:17.902968+0000","last_clean":"2024-02-02T13:31:17.902968+0000","last_became_active":"2024-02-02T13:31:16.908678+0000","last_became_peered":"2024-02-02T13:31:16.908678+0000","last_unstale":"2024-02-02T13:31:17.902968+0000","last_undegraded":"2024-02-02T13:31:17.902968+0000","last_fullsized":"2024-02-02T13:31:17.902968+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:39:39.190128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.908715+0000","last_change":"2024-02-02T13:31:16.434573+0000","last_active":"2024-02-02T13:31:17.908715+0000","last_peered":"2024-02-02T13:31:17.908715+0000","last_clean":"2024-02-02T13:31:17.908715+0000","last_became_active":"2024-02-02T13:31:16.433985+0000","last_became_peered":"2024-02-02T13:31:16.433985+0000","last_unstale":"2024-02-02T13:31:17.908715+0000","last_undegraded":"2024-02-02T13:31:17.908715+0000","last_fullsized":"2024-02-02T13:31:17.908715+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:10.211668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.903757+0000","last_change":"2024-02-02T13:31:16.913433+0000","last_active":"2024-02-02T13:31:17.903757+0000","last_peered":"2024-02-02T13:31:17.903757+0000","last_clean":"2024-02-02T13:31:17.903757+0000","last_became_active":"2024-02-02T13:31:16.913071+0000","last_became_peered":"2024-02-02T13:31:16.913071+0000","last_unstale":"2024-02-02T13:31:17.903757+0000","last_undegraded":"2024-02-02T13:31:17.903757+0000","last_fullsized":"2024-02-02T13:31:17.903757+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:02:37.732207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916092+0000","last_change":"2024-02-02T13:31:16.437984+0000","last_active":"2024-02-02T13:31:16.916092+0000","last_peered":"2024-02-02T13:31:16.916092+0000","last_clean":"2024-02-02T13:31:16.916092+0000","last_became_active":"2024-02-02T13:31:16.437426+0000","last_became_peered":"2024-02-02T13:31:16.437426+0000","last_unstale":"2024-02-02T13:31:16.916092+0000","last_undegraded":"2024-02-02T13:31:16.916092+0000","last_fullsized":"2024-02-02T13:31:16.916092+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:43:16.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411212+0000","last_change":"2024-02-02T13:31:16.914410+0000","last_active":"2024-02-02T13:31:18.411212+0000","last_peered":"2024-02-02T13:31:18.411212+0000","last_clean":"2024-02-02T13:31:18.411212+0000","last_became_active":"2024-02-02T13:31:16.913957+0000","last_became_peered":"2024-02-02T13:31:16.913957+0000","last_unstale":"2024-02-02T13:31:18.411212+0000","last_undegraded":"2024-02-02T13:31:18.411212+0000","last_fullsized":"2024-02-02T13:31:18.411212+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:18.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.908331+0000","last_change":"2024-02-02T13:31:16.428377+0000","last_active":"2024-02-02T13:31:16.908331+0000","last_peered":"2024-02-02T13:31:16.908331+0000","last_clean":"2024-02-02T13:31:16.908331+0000","last_became_active":"2024-02-02T13:31:16.427907+0000","last_became_peered":"2024-02-02T13:31:16.427907+0000","last_unstale":"2024-02-02T13:31:16.908331+0000","last_undegraded":"2024-02-02T13:31:16.908331+0000","last_fullsized":"2024-02-02T13:31:16.908331+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:58:45.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.906825+0000","last_change":"2024-02-02T13:31:16.925836+0000","last_active":"2024-02-02T13:31:17.906825+0000","last_peered":"2024-02-02T13:31:17.906825+0000","last_clean":"2024-02-02T13:31:17.906825+0000","last_became_active":"2024-02-02T13:31:16.925344+0000","last_became_peered":"2024-02-02T13:31:16.925344+0000","last_unstale":"2024-02-02T13:31:17.906825+0000","last_undegraded":"2024-02-02T13:31:17.906825+0000","last_fullsized":"2024-02-02T13:31:17.906825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:10:14.812842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.916292+0000","last_change":"2024-02-02T13:31:16.440211+0000","last_active":"2024-02-02T13:31:16.916292+0000","last_peered":"2024-02-02T13:31:16.916292+0000","last_clean":"2024-02-02T13:31:16.916292+0000","last_became_active":"2024-02-02T13:31:16.439815+0000","last_became_peered":"2024-02-02T13:31:16.439815+0000","last_unstale":"2024-02-02T13:31:16.916292+0000","last_undegraded":"2024-02-02T13:31:16.916292+0000","last_fullsized":"2024-02-02T13:31:16.916292+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:20.380628+0000","last_change":"2024-02-02T13:31:16.929773+0000","last_active":"2024-02-02T13:31:20.380628+0000","last_peered":"2024-02-02T13:31:20.380628+0000","last_clean":"2024-02-02T13:31:20.380628+0000","last_became_active":"2024-02-02T13:31:16.929288+0000","last_became_peered":"2024-02-02T13:31:16.929288+0000","last_unstale":"2024-02-02T13:31:20.380628+0000","last_undegraded":"2024-02-02T13:31:20.380628+0000","last_fullsized":"2024-02-02T13:31:20.380628+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:30.322652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.907605+0000","last_change":"2024-02-02T13:31:16.423671+0000","last_active":"2024-02-02T13:31:16.907605+0000","last_peered":"2024-02-02T13:31:16.907605+0000","last_clean":"2024-02-02T13:31:16.907605+0000","last_became_active":"2024-02-02T13:31:16.422898+0000","last_became_peered":"2024-02-02T13:31:16.422898+0000","last_unstale":"2024-02-02T13:31:16.907605+0000","last_undegraded":"2024-02-02T13:31:16.907605+0000","last_fullsized":"2024-02-02T13:31:16.907605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:57:01.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907355+0000","last_change":"2024-02-02T13:31:16.925478+0000","last_active":"2024-02-02T13:31:17.907355+0000","last_peered":"2024-02-02T13:31:17.907355+0000","last_clean":"2024-02-02T13:31:17.907355+0000","last_became_active":"2024-02-02T13:31:16.925257+0000","last_became_peered":"2024-02-02T13:31:16.925257+0000","last_unstale":"2024-02-02T13:31:17.907355+0000","last_undegraded":"2024-02-02T13:31:17.907355+0000","last_fullsized":"2024-02-02T13:31:17.907355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:28.646906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.903393+0000","last_change":"2024-02-02T13:31:16.439366+0000","last_active":"2024-02-02T13:31:16.903393+0000","last_peered":"2024-02-02T13:31:16.903393+0000","last_clean":"2024-02-02T13:31:16.903393+0000","last_became_active":"2024-02-02T13:31:16.438414+0000","last_became_peered":"2024-02-02T13:31:16.438414+0000","last_unstale":"2024-02-02T13:31:16.903393+0000","last_undegraded":"2024-02-02T13:31:16.903393+0000","last_fullsized":"2024-02-02T13:31:16.903393+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:04:29.317034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:18.411280+0000","last_change":"2024-02-02T13:31:16.919515+0000","last_active":"2024-02-02T13:31:18.411280+0000","last_peered":"2024-02-02T13:31:18.411280+0000","last_clean":"2024-02-02T13:31:18.411280+0000","last_became_active":"2024-02-02T13:31:16.918683+0000","last_became_peered":"2024-02-02T13:31:16.918683+0000","last_unstale":"2024-02-02T13:31:18.411280+0000","last_undegraded":"2024-02-02T13:31:18.411280+0000","last_fullsized":"2024-02-02T13:31:18.411280+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:36:05.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.906141+0000","last_change":"2024-02-02T13:31:16.433478+0000","last_active":"2024-02-02T13:31:16.906141+0000","last_peered":"2024-02-02T13:31:16.906141+0000","last_clean":"2024-02-02T13:31:16.906141+0000","last_became_active":"2024-02-02T13:31:16.433197+0000","last_became_peered":"2024-02-02T13:31:16.433197+0000","last_unstale":"2024-02-02T13:31:16.906141+0000","last_undegraded":"2024-02-02T13:31:16.906141+0000","last_fullsized":"2024-02-02T13:31:16.906141+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:54.153974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-02-02T13:31:17.907538+0000","last_change":"2024-02-02T13:31:16.910890+0000","last_active":"2024-02-02T13:31:17.907538+0000","last_peered":"2024-02-02T13:31:17.907538+0000","last_clean":"2024-02-02T13:31:17.907538+0000","last_became_active":"2024-02-02T13:31:16.910665+0000","last_became_peered":"2024-02-02T13:31:16.910665+0000","last_unstale":"2024-02-02T13:31:17.907538+0000","last_undegraded":"2024-02-02T13:31:17.907538+0000","last_fullsized":"2024-02-02T13:31:17.907538+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:16.392456+0000","last_clean_scrub_stamp":"2024-02-02T13:31:16.392456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:11:17.000587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"127'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.930725+0000","last_change":"2024-02-02T13:31:16.434228+0000","last_active":"2024-02-02T13:31:16.930725+0000","last_peered":"2024-02-02T13:31:16.930725+0000","last_clean":"2024-02-02T13:31:16.930725+0000","last_became_active":"2024-02-02T13:31:16.433955+0000","last_became_peered":"2024-02-02T13:31:16.433955+0000","last_unstale":"2024-02-02T13:31:16.930725+0000","last_undegraded":"2024-02-02T13:31:16.930725+0000","last_fullsized":"2024-02-02T13:31:16.930725+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:15.386838+0000","last_clean_scrub_stamp":"2024-02-02T13:31:15.386838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:30.290642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-02-02T13:31:16.913104+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:31:16.913104+0000","last_peered":"2024-02-02T13:31:16.913104+0000","last_clean":"2024-02-02T13:31:16.913104+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:31:16.913104+0000","last_undegraded":"2024-02-02T13:31:16.913104+0000","last_fullsized":"2024-02-02T13:31:16.913104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198703555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.0660000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.5149999999999999,"5min":1.6279999999999999,"15min":2.5310000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.105,"15min":1.1240000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4710000000000001,"5min":1.613,"15min":4.1609999999999996},"last":1.091}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.0940000000000001},"min":{"1min":0.86299999999999999,"5min":0.66700000000000004,"15min":0.66300000000000003},"max":{"1min":1.4650000000000001,"5min":1.8839999999999999,"15min":1.893},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.595,"5min":1.758,"15min":2.5840000000000001},"last":1.351}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.266,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.70099999999999996},"max":{"1min":1.7250000000000001,"5min":1.909,"15min":2.6669999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.623},"max":{"1min":1.4359999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.6339999999999999,"15min":1.819},"last":1.202},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.752,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.825,"5min":2.133,"15min":2.4620000000000002},"last":1.254}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.282,"15min":1.202},"min":{"1min":0.92500000000000004,"5min":0.60399999999999998,"15min":0.57599999999999996},"max":{"1min":1.6639999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":0.88400000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.629,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.3520000000000001},"min":{"1min":0.95199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9239999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.454},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.397,"15min":1.4450000000000001},"min":{"1min":0.90900000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7190000000000001,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.8899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4850000000000001,"15min":1.5289999999999999},"min":{"1min":1.246,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.645,"5min":2.1869999999999998,"15min":3.6840000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.421},"min":{"1min":1.087,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0960000000000001,"15min":1.024},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.381},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0820000000000001,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.1140000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0269999999999999},"min":{"1min":0.67600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.47,"5min":1.556,"15min":1.556},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1659999999999999,"15min":1.0840000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1719999999999999,"15min":1.079},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.052},{"interface":"front","average":{"1min":1.2,"5min":1.1799999999999999,"15min":1.1559999999999999},"min":{"1min":0.80300000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.3169999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2370000000000001,"15min":1.258},"min":{"1min":0.86299999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.218},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.89600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.282}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.4570000000000001,"15min":1.27},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":17.686,"5min":17.686,"15min":17.686},"last":0.89600000000000002},{"interface":"front","average":{"1min":2.1349999999999998,"5min":1.5900000000000001,"15min":1.4490000000000001},"min":{"1min":0.96899999999999997,"5min":0.89900000000000002,"15min":0.66000000000000003},"max":{"1min":17.388999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.419}]},{"osd":5,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.288,"15min":1.276},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":1.986,"15min":1.986},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4339999999999999,"15min":1.359},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.77400000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.153}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4730000000000001,"15min":1.4530000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.61199999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.472},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.101,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.365}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199293965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.74299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0309999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.26},"min":{"1min":0.76500000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.603,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.081,"15min":1.1419999999999999},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.2310000000000001},"min":{"1min":0.86399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.643,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.177},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1970000000000001,"15min":1.1779999999999999},"min":{"1min":0.79200000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.468,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.29}]},{"osd":3,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.202,"15min":1.1899999999999999},"min":{"1min":0.96799999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.409,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.23},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.1830000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6200000000000001,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.385}]},{"osd":4,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4570000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.85099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5569999999999999,"5min":1.833,"15min":1.833},"last":1.3400000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3380000000000001,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7010000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.492},{"interface":"front","average":{"1min":1.341,"5min":1.351,"15min":1.345},"min":{"1min":0.98499999999999999,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.6220000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.427}]},{"osd":7,"last update":"Fri Feb 2 13:30:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.0129999999999999,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.413},{"interface":"front","average":{"1min":1.492,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":1.1279999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.75,"5min":1.851,"15min":1.851},"last":1.2669999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194622,"statfs":{"total":200000000,"available":199293366,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.1080000000000001,"15min":0.997},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.6439999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4970000000000001,"5min":1.839,"15min":1.839},"last":1.2390000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":0.995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.1659999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.403},{"interface":"front","average":{"1min":1.105,"5min":1.1180000000000001,"15min":1.04},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.204}]},{"osd":2,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.097},"min":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9399999999999999,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":1.109},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1950000000000001,"15min":1.083},"min":{"1min":0.79100000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.077,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.484}]},{"osd":3,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.284,"15min":1.1859999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0470000000000002,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.339,"15min":1.2490000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1110000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.171}]},{"osd":5,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2529999999999999,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6479999999999999,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.3740000000000001,"15min":1.286},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4019999999999999,"15min":1.304},"min":{"1min":0.81499999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2869999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.466,"15min":1.395},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.266}]},{"osd":7,"last update":"Fri Feb 2 13:30:36 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4750000000000001,"15min":1.389},"min":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.431},{"interface":"front","average":{"1min":1.571,"5min":1.504,"15min":1.454},"min":{"1min":0.86099999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4900000000000002,"5min":2.698,"15min":2.698},"last":1.05}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194621,"statfs":{"total":200000000,"available":199292449,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.70999999999999996},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3480000000000001,"5min":1.619,"15min":2.6040000000000001},"last":1.8400000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94699999999999995,"15min":0.90100000000000002},"min":{"1min":0.497,"5min":0.46600000000000003,"15min":0.39700000000000002},"max":{"1min":1.4430000000000001,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":2.403},{"interface":"front","average":{"1min":1.014,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.516,"5min":1.6359999999999999,"15min":2.125},"last":2.2709999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.034,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4079999999999999,"5min":1.611,"15min":1.611},"last":1.304},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":0.89400000000000002},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":1.9159999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.72699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.593,"5min":1.7110000000000001,"15min":3.585},"last":1.99},{"interface":"front","average":{"1min":1.165,"5min":1.2529999999999999,"15min":1.206},"min":{"1min":0.70499999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5249999999999999,"5min":1.6659999999999999,"15min":1.877},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.202,"15min":1.218},"min":{"1min":0.64400000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3819999999999999,"5min":3.738,"15min":3.738},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.754,"5min":1.851,"15min":3.4950000000000001},"last":1.8899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.2689999999999999},"min":{"1min":0.81299999999999994,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.7},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.2989999999999999},"min":{"1min":0.80400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.462,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":2.0310000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:42 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2829999999999999,"15min":1.2629999999999999},"min":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6830000000000001,"5min":1.869,"15min":3.391},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3180000000000001},"min":{"1min":0.80900000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6379999999999999,"5min":1.9199999999999999,"15min":2.0409999999999999},"last":1.946}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194623,"statfs":{"total":200000000,"available":199294001,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4139999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.79900000000000004},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.30299999999999999},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.91200000000000003},"min":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.441},"max":{"1min":2.153,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.002,"15min":0.874},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.5369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.81000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.9790000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.077,"15min":0.996},"min":{"1min":0.45500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4259999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.296,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.748,"15min":0.748},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.26,"15min":1.1699999999999999},"min":{"1min":0.81899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.887,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.4710000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2490000000000001,"15min":1.1040000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.077},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.327,"15min":1.28},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.196},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.335},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3,"15min":1.212},"min":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.3779999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.367,"15min":1.2250000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.526},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3700000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.25}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.98999999999999999},"min":{"1min":0.624,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.113,"5min":1.425,"15min":3.5680000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91600000000000004,"15min":1.333},"min":{"1min":0.54700000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.355,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.91000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.133,"5min":1.431,"15min":1.639},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":1.0820000000000001},"min":{"1min":0.59699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3999999999999999,"5min":1.823,"15min":3.0259999999999998},"last":1.6890000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.046,"15min":1.24},"min":{"1min":0.75800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.5549999999999999,"5min":1.647,"15min":3.1539999999999999},"last":2.2029999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.26},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.7030000000000001,"15min":4.8760000000000003},"last":1.774}]},{"osd":4,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1220000000000001,"15min":1.306},"min":{"1min":0.70499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4490000000000001,"5min":1.837,"15min":2.9449999999999998},"last":1.109},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.3580000000000001},"min":{"1min":0.94399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.722,"5min":1.772,"15min":3.2970000000000002},"last":2.5129999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2929999999999999,"15min":1.448},"min":{"1min":1.085,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.589,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.278,"15min":1.4279999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5,"5min":1.8959999999999999,"15min":3.0249999999999999},"last":1.5669999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.282,"15min":1.5509999999999999},"min":{"1min":1.0069999999999999,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":1.149},{"interface":"front","average":{"1min":1.226,"5min":1.304,"15min":1.4790000000000001},"min":{"1min":0.96899999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4039999999999999,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3540000000000001,"15min":1.542},"min":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7889999999999999,"5min":2.1469999999999998,"15min":3.4689999999999999},"last":2.5750000000000002},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.363,"15min":1.6000000000000001},"min":{"1min":0.88700000000000001,"5min":0.86699999999999999,"15min":0.78000000000000003},"max":{"1min":1.6240000000000001,"5min":2.1019999999999999,"15min":4.3040000000000003},"last":1.226}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1265,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194046,"statfs":{"total":200000000,"available":198704104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77800000000000002,"15min":1.101},"min":{"1min":0.34899999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2030000000000001,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.753},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89600000000000002,"15min":1.444},"min":{"1min":0.432,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":1.038}]},{"osd":2,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92900000000000005,"15min":1.151},"min":{"1min":0.48999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4219999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92500000000000004,"15min":1.4390000000000001},"min":{"1min":0.47999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4079999999999999,"5min":1.706,"15min":10.781000000000001},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88600000000000001,"15min":1.2050000000000001},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1970000000000001,"5min":1.427,"15min":5.8410000000000002},"last":1.093},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.292},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.413,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":0.877}]},{"osd":4,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1719999999999999,"15min":1.3320000000000001},"min":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.669},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.397},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.895,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.256,"15min":1.7829999999999999},"min":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.2350000000000001,"15min":1.413},"min":{"1min":0.92100000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6859999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.218}]},{"osd":6,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2969999999999999,"15min":1.4610000000000001},"min":{"1min":0.77000000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.2749999999999999,"15min":1.8160000000000001},"min":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":10.708},"last":1.2749999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:30:37 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3420000000000001,"15min":1.833},"min":{"1min":0.873,"5min":0.82499999999999996,"15min":0.70999999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":10.74},"last":1.353},{"interface":"front","average":{"1min":1.296,"5min":1.2869999999999999,"15min":1.4379999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.869,"5min":1.869,"15min":3.863},"last":1.3400000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:31:27.218 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:31:27.218 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:31:27.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.471+0000 7fa509e00640 1 Processor -- start 2024-02-02T13:31:27.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.471+0000 7fa509e00640 1 -- start start 2024-02-02T13:31:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.472+0000 7fa509e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa50415a010 0x7fa5040c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.472+0000 7fa509e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.472+0000 7fa509e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040cc7c0 con 0x7fa5040ce130 2024-02-02T13:31:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.472+0000 7fa509e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5040b10d0 con 0x7fa50415a010 2024-02-02T13:31:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.472+0000 7fa509e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040ce930 con 0x7fa5040b0700 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.473+0000 7fa503400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.473+0000 7fa502a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa50415a010 0x7fa5040c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.473+0000 7fa502000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa5040ce130 0x7fa5040ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:46038/0 (socket says 172.21.15.91:46038) 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.473+0000 7fa502000640 1 -- 172.21.15.91:0/2509369297 learned_addr learned my addr 172.21.15.91:0/2509369297 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.474+0000 7fa502a00640 1 -- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 msgr2=0x7fa5040b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.474+0000 7fa502a00640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.474+0000 7fa502a00640 1 -- 172.21.15.91:0/2509369297 >> v1:172.21.15.113:6789/0 conn(0x7fa5040ce130 legacy=0x7fa5040ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.474+0000 7fa502a00640 1 -- 172.21.15.91:0/2509369297 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040cd640 con 0x7fa50415a010 2024-02-02T13:31:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.474+0000 7fa503400640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:31:27.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.475+0000 7fa502a00640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50415a010 0x7fa5040c9670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009d90 tx=0x7fa4f402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd63d051f27790df server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa501600640 1 -- 172.21.15.91:0/2509369297 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f402cc50 con 0x7fa50415a010 2024-02-02T13:31:27.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa501600640 1 -- 172.21.15.91:0/2509369297 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f402cdb0 con 0x7fa50415a010 2024-02-02T13:31:27.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa501600640 1 -- 172.21.15.91:0/2509369297 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4034660 con 0x7fa50415a010 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 -- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50415a010 msgr2=0x7fa5040c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50415a010 0x7fa5040c9670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009d90 tx=0x7fa4f402bea0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 -- 172.21.15.91:0/2509369297 shutdown_connections 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 --2- 172.21.15.91:0/2509369297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50415a010 0x7fa5040c9670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 -- 172.21.15.91:0/2509369297 >> 172.21.15.91:0/2509369297 conn(0x7fa5040cec80 msgr2=0x7fa5040cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:27.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 -- 172.21.15.91:0/2509369297 shutdown_connections 2024-02-02T13:31:27.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.476+0000 7fa509e00640 1 -- 172.21.15.91:0/2509369297 wait complete. 2024-02-02T13:31:27.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.478+0000 7fa509e00640 1 Processor -- start 2024-02-02T13:31:27.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.478+0000 7fa509e00640 1 -- start start 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 0x7fa504158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040c80e0 con 0x7fa50415a010 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504158660 con 0x7fa5040ce130 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa509e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040b32d0 con 0x7fa5040b0700 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49736/0 (socket says 172.21.15.91:49736) 2024-02-02T13:31:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.479+0000 7fa503400640 1 -- 172.21.15.91:0/1406168827 learned_addr learned my addr 172.21.15.91:0/1406168827 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa502000640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 0x7fa504158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa502a00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 -- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 msgr2=0x7fa5040b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 -- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 msgr2=0x7fa504158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 0x7fa504158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 -- 172.21.15.91:0/1406168827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc009400 con 0x7fa50415a010 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa502a00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa502000640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 0x7fa504158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa503400640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e880 tx=0x7fa4fc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.480+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc009650 con 0x7fa50415a010 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.481+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc004020 con 0x7fa50415a010 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.481+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f4009a70 con 0x7fa50415a010 2024-02-02T13:31:27.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.481+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0043b0 con 0x7fa50415a010 2024-02-02T13:31:27.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.481+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5040b3890 con 0x7fa50415a010 2024-02-02T13:31:27.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.482+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa504066cf0 con 0x7fa50415a010 2024-02-02T13:31:27.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.483+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa4fc00c940 con 0x7fa50415a010 2024-02-02T13:31:27.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.484+0000 7fa4fb400640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 0x7fa4dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.484+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4fc0887d0 con 0x7fa50415a010 2024-02-02T13:31:27.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.484+0000 7fa502a00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 0x7fa4dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.485+0000 7fa502a00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 0x7fa4dc06c8f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4036000 tx=0x7fa4f4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.488+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa4fc053db0 con 0x7fa50415a010 2024-02-02T13:31:27.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.745+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa5040b4c00 con 0x7fa50415a010 2024-02-02T13:31:27.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.752+0000 7fa4fb400640 1 -- 172.21.15.91:0/1406168827 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa4fc0573d0 con 0x7fa50415a010 2024-02-02T13:31:27.753 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:31:27.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 msgr2=0x7fa4dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 0x7fa4dc06c8f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4036000 tx=0x7fa4f4030000 comp rx=0 tx=0).stop 2024-02-02T13:31:27.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 msgr2=0x7fa5040b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e880 tx=0x7fa4fc00ed40 comp rx=0 tx=0).stop 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 shutdown_connections 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa4dc06a430 0x7fa4dc06c8f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa50415a010 0x7fa5040b28f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5040ce130 0x7fa504158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 --2- 172.21.15.91:0/1406168827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5040b0700 0x7fa5040b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.754+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 >> 172.21.15.91:0/1406168827 conn(0x7fa5040cec80 msgr2=0x7fa50415ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.755+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 shutdown_connections 2024-02-02T13:31:27.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.755+0000 7fa509e00640 1 -- 172.21.15.91:0/1406168827 wait complete. 2024-02-02T13:31:27.775 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:27.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.980+0000 7f72a4800640 1 Processor -- start 2024-02-02T13:31:27.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.980+0000 7f72a4800640 1 -- start start 2024-02-02T13:31:27.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.981+0000 7f72a4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f729c1597a0 0x7f729c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.981+0000 7f72a4800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f729c0c9950 0x7f729c0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.981+0000 7f72a4800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c06b5f0 con 0x7f729c0cd690 2024-02-02T13:31:27.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.981+0000 7f72a4800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c06c450 con 0x7f729c0c9950 2024-02-02T13:31:27.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.981+0000 7f72a4800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c159b80 con 0x7f729c1597a0 2024-02-02T13:31:27.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f729c0c9950 0x7f729c0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a1400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f729c0cd690 0x7f729c0c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44072/0 (socket says 172.21.15.91:44072) 2024-02-02T13:31:27.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a1400640 1 -- 172.21.15.91:0/3726145299 learned_addr learned my addr 172.21.15.91:0/3726145299 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:27.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a1e00640 1 --2- 172.21.15.91:0/3726145299 >> v2:172.21.15.113:3300/0 conn(0x7f729c1597a0 0x7f729c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a2800640 1 -- 172.21.15.91:0/3726145299 >> v2:172.21.15.113:3300/0 conn(0x7f729c1597a0 msgr2=0x7f729c06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:27.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a2800640 1 --2- 172.21.15.91:0/3726145299 >> v2:172.21.15.113:3300/0 conn(0x7f729c1597a0 0x7f729c06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a2800640 1 -- 172.21.15.91:0/3726145299 >> v1:172.21.15.91:6789/0 conn(0x7f729c0cd690 legacy=0x7f729c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a2800640 1 -- 172.21.15.91:0/3726145299 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c06b840 con 0x7f729c0c9950 2024-02-02T13:31:27.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.982+0000 7f72a0a00640 1 -- 172.21.15.91:0/3726145299 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663374512 0 0) 0x7f729c06b5f0 con 0x7f729c0cd690 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.984+0000 7f72a2800640 1 --2- 172.21.15.91:0/3726145299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c0c9950 0x7f729c0cdb20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f729c06d7d0 tx=0x7f729402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78670ce78965f52a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.984+0000 7f72a0a00640 1 -- 172.21.15.91:0/3726145299 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729402cc80 con 0x7f729c0c9950 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.984+0000 7f72a0a00640 1 -- 172.21.15.91:0/3726145299 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729402cde0 con 0x7f729c0c9950 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.984+0000 7f72a0a00640 1 -- 172.21.15.91:0/3726145299 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294034660 con 0x7f729c0c9950 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 -- 172.21.15.91:0/3726145299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c0c9950 msgr2=0x7f729c0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 --2- 172.21.15.91:0/3726145299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c0c9950 0x7f729c0cdb20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f729c06d7d0 tx=0x7f729402bed0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 -- 172.21.15.91:0/3726145299 shutdown_connections 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 --2- 172.21.15.91:0/3726145299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c0c9950 0x7f729c0cdb20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 --2- 172.21.15.91:0/3726145299 >> v2:172.21.15.113:3300/0 conn(0x7f729c1597a0 0x7f729c06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 -- 172.21.15.91:0/3726145299 >> 172.21.15.91:0/3726145299 conn(0x7f729c0cec80 msgr2=0x7f729c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 -- 172.21.15.91:0/3726145299 shutdown_connections 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.985+0000 7f72a4800640 1 -- 172.21.15.91:0/3726145299 wait complete. 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.986+0000 7f72a4800640 1 Processor -- start 2024-02-02T13:31:27.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 -- start start 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c0c9950 0x7f729c1f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cd690 0x7f729c0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1f4400 con 0x7f729c1597a0 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1f1af0 con 0x7f729c0c9950 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.987+0000 7f72a4800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1f1250 con 0x7f729c0cd690 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49750/0 (socket says 172.21.15.91:49750) 2024-02-02T13:31:27.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 -- 172.21.15.91:0/1113476714 learned_addr learned my addr 172.21.15.91:0/1113476714 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 -- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cd690 msgr2=0x7f729c0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cd690 0x7f729c0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 -- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c0c9950 msgr2=0x7f729c1f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c0c9950 0x7f729c1f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 -- 172.21.15.91:0/1113476714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009a70 con 0x7f729c1597a0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.988+0000 7f72a2800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f729402c4e0 tx=0x7f7294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.989+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294040650 con 0x7f729c1597a0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.989+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72940407d0 con 0x7f729c1597a0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.989+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294040b80 con 0x7f729c1597a0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.989+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c1f1530 con 0x7f729c1597a0 2024-02-02T13:31:27.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.989+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c0c7e30 con 0x7f729c1597a0 2024-02-02T13:31:27.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.990+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c066cf0 con 0x7f729c1597a0 2024-02-02T13:31:27.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.991+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7294040ce0 con 0x7f729c1597a0 2024-02-02T13:31:27.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.992+0000 7f729a000640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 0x7f727406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:27.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.992+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72940ab5f0 con 0x7f729c1597a0 2024-02-02T13:31:27.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.992+0000 7f72a1e00640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 0x7f727406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:27.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.993+0000 7f72a1e00640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 0x7f727406c8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f728c006fd0 tx=0x7f728c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:27.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:27.996+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7294076bd0 con 0x7f729c1597a0 2024-02-02T13:31:28.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.228+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729c0cc7c0 con 0x7f729c1597a0 2024-02-02T13:31:28.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.229+0000 7f729a000640 1 -- 172.21.15.91:0/1113476714 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f7294009450 con 0x7f729c1597a0 2024-02-02T13:31:28.231 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:28.231 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 msgr2=0x7f727406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 0x7f727406c8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f728c006fd0 tx=0x7f728c008040 comp rx=0 tx=0).stop 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 msgr2=0x7f729c14f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f729402c4e0 tx=0x7f7294004a40 comp rx=0 tx=0).stop 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 shutdown_connections 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f727406a430 0x7f727406c8f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cd690 0x7f729c0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f729c1597a0 0x7f729c14f460 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 --2- 172.21.15.91:0/1113476714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c0c9950 0x7f729c1f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 >> 172.21.15.91:0/1113476714 conn(0x7f729c0cec80 msgr2=0x7f729c15bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 shutdown_connections 2024-02-02T13:31:28.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.232+0000 7f72a4800640 1 -- 172.21.15.91:0/1113476714 wait complete. 2024-02-02T13:31:28.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:31:16.893346+0000', 'modified': '2024-02-02T13:31:20.388634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5578': {'gid': 5578, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/1833616710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 1833616710}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 1833616710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:31:28.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:31:28.656 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-02T13:31:28.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.872+0000 7f9509800640 1 Processor -- start 2024-02-02T13:31:28.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.873+0000 7f9509800640 1 -- start start 2024-02-02T13:31:28.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.874+0000 7f9509800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95040cb580 0x7f95040ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.874+0000 7f9509800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040ac960 0x7f95040acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.874+0000 7f9509800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040c94a0 con 0x7f950415d3a0 2024-02-02T13:31:28.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.874+0000 7f9509800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95040ad6e0 con 0x7f95040cb580 2024-02-02T13:31:28.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.874+0000 7f9509800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040c96f0 con 0x7f95040ac960 2024-02-02T13:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95040cb580 0x7f95040ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9503400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040ac960 0x7f95040acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95040cb580 0x7f95040ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:34398/0 (socket says 172.21.15.91:34398) 2024-02-02T13:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 -- 172.21.15.91:0/2380080569 learned_addr learned my addr 172.21.15.91:0/2380080569 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 -- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040ac960 msgr2=0x7f95040acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 --2- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040ac960 0x7f95040acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 -- 172.21.15.91:0/2380080569 >> v1:172.21.15.113:6789/0 conn(0x7f950415d3a0 legacy=0x7f95040cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.875+0000 7f9502a00640 1 -- 172.21.15.91:0/2380080569 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040c99d0 con 0x7f95040cb580 2024-02-02T13:31:28.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.877+0000 7f9502a00640 1 --2- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040cb580 0x7f95040ca1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009870 tx=0x7f94f402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda050fbf9cbc041 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9502000640 1 -- 172.21.15.91:0/2380080569 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4030840 con 0x7f95040cb580 2024-02-02T13:31:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9502000640 1 -- 172.21.15.91:0/2380080569 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40309a0 con 0x7f95040cb580 2024-02-02T13:31:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9502000640 1 -- 172.21.15.91:0/2380080569 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4030d30 con 0x7f95040cb580 2024-02-02T13:31:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 -- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040cb580 msgr2=0x7f95040ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 --2- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040cb580 0x7f95040ca1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009870 tx=0x7f94f402b980 comp rx=0 tx=0).stop 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 -- 172.21.15.91:0/2380080569 shutdown_connections 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 --2- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040ac960 0x7f95040acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 --2- 172.21.15.91:0/2380080569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040cb580 0x7f95040ca1d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.878+0000 7f9509800640 1 -- 172.21.15.91:0/2380080569 >> 172.21.15.91:0/2380080569 conn(0x7f95040b0700 msgr2=0x7f95040b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.879+0000 7f9509800640 1 -- 172.21.15.91:0/2380080569 shutdown_connections 2024-02-02T13:31:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.879+0000 7f9509800640 1 -- 172.21.15.91:0/2380080569 wait complete. 2024-02-02T13:31:28.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.880+0000 7f9509800640 1 Processor -- start 2024-02-02T13:31:28.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.880+0000 7f9509800640 1 -- start start 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040ac960 0x7f950415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040cb580 0x7f9504145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950414f930 con 0x7f950415d3a0 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040c3750 con 0x7f95040ac960 2024-02-02T13:31:28.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.881+0000 7f9509800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040c3a30 con 0x7f95040cb580 2024-02-02T13:31:28.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49758/0 (socket says 172.21.15.91:49758) 2024-02-02T13:31:28.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 -- 172.21.15.91:0/3940939904 learned_addr learned my addr 172.21.15.91:0/3940939904 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f94f9e00640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040cb580 0x7f9504145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9502a00640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040ac960 0x7f950415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 -- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040cb580 msgr2=0x7f9504145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040cb580 0x7f9504145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 -- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040ac960 msgr2=0x7f950415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040ac960 0x7f950415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.882+0000 7f9503400640 1 -- 172.21.15.91:0/3940939904 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f40094d0 con 0x7f950415d3a0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f9503400640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00efc0 tx=0x7f94fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f94fb400640 1 -- 172.21.15.91:0/3940939904 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc014480 con 0x7f950415d3a0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f94fb400640 1 -- 172.21.15.91:0/3940939904 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc00f040 con 0x7f950415d3a0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f94fb400640 1 -- 172.21.15.91:0/3940939904 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc014820 con 0x7f950415d3a0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95040ad4b0 con 0x7f950415d3a0 2024-02-02T13:31:28.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.883+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95040c3ff0 con 0x7f950415d3a0 2024-02-02T13:31:28.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.885+0000 7f94fb400640 1 -- 172.21.15.91:0/3940939904 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f94fc014980 con 0x7f950415d3a0 2024-02-02T13:31:28.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.886+0000 7f94fb400640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 0x7f94dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.886+0000 7f94fb400640 1 -- 172.21.15.91:0/3940939904 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94fc089840 con 0x7f950415d3a0 2024-02-02T13:31:28.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.886+0000 7f9502a00640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 0x7f94dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.887+0000 7f950c56f740 1 Processor -- start 2024-02-02T13:31:28.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.887+0000 7f950c56f740 1 -- start start 2024-02-02T13:31:28.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.887+0000 7f9502a00640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 0x7f94dc06c8f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f94f4036000 tx=0x7f94f4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c965ed0 0x55e80c9931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c993700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c941650 con 0x55e80c995f50 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c8d6000 con 0x55e80c965ed0 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e80c8d3790 con 0x55e80c969ad0 2024-02-02T13:31:28.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f9502a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f9502a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49760/0 (socket says 172.21.15.91:49760) 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.888+0000 7f9502a00640 1 -- 172.21.15.91:0/41137913 learned_addr learned my addr 172.21.15.91:0/41137913 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f94f9e00640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c965ed0 0x55e80c9931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9503400640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c993700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 -- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 msgr2=0x55e80c993700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c993700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 -- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c965ed0 msgr2=0x55e80c9931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c965ed0 0x55e80c9931c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 -- 172.21.15.91:0/41137913 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e80c896710 con 0x55e80c995f50 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9503400640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c993700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:28.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f9502a00640 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f660f460d5348ce server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f94f3400640 1 -- 172.21.15.91:0/41137913 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4045e10 con 0x55e80c995f50 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f94f3400640 1 -- 172.21.15.91:0/41137913 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4039450 con 0x55e80c995f50 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.889+0000 7f94f3400640 1 -- 172.21.15.91:0/41137913 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40397c0 con 0x55e80c995f50 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 -- 172.21.15.91:0/41137913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 msgr2=0x55e80c996330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f4030000 comp rx=0 tx=0).stop 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 -- 172.21.15.91:0/41137913 shutdown_connections 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c993700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c965ed0 0x55e80c9931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 --2- 172.21.15.91:0/41137913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c995f50 0x55e80c996330 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 -- 172.21.15.91:0/41137913 >> 172.21.15.91:0/41137913 conn(0x55e80c9681e0 msgr2=0x55e80c9b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 -- 172.21.15.91:0/41137913 shutdown_connections 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.890+0000 7f950c56f740 1 -- 172.21.15.91:0/41137913 wait complete. 2024-02-02T13:31:28.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.891+0000 7f950c56f740 1 -- start start 2024-02-02T13:31:28.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.891+0000 7f950c56f740 1 Processor -- start 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c969ad0 0x55e80c9895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c98f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c8cf080 con 0x55e80c996430 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c870850 con 0x55e80c969ad0 2024-02-02T13:31:28.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e80c8704f0 con 0x55e80c995f50 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f950c56f740 10 client.0 _ll_register_callbacks cb 0x55e80c8c89c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49762/0 (socket says 172.21.15.91:49762) 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.892+0000 7f9503400640 1 -- 172.21.15.91:0/491279060 learned_addr learned my addr 172.21.15.91:0/491279060 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 -- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 msgr2=0x55e80c98f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c98f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 -- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c969ad0 msgr2=0x55e80c9895e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c969ad0 0x55e80c9895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 -- 172.21.15.91:0/491279060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95040c99d0 con 0x55e80c996430 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f9503400640 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009a40 tx=0x7f94fc058b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9444c16c0adc90d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f94ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:31:28.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc070070 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94fc058cb0 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e80c8f95d0 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94fc058e70 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f950c56f740 10 client.7314 fetch_fsmap learned FSMap version 56 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.893+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e80c8cd700 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e80c8f95d0 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 56) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f94fc024020 con 0x55e80c996430 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 10 client.7314 fetch_fsmap finished waiting for FSMap version 56 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 10 client.7314 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 20 client.7314 populate_metadata read hostname 'smithi091' 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c98c350 0x55e80c9cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 4 client.7314 mds_command: new command op to 5578 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:31:28.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 --> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e80c8c18d0 con 0x55e80c98c350 2024-02-02T13:31:28.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.894+0000 7f9502a00640 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c98c350 0x55e80c9cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.895+0000 7f94ebe00640 10 client.7314 ms_handle_connect on v2:172.21.15.91:6835/1833616710 2024-02-02T13:31:28.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.895+0000 7f9502a00640 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c98c350 0x55e80c9cdd90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b99a5e6da4fea35 server_cookie=2fe2b7badb8e8179 in_seq=0 out_seq=0 2024-02-02T13:31:28.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.895+0000 7f94ebe00640 1 -- 172.21.15.91:0/491279060 <== mds.0 v2:172.21.15.91:6835/1833616710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e80c8c18d0 con 0x55e80c98c350 2024-02-02T13:31:28.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.895+0000 7f94ebe00640 10 client.7314 handle_command_reply: tid=0 2024-02-02T13:31:28.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.895+0000 7f950c56f740 1 client.7314 shutdown 2024-02-02T13:31:28.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 msgr2=0x55e80c98fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009a40 tx=0x7f94fc058b10 comp rx=0 tx=0).stop 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 shutdown_connections 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c98c350 0x55e80c9cdd90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c98f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c98fba0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 --2- 172.21.15.91:0/491279060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c969ad0 0x55e80c9895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f94ebe00640 0 client.7314 ms_handle_reset on v2:172.21.15.91:6835/1833616710 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 >> 172.21.15.91:0/491279060 conn(0x55e80c965ed0 msgr2=0x55e80c999d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 shutdown_connections 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 1 -- 172.21.15.91:0/491279060 wait complete. 2024-02-02T13:31:28.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.896+0000 7f950c56f740 20 client.7314 trim_cache size 0 max 16384 2024-02-02T13:31:28.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.912+0000 7f950c56f740 1 Processor -- start 2024-02-02T13:31:28.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.912+0000 7f950c56f740 1 -- start start 2024-02-02T13:31:28.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c965540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c995f50 0x55e80c9c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c636f20 con 0x55e80c996430 2024-02-02T13:31:28.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80c990510 con 0x55e80c995f50 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e80c98fe50 con 0x55e80c969ad0 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49778/0 (socket says 172.21.15.91:49778) 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 -- 172.21.15.91:0/1853106245 learned_addr learned my addr 172.21.15.91:0/1853106245 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 -- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 msgr2=0x55e80c965540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c965540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 -- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c995f50 msgr2=0x55e80c9c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c995f50 0x55e80c9c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.913+0000 7f9503400640 1 -- 172.21.15.91:0/1853106245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e80c903280 con 0x55e80c996430 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f9503400640 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0589e0 tx=0x7f94fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd840d0cdadc3c58 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f94f3e00640 1 -- 172.21.15.91:0/1853106245 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc070040 con 0x55e80c996430 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f94f3e00640 1 -- 172.21.15.91:0/1853106245 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc058cb0 con 0x55e80c996430 2024-02-02T13:31:28.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f94f3e00640 1 -- 172.21.15.91:0/1853106245 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc058e20 con 0x55e80c996430 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f950c56f740 1 -- 172.21.15.91:0/1853106245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 msgr2=0x55e80c9c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.914+0000 7f950c56f740 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0589e0 tx=0x7f94fc018000 comp rx=0 tx=0).stop 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 -- 172.21.15.91:0/1853106245 shutdown_connections 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c996430 0x55e80c9c9f90 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c995f50 0x55e80c9c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 --2- 172.21.15.91:0/1853106245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c969ad0 0x55e80c965540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 -- 172.21.15.91:0/1853106245 >> 172.21.15.91:0/1853106245 conn(0x55e80c965ed0 msgr2=0x55e80c98f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 -- 172.21.15.91:0/1853106245 shutdown_connections 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 -- 172.21.15.91:0/1853106245 wait complete. 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 -- start start 2024-02-02T13:31:28.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.915+0000 7f950c56f740 1 Processor -- start 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c9b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c996430 0x55e80ca1a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80ca1ae80 con 0x55e80c969ad0 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80ca190e0 con 0x55e80c996430 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e80c98db20 con 0x55e80c995f50 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.916+0000 7f950c56f740 10 client.0 _ll_register_callbacks cb 0x55e80c98efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49794/0 (socket says 172.21.15.91:49794) 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 -- 172.21.15.91:0/2250642795 learned_addr learned my addr 172.21.15.91:0/2250642795 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 -- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 msgr2=0x55e80c9b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c9b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 -- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c996430 msgr2=0x55e80ca1a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:28.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9503400640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c996430 0x55e80ca1a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c996430 0x55e80ca1a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 -- 172.21.15.91:0/2250642795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e80c8cd700 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f9502a00640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009990 tx=0x7f94f40091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cdfa9808c901aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.917+0000 7f94ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4062070 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94f4039450 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94f403e040 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f405d070 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 10 client.7320 fetch_fsmap learned FSMap version 56 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e80c86fa60 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4039450 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 56) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f94f4065020 con 0x55e80c969ad0 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 10 client.7320 fetch_fsmap finished waiting for FSMap version 56 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 10 client.7320 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 20 client.7320 populate_metadata read hostname 'smithi091' 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c9b08d0 0x55e80c9b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:28.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 4 client.7320 mds_command: new command op to 5578 tid=0[{"prefix": "status"}] 2024-02-02T13:31:28.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 --> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e80c8c8100 con 0x55e80c9b08d0 2024-02-02T13:31:28.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.918+0000 7f94f9e00640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c9b08d0 0x55e80c9b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f94f9e00640 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c9b08d0 0x55e80c9b0cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4db8da040f32aaa server_cookie=6632cef28a979aac in_seq=0 out_seq=0 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f94ebe00640 10 client.7320 ms_handle_connect on v2:172.21.15.91:6835/1833616710 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f94ebe00640 1 -- 172.21.15.91:0/2250642795 <== mds.0 v2:172.21.15.91:6835/1833616710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e80c8c8100 con 0x55e80c9b08d0 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f94ebe00640 10 client.7320 handle_command_reply: tid=0 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 client.7320 shutdown 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 msgr2=0x55e80c9c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009990 tx=0x7f94f40091e0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 shutdown_connections 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:6835/1833616710,v1:172.21.15.91:6837/1833616710] conn(0x55e80c9b08d0 0x55e80c9b0cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e80c995f50 0x55e80c9b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e80c996430 0x55e80ca1a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 --2- 172.21.15.91:0/2250642795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55e80c969ad0 0x55e80c9c9a50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f94ebe00640 0 client.7320 ms_handle_reset on v2:172.21.15.91:6835/1833616710 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.919+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 >> 172.21.15.91:0/2250642795 conn(0x55e80c965ed0 msgr2=0x55e80ca11ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 shutdown_connections 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f950c56f740 1 -- 172.21.15.91:0/2250642795 wait complete. 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f950c56f740 20 client.7320 trim_cache size 0 max 16384 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5578, 2024-02-02T13:31:28.925 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 12.01969085, 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 56, 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 128, 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 128, 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 17.533869332999998 2024-02-02T13:31:28.926 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 msgr2=0x7f94dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 0x7f94dc06c8f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f94f4036000 tx=0x7f94f4008030 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 msgr2=0x7f9504145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00efc0 tx=0x7f94fc00c490 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 shutdown_connections 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f94dc06a430 0x7f94dc06c8f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95040cb580 0x7f9504145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950415d3a0 0x7f9504145bd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 --2- 172.21.15.91:0/3940939904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95040ac960 0x7f950415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 >> 172.21.15.91:0/3940939904 conn(0x7f95040b0700 msgr2=0x7f95040b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.920+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 shutdown_connections 2024-02-02T13:31:28.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:28.921+0000 7f9509800640 1 -- 172.21.15.91:0/3940939904 wait complete. 2024-02-02T13:31:28.938 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:31:29.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.116+0000 7f1f5b200640 1 Processor -- start 2024-02-02T13:31:29.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.117+0000 7f1f5b200640 1 -- start start 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f5c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f5b200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0ad2e0 con 0x7f1f5c06c820 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f5b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0ad5c0 con 0x7f1f5c15d2c0 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f5b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c06b4f0 con 0x7f1f5c0ccd70 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f51200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1f5c06c820 0x7f1f5c0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44082/0 (socket says 172.21.15.91:44082) 2024-02-02T13:31:29.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.118+0000 7f1f51200640 1 -- 172.21.15.91:0/899738609 learned_addr learned my addr 172.21.15.91:0/899738609 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:29.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673749513 0 0) 0x7f1f5c0ad2e0 con 0x7f1f5c06c820 2024-02-02T13:31:29.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f3c003520 con 0x7f1f5c06c820 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f5a800640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f5c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438538267 0 0) 0x7f1f3c003520 con 0x7f1f5c06c820 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 msgr2=0x7f1f5c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f5c0c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 >> v1:172.21.15.113:6789/0 conn(0x7f1f5c15d2c0 legacy=0x7f1f5c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c06b7d0 con 0x7f1f5c06c820 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1f44001670 con 0x7f1f5c06c820 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 >> v1:172.21.15.91:6789/0 conn(0x7f1f5c06c820 legacy=0x7f1f5c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f3c0039a0 0x7f1f3c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c06b4f0 con 0x7f1f3c004d00 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c00bb40 con 0x7f1f3c0039a0 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.119+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c00bd00 con 0x7f1f5c0ccd70 2024-02-02T13:31:29.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f51200640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f59e00640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f3c0039a0 0x7f1f3c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 -- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 msgr2=0x7f1f3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 -- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f3c0039a0 msgr2=0x7f1f3c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f3c0039a0 0x7f1f3c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f5a800640 1 -- 172.21.15.91:0/899738609 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c003520 con 0x7f1f3c004d00 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f51200640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c0ccd70 0x7f1f3c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.120+0000 7f1f59e00640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f3c0039a0 0x7f1f3c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.121+0000 7f1f5a800640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f3c005180 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c06d590 tx=0x7f1f3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2225bbf12e4061f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.121+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5402cc50 con 0x7f1f3c004d00 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.121+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5402cdb0 con 0x7f1f3c004d00 2024-02-02T13:31:29.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.121+0000 7f1f59400640 1 -- 172.21.15.91:0/899738609 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54034660 con 0x7f1f3c004d00 2024-02-02T13:31:29.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 -- 172.21.15.91:0/899738609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 msgr2=0x7f1f3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 --2- 172.21.15.91:0/899738609 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f3c005180 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c06d590 tx=0x7f1f3c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5a800640 1 -- 172.21.15.91:0/899738609 reap_dead start 2024-02-02T13:31:29.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 -- 172.21.15.91:0/899738609 shutdown_connections 2024-02-02T13:31:29.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 -- 172.21.15.91:0/899738609 >> 172.21.15.91:0/899738609 conn(0x7f1f5c0cea90 msgr2=0x7f1f5c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 -- 172.21.15.91:0/899738609 shutdown_connections 2024-02-02T13:31:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.122+0000 7f1f5b200640 1 -- 172.21.15.91:0/899738609 wait complete. 2024-02-02T13:31:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.124+0000 7f1f5b200640 1 Processor -- start 2024-02-02T13:31:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.124+0000 7f1f5b200640 1 -- start start 2024-02-02T13:31:29.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.124+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.124+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f5c151d70 0x7f1f5c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c15c110 0x7f1f5c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f5b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c15cad0 con 0x7f1f3c004d00 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f5b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c89d0 con 0x7f1f5c151d70 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f5b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0aebd0 con 0x7f1f5c15c110 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:49808/0 (socket says 172.21.15.91:49808) 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 -- 172.21.15.91:0/500573037 learned_addr learned my addr 172.21.15.91:0/500573037 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 -- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c15c110 msgr2=0x7f1f5c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f5a800640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c15c110 0x7f1f5c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f51200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f5c151d70 0x7f1f5c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c15c110 0x7f1f5c15c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 -- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f5c151d70 msgr2=0x7f1f5c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f5c151d70 0x7f1f5c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.125+0000 7f1f59e00640 1 -- 172.21.15.91:0/500573037 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c15d820 con 0x7f1f3c004d00 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f59e00640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00d5c0 tx=0x7f1f4c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:29.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c007220 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c007380 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0067d0 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0aeeb0 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.126+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0c6720 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.127+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0c6940 con 0x7f1f3c004d00 2024-02-02T13:31:29.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.128+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f4c006930 con 0x7f1f3c004d00 2024-02-02T13:31:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.129+0000 7f1f52a00640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 0x7f1f3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.129+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f4c088a90 con 0x7f1f3c004d00 2024-02-02T13:31:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.129+0000 7f1f51200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 0x7f1f3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:29.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.130+0000 7f1f51200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 0x7f1f3406c8f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:29.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.134+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f4c053fe0 con 0x7f1f3c004d00 2024-02-02T13:31:29.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.363+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f5c04dff0 con 0x7f1f3c004d00 2024-02-02T13:31:29.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.364+0000 7f1f52a00640 1 -- 172.21.15.91:0/500573037 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1f4c01f020 con 0x7f1f3c004d00 2024-02-02T13:31:29.366 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:31:29.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 msgr2=0x7f1f3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 0x7f1f3406c8f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f44009040 comp rx=0 tx=0).stop 2024-02-02T13:31:29.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 msgr2=0x7f1f5c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:29.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00d5c0 tx=0x7f1f4c00da80 comp rx=0 tx=0).stop 2024-02-02T13:31:29.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 shutdown_connections 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1f3406a430 0x7f1f3406c8f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f5c15c110 0x7f1f5c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f5c151d70 0x7f1f5c160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 --2- 172.21.15.91:0/500573037 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f3c004d00 0x7f1f5c0ae4d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.366+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 >> 172.21.15.91:0/500573037 conn(0x7f1f5c0cea90 msgr2=0x7f1f5c15a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.367+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 shutdown_connections 2024-02-02T13:31:29.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:29.367+0000 7f1f5b200640 1 -- 172.21.15.91:0/500573037 wait complete. 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:31:29.387 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:31:29.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.388 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:31:29.436 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:31:29.436 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:31:29.436 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:31:29.436 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:31:29.437 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:31:29.438 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:29.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.439 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:31:29.516 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:31:29.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.521 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:31:29.572 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:31:29.573 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:31:29.573 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:31:29.573 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:31:29.573 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:31:29.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:31:29.574 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:31:29.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.752 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:31:29.801 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:31:29.801 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:31:29.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:31:29.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.802 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:31:29.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:31:29.892 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:31:29.944 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:31:29.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:31:29.945 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:31:29.993 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:31:29.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:29.994 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:30.122 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:30.122 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:31:30.158 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:30.158 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:31:30.158 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:31:30.158 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:31:30.158 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:31:30.162 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:31:30.162 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:31:30.162 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:31:30.162 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:31:30.162 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:31:30.166 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:31:30.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:30.166 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:31:30.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:30.242 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:30.303 INFO:teuthology.orchestra.run.smithi091.stdout:client5252 2024-02-02T13:31:30.304 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:30.304 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:30.352 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:31:30.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.597+0000 7f4a07000640 1 Processor -- start 2024-02-02T13:31:30.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.597+0000 7f4a07000640 1 -- start start 2024-02-02T13:31:30.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a07000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a000add80 0x7f4a000706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a07000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000ce3b0 con 0x7f4a0015d3a0 2024-02-02T13:31:30.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a07000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000ce8e0 con 0x7f4a000c9450 2024-02-02T13:31:30.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a07000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000adbd0 con 0x7f4a000add80 2024-02-02T13:31:30.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a05c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4a0015d3a0 0x7f4a0006b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50326/0 (socket says 172.21.15.91:50326) 2024-02-02T13:31:30.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a05c00640 1 -- 172.21.15.91:0/2406037836 learned_addr learned my addr 172.21.15.91:0/2406037836 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:30.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> v2:172.21.15.113:3300/0 conn(0x7f4a000add80 0x7f4a000706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.599+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972009850 0 0) 0x7f4a000ce3b0 con 0x7f4a0015d3a0 2024-02-02T13:31:30.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e0003520 con 0x7f4a0015d3a0 2024-02-02T13:31:30.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 906657167 0 0) 0x7f49e0003520 con 0x7f4a0015d3a0 2024-02-02T13:31:30.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 >> v2:172.21.15.113:3300/0 conn(0x7f4a000add80 msgr2=0x7f4a000706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 --2- 172.21.15.91:0/2406037836 >> v2:172.21.15.113:3300/0 conn(0x7f4a000add80 0x7f4a000706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 >> v1:172.21.15.113:6789/0 conn(0x7f4a000c9450 legacy=0x7f4a000c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0015d780 con 0x7f4a0015d3a0 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f49f0001670 con 0x7f4a0015d3a0 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 >> v1:172.21.15.91:6789/0 conn(0x7f4a0015d3a0 legacy=0x7f4a0006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00039c0 0x7f49e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 0x7f49e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49e0005380 0x7f49e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000adbd0 con 0x7f49e00047c0 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e000c240 con 0x7f49e00039c0 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e000c400 con 0x7f49e0005380 2024-02-02T13:31:30.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> v2:172.21.15.113:3300/0 conn(0x7f4a000add80 0x7f4a000706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.600+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 0x7f49e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05c00640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00039c0 0x7f49e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a06600640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49e0005380 0x7f49e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05200640 1 -- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49e0005380 msgr2=0x7f49e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49e0005380 0x7f49e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05200640 1 -- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00039c0 msgr2=0x7f49e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00039c0 0x7f49e0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a05200640 1 -- 172.21.15.91:0/2406037836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0003520 con 0x7f49e00047c0 2024-02-02T13:31:30.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.601+0000 7f4a06600640 1 -- 172.21.15.91:0/2406037836 reap_dead start 2024-02-02T13:31:30.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.602+0000 7f4a05200640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 0x7f49e0004c40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4a00071cb0 tx=0x7f49e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67030a9c55509700 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:30.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8004230 con 0x7f49e00047c0 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8004390 con 0x7f49e00047c0 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f49ffe00640 1 -- 172.21.15.91:0/2406037836 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8003850 con 0x7f49e00047c0 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 -- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 msgr2=0x7f49e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 0x7f49e0004c40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4a00071cb0 tx=0x7f49e00057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 -- 172.21.15.91:0/2406037836 shutdown_connections 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 --2- 172.21.15.91:0/2406037836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49e00047c0 0x7f49e0004c40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 -- 172.21.15.91:0/2406037836 >> 172.21.15.91:0/2406037836 conn(0x7f4a000cea90 msgr2=0x7f4a000ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 -- 172.21.15.91:0/2406037836 shutdown_connections 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.603+0000 7f4a07000640 1 -- 172.21.15.91:0/2406037836 wait complete. 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.605+0000 7f4a07000640 1 Processor -- start 2024-02-02T13:31:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.605+0000 7f4a07000640 1 -- start start 2024-02-02T13:31:30.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00047c0 0x7f4a000acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a000af140 0x7f4a000af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c4340 con 0x7f4a00150e00 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00151780 con 0x7f49e00047c0 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a07000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c3450 con 0x7f4a000af140 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a06600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a06600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38318/0 (socket says 172.21.15.91:38318) 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.606+0000 7f4a06600640 1 -- 172.21.15.91:0/2136710284 learned_addr learned my addr 172.21.15.91:0/2136710284 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a05c00640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00047c0 0x7f4a000acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 -- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a000af140 msgr2=0x7f4a000af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a000af140 0x7f4a000af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 -- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00047c0 msgr2=0x7f4a000acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00047c0 0x7f4a000acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 -- 172.21.15.91:0/2136710284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000c3730 con 0x7f4a00150e00 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a06600640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ae160 tx=0x7f49f800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8003df0 con 0x7f4a00150e00 2024-02-02T13:31:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f8013420 con 0x7f4a00150e00 2024-02-02T13:31:30.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80137d0 con 0x7f4a00150e00 2024-02-02T13:31:30.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.607+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a000c3a10 con 0x7f4a00150e00 2024-02-02T13:31:30.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.608+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a000c3fd0 con 0x7f4a00150e00 2024-02-02T13:31:30.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.609+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0004dff0 con 0x7f4a00150e00 2024-02-02T13:31:30.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.610+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f49f8013930 con 0x7f4a00150e00 2024-02-02T13:31:30.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.610+0000 7f49fd600640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 0x7f49d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:30.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.611+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49f8088570 con 0x7f4a00150e00 2024-02-02T13:31:30.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.611+0000 7f4a05c00640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 0x7f49d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:30.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.611+0000 7f4a05c00640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 0x7f49d806c8f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000c00 tx=0x7f49f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:30.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.615+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f49f8053b60 con 0x7f4a00150e00 2024-02-02T13:31:30.732 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:31:30.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:31:30.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.844+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a000b4a10 con 0x7f4a00150e00 2024-02-02T13:31:30.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.844+0000 7f49fd600640 1 -- 172.21.15.91:0/2136710284 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f49f8057180 con 0x7f4a00150e00 2024-02-02T13:31:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.847+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 msgr2=0x7f49d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.847+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 0x7f49d806c8f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000c00 tx=0x7f49f0009040 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 msgr2=0x7f4a00151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ae160 tx=0x7f49f800e960 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 shutdown_connections 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f49d806a430 0x7f49d806c8f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a000af140 0x7f4a000af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a00150e00 0x7f4a00151240 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 --2- 172.21.15.91:0/2136710284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00047c0 0x7f4a000acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 >> 172.21.15.91:0/2136710284 conn(0x7f4a000cea90 msgr2=0x7f4a0015c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 shutdown_connections 2024-02-02T13:31:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:30.848+0000 7f4a07000640 1 -- 172.21.15.91:0/2136710284 wait complete. 2024-02-02T13:31:30.866 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:31:30.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:31:30.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:31:30.866 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:31:30.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:31:30.867 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:31:30.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:31:30.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:31:30.867 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:31:30.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:30.867 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:31:30.918 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:31:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:31:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42275sec preferred_lft 42275sec 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:31:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.921 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:31:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:31:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:31:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:31:30.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:30.922 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:31:31.054 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:31:31.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.059 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:31:31.110 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:31:31.110 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:31:31.111 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:31:31.111 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:31:31.111 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:31:31.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:31:31.113 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:31:31.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.282 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:31:31.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:31:31.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:31:31.332 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:31:31.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:31:31.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:31:31.421 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:31:31.474 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:31:31.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:31:31.474 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:31:31.522 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:31:31.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.523 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:31.658 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:31.658 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:31:31.692 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:31:31.693 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:31:31.693 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:31:31.693 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:31:31.693 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:31:31.694 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:31:31.694 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:31:31.695 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:31:31.695 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:31:31.695 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:31:31.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:31:31.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:31:31.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:31.789 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:31.844 INFO:teuthology.orchestra.run.smithi113.stdout:client5662 2024-02-02T13:31:31.844 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:31.845 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:31.893 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:32.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.113+0000 7f1915e00640 1 Processor -- start 2024-02-02T13:31:32.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.114+0000 7f1915e00640 1 -- start start 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f19100b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19100b6990 0x7f19100cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19100ce000 0x7f19100b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100ce450 con 0x7f19100ce000 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100b7d40 con 0x7f19100b6990 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100b81d0 con 0x7f191015cee0 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19100ce000 0x7f19100b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19100ce000 0x7f19100b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38330/0 (socket says 172.21.15.91:38330) 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 -- 172.21.15.91:0/1562206473 learned_addr learned my addr 172.21.15.91:0/1562206473 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 -- 172.21.15.91:0/1562206473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 msgr2=0x7f19100b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:32.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 --2- 172.21.15.91:0/1562206473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f19100b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 -- 172.21.15.91:0/1562206473 >> v2:172.21.15.113:3300/0 conn(0x7f19100b6990 msgr2=0x7f19100cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:32.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 --2- 172.21.15.91:0/1562206473 >> v2:172.21.15.113:3300/0 conn(0x7f19100b6990 0x7f19100cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.115+0000 7f1915400640 1 -- 172.21.15.91:0/1562206473 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100b7130 con 0x7f19100ce000 2024-02-02T13:31:32.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f1915400640 1 --2- 172.21.15.91:0/1562206473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100b4870 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f19100b59a0 tx=0x7f190802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4213f4dc5012a5f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f190f400640 1 -- 172.21.15.91:0/1562206473 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190802ca70 con 0x7f19100ce000 2024-02-02T13:31:32.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f190f400640 1 -- 172.21.15.91:0/1562206473 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190802cbd0 con 0x7f19100ce000 2024-02-02T13:31:32.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f190f400640 1 -- 172.21.15.91:0/1562206473 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080343f0 con 0x7f19100ce000 2024-02-02T13:31:32.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f1915e00640 1 -- 172.21.15.91:0/1562206473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 msgr2=0x7f19100b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.117+0000 7f1915e00640 1 --2- 172.21.15.91:0/1562206473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100b4870 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f19100b59a0 tx=0x7f190802bed0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 -- 172.21.15.91:0/1562206473 shutdown_connections 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 --2- 172.21.15.91:0/1562206473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100b4870 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 --2- 172.21.15.91:0/1562206473 >> v2:172.21.15.113:3300/0 conn(0x7f19100b6990 0x7f19100cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 --2- 172.21.15.91:0/1562206473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f19100b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 -- 172.21.15.91:0/1562206473 >> 172.21.15.91:0/1562206473 conn(0x7f1910070690 msgr2=0x7f1910070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 -- 172.21.15.91:0/1562206473 shutdown_connections 2024-02-02T13:31:32.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.118+0000 7f1915e00640 1 -- 172.21.15.91:0/1562206473 wait complete. 2024-02-02T13:31:32.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.119+0000 7f1915e00640 1 Processor -- start 2024-02-02T13:31:32.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.119+0000 7f1915e00640 1 -- start start 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 0x7f1910158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f1910153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1910159190 con 0x7f19100ce000 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100ca040 con 0x7f19100b6990 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.120+0000 7f1915e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100ca320 con 0x7f191015cee0 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38334/0 (socket says 172.21.15.91:38334) 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 -- 172.21.15.91:0/1740059313 learned_addr learned my addr 172.21.15.91:0/1740059313 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 -- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 msgr2=0x7f1910153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f1910153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f1914a00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 0x7f1910158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 -- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 msgr2=0x7f1910158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 0x7f1910158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 -- 172.21.15.91:0/1740059313 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1908009b30 con 0x7f19100ce000 2024-02-02T13:31:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f1914a00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 0x7f1910158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.121+0000 7f190fe00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f18f800b4a0 tx=0x7f18f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.122+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8010610 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.122+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f8010770 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.122+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8010b20 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.122+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19100ca600 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.122+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19100ccad0 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.123+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19100cab20 con 0x7f19100ce000 2024-02-02T13:31:32.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.124+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f18f8010c80 con 0x7f19100ce000 2024-02-02T13:31:32.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.125+0000 7f190cc00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 0x7f18e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.125+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18f8088520 con 0x7f19100ce000 2024-02-02T13:31:32.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.125+0000 7f1914a00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 0x7f18e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.126+0000 7f1914a00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 0x7f18e806c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f19100b2430 tx=0x7f1900008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.131 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.129+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f19100cab20 con 0x7f19100ce000 2024-02-02T13:31:32.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.356+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19100cab20 con 0x7f19100ce000 2024-02-02T13:31:32.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.357+0000 7f190cc00640 1 -- 172.21.15.91:0/1740059313 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f19100cab20 con 0x7f19100ce000 2024-02-02T13:31:32.359 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:32.360 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:32.360 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:32.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 msgr2=0x7f18e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 0x7f18e806c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f19100b2430 tx=0x7f1900008040 comp rx=0 tx=0).stop 2024-02-02T13:31:32.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 msgr2=0x7f19100cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f18f800b4a0 tx=0x7f18f800b960 comp rx=0 tx=0).stop 2024-02-02T13:31:32.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 shutdown_connections 2024-02-02T13:31:32.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f18e806a430 0x7f18e806c8f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191015cee0 0x7f1910153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19100ce000 0x7f19100cc590 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 --2- 172.21.15.91:0/1740059313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19100b6990 0x7f1910158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.360+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 >> 172.21.15.91:0/1740059313 conn(0x7f1910070690 msgr2=0x7f19100b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.361+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 shutdown_connections 2024-02-02T13:31:32.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.361+0000 7f1915e00640 1 -- 172.21.15.91:0/1740059313 wait complete. 2024-02-02T13:31:32.381 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:31:32.605 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":5578,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.688784607000001,"mdsmap_epoch":56,"osdmap_epoch":128,"osdmap_epoch_barrier":128,"uptime":21.202980476} 2024-02-02T13:31:32.605 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 5578, "mdsmap_epoch": 56, "osdmap_epoch": 128, "osdmap_epoch_barrier": 128, "rank_uptime": 15.688784607, "state": "up:active", "uptime": 21.202980476, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:31:32.606 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-02-02T13:31:32.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.806+0000 7f8455e00640 1 Processor -- start 2024-02-02T13:31:32.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.806+0000 7f8455e00640 1 -- start start 2024-02-02T13:31:32.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f8455e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f845015a010 0x7f84500c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f8455e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84500ce130 0x7f84500ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f8455e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500cc7c0 con 0x7f84500cd470 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f8455e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84500ce930 con 0x7f845015a010 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f8455e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845006c890 con 0x7f84500ce130 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f845015a010 0x7f84500c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f845015a010 0x7f84500c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38338/0 (socket says 172.21.15.91:38338) 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 -- 172.21.15.91:0/3742144740 learned_addr learned my addr 172.21.15.91:0/3742144740 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:32.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 -- 172.21.15.91:0/3742144740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84500ce130 msgr2=0x7f84500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 --2- 172.21.15.91:0/3742144740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84500ce130 0x7f84500ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 -- 172.21.15.91:0/3742144740 >> v1:172.21.15.113:6789/0 conn(0x7f84500cd470 legacy=0x7f84500b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.808+0000 7f844ea00640 1 -- 172.21.15.91:0/3742144740 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845006cb70 con 0x7f845015a010 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.809+0000 7f844ea00640 1 --2- 172.21.15.91:0/3742144740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f845015a010 0x7f84500c9670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f844000bb10 tx=0x7f844002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e88a1ebde73a1e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.810+0000 7f844e000640 1 -- 172.21.15.91:0/3742144740 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844002ee70 con 0x7f845015a010 2024-02-02T13:31:32.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.810+0000 7f844e000640 1 -- 172.21.15.91:0/3742144740 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440036440 con 0x7f845015a010 2024-02-02T13:31:32.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.810+0000 7f844e000640 1 -- 172.21.15.91:0/3742144740 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400367d0 con 0x7f845015a010 2024-02-02T13:31:32.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.810+0000 7f8455e00640 1 -- 172.21.15.91:0/3742144740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f845015a010 msgr2=0x7f84500c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.810+0000 7f8455e00640 1 --2- 172.21.15.91:0/3742144740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f845015a010 0x7f84500c9670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f844000bb10 tx=0x7f844002e0c0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 -- 172.21.15.91:0/3742144740 shutdown_connections 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 --2- 172.21.15.91:0/3742144740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84500ce130 0x7f84500ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 --2- 172.21.15.91:0/3742144740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f845015a010 0x7f84500c9670 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 -- 172.21.15.91:0/3742144740 >> 172.21.15.91:0/3742144740 conn(0x7f84500cec80 msgr2=0x7f84500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 -- 172.21.15.91:0/3742144740 shutdown_connections 2024-02-02T13:31:32.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.811+0000 7f8455e00640 1 -- 172.21.15.91:0/3742144740 wait complete. 2024-02-02T13:31:32.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.812+0000 7f8455e00640 1 Processor -- start 2024-02-02T13:31:32.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.812+0000 7f8455e00640 1 -- start start 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f845015a010 0x7f84500c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500afdd0 0x7f84500b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84500c49a0 con 0x7f84500ce130 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84500c5200 con 0x7f84500afdd0 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.813+0000 7f8455e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84500c4c40 con 0x7f845015a010 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38340/0 (socket says 172.21.15.91:38340) 2024-02-02T13:31:32.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 -- 172.21.15.91:0/1574263171 learned_addr learned my addr 172.21.15.91:0/1574263171 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 -- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f845015a010 msgr2=0x7f84500c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f845015a010 0x7f84500c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 -- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500afdd0 msgr2=0x7f84500b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500afdd0 0x7f84500b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 -- 172.21.15.91:0/1574263171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844000b770 con 0x7f84500ce130 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.814+0000 7f844ea00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f844000b2c0 tx=0x7f8440004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.815+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440034950 con 0x7f84500ce130 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.815+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440034ad0 con 0x7f84500ce130 2024-02-02T13:31:32.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.815+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440034e80 con 0x7f84500ce130 2024-02-02T13:31:32.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.815+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84500c4f20 con 0x7f84500ce130 2024-02-02T13:31:32.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.815+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84500af8b0 con 0x7f84500ce130 2024-02-02T13:31:32.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.816+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84500afaa0 con 0x7f84500ce130 2024-02-02T13:31:32.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.817+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8440041480 con 0x7f84500ce130 2024-02-02T13:31:32.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.818+0000 7f8447e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 0x7f842806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:32.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.818+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84400abbf0 con 0x7f84500ce130 2024-02-02T13:31:32.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.818+0000 7f8445e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 0x7f842806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:32.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.819+0000 7f8445e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 0x7f842806c8f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8450152640 tx=0x7f8438006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:32.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:32.822+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8440077140 con 0x7f84500ce130 2024-02-02T13:31:33.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.005+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f84500afaa0 con 0x7f84500ce130 2024-02-02T13:31:33.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.447+0000 7f8447e00640 1 -- 172.21.15.91:0/1574263171 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f844003b0c0 con 0x7f84500ce130 2024-02-02T13:31:33.450 INFO:teuthology.orchestra.run.smithi091.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 msgr2=0x7f842806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 0x7f842806c8f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8450152640 tx=0x7f8438006c90 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 msgr2=0x7f84501519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f844000b2c0 tx=0x7f8440004730 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 shutdown_connections 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f842806a430 0x7f842806c8f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f845015a010 0x7f84500c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500afdd0 0x7f84500b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 --2- 172.21.15.91:0/1574263171 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84500ce130 0x7f84501519b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:33.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 >> 172.21.15.91:0/1574263171 conn(0x7f84500cec80 msgr2=0x7f84500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:33.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 shutdown_connections 2024-02-02T13:31:33.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:33.451+0000 7f8455e00640 1 -- 172.21.15.91:0/1574263171 wait complete. 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:31:33.472 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:31:33.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:33.786 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:33.886 INFO:teuthology.orchestra.run.smithi091.stdout:client5252 2024-02-02T13:31:33.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:33.887 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5252/osdmap') 2024-02-02T13:31:33.968 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 128 barrier 128 flags 0x588000 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:31:33.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:31:33.970 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:31:33.970 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:31:33.970 INFO:teuthology.orchestra.run.smithi091.stderr:799 bytes copied, 0.000100016 s, 8.0 MB/s 2024-02-02T13:31:33.972 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-02-02T13:31:34.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.174+0000 7f9eabe00640 1 Processor -- start 2024-02-02T13:31:34.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.174+0000 7f9eabe00640 1 -- start start 2024-02-02T13:31:34.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9eabe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ea415b0f0 0x7f9ea406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 0x7f9ea40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9eabe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea406d910 con 0x7f9ea415ad10 2024-02-02T13:31:34.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9eabe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea406d730 con 0x7f9ea415b0f0 2024-02-02T13:31:34.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9eabe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cbf50 con 0x7f9ea40cb580 2024-02-02T13:31:34.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea9400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9ea415ad10 0x7f9ea406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50336/0 (socket says 172.21.15.91:50336) 2024-02-02T13:31:34.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea9400640 1 -- 172.21.15.91:0/3895727316 learned_addr learned my addr 172.21.15.91:0/3895727316 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:34.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503774132 0 0) 0x7f9ea406d910 con 0x7f9ea415ad10 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea9e00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 0x7f9ea40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea8a00640 1 --2- 172.21.15.91:0/3895727316 >> v2:172.21.15.113:3300/0 conn(0x7f9ea415b0f0 0x7f9ea406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.176+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e84003520 con 0x7f9ea415ad10 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767027441 0 0) 0x7f9e84003520 con 0x7f9ea415ad10 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 msgr2=0x7f9ea40cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 0x7f9ea40cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 >> v2:172.21.15.113:3300/0 conn(0x7f9ea415b0f0 msgr2=0x7f9ea406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 --2- 172.21.15.91:0/3895727316 >> v2:172.21.15.113:3300/0 conn(0x7f9ea415b0f0 0x7f9ea406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40cc850 con 0x7f9ea415ad10 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9e94001670 con 0x7f9ea415ad10 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 >> v1:172.21.15.91:6789/0 conn(0x7f9ea415ad10 legacy=0x7f9ea406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e84003970 0x7f9e84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e840047c0 0x7f9e84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 0x7f9e840051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea406d730 con 0x7f9e84003970 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cbf50 con 0x7f9e840047c0 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8400bb10 con 0x7f9ea40cb580 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.177+0000 7f9ea9400640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e84003970 0x7f9e84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.178+0000 7f9ea9400640 1 -- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 msgr2=0x7f9e840051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.178+0000 7f9ea9400640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea40cb580 0x7f9e840051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.178+0000 7f9ea9400640 1 -- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e840047c0 msgr2=0x7f9e84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.178+0000 7f9ea9400640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e840047c0 0x7f9e84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.178+0000 7f9ea9400640 1 -- 172.21.15.91:0/3895727316 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e84003520 con 0x7f9e84003970 2024-02-02T13:31:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9ea9400640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e84003970 0x7f9e84003d90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c0e0 tx=0x7f9e840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9072133fa0d56e90 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94005ad0 con 0x7f9e84003970 2024-02-02T13:31:34.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94005c30 con 0x7f9e84003970 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9ea3e00640 1 -- 172.21.15.91:0/3895727316 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9402c7d0 con 0x7f9e84003970 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9eabe00640 1 -- 172.21.15.91:0/3895727316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e84003970 msgr2=0x7f9e84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9eabe00640 1 --2- 172.21.15.91:0/3895727316 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9e84003970 0x7f9e84003d90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c0e0 tx=0x7f9e840057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.179+0000 7f9ea9e00640 1 -- 172.21.15.91:0/3895727316 reap_dead start 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.180+0000 7f9eabe00640 1 -- 172.21.15.91:0/3895727316 shutdown_connections 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.180+0000 7f9eabe00640 1 -- 172.21.15.91:0/3895727316 >> 172.21.15.91:0/3895727316 conn(0x7f9ea40b0700 msgr2=0x7f9ea40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.180+0000 7f9eabe00640 1 -- 172.21.15.91:0/3895727316 shutdown_connections 2024-02-02T13:31:34.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.180+0000 7f9eabe00640 1 -- 172.21.15.91:0/3895727316 wait complete. 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.181+0000 7f9eabe00640 1 Processor -- start 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 -- start start 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e84003970 0x7f9ea4151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea41f2380 0x7f9ea414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40af320 con 0x7f9ea41f1f00 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40ae7a0 con 0x7f9e84003970 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.182+0000 7f9eabe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40aea80 con 0x7f9ea41f2380 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38354/0 (socket says 172.21.15.91:38354) 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 -- 172.21.15.91:0/2698399300 learned_addr learned my addr 172.21.15.91:0/2698399300 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea9e00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea41f2380 0x7f9ea414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea9400640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e84003970 0x7f9ea4151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 -- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea41f2380 msgr2=0x7f9ea414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea41f2380 0x7f9ea414e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 -- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e84003970 msgr2=0x7f9ea4151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e84003970 0x7f9ea4151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 -- 172.21.15.91:0/2698399300 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40aed60 con 0x7f9ea41f1f00 2024-02-02T13:31:34.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.183+0000 7f9ea8a00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00d800 tx=0x7f9e8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.184+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00b6c0 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.184+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c00b820 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.184+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00bbd0 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.184+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea40af040 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.184+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea40ae330 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.185+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea40ae550 con 0x7f9ea41f1f00 2024-02-02T13:31:34.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.186+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9e8c00bd30 con 0x7f9ea41f1f00 2024-02-02T13:31:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.187+0000 7f9ea1600640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 0x7f9e7c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.187+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e8c088db0 con 0x7f9ea41f1f00 2024-02-02T13:31:34.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.187+0000 7f9ea9400640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 0x7f9e7c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.188+0000 7f9ea9400640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 0x7f9e7c06c8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e94034000 tx=0x7f9e94037040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.191+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9e8c08d070 con 0x7f9ea41f1f00 2024-02-02T13:31:34.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.391+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f9ea4066cf0 con 0x7f9ea41f1f00 2024-02-02T13:31:34.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.466+0000 7f9ea1600640 1 -- 172.21.15.91:0/2698399300 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v130) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9e8c054390 con 0x7f9ea41f1f00 2024-02-02T13:31:34.468 INFO:teuthology.orchestra.run.smithi091.stderr:pauserd,pausewr is set 2024-02-02T13:31:34.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 msgr2=0x7f9e7c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 0x7f9e7c06c8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e94034000 tx=0x7f9e94037040 comp rx=0 tx=0).stop 2024-02-02T13:31:34.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 msgr2=0x7f9ea414dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00d800 tx=0x7f9e8c00dcc0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 shutdown_connections 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9e7c06a430 0x7f9e7c06c8f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ea41f2380 0x7f9ea414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ea41f1f00 0x7f9ea414dc60 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 --2- 172.21.15.91:0/2698399300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e84003970 0x7f9ea4151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.469+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 >> 172.21.15.91:0/2698399300 conn(0x7f9ea40b0700 msgr2=0x7f9ea40c1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.470+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 shutdown_connections 2024-02-02T13:31:34.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.470+0000 7f9eabe00640 1 -- 172.21.15.91:0/2698399300 wait complete. 2024-02-02T13:31:34.491 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-02-02T13:31:34.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.698+0000 7fed17200640 1 Processor -- start 2024-02-02T13:31:34.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.698+0000 7fed17200640 1 -- start start 2024-02-02T13:31:34.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed17200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fed180cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed17200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180b09a0 con 0x7fed180cc720 2024-02-02T13:31:34.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed17200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180cbf50 con 0x7fed180ccb00 2024-02-02T13:31:34.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed17200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cdc30 con 0x7fed180cd260 2024-02-02T13:31:34.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed15e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fed180cc720 0x7fed180cb840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50346/0 (socket says 172.21.15.91:50346) 2024-02-02T13:31:34.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed15e00640 1 -- 172.21.15.91:0/1193636200 learned_addr learned my addr 172.21.15.91:0/1193636200 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.700+0000 7fed16800640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fed180cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381248802 0 0) 0x7fed180b09a0 con 0x7fed180cc720 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecf8003520 con 0x7fed180cc720 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372485488 0 0) 0x7fecf8003520 con 0x7fed180cc720 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 msgr2=0x7fed180cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fed180cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 >> v1:172.21.15.113:6789/0 conn(0x7fed180ccb00 legacy=0x7fed180b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180cf3d0 con 0x7fed180cc720 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fed08001b70 con 0x7fed180cc720 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 >> v1:172.21.15.91:6789/0 conn(0x7fed180cc720 legacy=0x7fed180cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fecf8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fecf80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecf8004d00 0x7fecf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cdc30 con 0x7fecf80039a0 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf800ba60 con 0x7fecf8004d00 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf800bc20 con 0x7fed180cd260 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.701+0000 7fed15e00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fecf8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15e00640 1 -- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 msgr2=0x7fecf80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15400640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fecf80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed16800640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecf8004d00 0x7fecf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15e00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed180cd260 0x7fecf80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15e00640 1 -- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecf8004d00 msgr2=0x7fecf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15e00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecf8004d00 0x7fecf8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed15e00640 1 -- 172.21.15.91:0/1193636200 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8003520 con 0x7fecf80039a0 2024-02-02T13:31:34.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.702+0000 7fed16800640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecf8004d00 0x7fecf8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:34.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.703+0000 7fed15e00640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fecf8003dc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c4f0 tx=0x7fecf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f93fc0c5ae441c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.703+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0802ce10 con 0x7fecf80039a0 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.703+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed08005b20 con 0x7fecf80039a0 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.703+0000 7fed14a00640 1 -- 172.21.15.91:0/1193636200 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08007070 con 0x7fecf80039a0 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 -- 172.21.15.91:0/1193636200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 msgr2=0x7fecf8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 --2- 172.21.15.91:0/1193636200 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fecf8003dc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c4f0 tx=0x7fecf80057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed16800640 1 -- 172.21.15.91:0/1193636200 reap_dead start 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 -- 172.21.15.91:0/1193636200 shutdown_connections 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 -- 172.21.15.91:0/1193636200 >> 172.21.15.91:0/1193636200 conn(0x7fed180d0c30 msgr2=0x7fed180d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 -- 172.21.15.91:0/1193636200 shutdown_connections 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.704+0000 7fed17200640 1 -- 172.21.15.91:0/1193636200 wait complete. 2024-02-02T13:31:34.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.706+0000 7fed17200640 1 Processor -- start 2024-02-02T13:31:34.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.706+0000 7fed17200640 1 -- start start 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed1806e320 0x7fed180c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 0x7fed180c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ca100 con 0x7fecf80039a0 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c7aa0 con 0x7fed1806e320 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed17200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c7ea0 con 0x7fed1806e7a0 2024-02-02T13:31:34.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed15e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed15e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38376/0 (socket says 172.21.15.91:38376) 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.707+0000 7fed15e00640 1 -- 172.21.15.91:0/419656682 learned_addr learned my addr 172.21.15.91:0/419656682 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed16800640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 0x7fed180c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 -- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 msgr2=0x7fed180c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 0x7fed180c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 -- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed1806e320 msgr2=0x7fed180c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed1806e320 0x7fed180c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180c8180 con 0x7fecf80039a0 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed16800640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 0x7fed180c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:34.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed15e00640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c4f0 tx=0x7fed08001f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08007480 con 0x7fecf80039a0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed08007600 con 0x7fecf80039a0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080079b0 con 0x7fecf80039a0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.708+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180c8460 con 0x7fecf80039a0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.709+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed180c91a0 con 0x7fecf80039a0 2024-02-02T13:31:34.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.710+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1804dff0 con 0x7fecf80039a0 2024-02-02T13:31:34.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.710+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fed08032070 con 0x7fecf80039a0 2024-02-02T13:31:34.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.711+0000 7fed0e000640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 0x7fecf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:34.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.711+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed080a9ce0 con 0x7fecf80039a0 2024-02-02T13:31:34.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.711+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=131+}) v3 -- 0x7fecf0071f60 con 0x7fecf80039a0 2024-02-02T13:31:34.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.711+0000 7fed15400640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 0x7fecf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:34.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.712+0000 7fed15400640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 0x7fecf006c8f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed180ccf00 tx=0x7fed00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:34.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.716+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fed08045070 con 0x7fecf80039a0 2024-02-02T13:31:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:34.913+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fed180c7fe0 con 0x7fecf80039a0 2024-02-02T13:31:35.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.469+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(131..131 src has 1..131) v4 ==== 288+0+0 (secure 0 0 0) 0x7fed08079720 con 0x7fecf80039a0 2024-02-02T13:31:35.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.469+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=132}) v3 -- 0x7fecf00726a0 con 0x7fecf80039a0 2024-02-02T13:31:35.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.475+0000 7fed0e000640 1 -- 172.21.15.91:0/419656682 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7fed180c7fe0 con 0x7fecf80039a0 2024-02-02T13:31:35.477 INFO:teuthology.orchestra.run.smithi091.stderr:pauserd,pausewr is unset 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 msgr2=0x7fecf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 0x7fecf006c8f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed180ccf00 tx=0x7fed00009040 comp rx=0 tx=0).stop 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 msgr2=0x7fed180c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c4f0 tx=0x7fed08001f00 comp rx=0 tx=0).stop 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 shutdown_connections 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fecf006a430 0x7fecf006c8f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed1806e7a0 0x7fed180c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed1806e320 0x7fed180c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 --2- 172.21.15.91:0/419656682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fecf80039a0 0x7fed180c8c60 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 >> 172.21.15.91:0/419656682 conn(0x7fed180d0c30 msgr2=0x7fed1806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.479+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 shutdown_connections 2024-02-02T13:31:35.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.480+0000 7fed17200640 1 -- 172.21.15.91:0/419656682 wait complete. 2024-02-02T13:31:35.502 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.703+0000 7fd5ac200640 1 Processor -- start 2024-02-02T13:31:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.703+0000 7fd5ac200640 1 -- start start 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.704+0000 7fd5ac200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a415f5a0 0x7fd5a40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.704+0000 7fd5ac200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5a40b0c40 0x7fd5a40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.704+0000 7fd5ac200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a415fa00 con 0x7fd5a4151f90 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.704+0000 7fd5ac200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a40b10d0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.704+0000 7fd5ac200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4070f80 con 0x7fd5a415f5a0 2024-02-02T13:31:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5a40b0c40 0x7fd5a40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5a40b0c40 0x7fd5a40b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38380/0 (socket says 172.21.15.91:38380) 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 -- 172.21.15.91:0/2789032457 learned_addr learned my addr 172.21.15.91:0/2789032457 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5a8e00640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a415f5a0 0x7fd5a40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 -- 172.21.15.91:0/2789032457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a415f5a0 msgr2=0x7fd5a40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a415f5a0 0x7fd5a40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 -- 172.21.15.91:0/2789032457 >> v1:172.21.15.113:6789/0 conn(0x7fd5a4151f90 legacy=0x7fd5a406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.705+0000 7fd5aa200640 1 -- 172.21.15.91:0/2789032457 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4152370 con 0x7fd5a40b0c40 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.706+0000 7fd5aa200640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a40b3af0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009e50 tx=0x7fd59c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875759c0d22fc983 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5a3e00640 1 -- 172.21.15.91:0/2789032457 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c02cb00 con 0x7fd5a40b0c40 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5a3e00640 1 -- 172.21.15.91:0/2789032457 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c02cc60 con 0x7fd5a40b0c40 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5a3e00640 1 -- 172.21.15.91:0/2789032457 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0343f0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 -- 172.21.15.91:0/2789032457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 msgr2=0x7fd5a40b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a40b3af0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009e50 tx=0x7fd59c02bf60 comp rx=0 tx=0).stop 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 -- 172.21.15.91:0/2789032457 shutdown_connections 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a40b3af0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 --2- 172.21.15.91:0/2789032457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a415f5a0 0x7fd5a40b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.707+0000 7fd5ac200640 1 -- 172.21.15.91:0/2789032457 >> 172.21.15.91:0/2789032457 conn(0x7fd5a40cec80 msgr2=0x7fd5a40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:35.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.708+0000 7fd5ac200640 1 -- 172.21.15.91:0/2789032457 shutdown_connections 2024-02-02T13:31:35.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.708+0000 7fd5ac200640 1 -- 172.21.15.91:0/2789032457 wait complete. 2024-02-02T13:31:35.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.709+0000 7fd5ac200640 1 Processor -- start 2024-02-02T13:31:35.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.709+0000 7fd5ac200640 1 -- start start 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a4151f90 0x7fd5a40ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a415f5a0 0x7fd5a40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4148910 con 0x7fd5a40b0c40 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40ad930 con 0x7fd5a415f5a0 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.710+0000 7fd5ac200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40adc10 con 0x7fd5a4151f90 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38394/0 (socket says 172.21.15.91:38394) 2024-02-02T13:31:35.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 -- 172.21.15.91:0/1514868573 learned_addr learned my addr 172.21.15.91:0/1514868573 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:35.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 -- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a4151f90 msgr2=0x7fd5a40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a4151f90 0x7fd5a40ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 -- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a415f5a0 msgr2=0x7fd5a40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a415f5a0 0x7fd5a40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 -- 172.21.15.91:0/1514868573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c009b30 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a9800640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c400 tx=0x7fd59400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.711+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940052d0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.712+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594005430 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.712+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594003830 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.712+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a40adef0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.712+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a40c27c0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.713+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a4066cf0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.714+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd594003990 con 0x7fd5a40b0c40 2024-02-02T13:31:35.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.715+0000 7fd5a1600640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 0x7fd57c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:35.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.715+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd594087dd0 con 0x7fd5a40b0c40 2024-02-02T13:31:35.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.715+0000 7fd5a8e00640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 0x7fd57c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:35.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.716+0000 7fd5a8e00640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 0x7fd57c06c920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4151350 tx=0x7fd58c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:35.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.719+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd594053320 con 0x7fd5a40b0c40 2024-02-02T13:31:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.912+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5a415fa00 con 0x7fd5a40b0c40 2024-02-02T13:31:35.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.914+0000 7fd5a1600640 1 -- 172.21.15.91:0/1514868573 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7fd5a415fa00 con 0x7fd5a40b0c40 2024-02-02T13:31:35.916 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:35.916 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 msgr2=0x7fd57c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 0x7fd57c06c920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4151350 tx=0x7fd58c008040 comp rx=0 tx=0).stop 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 msgr2=0x7fd5a41483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c400 tx=0x7fd59400c8c0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 shutdown_connections 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57c06a460 0x7fd57c06c920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5a4151f90 0x7fd5a40ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a415f5a0 0x7fd5a40cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 --2- 172.21.15.91:0/1514868573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5a40b0c40 0x7fd5a41483d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.917+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 >> 172.21.15.91:0/1514868573 conn(0x7fd5a40cec80 msgr2=0x7fd5a40c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.918+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 shutdown_connections 2024-02-02T13:31:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:35.918+0000 7fd5ac200640 1 -- 172.21.15.91:0/1514868573 wait complete. 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:31:35.938 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:31:36.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:36.252 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:36.354 INFO:teuthology.orchestra.run.smithi091.stdout:client5252 2024-02-02T13:31:36.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:36.355 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5252/osdmap') 2024-02-02T13:31:36.445 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 128 barrier 128 flags 0x588000 2024-02-02T13:31:36.445 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:31:36.446 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:31:36.447 INFO:teuthology.orchestra.run.smithi091.stderr:799 bytes copied, 8.4938e-05 s, 9.4 MB/s 2024-02-02T13:31:36.450 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:36.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.658+0000 7f1ef3600640 1 Processor -- start 2024-02-02T13:31:36.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.659+0000 7f1ef3600640 1 -- start start 2024-02-02T13:31:36.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 0x7f1eec15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1eec06f4f0 0x7f1eec0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec0b10b0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec070f80 con 0x7f1eec151f90 2024-02-02T13:31:36.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1ef3600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec152370 con 0x7f1eec0b0c40 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1eebe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1eec06f4f0 0x7f1eec0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1eebe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1eec06f4f0 0x7f1eec0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38402/0 (socket says 172.21.15.91:38402) 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.660+0000 7f1eebe00640 1 -- 172.21.15.91:0/2905403999 learned_addr learned my addr 172.21.15.91:0/2905403999 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1ef0c00640 1 --2- 172.21.15.91:0/2905403999 >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 0x7f1eec15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1eebe00640 1 -- 172.21.15.91:0/2905403999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 msgr2=0x7f1eec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1ef1600640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1eebe00640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1eebe00640 1 -- 172.21.15.91:0/2905403999 >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 msgr2=0x7f1eec15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1eebe00640 1 --2- 172.21.15.91:0/2905403999 >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 0x7f1eec15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1eebe00640 1 -- 172.21.15.91:0/2905403999 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0cb010 con 0x7f1eec06f4f0 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1ef0c00640 1 --2- 172.21.15.91:0/2905403999 >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 0x7f1eec15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:36.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.661+0000 7f1ef1600640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:36.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.662+0000 7f1eebe00640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec0b0700 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cc770 tx=0x7f1ed402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ebed3e849be688 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:36.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.662+0000 7f1eeb400640 1 -- 172.21.15.91:0/2905403999 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4031880 con 0x7f1eec06f4f0 2024-02-02T13:31:36.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1eeb400640 1 -- 172.21.15.91:0/2905403999 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed40319e0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1eeb400640 1 -- 172.21.15.91:0/2905403999 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4031d70 con 0x7f1eec06f4f0 2024-02-02T13:31:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 -- 172.21.15.91:0/2905403999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 msgr2=0x7f1eec0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec0b0700 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cc770 tx=0x7f1ed402b920 comp rx=0 tx=0).stop 2024-02-02T13:31:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 -- 172.21.15.91:0/2905403999 shutdown_connections 2024-02-02T13:31:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2905403999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec0b0700 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2905403999 >> v2:172.21.15.113:3300/0 conn(0x7f1eec151f90 0x7f1eec15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.663+0000 7f1ef3600640 1 -- 172.21.15.91:0/2905403999 >> 172.21.15.91:0/2905403999 conn(0x7f1eec0cec80 msgr2=0x7f1eec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:36.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.664+0000 7f1ef3600640 1 -- 172.21.15.91:0/2905403999 shutdown_connections 2024-02-02T13:31:36.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.664+0000 7f1ef3600640 1 -- 172.21.15.91:0/2905403999 wait complete. 2024-02-02T13:31:36.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.665+0000 7f1ef3600640 1 Processor -- start 2024-02-02T13:31:36.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.665+0000 7f1ef3600640 1 -- start start 2024-02-02T13:31:36.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 0x7f1eec0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c6960 con 0x7f1eec06f4f0 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c75d0 con 0x7f1eec151f90 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.666+0000 7f1ef3600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c78b0 con 0x7f1eec0b0c40 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38418/0 (socket says 172.21.15.91:38418) 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 -- 172.21.15.91:0/2657513311 learned_addr learned my addr 172.21.15.91:0/2657513311 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef1600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 0x7f1eec0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 -- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 msgr2=0x7f1eec0c27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 -- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 msgr2=0x7f1eec0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 0x7f1eec0afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef0c00640 1 -- 172.21.15.91:0/2657513311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40094d0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.667+0000 7f1ef1600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 0x7f1eec0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:36.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ef0c00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c7f0 tx=0x7f1edc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0079d0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc007b30 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00ce40 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec0c7b90 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.668+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec148840 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.669+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec066cf0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.670+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1edc0113f0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.671+0000 7f1ee8c00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 0x7f1ec406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:36.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.671+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1edc088ac0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.671+0000 7f1eebe00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 0x7f1ec406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:36.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.672+0000 7f1eebe00640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 0x7f1ec406c8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4036000 tx=0x7f1ed4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:36.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.675+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1edc054010 con 0x7f1eec06f4f0 2024-02-02T13:31:36.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.904+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eec06c7b0 con 0x7f1eec06f4f0 2024-02-02T13:31:36.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.905+0000 7f1ee8c00640 1 -- 172.21.15.91:0/2657513311 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f1edc01f090 con 0x7f1eec06f4f0 2024-02-02T13:31:36.907 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:36.907 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:36.907 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:36.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.907+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 msgr2=0x7f1ec406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:36.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.907+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 0x7f1ec406c8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4036000 tx=0x7f1ed4030000 comp rx=0 tx=0).stop 2024-02-02T13:31:36.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.907+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 msgr2=0x7f1eec148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:36.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.907+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c7f0 tx=0x7f1edc00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 shutdown_connections 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec406a430 0x7f1ec406c8f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eec0b0c40 0x7f1eec0c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eec151f90 0x7f1eec0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 --2- 172.21.15.91:0/2657513311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec06f4f0 0x7f1eec148120 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 >> 172.21.15.91:0/2657513311 conn(0x7f1eec0cec80 msgr2=0x7f1eec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 shutdown_connections 2024-02-02T13:31:36.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:36.908+0000 7f1ef3600640 1 -- 172.21.15.91:0/2657513311 wait complete. 2024-02-02T13:31:36.927 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 131 2024-02-02T13:31:37.150 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> os.fchmod(fd, 0O666) 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(fd) 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> os.close(fd) 2024-02-02T13:31:37.151 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:31:37.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:37.472 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:37.571 INFO:teuthology.orchestra.run.smithi091.stdout:client5252 2024-02-02T13:31:37.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:37.572 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5252/osdmap') 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 131 barrier 131 flags 0x588000 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:31:37.667 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:31:37.668 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:31:37.668 INFO:teuthology.orchestra.run.smithi091.stderr:799 bytes copied, 9.7269e-05 s, 8.2 MB/s 2024-02-02T13:31:37.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:37.671 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:31:37.725 INFO:teuthology.orchestra.run.smithi091.stdout:client5252 2024-02-02T13:31:37.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:37.725 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cc006970-f900-4781-8947-81ed22cd959b.client5252/osdmap') 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 131 barrier 131 flags 0x588000 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd0 (2)172.21.15.91:6820 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd1 (2)172.21.15.91:6805 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd2 (2)172.21.15.91:6816 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd3 (2)172.21.15.91:6800 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd4 (2)172.21.15.113:6823 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd5 (2)172.21.15.113:6800 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd6 (2)172.21.15.113:6816 100% (exists, up) 100% -1 2024-02-02T13:31:37.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd7 (2)172.21.15.113:6806 100% (exists, up) 100% -1 2024-02-02T13:31:37.813 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records in 2024-02-02T13:31:37.813 INFO:teuthology.orchestra.run.smithi091.stderr:1+1 records out 2024-02-02T13:31:37.813 INFO:teuthology.orchestra.run.smithi091.stderr:799 bytes copied, 8.2507e-05 s, 9.7 MB/s 2024-02-02T13:31:37.816 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:31:37.816 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:31:37.816 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:37.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:38.042 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:31:38.042 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:38.042 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:38.099 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:38.099 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:38.152 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:38.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.356+0000 7fc91da00640 1 Processor -- start 2024-02-02T13:31:38.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.356+0000 7fc91da00640 1 -- start start 2024-02-02T13:31:38.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc91da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 0x7fc91806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc91da00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc9180b0700 0x7fc9180b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc91da00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91815d870 con 0x7fc91815d490 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc91da00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91806bce0 con 0x7fc9180b0700 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc91da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc918161220 con 0x7fc9180acfe0 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc917400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc9180b0700 0x7fc9180b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc917400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc9180b0700 0x7fc9180b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38434/0 (socket says 172.21.15.91:38434) 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc917400640 1 -- 172.21.15.91:0/92218738 learned_addr learned my addr 172.21.15.91:0/92218738 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:38.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc916a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc91815d490 0x7fc9180ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50348/0 (socket says 172.21.15.91:50348) 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.358+0000 7fc90de00640 1 --2- 172.21.15.91:0/92218738 >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 0x7fc91806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.359+0000 7fc917400640 1 -- 172.21.15.91:0/92218738 >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 msgr2=0x7fc91806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.359+0000 7fc917400640 1 --2- 172.21.15.91:0/92218738 >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 0x7fc91806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.359+0000 7fc917400640 1 -- 172.21.15.91:0/92218738 >> v1:172.21.15.91:6789/0 conn(0x7fc91815d490 legacy=0x7fc9180ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.359+0000 7fc917400640 1 -- 172.21.15.91:0/92218738 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180b1050 con 0x7fc9180b0700 2024-02-02T13:31:38.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.359+0000 7fc90de00640 1 --2- 172.21.15.91:0/92218738 >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 0x7fc91806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:38.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.360+0000 7fc917400640 1 --2- 172.21.15.91:0/92218738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180b0700 0x7fc9180b0b10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc9180b1870 tx=0x7fc91002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73c9b5a5b06aebf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.360+0000 7fc916000640 1 -- 172.21.15.91:0/92218738 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91002cb10 con 0x7fc9180b0700 2024-02-02T13:31:38.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.360+0000 7fc916000640 1 -- 172.21.15.91:0/92218738 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91002cc70 con 0x7fc9180b0700 2024-02-02T13:31:38.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.360+0000 7fc916000640 1 -- 172.21.15.91:0/92218738 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100343f0 con 0x7fc9180b0700 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 -- 172.21.15.91:0/92218738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180b0700 msgr2=0x7fc9180b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 --2- 172.21.15.91:0/92218738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180b0700 0x7fc9180b0b10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc9180b1870 tx=0x7fc91002bf70 comp rx=0 tx=0).stop 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 -- 172.21.15.91:0/92218738 shutdown_connections 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 --2- 172.21.15.91:0/92218738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180b0700 0x7fc9180b0b10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 --2- 172.21.15.91:0/92218738 >> v2:172.21.15.113:3300/0 conn(0x7fc9180acfe0 0x7fc91806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 -- 172.21.15.91:0/92218738 >> 172.21.15.91:0/92218738 conn(0x7fc9180cec80 msgr2=0x7fc9180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 -- 172.21.15.91:0/92218738 shutdown_connections 2024-02-02T13:31:38.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.361+0000 7fc91da00640 1 -- 172.21.15.91:0/92218738 wait complete. 2024-02-02T13:31:38.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.363+0000 7fc91da00640 1 Processor -- start 2024-02-02T13:31:38.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.363+0000 7fc91da00640 1 -- start start 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 0x7fc9180c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc91815d490 0x7fc9180c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181f6bd0 con 0x7fc9180acfe0 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180c2720 con 0x7fc9180b0700 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc91da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181f5fc0 con 0x7fc91815d490 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc916a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc916a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38436/0 (socket says 172.21.15.91:38436) 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc916a00640 1 -- 172.21.15.91:0/1514571737 learned_addr learned my addr 172.21.15.91:0/1514571737 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.364+0000 7fc90de00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 0x7fc9180c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 -- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc91815d490 msgr2=0x7fc9180c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc917400640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc91815d490 0x7fc9180c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc91815d490 0x7fc9180c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 -- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 msgr2=0x7fc9180c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 0x7fc9180c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 -- 172.21.15.91:0/1514571737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009b30 con 0x7fc9180acfe0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc90de00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 0x7fc9180c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc916a00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c450 tx=0x7fc90800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908007480 con 0x7fc9180acfe0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908007600 con 0x7fc9180acfe0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9181f62a0 con 0x7fc9180acfe0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908004a20 con 0x7fc9180acfe0 2024-02-02T13:31:38.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.365+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9181f6860 con 0x7fc9180acfe0 2024-02-02T13:31:38.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.367+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc918066cf0 con 0x7fc9180acfe0 2024-02-02T13:31:38.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.367+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc908004b80 con 0x7fc9180acfe0 2024-02-02T13:31:38.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.368+0000 7fc90f400640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 0x7fc8f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.368+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9080879f0 con 0x7fc9180acfe0 2024-02-02T13:31:38.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.369+0000 7fc90de00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 0x7fc8f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.369+0000 7fc90de00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 0x7fc8f006c8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc900006fd0 tx=0x7fc900008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.372+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc908052fd0 con 0x7fc9180acfe0 2024-02-02T13:31:38.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.564+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc91806b940 con 0x7fc9180acfe0 2024-02-02T13:31:38.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.567+0000 7fc90f400640 1 -- 172.21.15.91:0/1514571737 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7fc908018090 con 0x7fc9180acfe0 2024-02-02T13:31:38.569 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:38.569 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:38.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 msgr2=0x7fc8f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 0x7fc8f006c8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc900006fd0 tx=0x7fc900008040 comp rx=0 tx=0).stop 2024-02-02T13:31:38.571 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 msgr2=0x7fc918070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c450 tx=0x7fc90800c910 comp rx=0 tx=0).stop 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 shutdown_connections 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8f006a430 0x7fc8f006c8f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc91815d490 0x7fc9180c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9180b0700 0x7fc9180c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 --2- 172.21.15.91:0/1514571737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9180acfe0 0x7fc918070f50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.569+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 >> 172.21.15.91:0/1514571737 conn(0x7fc9180cec80 msgr2=0x7fc9180cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.570+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 shutdown_connections 2024-02-02T13:31:38.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.570+0000 7fc91da00640 1 -- 172.21.15.91:0/1514571737 wait complete. 2024-02-02T13:31:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:14.703691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:15.895506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:38.590 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.796+0000 7f552e800640 1 Processor -- start 2024-02-02T13:31:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.796+0000 7f552e800640 1 -- start start 2024-02-02T13:31:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.797+0000 7f552e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5528151f90 0x7f552815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.797+0000 7f552e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.797+0000 7f552e800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5528070f80 con 0x7f55280b0c40 2024-02-02T13:31:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.797+0000 7f552e800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55280b10d0 con 0x7f5528151f90 2024-02-02T13:31:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.797+0000 7f552e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5528152370 con 0x7f552806f4f0 2024-02-02T13:31:38.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.798+0000 7f5527400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5528151f90 0x7f552815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.798+0000 7f5527400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5528151f90 0x7f552815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38452/0 (socket says 172.21.15.91:38452) 2024-02-02T13:31:38.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.798+0000 7f5527400640 1 -- 172.21.15.91:0/2212606092 learned_addr learned my addr 172.21.15.91:0/2212606092 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:38.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.798+0000 7f5526a00640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.799+0000 7f5527400640 1 -- 172.21.15.91:0/2212606092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 msgr2=0x7f55280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:38.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.799+0000 7f5527400640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.799+0000 7f5527400640 1 -- 172.21.15.91:0/2212606092 >> v1:172.21.15.113:6789/0 conn(0x7f55280b0c40 legacy=0x7f55280b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:38.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.799+0000 7f5527400640 1 -- 172.21.15.91:0/2212606092 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280cb010 con 0x7f5528151f90 2024-02-02T13:31:38.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.800+0000 7f5527400640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f552815f5a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fadb9e1ce5725f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.800+0000 7f5526000640 1 -- 172.21.15.91:0/2212606092 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551802ca70 con 0x7f5528151f90 2024-02-02T13:31:38.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f5526000640 1 -- 172.21.15.91:0/2212606092 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551802cbd0 con 0x7f5528151f90 2024-02-02T13:31:38.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f5526000640 1 -- 172.21.15.91:0/2212606092 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180343f0 con 0x7f5528151f90 2024-02-02T13:31:38.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 -- 172.21.15.91:0/2212606092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 msgr2=0x7f552815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f552815f5a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e50 tx=0x7f551802bed0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 -- 172.21.15.91:0/2212606092 shutdown_connections 2024-02-02T13:31:38.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 --2- 172.21.15.91:0/2212606092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f552815f5a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 -- 172.21.15.91:0/2212606092 >> 172.21.15.91:0/2212606092 conn(0x7f55280cec80 msgr2=0x7f55280cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 -- 172.21.15.91:0/2212606092 shutdown_connections 2024-02-02T13:31:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.801+0000 7f552e800640 1 -- 172.21.15.91:0/2212606092 wait complete. 2024-02-02T13:31:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.803+0000 7f552e800640 1 Processor -- start 2024-02-02T13:31:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.803+0000 7f552e800640 1 -- start start 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55280b0c40 0x7f55280c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280c2a70 con 0x7f5528151f90 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280c6200 con 0x7f55280b0c40 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f552e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280c2720 con 0x7f552806f4f0 2024-02-02T13:31:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f5527e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f5527e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38456/0 (socket says 172.21.15.91:38456) 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.804+0000 7f5527e00640 1 -- 172.21.15.91:0/2256235356 learned_addr learned my addr 172.21.15.91:0/2256235356 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527400640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5526a00640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55280b0c40 0x7f55280c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 -- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 msgr2=0x7f55280c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 -- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55280b0c40 msgr2=0x7f55280c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55280b0c40 0x7f55280c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 -- 172.21.15.91:0/2256235356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009b30 con 0x7f5528151f90 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f5527e00640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f552000c440 tx=0x7f552000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520011790 con 0x7f5528151f90 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.805+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55200118f0 con 0x7f5528151f90 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.806+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520011ca0 con 0x7f5528151f90 2024-02-02T13:31:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.806+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55280c6430 con 0x7f5528151f90 2024-02-02T13:31:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.806+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55280c69f0 con 0x7f5528151f90 2024-02-02T13:31:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.807+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55280c2bb0 con 0x7f5528151f90 2024-02-02T13:31:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.808+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f552001c460 con 0x7f5528151f90 2024-02-02T13:31:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.809+0000 7f551f400640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 0x7f550006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:38.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.809+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5520088660 con 0x7f5528151f90 2024-02-02T13:31:38.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.809+0000 7f5527400640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 0x7f550006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:38.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.810+0000 7f5527400640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 0x7f550006c8f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5518035000 tx=0x7f5518030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:38.814+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f55280c2bb0 con 0x7f5528151f90 2024-02-02T13:31:39.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.005+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55280c2bb0 con 0x7f5528151f90 2024-02-02T13:31:39.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.006+0000 7f551f400640 1 -- 172.21.15.91:0/2256235356 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55280c2bb0 con 0x7f5528151f90 2024-02-02T13:31:39.007 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 msgr2=0x7f550006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 0x7f550006c8f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5518035000 tx=0x7f5518030000 comp rx=0 tx=0).stop 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 msgr2=0x7f55280c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f552000c440 tx=0x7f552000c900 comp rx=0 tx=0).stop 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 shutdown_connections 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f550006a430 0x7f550006c8f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5528151f90 0x7f55280c2e90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55280b0c40 0x7f55280c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 --2- 172.21.15.91:0/2256235356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f552806f4f0 0x7f55280c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 >> 172.21.15.91:0/2256235356 conn(0x7f55280cec80 msgr2=0x7f55280ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:39.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 shutdown_connections 2024-02-02T13:31:39.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.008+0000 7f552e800640 1 -- 172.21.15.91:0/2256235356 wait complete. 2024-02-02T13:31:39.029 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:31:39.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.237+0000 7f5655000640 1 Processor -- start 2024-02-02T13:31:39.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.237+0000 7f5655000640 1 -- start start 2024-02-02T13:31:39.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.238+0000 7f5655000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f56500c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.238+0000 7f5655000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500ce3b0 con 0x7f565015d3a0 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.238+0000 7f5655000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500ce8e0 con 0x7f56500add80 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.238+0000 7f5655000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500adbd0 con 0x7f56500c9450 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f565015d3a0 0x7f565006b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50358/0 (socket says 172.21.15.91:50358) 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564f400640 1 -- 172.21.15.91:0/2067218785 learned_addr learned my addr 172.21.15.91:0/2067218785 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154783479 0 0) 0x7f56500ce3b0 con 0x7f565015d3a0 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5630005730 con 0x7f565015d3a0 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564fe00640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f56500c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 418419748 0 0) 0x7f5630005730 con 0x7f565015d3a0 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 msgr2=0x7f56500c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f56500c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 >> v1:172.21.15.113:6789/0 conn(0x7f56500add80 legacy=0x7f56500706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650070db0 con 0x7f565015d3a0 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.239+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f56400019f0 con 0x7f565015d3a0 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 >> v1:172.21.15.91:6789/0 conn(0x7f565015d3a0 legacy=0x7f565006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f5630005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56300061e0 0x7f56300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5630006f60 0x7f56300073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500adbd0 con 0x7f56300061e0 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563000de70 con 0x7f5630006f60 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563000e030 con 0x7f56500c9450 2024-02-02T13:31:39.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56300061e0 0x7f56300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564f400640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f5630005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 -- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 msgr2=0x7f5630005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f5630005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 -- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5630006f60 msgr2=0x7f56300073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5630006f60 0x7f56300073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f5646800640 1 -- 172.21.15.91:0/2067218785 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5630005730 con 0x7f56300061e0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.240+0000 7f564f400640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56500c9450 0x7f5630005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.241+0000 7f5646800640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56300061e0 0x7f56300069d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5650071cb0 tx=0x7f5630007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2896cbe8e76019 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56380052d0 con 0x7f56300061e0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563802e050 con 0x7f56300061e0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f564ea00640 1 -- 172.21.15.91:0/2067218785 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638003830 con 0x7f56300061e0 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 -- 172.21.15.91:0/2067218785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56300061e0 msgr2=0x7f56300069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 --2- 172.21.15.91:0/2067218785 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56300061e0 0x7f56300069d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5650071cb0 tx=0x7f5630007a00 comp rx=0 tx=0).stop 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f564fe00640 1 -- 172.21.15.91:0/2067218785 reap_dead start 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 -- 172.21.15.91:0/2067218785 shutdown_connections 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 -- 172.21.15.91:0/2067218785 >> 172.21.15.91:0/2067218785 conn(0x7f56500cea90 msgr2=0x7f56500ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 -- 172.21.15.91:0/2067218785 shutdown_connections 2024-02-02T13:31:39.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.242+0000 7f5655000640 1 -- 172.21.15.91:0/2067218785 wait complete. 2024-02-02T13:31:39.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.244+0000 7f5655000640 1 Processor -- start 2024-02-02T13:31:39.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.244+0000 7f5655000640 1 -- start start 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56300061e0 0x7f56500c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 0x7f565015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c4370 con 0x7f56500b0f50 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015c340 con 0x7f56300061e0 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5655000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b13d0 con 0x7f565015b980 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5646800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5646800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38470/0 (socket says 172.21.15.91:38470) 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.245+0000 7f5646800640 1 -- 172.21.15.91:0/3765548776 learned_addr learned my addr 172.21.15.91:0/3765548776 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f564f400640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56300061e0 0x7f56500c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f564fe00640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 0x7f565015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 -- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 msgr2=0x7f565015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 0x7f565015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 -- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56300061e0 msgr2=0x7f56500c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56300061e0 0x7f56500c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 -- 172.21.15.91:0/3765548776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500c8920 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f564fe00640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 0x7f565015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5646800640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f563802bba0 tx=0x7f5638001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56380076c0 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638007a30 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638006c20 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.246+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501ee840 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.247+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501eee00 con 0x7f56500b0f50 2024-02-02T13:31:39.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.248+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565004dff0 con 0x7f56500b0f50 2024-02-02T13:31:39.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.248+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f56380035f0 con 0x7f56500b0f50 2024-02-02T13:31:39.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.249+0000 7f5647e00640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 0x7f562806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.249+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56380aa5d0 con 0x7f56500b0f50 2024-02-02T13:31:39.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.249+0000 7f564f400640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 0x7f562806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.250+0000 7f564f400640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 0x7f562806c8f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5640001370 tx=0x7f5640009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.254+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5638075b20 con 0x7f56500b0f50 2024-02-02T13:31:39.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.444+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f565015cb50 con 0x7f56500b0f50 2024-02-02T13:31:39.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.445+0000 7f5647e00640 1 -- 172.21.15.91:0/3765548776 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5638079140 con 0x7f56500b0f50 2024-02-02T13:31:39.447 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 msgr2=0x7f562806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 0x7f562806c8f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5640001370 tx=0x7f5640009040 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 msgr2=0x7f56501edfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f563802bba0 tx=0x7f5638001120 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 shutdown_connections 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f562806a430 0x7f562806c8f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f565015b980 0x7f565015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56500b0f50 0x7f56501edfb0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 --2- 172.21.15.91:0/3765548776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56300061e0 0x7f56500c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 >> 172.21.15.91:0/3765548776 conn(0x7f56500cea90 msgr2=0x7f565006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:39.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 shutdown_connections 2024-02-02T13:31:39.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.447+0000 7f5655000640 1 -- 172.21.15.91:0/3765548776 wait complete. 2024-02-02T13:31:39.467 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:31:39.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.674+0000 7fdc53c00640 1 Processor -- start 2024-02-02T13:31:39.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.675+0000 7fdc53c00640 1 -- start start 2024-02-02T13:31:39.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc53c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 0x7fdc4c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc53c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc4c06f4f0 0x7fdc4c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc53c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4c070f80 con 0x7fdc4c0b0c40 2024-02-02T13:31:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc53c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4c0b10d0 con 0x7fdc4c06f4f0 2024-02-02T13:31:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc53c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c152370 con 0x7fdc4c151f90 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc51c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdc4c0b0c40 0x7fdc4c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50360/0 (socket says 172.21.15.91:50360) 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.676+0000 7fdc51c00640 1 -- 172.21.15.91:0/2003460490 learned_addr learned my addr 172.21.15.91:0/2003460490 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc51200640 1 --2- 172.21.15.91:0/2003460490 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 0x7fdc4c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc4be00640 1 --2- 172.21.15.91:0/2003460490 >> v2:172.21.15.113:3300/0 conn(0x7fdc4c06f4f0 0x7fdc4c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc4b400640 1 -- 172.21.15.91:0/2003460490 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158490861 0 0) 0x7fdc4c070f80 con 0x7fdc4c0b0c40 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc4b400640 1 -- 172.21.15.91:0/2003460490 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2c003520 con 0x7fdc4c0b0c40 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc51200640 1 -- 172.21.15.91:0/2003460490 >> v2:172.21.15.113:3300/0 conn(0x7fdc4c06f4f0 msgr2=0x7fdc4c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc51200640 1 --2- 172.21.15.91:0/2003460490 >> v2:172.21.15.113:3300/0 conn(0x7fdc4c06f4f0 0x7fdc4c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc51200640 1 -- 172.21.15.91:0/2003460490 >> v1:172.21.15.91:6789/0 conn(0x7fdc4c0b0c40 legacy=0x7fdc4c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.677+0000 7fdc51200640 1 -- 172.21.15.91:0/2003460490 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c0cb010 con 0x7fdc4c151f90 2024-02-02T13:31:39.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.679+0000 7fdc51200640 1 --2- 172.21.15.91:0/2003460490 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 0x7fdc4c15f5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0097a0 tx=0x7fdc3c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d846ccd4bf4ba2e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.679+0000 7fdc4b400640 1 -- 172.21.15.91:0/2003460490 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c031830 con 0x7fdc4c151f90 2024-02-02T13:31:39.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.679+0000 7fdc4b400640 1 -- 172.21.15.91:0/2003460490 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c031990 con 0x7fdc4c151f90 2024-02-02T13:31:39.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.679+0000 7fdc4b400640 1 -- 172.21.15.91:0/2003460490 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c031d20 con 0x7fdc4c151f90 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 -- 172.21.15.91:0/2003460490 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 msgr2=0x7fdc4c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 --2- 172.21.15.91:0/2003460490 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 0x7fdc4c15f5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0097a0 tx=0x7fdc3c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 -- 172.21.15.91:0/2003460490 shutdown_connections 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 --2- 172.21.15.91:0/2003460490 >> v2:172.21.15.113:3300/0 conn(0x7fdc4c06f4f0 0x7fdc4c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 --2- 172.21.15.91:0/2003460490 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c151f90 0x7fdc4c15f5a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 -- 172.21.15.91:0/2003460490 >> 172.21.15.91:0/2003460490 conn(0x7fdc4c0cec80 msgr2=0x7fdc4c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 -- 172.21.15.91:0/2003460490 shutdown_connections 2024-02-02T13:31:39.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.680+0000 7fdc53c00640 1 -- 172.21.15.91:0/2003460490 wait complete. 2024-02-02T13:31:39.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.682+0000 7fdc53c00640 1 Processor -- start 2024-02-02T13:31:39.683 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.682+0000 7fdc53c00640 1 -- start start 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c06f4f0 0x7fdc4c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc4c0b0c40 0x7fdc4c15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c0c28c0 con 0x7fdc4c151f90 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c0c6f40 con 0x7fdc4c0b0c40 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc53c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c0c78a0 con 0x7fdc4c06f4f0 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc51c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc51c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:38476/0 (socket says 172.21.15.91:38476) 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc51c00640 1 -- 172.21.15.91:0/1588731181 learned_addr learned my addr 172.21.15.91:0/1588731181 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc4be00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc4c0b0c40 0x7fdc4c15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.683+0000 7fdc51200640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c06f4f0 0x7fdc4c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 -- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c06f4f0 msgr2=0x7fdc4c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c06f4f0 0x7fdc4c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 -- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc4c0b0c40 msgr2=0x7fdc4c15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc4c0b0c40 0x7fdc4c15a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 -- 172.21.15.91:0/1588731181 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009400 con 0x7fdc4c151f90 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc51c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c9f0 tx=0x7fdc4400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400ab70 con 0x7fdc4c151f90 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4400acd0 con 0x7fdc4c151f90 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c0c6980 con 0x7fdc4c151f90 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44012600 con 0x7fdc4c151f90 2024-02-02T13:31:39.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.684+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c0c7b40 con 0x7fdc4c151f90 2024-02-02T13:31:39.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.686+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c066cf0 con 0x7fdc4c151f90 2024-02-02T13:31:39.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.686+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc44007070 con 0x7fdc4c151f90 2024-02-02T13:31:39.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.687+0000 7fdc48c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 0x7fdc2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:39.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.687+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc44088760 con 0x7fdc4c151f90 2024-02-02T13:31:39.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.687+0000 7fdc51200640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 0x7fdc2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:39.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.688+0000 7fdc51200640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 0x7fdc2406c8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c036000 tx=0x7fdc3c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:39.693 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.692+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc44053d70 con 0x7fdc4c151f90 2024-02-02T13:31:39.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.883+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc4c15ef70 con 0x7fdc4c151f90 2024-02-02T13:31:39.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.884+0000 7fdc48c00640 1 -- 172.21.15.91:0/1588731181 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc44057390 con 0x7fdc4c151f90 2024-02-02T13:31:39.885 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 msgr2=0x7fdc2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 0x7fdc2406c8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c036000 tx=0x7fdc3c03c040 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 msgr2=0x7fdc4c14cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c9f0 tx=0x7fdc4400ceb0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 shutdown_connections 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdc2406a430 0x7fdc2406c8f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc4c151f90 0x7fdc4c14cd40 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc4c0b0c40 0x7fdc4c15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 --2- 172.21.15.91:0/1588731181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc4c06f4f0 0x7fdc4c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:39.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 >> 172.21.15.91:0/1588731181 conn(0x7fdc4c0cec80 msgr2=0x7fdc4c0c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:39.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 shutdown_connections 2024-02-02T13:31:39.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:39.887+0000 7fdc53c00640 1 -- 172.21.15.91:0/1588731181 wait complete. 2024-02-02T13:31:39.907 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:31:40.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.114+0000 7fd333a00640 1 Processor -- start 2024-02-02T13:31:40.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.114+0000 7fd333a00640 1 -- start start 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd333a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd32c15d3e0 0x7fd32c0b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd333a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd32c0ce4b0 0x7fd32c070930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd333a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c0704a0 con 0x7fd32c0d1a60 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd333a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0d2800 con 0x7fd32c15d3e0 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd333a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0d1f60 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd331000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd32c15d3e0 0x7fd32c0b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd331000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd32c15d3e0 0x7fd32c0b88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37546/0 (socket says 172.21.15.91:37546) 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd331000640 1 -- 172.21.15.91:0/3623034958 learned_addr learned my addr 172.21.15.91:0/3623034958 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.116+0000 7fd323c00640 1 --1- 172.21.15.91:0/3623034958 >> v1:172.21.15.91:6789/0 conn(0x7fd32c0d1a60 0x7fd32c0cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34128/0 (socket says 172.21.15.91:34128) 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd331a00640 1 --2- 172.21.15.91:0/3623034958 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0ce4b0 0x7fd32c070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd331000640 1 -- 172.21.15.91:0/3623034958 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0ce4b0 msgr2=0x7fd32c070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd331000640 1 --2- 172.21.15.91:0/3623034958 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0ce4b0 0x7fd32c070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd331000640 1 -- 172.21.15.91:0/3623034958 >> v1:172.21.15.91:6789/0 conn(0x7fd32c0d1a60 legacy=0x7fd32c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd331000640 1 -- 172.21.15.91:0/3623034958 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0d2240 con 0x7fd32c15d3e0 2024-02-02T13:31:40.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.117+0000 7fd32be00640 1 -- 172.21.15.91:0/3623034958 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502503475 0 0) 0x7fd32c0704a0 con 0x7fd32c0d1a60 2024-02-02T13:31:40.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.118+0000 7fd331000640 1 --2- 172.21.15.91:0/3623034958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0b88f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0b8690 tx=0x7fd31c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2cb7adc4ed80b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.118+0000 7fd32be00640 1 -- 172.21.15.91:0/3623034958 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c004230 con 0x7fd32c15d3e0 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.118+0000 7fd32be00640 1 -- 172.21.15.91:0/3623034958 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c004390 con 0x7fd32c15d3e0 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.118+0000 7fd32be00640 1 -- 172.21.15.91:0/3623034958 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c030840 con 0x7fd32c15d3e0 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 -- 172.21.15.91:0/3623034958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c15d3e0 msgr2=0x7fd32c0b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 --2- 172.21.15.91:0/3623034958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0b88f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0b8690 tx=0x7fd31c02b980 comp rx=0 tx=0).stop 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 -- 172.21.15.91:0/3623034958 shutdown_connections 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 --2- 172.21.15.91:0/3623034958 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0ce4b0 0x7fd32c070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 --2- 172.21.15.91:0/3623034958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0b88f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 -- 172.21.15.91:0/3623034958 >> 172.21.15.91:0/3623034958 conn(0x7fd32c0cfa50 msgr2=0x7fd32c0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 -- 172.21.15.91:0/3623034958 shutdown_connections 2024-02-02T13:31:40.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.119+0000 7fd333a00640 1 -- 172.21.15.91:0/3623034958 wait complete. 2024-02-02T13:31:40.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.121+0000 7fd333a00640 1 Processor -- start 2024-02-02T13:31:40.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.121+0000 7fd333a00640 1 -- start start 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 0x7fd32c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0cb270 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0aaac0 con 0x7fd32c15d3e0 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd333a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0aada0 con 0x7fd32c0d1a60 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37556/0 (socket says 172.21.15.91:37556) 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 -- 172.21.15.91:0/2544734131 learned_addr learned my addr 172.21.15.91:0/2544734131 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd323c00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 0x7fd32c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.124 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 -- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 msgr2=0x7fd32c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 0x7fd32c0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 -- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 msgr2=0x7fd32c0abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331000640 1 -- 172.21.15.91:0/2544734131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c009460 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd331a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.122+0000 7fd323c00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 0x7fd32c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:40.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd331000640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02b9c0 tx=0x7fd31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c03e070 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c004500 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c030e00 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c0ab080 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.123+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0ac2b0 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.124+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c0cd6c0 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.127 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.125+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd31c003610 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.126+0000 7fd329600640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 0x7fd30406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.126+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd31c0aa820 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.126+0000 7fd323c00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 0x7fd30406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.127+0000 7fd323c00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 0x7fd30406c8f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd32c151e60 tx=0x7fd314009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.132 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.131+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd31c075d70 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.334+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd32c0ac2b0 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.334+0000 7fd329600640 1 -- 172.21.15.91:0/2544734131 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v131) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd32c0ac2b0 con 0x7fd32c0ce4b0 2024-02-02T13:31:40.336 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:31:40.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 msgr2=0x7fd30406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 0x7fd30406c8f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd32c151e60 tx=0x7fd314009040 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 msgr2=0x7fd32c0aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02b9c0 tx=0x7fd31c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 shutdown_connections 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd30406a430 0x7fd30406c8f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c0d1a60 0x7fd32c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c15d3e0 0x7fd32c0abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 --2- 172.21.15.91:0/2544734131 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd32c0ce4b0 0x7fd32c0aa3c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 >> 172.21.15.91:0/2544734131 conn(0x7fd32c0cfa50 msgr2=0x7fd32c0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 shutdown_connections 2024-02-02T13:31:40.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.337+0000 7fd333a00640 1 -- 172.21.15.91:0/2544734131 wait complete. 2024-02-02T13:31:40.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:31:40.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:40.358 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:31:40.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:31:40.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:40.474 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:31:40.526 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:40.526 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:40.569 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:40.569 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:40.661 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:40.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.866+0000 7f19dde00640 1 Processor -- start 2024-02-02T13:31:40.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.866+0000 7f19dde00640 1 -- start start 2024-02-02T13:31:40.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.867+0000 7f19dde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 0x7f19d80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.867+0000 7f19dde00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19d80b0700 0x7f19d80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.867+0000 7f19dde00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d80cc7c0 con 0x7f19d80ce130 2024-02-02T13:31:40.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.867+0000 7f19dde00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d80b10d0 con 0x7f19d80b0700 2024-02-02T13:31:40.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.867+0000 7f19dde00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80ce930 con 0x7f19d815a010 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19d80b0700 0x7f19d80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19d80b0700 0x7f19d80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37572/0 (socket says 172.21.15.91:37572) 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 -- 172.21.15.91:0/3735807029 learned_addr learned my addr 172.21.15.91:0/3735807029 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d6a00640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 0x7f19d80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 -- 172.21.15.91:0/3735807029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 msgr2=0x7f19d80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 0x7f19d80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 -- 172.21.15.91:0/3735807029 >> v1:172.21.15.91:6789/0 conn(0x7f19d80ce130 legacy=0x7f19d80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d7400640 1 -- 172.21.15.91:0/3735807029 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80cd640 con 0x7f19d80b0700 2024-02-02T13:31:40.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.868+0000 7f19d6a00640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 0x7f19d80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.869+0000 7f19d7400640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80b0700 0x7f19d80b0b90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009e50 tx=0x7f19d002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ac1c9d787bc3f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.869+0000 7f19d6000640 1 -- 172.21.15.91:0/3735807029 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d002cb00 con 0x7f19d80b0700 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.869+0000 7f19d6000640 1 -- 172.21.15.91:0/3735807029 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d002cc60 con 0x7f19d80b0700 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.869+0000 7f19d6000640 1 -- 172.21.15.91:0/3735807029 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00343f0 con 0x7f19d80b0700 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 -- 172.21.15.91:0/3735807029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80b0700 msgr2=0x7f19d80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80b0700 0x7f19d80b0b90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009e50 tx=0x7f19d002bf60 comp rx=0 tx=0).stop 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 -- 172.21.15.91:0/3735807029 shutdown_connections 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80b0700 0x7f19d80b0b90 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 --2- 172.21.15.91:0/3735807029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d815a010 0x7f19d80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 -- 172.21.15.91:0/3735807029 >> 172.21.15.91:0/3735807029 conn(0x7f19d80cec80 msgr2=0x7f19d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 -- 172.21.15.91:0/3735807029 shutdown_connections 2024-02-02T13:31:40.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.870+0000 7f19dde00640 1 -- 172.21.15.91:0/3735807029 wait complete. 2024-02-02T13:31:40.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.872+0000 7f19dde00640 1 Processor -- start 2024-02-02T13:31:40.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.872+0000 7f19dde00640 1 -- start start 2024-02-02T13:31:40.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d80b0700 0x7f19d81513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d815a010 0x7f19d815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b26d0 con 0x7f19d80ce130 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b29b0 con 0x7f19d815a010 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19dde00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b2c90 con 0x7f19d80b0700 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37574/0 (socket says 172.21.15.91:37574) 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 -- 172.21.15.91:0/3685789940 learned_addr learned my addr 172.21.15.91:0/3685789940 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19d6a00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d80b0700 0x7f19d81513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 -- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d80b0700 msgr2=0x7f19d81513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19d7400640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d815a010 0x7f19d815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d80b0700 0x7f19d81513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 -- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d815a010 msgr2=0x7f19d815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d815a010 0x7f19d815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.873+0000 7f19cde00640 1 -- 172.21.15.91:0/3685789940 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009b30 con 0x7f19d80ce130 2024-02-02T13:31:40.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19cde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c450 tx=0x7f19c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c00052d0 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c0005430 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0012870 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d80b2f70 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.874+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81518f0 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.875+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d80b3490 con 0x7f19d80ce130 2024-02-02T13:31:40.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.876+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f19c00129d0 con 0x7f19d80ce130 2024-02-02T13:31:40.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.877+0000 7f19cfe00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 0x7f19b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:40.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.877+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19c00884b0 con 0x7f19d80ce130 2024-02-02T13:31:40.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.877+0000 7f19d6a00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 0x7f19b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:40.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.878+0000 7f19d6a00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 0x7f19b006c8f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a950 tx=0x7f19c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:40.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:40.881+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f19c0053a00 con 0x7f19d80ce130 2024-02-02T13:31:41.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.079+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19d8066cf0 con 0x7f19d80ce130 2024-02-02T13:31:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.081+0000 7f19cfe00640 1 -- 172.21.15.91:0/3685789940 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f19c0057020 con 0x7f19d80ce130 2024-02-02T13:31:41.083 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:41.083 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:41.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 msgr2=0x7f19b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 0x7f19b006c8f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a950 tx=0x7f19c8008040 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 msgr2=0x7f19d815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c450 tx=0x7f19c000c910 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 shutdown_connections 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19b006a430 0x7f19b006c8f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d815a010 0x7f19d815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19d80ce130 0x7f19d815a5b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 --2- 172.21.15.91:0/3685789940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19d80b0700 0x7f19d81513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 >> 172.21.15.91:0/3685789940 conn(0x7f19d80cec80 msgr2=0x7f19d80ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 shutdown_connections 2024-02-02T13:31:41.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.084+0000 7f19dde00640 1 -- 172.21.15.91:0/3685789940 wait complete. 2024-02-02T13:31:41.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:14.703691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:15.895506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:41.104 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:41.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.308+0000 7fd4c5400640 1 Processor -- start 2024-02-02T13:31:41.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.308+0000 7fd4c5400640 1 -- start start 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4c5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd4c00acfe0 0x7fd4c006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4c5400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c015d870 con 0x7fd4c00b0700 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4c5400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c006bce0 con 0x7fd4c015d490 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4c5400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c0161220 con 0x7fd4c00acfe0 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4bf200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd4c00b0700 0x7fd4c00b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34152/0 (socket says 172.21.15.91:34152) 2024-02-02T13:31:41.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.310+0000 7fd4bf200640 1 -- 172.21.15.91:0/1083694732 learned_addr learned my addr 172.21.15.91:0/1083694732 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:41.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bde00640 1 --2- 172.21.15.91:0/1083694732 >> v2:172.21.15.113:3300/0 conn(0x7fd4c00acfe0 0x7fd4c006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830130113 0 0) 0x7fd4c015d870 con 0x7fd4c00b0700 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a0003520 con 0x7fd4c00b0700 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139731732 0 0) 0x7fd4a0003520 con 0x7fd4c00b0700 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 >> v2:172.21.15.113:3300/0 conn(0x7fd4c00acfe0 msgr2=0x7fd4c006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 --2- 172.21.15.91:0/1083694732 >> v2:172.21.15.113:3300/0 conn(0x7fd4c00acfe0 0x7fd4c006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 >> v1:172.21.15.113:6789/0 conn(0x7fd4c015d490 legacy=0x7fd4c00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00ad490 con 0x7fd4c00b0700 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd4b8001690 con 0x7fd4c00b0700 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.311+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 >> v1:172.21.15.91:6789/0 conn(0x7fd4c00b0700 legacy=0x7fd4c00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a0003970 0x7fd4a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 0x7fd4a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a0005180 0x7fd4a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0161220 con 0x7fd4a00047c0 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a000c020 con 0x7fd4a0005180 2024-02-02T13:31:41.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a000c1e0 con 0x7fd4a0003970 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 0x7fd4a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bf200640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a0005180 0x7fd4a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 -- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a0003970 msgr2=0x7fd4a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a0003970 0x7fd4a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 -- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a0005180 msgr2=0x7fd4a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a0005180 0x7fd4a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bde00640 1 -- 172.21.15.91:0/1083694732 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0003520 con 0x7fd4a00047c0 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bf200640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a0005180 0x7fd4a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:41.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.312+0000 7fd4bf200640 1 -- 172.21.15.91:0/1083694732 reap_dead start 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.313+0000 7fd4bde00640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 0x7fd4a0004c40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009fc0 tx=0x7fd4a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6380504f6c34a7a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b002cc20 con 0x7fd4a00047c0 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b000f040 con 0x7fd4a00047c0 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4bd400640 1 -- 172.21.15.91:0/1083694732 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0034440 con 0x7fd4a00047c0 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 -- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 msgr2=0x7fd4a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 0x7fd4a0004c40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009fc0 tx=0x7fd4a00057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 -- 172.21.15.91:0/1083694732 shutdown_connections 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 --2- 172.21.15.91:0/1083694732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4a00047c0 0x7fd4a0004c40 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 -- 172.21.15.91:0/1083694732 >> 172.21.15.91:0/1083694732 conn(0x7fd4c00cec80 msgr2=0x7fd4c00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:41.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.314+0000 7fd4c5400640 1 -- 172.21.15.91:0/1083694732 shutdown_connections 2024-02-02T13:31:41.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.315+0000 7fd4c5400640 1 -- 172.21.15.91:0/1083694732 wait complete. 2024-02-02T13:31:41.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.316+0000 7fd4c5400640 1 Processor -- start 2024-02-02T13:31:41.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.316+0000 7fd4c5400640 1 -- start start 2024-02-02T13:31:41.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a00047c0 0x7fd4c00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 0x7fd4c0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00c47c0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00b27f0 con 0x7fd4c00b2250 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.317+0000 7fd4c5400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00b2ad0 con 0x7fd4a00047c0 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37590/0 (socket says 172.21.15.91:37590) 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 -- 172.21.15.91:0/3388443507 learned_addr learned my addr 172.21.15.91:0/3388443507 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bde00640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 0x7fd4c0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 -- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a00047c0 msgr2=0x7fd4c00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4be800640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a00047c0 0x7fd4c00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a00047c0 0x7fd4c00b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 -- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 msgr2=0x7fd4c0156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 0x7fd4c0156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 -- 172.21.15.91:0/3388443507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00b2db0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bde00640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 0x7fd4c0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.318+0000 7fd4bf200640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd4c006d930 tx=0x7fd4b800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.319+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80074e0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.319+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8007640 con 0x7fd4c00c3e00 2024-02-02T13:31:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.319+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8004a20 con 0x7fd4c00c3e00 2024-02-02T13:31:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.319+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c00b3090 con 0x7fd4c00c3e00 2024-02-02T13:31:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.319+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c0156660 con 0x7fd4c00c3e00 2024-02-02T13:31:41.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.320+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c0066cf0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.321+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd4b8004b80 con 0x7fd4c00c3e00 2024-02-02T13:31:41.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.322+0000 7fd4b6a00640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 0x7fd49806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.322+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b8088050 con 0x7fd4c00c3e00 2024-02-02T13:31:41.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.322+0000 7fd4be800640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 0x7fd49806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.323+0000 7fd4be800640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 0x7fd49806c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80015d0 tx=0x7fd4a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.326+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd4b80535a0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.516+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4c00b4c00 con 0x7fd4c00c3e00 2024-02-02T13:31:41.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.516+0000 7fd4b6a00640 1 -- 172.21.15.91:0/3388443507 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4b801a0a0 con 0x7fd4c00c3e00 2024-02-02T13:31:41.518 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:41.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 msgr2=0x7fd49806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 0x7fd49806c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80015d0 tx=0x7fd4a8009040 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 msgr2=0x7fd4c00c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd4c006d930 tx=0x7fd4b800d890 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 shutdown_connections 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd49806a430 0x7fd49806c8f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd4c00c3e00 0x7fd4c00c4280 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b2250 0x7fd4c0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 --2- 172.21.15.91:0/3388443507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4a00047c0 0x7fd4c00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 >> 172.21.15.91:0/3388443507 conn(0x7fd4c00cec80 msgr2=0x7fd4c006f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 shutdown_connections 2024-02-02T13:31:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.519+0000 7fd4c5400640 1 -- 172.21.15.91:0/3388443507 wait complete. 2024-02-02T13:31:41.539 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:31:41.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.745+0000 7fcfd4600640 1 Processor -- start 2024-02-02T13:31:41.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.745+0000 7fcfd4600640 1 -- start start 2024-02-02T13:31:41.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.746+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.746+0000 7fcfd4600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcfcc0b0c40 0x7fcfcc0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.746+0000 7fcfd4600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc15fa00 con 0x7fcfcc151f90 2024-02-02T13:31:41.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.746+0000 7fcfd4600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0b10d0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.746+0000 7fcfd4600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc070f80 con 0x7fcfcc15f5a0 2024-02-02T13:31:41.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd1c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcfcc151f90 0x7fcfcc06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34162/0 (socket says 172.21.15.91:34162) 2024-02-02T13:31:41.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd1c00640 1 -- 172.21.15.91:0/2508627260 learned_addr learned my addr 172.21.15.91:0/2508627260 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:41.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd2600640 1 --2- 172.21.15.91:0/2508627260 >> v2:172.21.15.91:3300/0 conn(0x7fcfcc0b0c40 0x7fcfcc0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd1200640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfcbe00640 1 -- 172.21.15.91:0/2508627260 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204434274 0 0) 0x7fcfbc001490 con 0x7fcfcc151f90 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfcbe00640 1 -- 172.21.15.91:0/2508627260 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfac003520 con 0x7fcfcc151f90 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd2600640 1 -- 172.21.15.91:0/2508627260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 msgr2=0x7fcfcc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd2600640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd2600640 1 -- 172.21.15.91:0/2508627260 >> v1:172.21.15.91:6789/0 conn(0x7fcfcc151f90 legacy=0x7fcfcc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.747+0000 7fcfd2600640 1 -- 172.21.15.91:0/2508627260 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc152370 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.748+0000 7fcfd2600640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc0b3af0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40097a0 tx=0x7fcfc402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4cfb9e53049f10 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfcbe00640 1 -- 172.21.15.91:0/2508627260 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc402d040 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfcbe00640 1 -- 172.21.15.91:0/2508627260 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4031810 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfcbe00640 1 -- 172.21.15.91:0/2508627260 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4031ba0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 -- 172.21.15.91:0/2508627260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 msgr2=0x7fcfcc0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc0b3af0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40097a0 tx=0x7fcfc402b8b0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 -- 172.21.15.91:0/2508627260 shutdown_connections 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc0b3af0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 --2- 172.21.15.91:0/2508627260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 -- 172.21.15.91:0/2508627260 >> 172.21.15.91:0/2508627260 conn(0x7fcfcc0cec80 msgr2=0x7fcfcc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 -- 172.21.15.91:0/2508627260 shutdown_connections 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.749+0000 7fcfd4600640 1 -- 172.21.15.91:0/2508627260 wait complete. 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.751+0000 7fcfd4600640 1 Processor -- start 2024-02-02T13:31:41.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.751+0000 7fcfd4600640 1 -- start start 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc151f90 0x7fcfcc0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c7800 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc1f6d30 con 0x7fcfcc151f90 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd4600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c7240 con 0x7fcfcc15f5a0 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd1c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd1c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37614/0 (socket says 172.21.15.91:37614) 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.752+0000 7fcfd1c00640 1 -- 172.21.15.91:0/3849852832 learned_addr learned my addr 172.21.15.91:0/3849852832 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 -- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 msgr2=0x7fcfcc0c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 -- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc151f90 msgr2=0x7fcfcc0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc151f90 0x7fcfcc0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 -- 172.21.15.91:0/3849852832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4009400 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd1c00640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00cbb0 tx=0x7fcfbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00ad40 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc012400 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0c7520 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00aea0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.753+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0b1d20 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.754+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc066cf0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.756+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcfbc007070 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.757+0000 7fcfc9600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 0x7fcfa406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:41.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.757+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfbc0887f0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.757+0000 7fcfd1200640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 0x7fcfa406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:41.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.758+0000 7fcfd1200640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 0x7fcfa406c920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc15b070 tx=0x7fcfb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:41.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.761+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcfbc0577d0 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.951+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfcc15fa00 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.952+0000 7fcfc9600640 1 -- 172.21.15.91:0/3849852832 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfbc017020 con 0x7fcfcc0b0c40 2024-02-02T13:31:41.954 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 msgr2=0x7fcfa406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 0x7fcfa406c920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc15b070 tx=0x7fcfb4008040 comp rx=0 tx=0).stop 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 msgr2=0x7fcfcc1f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00cbb0 tx=0x7fcfbc00a430 comp rx=0 tx=0).stop 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 shutdown_connections 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a460 0x7fcfa406c920 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15f5a0 0x7fcfcc0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc151f90 0x7fcfcc0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 --2- 172.21.15.91:0/3849852832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0b0c40 0x7fcfcc1f2030 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:41.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.954+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 >> 172.21.15.91:0/3849852832 conn(0x7fcfcc0cec80 msgr2=0x7fcfcc0ada80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:41.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.955+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 shutdown_connections 2024-02-02T13:31:41.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:41.955+0000 7fcfd4600640 1 -- 172.21.15.91:0/3849852832 wait complete. 2024-02-02T13:31:41.975 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:31:42.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.185+0000 7f2333000640 1 Processor -- start 2024-02-02T13:31:42.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.185+0000 7f2333000640 1 -- start start 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.186+0000 7f2333000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f232c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2333000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c0ce3b0 con 0x7f232c0add80 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2333000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c0ce8e0 con 0x7f232c0c9450 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2333000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0adbd0 con 0x7f232c15d3a0 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2331200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f232c0add80 0x7f232c0706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34168/0 (socket says 172.21.15.91:34168) 2024-02-02T13:31:42.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2331200640 1 -- 172.21.15.91:0/64418886 learned_addr learned my addr 172.21.15.91:0/64418886 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f2331c00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f232c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118590392 0 0) 0x7f232c0ce3b0 con 0x7f232c0add80 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230c003520 con 0x7f232c0add80 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.187+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495054614 0 0) 0x7f230c003520 con 0x7f232c0add80 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 msgr2=0x7f232c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f232c06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 >> v1:172.21.15.113:6789/0 conn(0x7f232c0c9450 legacy=0x7f232c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c070db0 con 0x7f232c0add80 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2314001670 con 0x7f232c0add80 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 >> v1:172.21.15.91:6789/0 conn(0x7f232c0add80 legacy=0x7f232c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f230c0039a0 0x7f230c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f230c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f230c004d00 0x7f230c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0adbd0 con 0x7f230c0039a0 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c00bb40 con 0x7f230c004d00 2024-02-02T13:31:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c00bd00 con 0x7f232c15d3a0 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331c00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f230c0039a0 0x7f230c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331c00640 1 -- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 msgr2=0x7f230c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331200640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f230c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331c00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f232c15d3a0 0x7f230c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331c00640 1 -- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f230c004d00 msgr2=0x7f230c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.188+0000 7f2331c00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f230c004d00 0x7f230c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.189+0000 7f2331c00640 1 -- 172.21.15.91:0/64418886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c003520 con 0x7f230c0039a0 2024-02-02T13:31:42.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.189+0000 7f2331c00640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f230c0039a0 0x7f230c003dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ac0 tx=0x7f230c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4aa53c1e23d5de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c02c6b0 con 0x7f230c0039a0 2024-02-02T13:31:42.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c02c810 con 0x7f230c0039a0 2024-02-02T13:31:42.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f232be00640 1 -- 172.21.15.91:0/64418886 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c02cba0 con 0x7f230c0039a0 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f2333000640 1 -- 172.21.15.91:0/64418886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f230c0039a0 msgr2=0x7f230c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f2333000640 1 --2- 172.21.15.91:0/64418886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f230c0039a0 0x7f230c003dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ac0 tx=0x7f230c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f2332600640 1 -- 172.21.15.91:0/64418886 reap_dead start 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f2333000640 1 -- 172.21.15.91:0/64418886 shutdown_connections 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.190+0000 7f2333000640 1 -- 172.21.15.91:0/64418886 >> 172.21.15.91:0/64418886 conn(0x7f232c0cea90 msgr2=0x7f232c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.191+0000 7f2333000640 1 -- 172.21.15.91:0/64418886 shutdown_connections 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.191+0000 7f2333000640 1 -- 172.21.15.91:0/64418886 wait complete. 2024-02-02T13:31:42.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.192+0000 7f2333000640 1 Processor -- start 2024-02-02T13:31:42.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.192+0000 7f2333000640 1 -- start start 2024-02-02T13:31:42.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f230c0039a0 0x7f232c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f232c0c8220 0x7f232c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0af480 con 0x7f232c0c7120 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0af760 con 0x7f232c0c8220 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.193+0000 7f2333000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0afa40 con 0x7f230c0039a0 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37632/0 (socket says 172.21.15.91:37632) 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 -- 172.21.15.91:0/2911708219 learned_addr learned my addr 172.21.15.91:0/2911708219 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331c00640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f230c0039a0 0x7f232c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2332600640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f232c0c8220 0x7f232c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 -- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f230c0039a0 msgr2=0x7f232c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f230c0039a0 0x7f232c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 -- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f232c0c8220 msgr2=0x7f232c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f232c0c8220 0x7f232c0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.194+0000 7f2331200640 1 -- 172.21.15.91:0/2911708219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0afd20 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2331200640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f231400d7e0 tx=0x7f231400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140074e0 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2314007640 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c0b0000 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140049e0 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.195+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c1592b0 con 0x7f232c0c7120 2024-02-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.196+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232c0b0240 con 0x7f232c0c7120 2024-02-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.197+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2314004b40 con 0x7f232c0c7120 2024-02-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.198+0000 7f2329600640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 0x7f230406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.198+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23140886b0 con 0x7f232c0c7120 2024-02-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.198+0000 7f2331c00640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 0x7f230406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.199+0000 7f2331c00640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 0x7f230406c920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f231c036000 tx=0x7f231c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.202+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f232c0b0240 con 0x7f232c0c7120 2024-02-02T13:31:42.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.390+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f232c15cf00 con 0x7f232c0c7120 2024-02-02T13:31:42.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.390+0000 7f2329600640 1 -- 172.21.15.91:0/2911708219 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2314053c00 con 0x7f232c0c7120 2024-02-02T13:31:42.392 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 msgr2=0x7f230406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 0x7f230406c920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f231c036000 tx=0x7f231c030000 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 msgr2=0x7f232c0b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f231400d7e0 tx=0x7f231400dca0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 shutdown_connections 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f230406a460 0x7f230406c920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f232c0c8220 0x7f232c0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f232c0c7120 0x7f232c0b0520 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 --2- 172.21.15.91:0/2911708219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f230c0039a0 0x7f232c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 >> 172.21.15.91:0/2911708219 conn(0x7f232c0cea90 msgr2=0x7f232c06fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 shutdown_connections 2024-02-02T13:31:42.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.393+0000 7f2333000640 1 -- 172.21.15.91:0/2911708219 wait complete. 2024-02-02T13:31:42.413 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:31:42.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.620+0000 7f2b58000640 1 Processor -- start 2024-02-02T13:31:42.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.621+0000 7f2b58000640 1 -- start start 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b58000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2b501609b0 0x7f2b500cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b58000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b50070980 0x7f2b500b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b58000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b50160d90 con 0x7f2b500ce530 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b58000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b50070fe0 con 0x7f2b501609b0 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b58000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500d2940 con 0x7f2b50070980 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b55600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2b501609b0 0x7f2b500cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b55600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2b501609b0 0x7f2b500cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37642/0 (socket says 172.21.15.91:37642) 2024-02-02T13:31:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.622+0000 7f2b55600640 1 -- 172.21.15.91:0/1209783920 learned_addr learned my addr 172.21.15.91:0/1209783920 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:42.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.623+0000 7f2b55600640 1 -- 172.21.15.91:0/1209783920 >> v2:172.21.15.113:3300/0 conn(0x7f2b50070980 msgr2=0x7f2b500b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:42.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.623+0000 7f2b55600640 1 --2- 172.21.15.91:0/1209783920 >> v2:172.21.15.113:3300/0 conn(0x7f2b50070980 0x7f2b500b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.623+0000 7f2b55600640 1 -- 172.21.15.91:0/1209783920 >> v1:172.21.15.113:6789/0 conn(0x7f2b500ce530 legacy=0x7f2b50070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:42.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.623+0000 7f2b55600640 1 -- 172.21.15.91:0/1209783920 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500d2c20 con 0x7f2b501609b0 2024-02-02T13:31:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.624+0000 7f2b55600640 1 --2- 172.21.15.91:0/1209783920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b501609b0 0x7f2b500cdff0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2b5008c450 tx=0x7f2b4002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec78a444197054 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.624+0000 7f2b4fe00640 1 -- 172.21.15.91:0/1209783920 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031750 con 0x7f2b501609b0 2024-02-02T13:31:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.624+0000 7f2b4fe00640 1 -- 172.21.15.91:0/1209783920 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b400318b0 con 0x7f2b501609b0 2024-02-02T13:31:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.624+0000 7f2b4fe00640 1 -- 172.21.15.91:0/1209783920 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031c40 con 0x7f2b501609b0 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 -- 172.21.15.91:0/1209783920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b501609b0 msgr2=0x7f2b500cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 --2- 172.21.15.91:0/1209783920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b501609b0 0x7f2b500cdff0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2b5008c450 tx=0x7f2b4002b7f0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 -- 172.21.15.91:0/1209783920 shutdown_connections 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 --2- 172.21.15.91:0/1209783920 >> v2:172.21.15.113:3300/0 conn(0x7f2b50070980 0x7f2b500b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 --2- 172.21.15.91:0/1209783920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b501609b0 0x7f2b500cdff0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 -- 172.21.15.91:0/1209783920 >> 172.21.15.91:0/1209783920 conn(0x7f2b500d3020 msgr2=0x7f2b500d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 -- 172.21.15.91:0/1209783920 shutdown_connections 2024-02-02T13:31:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.625+0000 7f2b58000640 1 -- 172.21.15.91:0/1209783920 wait complete. 2024-02-02T13:31:42.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.627+0000 7f2b58000640 1 Processor -- start 2024-02-02T13:31:42.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.627+0000 7f2b58000640 1 -- start start 2024-02-02T13:31:42.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b500ce530 0x7f2b500cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 0x7f2b500cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50151a50 con 0x7f2b50070980 2024-02-02T13:31:42.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50156010 con 0x7f2b501609b0 2024-02-02T13:31:42.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b58000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501562f0 con 0x7f2b500ce530 2024-02-02T13:31:42.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37650/0 (socket says 172.21.15.91:37650) 2024-02-02T13:31:42.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 -- 172.21.15.91:0/2462647626 learned_addr learned my addr 172.21.15.91:0/2462647626 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b54c00640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b500ce530 0x7f2b500cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b56000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 0x7f2b500cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 -- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b500ce530 msgr2=0x7f2b500cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b500ce530 0x7f2b500cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 -- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 msgr2=0x7f2b500cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 0x7f2b500cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b55600640 1 -- 172.21.15.91:0/2462647626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400094d0 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.628+0000 7f2b56000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 0x7f2b500cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b55600640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2b40000c00 tx=0x7f2b400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40033c60 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40033dc0 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b500ca0f0 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4003d6b0 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.629+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b500ca6b0 con 0x7f2b50070980 2024-02-02T13:31:42.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.630+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50066cf0 con 0x7f2b50070980 2024-02-02T13:31:42.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.632+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2b4003a040 con 0x7f2b50070980 2024-02-02T13:31:42.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.632+0000 7f2b4d600640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 0x7f2b2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:42.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.633+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b400aa910 con 0x7f2b50070980 2024-02-02T13:31:42.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.633+0000 7f2b54c00640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 0x7f2b2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:42.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.633+0000 7f2b54c00640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 0x7f2b2806c8f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2b500b5660 tx=0x7f2b38009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:42.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.637+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2b40075f00 con 0x7f2b50070980 2024-02-02T13:31:42.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.841+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b50160d90 con 0x7f2b50070980 2024-02-02T13:31:42.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.842+0000 7f2b4d600640 1 -- 172.21.15.91:0/2462647626 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v131) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b50160d90 con 0x7f2b50070980 2024-02-02T13:31:42.843 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 msgr2=0x7f2b2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 0x7f2b2806c8f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2b500b5660 tx=0x7f2b38009040 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 msgr2=0x7f2b50151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2b40000c00 tx=0x7f2b400046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 shutdown_connections 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2b2806a430 0x7f2b2806c8f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b500ce530 0x7f2b500cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b501609b0 0x7f2b500cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 --2- 172.21.15.91:0/2462647626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b50070980 0x7f2b50151510 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 >> 172.21.15.91:0/2462647626 conn(0x7f2b500d3020 msgr2=0x7f2b500d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 shutdown_connections 2024-02-02T13:31:42.847 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:42.845+0000 7f2b58000640 1 -- 172.21.15.91:0/2462647626 wait complete. 2024-02-02T13:31:42.865 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:31:42.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:42.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:31:42.973 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:31:42.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:31:42.973 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:31:43.025 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:43.025 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:43.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:31:43.083 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:43.083 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:43.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:31:43.137 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:43.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.351+0000 7ff893c00640 1 Processor -- start 2024-02-02T13:31:43.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.351+0000 7ff893c00640 1 -- start start 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff893c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff88c0b0e10 0x7ff88c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff893c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c15d870 con 0x7ff88c15d490 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff893c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c06bce0 con 0x7ff88c0acfe0 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff893c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c161220 con 0x7ff88c0b0e10 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff891200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff88c15d490 0x7ff88c0ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34178/0 (socket says 172.21.15.91:34178) 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff891200640 1 -- 172.21.15.91:0/667467828 learned_addr learned my addr 172.21.15.91:0/667467828 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.353+0000 7ff891c00640 1 --2- 172.21.15.91:0/667467828 >> v2:172.21.15.91:3300/0 conn(0x7ff88c0b0e10 0x7ff88c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.354+0000 7ff891c00640 1 -- 172.21.15.91:0/667467828 >> v1:172.21.15.113:6789/0 conn(0x7ff88c0acfe0 legacy=0x7ff88c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:43.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.354+0000 7ff88b400640 1 -- 172.21.15.91:0/667467828 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878403540 0 0) 0x7ff88c15d870 con 0x7ff88c15d490 2024-02-02T13:31:43.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.354+0000 7ff891c00640 1 -- 172.21.15.91:0/667467828 >> v1:172.21.15.91:6789/0 conn(0x7ff88c15d490 legacy=0x7ff88c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.354+0000 7ff891c00640 1 -- 172.21.15.91:0/667467828 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0ad490 con 0x7ff88c0b0e10 2024-02-02T13:31:43.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.355+0000 7ff891c00640 1 --2- 172.21.15.91:0/667467828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c06c170 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff884009d90 tx=0x7ff88402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1371d158dd0ab9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.355+0000 7ff88b400640 1 -- 172.21.15.91:0/667467828 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88402cc60 con 0x7ff88c0b0e10 2024-02-02T13:31:43.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.355+0000 7ff88b400640 1 -- 172.21.15.91:0/667467828 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88402cdc0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.355+0000 7ff88b400640 1 -- 172.21.15.91:0/667467828 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884034660 con 0x7ff88c0b0e10 2024-02-02T13:31:43.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 -- 172.21.15.91:0/667467828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 msgr2=0x7ff88c06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 --2- 172.21.15.91:0/667467828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c06c170 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff884009d90 tx=0x7ff88402beb0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 -- 172.21.15.91:0/667467828 shutdown_connections 2024-02-02T13:31:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 --2- 172.21.15.91:0/667467828 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c06c170 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 -- 172.21.15.91:0/667467828 >> 172.21.15.91:0/667467828 conn(0x7ff88c0cec80 msgr2=0x7ff88c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 -- 172.21.15.91:0/667467828 shutdown_connections 2024-02-02T13:31:43.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.356+0000 7ff893c00640 1 -- 172.21.15.91:0/667467828 wait complete. 2024-02-02T13:31:43.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.358+0000 7ff893c00640 1 Processor -- start 2024-02-02T13:31:43.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.358+0000 7ff893c00640 1 -- start start 2024-02-02T13:31:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88c0acfe0 0x7ff88c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff88c15d490 0x7ff88c0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b1870 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1f6d50 con 0x7ff88c15d490 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff893c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1f7030 con 0x7ff88c0acfe0 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37660/0 (socket says 172.21.15.91:37660) 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 -- 172.21.15.91:0/2021902600 learned_addr learned my addr 172.21.15.91:0/2021902600 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 -- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88c0acfe0 msgr2=0x7ff88c156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88c0acfe0 0x7ff88c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 -- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff88c15d490 msgr2=0x7ff88c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff88c15d490 0x7ff88c0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.359+0000 7ff88be00640 1 -- 172.21.15.91:0/2021902600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009a70 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff88be00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff87400b3e0 tx=0x7ff87400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874011830 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8740119b0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874011d60 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c0b1a80 con 0x7ff88c0b0e10 2024-02-02T13:31:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.360+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c0c3f40 con 0x7ff88c0b0e10 2024-02-02T13:31:43.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.361+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c1f71f0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.362+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff87401a400 con 0x7ff88c0b0e10 2024-02-02T13:31:43.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.363+0000 7ff888c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 0x7ff86406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.363+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff874088ad0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.363+0000 7ff891200640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 0x7ff86406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.364+0000 7ff891200640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 0x7ff86406c8f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005880 tx=0x7ff87c007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.367+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8740540b0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.599+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff88c0b1fa0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.600+0000 7ff888c00640 1 -- 172.21.15.91:0/2021902600 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7ff88c0b1fa0 con 0x7ff88c0b0e10 2024-02-02T13:31:43.602 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:43.602 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:43.602 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:43.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 msgr2=0x7ff86406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 0x7ff86406c8f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005880 tx=0x7ff87c007950 comp rx=0 tx=0).stop 2024-02-02T13:31:43.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 msgr2=0x7ff88c1f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff87400b3e0 tx=0x7ff87400b8a0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 shutdown_connections 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86406a430 0x7ff86406c8f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff88c15d490 0x7ff88c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff88c0b0e10 0x7ff88c1f62d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 --2- 172.21.15.91:0/2021902600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88c0acfe0 0x7ff88c156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 >> 172.21.15.91:0/2021902600 conn(0x7ff88c0cec80 msgr2=0x7ff88c0ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 shutdown_connections 2024-02-02T13:31:43.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.603+0000 7ff893c00640 1 -- 172.21.15.91:0/2021902600 wait complete. 2024-02-02T13:31:43.622 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:43.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.825+0000 7fac81c00640 1 Processor -- start 2024-02-02T13:31:43.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.825+0000 7fac81c00640 1 -- start start 2024-02-02T13:31:43.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac81c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 0x7fac7c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac81c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac7c0b1d70 0x7fac7c0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac81c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7c0ce210 con 0x7fac7c070270 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac81c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c0ce4f0 con 0x7fac7c0b1d70 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac81c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c0ce7d0 con 0x7fac7c160830 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac71e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac7c0b1d70 0x7fac7c0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac71e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac7c0b1d70 0x7fac7c0711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37668/0 (socket says 172.21.15.91:37668) 2024-02-02T13:31:43.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.827+0000 7fac71e00640 1 -- 172.21.15.91:0/4179316210 learned_addr learned my addr 172.21.15.91:0/4179316210 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac7aa00640 1 --2- 172.21.15.91:0/4179316210 >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 0x7fac7c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac71e00640 1 -- 172.21.15.91:0/4179316210 >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 msgr2=0x7fac7c0b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac71e00640 1 --2- 172.21.15.91:0/4179316210 >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 0x7fac7c0b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac71e00640 1 -- 172.21.15.91:0/4179316210 >> v1:172.21.15.113:6789/0 conn(0x7fac7c070270 legacy=0x7fac7c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac71e00640 1 -- 172.21.15.91:0/4179316210 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0d17b0 con 0x7fac7c0b1d70 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.828+0000 7fac7aa00640 1 --2- 172.21.15.91:0/4179316210 >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 0x7fac7c0b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:43.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.829+0000 7fac71e00640 1 --2- 172.21.15.91:0/4179316210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c0b1d70 0x7fac7c0711f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fac7c069d00 tx=0x7fac6402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d582653174177d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.829+0000 7fac7a000640 1 -- 172.21.15.91:0/4179316210 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac640317e0 con 0x7fac7c0b1d70 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.829+0000 7fac7a000640 1 -- 172.21.15.91:0/4179316210 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac64031940 con 0x7fac7c0b1d70 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.829+0000 7fac7a000640 1 -- 172.21.15.91:0/4179316210 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64031cd0 con 0x7fac7c0b1d70 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 -- 172.21.15.91:0/4179316210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c0b1d70 msgr2=0x7fac7c0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 --2- 172.21.15.91:0/4179316210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c0b1d70 0x7fac7c0711f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fac7c069d00 tx=0x7fac6402b880 comp rx=0 tx=0).stop 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 -- 172.21.15.91:0/4179316210 shutdown_connections 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 --2- 172.21.15.91:0/4179316210 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c0b1d70 0x7fac7c0711f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 --2- 172.21.15.91:0/4179316210 >> v2:172.21.15.113:3300/0 conn(0x7fac7c160830 0x7fac7c0b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 -- 172.21.15.91:0/4179316210 >> 172.21.15.91:0/4179316210 conn(0x7fac7c0d3030 msgr2=0x7fac7c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 -- 172.21.15.91:0/4179316210 shutdown_connections 2024-02-02T13:31:43.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.830+0000 7fac81c00640 1 -- 172.21.15.91:0/4179316210 wait complete. 2024-02-02T13:31:43.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.832+0000 7fac81c00640 1 Processor -- start 2024-02-02T13:31:43.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.832+0000 7fac81c00640 1 -- start start 2024-02-02T13:31:43.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac7c070270 0x7fac7c157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac7c0b1d70 0x7fac7c1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1f4440 con 0x7fac7c160830 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1f3e50 con 0x7fac7c070270 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac81c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c157d70 con 0x7fac7c0b1d70 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37676/0 (socket says 172.21.15.91:37676) 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 -- 172.21.15.91:0/1035277635 learned_addr learned my addr 172.21.15.91:0/1035277635 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7aa00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac7c070270 0x7fac7c157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac71e00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac7c0b1d70 0x7fac7c1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 -- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac7c0b1d70 msgr2=0x7fac7c1f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac7c0b1d70 0x7fac7c1f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 -- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac7c070270 msgr2=0x7fac7c157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac7c070270 0x7fac7c157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:43.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.833+0000 7fac7b400640 1 -- 172.21.15.91:0/1035277635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac640094d0 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac7b400640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c780 tx=0x7fac7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74003d80 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7400ce90 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac740052b0 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c1f37c0 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.834+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c1f4060 con 0x7fac7c160830 2024-02-02T13:31:43.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.835+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c1f4580 con 0x7fac7c160830 2024-02-02T13:31:43.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.836+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fac74005410 con 0x7fac7c160830 2024-02-02T13:31:43.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.837+0000 7fac73400640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 0x7fac5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:43.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.837+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac74087e00 con 0x7fac7c160830 2024-02-02T13:31:43.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.837+0000 7fac7aa00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 0x7fac5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:43.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.838+0000 7fac7aa00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 0x7fac5406c8f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0cc6e0 tx=0x7fac6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:43.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:43.841+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fac7c1f4580 con 0x7fac7c160830 2024-02-02T13:31:44.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.035+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac7c1f4580 con 0x7fac7c160830 2024-02-02T13:31:44.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.037+0000 7fac73400640 1 -- 172.21.15.91:0/1035277635 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7fac7c1f4580 con 0x7fac7c160830 2024-02-02T13:31:44.038 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 msgr2=0x7fac5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 0x7fac5406c8f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0cc6e0 tx=0x7fac6c008040 comp rx=0 tx=0).stop 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 msgr2=0x7fac7c1f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c780 tx=0x7fac7400cc40 comp rx=0 tx=0).stop 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 shutdown_connections 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac5406a430 0x7fac5406c8f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac7c0b1d70 0x7fac7c1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac7c160830 0x7fac7c1f31d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 --2- 172.21.15.91:0/1035277635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac7c070270 0x7fac7c157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.039+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 >> 172.21.15.91:0/1035277635 conn(0x7fac7c0d3030 msgr2=0x7fac7c15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:44.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.040+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 shutdown_connections 2024-02-02T13:31:44.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.040+0000 7fac81c00640 1 -- 172.21.15.91:0/1035277635 wait complete. 2024-02-02T13:31:44.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:14.703691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:15.895506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:44.061 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:44.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.269+0000 7fa41ca00640 1 Processor -- start 2024-02-02T13:31:44.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.269+0000 7fa41ca00640 1 -- start start 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4140ce130 0x7fa4140c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa4140b0700 0x7fa4140b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140cc7c0 con 0x7fa41415a010 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41ca00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140b10b0 con 0x7fa4140b0700 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140ce930 con 0x7fa4140ce130 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa4140b0700 0x7fa4140b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa4140b0700 0x7fa4140b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37684/0 (socket says 172.21.15.91:37684) 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 -- 172.21.15.91:0/2816377441 learned_addr learned my addr 172.21.15.91:0/2816377441 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa419600640 1 --2- 172.21.15.91:0/2816377441 >> v2:172.21.15.113:3300/0 conn(0x7fa4140ce130 0x7fa4140c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 -- 172.21.15.91:0/2816377441 >> v2:172.21.15.113:3300/0 conn(0x7fa4140ce130 msgr2=0x7fa4140c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:44.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 --2- 172.21.15.91:0/2816377441 >> v2:172.21.15.113:3300/0 conn(0x7fa4140ce130 0x7fa4140c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 -- 172.21.15.91:0/2816377441 >> v1:172.21.15.113:6789/0 conn(0x7fa41415a010 legacy=0x7fa4140ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:44.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.271+0000 7fa41aa00640 1 -- 172.21.15.91:0/2816377441 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140cd640 con 0x7fa4140b0700 2024-02-02T13:31:44.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.272+0000 7fa41aa00640 1 --2- 172.21.15.91:0/2816377441 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140b0b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b1870 tx=0x7fa40c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfae48bd97b0901 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.273+0000 7fa418c00640 1 -- 172.21.15.91:0/2816377441 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02ca80 con 0x7fa4140b0700 2024-02-02T13:31:44.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.273+0000 7fa418c00640 1 -- 172.21.15.91:0/2816377441 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c02cbe0 con 0x7fa4140b0700 2024-02-02T13:31:44.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.273+0000 7fa418c00640 1 -- 172.21.15.91:0/2816377441 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0343f0 con 0x7fa4140b0700 2024-02-02T13:31:44.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.273+0000 7fa41ca00640 1 -- 172.21.15.91:0/2816377441 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 msgr2=0x7fa4140b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.273+0000 7fa41ca00640 1 --2- 172.21.15.91:0/2816377441 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140b0b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b1870 tx=0x7fa40c02bee0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 -- 172.21.15.91:0/2816377441 shutdown_connections 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 --2- 172.21.15.91:0/2816377441 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140b0b70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 --2- 172.21.15.91:0/2816377441 >> v2:172.21.15.113:3300/0 conn(0x7fa4140ce130 0x7fa4140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 -- 172.21.15.91:0/2816377441 >> 172.21.15.91:0/2816377441 conn(0x7fa4140cec80 msgr2=0x7fa4140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 -- 172.21.15.91:0/2816377441 shutdown_connections 2024-02-02T13:31:44.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.274+0000 7fa41ca00640 1 -- 172.21.15.91:0/2816377441 wait complete. 2024-02-02T13:31:44.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.275+0000 7fa41ca00640 1 Processor -- start 2024-02-02T13:31:44.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.276+0000 7fa41ca00640 1 -- start start 2024-02-02T13:31:44.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.276+0000 7fa41ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4140ce130 0x7fa4140c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa41415a010 0x7fa414157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41ca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414070f30 con 0x7fa4140b0700 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140c5290 con 0x7fa41415a010 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140c4fb0 con 0x7fa4140ce130 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37692/0 (socket says 172.21.15.91:37692) 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 -- 172.21.15.91:0/3721357909 learned_addr learned my addr 172.21.15.91:0/3721357909 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 -- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4140ce130 msgr2=0x7fa4140c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4140ce130 0x7fa4140c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 -- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa41415a010 msgr2=0x7fa414157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa41415a010 0x7fa414157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.277+0000 7fa41a000640 1 -- 172.21.15.91:0/3721357909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c009b30 con 0x7fa4140b0700 2024-02-02T13:31:44.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa41a000640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b3f0 tx=0x7fa40400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011830 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404011990 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41415a490 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011d40 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.278+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41415aa50 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.279+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa414066cf0 con 0x7fa4140b0700 2024-02-02T13:31:44.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.280+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa40401a400 con 0x7fa4140b0700 2024-02-02T13:31:44.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.281+0000 7fa412000640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 0x7fa3ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.281+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa404088820 con 0x7fa4140b0700 2024-02-02T13:31:44.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.282+0000 7fa419600640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 0x7fa3ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.282+0000 7fa419600640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 0x7fa3ec06c8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4140adb70 tx=0x7fa3fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.286+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa404053d70 con 0x7fa4140b0700 2024-02-02T13:31:44.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.475+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4140cca00 con 0x7fa4140b0700 2024-02-02T13:31:44.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.476+0000 7fa412000640 1 -- 172.21.15.91:0/3721357909 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa404057390 con 0x7fa4140b0700 2024-02-02T13:31:44.478 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:44.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 msgr2=0x7fa3ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 0x7fa3ec06c8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4140adb70 tx=0x7fa3fc008040 comp rx=0 tx=0).stop 2024-02-02T13:31:44.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 msgr2=0x7fa4140c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b3f0 tx=0x7fa40400b8b0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 shutdown_connections 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa3ec06a430 0x7fa3ec06c8f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4140ce130 0x7fa4140c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa41415a010 0x7fa414157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 --2- 172.21.15.91:0/3721357909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140b0700 0x7fa4140c3890 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 >> 172.21.15.91:0/3721357909 conn(0x7fa4140cec80 msgr2=0x7fa41415e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 shutdown_connections 2024-02-02T13:31:44.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.479+0000 7fa41ca00640 1 -- 172.21.15.91:0/3721357909 wait complete. 2024-02-02T13:31:44.499 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:31:44.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.708+0000 7f0a9be00640 1 Processor -- start 2024-02-02T13:31:44.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.708+0000 7f0a9be00640 1 -- start start 2024-02-02T13:31:44.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.710+0000 7f0a9be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a9415b160 0x7f0a9406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.710+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.710+0000 7f0a9be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940cb960 con 0x7f0a940b0af0 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.710+0000 7f0a9be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a94070310 con 0x7f0a9415b160 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.710+0000 7f0a9be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9406fa70 con 0x7f0a94071730 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:55892/0 (socket says 172.21.15.91:55892) 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 -- 172.21.15.91:0/847552162 learned_addr learned my addr 172.21.15.91:0/847552162 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a99400640 1 --2- 172.21.15.91:0/847552162 >> v2:172.21.15.113:3300/0 conn(0x7f0a9415b160 0x7f0a9406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 -- 172.21.15.91:0/847552162 >> v2:172.21.15.113:3300/0 conn(0x7f0a9415b160 msgr2=0x7f0a9406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:44.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 --2- 172.21.15.91:0/847552162 >> v2:172.21.15.113:3300/0 conn(0x7f0a9415b160 0x7f0a9406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 -- 172.21.15.91:0/847552162 >> v1:172.21.15.113:6789/0 conn(0x7f0a940b0af0 legacy=0x7f0a9415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.711+0000 7f0a98a00640 1 -- 172.21.15.91:0/847552162 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9406fd50 con 0x7f0a94071730 2024-02-02T13:31:44.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.713+0000 7f0a98a00640 1 --2- 172.21.15.91:0/847552162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009870 tx=0x7f0a7c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635603b83ac5e953 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.713+0000 7f0a93e00640 1 -- 172.21.15.91:0/847552162 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c031640 con 0x7f0a94071730 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.713+0000 7f0a93e00640 1 -- 172.21.15.91:0/847552162 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c0317a0 con 0x7f0a94071730 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.713+0000 7f0a93e00640 1 -- 172.21.15.91:0/847552162 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c031b30 con 0x7f0a94071730 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 -- 172.21.15.91:0/847552162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 msgr2=0x7f0a94071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 --2- 172.21.15.91:0/847552162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009870 tx=0x7f0a7c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 -- 172.21.15.91:0/847552162 shutdown_connections 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 --2- 172.21.15.91:0/847552162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a94071730 0x7f0a94071b90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 --2- 172.21.15.91:0/847552162 >> v2:172.21.15.113:3300/0 conn(0x7f0a9415b160 0x7f0a9406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 -- 172.21.15.91:0/847552162 >> 172.21.15.91:0/847552162 conn(0x7f0a940cbaf0 msgr2=0x7f0a940cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:44.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 -- 172.21.15.91:0/847552162 shutdown_connections 2024-02-02T13:31:44.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.714+0000 7f0a9be00640 1 -- 172.21.15.91:0/847552162 wait complete. 2024-02-02T13:31:44.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.716+0000 7f0a9be00640 1 Processor -- start 2024-02-02T13:31:44.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.716+0000 7f0a9be00640 1 -- start start 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 0x7f0a940b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0af0 0x7f0a94150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941551d0 con 0x7f0a940b1c90 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940c6570 con 0x7f0a94071730 2024-02-02T13:31:44.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a9be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94155420 con 0x7f0a940b0af0 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a99e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a99e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37704/0 (socket says 172.21.15.91:37704) 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.717+0000 7f0a99e00640 1 -- 172.21.15.91:0/4104861453 learned_addr learned my addr 172.21.15.91:0/4104861453 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99400640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 0x7f0a940b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a98a00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0af0 0x7f0a94150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 -- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0af0 msgr2=0x7f0a94150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0af0 0x7f0a94150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 -- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 msgr2=0x7f0a940b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 0x7f0a940b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 -- 172.21.15.91:0/4104861453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0094d0 con 0x7f0a940b1c90 2024-02-02T13:31:44.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99400640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 0x7f0a940b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.718+0000 7f0a99e00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c7a0 tx=0x7f0a8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.719+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003b70 con 0x7f0a940b1c90 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.719+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c003cf0 con 0x7f0a940b1c90 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.719+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003ea0 con 0x7f0a940b1c90 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.719+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a94155700 con 0x7f0a940b1c90 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.719+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a94155cc0 con 0x7f0a940b1c90 2024-02-02T13:31:44.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.720+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94066cf0 con 0x7f0a940b1c90 2024-02-02T13:31:44.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.721+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a8c00aa90 con 0x7f0a940b1c90 2024-02-02T13:31:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.721+0000 7f0a91600640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.722+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a8c0886e0 con 0x7f0a940b1c90 2024-02-02T13:31:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.722+0000 7f0a99400640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:44.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.722+0000 7f0a99400640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009b90 tx=0x7f0a84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:44.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.726+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a8c053cb0 con 0x7f0a940b1c90 2024-02-02T13:31:44.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.917+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a940cbf70 con 0x7f0a940b1c90 2024-02-02T13:31:44.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.917+0000 7f0a91600640 1 -- 172.21.15.91:0/4104861453 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a8c0572d0 con 0x7f0a940b1c90 2024-02-02T13:31:44.919 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 msgr2=0x7f0a6c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009b90 tx=0x7f0a84008040 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 msgr2=0x7f0a941561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c7a0 tx=0x7f0a8c00cc60 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 shutdown_connections 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0a6c06a430 0x7f0a6c06c8f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a940b0af0 0x7f0a94150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a940b1c90 0x7f0a941561e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 --2- 172.21.15.91:0/4104861453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a94071730 0x7f0a940b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 >> 172.21.15.91:0/4104861453 conn(0x7f0a940cbaf0 msgr2=0x7f0a9415b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 shutdown_connections 2024-02-02T13:31:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:44.920+0000 7f0a9be00640 1 -- 172.21.15.91:0/4104861453 wait complete. 2024-02-02T13:31:44.941 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:31:45.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.144+0000 7fdf6e600640 1 Processor -- start 2024-02-02T13:31:45.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.144+0000 7fdf6e600640 1 -- start start 2024-02-02T13:31:45.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf6e600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf680ce130 0x7fdf680c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf6e600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf680cc7c0 con 0x7fdf680b0700 2024-02-02T13:31:45.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf6e600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf680ce930 con 0x7fdf6815a010 2024-02-02T13:31:45.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf6e600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf680cd640 con 0x7fdf680ce130 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf67e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdf680b0700 0x7fdf6806ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34194/0 (socket says 172.21.15.91:34194) 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf67e00640 1 -- 172.21.15.91:0/3420210455 learned_addr learned my addr 172.21.15.91:0/3420210455 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf5e800640 1 --2- 172.21.15.91:0/3420210455 >> v2:172.21.15.113:3300/0 conn(0x7fdf680ce130 0x7fdf680c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.146+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087730279 0 0) 0x7fdf680cc7c0 con 0x7fdf680b0700 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf48005730 con 0x7fdf680b0700 2024-02-02T13:31:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2482525452 0 0) 0x7fdf48005730 con 0x7fdf680b0700 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 >> v2:172.21.15.113:3300/0 conn(0x7fdf680ce130 msgr2=0x7fdf680c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 --2- 172.21.15.91:0/3420210455 >> v2:172.21.15.113:3300/0 conn(0x7fdf680ce130 0x7fdf680c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 >> v1:172.21.15.113:6789/0 conn(0x7fdf6815a010 legacy=0x7fdf680ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6806c890 con 0x7fdf680b0700 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fdf600019f0 con 0x7fdf680b0700 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 >> v1:172.21.15.91:6789/0 conn(0x7fdf680b0700 legacy=0x7fdf6806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf48005bd0 0x7fdf48005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf480069d0 0x7fdf48006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 0x7fdf4800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680cd640 con 0x7fdf48007390 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800e290 con 0x7fdf48005bd0 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800e450 con 0x7fdf480069d0 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.147+0000 7fdf67e00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 0x7fdf4800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf5e800640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf480069d0 0x7fdf48006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67400640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf48005bd0 0x7fdf48005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67e00640 1 -- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf480069d0 msgr2=0x7fdf48006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67e00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf480069d0 0x7fdf48006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67e00640 1 -- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf48005bd0 msgr2=0x7fdf48005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67e00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf48005bd0 0x7fdf48005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.148+0000 7fdf67e00640 1 -- 172.21.15.91:0/3420210455 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48005730 con 0x7fdf48007390 2024-02-02T13:31:45.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.149+0000 7fdf67e00640 1 -- 172.21.15.91:0/3420210455 reap_dead start 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.149+0000 7fdf67e00640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 0x7fdf4800db90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000a760 tx=0x7fdf48007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac9a194abf9ba44 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.149+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60034630 con 0x7fdf48007390 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.149+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6002f070 con 0x7fdf48007390 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.149+0000 7fdf66a00640 1 -- 172.21.15.91:0/3420210455 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60034a10 con 0x7fdf48007390 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 -- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 msgr2=0x7fdf4800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 0x7fdf4800db90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000a760 tx=0x7fdf48007a00 comp rx=0 tx=0).stop 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 -- 172.21.15.91:0/3420210455 shutdown_connections 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 --2- 172.21.15.91:0/3420210455 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf48007390 0x7fdf4800db90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 -- 172.21.15.91:0/3420210455 >> 172.21.15.91:0/3420210455 conn(0x7fdf680cec80 msgr2=0x7fdf680cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 -- 172.21.15.91:0/3420210455 shutdown_connections 2024-02-02T13:31:45.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.150+0000 7fdf6e600640 1 -- 172.21.15.91:0/3420210455 wait complete. 2024-02-02T13:31:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.151+0000 7fdf6e600640 1 Processor -- start 2024-02-02T13:31:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.152+0000 7fdf6e600640 1 -- start start 2024-02-02T13:31:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.152+0000 7fdf6e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf48007390 0x7fdf68150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf6e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf681572d0 0x7fdf68070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf6e600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf6e600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680b47d0 con 0x7fdf680af5d0 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf6e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680aff90 con 0x7fdf681572d0 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf6e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68150770 con 0x7fdf48007390 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37726/0 (socket says 172.21.15.91:37726) 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 -- 172.21.15.91:0/1102598968 learned_addr learned my addr 172.21.15.91:0/1102598968 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf5e800640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf681572d0 0x7fdf68070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67400640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf48007390 0x7fdf68150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 -- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf48007390 msgr2=0x7fdf68150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf48007390 0x7fdf68150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 -- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf681572d0 msgr2=0x7fdf68070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf681572d0 0x7fdf68070790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.153+0000 7fdf67e00640 1 -- 172.21.15.91:0/1102598968 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf680b3b50 con 0x7fdf680af5d0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf67e00640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fdf60002230 tx=0x7fdf60001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60001930 con 0x7fdf680af5d0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf600345b0 con 0x7fdf680af5d0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60004da0 con 0x7fdf680af5d0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf680b3e30 con 0x7fdf680af5d0 2024-02-02T13:31:45.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.154+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf680b43f0 con 0x7fdf680af5d0 2024-02-02T13:31:45.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.155+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf680b4580 con 0x7fdf680af5d0 2024-02-02T13:31:45.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.156+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdf60031030 con 0x7fdf680af5d0 2024-02-02T13:31:45.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.157+0000 7fdf5fe00640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 0x7fdf4006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.157+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf600a9700 con 0x7fdf680af5d0 2024-02-02T13:31:45.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.157+0000 7fdf67400640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 0x7fdf4006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.158+0000 7fdf67400640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 0x7fdf4006c8f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.161+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdf60074d90 con 0x7fdf680af5d0 2024-02-02T13:31:45.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.352+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf68066cf0 con 0x7fdf680af5d0 2024-02-02T13:31:45.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.353+0000 7fdf5fe00640 1 -- 172.21.15.91:0/1102598968 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf600430a0 con 0x7fdf680af5d0 2024-02-02T13:31:45.355 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.355+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 msgr2=0x7fdf4006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.355+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 0x7fdf4006c8f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf58009040 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.355+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 msgr2=0x7fdf680afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.355+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fdf60002230 tx=0x7fdf60001cb0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 shutdown_connections 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdf4006a430 0x7fdf4006c8f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf680af5d0 0x7fdf680afa50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf681572d0 0x7fdf68070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 --2- 172.21.15.91:0/1102598968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf48007390 0x7fdf68150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 >> 172.21.15.91:0/1102598968 conn(0x7fdf680cec80 msgr2=0x7fdf680b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 shutdown_connections 2024-02-02T13:31:45.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.356+0000 7fdf6e600640 1 -- 172.21.15.91:0/1102598968 wait complete. 2024-02-02T13:31:45.376 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:45.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.589+0000 7f5448400640 1 Processor -- start 2024-02-02T13:31:45.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.589+0000 7f5448400640 1 -- start start 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.590+0000 7f5448400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5440151f90 0x7f544015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.590+0000 7f5448400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5440070f80 con 0x7f544006f4f0 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.590+0000 7f5448400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544006c600 con 0x7f54400b0e10 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.590+0000 7f5448400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5440152370 con 0x7f5440151f90 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f5445000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f544006f4f0 0x7f54400b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34208/0 (socket says 172.21.15.91:34208) 2024-02-02T13:31:45.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f5445000640 1 -- 172.21.15.91:0/653806081 learned_addr learned my addr 172.21.15.91:0/653806081 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:45.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f5445a00640 1 --2- 172.21.15.91:0/653806081 >> v2:172.21.15.113:3300/0 conn(0x7f5440151f90 0x7f544015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960179825 0 0) 0x7f5440070f80 con 0x7f544006f4f0 2024-02-02T13:31:45.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5420003520 con 0x7f544006f4f0 2024-02-02T13:31:45.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223319232 0 0) 0x7f5420003520 con 0x7f544006f4f0 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 >> v2:172.21.15.113:3300/0 conn(0x7f5440151f90 msgr2=0x7f544015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 --2- 172.21.15.91:0/653806081 >> v2:172.21.15.113:3300/0 conn(0x7f5440151f90 0x7f544015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.591+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 >> v1:172.21.15.113:6789/0 conn(0x7f54400b0e10 legacy=0x7f54400b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400cb010 con 0x7f544006f4f0 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5428001670 con 0x7f544006f4f0 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 >> v1:172.21.15.91:6789/0 conn(0x7f544006f4f0 legacy=0x7f54400b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54200039c0 0x7f5420003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f5420004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5420005180 0x7f542000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440152370 con 0x7f54200047c0 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542000c080 con 0x7f54200039c0 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542000c240 con 0x7f5420005180 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f5420004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 -- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5420005180 msgr2=0x7f542000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5420005180 0x7f542000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 -- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54200039c0 msgr2=0x7f5420003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445a00640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54200039c0 0x7f5420003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54200039c0 0x7f5420003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5445000640 1 -- 172.21.15.91:0/653806081 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420003520 con 0x7f54200047c0 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.592+0000 7f5446400640 1 -- 172.21.15.91:0/653806081 reap_dead start 2024-02-02T13:31:45.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.593+0000 7f5445000640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f5420004c40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f542800c0e0 tx=0x7f54200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4376dc6b5327c3ba server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428005ad0 con 0x7f54200047c0 2024-02-02T13:31:45.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428005c30 con 0x7f54200047c0 2024-02-02T13:31:45.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f543fe00640 1 -- 172.21.15.91:0/653806081 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542802ca50 con 0x7f54200047c0 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 -- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 msgr2=0x7f5420004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f5420004c40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f542800c0e0 tx=0x7f54200057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 -- 172.21.15.91:0/653806081 shutdown_connections 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 --2- 172.21.15.91:0/653806081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f5420004c40 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 -- 172.21.15.91:0/653806081 >> 172.21.15.91:0/653806081 conn(0x7f54400cec80 msgr2=0x7f54400cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 -- 172.21.15.91:0/653806081 shutdown_connections 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.594+0000 7f5448400640 1 -- 172.21.15.91:0/653806081 wait complete. 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.596+0000 7f5448400640 1 Processor -- start 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.596+0000 7f5448400640 1 -- start start 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 0x7f54400afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544015d760 0x7f544015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c4bc0 con 0x7f54200047c0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c46d0 con 0x7f544015d760 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5448400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544015e120 con 0x7f54400c85b0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.597+0000 7f5445a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37742/0 (socket says 172.21.15.91:37742) 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 -- 172.21.15.91:0/2243862085 learned_addr learned my addr 172.21.15.91:0/2243862085 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445000640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 0x7f54400afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 -- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 msgr2=0x7f54400afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 0x7f54400afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 -- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544015d760 msgr2=0x7f544015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5446400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544015d760 0x7f544015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544015d760 0x7f544015dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 -- 172.21.15.91:0/2243862085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400c48e0 con 0x7f54200047c0 2024-02-02T13:31:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445000640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 0x7f54400afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.598+0000 7f5445a00640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5430010840 tx=0x7f5430010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.599+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430009630 con 0x7f54200047c0 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.599+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000eb60 con 0x7f54200047c0 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.599+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543001c400 con 0x7f54200047c0 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.599+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544014ea60 con 0x7f54200047c0 2024-02-02T13:31:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.599+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544014f020 con 0x7f54200047c0 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.600+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544014e8c0 con 0x7f54200047c0 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.601+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f543001c560 con 0x7f54200047c0 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.602+0000 7f543d600640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 0x7f541806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.602+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5430088e00 con 0x7f54200047c0 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.602+0000 7f5445000640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 0x7f541806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.603+0000 7f5445000640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 0x7f541806c8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5428034000 tx=0x7f5428009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:45.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.606+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f544014e8c0 con 0x7f54200047c0 2024-02-02T13:31:45.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.830+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f544014e8c0 con 0x7f54200047c0 2024-02-02T13:31:45.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.831+0000 7f543d600640 1 -- 172.21.15.91:0/2243862085 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f544014e8c0 con 0x7f54200047c0 2024-02-02T13:31:45.833 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:45.834 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:45.834 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 msgr2=0x7f541806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 0x7f541806c8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5428034000 tx=0x7f5428009030 comp rx=0 tx=0).stop 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 msgr2=0x7f54400c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5430010840 tx=0x7f5430010d00 comp rx=0 tx=0).stop 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 shutdown_connections 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f541806a430 0x7f541806c8f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54400c85b0 0x7f54400afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544015d760 0x7f544015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 --2- 172.21.15.91:0/2243862085 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54200047c0 0x7f54400c4190 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:45.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 >> 172.21.15.91:0/2243862085 conn(0x7f54400cec80 msgr2=0x7f54400cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:45.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 shutdown_connections 2024-02-02T13:31:45.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:45.834+0000 7f5448400640 1 -- 172.21.15.91:0/2243862085 wait complete. 2024-02-02T13:31:45.855 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:46.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.059+0000 7faa19c00640 1 Processor -- start 2024-02-02T13:31:46.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.059+0000 7faa19c00640 1 -- start start 2024-02-02T13:31:46.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.060+0000 7faa19c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.060+0000 7faa19c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa140ce270 0x7faa140b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.060+0000 7faa19c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140b10a0 con 0x7faa140b0700 2024-02-02T13:31:46.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.060+0000 7faa19c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa140ccdd0 con 0x7faa140ce270 2024-02-02T13:31:46.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.060+0000 7faa19c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa140cc7c0 con 0x7faa14161660 2024-02-02T13:31:46.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.061+0000 7faa13400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa140ce270 0x7faa140b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.061+0000 7faa12a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.061+0000 7faa12a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:55926/0 (socket says 172.21.15.91:55926) 2024-02-02T13:31:46.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.061+0000 7faa12a00640 1 -- 172.21.15.91:0/2420649625 learned_addr learned my addr 172.21.15.91:0/2420649625 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.062+0000 7faa12a00640 1 -- 172.21.15.91:0/2420649625 >> v2:172.21.15.113:3300/0 conn(0x7faa140ce270 msgr2=0x7faa140b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.062+0000 7faa12a00640 1 --2- 172.21.15.91:0/2420649625 >> v2:172.21.15.113:3300/0 conn(0x7faa140ce270 0x7faa140b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.062+0000 7faa12a00640 1 -- 172.21.15.91:0/2420649625 >> v1:172.21.15.113:6789/0 conn(0x7faa140b0700 legacy=0x7faa14165200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.062+0000 7faa12a00640 1 -- 172.21.15.91:0/2420649625 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1406c540 con 0x7faa14161660 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.063+0000 7faa12a00640 1 --2- 172.21.15.91:0/2420649625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa04009870 tx=0x7faa0402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624671bff4938ce7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa11600640 1 -- 172.21.15.91:0/2420649625 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04031640 con 0x7faa14161660 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa11600640 1 -- 172.21.15.91:0/2420649625 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040317a0 con 0x7faa14161660 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa11600640 1 -- 172.21.15.91:0/2420649625 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04031b30 con 0x7faa14161660 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 -- 172.21.15.91:0/2420649625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 msgr2=0x7faa1406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 --2- 172.21.15.91:0/2420649625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa04009870 tx=0x7faa0402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 -- 172.21.15.91:0/2420649625 shutdown_connections 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 --2- 172.21.15.91:0/2420649625 >> v2:172.21.15.113:3300/0 conn(0x7faa140ce270 0x7faa140b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 --2- 172.21.15.91:0/2420649625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa14161660 0x7faa1406c000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.064+0000 7faa19c00640 1 -- 172.21.15.91:0/2420649625 >> 172.21.15.91:0/2420649625 conn(0x7faa140cec80 msgr2=0x7faa140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.065+0000 7faa19c00640 1 -- 172.21.15.91:0/2420649625 shutdown_connections 2024-02-02T13:31:46.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.065+0000 7faa19c00640 1 -- 172.21.15.91:0/2420649625 wait complete. 2024-02-02T13:31:46.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.066+0000 7faa19c00640 1 Processor -- start 2024-02-02T13:31:46.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.066+0000 7faa19c00640 1 -- start start 2024-02-02T13:31:46.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa140b0700 0x7faa140c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 0x7faa140c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140b2a00 con 0x7faa14161660 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140b2ce0 con 0x7faa140ce270 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.067+0000 7faa19c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa140b2fc0 con 0x7faa140b0700 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37746/0 (socket says 172.21.15.91:37746) 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 -- 172.21.15.91:0/3985196398 learned_addr learned my addr 172.21.15.91:0/3985196398 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa12a00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa140b0700 0x7faa140c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa12000640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 0x7faa140c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 -- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa140b0700 msgr2=0x7faa140c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa140b0700 0x7faa140c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 -- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 msgr2=0x7faa140c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 0x7faa140c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 -- 172.21.15.91:0/3985196398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040094d0 con 0x7faa14161660 2024-02-02T13:31:46.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa12000640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 0x7faa140c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.068+0000 7faa13400640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00efc0 tx=0x7faa0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.069+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c009b10 con 0x7faa14161660 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.069+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00f040 con 0x7faa14161660 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.069+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa140b32a0 con 0x7faa14161660 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.069+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0146c0 con 0x7faa14161660 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.069+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa140c6970 con 0x7faa14161660 2024-02-02T13:31:46.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.070+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa140b34c0 con 0x7faa14161660 2024-02-02T13:31:46.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.071+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faa0c014820 con 0x7faa14161660 2024-02-02T13:31:46.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.072+0000 7faa0aa00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 0x7fa9ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.072+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa0c089390 con 0x7faa14161660 2024-02-02T13:31:46.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.072+0000 7faa12a00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 0x7fa9ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.073+0000 7faa12a00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 0x7fa9ec06c8f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faa04036000 tx=0x7faa0403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.077+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faa0c054a10 con 0x7faa14161660 2024-02-02T13:31:46.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.266+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa140b2b40 con 0x7faa14161660 2024-02-02T13:31:46.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.268+0000 7faa0aa00640 1 -- 172.21.15.91:0/3985196398 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7faa140b2b40 con 0x7faa14161660 2024-02-02T13:31:46.270 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:46.270 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:46.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 msgr2=0x7fa9ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 0x7fa9ec06c8f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faa04036000 tx=0x7faa0403c040 comp rx=0 tx=0).stop 2024-02-02T13:31:46.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 msgr2=0x7faa140c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00efc0 tx=0x7faa0c00c490 comp rx=0 tx=0).stop 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 shutdown_connections 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.270+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa9ec06a430 0x7fa9ec06c8f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa14161660 0x7faa140c26f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa140ce270 0x7faa140c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 --2- 172.21.15.91:0/3985196398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa140b0700 0x7faa140c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 >> 172.21.15.91:0/3985196398 conn(0x7faa140cec80 msgr2=0x7faa14160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 shutdown_connections 2024-02-02T13:31:46.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.271+0000 7faa19c00640 1 -- 172.21.15.91:0/3985196398 wait complete. 2024-02-02T13:31:46.291 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:31:46.291 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:46.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.502+0000 7f5797000640 1 Processor -- start 2024-02-02T13:31:46.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.502+0000 7f5797000640 1 -- start start 2024-02-02T13:31:46.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.503+0000 7f5797000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f579015d3a0 0x7f579006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.503+0000 7f5797000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57900add80 0x7f57900706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.503+0000 7f5797000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900ce3b0 con 0x7f57900c9450 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.503+0000 7f5797000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900adbd0 con 0x7f57900add80 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.503+0000 7f5797000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900c59d0 con 0x7f579015d3a0 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57900add80 0x7f57900706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57900add80 0x7f57900706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37758/0 (socket says 172.21.15.91:37758) 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 -- 172.21.15.91:0/3927371629 learned_addr learned my addr 172.21.15.91:0/3927371629 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:46.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795c00640 1 --2- 172.21.15.91:0/3927371629 >> v2:172.21.15.113:3300/0 conn(0x7f579015d3a0 0x7f579006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 -- 172.21.15.91:0/3927371629 >> v2:172.21.15.113:3300/0 conn(0x7f579015d3a0 msgr2=0x7f579006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:46.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 --2- 172.21.15.91:0/3927371629 >> v2:172.21.15.113:3300/0 conn(0x7f579015d3a0 0x7f579006b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 -- 172.21.15.91:0/3927371629 >> v1:172.21.15.91:6789/0 conn(0x7f57900c9450 legacy=0x7f57900c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.504+0000 7f5795200640 1 -- 172.21.15.91:0/3927371629 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900ce8e0 con 0x7f57900add80 2024-02-02T13:31:46.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.505+0000 7f5795200640 1 --2- 172.21.15.91:0/3927371629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900706a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f579006b360 tx=0x7f577802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d17a0f98304a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f578fe00640 1 -- 172.21.15.91:0/3927371629 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57780317e0 con 0x7f57900add80 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f578fe00640 1 -- 172.21.15.91:0/3927371629 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778031940 con 0x7f57900add80 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f578fe00640 1 -- 172.21.15.91:0/3927371629 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778031cd0 con 0x7f57900add80 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 -- 172.21.15.91:0/3927371629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 msgr2=0x7f57900706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 --2- 172.21.15.91:0/3927371629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900706a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f579006b360 tx=0x7f577802b880 comp rx=0 tx=0).stop 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 -- 172.21.15.91:0/3927371629 shutdown_connections 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 --2- 172.21.15.91:0/3927371629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900706a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 --2- 172.21.15.91:0/3927371629 >> v2:172.21.15.113:3300/0 conn(0x7f579015d3a0 0x7f579006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 -- 172.21.15.91:0/3927371629 >> 172.21.15.91:0/3927371629 conn(0x7f57900cea90 msgr2=0x7f57900ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 -- 172.21.15.91:0/3927371629 shutdown_connections 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.506+0000 7f5797000640 1 -- 172.21.15.91:0/3927371629 wait complete. 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.508+0000 7f5797000640 1 Processor -- start 2024-02-02T13:31:46.510 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.508+0000 7f5797000640 1 -- start start 2024-02-02T13:31:46.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 0x7f57900b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f579015d3a0 0x7f57900aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c2be0 con 0x7f57900add80 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900b4060 con 0x7f579015d3a0 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5797000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900aea90 con 0x7f57900c9450 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.509+0000 7f5795c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37770/0 (socket says 172.21.15.91:37770) 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 -- 172.21.15.91:0/2191490627 learned_addr learned my addr 172.21.15.91:0/2191490627 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 -- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 msgr2=0x7f57900b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795200640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 0x7f57900b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5796600640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f579015d3a0 0x7f57900aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 0x7f57900b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 -- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f579015d3a0 msgr2=0x7f57900aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f579015d3a0 0x7f57900aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 -- 172.21.15.91:0/2191490627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780094d0 con 0x7f57900add80 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795200640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 0x7f57900b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.510+0000 7f5795c00640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f578000d5f0 tx=0x7f578000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.511+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800125f0 con 0x7f57900add80 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.511+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780012750 con 0x7f57900add80 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.511+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780012b00 con 0x7f57900add80 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.511+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57900b37c0 con 0x7f57900add80 2024-02-02T13:31:46.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.511+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57900b3d80 con 0x7f57900add80 2024-02-02T13:31:46.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.512+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57900b14f0 con 0x7f57900add80 2024-02-02T13:31:46.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.513+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5780012c60 con 0x7f57900add80 2024-02-02T13:31:46.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.514+0000 7f578d600640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 0x7f576806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.514+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5780088dd0 con 0x7f57900add80 2024-02-02T13:31:46.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.514+0000 7f5795200640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 0x7f576806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.515+0000 7f5795200640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 0x7f576806c8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5778036000 tx=0x7f577803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.519+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5780054450 con 0x7f57900add80 2024-02-02T13:31:46.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.748+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f579004dff0 con 0x7f57900add80 2024-02-02T13:31:46.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.750+0000 7f578d600640 1 -- 172.21.15.91:0/2191490627 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f5780057a70 con 0x7f57900add80 2024-02-02T13:31:46.751 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:46.752 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:46.752 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 msgr2=0x7f576806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 0x7f576806c8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5778036000 tx=0x7f577803d040 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 msgr2=0x7f57900ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f578000d5f0 tx=0x7f578000dab0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 shutdown_connections 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f576806a430 0x7f576806c8f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57900c9450 0x7f57900b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f579015d3a0 0x7f57900aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 --2- 172.21.15.91:0/2191490627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57900add80 0x7f57900ae550 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 >> 172.21.15.91:0/2191490627 conn(0x7f57900cea90 msgr2=0x7f57900ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 shutdown_connections 2024-02-02T13:31:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.752+0000 7f5797000640 1 -- 172.21.15.91:0/2191490627 wait complete. 2024-02-02T13:31:46.773 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:46.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.966+0000 7f56d1a00640 1 Processor -- start 2024-02-02T13:31:46.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.966+0000 7f56d1a00640 1 -- start start 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.967+0000 7f56d1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56cc0add80 0x7f56cc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.967+0000 7f56d1a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0ce3b0 con 0x7f56cc0c9450 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.967+0000 7f56d1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0ce8e0 con 0x7f56cc15d3a0 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.967+0000 7f56d1a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0adbd0 con 0x7f56cc0add80 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56d1000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f56cc0c9450 0x7f56cc0c98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34226/0 (socket says 172.21.15.91:34226) 2024-02-02T13:31:46.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56d1000640 1 -- 172.21.15.91:0/2240374970 learned_addr learned my addr 172.21.15.91:0/2240374970 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56c3200640 1 --2- 172.21.15.91:0/2240374970 >> v2:172.21.15.113:3300/0 conn(0x7f56cc0add80 0x7f56cc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171392988 0 0) 0x7f56cc0ce3b0 con 0x7f56cc0c9450 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ac003520 con 0x7f56cc0c9450 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127376729 0 0) 0x7f56ac003520 con 0x7f56cc0c9450 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 >> v2:172.21.15.113:3300/0 conn(0x7f56cc0add80 msgr2=0x7f56cc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 --2- 172.21.15.91:0/2240374970 >> v2:172.21.15.113:3300/0 conn(0x7f56cc0add80 0x7f56cc0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 >> v1:172.21.15.113:6789/0 conn(0x7f56cc15d3a0 legacy=0x7f56cc06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc15d780 con 0x7f56cc0c9450 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.968+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f56c40019f0 con 0x7f56cc0c9450 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 >> v1:172.21.15.91:6789/0 conn(0x7f56cc0c9450 legacy=0x7f56cc0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac003970 0x7f56ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0047c0 0x7f56ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 0x7f56ac00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0adbd0 con 0x7f56ac0052a0 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac00e380 con 0x7f56ac003970 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac00e540 con 0x7f56ac0047c0 2024-02-02T13:31:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 0x7f56ac00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56cbe00640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac003970 0x7f56ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56c3200640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0047c0 0x7f56ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 -- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0047c0 msgr2=0x7f56ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0047c0 0x7f56ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 -- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac003970 msgr2=0x7f56ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac003970 0x7f56ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56d1000640 1 -- 172.21.15.91:0/2240374970 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac003520 con 0x7f56ac0052a0 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.969+0000 7f56c3200640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0047c0 0x7f56ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.970+0000 7f56d1000640 1 -- 172.21.15.91:0/2240374970 reap_dead start 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1000640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 0x7f56ac00db90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f56c400a800 tx=0x7f56ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb15d8e4c285c39 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40072d0 con 0x7f56ac0052a0 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c402f070 con 0x7f56ac0052a0 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56cb400640 1 -- 172.21.15.91:0/2240374970 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4007650 con 0x7f56ac0052a0 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 -- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 msgr2=0x7f56ac00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 0x7f56ac00db90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f56c400a800 tx=0x7f56ac0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 -- 172.21.15.91:0/2240374970 shutdown_connections 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 --2- 172.21.15.91:0/2240374970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56ac0052a0 0x7f56ac00db90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 -- 172.21.15.91:0/2240374970 >> 172.21.15.91:0/2240374970 conn(0x7f56cc0cea90 msgr2=0x7f56cc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 -- 172.21.15.91:0/2240374970 shutdown_connections 2024-02-02T13:31:46.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.971+0000 7f56d1a00640 1 -- 172.21.15.91:0/2240374970 wait complete. 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.973+0000 7f56d1a00640 1 Processor -- start 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.973+0000 7f56d1a00640 1 -- start start 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 0x7f56cc1f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 0x7f56cc0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ae290 con 0x7f56cc0c7270 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ae570 con 0x7f56cc1f4ec0 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ae850 con 0x7f56ac0052a0 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.974+0000 7f56d1000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37794/0 (socket says 172.21.15.91:37794) 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 -- 172.21.15.91:0/3148347170 learned_addr learned my addr 172.21.15.91:0/3148347170 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56c3200640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 0x7f56cc0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56cbe00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 0x7f56cc1f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 -- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 msgr2=0x7f56cc1f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 0x7f56cc1f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 -- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 msgr2=0x7f56cc0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 0x7f56cc0c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 -- 172.21.15.91:0/3148347170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0aeb30 con 0x7f56cc0c7270 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56c3200640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 0x7f56cc0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56cbe00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 0x7f56cc1f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.975+0000 7f56d1000640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000f80 tx=0x7f56c4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.976+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4042070 con 0x7f56cc0c7270 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.976+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4004ce0 con 0x7f56cc0c7270 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.976+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc0aee10 con 0x7f56cc0c7270 2024-02-02T13:31:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.976+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4001970 con 0x7f56cc0c7270 2024-02-02T13:31:46.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.976+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc1f5340 con 0x7f56cc0c7270 2024-02-02T13:31:46.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.977+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc04dff0 con 0x7f56cc0c7270 2024-02-02T13:31:46.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.978+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f56c4031030 con 0x7f56cc0c7270 2024-02-02T13:31:46.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.979+0000 7f56c8c00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 0x7f56a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:46.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.979+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56c40a9cc0 con 0x7f56cc0c7270 2024-02-02T13:31:46.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.979+0000 7f56cbe00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 0x7f56a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:46.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.980+0000 7f56cbe00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 0x7f56a406c8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:46.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:46.984+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f56c40752b0 con 0x7f56cc0c7270 2024-02-02T13:31:47.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.211+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56cc04dff0 con 0x7f56cc0c7270 2024-02-02T13:31:47.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.212+0000 7f56c8c00640 1 -- 172.21.15.91:0/3148347170 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f56c40788d0 con 0x7f56cc0c7270 2024-02-02T13:31:47.213 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:47.214 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:31:16.893346+0000","modified":"2024-02-02T13:31:20.388634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5578":{"gid":5578,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1833616710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1833616710},{"type":"v1","addr":"172.21.15.91:6837","nonce":1833616710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-02T13:31:47.214 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 56 2024-02-02T13:31:47.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.214+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 msgr2=0x7f56a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.214+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 0x7f56a406c8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc009040 comp rx=0 tx=0).stop 2024-02-02T13:31:47.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.214+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 msgr2=0x7f56cc0c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.214+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000f80 tx=0x7f56c4001d80 comp rx=0 tx=0).stop 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 shutdown_connections 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f56a406a430 0x7f56a406c8f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56cc0c7270 0x7f56cc0c7830 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc1f4ec0 0x7f56cc0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 --2- 172.21.15.91:0/3148347170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56ac0052a0 0x7f56cc1f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 >> 172.21.15.91:0/3148347170 conn(0x7f56cc0cea90 msgr2=0x7f56cc0c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 shutdown_connections 2024-02-02T13:31:47.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.215+0000 7f56d1a00640 1 -- 172.21.15.91:0/3148347170 wait complete. 2024-02-02T13:31:47.234 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:47.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.438+0000 7f5e95000640 1 Processor -- start 2024-02-02T13:31:47.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.439+0000 7f5e95000640 1 -- start start 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5e900d0070 0x7f5e900b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b2260 0x7f5e900d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e900b6260 0x7f5e900b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e900b8f90 con 0x7f5e900d0070 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e900cd930 con 0x7f5e900b6260 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e95000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900d1680 con 0x7f5e900b2260 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e8de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5e900d0070 0x7f5e900b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e8de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5e900d0070 0x7f5e900b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37810/0 (socket says 172.21.15.91:37810) 2024-02-02T13:31:47.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.440+0000 7f5e8de00640 1 -- 172.21.15.91:0/3638410069 learned_addr learned my addr 172.21.15.91:0/3638410069 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.441+0000 7f5e8de00640 1 -- 172.21.15.91:0/3638410069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b2260 msgr2=0x7f5e900d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.441+0000 7f5e8de00640 1 --2- 172.21.15.91:0/3638410069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b2260 0x7f5e900d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.441+0000 7f5e8de00640 1 -- 172.21.15.91:0/3638410069 >> v2:172.21.15.113:3300/0 conn(0x7f5e900b6260 msgr2=0x7f5e900b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.441+0000 7f5e8de00640 1 --2- 172.21.15.91:0/3638410069 >> v2:172.21.15.113:3300/0 conn(0x7f5e900b6260 0x7f5e900b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.441+0000 7f5e8de00640 1 -- 172.21.15.91:0/3638410069 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900d1960 con 0x7f5e900d0070 2024-02-02T13:31:47.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.442+0000 7f5e8de00640 1 --2- 172.21.15.91:0/3638410069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900d0070 0x7f5e900b8a50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5e900d0fc0 tx=0x7f5e8002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3a7a8f8b9f9c08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.442+0000 7f5e8d400640 1 -- 172.21.15.91:0/3638410069 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80031880 con 0x7f5e900d0070 2024-02-02T13:31:47.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.442+0000 7f5e8d400640 1 -- 172.21.15.91:0/3638410069 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800319e0 con 0x7f5e900d0070 2024-02-02T13:31:47.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.442+0000 7f5e8d400640 1 -- 172.21.15.91:0/3638410069 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80031d70 con 0x7f5e900d0070 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 -- 172.21.15.91:0/3638410069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900d0070 msgr2=0x7f5e900b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 --2- 172.21.15.91:0/3638410069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900d0070 0x7f5e900b8a50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5e900d0fc0 tx=0x7f5e8002b920 comp rx=0 tx=0).stop 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 -- 172.21.15.91:0/3638410069 shutdown_connections 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 --2- 172.21.15.91:0/3638410069 >> v2:172.21.15.113:3300/0 conn(0x7f5e900b6260 0x7f5e900b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 --2- 172.21.15.91:0/3638410069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b2260 0x7f5e900d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 --2- 172.21.15.91:0/3638410069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900d0070 0x7f5e900b8a50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 -- 172.21.15.91:0/3638410069 >> 172.21.15.91:0/3638410069 conn(0x7f5e90070660 msgr2=0x7f5e90070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 -- 172.21.15.91:0/3638410069 shutdown_connections 2024-02-02T13:31:47.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.443+0000 7f5e95000640 1 -- 172.21.15.91:0/3638410069 wait complete. 2024-02-02T13:31:47.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.445+0000 7f5e95000640 1 Processor -- start 2024-02-02T13:31:47.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.445+0000 7f5e95000640 1 -- start start 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b6260 0x7f5e901f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e900d0070 0x7f5e9015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9015f1a0 con 0x7f5e900b2260 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90158a50 con 0x7f5e900d0070 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.446+0000 7f5e95000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90158490 con 0x7f5e900b6260 2024-02-02T13:31:47.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37816/0 (socket says 172.21.15.91:37816) 2024-02-02T13:31:47.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 -- 172.21.15.91:0/2427339956 learned_addr learned my addr 172.21.15.91:0/2427339956 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8e800640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e900d0070 0x7f5e9015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e87e00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b6260 0x7f5e901f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 -- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b6260 msgr2=0x7f5e901f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b6260 0x7f5e901f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 -- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e900d0070 msgr2=0x7f5e9015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e900d0070 0x7f5e9015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.447+0000 7f5e8de00640 1 -- 172.21.15.91:0/2427339956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800094d0 con 0x7f5e900b2260 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e8de00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000c00 tx=0x7f5e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80031850 con 0x7f5e900b2260 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800443f0 con 0x7f5e900b2260 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e90158770 con 0x7f5e900b2260 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800447a0 con 0x7f5e900b2260 2024-02-02T13:31:47.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.448+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9015e500 con 0x7f5e900b2260 2024-02-02T13:31:47.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.449+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9015ec20 con 0x7f5e900b2260 2024-02-02T13:31:47.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.450+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5e800303d0 con 0x7f5e900b2260 2024-02-02T13:31:47.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.451+0000 7f5e86000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 0x7f5e6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.451+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e800ab360 con 0x7f5e900b2260 2024-02-02T13:31:47.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.451+0000 7f5e87e00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 0x7f5e6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.452+0000 7f5e87e00640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 0x7f5e6806c8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e900b2230 tx=0x7f5e78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.455+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5e9015ec20 con 0x7f5e900b2260 2024-02-02T13:31:47.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.648+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e9015ec20 con 0x7f5e900b2260 2024-02-02T13:31:47.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.651+0000 7f5e86000640 1 -- 172.21.15.91:0/2427339956 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f5e9015ec20 con 0x7f5e900b2260 2024-02-02T13:31:47.653 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:47.653 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":131,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:35.464196+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:14.703691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:15.895506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 msgr2=0x7f5e6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 0x7f5e6806c8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e900b2230 tx=0x7f5e78008040 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 msgr2=0x7f5e90157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000c00 tx=0x7f5e800046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 shutdown_connections 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e6806a430 0x7f5e6806c8f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e900b6260 0x7f5e901f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e900d0070 0x7f5e9015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 --2- 172.21.15.91:0/2427339956 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e900b2260 0x7f5e90157eb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 >> 172.21.15.91:0/2427339956 conn(0x7f5e90070660 msgr2=0x7f5e900cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 shutdown_connections 2024-02-02T13:31:47.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.654+0000 7f5e95000640 1 -- 172.21.15.91:0/2427339956 wait complete. 2024-02-02T13:31:47.673 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:31:47.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.872+0000 7f6970400640 1 Processor -- start 2024-02-02T13:31:47.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.873+0000 7f6970400640 1 -- start start 2024-02-02T13:31:47.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f6970400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6968159de0 0x7f69680cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f6970400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f696815e340 0x7f696806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f6970400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69680cbf20 con 0x7f69680cbac0 2024-02-02T13:31:47.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f6970400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696815e790 con 0x7f696815e340 2024-02-02T13:31:47.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f6970400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69680ca0c0 con 0x7f6968159de0 2024-02-02T13:31:47.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f696e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f696815e340 0x7f696806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.874+0000 7f696da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6968159de0 0x7f69680cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f696815e340 0x7f696806fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:53596/0 (socket says 172.21.15.91:53596) 2024-02-02T13:31:47.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 -- 172.21.15.91:0/3324056570 learned_addr learned my addr 172.21.15.91:0/3324056570 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:47.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 -- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6968159de0 msgr2=0x7f69680cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:47.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 --2- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6968159de0 0x7f69680cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 -- 172.21.15.91:0/3324056570 >> v1:172.21.15.113:6789/0 conn(0x7f69680cbac0 legacy=0x7f696815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.875+0000 7f696e400640 1 -- 172.21.15.91:0/3324056570 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680ca380 con 0x7f696815e340 2024-02-02T13:31:47.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.877+0000 7f696e400640 1 --2- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f696806fc90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e50 tx=0x7f696002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627c5432170799b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.877+0000 7f6967e00640 1 -- 172.21.15.91:0/3324056570 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696002cb00 con 0x7f696815e340 2024-02-02T13:31:47.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.877+0000 7f6967e00640 1 -- 172.21.15.91:0/3324056570 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696002cc60 con 0x7f696815e340 2024-02-02T13:31:47.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.877+0000 7f6967e00640 1 -- 172.21.15.91:0/3324056570 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600343f0 con 0x7f696815e340 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 -- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 msgr2=0x7f696806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 --2- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f696806fc90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e50 tx=0x7f696002bf60 comp rx=0 tx=0).stop 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 -- 172.21.15.91:0/3324056570 shutdown_connections 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 --2- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f696806fc90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 --2- 172.21.15.91:0/3324056570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6968159de0 0x7f69680cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 -- 172.21.15.91:0/3324056570 >> 172.21.15.91:0/3324056570 conn(0x7f69680b0700 msgr2=0x7f69680b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 -- 172.21.15.91:0/3324056570 shutdown_connections 2024-02-02T13:31:47.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.878+0000 7f6970400640 1 -- 172.21.15.91:0/3324056570 wait complete. 2024-02-02T13:31:47.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.880+0000 7f6970400640 1 Processor -- start 2024-02-02T13:31:47.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.880+0000 7f6970400640 1 -- start start 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69680cbac0 0x7f6968153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f6968144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968155140 con 0x7f6968159de0 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968152900 con 0x7f696815e340 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f6970400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69681544c0 con 0x7f69680cbac0 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f696d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f696d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37822/0 (socket says 172.21.15.91:37822) 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.881+0000 7f696d000640 1 -- 172.21.15.91:0/1030108972 learned_addr learned my addr 172.21.15.91:0/1030108972 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696e400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f6968144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696da00640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69680cbac0 0x7f6968153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 -- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69680cbac0 msgr2=0x7f6968153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69680cbac0 0x7f6968153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 -- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 msgr2=0x7f6968144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f6968144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 -- 172.21.15.91:0/1030108972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960009b30 con 0x7f6968159de0 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696e400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f6968144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:47.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f696d000640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f695000c3c0 tx=0x7f695000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500052d0 con 0x7f6968159de0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950005450 con 0x7f6968159de0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950007260 con 0x7f6968159de0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681547a0 con 0x7f6968159de0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.882+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6968154d60 con 0x7f6968159de0 2024-02-02T13:31:47.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.884+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6968066cf0 con 0x7f6968159de0 2024-02-02T13:31:47.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.885+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f69500073c0 con 0x7f6968159de0 2024-02-02T13:31:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.885+0000 7f6965600640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 0x7f694006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.885+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6950087c00 con 0x7f6968159de0 2024-02-02T13:31:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.886+0000 7f696da00640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 0x7f694006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:47.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.886+0000 7f696da00640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 0x7f694006c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6958009a10 tx=0x7f6958008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:47.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:47.890+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f695008e080 con 0x7f6968159de0 2024-02-02T13:31:48.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.117+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f69680c9a60 con 0x7f6968159de0 2024-02-02T13:31:48.120 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:31:48.119+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:31:48.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.135+0000 7f6965600640 1 -- 172.21.15.91:0/1030108972 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6950053280 con 0x7f6968159de0 2024-02-02T13:31:48.137 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 msgr2=0x7f694006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 0x7f694006c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6958009a10 tx=0x7f6958008040 comp rx=0 tx=0).stop 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 msgr2=0x7f6968155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f695000c3c0 tx=0x7f695000c880 comp rx=0 tx=0).stop 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 shutdown_connections 2024-02-02T13:31:48.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f694006a430 0x7f694006c8f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696815e340 0x7f6968144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6968159de0 0x7f6968155560 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 --2- 172.21.15.91:0/1030108972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69680cbac0 0x7f6968153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.138+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 >> 172.21.15.91:0/1030108972 conn(0x7f69680b0700 msgr2=0x7f696815dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.139+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 shutdown_connections 2024-02-02T13:31:48.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.139+0000 7f6970400640 1 -- 172.21.15.91:0/1030108972 wait complete. 2024-02-02T13:31:48.153 INFO:tasks.ceph.mds.a.smithi091.stderr: -4995> 2024-02-02T13:31:22.207+0000 7fd1e3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:31:48.157 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:31:48.216 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:31:48.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.345+0000 7fd491a00640 1 Processor -- start 2024-02-02T13:31:48.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.345+0000 7fd491a00640 1 -- start start 2024-02-02T13:31:48.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd491a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd48c15a1e0 0x7fd48c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd491a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0cca70 con 0x7fd48c0b0da0 2024-02-02T13:31:48.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd491a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0cc770 con 0x7fd48c070ea0 2024-02-02T13:31:48.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd491a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0ca570 con 0x7fd48c15a1e0 2024-02-02T13:31:48.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd48b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd48c0b0da0 0x7fd48c0ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34242/0 (socket says 172.21.15.91:34242) 2024-02-02T13:31:48.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.346+0000 7fd48b400640 1 -- 172.21.15.91:0/3276036985 learned_addr learned my addr 172.21.15.91:0/3276036985 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd48aa00640 1 --2- 172.21.15.91:0/3276036985 >> v2:172.21.15.113:3300/0 conn(0x7fd48c15a1e0 0x7fd48c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940107436 0 0) 0x7fd48c0cca70 con 0x7fd48c0b0da0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46c005730 con 0x7fd48c0b0da0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417488840 0 0) 0x7fd46c005730 con 0x7fd48c0b0da0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 >> v2:172.21.15.113:3300/0 conn(0x7fd48c15a1e0 msgr2=0x7fd48c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 --2- 172.21.15.91:0/3276036985 >> v2:172.21.15.113:3300/0 conn(0x7fd48c15a1e0 0x7fd48c0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 >> v1:172.21.15.113:6789/0 conn(0x7fd48c070ea0 legacy=0x7fd48c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0ca850 con 0x7fd48c0b0da0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd4840019f0 con 0x7fd48c0b0da0 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd48aa00640 1 --2- 172.21.15.91:0/3276036985 >> v2:172.21.15.113:3300/0 conn(0x7fd48c15a1e0 0x7fd48c0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 >> v1:172.21.15.91:6789/0 conn(0x7fd48c0b0da0 legacy=0x7fd48c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c005b80 0x7fd46c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 0x7fd46c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd46c007390 0x7fd46c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0ca570 con 0x7fd46c0069d0 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00e2e0 con 0x7fd46c005b80 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.347+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00e4a0 con 0x7fd46c007390 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 0x7fd46c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48b400640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd46c007390 0x7fd46c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48aa00640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c005b80 0x7fd46c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 -- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd46c007390 msgr2=0x7fd46c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd46c007390 0x7fd46c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 -- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c005b80 msgr2=0x7fd46c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c005b80 0x7fd46c005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48a000640 1 -- 172.21.15.91:0/3276036985 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c005730 con 0x7fd46c0069d0 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48b400640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd46c007390 0x7fd46c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.348+0000 7fd48b400640 1 -- 172.21.15.91:0/3276036985 reap_dead start 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.349+0000 7fd48a000640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 0x7fd46c006e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd474001750 tx=0x7fd46c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcba862800c238 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.349+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740052d0 con 0x7fd46c0069d0 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.349+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47402e050 con 0x7fd46c0069d0 2024-02-02T13:31:48.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.349+0000 7fd489600640 1 -- 172.21.15.91:0/3276036985 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474003830 con 0x7fd46c0069d0 2024-02-02T13:31:48.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 -- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 msgr2=0x7fd46c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 0x7fd46c006e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd474001750 tx=0x7fd46c007a00 comp rx=0 tx=0).stop 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 -- 172.21.15.91:0/3276036985 shutdown_connections 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 --2- 172.21.15.91:0/3276036985 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd46c0069d0 0x7fd46c006e50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 -- 172.21.15.91:0/3276036985 >> 172.21.15.91:0/3276036985 conn(0x7fd48c0ceb90 msgr2=0x7fd48c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 -- 172.21.15.91:0/3276036985 shutdown_connections 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.350+0000 7fd491a00640 1 -- 172.21.15.91:0/3276036985 wait complete. 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.352+0000 7fd491a00640 1 Processor -- start 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.352+0000 7fd491a00640 1 -- start start 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0069d0 0x7fd48c15c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd48c0cdaa0 0x7fd48c0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0ce460 con 0x7fd48c0b36f0 2024-02-02T13:31:48.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0b2650 con 0x7fd46c0069d0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd491a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0b2930 con 0x7fd48c0cdaa0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37850/0 (socket says 172.21.15.91:37850) 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 -- 172.21.15.91:0/611029928 learned_addr learned my addr 172.21.15.91:0/611029928 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 -- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd48c0cdaa0 msgr2=0x7fd48c0cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd48c0cdaa0 0x7fd48c0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 -- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0069d0 msgr2=0x7fd48c15c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0069d0 0x7fd48c15c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.353+0000 7fd48a000640 1 -- 172.21.15.91:0/611029928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0b2c10 con 0x7fd48c0b36f0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd48a000640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd47402bc00 tx=0x7fd474001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740076c0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474007a30 con 0x7fd48c0b36f0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474006be0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c0b2ef0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.354+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c15c7f0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.355+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c0c23f0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.356+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd474006d40 con 0x7fd48c0b36f0 2024-02-02T13:31:48.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.357+0000 7fd482a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 0x7fd46406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:48.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.357+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd4740aa5c0 con 0x7fd48c0b36f0 2024-02-02T13:31:48.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.357+0000 7fd48aa00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 0x7fd46406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:48.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.358+0000 7fd48aa00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 0x7fd46406c920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006e80 tx=0x7fd47c006df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:48.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.361+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd474075b40 con 0x7fd48c0b36f0 2024-02-02T13:31:48.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:48.591+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd48c066cf0 con 0x7fd48c0b36f0 2024-02-02T13:31:49.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.146+0000 7fd482a00640 1 -- 172.21.15.91:0/611029928 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd474079160 con 0x7fd48c0b36f0 2024-02-02T13:31:49.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 msgr2=0x7fd46406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 0x7fd46406c920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006e80 tx=0x7fd47c006df0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 msgr2=0x7fd48c1584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd47402bc00 tx=0x7fd474001120 comp rx=0 tx=0).stop 2024-02-02T13:31:49.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 shutdown_connections 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd46406a460 0x7fd46406c920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd48c0cdaa0 0x7fd48c0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd48c0b36f0 0x7fd48c1584e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 --2- 172.21.15.91:0/611029928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0069d0 0x7fd48c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 >> 172.21.15.91:0/611029928 conn(0x7fd48c0ceb90 msgr2=0x7fd48c0ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.150+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 shutdown_connections 2024-02-02T13:31:49.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.151+0000 7fd491a00640 1 -- 172.21.15.91:0/611029928 wait complete. 2024-02-02T13:31:49.171 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:31:49.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.380+0000 7fb09b600640 1 Processor -- start 2024-02-02T13:31:49.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.381+0000 7fb09b600640 1 -- start start 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.382+0000 7fb09b600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb094160830 0x7fb0940b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.382+0000 7fb09b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 0x7fb0940711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb09b600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940ce210 con 0x7fb0940b1d70 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb09b600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940ce4f0 con 0x7fb094160830 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb09b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940ce7d0 con 0x7fb094070980 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb094160830 0x7fb0940b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb093e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb0940b1d70 0x7fb094070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:34248/0 (socket says 172.21.15.91:34248) 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb094160830 0x7fb0940b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37858/0 (socket says 172.21.15.91:37858) 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb093e00640 1 -- 172.21.15.91:0/916046248 learned_addr learned my addr 172.21.15.91:0/916046248 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:49.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb099600640 1 --2- 172.21.15.91:0/916046248 >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 0x7fb0940711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 -- 172.21.15.91:0/916046248 >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 msgr2=0x7fb0940711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 --2- 172.21.15.91:0/916046248 >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 0x7fb0940711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 -- 172.21.15.91:0/916046248 >> v1:172.21.15.91:6789/0 conn(0x7fb0940b1d70 legacy=0x7fb094070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb098c00640 1 -- 172.21.15.91:0/916046248 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940d17b0 con 0x7fb094160830 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb099600640 1 --2- 172.21.15.91:0/916046248 >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 0x7fb0940711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:49.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.383+0000 7fb093400640 1 -- 172.21.15.91:0/916046248 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191581704 0 0) 0x7fb0940ce210 con 0x7fb0940b1d70 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.384+0000 7fb098c00640 1 --2- 172.21.15.91:0/916046248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094160830 0x7fb0940b61d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb0940b5b30 tx=0x7fb08402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b887c540d14b33d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb093400640 1 -- 172.21.15.91:0/916046248 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08402e070 con 0x7fb094160830 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb093400640 1 -- 172.21.15.91:0/916046248 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0840315a0 con 0x7fb094160830 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb093400640 1 -- 172.21.15.91:0/916046248 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084031930 con 0x7fb094160830 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 -- 172.21.15.91:0/916046248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094160830 msgr2=0x7fb0940b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 --2- 172.21.15.91:0/916046248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094160830 0x7fb0940b61d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb0940b5b30 tx=0x7fb08402b840 comp rx=0 tx=0).stop 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 -- 172.21.15.91:0/916046248 shutdown_connections 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 --2- 172.21.15.91:0/916046248 >> v2:172.21.15.113:3300/0 conn(0x7fb094070980 0x7fb0940711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 --2- 172.21.15.91:0/916046248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094160830 0x7fb0940b61d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 -- 172.21.15.91:0/916046248 >> 172.21.15.91:0/916046248 conn(0x7fb0940d3030 msgr2=0x7fb0940d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.385+0000 7fb09b600640 1 -- 172.21.15.91:0/916046248 shutdown_connections 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.386+0000 7fb09b600640 1 -- 172.21.15.91:0/916046248 wait complete. 2024-02-02T13:31:49.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.387+0000 7fb09b600640 1 Processor -- start 2024-02-02T13:31:49.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.387+0000 7fb09b600640 1 -- start start 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940b1d70 0x7fb0940cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb094160830 0x7fb09415f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415dea0 con 0x7fb094070980 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415ed00 con 0x7fb0940b1d70 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.388+0000 7fb09b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415e0f0 con 0x7fb094160830 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:37860/0 (socket says 172.21.15.91:37860) 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 -- 172.21.15.91:0/3312970543 learned_addr learned my addr 172.21.15.91:0/3312970543 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 -- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb094160830 msgr2=0x7fb09415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb094160830 0x7fb09415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 -- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940b1d70 msgr2=0x7fb0940cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:49.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940b1d70 0x7fb0940cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:49.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.389+0000 7fb098c00640 1 -- 172.21.15.91:0/3312970543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084009400 con 0x7fb094070980 2024-02-02T13:31:49.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb098c00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb084009890 tx=0x7fb0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:49.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084032840 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0840329a0 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084032d50 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09415e3d0 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.390+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09415e990 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.391+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0940d1a70 con 0x7fb094070980 2024-02-02T13:31:49.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.392+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb0840453f0 con 0x7fb094070980 2024-02-02T13:31:49.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.393+0000 7fb090c00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 0x7fb06c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:49.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.393+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb0840aac50 con 0x7fb094070980 2024-02-02T13:31:49.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.394+0000 7fb093e00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 0x7fb06c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:49.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.394+0000 7fb093e00640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 0x7fb06c06c920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb07c007fd0 tx=0x7fb07c007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:49.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.398+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb0940d1a70 con 0x7fb094070980 2024-02-02T13:31:49.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:49.595+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0940d1a70 con 0x7fb094070980 2024-02-02T13:31:50.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.560+0000 7fb090c00640 1 -- 172.21.15.91:0/3312970543 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0940d1a70 con 0x7fb094070980 2024-02-02T13:31:50.561 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:31:50.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.563+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 msgr2=0x7fb06c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:50.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.563+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 0x7fb06c06c920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb07c007fd0 tx=0x7fb07c007920 comp rx=0 tx=0).stop 2024-02-02T13:31:50.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.563+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 msgr2=0x7fb0940a9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.563+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb084009890 tx=0x7fb0840046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.563+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 shutdown_connections 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb06c06a460 0x7fb06c06c920 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb094160830 0x7fb09415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940b1d70 0x7fb0940cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 --2- 172.21.15.91:0/3312970543 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb094070980 0x7fb0940a9a30 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 >> 172.21.15.91:0/3312970543 conn(0x7fb0940d3030 msgr2=0x7fb0940d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 shutdown_connections 2024-02-02T13:31:50.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.564+0000 7fb09b600640 1 -- 172.21.15.91:0/3312970543 wait complete. 2024-02-02T13:31:50.585 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:31:50.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.791+0000 7fd849800640 1 Processor -- start 2024-02-02T13:31:50.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.792+0000 7fd849800640 1 -- start start 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd849800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd844160830 0x7fd8440b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd849800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8440b1d70 0x7fd8440711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd849800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440ce210 con 0x7fd844070270 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd849800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440ce4f0 con 0x7fd844160830 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd849800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440ce7d0 con 0x7fd8440b1d70 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd842a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd844160830 0x7fd8440b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:50.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd842a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd844160830 0x7fd8440b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34594/0 (socket says 172.21.15.91:34594) 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.793+0000 7fd842a00640 1 -- 172.21.15.91:0/773995064 learned_addr learned my addr 172.21.15.91:0/773995064 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.794+0000 7fd842a00640 1 -- 172.21.15.91:0/773995064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8440b1d70 msgr2=0x7fd8440711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.794+0000 7fd842a00640 1 --2- 172.21.15.91:0/773995064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8440b1d70 0x7fd8440711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.794+0000 7fd842a00640 1 -- 172.21.15.91:0/773995064 >> v1:172.21.15.113:6789/0 conn(0x7fd844070270 legacy=0x7fd844160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.794+0000 7fd842a00640 1 -- 172.21.15.91:0/773995064 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440d17b0 con 0x7fd844160830 2024-02-02T13:31:50.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.795+0000 7fd842a00640 1 --2- 172.21.15.91:0/773995064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440b61d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009e50 tx=0x7fd83c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fde069fb3b8103 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:50.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.795+0000 7fd841600640 1 -- 172.21.15.91:0/773995064 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c02c9f0 con 0x7fd844160830 2024-02-02T13:31:50.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.795+0000 7fd841600640 1 -- 172.21.15.91:0/773995064 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c02cb50 con 0x7fd844160830 2024-02-02T13:31:50.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.795+0000 7fd841600640 1 -- 172.21.15.91:0/773995064 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0343f0 con 0x7fd844160830 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 -- 172.21.15.91:0/773995064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 msgr2=0x7fd8440b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 --2- 172.21.15.91:0/773995064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440b61d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009e50 tx=0x7fd83c02be50 comp rx=0 tx=0).stop 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 -- 172.21.15.91:0/773995064 shutdown_connections 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 --2- 172.21.15.91:0/773995064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8440b1d70 0x7fd8440711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 --2- 172.21.15.91:0/773995064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440b61d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 -- 172.21.15.91:0/773995064 >> 172.21.15.91:0/773995064 conn(0x7fd8440d3030 msgr2=0x7fd8440d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 -- 172.21.15.91:0/773995064 shutdown_connections 2024-02-02T13:31:50.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.796+0000 7fd849800640 1 -- 172.21.15.91:0/773995064 wait complete. 2024-02-02T13:31:50.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.798+0000 7fd849800640 1 Processor -- start 2024-02-02T13:31:50.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.798+0000 7fd849800640 1 -- start start 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd844070270 0x7fd844154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8440b1d70 0x7fd844150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440caf70 con 0x7fd844160830 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd844153150 con 0x7fd8440b1d70 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd849800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440cb1c0 con 0x7fd844070270 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd843400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd843400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34610/0 (socket says 172.21.15.91:34610) 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd843400640 1 -- 172.21.15.91:0/1955579075 learned_addr learned my addr 172.21.15.91:0/1955579075 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd842a00640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd844070270 0x7fd844154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd842000640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8440b1d70 0x7fd844150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.799+0000 7fd843400640 1 -- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd844070270 msgr2=0x7fd844154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd843400640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd844070270 0x7fd844154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd843400640 1 -- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8440b1d70 msgr2=0x7fd844150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd843400640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8440b1d70 0x7fd844150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd843400640 1 -- 172.21.15.91:0/1955579075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009b30 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd843400640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c390 tx=0x7fd83400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340035f0 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834003750 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834003b00 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8440cb4a0 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.800+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440cba60 con 0x7fd844160830 2024-02-02T13:31:50.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.801+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd844066cf0 con 0x7fd844160830 2024-02-02T13:31:50.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.802+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd834003c60 con 0x7fd844160830 2024-02-02T13:31:50.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.803+0000 7fd83aa00640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 0x7fd81c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:50.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.803+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd8340879f0 con 0x7fd844160830 2024-02-02T13:31:50.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.804+0000 7fd842a00640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 0x7fd81c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:50.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.804+0000 7fd842a00640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 0x7fd81c06c8f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd83c035000 tx=0x7fd83c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:50.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:50.808+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd8340530a0 con 0x7fd844160830 2024-02-02T13:31:51.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.001+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8440d3f40 con 0x7fd844160830 2024-02-02T13:31:51.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.571+0000 7fd83aa00640 1 -- 172.21.15.91:0/1955579075 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd8340566c0 con 0x7fd844160830 2024-02-02T13:31:51.572 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:31:51.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.575+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 msgr2=0x7fd81c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:51.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.575+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 0x7fd81c06c8f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd83c035000 tx=0x7fd83c030000 comp rx=0 tx=0).stop 2024-02-02T13:31:51.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.575+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 msgr2=0x7fd8440cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:51.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.575+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c390 tx=0x7fd83400c850 comp rx=0 tx=0).stop 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 shutdown_connections 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd81c06a430 0x7fd81c06c8f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd844160830 0x7fd8440cc260 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8440b1d70 0x7fd844150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 --2- 172.21.15.91:0/1955579075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd844070270 0x7fd844154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 >> 172.21.15.91:0/1955579075 conn(0x7fd8440d3030 msgr2=0x7fd8440d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 shutdown_connections 2024-02-02T13:31:51.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.576+0000 7fd849800640 1 -- 172.21.15.91:0/1955579075 wait complete. 2024-02-02T13:31:51.599 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:31:51.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.808+0000 7f54be800640 1 Processor -- start 2024-02-02T13:31:51.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.808+0000 7f54be800640 1 -- start start 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80b0ee0 0x7f54b8165380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54b80cde70 0x7f54b80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80c9150 0x7f54b80c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80ce860 con 0x7f54b80cde70 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80c9b00 con 0x7f54b80b0ee0 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54be800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b806b630 con 0x7f54b80c9150 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54ae800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54b80cde70 0x7f54b80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54ae800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f54b80cde70 0x7f54b80ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34616/0 (socket says 172.21.15.91:34616) 2024-02-02T13:31:51.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54ae800640 1 -- 172.21.15.91:0/261925643 learned_addr learned my addr 172.21.15.91:0/261925643 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54b7400640 1 --2- 172.21.15.91:0/261925643 >> v2:172.21.15.113:3300/0 conn(0x7f54b80b0ee0 0x7f54b8165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.810+0000 7f54b7e00640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80c9150 0x7f54b80c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.811+0000 7f54ae800640 1 -- 172.21.15.91:0/261925643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80c9150 msgr2=0x7f54b80c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.811+0000 7f54ae800640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80c9150 0x7f54b80c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.811+0000 7f54ae800640 1 -- 172.21.15.91:0/261925643 >> v2:172.21.15.113:3300/0 conn(0x7f54b80b0ee0 msgr2=0x7f54b8165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.811+0000 7f54ae800640 1 --2- 172.21.15.91:0/261925643 >> v2:172.21.15.113:3300/0 conn(0x7f54b80b0ee0 0x7f54b8165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.811+0000 7f54ae800640 1 -- 172.21.15.91:0/261925643 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b8161c30 con 0x7f54b80cde70 2024-02-02T13:31:51.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.812+0000 7f54ae800640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80cde70 0x7f54b80ce320 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f54b80cb570 tx=0x7f54a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332a5eef78d4b783 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:51.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.812+0000 7f54b6a00640 1 -- 172.21.15.91:0/261925643 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0031640 con 0x7f54b80cde70 2024-02-02T13:31:51.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54b6a00640 1 -- 172.21.15.91:0/261925643 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a00317a0 con 0x7f54b80cde70 2024-02-02T13:31:51.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54b6a00640 1 -- 172.21.15.91:0/261925643 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0031b30 con 0x7f54b80cde70 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 -- 172.21.15.91:0/261925643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80cde70 msgr2=0x7f54b80ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80cde70 0x7f54b80ce320 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f54b80cb570 tx=0x7f54a002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 -- 172.21.15.91:0/261925643 shutdown_connections 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80c9150 0x7f54b80c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 --2- 172.21.15.91:0/261925643 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80cde70 0x7f54b80ce320 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 --2- 172.21.15.91:0/261925643 >> v2:172.21.15.113:3300/0 conn(0x7f54b80b0ee0 0x7f54b8165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 -- 172.21.15.91:0/261925643 >> 172.21.15.91:0/261925643 conn(0x7f54b80ceb90 msgr2=0x7f54b80cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:51.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.813+0000 7f54be800640 1 -- 172.21.15.91:0/261925643 shutdown_connections 2024-02-02T13:31:51.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.814+0000 7f54be800640 1 -- 172.21.15.91:0/261925643 wait complete. 2024-02-02T13:31:51.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.815+0000 7f54be800640 1 Processor -- start 2024-02-02T13:31:51.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.815+0000 7f54be800640 1 -- start start 2024-02-02T13:31:51.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80b0ee0 0x7f54b81f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b80cde70 0x7f54b8160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8162f30 con 0x7f54b80c9150 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81623f0 con 0x7f54b80cde70 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.816+0000 7f54be800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8162690 con 0x7f54b80b0ee0 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34626/0 (socket says 172.21.15.91:34626) 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 -- 172.21.15.91:0/2870164115 learned_addr learned my addr 172.21.15.91:0/2870164115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54b7400640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80b0ee0 0x7f54b81f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 -- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80b0ee0 msgr2=0x7f54b81f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54b7e00640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b80cde70 0x7f54b8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80b0ee0 0x7f54b81f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 -- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b80cde70 msgr2=0x7f54b8160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:51.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b80cde70 0x7f54b8160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:51.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 -- 172.21.15.91:0/2870164115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00094d0 con 0x7f54b80c9150 2024-02-02T13:31:51.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.817+0000 7f54ae800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000c00 tx=0x7f54a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:51.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.818+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a003ac60 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.818+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a003adc0 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.818+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0044630 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.818+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b8162970 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.818+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b81f72f0 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.819+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b8066cf0 con 0x7f54b80c9150 2024-02-02T13:31:51.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.820+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f54a002f050 con 0x7f54b80c9150 2024-02-02T13:31:51.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.821+0000 7f54afe00640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 0x7f549006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:51.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.822+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54a00aa530 con 0x7f54b80c9150 2024-02-02T13:31:51.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.822+0000 7f54b7400640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 0x7f549006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:51.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.822+0000 7f54b7400640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 0x7f549006c8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f54a8006fd0 tx=0x7f54a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:51.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:51.826+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f54a0075e20 con 0x7f54b80c9150 2024-02-02T13:31:52.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.016+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f54b806be10 con 0x7f54b80c9150 2024-02-02T13:31:52.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.023+0000 7f54afe00640 1 -- 172.21.15.91:0/2870164115 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f54a0031e60 con 0x7f54b80c9150 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 msgr2=0x7f549006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 0x7f549006c8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f54a8006fd0 tx=0x7f54a8008040 comp rx=0 tx=0).stop 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 msgr2=0x7f54b8161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000c00 tx=0x7f54a00046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 shutdown_connections 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f549006a430 0x7f549006c8f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54b80cde70 0x7f54b8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54b80c9150 0x7f54b8161eb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 --2- 172.21.15.91:0/2870164115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54b80b0ee0 0x7f54b81f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.025+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 >> 172.21.15.91:0/2870164115 conn(0x7f54b80ceb90 msgr2=0x7f54b80cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:52.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.026+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 shutdown_connections 2024-02-02T13:31:52.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.026+0000 7f54be800640 1 -- 172.21.15.91:0/2870164115 wait complete. 2024-02-02T13:31:52.045 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-02-02T13:31:52.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.253+0000 7f4f9ee00640 1 Processor -- start 2024-02-02T13:31:52.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.254+0000 7f4f9ee00640 1 -- start start 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4f98151f90 0x7f4f9815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 0x7f4f980b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980b10b0 con 0x7f4f98151f90 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f98070f80 con 0x7f4f9806f4f0 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f9ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98152370 con 0x7f4f980b0c40 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.255+0000 7f4f97e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4f98151f90 0x7f4f9815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4f98151f90 0x7f4f9815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34634/0 (socket says 172.21.15.91:34634) 2024-02-02T13:31:52.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 -- 172.21.15.91:0/2440574526 learned_addr learned my addr 172.21.15.91:0/2440574526 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 -- 172.21.15.91:0/2440574526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 msgr2=0x7f4f980b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f9ce00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 -- 172.21.15.91:0/2440574526 >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 msgr2=0x7f4f980b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97400640 1 --2- 172.21.15.91:0/2440574526 >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 0x7f4f980b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 --2- 172.21.15.91:0/2440574526 >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 0x7f4f980b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97e00640 1 -- 172.21.15.91:0/2440574526 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980cb010 con 0x7f4f98151f90 2024-02-02T13:31:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.256+0000 7f4f97400640 1 --2- 172.21.15.91:0/2440574526 >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 0x7f4f980b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.257+0000 7f4f97e00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f98151f90 0x7f4f9815f5a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4f98071a00 tx=0x7f4f8802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87747f9cc4fc4f39 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.257+0000 7f4f96a00640 1 -- 172.21.15.91:0/2440574526 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88031860 con 0x7f4f98151f90 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.257+0000 7f4f96a00640 1 -- 172.21.15.91:0/2440574526 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f880319c0 con 0x7f4f98151f90 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.257+0000 7f4f96a00640 1 -- 172.21.15.91:0/2440574526 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88031d50 con 0x7f4f98151f90 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 -- 172.21.15.91:0/2440574526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f98151f90 msgr2=0x7f4f9815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f98151f90 0x7f4f9815f5a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4f98071a00 tx=0x7f4f8802b900 comp rx=0 tx=0).stop 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 -- 172.21.15.91:0/2440574526 shutdown_connections 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/2440574526 >> v2:172.21.15.113:3300/0 conn(0x7f4f9806f4f0 0x7f4f980b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/2440574526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f98151f90 0x7f4f9815f5a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 -- 172.21.15.91:0/2440574526 >> 172.21.15.91:0/2440574526 conn(0x7f4f980cec80 msgr2=0x7f4f980cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 -- 172.21.15.91:0/2440574526 shutdown_connections 2024-02-02T13:31:52.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.258+0000 7f4f9ee00640 1 -- 172.21.15.91:0/2440574526 wait complete. 2024-02-02T13:31:52.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.260+0000 7f4f9ee00640 1 Processor -- start 2024-02-02T13:31:52.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.261+0000 7f4f9ee00640 1 -- start start 2024-02-02T13:31:52.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f98151f90 0x7f4f98150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980b3620 con 0x7f4f9806f4f0 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98150b50 con 0x7f4f98151f90 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f9ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980b38f0 con 0x7f4f980b0c40 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f97e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f97e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34644/0 (socket says 172.21.15.91:34644) 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.262+0000 7f4f97e00640 1 -- 172.21.15.91:0/1653526429 learned_addr learned my addr 172.21.15.91:0/1653526429 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f9ce00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f98151f90 0x7f4f98150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 -- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 msgr2=0x7f4f980b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 -- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f98151f90 msgr2=0x7f4f98150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f98151f90 0x7f4f98150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 -- 172.21.15.91:0/1653526429 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880094d0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f97e00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4f8802b940 tx=0x7f4f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88031850 con 0x7f4f9806f4f0 2024-02-02T13:31:52.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f880443f0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880447a0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.263+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f980c3c80 con 0x7f4f9806f4f0 2024-02-02T13:31:52.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.264+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980c4240 con 0x7f4f9806f4f0 2024-02-02T13:31:52.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.265+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98066cf0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.266+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4f8804d450 con 0x7f4f9806f4f0 2024-02-02T13:31:52.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.267+0000 7f4f8fe00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 0x7f4f7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.267+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4f880aae10 con 0x7f4f9806f4f0 2024-02-02T13:31:52.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.267+0000 7f4f97400640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 0x7f4f7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.268+0000 7f4f97400640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 0x7f4f7006c8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006fd0 tx=0x7f4f80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.271+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4f880ae030 con 0x7f4f9806f4f0 2024-02-02T13:31:52.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.538+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f4f981516b0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.572+0000 7f4f8fe00640 1 -- 172.21.15.91:0/1653526429 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v376) v1 ==== 128+0+0 (secure 0 0 0) 0x7f4f880350c0 con 0x7f4f9806f4f0 2024-02-02T13:31:52.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.574+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 msgr2=0x7f4f7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.574+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 0x7f4f7006c8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006fd0 tx=0x7f4f80008040 comp rx=0 tx=0).stop 2024-02-02T13:31:52.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.574+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 msgr2=0x7f4f980b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.574+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4f8802b940 tx=0x7f4f880046c0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 shutdown_connections 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f7006a430 0x7f4f7006c8f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f980b0c40 0x7f4f980b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f98151f90 0x7f4f98150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 --2- 172.21.15.91:0/1653526429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f9806f4f0 0x7f4f980b4530 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 >> 172.21.15.91:0/1653526429 conn(0x7f4f980cec80 msgr2=0x7f4f980cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 shutdown_connections 2024-02-02T13:31:52.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.575+0000 7f4f9ee00640 1 -- 172.21.15.91:0/1653526429 wait complete. 2024-02-02T13:31:52.593 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-02-02T13:31:52.593 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-02-02T13:31:52.593 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-02-02T13:31:52.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.792+0000 7f4fbc400640 1 Processor -- start 2024-02-02T13:31:52.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.793+0000 7f4fbc400640 1 -- start start 2024-02-02T13:31:52.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fbc400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4fb4159de0 0x7f4fb40cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fbc400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 0x7f4fb406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fbc400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb40ca0c0 con 0x7f4fb40cbac0 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fbc400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb415e790 con 0x7f4fb4159de0 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fbc400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40cbf50 con 0x7f4fb415e340 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fb9000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4fb40cbac0 0x7f4fb415dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53684/0 (socket says 172.21.15.91:53684) 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.794+0000 7f4fb9a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4fb4159de0 0x7f4fb40cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fb9000640 1 -- 172.21.15.91:0/1972840948 learned_addr learned my addr 172.21.15.91:0/1972840948 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fba400640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 0x7f4fb406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fb9a00640 1 -- 172.21.15.91:0/1972840948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 msgr2=0x7f4fb406fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fb9a00640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 0x7f4fb406fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fb9a00640 1 -- 172.21.15.91:0/1972840948 >> v1:172.21.15.91:6789/0 conn(0x7f4fb40cbac0 legacy=0x7f4fb415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fb9a00640 1 -- 172.21.15.91:0/1972840948 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40ca380 con 0x7f4fb4159de0 2024-02-02T13:31:52.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.795+0000 7f4fba400640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 0x7f4fb406fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:52.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.796+0000 7f4fb9a00640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb4159de0 0x7f4fb40cb580 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009dc0 tx=0x7f4fa402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0893632de68d814 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fb3e00640 1 -- 172.21.15.91:0/1972840948 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa402cc40 con 0x7f4fb4159de0 2024-02-02T13:31:52.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fb3e00640 1 -- 172.21.15.91:0/1972840948 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa402cda0 con 0x7f4fb4159de0 2024-02-02T13:31:52.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fb3e00640 1 -- 172.21.15.91:0/1972840948 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4034660 con 0x7f4fb4159de0 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 -- 172.21.15.91:0/1972840948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb4159de0 msgr2=0x7f4fb40cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb4159de0 0x7f4fb40cb580 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009dc0 tx=0x7f4fa402be90 comp rx=0 tx=0).stop 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 -- 172.21.15.91:0/1972840948 shutdown_connections 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb415e340 0x7f4fb406fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1972840948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb4159de0 0x7f4fb40cb580 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 -- 172.21.15.91:0/1972840948 >> 172.21.15.91:0/1972840948 conn(0x7f4fb40b0700 msgr2=0x7f4fb40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 -- 172.21.15.91:0/1972840948 shutdown_connections 2024-02-02T13:31:52.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.797+0000 7f4fbc400640 1 -- 172.21.15.91:0/1972840948 wait complete. 2024-02-02T13:31:52.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.799+0000 7f4fbc400640 1 Processor -- start 2024-02-02T13:31:52.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.799+0000 7f4fbc400640 1 -- start start 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb40cbac0 0x7f4fb414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb4159de0 0x7f4fb40c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40c6b30 con 0x7f4fb415e340 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40c6e10 con 0x7f4fb40cbac0 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fbc400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40c70f0 con 0x7f4fb4159de0 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.800+0000 7f4fba400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34660/0 (socket says 172.21.15.91:34660) 2024-02-02T13:31:52.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 -- 172.21.15.91:0/1378435717 learned_addr learned my addr 172.21.15.91:0/1378435717 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:52.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fb9000640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb4159de0 0x7f4fb40c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fb9a00640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb40cbac0 0x7f4fb414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 -- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb4159de0 msgr2=0x7f4fb40c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb4159de0 0x7f4fb40c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 -- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb40cbac0 msgr2=0x7f4fb414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb40cbac0 0x7f4fb414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 -- 172.21.15.91:0/1378435717 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009a70 con 0x7f4fb415e340 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.801+0000 7f4fba400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c7c0 tx=0x7f4fac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.802+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004230 con 0x7f4fb415e340 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.802+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac004390 con 0x7f4fb415e340 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.802+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac007860 con 0x7f4fb415e340 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.802+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb40c73d0 con 0x7f4fb415e340 2024-02-02T13:31:52.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.802+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb41503b0 con 0x7f4fb415e340 2024-02-02T13:31:52.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.803+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb40c75c0 con 0x7f4fb415e340 2024-02-02T13:31:52.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.804+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4fac0079c0 con 0x7f4fb415e340 2024-02-02T13:31:52.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.805+0000 7f4fb1600640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 0x7f4f8c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:52.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.805+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4fac0887e0 con 0x7f4fb415e340 2024-02-02T13:31:52.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.805+0000 7f4fb9a00640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 0x7f4f8c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:52.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.806+0000 7f4fb9a00640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 0x7f4f8c06c8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4036000 tx=0x7f4fa4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:52.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:52.809+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4fac054220 con 0x7f4fb415e340 2024-02-02T13:31:53.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.068+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f4fb4066cf0 con 0x7f4fb415e340 2024-02-02T13:31:53.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.576+0000 7f4fb1600640 1 -- 172.21.15.91:0/1378435717 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v377) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4fac0150d0 con 0x7f4fb415e340 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.579+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 msgr2=0x7f4f8c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.579+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 0x7f4f8c06c8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4036000 tx=0x7f4fa4030000 comp rx=0 tx=0).stop 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.579+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 msgr2=0x7f4fb40c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.579+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c7c0 tx=0x7f4fac00cc80 comp rx=0 tx=0).stop 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.579+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 shutdown_connections 2024-02-02T13:31:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4f8c06a430 0x7f4f8c06c8f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb4159de0 0x7f4fb40c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4fb415e340 0x7f4fb40c8640 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 --2- 172.21.15.91:0/1378435717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb40cbac0 0x7f4fb414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 >> 172.21.15.91:0/1378435717 conn(0x7f4fb40b0700 msgr2=0x7f4fb40b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 shutdown_connections 2024-02-02T13:31:53.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.580+0000 7f4fbc400640 1 -- 172.21.15.91:0/1378435717 wait complete. 2024-02-02T13:31:53.603 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:31:53.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.811+0000 7fdae2e00640 1 Processor -- start 2024-02-02T13:31:53.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.812+0000 7fdae2e00640 1 -- start start 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae2e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0acfe0 0x7fdadc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae2e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdadc0b0700 0x7fdadc0b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae2e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc15d870 con 0x7fdadc15d490 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae2e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc0b10b0 con 0x7fdadc0b0700 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae2e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc06bce0 con 0x7fdadc0acfe0 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdae0e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdadc0b0700 0x7fdadc0b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdadbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdadc15d490 0x7fdadc0ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53690/0 (socket says 172.21.15.91:53690) 2024-02-02T13:31:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.813+0000 7fdadbe00640 1 -- 172.21.15.91:0/1411924561 learned_addr learned my addr 172.21.15.91:0/1411924561 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdae0e00640 1 -- 172.21.15.91:0/1411924561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0acfe0 msgr2=0x7fdadc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdad3200640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0acfe0 0x7fdadc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdae0e00640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0acfe0 0x7fdadc06c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdae0e00640 1 -- 172.21.15.91:0/1411924561 >> v1:172.21.15.91:6789/0 conn(0x7fdadc15d490 legacy=0x7fdadc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdae0e00640 1 -- 172.21.15.91:0/1411924561 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc161220 con 0x7fdadc0b0700 2024-02-02T13:31:53.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.814+0000 7fdadb400640 1 -- 172.21.15.91:0/1411924561 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706506932 0 0) 0x7fdadc15d870 con 0x7fdadc15d490 2024-02-02T13:31:53.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.815+0000 7fdae0e00640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0b0700 0x7fdadc0b0b70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009d90 tx=0x7fdad402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed4eab5dda75be3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.815+0000 7fdadb400640 1 -- 172.21.15.91:0/1411924561 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad402cc50 con 0x7fdadc0b0700 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.815+0000 7fdadb400640 1 -- 172.21.15.91:0/1411924561 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad402cdb0 con 0x7fdadc0b0700 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.815+0000 7fdadb400640 1 -- 172.21.15.91:0/1411924561 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4034660 con 0x7fdadc0b0700 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 -- 172.21.15.91:0/1411924561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0b0700 msgr2=0x7fdadc0b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0b0700 0x7fdadc0b0b70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009d90 tx=0x7fdad402bea0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 -- 172.21.15.91:0/1411924561 shutdown_connections 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0b0700 0x7fdadc0b0b70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 --2- 172.21.15.91:0/1411924561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0acfe0 0x7fdadc06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 -- 172.21.15.91:0/1411924561 >> 172.21.15.91:0/1411924561 conn(0x7fdadc0cec80 msgr2=0x7fdadc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 -- 172.21.15.91:0/1411924561 shutdown_connections 2024-02-02T13:31:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.816+0000 7fdae2e00640 1 -- 172.21.15.91:0/1411924561 wait complete. 2024-02-02T13:31:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.818+0000 7fdae2e00640 1 Processor -- start 2024-02-02T13:31:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.818+0000 7fdae2e00640 1 -- start start 2024-02-02T13:31:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0b0700 0x7fdadc1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdadc15d490 0x7fdadc0c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc15ec20 con 0x7fdadc0acfe0 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0c5670 con 0x7fdadc15d490 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdae2e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0c8f30 con 0x7fdadc0b0700 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34674/0 (socket says 172.21.15.91:34674) 2024-02-02T13:31:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdadbe00640 1 -- 172.21.15.91:0/2782825088 learned_addr learned my addr 172.21.15.91:0/2782825088 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:53.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdad3200640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0b0700 0x7fdadc1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdadbe00640 1 -- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0b0700 msgr2=0x7fdadc1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.819+0000 7fdadbe00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0b0700 0x7fdadc1f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdadbe00640 1 -- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdadc15d490 msgr2=0x7fdadc0c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdae0e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdadc15d490 0x7fdadc0c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdadbe00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdadc15d490 0x7fdadc0c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdadbe00640 1 -- 172.21.15.91:0/2782825088 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4009a70 con 0x7fdadc0acfe0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdadbe00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c4d0 tx=0x7fdacc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0074e0 con 0x7fdadc0acfe0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc007660 con 0x7fdadc0acfe0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc1f3c50 con 0x7fdadc0acfe0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc004a20 con 0x7fdadc0acfe0 2024-02-02T13:31:53.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.820+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc1f3690 con 0x7fdadc0acfe0 2024-02-02T13:31:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.821+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc0c9d40 con 0x7fdadc0acfe0 2024-02-02T13:31:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.823+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdacc004b80 con 0x7fdadc0acfe0 2024-02-02T13:31:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.824+0000 7fdad8c00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 0x7fdab406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.824+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdacc087b00 con 0x7fdadc0acfe0 2024-02-02T13:31:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.824+0000 7fdad3200640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 0x7fdab406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.825+0000 7fdad3200640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 0x7fdab406c8f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0c8440 tx=0x7fdac4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:53.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:53.828+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdacc053480 con 0x7fdadc0acfe0 2024-02-02T13:31:54.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.017+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdadc066cf0 con 0x7fdadc0acfe0 2024-02-02T13:31:54.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.023+0000 7fdad8c00640 1 -- 172.21.15.91:0/2782825088 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdacc0180c0 con 0x7fdadc0acfe0 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 msgr2=0x7fdab406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 0x7fdab406c8f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0c8440 tx=0x7fdac4008040 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 msgr2=0x7fdadc15c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c4d0 tx=0x7fdacc00c990 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 shutdown_connections 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdab406a430 0x7fdab406c8f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdadc0b0700 0x7fdadc1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdadc15d490 0x7fdadc0c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 --2- 172.21.15.91:0/2782825088 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdadc0acfe0 0x7fdadc15c460 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 >> 172.21.15.91:0/2782825088 conn(0x7fdadc0cec80 msgr2=0x7fdadc0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:54.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 shutdown_connections 2024-02-02T13:31:54.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.026+0000 7fdae2e00640 1 -- 172.21.15.91:0/2782825088 wait complete. 2024-02-02T13:31:54.046 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:31:54.046 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:54.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:31:54.261 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:31:54.261 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:54.261 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:54.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:31:54.318 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:54.319 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:54.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:31:54.375 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:31:54.376 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:54.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:31:54.430 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:31:54.430 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:31:54.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:31:54.487 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:31:54.487 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:31:54.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.702+0000 7f3808a00640 1 Processor -- start 2024-02-02T13:31:54.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.702+0000 7f3808a00640 1 -- start start 2024-02-02T13:31:54.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3808a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f38000ce130 0x7f38000c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3808a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f38000b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3808a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000cc7c0 con 0x7f380015a010 2024-02-02T13:31:54.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3808a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38000b10d0 con 0x7f38000ce130 2024-02-02T13:31:54.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3808a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000ce930 con 0x7f38000b0700 2024-02-02T13:31:54.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3806000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f380015a010 0x7f38000ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53694/0 (socket says 172.21.15.91:53694) 2024-02-02T13:31:54.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3806000640 1 -- 172.21.15.91:0/3151544744 learned_addr learned my addr 172.21.15.91:0/3151544744 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.704+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118677582 0 0) 0x7f38000cc7c0 con 0x7f380015a010 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e0003520 con 0x7f380015a010 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3805600640 1 --2- 172.21.15.91:0/3151544744 >> v2:172.21.15.113:3300/0 conn(0x7f38000ce130 0x7f38000c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3806a00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f38000b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1298797380 0 0) 0x7f37e0003520 con 0x7f380015a010 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 msgr2=0x7f38000b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f38000b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 >> v2:172.21.15.113:3300/0 conn(0x7f38000ce130 msgr2=0x7f38000c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 --2- 172.21.15.91:0/3151544744 >> v2:172.21.15.113:3300/0 conn(0x7f38000ce130 0x7f38000c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000cd640 con 0x7f380015a010 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f37f80019f0 con 0x7f380015a010 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 >> v1:172.21.15.91:6789/0 conn(0x7f380015a010 legacy=0x7f38000ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37e0003970 0x7f37e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f37e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e0004d00 0x7f37e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000b10d0 con 0x7f37e0003970 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000ce930 con 0x7f37e0004d00 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000ba30 con 0x7f38000b0700 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.705+0000 7f3806000640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37e0003970 0x7f37e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806a00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e0004d00 0x7f37e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3805600640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f37e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806000640 1 -- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 msgr2=0x7f37e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806000640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38000b0700 0x7f37e00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806000640 1 -- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e0004d00 msgr2=0x7f37e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806000640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e0004d00 0x7f37e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806000640 1 -- 172.21.15.91:0/3151544744 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0003520 con 0x7f37e0003970 2024-02-02T13:31:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.706+0000 7f3806a00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e0004d00 0x7f37e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.707+0000 7f3806000640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37e0003970 0x7f37e0003d90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f37f800a890 tx=0x7f37e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9719a6334d78cbec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.707+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8034630 con 0x7f37e0003970 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.707+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f802f070 con 0x7f37e0003970 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.707+0000 7f3804c00640 1 -- 172.21.15.91:0/3151544744 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80349f0 con 0x7f37e0003970 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 -- 172.21.15.91:0/3151544744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37e0003970 msgr2=0x7f37e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 --2- 172.21.15.91:0/3151544744 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37e0003970 0x7f37e0003d90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f37f800a890 tx=0x7f37e00057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3806a00640 1 -- 172.21.15.91:0/3151544744 reap_dead start 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 -- 172.21.15.91:0/3151544744 shutdown_connections 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 -- 172.21.15.91:0/3151544744 >> 172.21.15.91:0/3151544744 conn(0x7f38000cec80 msgr2=0x7f38000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 -- 172.21.15.91:0/3151544744 shutdown_connections 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.708+0000 7f3808a00640 1 -- 172.21.15.91:0/3151544744 wait complete. 2024-02-02T13:31:54.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.710+0000 7f3808a00640 1 Processor -- start 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.710+0000 7f3808a00640 1 -- start start 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e0003970 0x7f38001581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 0x7f380006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001f1030 con 0x7f38000c2390 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001f0700 con 0x7f38000c7a70 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3808a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38001f09e0 con 0x7f37e0003970 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34704/0 (socket says 172.21.15.91:34704) 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 -- 172.21.15.91:0/2937980286 learned_addr learned my addr 172.21.15.91:0/2937980286 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3806000640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e0003970 0x7f38001581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3806a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 0x7f380006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 -- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e0003970 msgr2=0x7f38001581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e0003970 0x7f38001581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 -- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 msgr2=0x7f380006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 0x7f380006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3805600640 1 -- 172.21.15.91:0/2937980286 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38001f0cc0 con 0x7f38000c2390 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.711+0000 7f3806a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 0x7f380006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f3805600640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e7e0 tx=0x7f37e800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800c6d0 con 0x7f38000c2390 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e800c830 con 0x7f38000c2390 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800cba0 con 0x7f38000c2390 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38001f03b0 con 0x7f38000c2390 2024-02-02T13:31:54.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.712+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38000c89b0 con 0x7f38000c2390 2024-02-02T13:31:54.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.714+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800066cf0 con 0x7f38000c2390 2024-02-02T13:31:54.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.714+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f37e800cd00 con 0x7f38000c2390 2024-02-02T13:31:54.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.715+0000 7f37fe000640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 0x7f37d806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:54.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.715+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f37e8088280 con 0x7f38000c2390 2024-02-02T13:31:54.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.715+0000 7f3806000640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 0x7f37d806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:54.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.716+0000 7f3806000640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 0x7f37d806c920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f37f8034000 tx=0x7f37f8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:54.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.720+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f37e8053c00 con 0x7f38000c2390 2024-02-02T13:31:54.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.952+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38000b4c00 con 0x7f38000c2390 2024-02-02T13:31:54.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.953+0000 7f37fe000640 1 -- 172.21.15.91:0/2937980286 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f37e8021de0 con 0x7f38000c2390 2024-02-02T13:31:54.955 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:54.955 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/34480687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":34480687},{"type":"v1","addr":"172.21.15.91:6836","nonce":34480687}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4071551826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4071551826},{"type":"v1","addr":"172.21.15.113:6834","nonce":4071551826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3583896286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":3583896286},{"type":"v1","addr":"172.21.15.113:6835","nonce":3583896286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3083922910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3083922910},{"type":"v1","addr":"172.21.15.91:6837","nonce":3083922910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[]} 2024-02-02T13:31:54.955 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 58 2024-02-02T13:31:54.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 msgr2=0x7f37d806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 0x7f37d806c920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f37f8034000 tx=0x7f37f8009030 comp rx=0 tx=0).stop 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 msgr2=0x7f38000c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e7e0 tx=0x7f37e800eca0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 shutdown_connections 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f37d806a460 0x7f37d806c920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38000c7a70 0x7f380006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38000c2390 0x7f38000c7530 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 --2- 172.21.15.91:0/2937980286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e0003970 0x7f38001581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:54.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 >> 172.21.15.91:0/2937980286 conn(0x7f38000cec80 msgr2=0x7f3800073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:54.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.956+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 shutdown_connections 2024-02-02T13:31:54.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:54.957+0000 7f3808a00640 1 -- 172.21.15.91:0/2937980286 wait complete. 2024-02-02T13:31:54.978 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:31:54.978 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:31:54.978 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:31:54.978 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:54.978 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:31:54.979 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:31:54.979 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:31:54.979 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:54.979 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:31:54.979 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:31:54.979 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:31:54.979 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:54.980 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:31:54.980 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:31:54.980 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:31:54.980 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:31:54.999 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:31:54.999 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:31:55.000 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:31:55.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:31:55.002 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:31:55.002 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:31:55.003 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:31:55.004 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:31:55.049 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:31:55.050 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:55.050 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:31:55.050 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:55.055 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:31:55.055 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:55.057 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:31:55.057 DEBUG:teuthology.parallel:result is None 2024-02-02T13:31:55.057 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:31:55.207 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:31:55.208 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:31:55.215 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:31:55.216 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:31:55.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.243+0000 7f8d21a00640 1 Processor -- start 2024-02-02T13:31:55.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.243+0000 7f8d21a00640 1 -- start start 2024-02-02T13:31:55.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 0x7f8d1c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d1c0d1750 0x7f8d1c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c1613d0 0x7f8d1c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0cfa20 con 0x7f8d1c0d1750 2024-02-02T13:31:55.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c161ba0 con 0x7f8d1c15d200 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.244+0000 7f8d21a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0cfc50 con 0x7f8d1c1613d0 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d1c0d1750 0x7f8d1c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d1c0d1750 0x7f8d1c160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34730/0 (socket says 172.21.15.91:34730) 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 -- 172.21.15.91:0/3799016354 learned_addr learned my addr 172.21.15.91:0/3799016354 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d21000640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c1613d0 0x7f8d1c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1be00640 1 --2- 172.21.15.91:0/3799016354 >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 0x7f8d1c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 -- 172.21.15.91:0/3799016354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c1613d0 msgr2=0x7f8d1c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c1613d0 0x7f8d1c069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 -- 172.21.15.91:0/3799016354 >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 msgr2=0x7f8d1c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 --2- 172.21.15.91:0/3799016354 >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 0x7f8d1c0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1b400640 1 -- 172.21.15.91:0/3799016354 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0cff30 con 0x7f8d1c0d1750 2024-02-02T13:31:55.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.245+0000 7f8d1be00640 1 --2- 172.21.15.91:0/3799016354 >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 0x7f8d1c0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:31:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.247+0000 7f8d1b400640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c160e90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0c4730 tx=0x7f8d0402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21062673b1ae0cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.248+0000 7f8d1aa00640 1 -- 172.21.15.91:0/3799016354 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040317e0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.248+0000 7f8d1aa00640 1 -- 172.21.15.91:0/3799016354 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d04031940 con 0x7f8d1c0d1750 2024-02-02T13:31:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.248+0000 7f8d1aa00640 1 -- 172.21.15.91:0/3799016354 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04031cd0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.248+0000 7f8d21a00640 1 -- 172.21.15.91:0/3799016354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 msgr2=0x7f8d1c160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.248+0000 7f8d21a00640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c160e90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0c4730 tx=0x7f8d0402b880 comp rx=0 tx=0).stop 2024-02-02T13:31:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 -- 172.21.15.91:0/3799016354 shutdown_connections 2024-02-02T13:31:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c1613d0 0x7f8d1c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 --2- 172.21.15.91:0/3799016354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c160e90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 --2- 172.21.15.91:0/3799016354 >> v2:172.21.15.113:3300/0 conn(0x7f8d1c15d200 0x7f8d1c0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 -- 172.21.15.91:0/3799016354 >> 172.21.15.91:0/3799016354 conn(0x7f8d1c0d0c30 msgr2=0x7f8d1c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 -- 172.21.15.91:0/3799016354 shutdown_connections 2024-02-02T13:31:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.249+0000 7f8d21a00640 1 -- 172.21.15.91:0/3799016354 wait complete. 2024-02-02T13:31:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.251+0000 7f8d21a00640 1 Processor -- start 2024-02-02T13:31:55.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.251+0000 7f8d21a00640 1 -- start start 2024-02-02T13:31:55.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 0x7f8d1c1f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d1c1613d0 0x7f8d1c1f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0aefd0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0adbd0 con 0x7f8d1c1613d0 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c160050 con 0x7f8d1c15d200 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34746/0 (socket says 172.21.15.91:34746) 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d1be00640 1 -- 172.21.15.91:0/1882724396 learned_addr learned my addr 172.21.15.91:0/1882724396 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d1b400640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 0x7f8d1c1f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.252+0000 7f8d21000640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d1c1613d0 0x7f8d1c1f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 -- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 msgr2=0x7f8d1c1f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 0x7f8d1c1f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 -- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d1c1613d0 msgr2=0x7f8d1c1f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d1c1613d0 0x7f8d1c1f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 -- 172.21.15.91:0/1882724396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0093a0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1b400640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 0x7f8d1c1f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:55.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.253+0000 7f8d1be00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00e9e0 tx=0x7f8d0c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:55.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.256+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0096d0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.256+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d040094d0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.256+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c00ca30 con 0x7f8d1c0d1750 2024-02-02T13:31:55.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.256+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00cdc0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.256+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c0c8230 con 0x7f8d1c0d1750 2024-02-02T13:31:55.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.257+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c04dff0 con 0x7f8d1c0d1750 2024-02-02T13:31:55.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.258+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8d0c005990 con 0x7f8d1c0d1750 2024-02-02T13:31:55.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.260+0000 7f8d13e00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 0x7f8cf406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:55.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.260+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d0c088a10 con 0x7f8d1c0d1750 2024-02-02T13:31:55.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.260+0000 7f8d1b400640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 0x7f8cf406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:55.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.261+0000 7f8d1b400640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 0x7f8cf406c8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8d04036000 tx=0x7f8d0403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:55.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.264+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8d0c054320 con 0x7f8d1c0d1750 2024-02-02T13:31:55.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:55.462+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8d1c06d2f0 con 0x7f8d1c0d1750 2024-02-02T13:31:56.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.266+0000 7f8d13e00640 1 -- 172.21.15.91:0/1882724396 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d0c020080 con 0x7f8d1c0d1750 2024-02-02T13:31:56.268 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 msgr2=0x7f8cf406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 0x7f8cf406c8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8d04036000 tx=0x7f8d0403d040 comp rx=0 tx=0).stop 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 msgr2=0x7f8d1c150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00e9e0 tx=0x7f8d0c00eea0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 shutdown_connections 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8cf406a430 0x7f8cf406c8f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1c15d200 0x7f8d1c1f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d1c1613d0 0x7f8d1c1f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 --2- 172.21.15.91:0/1882724396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d1c0d1750 0x7f8d1c150e30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.269+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 >> 172.21.15.91:0/1882724396 conn(0x7f8d1c0d0c30 msgr2=0x7f8d1c0cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:56.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.270+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 shutdown_connections 2024-02-02T13:31:56.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.270+0000 7f8d21a00640 1 -- 172.21.15.91:0/1882724396 wait complete. 2024-02-02T13:31:56.289 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:31:56.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.487+0000 7f32f6200640 1 Processor -- start 2024-02-02T13:31:56.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.487+0000 7f32f6200640 1 -- start start 2024-02-02T13:31:56.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.488+0000 7f32f6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f015aef0 0x7f32f00b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.488+0000 7f32f6200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f32f00adf20 0x7f32f006b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.488+0000 7f32f6200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f00b0540 con 0x7f32f006ce50 2024-02-02T13:31:56.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f6200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f015b2d0 con 0x7f32f00adf20 2024-02-02T13:31:56.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f6200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00ca410 con 0x7f32f015aef0 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f32f00adf20 0x7f32f006b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f32f00adf20 0x7f32f006b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34758/0 (socket says 172.21.15.91:34758) 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 -- 172.21.15.91:0/753940204 learned_addr learned my addr 172.21.15.91:0/753940204 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f4e00640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f015aef0 0x7f32f00b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 -- 172.21.15.91:0/753940204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f015aef0 msgr2=0x7f32f00b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f015aef0 0x7f32f00b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 -- 172.21.15.91:0/753940204 >> v1:172.21.15.113:6789/0 conn(0x7f32f006ce50 legacy=0x7f32f0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:56.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.489+0000 7f32f5800640 1 -- 172.21.15.91:0/753940204 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f00b07e0 con 0x7f32f00adf20 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.490+0000 7f32f5800640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f00adf20 0x7f32f006b400 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009e50 tx=0x7f32e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed0964005308c89 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32efe00640 1 -- 172.21.15.91:0/753940204 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e802ca70 con 0x7f32f00adf20 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32efe00640 1 -- 172.21.15.91:0/753940204 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e802cbd0 con 0x7f32f00adf20 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32efe00640 1 -- 172.21.15.91:0/753940204 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e80343f0 con 0x7f32f00adf20 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 -- 172.21.15.91:0/753940204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f00adf20 msgr2=0x7f32f006b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f00adf20 0x7f32f006b400 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009e50 tx=0x7f32e802bed0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 -- 172.21.15.91:0/753940204 shutdown_connections 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f00adf20 0x7f32f006b400 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 --2- 172.21.15.91:0/753940204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f015aef0 0x7f32f00b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 -- 172.21.15.91:0/753940204 >> 172.21.15.91:0/753940204 conn(0x7f32f00cb4a0 msgr2=0x7f32f00cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 -- 172.21.15.91:0/753940204 shutdown_connections 2024-02-02T13:31:56.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.491+0000 7f32f6200640 1 -- 172.21.15.91:0/753940204 wait complete. 2024-02-02T13:31:56.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.493+0000 7f32f6200640 1 Processor -- start 2024-02-02T13:31:56.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.493+0000 7f32f6200640 1 -- start start 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f00adf20 0x7f32f015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32f015aef0 0x7f32f0070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f015b950 con 0x7f32f006ce50 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f014e900 con 0x7f32f015aef0 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f6200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f015e230 con 0x7f32f00adf20 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34764/0 (socket says 172.21.15.91:34764) 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.494+0000 7f32f4e00640 1 -- 172.21.15.91:0/484423245 learned_addr learned my addr 172.21.15.91:0/484423245 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:56.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 -- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f00adf20 msgr2=0x7f32f015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f00adf20 0x7f32f015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 -- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32f015aef0 msgr2=0x7f32f0070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32f015aef0 0x7f32f0070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 -- 172.21.15.91:0/484423245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009b30 con 0x7f32f006ce50 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f4e00640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c7f0 tx=0x7f32e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0004220 con 0x7f32f006ce50 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e0004380 con 0x7f32f006ce50 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f014eb10 con 0x7f32f006ce50 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0007840 con 0x7f32f006ce50 2024-02-02T13:31:56.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.495+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f014f0d0 con 0x7f32f006ce50 2024-02-02T13:31:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.496+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f004dff0 con 0x7f32f006ce50 2024-02-02T13:31:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.497+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f32e0007bb0 con 0x7f32f006ce50 2024-02-02T13:31:56.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.498+0000 7f32ed600640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 0x7f32c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:56.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.498+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f32e00885a0 con 0x7f32f006ce50 2024-02-02T13:31:56.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.498+0000 7f32e7c00640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 0x7f32c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:56.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.499+0000 7f32e7c00640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 0x7f32c806c8f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f32f0159600 tx=0x7f32d80079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:56.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.502+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f32e0053e40 con 0x7f32f006ce50 2024-02-02T13:31:56.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.766+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f32f006fce0 con 0x7f32f006ce50 2024-02-02T13:31:56.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.767+0000 7f32ed600640 1 -- 172.21.15.91:0/484423245 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f32e0007e60 con 0x7f32f006ce50 2024-02-02T13:31:56.769 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:56.769 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:31:56.769 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:31:56.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:31:56.773 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.775 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:31:56.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.777 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:56.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:31:56.780 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:31:56.781 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 msgr2=0x7f32c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 0x7f32c806c8f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f32f0159600 tx=0x7f32d80079b0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 msgr2=0x7f32f015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c7f0 tx=0x7f32e000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 shutdown_connections 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f32c806a430 0x7f32c806c8f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32f00adf20 0x7f32f015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32f015aef0 0x7f32f0070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 --2- 172.21.15.91:0/484423245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32f006ce50 0x7f32f015b410 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 >> 172.21.15.91:0/484423245 conn(0x7f32f00cb4a0 msgr2=0x7f32f00c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.770+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 shutdown_connections 2024-02-02T13:31:56.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:56.771+0000 7f32f6200640 1 -- 172.21.15.91:0/484423245 wait complete. 2024-02-02T13:31:56.791 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:31:57.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.004+0000 7f1450a00640 1 Processor -- start 2024-02-02T13:31:57.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.004+0000 7f1450a00640 1 -- start start 2024-02-02T13:31:57.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f1450a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f144c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f1450a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f144c0b6990 0x7f144c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f1450a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c0b7d40 con 0x7f144c15cee0 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f1450a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c0b81d0 con 0x7f144c0b6990 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f1450a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0b7130 con 0x7f144c0ce1d0 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f144a800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f144c15cee0 0x7f144c0cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53724/0 (socket says 172.21.15.91:53724) 2024-02-02T13:31:57.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.006+0000 7f144a800640 1 -- 172.21.15.91:0/3564557721 learned_addr learned my addr 172.21.15.91:0/3564557721 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534648116 0 0) 0x7f144c0b7d40 con 0x7f144c15cee0 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c003520 con 0x7f144c15cee0 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f144b200640 1 --2- 172.21.15.91:0/3564557721 >> v2:172.21.15.113:3300/0 conn(0x7f144c0b6990 0x7f144c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1441c00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f144c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768677001 0 0) 0x7f142c003520 con 0x7f144c15cee0 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 msgr2=0x7f144c0b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f144c0b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 >> v2:172.21.15.113:3300/0 conn(0x7f144c0b6990 msgr2=0x7f144c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 --2- 172.21.15.91:0/3564557721 >> v2:172.21.15.113:3300/0 conn(0x7f144c0b6990 0x7f144c0d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0b7410 con 0x7f144c15cee0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f143c0019f0 con 0x7f144c15cee0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 >> v1:172.21.15.91:6789/0 conn(0x7f144c15cee0 legacy=0x7f144c0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f142c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c003ff0 0x7f142c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142c004d00 0x7f142c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0b81d0 con 0x7f142c004d00 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0b7130 con 0x7f142c003ff0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.007+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c00bae0 con 0x7f144c0ce1d0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142c004d00 0x7f142c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f1441c00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c003ff0 0x7f142c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144a800640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f142c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 -- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 msgr2=0x7f142c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0ce1d0 0x7f142c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 -- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c003ff0 msgr2=0x7f142c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c003ff0 0x7f142c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.008+0000 7f144b200640 1 -- 172.21.15.91:0/3564557721 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c003520 con 0x7f142c004d00 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.009+0000 7f144b200640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142c004d00 0x7f142c005180 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f144400ba30 tx=0x7f142c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e631ba11fca126b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.009+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444032680 con 0x7f142c004d00 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.009+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440327e0 con 0x7f142c004d00 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.009+0000 7f1449e00640 1 -- 172.21.15.91:0/3564557721 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444032b70 con 0x7f142c004d00 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 -- 172.21.15.91:0/3564557721 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142c004d00 msgr2=0x7f142c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 --2- 172.21.15.91:0/3564557721 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f142c004d00 0x7f142c005180 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f144400ba30 tx=0x7f142c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f144b200640 1 -- 172.21.15.91:0/3564557721 reap_dead start 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 -- 172.21.15.91:0/3564557721 shutdown_connections 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 -- 172.21.15.91:0/3564557721 >> 172.21.15.91:0/3564557721 conn(0x7f144c070690 msgr2=0x7f144c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 -- 172.21.15.91:0/3564557721 shutdown_connections 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.010+0000 7f1450a00640 1 -- 172.21.15.91:0/3564557721 wait complete. 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.012+0000 7f1450a00640 1 Processor -- start 2024-02-02T13:31:57.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.012+0000 7f1450a00640 1 -- start start 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c004d00 0x7f144c1f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 0x7f144c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0cbea0 con 0x7f144c14b1a0 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c1f3030 con 0x7f142c004d00 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1450a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0b2850 con 0x7f144c0cb4b0 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1441c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1441c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34782/0 (socket says 172.21.15.91:34782) 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f1441c00640 1 -- 172.21.15.91:0/1605861887 learned_addr learned my addr 172.21.15.91:0/1605861887 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.013+0000 7f144a800640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c004d00 0x7f144c1f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f144b200640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 0x7f144c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 -- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 msgr2=0x7f144c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 0x7f144c0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 -- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c004d00 msgr2=0x7f144c1f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c004d00 0x7f144c1f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 -- 172.21.15.91:0/1605861887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c1f5a90 con 0x7f144c14b1a0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f144b200640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 0x7f144c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1441c00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f143400e810 tx=0x7f143400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400c6d0 con 0x7f144c14b1a0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143400c830 con 0x7f144c14b1a0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c1f5d70 con 0x7f144c14b1a0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400cbe0 con 0x7f144c14b1a0 2024-02-02T13:31:57.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.014+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c1f6330 con 0x7f144c14b1a0 2024-02-02T13:31:57.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.016+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c1f6550 con 0x7f144c14b1a0 2024-02-02T13:31:57.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.016+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f143400cd40 con 0x7f144c14b1a0 2024-02-02T13:31:57.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.017+0000 7f1443400640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 0x7f142406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.017+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1434088420 con 0x7f144c14b1a0 2024-02-02T13:31:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.018+0000 7f144a800640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 0x7f142406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.018+0000 7f144a800640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 0x7f142406c920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f143c001370 tx=0x7f143c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.021+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1434053cc0 con 0x7f144c14b1a0 2024-02-02T13:31:57.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.211+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f144c04dff0 con 0x7f144c14b1a0 2024-02-02T13:31:57.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.213+0000 7f1443400640 1 -- 172.21.15.91:0/1605861887 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f1434009380 con 0x7f144c14b1a0 2024-02-02T13:31:57.215 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:57.215 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":139,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:31:56.258284+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 msgr2=0x7f142406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 0x7f142406c920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f143c001370 tx=0x7f143c009040 comp rx=0 tx=0).stop 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 msgr2=0x7f144c0b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f143400e810 tx=0x7f143400ecd0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 shutdown_connections 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f142406a460 0x7f142406c920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144c0cb4b0 0x7f144c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f144c14b1a0 0x7f144c0b3b70 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 --2- 172.21.15.91:0/1605861887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c004d00 0x7f144c1f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 >> 172.21.15.91:0/1605861887 conn(0x7f144c070690 msgr2=0x7f144c0c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:57.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 shutdown_connections 2024-02-02T13:31:57.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.216+0000 7f1450a00640 1 -- 172.21.15.91:0/1605861887 wait complete. 2024-02-02T13:31:57.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:31:57.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:31:57.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.437+0000 7f2289e00640 1 Processor -- start 2024-02-02T13:31:57.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.438+0000 7f2289e00640 1 -- start start 2024-02-02T13:31:57.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.439+0000 7f2289e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f228415f5a0 0x7f22840b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.439+0000 7f2289e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840b1070 con 0x7f22840b0c40 2024-02-02T13:31:57.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.439+0000 7f2289e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228406c600 con 0x7f2284151f90 2024-02-02T13:31:57.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.439+0000 7f2289e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228415fa30 con 0x7f228415f5a0 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.439+0000 7f2282000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f228415f5a0 0x7f22840b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2283400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f22840b0c40 0x7f22840b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53732/0 (socket says 172.21.15.91:53732) 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2282000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f228415f5a0 0x7f22840b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34798/0 (socket says 172.21.15.91:34798) 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2283400640 1 -- 172.21.15.91:0/2026227652 learned_addr learned my addr 172.21.15.91:0/2026227652 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2282000640 1 -- 172.21.15.91:0/2026227652 >> v1:172.21.15.113:6789/0 conn(0x7f2284151f90 legacy=0x7f228406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2282000640 1 -- 172.21.15.91:0/2026227652 >> v1:172.21.15.91:6789/0 conn(0x7f22840b0c40 legacy=0x7f22840b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.440+0000 7f2282000640 1 -- 172.21.15.91:0/2026227652 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284070f80 con 0x7f228415f5a0 2024-02-02T13:31:57.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.441+0000 7f2282000640 1 --2- 172.21.15.91:0/2026227652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f22840b0700 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f226c0097a0 tx=0x7f226c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07f300aba5c987a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.442+0000 7f2281600640 1 -- 172.21.15.91:0/2026227652 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c031830 con 0x7f228415f5a0 2024-02-02T13:31:57.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.442+0000 7f2281600640 1 -- 172.21.15.91:0/2026227652 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c031990 con 0x7f228415f5a0 2024-02-02T13:31:57.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.442+0000 7f2281600640 1 -- 172.21.15.91:0/2026227652 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c031d20 con 0x7f228415f5a0 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 -- 172.21.15.91:0/2026227652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 msgr2=0x7f22840b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 --2- 172.21.15.91:0/2026227652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f22840b0700 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f226c0097a0 tx=0x7f226c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 -- 172.21.15.91:0/2026227652 shutdown_connections 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 --2- 172.21.15.91:0/2026227652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f22840b0700 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 -- 172.21.15.91:0/2026227652 >> 172.21.15.91:0/2026227652 conn(0x7f22840cec80 msgr2=0x7f22840cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 -- 172.21.15.91:0/2026227652 shutdown_connections 2024-02-02T13:31:57.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.443+0000 7f2289e00640 1 -- 172.21.15.91:0/2026227652 wait complete. 2024-02-02T13:31:57.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.444+0000 7f2289e00640 1 Processor -- start 2024-02-02T13:31:57.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 -- start start 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22840b0c40 0x7f2284150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 0x7f22841503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840c6b20 con 0x7f228415f5a0 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840c6280 con 0x7f2284151f90 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.445+0000 7f2289e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840c6560 con 0x7f22840b0c40 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34806/0 (socket says 172.21.15.91:34806) 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 -- 172.21.15.91:0/3572868590 learned_addr learned my addr 172.21.15.91:0/3572868590 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2282000640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 0x7f22841503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2282a00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22840b0c40 0x7f2284150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 -- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22840b0c40 msgr2=0x7f2284150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22840b0c40 0x7f2284150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 -- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 msgr2=0x7f22841503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 0x7f22841503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2283400640 1 -- 172.21.15.91:0/3572868590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c009400 con 0x7f228415f5a0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.446+0000 7f2282000640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 0x7f22841503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f2283400640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f227c00caa0 tx=0x7f227c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ab40 con 0x7f228415f5a0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00aca0 con 0x7f228415f5a0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22840c6840 con 0x7f228415f5a0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012600 con 0x7f228415f5a0 2024-02-02T13:31:57.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.447+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2284151170 con 0x7f228415f5a0 2024-02-02T13:31:57.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.448+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22840c40e0 con 0x7f228415f5a0 2024-02-02T13:31:57.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.449+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f227c007070 con 0x7f228415f5a0 2024-02-02T13:31:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.450+0000 7f227b400640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 0x7f225c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.450+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f227c0882a0 con 0x7f228415f5a0 2024-02-02T13:31:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.450+0000 7f2282a00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 0x7f225c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.451+0000 7f2282a00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 0x7f225c06c8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.454+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22840c40e0 con 0x7f228415f5a0 2024-02-02T13:31:57.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.649+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22840c40e0 con 0x7f228415f5a0 2024-02-02T13:31:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.650+0000 7f227b400640 1 -- 172.21.15.91:0/3572868590 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f227c017020 con 0x7f228415f5a0 2024-02-02T13:31:57.651 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 msgr2=0x7f225c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 0x7f225c06c8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274008040 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 msgr2=0x7f2284153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f227c00caa0 tx=0x7f227c00cf60 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 shutdown_connections 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f225c06a430 0x7f225c06c8f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f228415f5a0 0x7f2284153020 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2284151f90 0x7f22841503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 --2- 172.21.15.91:0/3572868590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22840b0c40 0x7f2284150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 >> 172.21.15.91:0/3572868590 conn(0x7f22840cec80 msgr2=0x7f22840c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:57.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 shutdown_connections 2024-02-02T13:31:57.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.652+0000 7f2289e00640 1 -- 172.21.15.91:0/3572868590 wait complete. 2024-02-02T13:31:57.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:31:57.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.873+0000 7fdbee800640 1 Processor -- start 2024-02-02T13:31:57.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.873+0000 7fdbee800640 1 -- start start 2024-02-02T13:31:57.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.874+0000 7fdbee800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbe815ad10 0x7fdbe806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.874+0000 7fdbee800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.874+0000 7fdbee800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe806d910 con 0x7fdbe815b0f0 2024-02-02T13:31:57.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.874+0000 7fdbee800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe806d730 con 0x7fdbe815ad10 2024-02-02T13:31:57.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.874+0000 7fdbee800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80cbf50 con 0x7fdbe80cb580 2024-02-02T13:31:57.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.875+0000 7fdbe7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbe815ad10 0x7fdbe806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.875+0000 7fdbe7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbe815ad10 0x7fdbe806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:44296/0 (socket says 172.21.15.91:44296) 2024-02-02T13:31:57.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.875+0000 7fdbe7400640 1 -- 172.21.15.91:0/2739305265 learned_addr learned my addr 172.21.15.91:0/2739305265 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:57.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.875+0000 7fdbe7e00640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.876+0000 7fdbe7400640 1 -- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 msgr2=0x7fdbe80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:57.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.876+0000 7fdbe7400640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.876+0000 7fdbe7400640 1 -- 172.21.15.91:0/2739305265 >> v1:172.21.15.113:6789/0 conn(0x7fdbe815b0f0 legacy=0x7fdbe806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.876+0000 7fdbe7400640 1 -- 172.21.15.91:0/2739305265 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80cc850 con 0x7fdbe815ad10 2024-02-02T13:31:57.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.877+0000 7fdbe7400640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815ad10 0x7fdbe806b7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098a0 tx=0x7fdbd802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7687e3454c96046b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.878+0000 7fdbe6a00640 1 -- 172.21.15.91:0/2739305265 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8031640 con 0x7fdbe815ad10 2024-02-02T13:31:57.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.878+0000 7fdbe6a00640 1 -- 172.21.15.91:0/2739305265 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd80317a0 con 0x7fdbe815ad10 2024-02-02T13:31:57.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.878+0000 7fdbe6a00640 1 -- 172.21.15.91:0/2739305265 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8031b30 con 0x7fdbe815ad10 2024-02-02T13:31:57.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.878+0000 7fdbee800640 1 -- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815ad10 msgr2=0x7fdbe806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.878+0000 7fdbee800640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815ad10 0x7fdbe806b7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098a0 tx=0x7fdbd802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 -- 172.21.15.91:0/2739305265 shutdown_connections 2024-02-02T13:31:57.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 --2- 172.21.15.91:0/2739305265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815ad10 0x7fdbe806b7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 -- 172.21.15.91:0/2739305265 >> 172.21.15.91:0/2739305265 conn(0x7fdbe80b0700 msgr2=0x7fdbe80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:57.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 -- 172.21.15.91:0/2739305265 shutdown_connections 2024-02-02T13:31:57.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.879+0000 7fdbee800640 1 -- 172.21.15.91:0/2739305265 wait complete. 2024-02-02T13:31:57.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.881+0000 7fdbee800640 1 Processor -- start 2024-02-02T13:31:57.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.881+0000 7fdbee800640 1 -- start start 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe813a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 0x7fdbe81f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe813b040 con 0x7fdbe815ad10 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe813b320 con 0x7fdbe815b0f0 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbee800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe813b600 con 0x7fdbe80cb580 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbde800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbde800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34818/0 (socket says 172.21.15.91:34818) 2024-02-02T13:31:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.882+0000 7fdbde800640 1 -- 172.21.15.91:0/3950651012 learned_addr learned my addr 172.21.15.91:0/3950651012 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbe7e00640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 0x7fdbe81f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbe7400640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe813a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 -- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 msgr2=0x7fdbe813a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe813a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 -- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 msgr2=0x7fdbe81f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 0x7fdbe81f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 -- 172.21.15.91:0/3950651012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80094d0 con 0x7fdbe815ad10 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbe7e00640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 0x7fdbe81f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbde800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c400 tx=0x7fdbd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00052d0 con 0x7fdbe815ad10 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd0005430 con 0x7fdbe815ad10 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003830 con 0x7fdbe815ad10 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.883+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe813b8e0 con 0x7fdbe815ad10 2024-02-02T13:31:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.884+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe813ad30 con 0x7fdbe815ad10 2024-02-02T13:31:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.885+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe813be00 con 0x7fdbe815ad10 2024-02-02T13:31:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.886+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdbd0003990 con 0x7fdbe815ad10 2024-02-02T13:31:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.886+0000 7fdbdfe00640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 0x7fdbc006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.887+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbd0087c80 con 0x7fdbe815ad10 2024-02-02T13:31:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.887+0000 7fdbe7400640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 0x7fdbc006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:57.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.888+0000 7fdbe7400640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 0x7fdbc006c8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8036000 tx=0x7fdbd803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:57.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:57.891+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdbe813be00 con 0x7fdbe815ad10 2024-02-02T13:31:58.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.086+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbe813be00 con 0x7fdbe815ad10 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.087+0000 7fdbdfe00640 1 -- 172.21.15.91:0/3950651012 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdbe813be00 con 0x7fdbe815ad10 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:31:58.089 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 msgr2=0x7fdbc006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 0x7fdbc006c8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8036000 tx=0x7fdbd803c040 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 msgr2=0x7fdbe80c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c400 tx=0x7fdbd000c8c0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 shutdown_connections 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdbc006a430 0x7fdbc006c8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbe815b0f0 0x7fdbe81f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbe815ad10 0x7fdbe80c4300 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 --2- 172.21.15.91:0/3950651012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbe80cb580 0x7fdbe813a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 >> 172.21.15.91:0/3950651012 conn(0x7fdbe80b0700 msgr2=0x7fdbe806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 shutdown_connections 2024-02-02T13:31:58.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.090+0000 7fdbee800640 1 -- 172.21.15.91:0/3950651012 wait complete. 2024-02-02T13:31:58.111 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:31:58.111 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:31:58.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.319+0000 7fa6de600640 1 Processor -- start 2024-02-02T13:31:58.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.319+0000 7fa6de600640 1 -- start start 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.320+0000 7fa6de600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80ce130 0x7fa6d80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.320+0000 7fa6de600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa6d80b0700 0x7fa6d80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.320+0000 7fa6de600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d80cc7c0 con 0x7fa6d815a010 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.320+0000 7fa6de600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80b10d0 con 0x7fa6d80b0700 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.320+0000 7fa6de600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80ce930 con 0x7fa6d80ce130 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa6d80b0700 0x7fa6d80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa6d80b0700 0x7fa6d80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34824/0 (socket says 172.21.15.91:34824) 2024-02-02T13:31:58.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 -- 172.21.15.91:0/970681949 learned_addr learned my addr 172.21.15.91:0/970681949 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:58.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d6a00640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80ce130 0x7fa6d80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:58.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 -- 172.21.15.91:0/970681949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80ce130 msgr2=0x7fa6d80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:58.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80ce130 0x7fa6d80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 -- 172.21.15.91:0/970681949 >> v1:172.21.15.113:6789/0 conn(0x7fa6d815a010 legacy=0x7fa6d80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:58.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.321+0000 7fa6d7e00640 1 -- 172.21.15.91:0/970681949 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80cd640 con 0x7fa6d80b0700 2024-02-02T13:31:58.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.322+0000 7fa6d7e00640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80b0700 0x7fa6d80b0b90 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009e50 tx=0x7fa6d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79fc4b7b5dc8344 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:58.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6d6000640 1 -- 172.21.15.91:0/970681949 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d002ca70 con 0x7fa6d80b0700 2024-02-02T13:31:58.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6d6000640 1 -- 172.21.15.91:0/970681949 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d002cbd0 con 0x7fa6d80b0700 2024-02-02T13:31:58.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6d6000640 1 -- 172.21.15.91:0/970681949 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00343f0 con 0x7fa6d80b0700 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 -- 172.21.15.91:0/970681949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80b0700 msgr2=0x7fa6d80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80b0700 0x7fa6d80b0b90 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009e50 tx=0x7fa6d002bed0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 -- 172.21.15.91:0/970681949 shutdown_connections 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80b0700 0x7fa6d80b0b90 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 --2- 172.21.15.91:0/970681949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80ce130 0x7fa6d80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 -- 172.21.15.91:0/970681949 >> 172.21.15.91:0/970681949 conn(0x7fa6d80cec80 msgr2=0x7fa6d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 -- 172.21.15.91:0/970681949 shutdown_connections 2024-02-02T13:31:58.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.323+0000 7fa6de600640 1 -- 172.21.15.91:0/970681949 wait complete. 2024-02-02T13:31:58.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.325+0000 7fa6de600640 1 Processor -- start 2024-02-02T13:31:58.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.325+0000 7fa6de600640 1 -- start start 2024-02-02T13:31:58.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80b0700 0x7fa6d80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 0x7fa6d81f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80c35d0 con 0x7fa6d80ce130 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b1980 con 0x7fa6d815a010 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.326+0000 7fa6de600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b1c60 con 0x7fa6d80b0700 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34832/0 (socket says 172.21.15.91:34832) 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 -- 172.21.15.91:0/2768999516 learned_addr learned my addr 172.21.15.91:0/2768999516 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d7e00640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 0x7fa6d81f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 -- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80b0700 msgr2=0x7fa6d80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80b0700 0x7fa6d80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 -- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 msgr2=0x7fa6d81f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:58.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 0x7fa6d81f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 -- 172.21.15.91:0/2768999516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0009b30 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d7e00640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 0x7fa6d81f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.327+0000 7fa6d6a00640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c7f0 tx=0x7fa6c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.328+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004220 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.328+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c00043a0 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.328+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0007860 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.328+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d80b1f40 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.328+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d80c32b0 con 0x7fa6d80ce130 2024-02-02T13:31:58.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.329+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d80b2460 con 0x7fa6d80ce130 2024-02-02T13:31:58.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.330+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa6c0007bd0 con 0x7fa6d80ce130 2024-02-02T13:31:58.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.331+0000 7fa6cf400640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 0x7fa6b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:58.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.331+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6c0088650 con 0x7fa6d80ce130 2024-02-02T13:31:58.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.331+0000 7fa6d7400640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 0x7fa6b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:58.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.332+0000 7fa6d7400640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 0x7fa6b006c8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006fd0 tx=0x7fa6c8006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:58.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.335+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa6d80b2460 con 0x7fa6d80ce130 2024-02-02T13:31:58.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:58.523+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6d80b2460 con 0x7fa6d80ce130 2024-02-02T13:31:59.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.055+0000 7fa6cf400640 1 -- 172.21.15.91:0/2768999516 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6d80b2460 con 0x7fa6d80ce130 2024-02-02T13:31:59.057 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:31:59.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.061+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 msgr2=0x7fa6b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.062+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 0x7fa6b006c8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006fd0 tx=0x7fa6c8006bc0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.062+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 msgr2=0x7fa6d80c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.062+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c7f0 tx=0x7fa6c000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 shutdown_connections 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa6b006a430 0x7fa6b006c8f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d815a010 0x7fa6d81f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6d80ce130 0x7fa6d80c2d70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 --2- 172.21.15.91:0/2768999516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6d80b0700 0x7fa6d80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.063+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 >> 172.21.15.91:0/2768999516 conn(0x7fa6d80cec80 msgr2=0x7fa6d80c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:59.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.064+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 shutdown_connections 2024-02-02T13:31:59.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.064+0000 7fa6de600640 1 -- 172.21.15.91:0/2768999516 wait complete. 2024-02-02T13:31:59.085 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:31:59.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.289+0000 7f774fe00640 1 Processor -- start 2024-02-02T13:31:59.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.289+0000 7f774fe00640 1 -- start start 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.290+0000 7f774fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77500cb730 0x7f77500cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.290+0000 7f774fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775006f9f0 con 0x7f77500b1c00 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.290+0000 7f774fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77500719a0 con 0x7f775015aec0 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f774fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775006c600 con 0x7f77500cb730 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f774f400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f77500b1c00 0x7f77500b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:53742/0 (socket says 172.21.15.91:53742) 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f774f400640 1 -- 172.21.15.91:0/1511867435 learned_addr learned my addr 172.21.15.91:0/1511867435 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f7747e00640 1 --2- 172.21.15.91:0/1511867435 >> v2:172.21.15.113:3300/0 conn(0x7f77500cb730 0x7f77500cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803671083 0 0) 0x7f775006f9f0 con 0x7f77500b1c00 2024-02-02T13:31:59.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.291+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7730003520 con 0x7f77500b1c00 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2881252419 0 0) 0x7f7730003520 con 0x7f77500b1c00 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 >> v2:172.21.15.113:3300/0 conn(0x7f77500cb730 msgr2=0x7f77500cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 --2- 172.21.15.91:0/1511867435 >> v2:172.21.15.113:3300/0 conn(0x7f77500cb730 0x7f77500cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 >> v1:172.21.15.113:6789/0 conn(0x7f775015aec0 legacy=0x7f7750071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500cc0e0 con 0x7f77500b1c00 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f77480019f0 con 0x7f77500b1c00 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 >> v1:172.21.15.91:6789/0 conn(0x7f77500b1c00 legacy=0x7f77500b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7730003970 0x7f7730003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77300047c0 0x7f7730004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f773000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775006c600 con 0x7f7730005180 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773000c120 con 0x7f7730003970 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773000c2e0 con 0x7f77300047c0 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774f400640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f773000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f774ea00640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7730003970 0x7f7730003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.292+0000 7f7747e00640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77300047c0 0x7f7730004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774f400640 1 -- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77300047c0 msgr2=0x7f7730004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774f400640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77300047c0 0x7f7730004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774f400640 1 -- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7730003970 msgr2=0x7f7730003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774f400640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7730003970 0x7f7730003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774f400640 1 -- 172.21.15.91:0/1511867435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7730003520 con 0x7f7730005180 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.293+0000 7f774ea00640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7730003970 0x7f7730003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.294+0000 7f774f400640 1 -- 172.21.15.91:0/1511867435 reap_dead start 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.294+0000 7f774f400640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f773000b980 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f774800a830 tx=0x7f77300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725f8b1c73cf894d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.294+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480072d0 con 0x7f7730005180 2024-02-02T13:31:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.294+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774802f070 con 0x7f7730005180 2024-02-02T13:31:59.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.294+0000 7f774e000640 1 -- 172.21.15.91:0/1511867435 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480076b0 con 0x7f7730005180 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 -- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 msgr2=0x7f773000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f773000b980 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f774800a830 tx=0x7f77300057f0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 -- 172.21.15.91:0/1511867435 shutdown_connections 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 --2- 172.21.15.91:0/1511867435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f773000b980 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 -- 172.21.15.91:0/1511867435 >> 172.21.15.91:0/1511867435 conn(0x7f77500b08e0 msgr2=0x7f77500b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 -- 172.21.15.91:0/1511867435 shutdown_connections 2024-02-02T13:31:59.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.295+0000 7f774fe00640 1 -- 172.21.15.91:0/1511867435 wait complete. 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.297+0000 7f774fe00640 1 Processor -- start 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.297+0000 7f774fe00640 1 -- start start 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f775015cca0 0x7f775014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77500cd510 0x7f77501599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775014e450 con 0x7f7730005180 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77500cd9a0 con 0x7f775015cca0 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77500cdc80 con 0x7f77500cd510 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:34852/0 (socket says 172.21.15.91:34852) 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 -- 172.21.15.91:0/1633975100 learned_addr learned my addr 172.21.15.91:0/1633975100 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774f400640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77500cd510 0x7f77501599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f7747e00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f775015cca0 0x7f775014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 -- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77500cd510 msgr2=0x7f77501599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77500cd510 0x7f77501599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 -- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f775015cca0 msgr2=0x7f775014df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f775015cca0 0x7f775014df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.298+0000 7f774ea00640 1 -- 172.21.15.91:0/1633975100 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500cdf60 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f774ea00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f774000e4d0 tx=0x7f774000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400052e0 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7740005440 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400037f0 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77500ce240 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.299+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775015a110 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.300+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775015a330 con 0x7f7730005180 2024-02-02T13:31:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.302+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7740003950 con 0x7f7730005180 2024-02-02T13:31:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.302+0000 7f7746a00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 0x7f772806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:31:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.303+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7740088ee0 con 0x7f7730005180 2024-02-02T13:31:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.303+0000 7f7747e00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 0x7f772806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:31:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.304+0000 7f7747e00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 0x7f772806c8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7750157020 tx=0x7f7738008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:31:59.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.307+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7740054790 con 0x7f7730005180 2024-02-02T13:31:59.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:31:59.502+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f775004dff0 con 0x7f7730005180 2024-02-02T13:32:00.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.072+0000 7f7746a00640 1 -- 172.21.15.91:0/1633975100 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7740057db0 con 0x7f7730005180 2024-02-02T13:32:00.075 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:32:00.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.078+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 msgr2=0x7f772806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.078+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 0x7f772806c8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7750157020 tx=0x7f7738008040 comp rx=0 tx=0).stop 2024-02-02T13:32:00.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.078+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 msgr2=0x7f775015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.078+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f774000e4d0 tx=0x7f774000e990 comp rx=0 tx=0).stop 2024-02-02T13:32:00.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.078+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 shutdown_connections 2024-02-02T13:32:00.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.079+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f772806a430 0x7f772806c8f0 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7750157020 tx=0x7f7738008040 comp rx=0 tx=0).stop 2024-02-02T13:32:00.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.079+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77500cd510 0x7f77501599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.079+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f775015cca0 0x7f775014df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.079+0000 7f774fe00640 1 --2- 172.21.15.91:0/1633975100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7730005180 0x7f775015da90 secure :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f774000e4d0 tx=0x7f774000e990 comp rx=0 tx=0).stop 2024-02-02T13:32:00.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.079+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 >> 172.21.15.91:0/1633975100 conn(0x7f77500b08e0 msgr2=0x7f7750150150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:00.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.081+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 shutdown_connections 2024-02-02T13:32:00.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.082+0000 7f774fe00640 1 -- 172.21.15.91:0/1633975100 wait complete. 2024-02-02T13:32:00.114 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:32:00.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.299+0000 7f1830400640 1 Processor -- start 2024-02-02T13:32:00.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.299+0000 7f1830400640 1 -- start start 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.300+0000 7f1830400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18280b0e10 0x7f18280c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.300+0000 7f1830400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280cc7c0 con 0x7f182815a010 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.300+0000 7f1830400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280ce930 con 0x7f18280ce130 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f1830400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18280cd640 con 0x7f18280b0e10 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f182da00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f182815a010 0x7f18280ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45410/0 (socket says 172.21.15.91:45410) 2024-02-02T13:32:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f182da00640 1 -- 172.21.15.91:0/207982645 learned_addr learned my addr 172.21.15.91:0/207982645 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f182e400640 1 --2- 172.21.15.91:0/207982645 >> v2:172.21.15.113:3300/0 conn(0x7f18280b0e10 0x7f18280c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353251953 0 0) 0x7f18280cc7c0 con 0x7f182815a010 2024-02-02T13:32:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.301+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1808005730 con 0x7f182815a010 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305697141 0 0) 0x7f1808005730 con 0x7f182815a010 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 >> v2:172.21.15.113:3300/0 conn(0x7f18280b0e10 msgr2=0x7f18280c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 --2- 172.21.15.91:0/207982645 >> v2:172.21.15.113:3300/0 conn(0x7f18280b0e10 0x7f18280c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 >> v1:172.21.15.113:6789/0 conn(0x7f18280ce130 legacy=0x7f18280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182806c890 con 0x7f182815a010 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1818001670 con 0x7f182815a010 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 >> v1:172.21.15.91:6789/0 conn(0x7f182815a010 legacy=0x7f18280ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1808005bd0 0x7f1808005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f1808006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18080073e0 0x7f180800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280cd640 con 0x7f18080069d0 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180800e330 con 0x7f18080073e0 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180800e4f0 con 0x7f1808005bd0 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.302+0000 7f182d000640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f1808006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182da00640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1808005bd0 0x7f1808005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182e400640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18080073e0 0x7f180800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182d000640 1 -- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1808005bd0 msgr2=0x7f1808005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182d000640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1808005bd0 0x7f1808005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182d000640 1 -- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18080073e0 msgr2=0x7f180800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182d000640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18080073e0 0x7f180800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182d000640 1 -- 172.21.15.91:0/207982645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808005730 con 0x7f18080069d0 2024-02-02T13:32:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.303+0000 7f182e400640 1 -- 172.21.15.91:0/207982645 reap_dead start 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.304+0000 7f182d000640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f1808006e50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1810001720 tx=0x7f1808007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb3dec26afc4cde server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.304+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100052d0 con 0x7f18080069d0 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.304+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181002e050 con 0x7f18080069d0 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.304+0000 7f1827e00640 1 -- 172.21.15.91:0/207982645 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810003830 con 0x7f18080069d0 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 -- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 msgr2=0x7f1808006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f1808006e50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1810001720 tx=0x7f1808007a00 comp rx=0 tx=0).stop 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 -- 172.21.15.91:0/207982645 shutdown_connections 2024-02-02T13:32:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 --2- 172.21.15.91:0/207982645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f1808006e50 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 -- 172.21.15.91:0/207982645 >> 172.21.15.91:0/207982645 conn(0x7f18280cec80 msgr2=0x7f18280cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:00.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 -- 172.21.15.91:0/207982645 shutdown_connections 2024-02-02T13:32:00.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.305+0000 7f1830400640 1 -- 172.21.15.91:0/207982645 wait complete. 2024-02-02T13:32:00.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.307+0000 7f1830400640 1 Processor -- start 2024-02-02T13:32:00.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.307+0000 7f1830400640 1 -- start start 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18280c4ca0 0x7f18280c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18280c5cc0 0x7f182815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18281ee650 con 0x7f18080069d0 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280c5180 con 0x7f18280c4ca0 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f1830400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280c5460 con 0x7f18280c5cc0 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58322/0 (socket says 172.21.15.91:58322) 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 -- 172.21.15.91:0/2045624817 learned_addr learned my addr 172.21.15.91:0/2045624817 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182d000640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18280c4ca0 0x7f18280c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 -- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18280c5cc0 msgr2=0x7f182815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18280c5cc0 0x7f182815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 -- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18280c4ca0 msgr2=0x7f18280c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18280c4ca0 0x7f18280c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.308+0000 7f182da00640 1 -- 172.21.15.91:0/2045624817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280c5740 con 0x7f18080069d0 2024-02-02T13:32:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f182da00640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f181800e730 tx=0x7f181800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800c640 con 0x7f18080069d0 2024-02-02T13:32:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181800c7a0 con 0x7f18080069d0 2024-02-02T13:32:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800cb50 con 0x7f18080069d0 2024-02-02T13:32:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18280c5a20 con 0x7f18080069d0 2024-02-02T13:32:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.309+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18280c8e30 con 0x7f18080069d0 2024-02-02T13:32:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.310+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828066cf0 con 0x7f18080069d0 2024-02-02T13:32:00.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.312+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f181800ccb0 con 0x7f18080069d0 2024-02-02T13:32:00.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.313+0000 7f1825600640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 0x7f180006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.313+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f18180891d0 con 0x7f18080069d0 2024-02-02T13:32:00.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.313+0000 7f182d000640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 0x7f180006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.314+0000 7f182d000640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 0x7f180006c920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1810033000 tx=0x7f1810009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.317+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1818054800 con 0x7f18080069d0 2024-02-02T13:32:00.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.542+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f18280b4c00 con 0x7f18080069d0 2024-02-02T13:32:00.557 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 16 and data pool 17 2024-02-02T13:32:00.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.554+0000 7f1825600640 1 -- 172.21.15.91:0/2045624817 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1818057e20 con 0x7f18080069d0 2024-02-02T13:32:00.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 msgr2=0x7f180006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 0x7f180006c920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1810033000 tx=0x7f1810009030 comp rx=0 tx=0).stop 2024-02-02T13:32:00.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 msgr2=0x7f18280c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f181800e730 tx=0x7f181800ebf0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.564 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 shutdown_connections 2024-02-02T13:32:00.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.562+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f180006a460 0x7f180006c920 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.563+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18280c5cc0 0x7f182815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.563+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18280c4ca0 0x7f18280c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.563+0000 7f1830400640 1 --2- 172.21.15.91:0/2045624817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18080069d0 0x7f18280c2350 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.563+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 >> 172.21.15.91:0/2045624817 conn(0x7f18280cec80 msgr2=0x7f1828073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:00.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.567+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 shutdown_connections 2024-02-02T13:32:00.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.567+0000 7f1830400640 1 -- 172.21.15.91:0/2045624817 wait complete. 2024-02-02T13:32:00.594 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:00.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.768+0000 7feea6800640 1 Processor -- start 2024-02-02T13:32:00.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.768+0000 7feea6800640 1 -- start start 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.769+0000 7feea6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feea0151f90 0x7feea015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.769+0000 7feea6800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea00b0e10 0x7feea00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.769+0000 7feea6800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea0070f80 con 0x7feea006f4f0 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.769+0000 7feea6800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea0152370 con 0x7feea00b0e10 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.769+0000 7feea6800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00cb010 con 0x7feea0151f90 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea00b0e10 0x7feea00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feea00b0e10 0x7feea00b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58330/0 (socket says 172.21.15.91:58330) 2024-02-02T13:32:00.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 -- 172.21.15.91:0/495399427 learned_addr learned my addr 172.21.15.91:0/495399427 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9f400640 1 --2- 172.21.15.91:0/495399427 >> v2:172.21.15.113:3300/0 conn(0x7feea0151f90 0x7feea015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 -- 172.21.15.91:0/495399427 >> v2:172.21.15.113:3300/0 conn(0x7feea0151f90 msgr2=0x7feea015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 --2- 172.21.15.91:0/495399427 >> v2:172.21.15.113:3300/0 conn(0x7feea0151f90 0x7feea015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 -- 172.21.15.91:0/495399427 >> v1:172.21.15.113:6789/0 conn(0x7feea006f4f0 legacy=0x7feea00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.770+0000 7fee9fe00640 1 -- 172.21.15.91:0/495399427 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00cb2f0 con 0x7feea00b0e10 2024-02-02T13:32:00.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.771+0000 7fee9fe00640 1 --2- 172.21.15.91:0/495399427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00b0e10 0x7feea00b3af0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7feea00cc770 tx=0x7fee9802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff741b3a4182a6a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.772+0000 7fee9ea00640 1 -- 172.21.15.91:0/495399427 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9802ca80 con 0x7feea00b0e10 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.772+0000 7fee9ea00640 1 -- 172.21.15.91:0/495399427 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9802cbe0 con 0x7feea00b0e10 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.772+0000 7fee9ea00640 1 -- 172.21.15.91:0/495399427 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980343f0 con 0x7feea00b0e10 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.772+0000 7feea6800640 1 -- 172.21.15.91:0/495399427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00b0e10 msgr2=0x7feea00b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.772+0000 7feea6800640 1 --2- 172.21.15.91:0/495399427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00b0e10 0x7feea00b3af0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7feea00cc770 tx=0x7fee9802bee0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 -- 172.21.15.91:0/495399427 shutdown_connections 2024-02-02T13:32:00.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 --2- 172.21.15.91:0/495399427 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00b0e10 0x7feea00b3af0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 --2- 172.21.15.91:0/495399427 >> v2:172.21.15.113:3300/0 conn(0x7feea0151f90 0x7feea015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 -- 172.21.15.91:0/495399427 >> 172.21.15.91:0/495399427 conn(0x7feea00cec80 msgr2=0x7feea00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:00.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 -- 172.21.15.91:0/495399427 shutdown_connections 2024-02-02T13:32:00.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.773+0000 7feea6800640 1 -- 172.21.15.91:0/495399427 wait complete. 2024-02-02T13:32:00.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.774+0000 7feea6800640 1 Processor -- start 2024-02-02T13:32:00.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.774+0000 7feea6800640 1 -- start start 2024-02-02T13:32:00.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea00b0e10 0x7feea0150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea0151f90 0x7feea00c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00c42a0 con 0x7feea006f4f0 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0151210 con 0x7feea0151f90 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.775+0000 7feea6800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00c3a10 con 0x7feea00b0e10 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58342/0 (socket says 172.21.15.91:58342) 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 -- 172.21.15.91:0/305714061 learned_addr learned my addr 172.21.15.91:0/305714061 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 -- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea00b0e10 msgr2=0x7feea0150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee97e00640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea00b0e10 0x7feea0150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9fe00640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea0151f90 0x7feea00c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea00b0e10 0x7feea0150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 -- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea0151f90 msgr2=0x7feea00c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea0151f90 0x7feea00c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 -- 172.21.15.91:0/305714061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98009b30 con 0x7feea006f4f0 2024-02-02T13:32:00.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.776+0000 7fee9f400640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c820 tx=0x7fee9000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.777+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90004220 con 0x7feea006f4f0 2024-02-02T13:32:00.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.777+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee900043a0 con 0x7feea006f4f0 2024-02-02T13:32:00.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.777+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90007a70 con 0x7feea006f4f0 2024-02-02T13:32:00.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.777+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea00c3170 con 0x7feea006f4f0 2024-02-02T13:32:00.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.777+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea00c3730 con 0x7feea006f4f0 2024-02-02T13:32:00.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.778+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea0066cf0 con 0x7feea006f4f0 2024-02-02T13:32:00.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.779+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fee90007bd0 con 0x7feea006f4f0 2024-02-02T13:32:00.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.780+0000 7fee97400640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 0x7fee7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:00.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.780+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee90088730 con 0x7feea006f4f0 2024-02-02T13:32:00.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.780+0000 7fee97e00640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 0x7fee7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:00.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.781+0000 7fee97e00640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 0x7fee7806c8f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feea0159d20 tx=0x7fee88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:00.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.784+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fee90053d10 con 0x7feea006f4f0 2024-02-02T13:32:00.905 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:32:00.913 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:32:00.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.977+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea006ffe0 con 0x7feea006f4f0 2024-02-02T13:32:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.978+0000 7fee97400640 1 -- 172.21.15.91:0/305714061 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16073 (secure 0 0 0) 0x7fee90057330 con 0x7feea006f4f0 2024-02-02T13:32:00.980 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:00.980 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":142,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:00.544232+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 msgr2=0x7fee7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 0x7fee7806c8f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feea0159d20 tx=0x7fee88008040 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 msgr2=0x7feea0156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c820 tx=0x7fee9000cce0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 shutdown_connections 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fee7806a430 0x7fee7806c8f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea00b0e10 0x7feea0150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea0151f90 0x7feea00c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 --2- 172.21.15.91:0/305714061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea006f4f0 0x7feea0156ef0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 >> 172.21.15.91:0/305714061 conn(0x7feea00cec80 msgr2=0x7feea01590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 shutdown_connections 2024-02-02T13:32:00.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:00.982+0000 7feea6800640 1 -- 172.21.15.91:0/305714061 wait complete. 2024-02-02T13:32:01.001 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:01.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.192+0000 7fb2eee00640 1 Processor -- start 2024-02-02T13:32:01.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.192+0000 7fb2eee00640 1 -- start start 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.193+0000 7fb2eee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2e8151f90 0x7fb2e815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.193+0000 7fb2eee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2e80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.193+0000 7fb2eee00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e8070f80 con 0x7fb2e80b0c40 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.193+0000 7fb2eee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80b10d0 con 0x7fb2e8151f90 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.193+0000 7fb2eee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8152370 con 0x7fb2e806f4f0 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2ece00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb2e80b0c40 0x7fb2e80b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45416/0 (socket says 172.21.15.91:45416) 2024-02-02T13:32:01.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2ece00640 1 -- 172.21.15.91:0/957679799 learned_addr learned my addr 172.21.15.91:0/957679799 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2e7e00640 1 --2- 172.21.15.91:0/957679799 >> v2:172.21.15.113:3300/0 conn(0x7fb2e8151f90 0x7fb2e815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2e80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302347016 0 0) 0x7fb2e8070f80 con 0x7fb2e80b0c40 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c8003520 con 0x7fb2e80b0c40 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.194+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721882193 0 0) 0x7fb2c8003520 con 0x7fb2e80b0c40 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 msgr2=0x7fb2e80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2e80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 >> v2:172.21.15.113:3300/0 conn(0x7fb2e8151f90 msgr2=0x7fb2e815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 --2- 172.21.15.91:0/957679799 >> v2:172.21.15.113:3300/0 conn(0x7fb2e8151f90 0x7fb2e815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e80cb010 con 0x7fb2e80b0c40 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb2e00019f0 con 0x7fb2e80b0c40 2024-02-02T13:32:01.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7e00640 1 --2- 172.21.15.91:0/957679799 >> v2:172.21.15.113:3300/0 conn(0x7fb2e8151f90 0x7fb2e815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2e80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 >> v1:172.21.15.91:6789/0 conn(0x7fb2e80b0c40 legacy=0x7fb2e80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2c8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c8003fb0 0x7fb2c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80b10d0 con 0x7fb2c8003fb0 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8152370 con 0x7fb2c8004d00 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c800bab0 con 0x7fb2e806f4f0 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c8003fb0 0x7fb2c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2ece00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2e7e00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2c8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 -- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 msgr2=0x7fb2c8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e806f4f0 0x7fb2c8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 -- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2c8004d00 msgr2=0x7fb2c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:01.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.195+0000 7fb2dfe00640 1 -- 172.21.15.91:0/957679799 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8003520 con 0x7fb2c8003fb0 2024-02-02T13:32:01.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.196+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c8003fb0 0x7fb2c80047c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b34166de48a1989 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004230 con 0x7fb2c8003fb0 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0004390 con 0x7fb2c8003fb0 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2e7400640 1 -- 172.21.15.91:0/957679799 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003830 con 0x7fb2c8003fb0 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 -- 172.21.15.91:0/957679799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c8003fb0 msgr2=0x7fb2c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 --2- 172.21.15.91:0/957679799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c8003fb0 0x7fb2c80047c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2c80057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2ece00640 1 -- 172.21.15.91:0/957679799 reap_dead start 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 -- 172.21.15.91:0/957679799 shutdown_connections 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 -- 172.21.15.91:0/957679799 >> 172.21.15.91:0/957679799 conn(0x7fb2e80cec80 msgr2=0x7fb2e80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 -- 172.21.15.91:0/957679799 shutdown_connections 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.197+0000 7fb2eee00640 1 -- 172.21.15.91:0/957679799 wait complete. 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.199+0000 7fb2eee00640 1 Processor -- start 2024-02-02T13:32:01.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.199+0000 7fb2eee00640 1 -- start start 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2c8003fb0 0x7fb2e80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e8153840 0x7fb2e80c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8154e30 con 0x7fb2e8153cc0 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8154220 con 0x7fb2e8153840 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.200+0000 7fb2eee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8154500 con 0x7fb2c8003fb0 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58370/0 (socket says 172.21.15.91:58370) 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 -- 172.21.15.91:0/3375541534 learned_addr learned my addr 172.21.15.91:0/3375541534 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2dfe00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e8153840 0x7fb2e80c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 -- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2c8003fb0 msgr2=0x7fb2e80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2c8003fb0 0x7fb2e80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 -- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e8153840 msgr2=0x7fb2e80c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e8153840 0x7fb2e80c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 -- 172.21.15.91:0/3375541534 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e81547e0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.201+0000 7fb2ece00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b1870 tx=0x7fb2e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.202+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004d60 con 0x7fb2e8153cc0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.202+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e000bb80 con 0x7fb2e8153cc0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.202+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0007110 con 0x7fb2e8153cc0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.202+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e8154ac0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.202+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e80cdf70 con 0x7fb2e8153cc0 2024-02-02T13:32:01.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.203+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e8066cf0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.204+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb2e0007270 con 0x7fb2e8153cc0 2024-02-02T13:32:01.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.204+0000 7fb2e4c00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 0x7fb2c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.205+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e0088cc0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.205+0000 7fb2e7e00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 0x7fb2c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.205+0000 7fb2e7e00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 0x7fb2c006c8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006e80 tx=0x7fb2d8006df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.209+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb2e00542b0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.395+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e806c7b0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.398+0000 7fb2e4c00640 1 -- 172.21.15.91:0/3375541534 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16073 (secure 0 0 0) 0x7fb2e00578d0 con 0x7fb2e8153cc0 2024-02-02T13:32:01.400 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:01.400 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":142,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:00.544232+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:01.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 msgr2=0x7fb2c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 0x7fb2c006c8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006e80 tx=0x7fb2d8006df0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 msgr2=0x7fb2e80c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b1870 tx=0x7fb2e000b4a0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 shutdown_connections 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb2c006a430 0x7fb2c006c8f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2e8153cc0 0x7fb2e80c8210 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e8153840 0x7fb2e80c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 --2- 172.21.15.91:0/3375541534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2c8003fb0 0x7fb2e80cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 >> 172.21.15.91:0/3375541534 conn(0x7fb2e80cec80 msgr2=0x7fb2e815ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 shutdown_connections 2024-02-02T13:32:01.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.401+0000 7fb2eee00640 1 -- 172.21.15.91:0/3375541534 wait complete. 2024-02-02T13:32:01.420 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:32:01.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.622+0000 7f8853400640 1 Processor -- start 2024-02-02T13:32:01.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.622+0000 7f8853400640 1 -- start start 2024-02-02T13:32:01.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.623+0000 7f8853400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f884c0b0da0 0x7f884c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.623+0000 7f8853400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0b1e60 con 0x7f884c15b2f0 2024-02-02T13:32:01.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.623+0000 7f8853400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c15f1b0 con 0x7f884c0b1a00 2024-02-02T13:32:01.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.623+0000 7f8853400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c15fdc0 con 0x7f884c0b0da0 2024-02-02T13:32:01.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8850a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f884c15b2f0 0x7f884c0711f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45432/0 (socket says 172.21.15.91:45432) 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8850a00640 1 -- 172.21.15.91:0/2537996026 learned_addr learned my addr 172.21.15.91:0/2537996026 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8851400640 1 --2- 172.21.15.91:0/2537996026 >> v2:172.21.15.91:3300/0 conn(0x7f884c0b0da0 0x7f884c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f884be00640 1 -- 172.21.15.91:0/2537996026 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791203936 0 0) 0x7f884c0b1e60 con 0x7f884c15b2f0 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f884be00640 1 -- 172.21.15.91:0/2537996026 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f882c003520 con 0x7f884c15b2f0 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8851400640 1 -- 172.21.15.91:0/2537996026 >> v1:172.21.15.113:6789/0 conn(0x7f884c0b1a00 legacy=0x7f884c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:01.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8851400640 1 -- 172.21.15.91:0/2537996026 >> v1:172.21.15.91:6789/0 conn(0x7f884c15b2f0 legacy=0x7f884c0711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.624+0000 7f8851400640 1 -- 172.21.15.91:0/2537996026 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c15f490 con 0x7f884c0b0da0 2024-02-02T13:32:01.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f8851400640 1 --2- 172.21.15.91:0/2537996026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b0da0 0x7f884c0c9480 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8844009d90 tx=0x7f884402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6470cced417a32c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f884be00640 1 -- 172.21.15.91:0/2537996026 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884402cbd0 con 0x7f884c0b0da0 2024-02-02T13:32:01.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f884be00640 1 -- 172.21.15.91:0/2537996026 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884402cd30 con 0x7f884c0b0da0 2024-02-02T13:32:01.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f884be00640 1 -- 172.21.15.91:0/2537996026 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884402cea0 con 0x7f884c0b0da0 2024-02-02T13:32:01.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f8853400640 1 -- 172.21.15.91:0/2537996026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b0da0 msgr2=0x7f884c0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.626+0000 7f8853400640 1 --2- 172.21.15.91:0/2537996026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b0da0 0x7f884c0c9480 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8844009d90 tx=0x7f884402be20 comp rx=0 tx=0).stop 2024-02-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.627+0000 7f8853400640 1 -- 172.21.15.91:0/2537996026 shutdown_connections 2024-02-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.627+0000 7f8853400640 1 --2- 172.21.15.91:0/2537996026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b0da0 0x7f884c0c9480 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.627+0000 7f8853400640 1 -- 172.21.15.91:0/2537996026 >> 172.21.15.91:0/2537996026 conn(0x7f884c0ceb90 msgr2=0x7f884c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.627+0000 7f8853400640 1 -- 172.21.15.91:0/2537996026 shutdown_connections 2024-02-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.627+0000 7f8853400640 1 -- 172.21.15.91:0/2537996026 wait complete. 2024-02-02T13:32:01.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.628+0000 7f8853400640 1 Processor -- start 2024-02-02T13:32:01.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.629+0000 7f8853400640 1 -- start start 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 0x7f884c0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f884c15b2f0 0x7f884c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c8370 con 0x7f884c0b1a00 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c7450 con 0x7f884c0b0da0 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8853400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c14c620 con 0x7f884c15b2f0 2024-02-02T13:32:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58390/0 (socket says 172.21.15.91:58390) 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 -- 172.21.15.91:0/857685011 learned_addr learned my addr 172.21.15.91:0/857685011 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8850a00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 0x7f884c0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8851400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f884c15b2f0 0x7f884c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 -- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f884c15b2f0 msgr2=0x7f884c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f884c15b2f0 0x7f884c15e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 -- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 msgr2=0x7f884c0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 0x7f884c0c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8843e00640 1 -- 172.21.15.91:0/857685011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8844009a70 con 0x7f884c0b1a00 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.630+0000 7f8850a00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 0x7f884c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8843e00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f883400b3f0 tx=0x7f883400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834011830 con 0x7f884c0b1a00 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834011990 con 0x7f884c0b1a00 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834011d40 con 0x7f884c0b1a00 2024-02-02T13:32:01.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c15e630 con 0x7f884c0b1a00 2024-02-02T13:32:01.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.631+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0c79d0 con 0x7f884c0b1a00 2024-02-02T13:32:01.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.633+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c066cf0 con 0x7f884c0b1a00 2024-02-02T13:32:01.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.633+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f883401a400 con 0x7f884c0b1a00 2024-02-02T13:32:01.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.634+0000 7f8849600640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 0x7f882406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:01.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.634+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8834088710 con 0x7f884c0b1a00 2024-02-02T13:32:01.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.634+0000 7f8850a00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 0x7f882406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:01.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.635+0000 7f8850a00640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 0x7f882406c8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f883c0011e0 tx=0x7f883c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:01.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.638+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8834053d90 con 0x7f884c0b1a00 2024-02-02T13:32:01.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:01.864+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f884c06c950 con 0x7f884c0b1a00 2024-02-02T13:32:02.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.567+0000 7f8849600640 1 -- 172.21.15.91:0/857685011 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f88340573b0 con 0x7f884c0b1a00 2024-02-02T13:32:02.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 msgr2=0x7f882406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 0x7f882406c8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f883c0011e0 tx=0x7f883c009040 comp rx=0 tx=0).stop 2024-02-02T13:32:02.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 msgr2=0x7f884c14c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f883400b3f0 tx=0x7f883400b8b0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 shutdown_connections 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f882406a430 0x7f882406c8f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.570+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f884c15b2f0 0x7f884c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.571+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f884c0b1a00 0x7f884c14c0e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.571+0000 7f8853400640 1 --2- 172.21.15.91:0/857685011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f884c0b0da0 0x7f884c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.571+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 >> 172.21.15.91:0/857685011 conn(0x7f884c0ceb90 msgr2=0x7f884c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.571+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 shutdown_connections 2024-02-02T13:32:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.571+0000 7f8853400640 1 -- 172.21.15.91:0/857685011 wait complete. 2024-02-02T13:32:02.590 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:32:02.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:32:02.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.800+0000 7f77e7400640 1 Processor -- start 2024-02-02T13:32:02.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.800+0000 7f77e7400640 1 -- start start 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.801+0000 7f77e7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77e00c9f90 0x7f77e00b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.801+0000 7f77e7400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e00b1020 con 0x7f77e00b0bd0 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.801+0000 7f77e7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e00cca70 con 0x7f77e015a1e0 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.801+0000 7f77e7400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e00cc770 con 0x7f77e00c9f90 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77e00c9f90 0x7f77e00b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:02.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77e00c9f90 0x7f77e00b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58394/0 (socket says 172.21.15.91:58394) 2024-02-02T13:32:02.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 -- 172.21.15.91:0/3660412553 learned_addr learned my addr 172.21.15.91:0/3660412553 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:02.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 -- 172.21.15.91:0/3660412553 >> v1:172.21.15.113:6789/0 conn(0x7f77e015a1e0 legacy=0x7f77e0070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:02.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 -- 172.21.15.91:0/3660412553 >> v1:172.21.15.91:6789/0 conn(0x7f77e00b0bd0 legacy=0x7f77e00ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.802+0000 7f77d7c00640 1 -- 172.21.15.91:0/3660412553 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00ca570 con 0x7f77e00c9f90 2024-02-02T13:32:02.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.803+0000 7f77d7c00640 1 --2- 172.21.15.91:0/3660412553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00c9f90 0x7f77e00b0690 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a246f70938f1413 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:02.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.803+0000 7f77dfe00640 1 -- 172.21.15.91:0/3660412553 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030840 con 0x7f77e00c9f90 2024-02-02T13:32:02.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77dfe00640 1 -- 172.21.15.91:0/3660412553 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c80309a0 con 0x7f77e00c9f90 2024-02-02T13:32:02.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77dfe00640 1 -- 172.21.15.91:0/3660412553 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030d30 con 0x7f77e00c9f90 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 -- 172.21.15.91:0/3660412553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00c9f90 msgr2=0x7f77e00b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 --2- 172.21.15.91:0/3660412553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00c9f90 0x7f77e00b0690 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c802b980 comp rx=0 tx=0).stop 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 -- 172.21.15.91:0/3660412553 shutdown_connections 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 --2- 172.21.15.91:0/3660412553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00c9f90 0x7f77e00b0690 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 -- 172.21.15.91:0/3660412553 >> 172.21.15.91:0/3660412553 conn(0x7f77e00ceb90 msgr2=0x7f77e00cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 -- 172.21.15.91:0/3660412553 shutdown_connections 2024-02-02T13:32:02.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.804+0000 7f77e7400640 1 -- 172.21.15.91:0/3660412553 wait complete. 2024-02-02T13:32:02.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.806+0000 7f77e7400640 1 Processor -- start 2024-02-02T13:32:02.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.806+0000 7f77e7400640 1 -- start start 2024-02-02T13:32:02.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:02.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e00c9f90 0x7f77e01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:02.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e015a1e0 0x7f77e00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00b1a80 con 0x7f77e00b0bd0 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00c6b50 con 0x7f77e00c9f90 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e7400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00b1500 con 0x7f77e015a1e0 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e4a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58404/0 (socket says 172.21.15.91:58404) 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.807+0000 7f77e4a00640 1 -- 172.21.15.91:0/974716743 learned_addr learned my addr 172.21.15.91:0/974716743 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77d7c00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e00c9f90 0x7f77e01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 -- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e015a1e0 msgr2=0x7f77e00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e015a1e0 0x7f77e00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 -- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e00c9f90 msgr2=0x7f77e01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e00c9f90 0x7f77e01556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 -- 172.21.15.91:0/974716743 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80094d0 con 0x7f77e00b0bd0 2024-02-02T13:32:02.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e4a00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f77d000ba80 tx=0x7f77d000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:02.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000cc00 con 0x7f77e00b0bd0 2024-02-02T13:32:02.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000cd60 con 0x7f77e00b0bd0 2024-02-02T13:32:02.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0003bb0 con 0x7f77e00b0bd0 2024-02-02T13:32:02.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.808+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e00b17a0 con 0x7f77e00b0bd0 2024-02-02T13:32:02.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.809+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e00b22c0 con 0x7f77e00b0bd0 2024-02-02T13:32:02.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.810+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e0066cf0 con 0x7f77e00b0bd0 2024-02-02T13:32:02.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.810+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77d0014440 con 0x7f77e00b0bd0 2024-02-02T13:32:02.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.811+0000 7f77dd600640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 0x7f77b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:02.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.811+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77d0088730 con 0x7f77e00b0bd0 2024-02-02T13:32:02.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.812+0000 7f77d7c00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 0x7f77b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:02.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.812+0000 7f77d7c00640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 0x7f77b806c8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f77c8036000 tx=0x7f77c803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:02.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:02.816+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77d0053d10 con 0x7f77e00b0bd0 2024-02-02T13:32:03.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.025+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f77e006ccb0 con 0x7f77e00b0bd0 2024-02-02T13:32:03.570 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.569+0000 7f77dd600640 1 -- 172.21.15.91:0/974716743 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f77d0057330 con 0x7f77e00b0bd0 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 msgr2=0x7f77b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 0x7f77b806c8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f77c8036000 tx=0x7f77c803b040 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 msgr2=0x7f77e00b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f77d000ba80 tx=0x7f77d000bf40 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 shutdown_connections 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f77b806a430 0x7f77b806c8f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77e015a1e0 0x7f77e00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77e00c9f90 0x7f77e01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 --2- 172.21.15.91:0/974716743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e00b0bd0 0x7f77e00b1d80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 >> 172.21.15.91:0/974716743 conn(0x7f77e00ceb90 msgr2=0x7f77e00add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 shutdown_connections 2024-02-02T13:32:03.574 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.572+0000 7f77e7400640 1 -- 172.21.15.91:0/974716743 wait complete. 2024-02-02T13:32:03.595 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:32:03.596 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:03.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.800+0000 7f6c18800640 1 Processor -- start 2024-02-02T13:32:03.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.801+0000 7f6c18800640 1 -- start start 2024-02-02T13:32:03.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c18800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c1015a010 0x7f6c100c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c18800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c100b0700 0x7f6c100b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c18800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100cc7c0 con 0x7f6c100ce130 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c18800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100b10b0 con 0x7f6c1015a010 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c18800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100ce930 con 0x7f6c100b0700 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c15e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c1015a010 0x7f6c100c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c15e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c1015a010 0x7f6c100c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58410/0 (socket says 172.21.15.91:58410) 2024-02-02T13:32:03.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.802+0000 7f6c15e00640 1 -- 172.21.15.91:0/1159445050 learned_addr learned my addr 172.21.15.91:0/1159445050 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.803+0000 7f6c15e00640 1 -- 172.21.15.91:0/1159445050 >> v2:172.21.15.113:3300/0 conn(0x7f6c100b0700 msgr2=0x7f6c100b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.803+0000 7f6c16800640 1 --2- 172.21.15.91:0/1159445050 >> v2:172.21.15.113:3300/0 conn(0x7f6c100b0700 0x7f6c100b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.803+0000 7f6c15e00640 1 --2- 172.21.15.91:0/1159445050 >> v2:172.21.15.113:3300/0 conn(0x7f6c100b0700 0x7f6c100b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.803+0000 7f6c15e00640 1 -- 172.21.15.91:0/1159445050 >> v1:172.21.15.113:6789/0 conn(0x7f6c100ce130 legacy=0x7f6c100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.803+0000 7f6c15e00640 1 -- 172.21.15.91:0/1159445050 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100cd640 con 0x7f6c1015a010 2024-02-02T13:32:03.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.804+0000 7f6c15e00640 1 --2- 172.21.15.91:0/1159445050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c100c9670 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cec10 tx=0x7f6c0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45e8f5f06b6db42 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.804+0000 7f6c14a00640 1 -- 172.21.15.91:0/1159445050 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00031870 con 0x7f6c1015a010 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.804+0000 7f6c14a00640 1 -- 172.21.15.91:0/1159445050 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000319d0 con 0x7f6c1015a010 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.804+0000 7f6c14a00640 1 -- 172.21.15.91:0/1159445050 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00031d60 con 0x7f6c1015a010 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 -- 172.21.15.91:0/1159445050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 msgr2=0x7f6c100c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 --2- 172.21.15.91:0/1159445050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c100c9670 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cec10 tx=0x7f6c0002b910 comp rx=0 tx=0).stop 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 -- 172.21.15.91:0/1159445050 shutdown_connections 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 --2- 172.21.15.91:0/1159445050 >> v2:172.21.15.113:3300/0 conn(0x7f6c100b0700 0x7f6c100b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 --2- 172.21.15.91:0/1159445050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c100c9670 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 -- 172.21.15.91:0/1159445050 >> 172.21.15.91:0/1159445050 conn(0x7f6c100cec80 msgr2=0x7f6c100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 -- 172.21.15.91:0/1159445050 shutdown_connections 2024-02-02T13:32:03.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.805+0000 7f6c18800640 1 -- 172.21.15.91:0/1159445050 wait complete. 2024-02-02T13:32:03.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.806+0000 7f6c18800640 1 Processor -- start 2024-02-02T13:32:03.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.807+0000 7f6c18800640 1 -- start start 2024-02-02T13:32:03.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 0x7f6c100c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c100ce130 0x7f6c101502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101517e0 con 0x7f6c1015a010 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10150ef0 con 0x7f6c100b0700 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c18800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10151190 con 0x7f6c100ce130 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58420/0 (socket says 172.21.15.91:58420) 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 -- 172.21.15.91:0/2804335424 learned_addr learned my addr 172.21.15.91:0/2804335424 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c15e00640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 0x7f6c100c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c15400640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c100ce130 0x7f6c101502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 -- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c100ce130 msgr2=0x7f6c101502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:03.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c100ce130 0x7f6c101502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 -- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 msgr2=0x7f6c100c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 0x7f6c100c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c16800640 1 -- 172.21.15.91:0/2804335424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c000094d0 con 0x7f6c1015a010 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.808+0000 7f6c15e00640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 0x7f6c100c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c16800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800cd20 tx=0x7f6c0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003b90 con 0x7f6c1015a010 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c08003cf0 con 0x7f6c1015a010 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10151470 con 0x7f6c1015a010 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08004240 con 0x7f6c1015a010 2024-02-02T13:32:03.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.809+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c101507e0 con 0x7f6c1015a010 2024-02-02T13:32:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.810+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c10151990 con 0x7f6c1015a010 2024-02-02T13:32:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.811+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c080043a0 con 0x7f6c1015a010 2024-02-02T13:32:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.812+0000 7f6c0e000640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.812+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c08012070 con 0x7f6c1015a010 2024-02-02T13:32:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.812+0000 7f6c15e00640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:03.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.813+0000 7f6c15e00640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 0x7f6be806c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c0003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:03.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:03.816+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c10151990 con 0x7f6c1015a010 2024-02-02T13:32:04.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.027+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c10066cf0 con 0x7f6c1015a010 2024-02-02T13:32:04.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.028+0000 7f6c0e000640 1 -- 172.21.15.91:0/2804335424 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f6c08054840 con 0x7f6c1015a010 2024-02-02T13:32:04.030 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:04.030 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:04.030 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 msgr2=0x7f6be806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 0x7f6be806c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c0003c040 comp rx=0 tx=0).stop 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 msgr2=0x7f6c1014c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800cd20 tx=0x7f6c0800a660 comp rx=0 tx=0).stop 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 shutdown_connections 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c100ce130 0x7f6c101502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c1015a010 0x7f6c1014c640 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 --2- 172.21.15.91:0/2804335424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c100b0700 0x7f6c100c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.030+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 >> 172.21.15.91:0/2804335424 conn(0x7f6c100cec80 msgr2=0x7f6c10070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:04.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.031+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 shutdown_connections 2024-02-02T13:32:04.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.031+0000 7f6c18800640 1 -- 172.21.15.91:0/2804335424 wait complete. 2024-02-02T13:32:04.049 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:04.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.243+0000 7f2ac7800640 1 Processor -- start 2024-02-02T13:32:04.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.243+0000 7f2ac7800640 1 -- start start 2024-02-02T13:32:04.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac7800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2ac00b0e10 0x7f2ac00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac7800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac015d870 con 0x7f2ac00ac8d0 2024-02-02T13:32:04.247 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac7800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac00ad270 con 0x7f2ac00b0e10 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac006bce0 con 0x7f2ac015d490 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2ac5800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2ac00b0e10 0x7f2ac00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2abfe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2ac00ac8d0 0x7f2ac00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45458/0 (socket says 172.21.15.91:45458) 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.245+0000 7f2abfe00640 1 -- 172.21.15.91:0/1108132970 learned_addr learned my addr 172.21.15.91:0/1108132970 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2ac4e00640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2ac5800640 1 -- 172.21.15.91:0/1108132970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 msgr2=0x7f2ac006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2ac5800640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2ac5800640 1 -- 172.21.15.91:0/1108132970 >> v1:172.21.15.91:6789/0 conn(0x7f2ac00ac8d0 legacy=0x7f2ac00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2abf400640 1 -- 172.21.15.91:0/1108132970 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292260780 0 0) 0x7f2ac015d870 con 0x7f2ac00ac8d0 2024-02-02T13:32:04.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.246+0000 7f2ac5800640 1 -- 172.21.15.91:0/1108132970 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0161220 con 0x7f2ac00b0e10 2024-02-02T13:32:04.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.247+0000 7f2ac5800640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00acd30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009d90 tx=0x7f2ab802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eb90300ae39a68 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2abf400640 1 -- 172.21.15.91:0/1108132970 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802cc50 con 0x7f2ac00b0e10 2024-02-02T13:32:04.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2abf400640 1 -- 172.21.15.91:0/1108132970 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab802cdb0 con 0x7f2ac00b0e10 2024-02-02T13:32:04.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2abf400640 1 -- 172.21.15.91:0/1108132970 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8034660 con 0x7f2ac00b0e10 2024-02-02T13:32:04.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 -- 172.21.15.91:0/1108132970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 msgr2=0x7f2ac00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00acd30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009d90 tx=0x7f2ab802bea0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 -- 172.21.15.91:0/1108132970 shutdown_connections 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00acd30 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 --2- 172.21.15.91:0/1108132970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 -- 172.21.15.91:0/1108132970 >> 172.21.15.91:0/1108132970 conn(0x7f2ac00cec80 msgr2=0x7f2ac00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 -- 172.21.15.91:0/1108132970 shutdown_connections 2024-02-02T13:32:04.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.248+0000 7f2ac7800640 1 -- 172.21.15.91:0/1108132970 wait complete. 2024-02-02T13:32:04.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.250+0000 7f2ac7800640 1 Processor -- start 2024-02-02T13:32:04.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.250+0000 7f2ac7800640 1 -- start start 2024-02-02T13:32:04.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac00ac8d0 0x7f2ac014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac00b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac014e6e0 con 0x7f2ac00b0e10 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00b21c0 con 0x7f2ac00ac8d0 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2ac7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00b24a0 con 0x7f2ac015d490 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.251+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58436/0 (socket says 172.21.15.91:58436) 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 -- 172.21.15.91:0/67304705 learned_addr learned my addr 172.21.15.91:0/67304705 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2ac4e00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac00ac8d0 0x7f2ac014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2ac5800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac00b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 -- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 msgr2=0x7f2ac00b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac00b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 -- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac00ac8d0 msgr2=0x7f2ac014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac00ac8d0 0x7f2ac014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 -- 172.21.15.91:0/67304705 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8009a70 con 0x7f2ac00b0e10 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abfe00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c430 tx=0x7f2aa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.252+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80074e0 con 0x7f2ac00b0e10 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.253+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8007660 con 0x7f2ac00b0e10 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.253+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8004a20 con 0x7f2ac00b0e10 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.253+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac00b2780 con 0x7f2ac00b0e10 2024-02-02T13:32:04.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.253+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac00b2d40 con 0x7f2ac00b0e10 2024-02-02T13:32:04.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.254+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac0066cf0 con 0x7f2ac00b0e10 2024-02-02T13:32:04.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.255+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2aa8004b80 con 0x7f2ac00b0e10 2024-02-02T13:32:04.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.256+0000 7f2abcc00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 0x7f2a9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.256+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aa8088760 con 0x7f2ac00b0e10 2024-02-02T13:32:04.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.256+0000 7f2ac4e00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 0x7f2a9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.257+0000 7f2ac4e00640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 0x7f2a9806c8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006fd0 tx=0x7f2ab0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.260+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2aa8053d40 con 0x7f2ac00b0e10 2024-02-02T13:32:04.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.453+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ac00cb470 con 0x7f2ac00b0e10 2024-02-02T13:32:04.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.456+0000 7f2abcc00640 1 -- 172.21.15.91:0/67304705 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16060 (secure 0 0 0) 0x7f2aa8057360 con 0x7f2ac00b0e10 2024-02-02T13:32:04.458 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:04.458 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":143,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:01.549761+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 msgr2=0x7f2a9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 0x7f2a9806c8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0006fd0 tx=0x7f2ab0008040 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 msgr2=0x7f2ac00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c430 tx=0x7f2aa800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 shutdown_connections 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2a9806a430 0x7f2a9806c8f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ac015d490 0x7f2ac00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ac00b0e10 0x7f2ac00b3260 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 --2- 172.21.15.91:0/67304705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac00ac8d0 0x7f2ac014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 >> 172.21.15.91:0/67304705 conn(0x7f2ac00cec80 msgr2=0x7f2ac00cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 shutdown_connections 2024-02-02T13:32:04.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.459+0000 7f2ac7800640 1 -- 172.21.15.91:0/67304705 wait complete. 2024-02-02T13:32:04.479 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:32:04.479 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:32:04.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.684+0000 7faf78e00640 1 Processor -- start 2024-02-02T13:32:04.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.684+0000 7faf78e00640 1 -- start start 2024-02-02T13:32:04.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf78e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf7415aec0 0x7faf740cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf78e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf7415b2a0 0x7faf7406c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf78e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7406c1c0 con 0x7faf7406cb90 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf78e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7406b5f0 con 0x7faf7415aec0 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf78e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7406b890 con 0x7faf7415b2a0 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf73400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf7415aec0 0x7faf740cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf73400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf7415aec0 0x7faf740cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58452/0 (socket says 172.21.15.91:58452) 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf73400640 1 -- 172.21.15.91:0/1489999756 learned_addr learned my addr 172.21.15.91:0/1489999756 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.686+0000 7faf6be00640 1 --2- 172.21.15.91:0/1489999756 >> v2:172.21.15.113:3300/0 conn(0x7faf7415b2a0 0x7faf7406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.687+0000 7faf73400640 1 -- 172.21.15.91:0/1489999756 >> v2:172.21.15.113:3300/0 conn(0x7faf7415b2a0 msgr2=0x7faf7406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:04.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.687+0000 7faf73400640 1 --2- 172.21.15.91:0/1489999756 >> v2:172.21.15.113:3300/0 conn(0x7faf7415b2a0 0x7faf7406c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.687+0000 7faf73400640 1 -- 172.21.15.91:0/1489999756 >> v1:172.21.15.113:6789/0 conn(0x7faf7406cb90 legacy=0x7faf740ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:04.689 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.687+0000 7faf73400640 1 -- 172.21.15.91:0/1489999756 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7406bb70 con 0x7faf7415aec0 2024-02-02T13:32:04.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.688+0000 7faf73400640 1 --2- 172.21.15.91:0/1489999756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7415aec0 0x7faf740cf5b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faf7406ee00 tx=0x7faf6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7bf84669e33002 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.689+0000 7faf72a00640 1 -- 172.21.15.91:0/1489999756 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64030860 con 0x7faf7415aec0 2024-02-02T13:32:04.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.689+0000 7faf72a00640 1 -- 172.21.15.91:0/1489999756 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf640309c0 con 0x7faf7415aec0 2024-02-02T13:32:04.690 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.689+0000 7faf72a00640 1 -- 172.21.15.91:0/1489999756 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64030d50 con 0x7faf7415aec0 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 -- 172.21.15.91:0/1489999756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7415aec0 msgr2=0x7faf740cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 --2- 172.21.15.91:0/1489999756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7415aec0 0x7faf740cf5b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faf7406ee00 tx=0x7faf6402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 -- 172.21.15.91:0/1489999756 shutdown_connections 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 --2- 172.21.15.91:0/1489999756 >> v2:172.21.15.113:3300/0 conn(0x7faf7415b2a0 0x7faf7406c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 --2- 172.21.15.91:0/1489999756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7415aec0 0x7faf740cf5b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 -- 172.21.15.91:0/1489999756 >> 172.21.15.91:0/1489999756 conn(0x7faf740cea90 msgr2=0x7faf740ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 -- 172.21.15.91:0/1489999756 shutdown_connections 2024-02-02T13:32:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.690+0000 7faf78e00640 1 -- 172.21.15.91:0/1489999756 wait complete. 2024-02-02T13:32:04.693 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.692+0000 7faf78e00640 1 Processor -- start 2024-02-02T13:32:04.693 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.692+0000 7faf78e00640 1 -- start start 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 0x7faf7415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7415b2a0 0x7faf7414aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c54f0 con 0x7faf7406cb90 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c2ca0 con 0x7faf7415aec0 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.693+0000 7faf78e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c29c0 con 0x7faf7415b2a0 2024-02-02T13:32:04.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.696 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58462/0 (socket says 172.21.15.91:58462) 2024-02-02T13:32:04.696 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 -- 172.21.15.91:0/3083968902 learned_addr learned my addr 172.21.15.91:0/3083968902 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf6be00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 0x7faf7415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 -- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7415b2a0 msgr2=0x7faf7414aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7415b2a0 0x7faf7414aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7415b2a0 0x7faf7414aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 -- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 msgr2=0x7faf7415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 0x7faf7415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 -- 172.21.15.91:0/3083968902 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640094d0 con 0x7faf7406cb90 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf6be00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 0x7faf7415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:04.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.694+0000 7faf73400640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7faf64000c00 tx=0x7faf64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.695+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64039c90 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.695+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64039df0 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.695+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64043620 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.695+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740c4ea0 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.695+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf74152d70 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.696+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7404dff0 con 0x7faf7406cb90 2024-02-02T13:32:04.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.697+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf64043780 con 0x7faf7406cb90 2024-02-02T13:32:04.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.698+0000 7faf6b400640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 0x7faf4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:04.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.699+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf6403e070 con 0x7faf7406cb90 2024-02-02T13:32:04.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.699+0000 7faf6be00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 0x7faf4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:04.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.699+0000 7faf6be00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 0x7faf4c06c8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf5c006fd0 tx=0x7faf5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:04.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.703+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf640770f0 con 0x7faf7406cb90 2024-02-02T13:32:04.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.891+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf7406be50 con 0x7faf4c06a430 2024-02-02T13:32:04.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.914+0000 7faf6b400640 1 -- 172.21.15.91:0/3083968902 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283140 (secure 0 0 0) 0x7faf5c009550 con 0x7faf4c06a430 2024-02-02T13:32:04.920 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:04.921 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:32:04.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 msgr2=0x7faf4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 0x7faf4c06c8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf5c006fd0 tx=0x7faf5c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:04.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 msgr2=0x7faf7414a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:04.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7faf64000c00 tx=0x7faf64004730 comp rx=0 tx=0).stop 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 shutdown_connections 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf4c06a430 0x7faf4c06c8f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7415b2a0 0x7faf7414aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7415aec0 0x7faf7415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 --2- 172.21.15.91:0/3083968902 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf7406cb90 0x7faf7414a270 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 >> 172.21.15.91:0/3083968902 conn(0x7faf740cea90 msgr2=0x7faf74070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 shutdown_connections 2024-02-02T13:32:04.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:04.920+0000 7faf78e00640 1 -- 172.21.15.91:0/3083968902 wait complete. 2024-02-02T13:32:04.939 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-02-02T13:32:04.381818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7289,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555210,"statfs":{"total":1600000000,"available":1592535623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.331680"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582181+0000","last_change":"2024-02-02T13:32:00.582114+0000","last_active":"2024-02-02T13:32:00.582181+0000","last_peered":"2024-02-02T13:32:00.582181+0000","last_clean":"2024-02-02T13:32:00.582181+0000","last_became_active":"2024-02-02T13:32:00.581837+0000","last_became_peered":"2024-02-02T13:32:00.581837+0000","last_unstale":"2024-02-02T13:32:00.582181+0000","last_undegraded":"2024-02-02T13:32:00.582181+0000","last_fullsized":"2024-02-02T13:32:00.582181+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:06:12.560424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.584011+0000","last_change":"2024-02-02T13:32:00.583918+0000","last_active":"2024-02-02T13:32:00.584011+0000","last_peered":"2024-02-02T13:32:00.584011+0000","last_clean":"2024-02-02T13:32:00.584011+0000","last_became_active":"2024-02-02T13:32:00.583015+0000","last_became_peered":"2024-02-02T13:32:00.583015+0000","last_unstale":"2024-02-02T13:32:00.584011+0000","last_undegraded":"2024-02-02T13:32:00.584011+0000","last_fullsized":"2024-02-02T13:32:00.584011+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:24:03.225415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578591+0000","last_change":"2024-02-02T13:32:00.575269+0000","last_active":"2024-02-02T13:32:00.578591+0000","last_peered":"2024-02-02T13:32:00.578591+0000","last_clean":"2024-02-02T13:32:00.578591+0000","last_became_active":"2024-02-02T13:32:00.573886+0000","last_became_peered":"2024-02-02T13:32:00.573886+0000","last_unstale":"2024-02-02T13:32:00.578591+0000","last_undegraded":"2024-02-02T13:32:00.578591+0000","last_fullsized":"2024-02-02T13:32:00.578591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:47:44.521870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.575517+0000","last_change":"2024-02-02T13:32:00.574342+0000","last_active":"2024-02-02T13:32:00.575517+0000","last_peered":"2024-02-02T13:32:00.575517+0000","last_clean":"2024-02-02T13:32:00.575517+0000","last_became_active":"2024-02-02T13:32:00.573989+0000","last_became_peered":"2024-02-02T13:32:00.573989+0000","last_unstale":"2024-02-02T13:32:00.575517+0000","last_undegraded":"2024-02-02T13:32:00.575517+0000","last_fullsized":"2024-02-02T13:32:00.575517+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:23.712750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568785+0000","last_change":"2024-02-02T13:32:00.566419+0000","last_active":"2024-02-02T13:32:00.568785+0000","last_peered":"2024-02-02T13:32:00.568785+0000","last_clean":"2024-02-02T13:32:00.568785+0000","last_became_active":"2024-02-02T13:32:00.566216+0000","last_became_peered":"2024-02-02T13:32:00.566216+0000","last_unstale":"2024-02-02T13:32:00.568785+0000","last_undegraded":"2024-02-02T13:32:00.568785+0000","last_fullsized":"2024-02-02T13:32:00.568785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:29:00.525568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566227+0000","last_change":"2024-02-02T13:32:00.564894+0000","last_active":"2024-02-02T13:32:00.566227+0000","last_peered":"2024-02-02T13:32:00.566227+0000","last_clean":"2024-02-02T13:32:00.566227+0000","last_became_active":"2024-02-02T13:32:00.564507+0000","last_became_peered":"2024-02-02T13:32:00.564507+0000","last_unstale":"2024-02-02T13:32:00.566227+0000","last_undegraded":"2024-02-02T13:32:00.566227+0000","last_fullsized":"2024-02-02T13:32:00.566227+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:42.546275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580468+0000","last_change":"2024-02-02T13:32:00.579619+0000","last_active":"2024-02-02T13:32:00.580468+0000","last_peered":"2024-02-02T13:32:00.580468+0000","last_clean":"2024-02-02T13:32:00.580468+0000","last_became_active":"2024-02-02T13:32:00.579242+0000","last_became_peered":"2024-02-02T13:32:00.579242+0000","last_unstale":"2024-02-02T13:32:00.580468+0000","last_undegraded":"2024-02-02T13:32:00.580468+0000","last_fullsized":"2024-02-02T13:32:00.580468+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:09:17.065112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578643+0000","last_change":"2024-02-02T13:32:00.576042+0000","last_active":"2024-02-02T13:32:00.578643+0000","last_peered":"2024-02-02T13:32:00.578643+0000","last_clean":"2024-02-02T13:32:00.578643+0000","last_became_active":"2024-02-02T13:32:00.574323+0000","last_became_peered":"2024-02-02T13:32:00.574323+0000","last_unstale":"2024-02-02T13:32:00.578643+0000","last_undegraded":"2024-02-02T13:32:00.578643+0000","last_fullsized":"2024-02-02T13:32:00.578643+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:27.030798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568204+0000","last_change":"2024-02-02T13:32:00.564327+0000","last_active":"2024-02-02T13:32:00.568204+0000","last_peered":"2024-02-02T13:32:00.568204+0000","last_clean":"2024-02-02T13:32:00.568204+0000","last_became_active":"2024-02-02T13:32:00.564174+0000","last_became_peered":"2024-02-02T13:32:00.564174+0000","last_unstale":"2024-02-02T13:32:00.568204+0000","last_undegraded":"2024-02-02T13:32:00.568204+0000","last_fullsized":"2024-02-02T13:32:00.568204+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:19:28.741542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572316+0000","last_change":"2024-02-02T13:32:00.571424+0000","last_active":"2024-02-02T13:32:00.572316+0000","last_peered":"2024-02-02T13:32:00.572316+0000","last_clean":"2024-02-02T13:32:00.572316+0000","last_became_active":"2024-02-02T13:32:00.570955+0000","last_became_peered":"2024-02-02T13:32:00.570955+0000","last_unstale":"2024-02-02T13:32:00.572316+0000","last_undegraded":"2024-02-02T13:32:00.572316+0000","last_fullsized":"2024-02-02T13:32:00.572316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:48:11.865231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583551+0000","last_change":"2024-02-02T13:32:00.583192+0000","last_active":"2024-02-02T13:32:00.583551+0000","last_peered":"2024-02-02T13:32:00.583551+0000","last_clean":"2024-02-02T13:32:00.583551+0000","last_became_active":"2024-02-02T13:32:00.582964+0000","last_became_peered":"2024-02-02T13:32:00.582964+0000","last_unstale":"2024-02-02T13:32:00.583551+0000","last_undegraded":"2024-02-02T13:32:00.583551+0000","last_fullsized":"2024-02-02T13:32:00.583551+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:37.598898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573749+0000","last_change":"2024-02-02T13:32:00.572710+0000","last_active":"2024-02-02T13:32:00.573749+0000","last_peered":"2024-02-02T13:32:00.573749+0000","last_clean":"2024-02-02T13:32:00.573749+0000","last_became_active":"2024-02-02T13:32:00.572494+0000","last_became_peered":"2024-02-02T13:32:00.572494+0000","last_unstale":"2024-02-02T13:32:00.573749+0000","last_undegraded":"2024-02-02T13:32:00.573749+0000","last_fullsized":"2024-02-02T13:32:00.573749+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:33:21.843675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578866+0000","last_change":"2024-02-02T13:32:00.577217+0000","last_active":"2024-02-02T13:32:00.578866+0000","last_peered":"2024-02-02T13:32:00.578866+0000","last_clean":"2024-02-02T13:32:00.578866+0000","last_became_active":"2024-02-02T13:32:00.574853+0000","last_became_peered":"2024-02-02T13:32:00.574853+0000","last_unstale":"2024-02-02T13:32:00.578866+0000","last_undegraded":"2024-02-02T13:32:00.578866+0000","last_fullsized":"2024-02-02T13:32:00.578866+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:17.955007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581712+0000","last_change":"2024-02-02T13:32:00.581102+0000","last_active":"2024-02-02T13:32:00.581712+0000","last_peered":"2024-02-02T13:32:00.581712+0000","last_clean":"2024-02-02T13:32:00.581712+0000","last_became_active":"2024-02-02T13:32:00.579971+0000","last_became_peered":"2024-02-02T13:32:00.579971+0000","last_unstale":"2024-02-02T13:32:00.581712+0000","last_undegraded":"2024-02-02T13:32:00.581712+0000","last_fullsized":"2024-02-02T13:32:00.581712+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:41:52.272275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566140+0000","last_change":"2024-02-02T13:32:00.560426+0000","last_active":"2024-02-02T13:32:00.566140+0000","last_peered":"2024-02-02T13:32:00.566140+0000","last_clean":"2024-02-02T13:32:00.566140+0000","last_became_active":"2024-02-02T13:32:00.560188+0000","last_became_peered":"2024-02-02T13:32:00.560188+0000","last_unstale":"2024-02-02T13:32:00.566140+0000","last_undegraded":"2024-02-02T13:32:00.566140+0000","last_fullsized":"2024-02-02T13:32:00.566140+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:31:42.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581674+0000","last_change":"2024-02-02T13:32:00.581610+0000","last_active":"2024-02-02T13:32:00.581674+0000","last_peered":"2024-02-02T13:32:00.581674+0000","last_clean":"2024-02-02T13:32:00.581674+0000","last_became_active":"2024-02-02T13:32:00.581328+0000","last_became_peered":"2024-02-02T13:32:00.581328+0000","last_unstale":"2024-02-02T13:32:00.581674+0000","last_undegraded":"2024-02-02T13:32:00.581674+0000","last_fullsized":"2024-02-02T13:32:00.581674+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:24:05.692536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583570+0000","last_change":"2024-02-02T13:32:00.583252+0000","last_active":"2024-02-02T13:32:00.583570+0000","last_peered":"2024-02-02T13:32:00.583570+0000","last_clean":"2024-02-02T13:32:00.583570+0000","last_became_active":"2024-02-02T13:32:00.581002+0000","last_became_peered":"2024-02-02T13:32:00.581002+0000","last_unstale":"2024-02-02T13:32:00.583570+0000","last_undegraded":"2024-02-02T13:32:00.583570+0000","last_fullsized":"2024-02-02T13:32:00.583570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:34.984416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582848+0000","last_change":"2024-02-02T13:32:00.581398+0000","last_active":"2024-02-02T13:32:00.582848+0000","last_peered":"2024-02-02T13:32:00.582848+0000","last_clean":"2024-02-02T13:32:00.582848+0000","last_became_active":"2024-02-02T13:32:00.580621+0000","last_became_peered":"2024-02-02T13:32:00.580621+0000","last_unstale":"2024-02-02T13:32:00.582848+0000","last_undegraded":"2024-02-02T13:32:00.582848+0000","last_fullsized":"2024-02-02T13:32:00.582848+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:45.721887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580119+0000","last_change":"2024-02-02T13:32:00.579565+0000","last_active":"2024-02-02T13:32:00.580119+0000","last_peered":"2024-02-02T13:32:00.580119+0000","last_clean":"2024-02-02T13:32:00.580119+0000","last_became_active":"2024-02-02T13:32:00.579061+0000","last_became_peered":"2024-02-02T13:32:00.579061+0000","last_unstale":"2024-02-02T13:32:00.580119+0000","last_undegraded":"2024-02-02T13:32:00.580119+0000","last_fullsized":"2024-02-02T13:32:00.580119+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:59.245267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579518+0000","last_change":"2024-02-02T13:32:00.579429+0000","last_active":"2024-02-02T13:32:00.579518+0000","last_peered":"2024-02-02T13:32:00.579518+0000","last_clean":"2024-02-02T13:32:00.579518+0000","last_became_active":"2024-02-02T13:32:00.579202+0000","last_became_peered":"2024-02-02T13:32:00.579202+0000","last_unstale":"2024-02-02T13:32:00.579518+0000","last_undegraded":"2024-02-02T13:32:00.579518+0000","last_fullsized":"2024-02-02T13:32:00.579518+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:06:08.782214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.570651+0000","last_change":"2024-02-02T13:32:00.569953+0000","last_active":"2024-02-02T13:32:00.570651+0000","last_peered":"2024-02-02T13:32:00.570651+0000","last_clean":"2024-02-02T13:32:00.570651+0000","last_became_active":"2024-02-02T13:32:00.569373+0000","last_became_peered":"2024-02-02T13:32:00.569373+0000","last_unstale":"2024-02-02T13:32:00.570651+0000","last_undegraded":"2024-02-02T13:32:00.570651+0000","last_fullsized":"2024-02-02T13:32:00.570651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:50.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579565+0000","last_change":"2024-02-02T13:32:00.579436+0000","last_active":"2024-02-02T13:32:00.579565+0000","last_peered":"2024-02-02T13:32:00.579565+0000","last_clean":"2024-02-02T13:32:00.579565+0000","last_became_active":"2024-02-02T13:32:00.579221+0000","last_became_peered":"2024-02-02T13:32:00.579221+0000","last_unstale":"2024-02-02T13:32:00.579565+0000","last_undegraded":"2024-02-02T13:32:00.579565+0000","last_fullsized":"2024-02-02T13:32:00.579565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:18.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577519+0000","last_change":"2024-02-02T13:32:00.577449+0000","last_active":"2024-02-02T13:32:00.577519+0000","last_peered":"2024-02-02T13:32:00.577519+0000","last_clean":"2024-02-02T13:32:00.577519+0000","last_became_active":"2024-02-02T13:32:00.576891+0000","last_became_peered":"2024-02-02T13:32:00.576891+0000","last_unstale":"2024-02-02T13:32:00.577519+0000","last_undegraded":"2024-02-02T13:32:00.577519+0000","last_fullsized":"2024-02-02T13:32:00.577519+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:35.351007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569489+0000","last_change":"2024-02-02T13:32:00.568508+0000","last_active":"2024-02-02T13:32:00.569489+0000","last_peered":"2024-02-02T13:32:00.569489+0000","last_clean":"2024-02-02T13:32:00.569489+0000","last_became_active":"2024-02-02T13:32:00.568122+0000","last_became_peered":"2024-02-02T13:32:00.568122+0000","last_unstale":"2024-02-02T13:32:00.569489+0000","last_undegraded":"2024-02-02T13:32:00.569489+0000","last_fullsized":"2024-02-02T13:32:00.569489+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:45.330479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581277+0000","last_change":"2024-02-02T13:32:00.581117+0000","last_active":"2024-02-02T13:32:00.581277+0000","last_peered":"2024-02-02T13:32:00.581277+0000","last_clean":"2024-02-02T13:32:00.581277+0000","last_became_active":"2024-02-02T13:32:00.580217+0000","last_became_peered":"2024-02-02T13:32:00.580217+0000","last_unstale":"2024-02-02T13:32:00.581277+0000","last_undegraded":"2024-02-02T13:32:00.581277+0000","last_fullsized":"2024-02-02T13:32:00.581277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:42:53.556889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577922+0000","last_change":"2024-02-02T13:32:00.577387+0000","last_active":"2024-02-02T13:32:00.577922+0000","last_peered":"2024-02-02T13:32:00.577922+0000","last_clean":"2024-02-02T13:32:00.577922+0000","last_became_active":"2024-02-02T13:32:00.573660+0000","last_became_peered":"2024-02-02T13:32:00.573660+0000","last_unstale":"2024-02-02T13:32:00.577922+0000","last_undegraded":"2024-02-02T13:32:00.577922+0000","last_fullsized":"2024-02-02T13:32:00.577922+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:47:59.184897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577829+0000","last_change":"2024-02-02T13:32:00.577513+0000","last_active":"2024-02-02T13:32:00.577829+0000","last_peered":"2024-02-02T13:32:00.577829+0000","last_clean":"2024-02-02T13:32:00.577829+0000","last_became_active":"2024-02-02T13:32:00.577306+0000","last_became_peered":"2024-02-02T13:32:00.577306+0000","last_unstale":"2024-02-02T13:32:00.577829+0000","last_undegraded":"2024-02-02T13:32:00.577829+0000","last_fullsized":"2024-02-02T13:32:00.577829+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:47.908651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574734+0000","last_change":"2024-02-02T13:32:00.573796+0000","last_active":"2024-02-02T13:32:00.574734+0000","last_peered":"2024-02-02T13:32:00.574734+0000","last_clean":"2024-02-02T13:32:00.574734+0000","last_became_active":"2024-02-02T13:32:00.573580+0000","last_became_peered":"2024-02-02T13:32:00.573580+0000","last_unstale":"2024-02-02T13:32:00.574734+0000","last_undegraded":"2024-02-02T13:32:00.574734+0000","last_fullsized":"2024-02-02T13:32:00.574734+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:11.808415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571986+0000","last_change":"2024-02-02T13:32:00.571784+0000","last_active":"2024-02-02T13:32:00.571986+0000","last_peered":"2024-02-02T13:32:00.571986+0000","last_clean":"2024-02-02T13:32:00.571986+0000","last_became_active":"2024-02-02T13:32:00.571416+0000","last_became_peered":"2024-02-02T13:32:00.571416+0000","last_unstale":"2024-02-02T13:32:00.571986+0000","last_undegraded":"2024-02-02T13:32:00.571986+0000","last_fullsized":"2024-02-02T13:32:00.571986+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:29.953865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.564952+0000","last_change":"2024-02-02T13:32:00.564708+0000","last_active":"2024-02-02T13:32:00.564952+0000","last_peered":"2024-02-02T13:32:00.564952+0000","last_clean":"2024-02-02T13:32:00.564952+0000","last_became_active":"2024-02-02T13:32:00.563986+0000","last_became_peered":"2024-02-02T13:32:00.563986+0000","last_unstale":"2024-02-02T13:32:00.564952+0000","last_undegraded":"2024-02-02T13:32:00.564952+0000","last_fullsized":"2024-02-02T13:32:00.564952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:55:24.734834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577903+0000","last_change":"2024-02-02T13:32:00.577227+0000","last_active":"2024-02-02T13:32:00.577903+0000","last_peered":"2024-02-02T13:32:00.577903+0000","last_clean":"2024-02-02T13:32:00.577903+0000","last_became_active":"2024-02-02T13:32:00.573346+0000","last_became_peered":"2024-02-02T13:32:00.573346+0000","last_unstale":"2024-02-02T13:32:00.577903+0000","last_undegraded":"2024-02-02T13:32:00.577903+0000","last_fullsized":"2024-02-02T13:32:00.577903+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:03:26.888341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560160+0000","last_change":"2024-02-02T13:32:00.559340+0000","last_active":"2024-02-02T13:32:00.560160+0000","last_peered":"2024-02-02T13:32:00.560160+0000","last_clean":"2024-02-02T13:32:00.560160+0000","last_became_active":"2024-02-02T13:32:00.559122+0000","last_became_peered":"2024-02-02T13:32:00.559122+0000","last_unstale":"2024-02-02T13:32:00.560160+0000","last_undegraded":"2024-02-02T13:32:00.560160+0000","last_fullsized":"2024-02-02T13:32:00.560160+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:51:44.876724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.576036+0000","last_change":"2024-02-02T13:32:00.575891+0000","last_active":"2024-02-02T13:32:00.576036+0000","last_peered":"2024-02-02T13:32:00.576036+0000","last_clean":"2024-02-02T13:32:00.576036+0000","last_became_active":"2024-02-02T13:32:00.575414+0000","last_became_peered":"2024-02-02T13:32:00.575414+0000","last_unstale":"2024-02-02T13:32:00.576036+0000","last_undegraded":"2024-02-02T13:32:00.576036+0000","last_fullsized":"2024-02-02T13:32:00.576036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:55.558567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"142'4","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580970+0000","last_change":"2024-02-02T13:32:00.079236+0000","last_active":"2024-02-02T13:32:00.580970+0000","last_peered":"2024-02-02T13:32:00.580970+0000","last_clean":"2024-02-02T13:32:00.580970+0000","last_became_active":"2024-02-02T13:32:00.078620+0000","last_became_peered":"2024-02-02T13:32:00.078620+0000","last_unstale":"2024-02-02T13:32:00.580970+0000","last_undegraded":"2024-02-02T13:32:00.580970+0000","last_fullsized":"2024-02-02T13:32:00.580970+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:46.893892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582878+0000","last_change":"2024-02-02T13:32:00.582490+0000","last_active":"2024-02-02T13:32:00.582878+0000","last_peered":"2024-02-02T13:32:00.582878+0000","last_clean":"2024-02-02T13:32:00.582878+0000","last_became_active":"2024-02-02T13:32:00.581686+0000","last_became_peered":"2024-02-02T13:32:00.581686+0000","last_unstale":"2024-02-02T13:32:00.582878+0000","last_undegraded":"2024-02-02T13:32:00.582878+0000","last_fullsized":"2024-02-02T13:32:00.582878+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:24:03.225415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.584122+0000","last_change":"2024-02-02T13:32:00.077561+0000","last_active":"2024-02-02T13:32:00.584122+0000","last_peered":"2024-02-02T13:32:00.584122+0000","last_clean":"2024-02-02T13:32:00.584122+0000","last_became_active":"2024-02-02T13:32:00.076903+0000","last_became_peered":"2024-02-02T13:32:00.076903+0000","last_unstale":"2024-02-02T13:32:00.584122+0000","last_undegraded":"2024-02-02T13:32:00.584122+0000","last_fullsized":"2024-02-02T13:32:00.584122+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:49.142735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566194+0000","last_change":"2024-02-02T13:32:00.564733+0000","last_active":"2024-02-02T13:32:00.566194+0000","last_peered":"2024-02-02T13:32:00.566194+0000","last_clean":"2024-02-02T13:32:00.566194+0000","last_became_active":"2024-02-02T13:32:00.564157+0000","last_became_peered":"2024-02-02T13:32:00.564157+0000","last_unstale":"2024-02-02T13:32:00.566194+0000","last_undegraded":"2024-02-02T13:32:00.566194+0000","last_fullsized":"2024-02-02T13:32:00.566194+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:32:33.903977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578791+0000","last_change":"2024-02-02T13:32:00.096516+0000","last_active":"2024-02-02T13:32:00.578791+0000","last_peered":"2024-02-02T13:32:00.578791+0000","last_clean":"2024-02-02T13:32:00.578791+0000","last_became_active":"2024-02-02T13:32:00.096282+0000","last_became_peered":"2024-02-02T13:32:00.096282+0000","last_unstale":"2024-02-02T13:32:00.578791+0000","last_undegraded":"2024-02-02T13:32:00.578791+0000","last_fullsized":"2024-02-02T13:32:00.578791+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:34:49.959878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578849+0000","last_change":"2024-02-02T13:32:00.577094+0000","last_active":"2024-02-02T13:32:00.578849+0000","last_peered":"2024-02-02T13:32:00.578849+0000","last_clean":"2024-02-02T13:32:00.578849+0000","last_became_active":"2024-02-02T13:32:00.575937+0000","last_became_peered":"2024-02-02T13:32:00.575937+0000","last_unstale":"2024-02-02T13:32:00.578849+0000","last_undegraded":"2024-02-02T13:32:00.578849+0000","last_fullsized":"2024-02-02T13:32:00.578849+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:11.808415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580865+0000","last_change":"2024-02-02T13:32:00.090475+0000","last_active":"2024-02-02T13:32:00.580865+0000","last_peered":"2024-02-02T13:32:00.580865+0000","last_clean":"2024-02-02T13:32:00.580865+0000","last_became_active":"2024-02-02T13:32:00.089886+0000","last_became_peered":"2024-02-02T13:32:00.089886+0000","last_unstale":"2024-02-02T13:32:00.580865+0000","last_undegraded":"2024-02-02T13:32:00.580865+0000","last_fullsized":"2024-02-02T13:32:00.580865+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:34.636450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577937+0000","last_change":"2024-02-02T13:32:00.577540+0000","last_active":"2024-02-02T13:32:00.577937+0000","last_peered":"2024-02-02T13:32:00.577937+0000","last_clean":"2024-02-02T13:32:00.577937+0000","last_became_active":"2024-02-02T13:32:00.575838+0000","last_became_peered":"2024-02-02T13:32:00.575838+0000","last_unstale":"2024-02-02T13:32:00.577937+0000","last_undegraded":"2024-02-02T13:32:00.577937+0000","last_fullsized":"2024-02-02T13:32:00.577937+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:39.975516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579409+0000","last_change":"2024-02-02T13:32:00.090351+0000","last_active":"2024-02-02T13:32:00.579409+0000","last_peered":"2024-02-02T13:32:00.579409+0000","last_clean":"2024-02-02T13:32:00.579409+0000","last_became_active":"2024-02-02T13:32:00.087899+0000","last_became_peered":"2024-02-02T13:32:00.087899+0000","last_unstale":"2024-02-02T13:32:00.579409+0000","last_undegraded":"2024-02-02T13:32:00.579409+0000","last_fullsized":"2024-02-02T13:32:00.579409+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:41.531516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583713+0000","last_change":"2024-02-02T13:32:00.583636+0000","last_active":"2024-02-02T13:32:00.583713+0000","last_peered":"2024-02-02T13:32:00.583713+0000","last_clean":"2024-02-02T13:32:00.583713+0000","last_became_active":"2024-02-02T13:32:00.582063+0000","last_became_peered":"2024-02-02T13:32:00.582063+0000","last_unstale":"2024-02-02T13:32:00.583713+0000","last_undegraded":"2024-02-02T13:32:00.583713+0000","last_fullsized":"2024-02-02T13:32:00.583713+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:38:11.453071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580913+0000","last_change":"2024-02-02T13:32:00.094741+0000","last_active":"2024-02-02T13:32:00.580913+0000","last_peered":"2024-02-02T13:32:00.580913+0000","last_clean":"2024-02-02T13:32:00.580913+0000","last_became_active":"2024-02-02T13:32:00.094330+0000","last_became_peered":"2024-02-02T13:32:00.094330+0000","last_unstale":"2024-02-02T13:32:00.580913+0000","last_undegraded":"2024-02-02T13:32:00.580913+0000","last_fullsized":"2024-02-02T13:32:00.580913+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:12:55.870178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572301+0000","last_change":"2024-02-02T13:32:00.571300+0000","last_active":"2024-02-02T13:32:00.572301+0000","last_peered":"2024-02-02T13:32:00.572301+0000","last_clean":"2024-02-02T13:32:00.572301+0000","last_became_active":"2024-02-02T13:32:00.570646+0000","last_became_peered":"2024-02-02T13:32:00.570646+0000","last_unstale":"2024-02-02T13:32:00.572301+0000","last_undegraded":"2024-02-02T13:32:00.572301+0000","last_fullsized":"2024-02-02T13:32:00.572301+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:36:05.151351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562794+0000","last_change":"2024-02-02T13:32:00.090412+0000","last_active":"2024-02-02T13:32:00.562794+0000","last_peered":"2024-02-02T13:32:00.562794+0000","last_clean":"2024-02-02T13:32:00.562794+0000","last_became_active":"2024-02-02T13:32:00.090122+0000","last_became_peered":"2024-02-02T13:32:00.090122+0000","last_unstale":"2024-02-02T13:32:00.562794+0000","last_undegraded":"2024-02-02T13:32:00.562794+0000","last_fullsized":"2024-02-02T13:32:00.562794+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:39.901199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581234+0000","last_change":"2024-02-02T13:32:00.580995+0000","last_active":"2024-02-02T13:32:00.581234+0000","last_peered":"2024-02-02T13:32:00.581234+0000","last_clean":"2024-02-02T13:32:00.581234+0000","last_became_active":"2024-02-02T13:32:00.580767+0000","last_became_peered":"2024-02-02T13:32:00.580767+0000","last_unstale":"2024-02-02T13:32:00.581234+0000","last_undegraded":"2024-02-02T13:32:00.581234+0000","last_fullsized":"2024-02-02T13:32:00.581234+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:22:40.859966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.563399+0000","last_change":"2024-02-02T13:32:00.097067+0000","last_active":"2024-02-02T13:32:00.563399+0000","last_peered":"2024-02-02T13:32:00.563399+0000","last_clean":"2024-02-02T13:32:00.563399+0000","last_became_active":"2024-02-02T13:32:00.096803+0000","last_became_peered":"2024-02-02T13:32:00.096803+0000","last_unstale":"2024-02-02T13:32:00.563399+0000","last_undegraded":"2024-02-02T13:32:00.563399+0000","last_fullsized":"2024-02-02T13:32:00.563399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:29.156726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573961+0000","last_change":"2024-02-02T13:32:00.571908+0000","last_active":"2024-02-02T13:32:00.573961+0000","last_peered":"2024-02-02T13:32:00.573961+0000","last_clean":"2024-02-02T13:32:00.573961+0000","last_became_active":"2024-02-02T13:32:00.571350+0000","last_became_peered":"2024-02-02T13:32:00.571350+0000","last_unstale":"2024-02-02T13:32:00.573961+0000","last_undegraded":"2024-02-02T13:32:00.573961+0000","last_fullsized":"2024-02-02T13:32:00.573961+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:23:59.823167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.561475+0000","last_change":"2024-02-02T13:32:00.090227+0000","last_active":"2024-02-02T13:32:00.561475+0000","last_peered":"2024-02-02T13:32:00.561475+0000","last_clean":"2024-02-02T13:32:00.561475+0000","last_became_active":"2024-02-02T13:32:00.089426+0000","last_became_peered":"2024-02-02T13:32:00.089426+0000","last_unstale":"2024-02-02T13:32:00.561475+0000","last_undegraded":"2024-02-02T13:32:00.561475+0000","last_fullsized":"2024-02-02T13:32:00.561475+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:56:44.272405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582863+0000","last_change":"2024-02-02T13:32:00.580509+0000","last_active":"2024-02-02T13:32:00.582863+0000","last_peered":"2024-02-02T13:32:00.582863+0000","last_clean":"2024-02-02T13:32:00.582863+0000","last_became_active":"2024-02-02T13:32:00.578471+0000","last_became_peered":"2024-02-02T13:32:00.578471+0000","last_unstale":"2024-02-02T13:32:00.582863+0000","last_undegraded":"2024-02-02T13:32:00.582863+0000","last_fullsized":"2024-02-02T13:32:00.582863+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:59.311899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.552101+0000","last_change":"2024-02-02T13:32:00.091859+0000","last_active":"2024-02-02T13:32:00.552101+0000","last_peered":"2024-02-02T13:32:00.552101+0000","last_clean":"2024-02-02T13:32:00.552101+0000","last_became_active":"2024-02-02T13:32:00.091294+0000","last_became_peered":"2024-02-02T13:32:00.091294+0000","last_unstale":"2024-02-02T13:32:00.552101+0000","last_undegraded":"2024-02-02T13:32:00.552101+0000","last_fullsized":"2024-02-02T13:32:00.552101+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:49.142735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569968+0000","last_change":"2024-02-02T13:32:00.569903+0000","last_active":"2024-02-02T13:32:00.569968+0000","last_peered":"2024-02-02T13:32:00.569968+0000","last_clean":"2024-02-02T13:32:00.569968+0000","last_became_active":"2024-02-02T13:32:00.569700+0000","last_became_peered":"2024-02-02T13:32:00.569700+0000","last_unstale":"2024-02-02T13:32:00.569968+0000","last_undegraded":"2024-02-02T13:32:00.569968+0000","last_fullsized":"2024-02-02T13:32:00.569968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:59.245267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581217+0000","last_change":"2024-02-02T13:32:00.089704+0000","last_active":"2024-02-02T13:32:00.581217+0000","last_peered":"2024-02-02T13:32:00.581217+0000","last_clean":"2024-02-02T13:32:00.581217+0000","last_became_active":"2024-02-02T13:32:00.089447+0000","last_became_peered":"2024-02-02T13:32:00.089447+0000","last_unstale":"2024-02-02T13:32:00.581217+0000","last_undegraded":"2024-02-02T13:32:00.581217+0000","last_fullsized":"2024-02-02T13:32:00.581217+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:59:08.318315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.570316+0000","last_change":"2024-02-02T13:32:00.569392+0000","last_active":"2024-02-02T13:32:00.570316+0000","last_peered":"2024-02-02T13:32:00.570316+0000","last_clean":"2024-02-02T13:32:00.570316+0000","last_became_active":"2024-02-02T13:32:00.568960+0000","last_became_peered":"2024-02-02T13:32:00.568960+0000","last_unstale":"2024-02-02T13:32:00.570316+0000","last_undegraded":"2024-02-02T13:32:00.570316+0000","last_fullsized":"2024-02-02T13:32:00.570316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:58:12.612972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582144+0000","last_change":"2024-02-02T13:32:00.092756+0000","last_active":"2024-02-02T13:32:00.582144+0000","last_peered":"2024-02-02T13:32:00.582144+0000","last_clean":"2024-02-02T13:32:00.582144+0000","last_became_active":"2024-02-02T13:32:00.092077+0000","last_became_peered":"2024-02-02T13:32:00.092077+0000","last_unstale":"2024-02-02T13:32:00.582144+0000","last_undegraded":"2024-02-02T13:32:00.582144+0000","last_fullsized":"2024-02-02T13:32:00.582144+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:58.230507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.563343+0000","last_change":"2024-02-02T13:32:00.563274+0000","last_active":"2024-02-02T13:32:00.563343+0000","last_peered":"2024-02-02T13:32:00.563343+0000","last_clean":"2024-02-02T13:32:00.563343+0000","last_became_active":"2024-02-02T13:32:00.563041+0000","last_became_peered":"2024-02-02T13:32:00.563041+0000","last_unstale":"2024-02-02T13:32:00.563343+0000","last_undegraded":"2024-02-02T13:32:00.563343+0000","last_fullsized":"2024-02-02T13:32:00.563343+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:30.171486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578892+0000","last_change":"2024-02-02T13:32:00.087069+0000","last_active":"2024-02-02T13:32:00.578892+0000","last_peered":"2024-02-02T13:32:00.578892+0000","last_clean":"2024-02-02T13:32:00.578892+0000","last_became_active":"2024-02-02T13:32:00.086799+0000","last_became_peered":"2024-02-02T13:32:00.086799+0000","last_unstale":"2024-02-02T13:32:00.578892+0000","last_undegraded":"2024-02-02T13:32:00.578892+0000","last_fullsized":"2024-02-02T13:32:00.578892+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:45:51.365321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.565028+0000","last_change":"2024-02-02T13:32:00.564934+0000","last_active":"2024-02-02T13:32:00.565028+0000","last_peered":"2024-02-02T13:32:00.565028+0000","last_clean":"2024-02-02T13:32:00.565028+0000","last_became_active":"2024-02-02T13:32:00.564577+0000","last_became_peered":"2024-02-02T13:32:00.564577+0000","last_unstale":"2024-02-02T13:32:00.565028+0000","last_undegraded":"2024-02-02T13:32:00.565028+0000","last_fullsized":"2024-02-02T13:32:00.565028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:47.088654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571138+0000","last_change":"2024-02-02T13:32:00.095149+0000","last_active":"2024-02-02T13:32:00.571138+0000","last_peered":"2024-02-02T13:32:00.571138+0000","last_clean":"2024-02-02T13:32:00.571138+0000","last_became_active":"2024-02-02T13:32:00.094875+0000","last_became_peered":"2024-02-02T13:32:00.094875+0000","last_unstale":"2024-02-02T13:32:00.571138+0000","last_undegraded":"2024-02-02T13:32:00.571138+0000","last_fullsized":"2024-02-02T13:32:00.571138+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:28.939106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572214+0000","last_change":"2024-02-02T13:32:00.572115+0000","last_active":"2024-02-02T13:32:00.572214+0000","last_peered":"2024-02-02T13:32:00.572214+0000","last_clean":"2024-02-02T13:32:00.572214+0000","last_became_active":"2024-02-02T13:32:00.571775+0000","last_became_peered":"2024-02-02T13:32:00.571775+0000","last_unstale":"2024-02-02T13:32:00.572214+0000","last_undegraded":"2024-02-02T13:32:00.572214+0000","last_fullsized":"2024-02-02T13:32:00.572214+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:58.765170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579332+0000","last_change":"2024-02-02T13:32:00.095339+0000","last_active":"2024-02-02T13:32:00.579332+0000","last_peered":"2024-02-02T13:32:00.579332+0000","last_clean":"2024-02-02T13:32:00.579332+0000","last_became_active":"2024-02-02T13:32:00.095122+0000","last_became_peered":"2024-02-02T13:32:00.095122+0000","last_unstale":"2024-02-02T13:32:00.579332+0000","last_undegraded":"2024-02-02T13:32:00.579332+0000","last_fullsized":"2024-02-02T13:32:00.579332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:59.441599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571546+0000","last_change":"2024-02-02T13:32:00.571311+0000","last_active":"2024-02-02T13:32:00.571546+0000","last_peered":"2024-02-02T13:32:00.571546+0000","last_clean":"2024-02-02T13:32:00.571546+0000","last_became_active":"2024-02-02T13:32:00.571101+0000","last_became_peered":"2024-02-02T13:32:00.571101+0000","last_unstale":"2024-02-02T13:32:00.571546+0000","last_undegraded":"2024-02-02T13:32:00.571546+0000","last_fullsized":"2024-02-02T13:32:00.571546+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:11.434157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574212+0000","last_change":"2024-02-02T13:32:00.092916+0000","last_active":"2024-02-02T13:32:00.574212+0000","last_peered":"2024-02-02T13:32:00.574212+0000","last_clean":"2024-02-02T13:32:00.574212+0000","last_became_active":"2024-02-02T13:32:00.092397+0000","last_became_peered":"2024-02-02T13:32:00.092397+0000","last_unstale":"2024-02-02T13:32:00.574212+0000","last_undegraded":"2024-02-02T13:32:00.574212+0000","last_fullsized":"2024-02-02T13:32:00.574212+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:05.207739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566271+0000","last_change":"2024-02-02T13:32:00.565792+0000","last_active":"2024-02-02T13:32:00.566271+0000","last_peered":"2024-02-02T13:32:00.566271+0000","last_clean":"2024-02-02T13:32:00.566271+0000","last_became_active":"2024-02-02T13:32:00.565566+0000","last_became_peered":"2024-02-02T13:32:00.565566+0000","last_unstale":"2024-02-02T13:32:00.566271+0000","last_undegraded":"2024-02-02T13:32:00.566271+0000","last_fullsized":"2024-02-02T13:32:00.566271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:19:07.790844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578554+0000","last_change":"2024-02-02T13:32:00.093809+0000","last_active":"2024-02-02T13:32:00.578554+0000","last_peered":"2024-02-02T13:32:00.578554+0000","last_clean":"2024-02-02T13:32:00.578554+0000","last_became_active":"2024-02-02T13:32:00.093246+0000","last_became_peered":"2024-02-02T13:32:00.093246+0000","last_unstale":"2024-02-02T13:32:00.578554+0000","last_undegraded":"2024-02-02T13:32:00.578554+0000","last_fullsized":"2024-02-02T13:32:00.578554+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:54:08.561101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578611+0000","last_change":"2024-02-02T13:32:00.575319+0000","last_active":"2024-02-02T13:32:00.578611+0000","last_peered":"2024-02-02T13:32:00.578611+0000","last_clean":"2024-02-02T13:32:00.578611+0000","last_became_active":"2024-02-02T13:32:00.574981+0000","last_became_peered":"2024-02-02T13:32:00.574981+0000","last_unstale":"2024-02-02T13:32:00.578611+0000","last_undegraded":"2024-02-02T13:32:00.578611+0000","last_fullsized":"2024-02-02T13:32:00.578611+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:47.109639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578846+0000","last_change":"2024-02-02T13:32:00.092851+0000","last_active":"2024-02-02T13:32:00.578846+0000","last_peered":"2024-02-02T13:32:00.578846+0000","last_clean":"2024-02-02T13:32:00.578846+0000","last_became_active":"2024-02-02T13:32:00.090804+0000","last_became_peered":"2024-02-02T13:32:00.090804+0000","last_unstale":"2024-02-02T13:32:00.578846+0000","last_undegraded":"2024-02-02T13:32:00.578846+0000","last_fullsized":"2024-02-02T13:32:00.578846+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:15.508475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.567719+0000","last_change":"2024-02-02T13:32:00.567461+0000","last_active":"2024-02-02T13:32:00.567719+0000","last_peered":"2024-02-02T13:32:00.567719+0000","last_clean":"2024-02-02T13:32:00.567719+0000","last_became_active":"2024-02-02T13:32:00.566827+0000","last_became_peered":"2024-02-02T13:32:00.566827+0000","last_unstale":"2024-02-02T13:32:00.567719+0000","last_undegraded":"2024-02-02T13:32:00.567719+0000","last_fullsized":"2024-02-02T13:32:00.567719+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:36:10.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562859+0000","last_change":"2024-02-02T13:32:00.094242+0000","last_active":"2024-02-02T13:32:00.562859+0000","last_peered":"2024-02-02T13:32:00.562859+0000","last_clean":"2024-02-02T13:32:00.562859+0000","last_became_active":"2024-02-02T13:32:00.093786+0000","last_became_peered":"2024-02-02T13:32:00.093786+0000","last_unstale":"2024-02-02T13:32:00.562859+0000","last_undegraded":"2024-02-02T13:32:00.562859+0000","last_fullsized":"2024-02-02T13:32:00.562859+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:19.155804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578495+0000","last_change":"2024-02-02T13:32:00.572255+0000","last_active":"2024-02-02T13:32:00.578495+0000","last_peered":"2024-02-02T13:32:00.578495+0000","last_clean":"2024-02-02T13:32:00.578495+0000","last_became_active":"2024-02-02T13:32:00.571907+0000","last_became_peered":"2024-02-02T13:32:00.571907+0000","last_unstale":"2024-02-02T13:32:00.578495+0000","last_undegraded":"2024-02-02T13:32:00.578495+0000","last_fullsized":"2024-02-02T13:32:00.578495+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:33.795196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580432+0000","last_change":"2024-02-02T13:32:00.087915+0000","last_active":"2024-02-02T13:32:00.580432+0000","last_peered":"2024-02-02T13:32:00.580432+0000","last_clean":"2024-02-02T13:32:00.580432+0000","last_became_active":"2024-02-02T13:32:00.087644+0000","last_became_peered":"2024-02-02T13:32:00.087644+0000","last_unstale":"2024-02-02T13:32:00.580432+0000","last_undegraded":"2024-02-02T13:32:00.580432+0000","last_fullsized":"2024-02-02T13:32:00.580432+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:22:21.773079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.575624+0000","last_change":"2024-02-02T13:32:00.575567+0000","last_active":"2024-02-02T13:32:00.575624+0000","last_peered":"2024-02-02T13:32:00.575624+0000","last_clean":"2024-02-02T13:32:00.575624+0000","last_became_active":"2024-02-02T13:32:00.574030+0000","last_became_peered":"2024-02-02T13:32:00.574030+0000","last_unstale":"2024-02-02T13:32:00.575624+0000","last_undegraded":"2024-02-02T13:32:00.575624+0000","last_fullsized":"2024-02-02T13:32:00.575624+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:06.430460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560561+0000","last_change":"2024-02-02T13:32:00.088439+0000","last_active":"2024-02-02T13:32:00.560561+0000","last_peered":"2024-02-02T13:32:00.560561+0000","last_clean":"2024-02-02T13:32:00.560561+0000","last_became_active":"2024-02-02T13:32:00.087866+0000","last_became_peered":"2024-02-02T13:32:00.087866+0000","last_unstale":"2024-02-02T13:32:00.560561+0000","last_undegraded":"2024-02-02T13:32:00.560561+0000","last_fullsized":"2024-02-02T13:32:00.560561+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:44:18.471870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578626+0000","last_change":"2024-02-02T13:32:00.575655+0000","last_active":"2024-02-02T13:32:00.578626+0000","last_peered":"2024-02-02T13:32:00.578626+0000","last_clean":"2024-02-02T13:32:00.578626+0000","last_became_active":"2024-02-02T13:32:00.575354+0000","last_became_peered":"2024-02-02T13:32:00.575354+0000","last_unstale":"2024-02-02T13:32:00.578626+0000","last_undegraded":"2024-02-02T13:32:00.578626+0000","last_fullsized":"2024-02-02T13:32:00.578626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:45.330479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.556643+0000","last_change":"2024-02-02T13:32:00.089831+0000","last_active":"2024-02-02T13:32:00.556643+0000","last_peered":"2024-02-02T13:32:00.556643+0000","last_clean":"2024-02-02T13:32:00.556643+0000","last_became_active":"2024-02-02T13:32:00.089471+0000","last_became_peered":"2024-02-02T13:32:00.089471+0000","last_unstale":"2024-02-02T13:32:00.556643+0000","last_undegraded":"2024-02-02T13:32:00.556643+0000","last_fullsized":"2024-02-02T13:32:00.556643+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:11.731967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573100+0000","last_change":"2024-02-02T13:32:00.573034+0000","last_active":"2024-02-02T13:32:00.573100+0000","last_peered":"2024-02-02T13:32:00.573100+0000","last_clean":"2024-02-02T13:32:00.573100+0000","last_became_active":"2024-02-02T13:32:00.572713+0000","last_became_peered":"2024-02-02T13:32:00.572713+0000","last_unstale":"2024-02-02T13:32:00.573100+0000","last_undegraded":"2024-02-02T13:32:00.573100+0000","last_fullsized":"2024-02-02T13:32:00.573100+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:17.955007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568999+0000","last_change":"2024-02-02T13:32:00.094234+0000","last_active":"2024-02-02T13:32:00.568999+0000","last_peered":"2024-02-02T13:32:00.568999+0000","last_clean":"2024-02-02T13:32:00.568999+0000","last_became_active":"2024-02-02T13:32:00.093955+0000","last_became_peered":"2024-02-02T13:32:00.093955+0000","last_unstale":"2024-02-02T13:32:00.568999+0000","last_undegraded":"2024-02-02T13:32:00.568999+0000","last_fullsized":"2024-02-02T13:32:00.568999+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:46.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582893+0000","last_change":"2024-02-02T13:32:00.582667+0000","last_active":"2024-02-02T13:32:00.582893+0000","last_peered":"2024-02-02T13:32:00.582893+0000","last_clean":"2024-02-02T13:32:00.582893+0000","last_became_active":"2024-02-02T13:32:00.582204+0000","last_became_peered":"2024-02-02T13:32:00.582204+0000","last_unstale":"2024-02-02T13:32:00.582893+0000","last_undegraded":"2024-02-02T13:32:00.582893+0000","last_fullsized":"2024-02-02T13:32:00.582893+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:59:50.613919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574886+0000","last_change":"2024-02-02T13:32:00.096526+0000","last_active":"2024-02-02T13:32:00.574886+0000","last_peered":"2024-02-02T13:32:00.574886+0000","last_clean":"2024-02-02T13:32:00.574886+0000","last_became_active":"2024-02-02T13:32:00.094279+0000","last_became_peered":"2024-02-02T13:32:00.094279+0000","last_unstale":"2024-02-02T13:32:00.574886+0000","last_undegraded":"2024-02-02T13:32:00.574886+0000","last_fullsized":"2024-02-02T13:32:00.574886+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:28.939106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573776+0000","last_change":"2024-02-02T13:32:00.573199+0000","last_active":"2024-02-02T13:32:00.573776+0000","last_peered":"2024-02-02T13:32:00.573776+0000","last_clean":"2024-02-02T13:32:00.573776+0000","last_became_active":"2024-02-02T13:32:00.572964+0000","last_became_peered":"2024-02-02T13:32:00.572964+0000","last_unstale":"2024-02-02T13:32:00.573776+0000","last_undegraded":"2024-02-02T13:32:00.573776+0000","last_fullsized":"2024-02-02T13:32:00.573776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:15:49.748992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573628+0000","last_change":"2024-02-02T13:32:00.093531+0000","last_active":"2024-02-02T13:32:00.573628+0000","last_peered":"2024-02-02T13:32:00.573628+0000","last_clean":"2024-02-02T13:32:00.573628+0000","last_became_active":"2024-02-02T13:32:00.091671+0000","last_became_peered":"2024-02-02T13:32:00.091671+0000","last_unstale":"2024-02-02T13:32:00.573628+0000","last_undegraded":"2024-02-02T13:32:00.573628+0000","last_fullsized":"2024-02-02T13:32:00.573628+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:22:20.921907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572281+0000","last_change":"2024-02-02T13:32:00.571172+0000","last_active":"2024-02-02T13:32:00.572281+0000","last_peered":"2024-02-02T13:32:00.572281+0000","last_clean":"2024-02-02T13:32:00.572281+0000","last_became_active":"2024-02-02T13:32:00.570352+0000","last_became_peered":"2024-02-02T13:32:00.570352+0000","last_unstale":"2024-02-02T13:32:00.572281+0000","last_undegraded":"2024-02-02T13:32:00.572281+0000","last_fullsized":"2024-02-02T13:32:00.572281+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:04:58.904061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562610+0000","last_change":"2024-02-02T13:32:00.092626+0000","last_active":"2024-02-02T13:32:00.562610+0000","last_peered":"2024-02-02T13:32:00.562610+0000","last_clean":"2024-02-02T13:32:00.562610+0000","last_became_active":"2024-02-02T13:32:00.091569+0000","last_became_peered":"2024-02-02T13:32:00.091569+0000","last_unstale":"2024-02-02T13:32:00.562610+0000","last_undegraded":"2024-02-02T13:32:00.562610+0000","last_fullsized":"2024-02-02T13:32:00.562610+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:44.315720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582907+0000","last_change":"2024-02-02T13:32:00.582776+0000","last_active":"2024-02-02T13:32:00.582907+0000","last_peered":"2024-02-02T13:32:00.582907+0000","last_clean":"2024-02-02T13:32:00.582907+0000","last_became_active":"2024-02-02T13:32:00.581988+0000","last_became_peered":"2024-02-02T13:32:00.581988+0000","last_unstale":"2024-02-02T13:32:00.582907+0000","last_undegraded":"2024-02-02T13:32:00.582907+0000","last_fullsized":"2024-02-02T13:32:00.582907+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:00:30.493159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.551947+0000","last_change":"2024-02-02T13:32:00.083671+0000","last_active":"2024-02-02T13:32:00.551947+0000","last_peered":"2024-02-02T13:32:00.551947+0000","last_clean":"2024-02-02T13:32:00.551947+0000","last_became_active":"2024-02-02T13:32:00.083395+0000","last_became_peered":"2024-02-02T13:32:00.083395+0000","last_unstale":"2024-02-02T13:32:00.551947+0000","last_undegraded":"2024-02-02T13:32:00.551947+0000","last_fullsized":"2024-02-02T13:32:00.551947+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:15.508475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.576785+0000","last_change":"2024-02-02T13:32:00.575210+0000","last_active":"2024-02-02T13:32:00.576785+0000","last_peered":"2024-02-02T13:32:00.576785+0000","last_clean":"2024-02-02T13:32:00.576785+0000","last_became_active":"2024-02-02T13:32:00.574736+0000","last_became_peered":"2024-02-02T13:32:00.574736+0000","last_unstale":"2024-02-02T13:32:00.576785+0000","last_undegraded":"2024-02-02T13:32:00.576785+0000","last_fullsized":"2024-02-02T13:32:00.576785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:20.518706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560590+0000","last_change":"2024-02-02T13:32:00.095860+0000","last_active":"2024-02-02T13:32:00.560590+0000","last_peered":"2024-02-02T13:32:00.560590+0000","last_clean":"2024-02-02T13:32:00.560590+0000","last_became_active":"2024-02-02T13:32:00.094658+0000","last_became_peered":"2024-02-02T13:32:00.094658+0000","last_unstale":"2024-02-02T13:32:00.560590+0000","last_undegraded":"2024-02-02T13:32:00.560590+0000","last_fullsized":"2024-02-02T13:32:00.560590+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:54.543807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573427+0000","last_change":"2024-02-02T13:32:00.573299+0000","last_active":"2024-02-02T13:32:00.573427+0000","last_peered":"2024-02-02T13:32:00.573427+0000","last_clean":"2024-02-02T13:32:00.573427+0000","last_became_active":"2024-02-02T13:32:00.572947+0000","last_became_peered":"2024-02-02T13:32:00.572947+0000","last_unstale":"2024-02-02T13:32:00.573427+0000","last_undegraded":"2024-02-02T13:32:00.573427+0000","last_fullsized":"2024-02-02T13:32:00.573427+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:16.523235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568345+0000","last_change":"2024-02-02T13:32:00.093624+0000","last_active":"2024-02-02T13:32:00.568345+0000","last_peered":"2024-02-02T13:32:00.568345+0000","last_clean":"2024-02-02T13:32:00.568345+0000","last_became_active":"2024-02-02T13:32:00.093387+0000","last_became_peered":"2024-02-02T13:32:00.093387+0000","last_unstale":"2024-02-02T13:32:00.568345+0000","last_undegraded":"2024-02-02T13:32:00.568345+0000","last_fullsized":"2024-02-02T13:32:00.568345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:43:38.592631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582348+0000","last_change":"2024-02-02T13:32:00.581946+0000","last_active":"2024-02-02T13:32:00.582348+0000","last_peered":"2024-02-02T13:32:00.582348+0000","last_clean":"2024-02-02T13:32:00.582348+0000","last_became_active":"2024-02-02T13:32:00.580825+0000","last_became_peered":"2024-02-02T13:32:00.580825+0000","last_unstale":"2024-02-02T13:32:00.582348+0000","last_undegraded":"2024-02-02T13:32:00.582348+0000","last_fullsized":"2024-02-02T13:32:00.582348+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:58.765170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562368+0000","last_change":"2024-02-02T13:32:00.089294+0000","last_active":"2024-02-02T13:32:00.562368+0000","last_peered":"2024-02-02T13:32:00.562368+0000","last_clean":"2024-02-02T13:32:00.562368+0000","last_became_active":"2024-02-02T13:32:00.088991+0000","last_became_peered":"2024-02-02T13:32:00.088991+0000","last_unstale":"2024-02-02T13:32:00.562368+0000","last_undegraded":"2024-02-02T13:32:00.562368+0000","last_fullsized":"2024-02-02T13:32:00.562368+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:30.325522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580131+0000","last_change":"2024-02-02T13:32:00.580047+0000","last_active":"2024-02-02T13:32:00.580131+0000","last_peered":"2024-02-02T13:32:00.580131+0000","last_clean":"2024-02-02T13:32:00.580131+0000","last_became_active":"2024-02-02T13:32:00.577888+0000","last_became_peered":"2024-02-02T13:32:00.577888+0000","last_unstale":"2024-02-02T13:32:00.580131+0000","last_undegraded":"2024-02-02T13:32:00.580131+0000","last_fullsized":"2024-02-02T13:32:00.580131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:19.870695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562663+0000","last_change":"2024-02-02T13:32:00.090802+0000","last_active":"2024-02-02T13:32:00.562663+0000","last_peered":"2024-02-02T13:32:00.562663+0000","last_clean":"2024-02-02T13:32:00.562663+0000","last_became_active":"2024-02-02T13:32:00.090403+0000","last_became_peered":"2024-02-02T13:32:00.090403+0000","last_unstale":"2024-02-02T13:32:00.562663+0000","last_undegraded":"2024-02-02T13:32:00.562663+0000","last_fullsized":"2024-02-02T13:32:00.562663+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:17.166864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569506+0000","last_change":"2024-02-02T13:32:00.568995+0000","last_active":"2024-02-02T13:32:00.569506+0000","last_peered":"2024-02-02T13:32:00.569506+0000","last_clean":"2024-02-02T13:32:00.569506+0000","last_became_active":"2024-02-02T13:32:00.568789+0000","last_became_peered":"2024-02-02T13:32:00.568789+0000","last_unstale":"2024-02-02T13:32:00.569506+0000","last_undegraded":"2024-02-02T13:32:00.569506+0000","last_fullsized":"2024-02-02T13:32:00.569506+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:36:10.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562621+0000","last_change":"2024-02-02T13:32:00.094070+0000","last_active":"2024-02-02T13:32:00.562621+0000","last_peered":"2024-02-02T13:32:00.562621+0000","last_clean":"2024-02-02T13:32:00.562621+0000","last_became_active":"2024-02-02T13:32:00.092626+0000","last_became_peered":"2024-02-02T13:32:00.092626+0000","last_unstale":"2024-02-02T13:32:00.562621+0000","last_undegraded":"2024-02-02T13:32:00.562621+0000","last_fullsized":"2024-02-02T13:32:00.562621+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:21:05.945643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.072240+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:32:00.072240+0000","last_peered":"2024-02-02T13:32:00.072240+0000","last_clean":"2024-02-02T13:32:00.072240+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:32:00.072240+0000","last_undegraded":"2024-02-02T13:32:00.072240+0000","last_fullsized":"2024-02-02T13:32:00.072240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193968,"statfs":{"total":200000000,"available":198623334,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.103,"15min":1.0700000000000001},"min":{"1min":0.71099999999999997,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.5310000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1060000000000001,"15min":1.121},"min":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.1609999999999996},"last":0.98399999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.73899999999999999,"5min":0.68999999999999995,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":1.8839999999999999,"15min":1.893},"last":1.284},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.73499999999999999,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.232},"min":{"1min":0.90800000000000003,"5min":0.81999999999999995,"15min":0.70099999999999996},"max":{"1min":1.675,"5min":1.909,"15min":2.6669999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.2070000000000001,"15min":1.1859999999999999},"min":{"1min":0.88400000000000001,"5min":0.80000000000000004,"15min":0.623},"max":{"1min":1.7949999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":1.2310000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1819999999999999,"15min":1.1799999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3819999999999999,"5min":1.6339999999999999,"15min":1.819},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.345,"15min":1.3180000000000001},"min":{"1min":0.79600000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0259999999999998,"5min":2.133,"15min":2.4620000000000002},"last":1.53}]},{"osd":4,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3120000000000001,"15min":1.21},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.619,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3100000000000001},"min":{"1min":0.88,"5min":0.77600000000000002,"15min":0.61199999999999999},"max":{"1min":1.6180000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.3879999999999999,"15min":1.3580000000000001},"min":{"1min":1.0920000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.788,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.84199999999999997,"5min":0.76000000000000001,"15min":0.56200000000000006},"max":{"1min":1.921,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.5089999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.506,"15min":1.526},"min":{"1min":0.96999999999999997,"5min":0.88700000000000001,"15min":0.64200000000000002},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.4399999999999999,"15min":1.419},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.7809999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194545,"statfs":{"total":200000000,"available":199214651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.0309999999999999},"min":{"1min":0.64000000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.5840000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.028},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6060000000000001,"5min":1.768,"15min":1.768},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0900000000000001,"15min":1.028},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4790000000000001,"5min":1.556,"15min":1.556},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1719999999999999,"15min":1.091},"min":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.861,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.0109999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1899999999999999,"15min":1.0900000000000001},"min":{"1min":0.747,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6659999999999999,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.345},{"interface":"front","average":{"1min":1.196,"5min":1.1919999999999999,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8169999999999999,"5min":1.849,"15min":1.849},"last":1.157}]},{"osd":3,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2529999999999999,"15min":1.2589999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8149999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1990000000000001,"15min":1.147},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.643,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.103}]},{"osd":4,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.4530000000000001,"15min":1.276},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.8640000000000001,"5min":17.686,"15min":17.686},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5920000000000001,"15min":1.4530000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.1099999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.431}]},{"osd":5,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.3009999999999999,"15min":1.2769999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6399999999999999,"5min":1.986,"15min":1.986},"last":1.232},{"interface":"front","average":{"1min":1.423,"5min":1.4370000000000001,"15min":1.3660000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9119999999999999,"5min":3.331,"15min":3.331},"last":1.556}]},{"osd":7,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.494,"15min":1.4590000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.61199999999999999},"max":{"1min":2.0089999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.4670000000000001,"15min":1.4390000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.901,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.395}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":768,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194544,"statfs":{"total":200000000,"available":199213098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.028,"15min":1.022},"min":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.081,"15min":1.048},"min":{"1min":0.74099999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.554,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.153,"15min":1.2430000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.494,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.085},{"interface":"front","average":{"1min":1.044,"5min":1.0740000000000001,"15min":1.133},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.877,"15min":4.4500000000000002},"last":1.4159999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.286,"15min":1.2310000000000001},"min":{"1min":0.78400000000000003,"5min":0.753,"15min":0.72499999999999998},"max":{"1min":1.6379999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.335},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1890000000000001,"15min":1.1759999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5129999999999999,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.4390000000000001,"5min":1.591,"15min":1.7769999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.29,"15min":1.1879999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.5349999999999999,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.4690000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1890000000000001,"15min":1.2190000000000001},"min":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.64000000000000001},"max":{"1min":1.615,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.012},{"interface":"front","average":{"1min":1.272,"5min":1.296,"15min":1.2829999999999999},"min":{"1min":0.92600000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5129999999999999,"5min":1.8029999999999999,"15min":1.833},"last":1.171}]},{"osd":6,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3300000000000001,"15min":1.3069999999999999},"min":{"1min":1.01,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7669999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.345,"15min":1.3460000000000001},"min":{"1min":0.82599999999999996,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.724,"5min":1.7889999999999999,"15min":1.8640000000000001},"last":1.375}]},{"osd":7,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.111,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.589,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.444,"15min":1.385},"min":{"1min":1.1299999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.71,"5min":1.851,"15min":1.851},"last":1.2849999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.091,"15min":0.998},"min":{"1min":0.67900000000000005,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.3959999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.079},"min":{"1min":0.71399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3999999999999999,"5min":1.839,"15min":1.839},"last":1.4630000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.0029999999999999},"min":{"1min":0.66000000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.403,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.1459999999999999,"15min":1.0489999999999999},"min":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5740000000000001,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.056}]},{"osd":2,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.177,"15min":1.099},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.478,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1830000000000001,"15min":1.0900000000000001},"min":{"1min":0.70099999999999996,"5min":0.68000000000000005,"15min":0.58799999999999997},"max":{"1min":1.5620000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.274,"15min":1.1899999999999999},"min":{"1min":0.871,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5349999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.331,"15min":1.252},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.2330000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.27,"15min":1.1619999999999999},"min":{"1min":0.88100000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.829,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3660000000000001,"15min":1.29},"min":{"1min":0.89900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.722,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.6140000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.403,"15min":1.3120000000000001},"min":{"1min":0.96899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.712,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.327},{"interface":"front","average":{"1min":1.454,"5min":1.48,"15min":1.401},"min":{"1min":1.0580000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7849999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.351}]},{"osd":7,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.49,"15min":1.3979999999999999},"min":{"1min":1.155,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.671},{"interface":"front","average":{"1min":1.401,"5min":1.498,"15min":1.452},"min":{"1min":1.0089999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.869,"5min":2.698,"15min":2.698},"last":1.5509999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":769,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194543,"statfs":{"total":200000000,"available":199212068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.71599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.88700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":1.014}]},{"osd":1,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96099999999999997,"15min":0.90700000000000003},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.097,"5min":1.077,"15min":1.0229999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1299999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":0.96799999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4370000000000001,"5min":1.611,"15min":1.611},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92200000000000004,"15min":0.89600000000000002},"min":{"1min":0.443,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.9159999999999999,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.181},"min":{"1min":0.71099999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.99,"5min":1.99,"15min":3.585},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2649999999999999,"15min":1.2110000000000001},"min":{"1min":0.80000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.877},"last":1.3600000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1819999999999999,"15min":1.21},"min":{"1min":0.72099999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8129999999999999,"5min":3.738,"15min":3.738},"last":1.399},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.28,"15min":1.2589999999999999},"min":{"1min":0.83499999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":1.1759999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3640000000000001,"15min":1.2749999999999999},"min":{"1min":0.91500000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.895,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.2569999999999999,"15min":1.2889999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":2.0310000000000001,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.222}]},{"osd":7,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.276,"15min":1.26},"min":{"1min":0.75,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":1.869,"15min":3.391},"last":1.079},{"interface":"front","average":{"1min":1.351,"5min":1.3779999999999999,"15min":1.321},"min":{"1min":0.877,"5min":0.73999999999999999,"15min":0.625},"max":{"1min":1.946,"5min":1.946,"15min":2.0409999999999999},"last":1.27}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":767,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194544,"statfs":{"total":200000000,"available":199213702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":0.78700000000000003},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":1.401,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.91600000000000004,"15min":0.80200000000000005},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.2410000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.002}]},{"osd":1,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97899999999999998,"15min":0.91100000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.218,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0169999999999999,"15min":0.88100000000000001},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.4990000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1060000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.93000000000000005,"15min":0.81200000000000006},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.145,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.091,"15min":0.995},"min":{"1min":0.435,"5min":0.435,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.84299999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.288,"15min":1.1930000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.298,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2470000000000001,"15min":1.1699999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4710000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.226,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3129999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.331},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.343,"15min":1.2789999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.635,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.89400000000000002}]},{"osd":6,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.3220000000000001,"15min":1.198},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.571,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.361},{"interface":"front","average":{"1min":1.137,"5min":1.29,"15min":1.21},"min":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":1.3799999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.446}]},{"osd":7,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.379,"15min":1.232},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6000000000000001,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.3400000000000001,"15min":1.2330000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4690000000000001,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.3939999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194545,"statfs":{"total":200000000,"available":199214597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.425,"5min":1.425,"15min":3.5680000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.93000000000000005,"15min":1.2869999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4870000000000001,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.875,"15min":0.90300000000000002},"min":{"1min":0.52500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3700000000000001,"5min":1.431,"15min":1.639},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0049999999999999,"15min":1.077},"min":{"1min":0.50800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6890000000000001,"5min":1.823,"15min":3.0259999999999998},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.097,"15min":1.228},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.002,"5min":1.0189999999999999,"15min":1.234},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.774,"5min":1.774,"15min":4.8760000000000003},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.2889999999999999},"min":{"1min":0.85499999999999998,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.837,"15min":2.9449999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.2829999999999999,"15min":1.351},"min":{"1min":0.83799999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":1.032}]},{"osd":5,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2869999999999999,"15min":1.4279999999999999},"min":{"1min":0.88900000000000001,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.5569999999999999,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2789999999999999,"15min":1.4099999999999999},"min":{"1min":0.91400000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.657,"5min":1.7090000000000001,"15min":3.0249999999999999},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.306,"15min":1.526},"min":{"1min":0.97399999999999998,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.966,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":1.417},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3260000000000001,"15min":1.4630000000000001},"min":{"1min":0.96399999999999997,"5min":0.92100000000000004,"15min":0.81000000000000005},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.375,"15min":1.524},"min":{"1min":0.82099999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.177},{"interface":"front","average":{"1min":1.373,"5min":1.3819999999999999,"15min":1.577},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.78000000000000003},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.286}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1345,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193967,"statfs":{"total":200000000,"available":198622677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":1.069},"min":{"1min":0.35599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2629999999999999,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.90800000000000003,"15min":1.3859999999999999},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3879999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.93500000000000005,"15min":1.125},"min":{"1min":0.48399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4319999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.93100000000000005,"15min":1.387},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6240000000000001,"5min":1.706,"15min":10.781000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":1.1739999999999999},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":5.8410000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0369999999999999,"15min":1.2649999999999999},"min":{"1min":0.51600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6459999999999999,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.3140000000000001},"min":{"1min":0.73399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.55,"5min":1.9670000000000001,"15min":3.669},"last":1.45},{"interface":"front","average":{"1min":1.256,"5min":1.2689999999999999,"15min":1.383},"min":{"1min":0.77300000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.6599999999999999,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.169}]},{"osd":5,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2589999999999999,"15min":1.73},"min":{"1min":0.89100000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.236,"15min":1.395},"min":{"1min":0.69799999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.153}]},{"osd":6,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3029999999999999,"15min":1.446},"min":{"1min":0.87,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":2.319,"15min":4.9210000000000003},"last":1.262},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2709999999999999,"15min":1.7609999999999999},"min":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.75,"5min":2.0510000000000002,"15min":10.708},"last":1.29}]},{"osd":7,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.337,"15min":1.7849999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":1.7390000000000001,"5min":2.3010000000000002,"15min":10.74},"last":1.216},{"interface":"front","average":{"1min":1.296,"5min":1.2949999999999999,"15min":1.423},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":3.863},"last":1.044}]}]}],"pool_statfs":[]}} 2024-02-02T13:32:04.944 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:32:05.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.194+0000 7fe02e200640 1 Processor -- start 2024-02-02T13:32:05.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.194+0000 7fe02e200640 1 -- start start 2024-02-02T13:32:05.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe02e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe0280c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe02e200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0280cc7c0 con 0x7fe0280ce130 2024-02-02T13:32:05.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe02e200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0280ce930 con 0x7fe02815a010 2024-02-02T13:32:05.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe02e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280cd640 con 0x7fe0280b0e10 2024-02-02T13:32:05.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe01e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe0280ce130 0x7fe0280b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45470/0 (socket says 172.21.15.91:45470) 2024-02-02T13:32:05.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe01e800640 1 -- 172.21.15.91:0/3744015780 learned_addr learned my addr 172.21.15.91:0/3744015780 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:05.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.196+0000 7fe027e00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe0280c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214448349 0 0) 0x7fe0280cc7c0 con 0x7fe0280ce130 2024-02-02T13:32:05.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe008005730 con 0x7fe0280ce130 2024-02-02T13:32:05.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1690164097 0 0) 0x7fe008005730 con 0x7fe0280ce130 2024-02-02T13:32:05.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 msgr2=0x7fe0280c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe0280c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 >> v1:172.21.15.113:6789/0 conn(0x7fe02815a010 legacy=0x7fe0280ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02806c890 con 0x7fe0280ce130 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe0100019f0 con 0x7fe0280ce130 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 >> v1:172.21.15.91:6789/0 conn(0x7fe0280ce130 legacy=0x7fe0280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe008005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe008006130 0x7fe0080069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe008007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280cd640 con 0x7fe008006f10 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00800dc70 con 0x7fe008006130 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.197+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00800de30 con 0x7fe0280b0e10 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe008007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe01e800640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe008006130 0x7fe0080069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027400640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe008005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 -- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 msgr2=0x7fe008005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:05.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0280b0e10 0x7fe008005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 -- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe008006130 msgr2=0x7fe0080069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:05.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe008006130 0x7fe0080069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe027e00640 1 -- 172.21.15.91:0/3744015780 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008005730 con 0x7fe008006f10 2024-02-02T13:32:05.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.198+0000 7fe01e800640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe008006130 0x7fe0080069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.199+0000 7fe027e00640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe008007390 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe020009a90 tx=0x7fe008007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914ae18d5112fbfb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.199+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02002c680 con 0x7fe008006f10 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.199+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02002c7e0 con 0x7fe008006f10 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.199+0000 7fe026a00640 1 -- 172.21.15.91:0/3744015780 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02002cb70 con 0x7fe008006f10 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 -- 172.21.15.91:0/3744015780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 msgr2=0x7fe008007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 --2- 172.21.15.91:0/3744015780 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe008007390 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe020009a90 tx=0x7fe008007a00 comp rx=0 tx=0).stop 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe027e00640 1 -- 172.21.15.91:0/3744015780 reap_dead start 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 -- 172.21.15.91:0/3744015780 shutdown_connections 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 -- 172.21.15.91:0/3744015780 >> 172.21.15.91:0/3744015780 conn(0x7fe0280cec80 msgr2=0x7fe0280cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 -- 172.21.15.91:0/3744015780 shutdown_connections 2024-02-02T13:32:05.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.200+0000 7fe02e200640 1 -- 172.21.15.91:0/3744015780 wait complete. 2024-02-02T13:32:05.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.202+0000 7fe02e200640 1 Processor -- start 2024-02-02T13:32:05.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.202+0000 7fe02e200640 1 -- start start 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 0x7fe0280c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe028154d70 0x7fe0281551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280c8090 con 0x7fe008006f10 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280c7760 con 0x7fe0281f35c0 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe02e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280c7a40 con 0x7fe028154d70 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.203+0000 7fe027400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58484/0 (socket says 172.21.15.91:58484) 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 -- 172.21.15.91:0/3089219859 learned_addr learned my addr 172.21.15.91:0/3089219859 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe01e800640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 0x7fe0280c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027e00640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe028154d70 0x7fe0281551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 -- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe028154d70 msgr2=0x7fe0281551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe028154d70 0x7fe0281551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 -- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 msgr2=0x7fe0280c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 0x7fe0280c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 -- 172.21.15.91:0/3089219859 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0280c7d20 con 0x7fe008006f10 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe01e800640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 0x7fe0280c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe027400640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e550 tx=0x7fe01800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.204+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180052e0 con 0x7fe008006f10 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.205+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe018005440 con 0x7fe008006f10 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.205+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018003810 con 0x7fe008006f10 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.205+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe028155730 con 0x7fe008006f10 2024-02-02T13:32:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.205+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0280b4470 con 0x7fe008006f10 2024-02-02T13:32:05.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.206+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe028066cf0 con 0x7fe008006f10 2024-02-02T13:32:05.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.207+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe018003970 con 0x7fe008006f10 2024-02-02T13:32:05.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.208+0000 7fe01fe00640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 0x7fe00006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.208+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0180886b0 con 0x7fe008006f10 2024-02-02T13:32:05.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.208+0000 7fe01e800640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 0x7fe00006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.208+0000 7fe01e800640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 0x7fe00006c920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe010002230 tx=0x7fe010009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.212+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe018053ca0 con 0x7fe008006f10 2024-02-02T13:32:05.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.396+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe02806ce50 con 0x7fe00006a460 2024-02-02T13:32:05.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.417+0000 7fe01fe00640 1 -- 172.21.15.91:0/3089219859 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283140 (secure 0 0 0) 0x7fe01000d040 con 0x7fe00006a460 2024-02-02T13:32:05.423 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:05.424 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 msgr2=0x7fe00006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 0x7fe00006c920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe010002230 tx=0x7fe010009040 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 msgr2=0x7fe0280b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e550 tx=0x7fe01800ea10 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 shutdown_connections 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe00006a460 0x7fe00006c920 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe028154d70 0x7fe0281551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0281f35c0 0x7fe0280c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 --2- 172.21.15.91:0/3089219859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe008006f10 0x7fe0280b3f30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 >> 172.21.15.91:0/3089219859 conn(0x7fe0280cec80 msgr2=0x7fe02815a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:05.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.423+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 shutdown_connections 2024-02-02T13:32:05.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.424+0000 7fe02e200640 1 -- 172.21.15.91:0/3089219859 wait complete. 2024-02-02T13:32:05.444 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-02-02T13:32:04.381818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7289,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555210,"statfs":{"total":1600000000,"available":1592535623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.331680"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582181+0000","last_change":"2024-02-02T13:32:00.582114+0000","last_active":"2024-02-02T13:32:00.582181+0000","last_peered":"2024-02-02T13:32:00.582181+0000","last_clean":"2024-02-02T13:32:00.582181+0000","last_became_active":"2024-02-02T13:32:00.581837+0000","last_became_peered":"2024-02-02T13:32:00.581837+0000","last_unstale":"2024-02-02T13:32:00.582181+0000","last_undegraded":"2024-02-02T13:32:00.582181+0000","last_fullsized":"2024-02-02T13:32:00.582181+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:06:12.560424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.584011+0000","last_change":"2024-02-02T13:32:00.583918+0000","last_active":"2024-02-02T13:32:00.584011+0000","last_peered":"2024-02-02T13:32:00.584011+0000","last_clean":"2024-02-02T13:32:00.584011+0000","last_became_active":"2024-02-02T13:32:00.583015+0000","last_became_peered":"2024-02-02T13:32:00.583015+0000","last_unstale":"2024-02-02T13:32:00.584011+0000","last_undegraded":"2024-02-02T13:32:00.584011+0000","last_fullsized":"2024-02-02T13:32:00.584011+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:24:03.225415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578591+0000","last_change":"2024-02-02T13:32:00.575269+0000","last_active":"2024-02-02T13:32:00.578591+0000","last_peered":"2024-02-02T13:32:00.578591+0000","last_clean":"2024-02-02T13:32:00.578591+0000","last_became_active":"2024-02-02T13:32:00.573886+0000","last_became_peered":"2024-02-02T13:32:00.573886+0000","last_unstale":"2024-02-02T13:32:00.578591+0000","last_undegraded":"2024-02-02T13:32:00.578591+0000","last_fullsized":"2024-02-02T13:32:00.578591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:47:44.521870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.575517+0000","last_change":"2024-02-02T13:32:00.574342+0000","last_active":"2024-02-02T13:32:00.575517+0000","last_peered":"2024-02-02T13:32:00.575517+0000","last_clean":"2024-02-02T13:32:00.575517+0000","last_became_active":"2024-02-02T13:32:00.573989+0000","last_became_peered":"2024-02-02T13:32:00.573989+0000","last_unstale":"2024-02-02T13:32:00.575517+0000","last_undegraded":"2024-02-02T13:32:00.575517+0000","last_fullsized":"2024-02-02T13:32:00.575517+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:23.712750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568785+0000","last_change":"2024-02-02T13:32:00.566419+0000","last_active":"2024-02-02T13:32:00.568785+0000","last_peered":"2024-02-02T13:32:00.568785+0000","last_clean":"2024-02-02T13:32:00.568785+0000","last_became_active":"2024-02-02T13:32:00.566216+0000","last_became_peered":"2024-02-02T13:32:00.566216+0000","last_unstale":"2024-02-02T13:32:00.568785+0000","last_undegraded":"2024-02-02T13:32:00.568785+0000","last_fullsized":"2024-02-02T13:32:00.568785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:29:00.525568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566227+0000","last_change":"2024-02-02T13:32:00.564894+0000","last_active":"2024-02-02T13:32:00.566227+0000","last_peered":"2024-02-02T13:32:00.566227+0000","last_clean":"2024-02-02T13:32:00.566227+0000","last_became_active":"2024-02-02T13:32:00.564507+0000","last_became_peered":"2024-02-02T13:32:00.564507+0000","last_unstale":"2024-02-02T13:32:00.566227+0000","last_undegraded":"2024-02-02T13:32:00.566227+0000","last_fullsized":"2024-02-02T13:32:00.566227+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:42.546275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580468+0000","last_change":"2024-02-02T13:32:00.579619+0000","last_active":"2024-02-02T13:32:00.580468+0000","last_peered":"2024-02-02T13:32:00.580468+0000","last_clean":"2024-02-02T13:32:00.580468+0000","last_became_active":"2024-02-02T13:32:00.579242+0000","last_became_peered":"2024-02-02T13:32:00.579242+0000","last_unstale":"2024-02-02T13:32:00.580468+0000","last_undegraded":"2024-02-02T13:32:00.580468+0000","last_fullsized":"2024-02-02T13:32:00.580468+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:09:17.065112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578643+0000","last_change":"2024-02-02T13:32:00.576042+0000","last_active":"2024-02-02T13:32:00.578643+0000","last_peered":"2024-02-02T13:32:00.578643+0000","last_clean":"2024-02-02T13:32:00.578643+0000","last_became_active":"2024-02-02T13:32:00.574323+0000","last_became_peered":"2024-02-02T13:32:00.574323+0000","last_unstale":"2024-02-02T13:32:00.578643+0000","last_undegraded":"2024-02-02T13:32:00.578643+0000","last_fullsized":"2024-02-02T13:32:00.578643+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:27.030798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568204+0000","last_change":"2024-02-02T13:32:00.564327+0000","last_active":"2024-02-02T13:32:00.568204+0000","last_peered":"2024-02-02T13:32:00.568204+0000","last_clean":"2024-02-02T13:32:00.568204+0000","last_became_active":"2024-02-02T13:32:00.564174+0000","last_became_peered":"2024-02-02T13:32:00.564174+0000","last_unstale":"2024-02-02T13:32:00.568204+0000","last_undegraded":"2024-02-02T13:32:00.568204+0000","last_fullsized":"2024-02-02T13:32:00.568204+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:19:28.741542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572316+0000","last_change":"2024-02-02T13:32:00.571424+0000","last_active":"2024-02-02T13:32:00.572316+0000","last_peered":"2024-02-02T13:32:00.572316+0000","last_clean":"2024-02-02T13:32:00.572316+0000","last_became_active":"2024-02-02T13:32:00.570955+0000","last_became_peered":"2024-02-02T13:32:00.570955+0000","last_unstale":"2024-02-02T13:32:00.572316+0000","last_undegraded":"2024-02-02T13:32:00.572316+0000","last_fullsized":"2024-02-02T13:32:00.572316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:48:11.865231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583551+0000","last_change":"2024-02-02T13:32:00.583192+0000","last_active":"2024-02-02T13:32:00.583551+0000","last_peered":"2024-02-02T13:32:00.583551+0000","last_clean":"2024-02-02T13:32:00.583551+0000","last_became_active":"2024-02-02T13:32:00.582964+0000","last_became_peered":"2024-02-02T13:32:00.582964+0000","last_unstale":"2024-02-02T13:32:00.583551+0000","last_undegraded":"2024-02-02T13:32:00.583551+0000","last_fullsized":"2024-02-02T13:32:00.583551+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:37.598898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573749+0000","last_change":"2024-02-02T13:32:00.572710+0000","last_active":"2024-02-02T13:32:00.573749+0000","last_peered":"2024-02-02T13:32:00.573749+0000","last_clean":"2024-02-02T13:32:00.573749+0000","last_became_active":"2024-02-02T13:32:00.572494+0000","last_became_peered":"2024-02-02T13:32:00.572494+0000","last_unstale":"2024-02-02T13:32:00.573749+0000","last_undegraded":"2024-02-02T13:32:00.573749+0000","last_fullsized":"2024-02-02T13:32:00.573749+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:33:21.843675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578866+0000","last_change":"2024-02-02T13:32:00.577217+0000","last_active":"2024-02-02T13:32:00.578866+0000","last_peered":"2024-02-02T13:32:00.578866+0000","last_clean":"2024-02-02T13:32:00.578866+0000","last_became_active":"2024-02-02T13:32:00.574853+0000","last_became_peered":"2024-02-02T13:32:00.574853+0000","last_unstale":"2024-02-02T13:32:00.578866+0000","last_undegraded":"2024-02-02T13:32:00.578866+0000","last_fullsized":"2024-02-02T13:32:00.578866+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:17.955007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581712+0000","last_change":"2024-02-02T13:32:00.581102+0000","last_active":"2024-02-02T13:32:00.581712+0000","last_peered":"2024-02-02T13:32:00.581712+0000","last_clean":"2024-02-02T13:32:00.581712+0000","last_became_active":"2024-02-02T13:32:00.579971+0000","last_became_peered":"2024-02-02T13:32:00.579971+0000","last_unstale":"2024-02-02T13:32:00.581712+0000","last_undegraded":"2024-02-02T13:32:00.581712+0000","last_fullsized":"2024-02-02T13:32:00.581712+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:41:52.272275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566140+0000","last_change":"2024-02-02T13:32:00.560426+0000","last_active":"2024-02-02T13:32:00.566140+0000","last_peered":"2024-02-02T13:32:00.566140+0000","last_clean":"2024-02-02T13:32:00.566140+0000","last_became_active":"2024-02-02T13:32:00.560188+0000","last_became_peered":"2024-02-02T13:32:00.560188+0000","last_unstale":"2024-02-02T13:32:00.566140+0000","last_undegraded":"2024-02-02T13:32:00.566140+0000","last_fullsized":"2024-02-02T13:32:00.566140+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:31:42.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581674+0000","last_change":"2024-02-02T13:32:00.581610+0000","last_active":"2024-02-02T13:32:00.581674+0000","last_peered":"2024-02-02T13:32:00.581674+0000","last_clean":"2024-02-02T13:32:00.581674+0000","last_became_active":"2024-02-02T13:32:00.581328+0000","last_became_peered":"2024-02-02T13:32:00.581328+0000","last_unstale":"2024-02-02T13:32:00.581674+0000","last_undegraded":"2024-02-02T13:32:00.581674+0000","last_fullsized":"2024-02-02T13:32:00.581674+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:24:05.692536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583570+0000","last_change":"2024-02-02T13:32:00.583252+0000","last_active":"2024-02-02T13:32:00.583570+0000","last_peered":"2024-02-02T13:32:00.583570+0000","last_clean":"2024-02-02T13:32:00.583570+0000","last_became_active":"2024-02-02T13:32:00.581002+0000","last_became_peered":"2024-02-02T13:32:00.581002+0000","last_unstale":"2024-02-02T13:32:00.583570+0000","last_undegraded":"2024-02-02T13:32:00.583570+0000","last_fullsized":"2024-02-02T13:32:00.583570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:07:34.984416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582848+0000","last_change":"2024-02-02T13:32:00.581398+0000","last_active":"2024-02-02T13:32:00.582848+0000","last_peered":"2024-02-02T13:32:00.582848+0000","last_clean":"2024-02-02T13:32:00.582848+0000","last_became_active":"2024-02-02T13:32:00.580621+0000","last_became_peered":"2024-02-02T13:32:00.580621+0000","last_unstale":"2024-02-02T13:32:00.582848+0000","last_undegraded":"2024-02-02T13:32:00.582848+0000","last_fullsized":"2024-02-02T13:32:00.582848+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:34:45.721887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580119+0000","last_change":"2024-02-02T13:32:00.579565+0000","last_active":"2024-02-02T13:32:00.580119+0000","last_peered":"2024-02-02T13:32:00.580119+0000","last_clean":"2024-02-02T13:32:00.580119+0000","last_became_active":"2024-02-02T13:32:00.579061+0000","last_became_peered":"2024-02-02T13:32:00.579061+0000","last_unstale":"2024-02-02T13:32:00.580119+0000","last_undegraded":"2024-02-02T13:32:00.580119+0000","last_fullsized":"2024-02-02T13:32:00.580119+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:59.245267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579518+0000","last_change":"2024-02-02T13:32:00.579429+0000","last_active":"2024-02-02T13:32:00.579518+0000","last_peered":"2024-02-02T13:32:00.579518+0000","last_clean":"2024-02-02T13:32:00.579518+0000","last_became_active":"2024-02-02T13:32:00.579202+0000","last_became_peered":"2024-02-02T13:32:00.579202+0000","last_unstale":"2024-02-02T13:32:00.579518+0000","last_undegraded":"2024-02-02T13:32:00.579518+0000","last_fullsized":"2024-02-02T13:32:00.579518+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:06:08.782214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.570651+0000","last_change":"2024-02-02T13:32:00.569953+0000","last_active":"2024-02-02T13:32:00.570651+0000","last_peered":"2024-02-02T13:32:00.570651+0000","last_clean":"2024-02-02T13:32:00.570651+0000","last_became_active":"2024-02-02T13:32:00.569373+0000","last_became_peered":"2024-02-02T13:32:00.569373+0000","last_unstale":"2024-02-02T13:32:00.570651+0000","last_undegraded":"2024-02-02T13:32:00.570651+0000","last_fullsized":"2024-02-02T13:32:00.570651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:50.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579565+0000","last_change":"2024-02-02T13:32:00.579436+0000","last_active":"2024-02-02T13:32:00.579565+0000","last_peered":"2024-02-02T13:32:00.579565+0000","last_clean":"2024-02-02T13:32:00.579565+0000","last_became_active":"2024-02-02T13:32:00.579221+0000","last_became_peered":"2024-02-02T13:32:00.579221+0000","last_unstale":"2024-02-02T13:32:00.579565+0000","last_undegraded":"2024-02-02T13:32:00.579565+0000","last_fullsized":"2024-02-02T13:32:00.579565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:18.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577519+0000","last_change":"2024-02-02T13:32:00.577449+0000","last_active":"2024-02-02T13:32:00.577519+0000","last_peered":"2024-02-02T13:32:00.577519+0000","last_clean":"2024-02-02T13:32:00.577519+0000","last_became_active":"2024-02-02T13:32:00.576891+0000","last_became_peered":"2024-02-02T13:32:00.576891+0000","last_unstale":"2024-02-02T13:32:00.577519+0000","last_undegraded":"2024-02-02T13:32:00.577519+0000","last_fullsized":"2024-02-02T13:32:00.577519+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:48:35.351007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569489+0000","last_change":"2024-02-02T13:32:00.568508+0000","last_active":"2024-02-02T13:32:00.569489+0000","last_peered":"2024-02-02T13:32:00.569489+0000","last_clean":"2024-02-02T13:32:00.569489+0000","last_became_active":"2024-02-02T13:32:00.568122+0000","last_became_peered":"2024-02-02T13:32:00.568122+0000","last_unstale":"2024-02-02T13:32:00.569489+0000","last_undegraded":"2024-02-02T13:32:00.569489+0000","last_fullsized":"2024-02-02T13:32:00.569489+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:45.330479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581277+0000","last_change":"2024-02-02T13:32:00.581117+0000","last_active":"2024-02-02T13:32:00.581277+0000","last_peered":"2024-02-02T13:32:00.581277+0000","last_clean":"2024-02-02T13:32:00.581277+0000","last_became_active":"2024-02-02T13:32:00.580217+0000","last_became_peered":"2024-02-02T13:32:00.580217+0000","last_unstale":"2024-02-02T13:32:00.581277+0000","last_undegraded":"2024-02-02T13:32:00.581277+0000","last_fullsized":"2024-02-02T13:32:00.581277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:42:53.556889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577922+0000","last_change":"2024-02-02T13:32:00.577387+0000","last_active":"2024-02-02T13:32:00.577922+0000","last_peered":"2024-02-02T13:32:00.577922+0000","last_clean":"2024-02-02T13:32:00.577922+0000","last_became_active":"2024-02-02T13:32:00.573660+0000","last_became_peered":"2024-02-02T13:32:00.573660+0000","last_unstale":"2024-02-02T13:32:00.577922+0000","last_undegraded":"2024-02-02T13:32:00.577922+0000","last_fullsized":"2024-02-02T13:32:00.577922+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:47:59.184897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577829+0000","last_change":"2024-02-02T13:32:00.577513+0000","last_active":"2024-02-02T13:32:00.577829+0000","last_peered":"2024-02-02T13:32:00.577829+0000","last_clean":"2024-02-02T13:32:00.577829+0000","last_became_active":"2024-02-02T13:32:00.577306+0000","last_became_peered":"2024-02-02T13:32:00.577306+0000","last_unstale":"2024-02-02T13:32:00.577829+0000","last_undegraded":"2024-02-02T13:32:00.577829+0000","last_fullsized":"2024-02-02T13:32:00.577829+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:47.908651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574734+0000","last_change":"2024-02-02T13:32:00.573796+0000","last_active":"2024-02-02T13:32:00.574734+0000","last_peered":"2024-02-02T13:32:00.574734+0000","last_clean":"2024-02-02T13:32:00.574734+0000","last_became_active":"2024-02-02T13:32:00.573580+0000","last_became_peered":"2024-02-02T13:32:00.573580+0000","last_unstale":"2024-02-02T13:32:00.574734+0000","last_undegraded":"2024-02-02T13:32:00.574734+0000","last_fullsized":"2024-02-02T13:32:00.574734+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:11.808415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571986+0000","last_change":"2024-02-02T13:32:00.571784+0000","last_active":"2024-02-02T13:32:00.571986+0000","last_peered":"2024-02-02T13:32:00.571986+0000","last_clean":"2024-02-02T13:32:00.571986+0000","last_became_active":"2024-02-02T13:32:00.571416+0000","last_became_peered":"2024-02-02T13:32:00.571416+0000","last_unstale":"2024-02-02T13:32:00.571986+0000","last_undegraded":"2024-02-02T13:32:00.571986+0000","last_fullsized":"2024-02-02T13:32:00.571986+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:29.953865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.564952+0000","last_change":"2024-02-02T13:32:00.564708+0000","last_active":"2024-02-02T13:32:00.564952+0000","last_peered":"2024-02-02T13:32:00.564952+0000","last_clean":"2024-02-02T13:32:00.564952+0000","last_became_active":"2024-02-02T13:32:00.563986+0000","last_became_peered":"2024-02-02T13:32:00.563986+0000","last_unstale":"2024-02-02T13:32:00.564952+0000","last_undegraded":"2024-02-02T13:32:00.564952+0000","last_fullsized":"2024-02-02T13:32:00.564952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:55:24.734834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577903+0000","last_change":"2024-02-02T13:32:00.577227+0000","last_active":"2024-02-02T13:32:00.577903+0000","last_peered":"2024-02-02T13:32:00.577903+0000","last_clean":"2024-02-02T13:32:00.577903+0000","last_became_active":"2024-02-02T13:32:00.573346+0000","last_became_peered":"2024-02-02T13:32:00.573346+0000","last_unstale":"2024-02-02T13:32:00.577903+0000","last_undegraded":"2024-02-02T13:32:00.577903+0000","last_fullsized":"2024-02-02T13:32:00.577903+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:03:26.888341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560160+0000","last_change":"2024-02-02T13:32:00.559340+0000","last_active":"2024-02-02T13:32:00.560160+0000","last_peered":"2024-02-02T13:32:00.560160+0000","last_clean":"2024-02-02T13:32:00.560160+0000","last_became_active":"2024-02-02T13:32:00.559122+0000","last_became_peered":"2024-02-02T13:32:00.559122+0000","last_unstale":"2024-02-02T13:32:00.560160+0000","last_undegraded":"2024-02-02T13:32:00.560160+0000","last_fullsized":"2024-02-02T13:32:00.560160+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:51:44.876724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.576036+0000","last_change":"2024-02-02T13:32:00.575891+0000","last_active":"2024-02-02T13:32:00.576036+0000","last_peered":"2024-02-02T13:32:00.576036+0000","last_clean":"2024-02-02T13:32:00.576036+0000","last_became_active":"2024-02-02T13:32:00.575414+0000","last_became_peered":"2024-02-02T13:32:00.575414+0000","last_unstale":"2024-02-02T13:32:00.576036+0000","last_undegraded":"2024-02-02T13:32:00.576036+0000","last_fullsized":"2024-02-02T13:32:00.576036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:55.558567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"142'4","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580970+0000","last_change":"2024-02-02T13:32:00.079236+0000","last_active":"2024-02-02T13:32:00.580970+0000","last_peered":"2024-02-02T13:32:00.580970+0000","last_clean":"2024-02-02T13:32:00.580970+0000","last_became_active":"2024-02-02T13:32:00.078620+0000","last_became_peered":"2024-02-02T13:32:00.078620+0000","last_unstale":"2024-02-02T13:32:00.580970+0000","last_undegraded":"2024-02-02T13:32:00.580970+0000","last_fullsized":"2024-02-02T13:32:00.580970+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:46.893892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582878+0000","last_change":"2024-02-02T13:32:00.582490+0000","last_active":"2024-02-02T13:32:00.582878+0000","last_peered":"2024-02-02T13:32:00.582878+0000","last_clean":"2024-02-02T13:32:00.582878+0000","last_became_active":"2024-02-02T13:32:00.581686+0000","last_became_peered":"2024-02-02T13:32:00.581686+0000","last_unstale":"2024-02-02T13:32:00.582878+0000","last_undegraded":"2024-02-02T13:32:00.582878+0000","last_fullsized":"2024-02-02T13:32:00.582878+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:24:03.225415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.584122+0000","last_change":"2024-02-02T13:32:00.077561+0000","last_active":"2024-02-02T13:32:00.584122+0000","last_peered":"2024-02-02T13:32:00.584122+0000","last_clean":"2024-02-02T13:32:00.584122+0000","last_became_active":"2024-02-02T13:32:00.076903+0000","last_became_peered":"2024-02-02T13:32:00.076903+0000","last_unstale":"2024-02-02T13:32:00.584122+0000","last_undegraded":"2024-02-02T13:32:00.584122+0000","last_fullsized":"2024-02-02T13:32:00.584122+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:49.142735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566194+0000","last_change":"2024-02-02T13:32:00.564733+0000","last_active":"2024-02-02T13:32:00.566194+0000","last_peered":"2024-02-02T13:32:00.566194+0000","last_clean":"2024-02-02T13:32:00.566194+0000","last_became_active":"2024-02-02T13:32:00.564157+0000","last_became_peered":"2024-02-02T13:32:00.564157+0000","last_unstale":"2024-02-02T13:32:00.566194+0000","last_undegraded":"2024-02-02T13:32:00.566194+0000","last_fullsized":"2024-02-02T13:32:00.566194+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:32:33.903977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578791+0000","last_change":"2024-02-02T13:32:00.096516+0000","last_active":"2024-02-02T13:32:00.578791+0000","last_peered":"2024-02-02T13:32:00.578791+0000","last_clean":"2024-02-02T13:32:00.578791+0000","last_became_active":"2024-02-02T13:32:00.096282+0000","last_became_peered":"2024-02-02T13:32:00.096282+0000","last_unstale":"2024-02-02T13:32:00.578791+0000","last_undegraded":"2024-02-02T13:32:00.578791+0000","last_fullsized":"2024-02-02T13:32:00.578791+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:34:49.959878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578849+0000","last_change":"2024-02-02T13:32:00.577094+0000","last_active":"2024-02-02T13:32:00.578849+0000","last_peered":"2024-02-02T13:32:00.578849+0000","last_clean":"2024-02-02T13:32:00.578849+0000","last_became_active":"2024-02-02T13:32:00.575937+0000","last_became_peered":"2024-02-02T13:32:00.575937+0000","last_unstale":"2024-02-02T13:32:00.578849+0000","last_undegraded":"2024-02-02T13:32:00.578849+0000","last_fullsized":"2024-02-02T13:32:00.578849+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:11.808415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580865+0000","last_change":"2024-02-02T13:32:00.090475+0000","last_active":"2024-02-02T13:32:00.580865+0000","last_peered":"2024-02-02T13:32:00.580865+0000","last_clean":"2024-02-02T13:32:00.580865+0000","last_became_active":"2024-02-02T13:32:00.089886+0000","last_became_peered":"2024-02-02T13:32:00.089886+0000","last_unstale":"2024-02-02T13:32:00.580865+0000","last_undegraded":"2024-02-02T13:32:00.580865+0000","last_fullsized":"2024-02-02T13:32:00.580865+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:34.636450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.577937+0000","last_change":"2024-02-02T13:32:00.577540+0000","last_active":"2024-02-02T13:32:00.577937+0000","last_peered":"2024-02-02T13:32:00.577937+0000","last_clean":"2024-02-02T13:32:00.577937+0000","last_became_active":"2024-02-02T13:32:00.575838+0000","last_became_peered":"2024-02-02T13:32:00.575838+0000","last_unstale":"2024-02-02T13:32:00.577937+0000","last_undegraded":"2024-02-02T13:32:00.577937+0000","last_fullsized":"2024-02-02T13:32:00.577937+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:49:39.975516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579409+0000","last_change":"2024-02-02T13:32:00.090351+0000","last_active":"2024-02-02T13:32:00.579409+0000","last_peered":"2024-02-02T13:32:00.579409+0000","last_clean":"2024-02-02T13:32:00.579409+0000","last_became_active":"2024-02-02T13:32:00.087899+0000","last_became_peered":"2024-02-02T13:32:00.087899+0000","last_unstale":"2024-02-02T13:32:00.579409+0000","last_undegraded":"2024-02-02T13:32:00.579409+0000","last_fullsized":"2024-02-02T13:32:00.579409+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:42:41.531516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.583713+0000","last_change":"2024-02-02T13:32:00.583636+0000","last_active":"2024-02-02T13:32:00.583713+0000","last_peered":"2024-02-02T13:32:00.583713+0000","last_clean":"2024-02-02T13:32:00.583713+0000","last_became_active":"2024-02-02T13:32:00.582063+0000","last_became_peered":"2024-02-02T13:32:00.582063+0000","last_unstale":"2024-02-02T13:32:00.583713+0000","last_undegraded":"2024-02-02T13:32:00.583713+0000","last_fullsized":"2024-02-02T13:32:00.583713+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:38:11.453071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580913+0000","last_change":"2024-02-02T13:32:00.094741+0000","last_active":"2024-02-02T13:32:00.580913+0000","last_peered":"2024-02-02T13:32:00.580913+0000","last_clean":"2024-02-02T13:32:00.580913+0000","last_became_active":"2024-02-02T13:32:00.094330+0000","last_became_peered":"2024-02-02T13:32:00.094330+0000","last_unstale":"2024-02-02T13:32:00.580913+0000","last_undegraded":"2024-02-02T13:32:00.580913+0000","last_fullsized":"2024-02-02T13:32:00.580913+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:12:55.870178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572301+0000","last_change":"2024-02-02T13:32:00.571300+0000","last_active":"2024-02-02T13:32:00.572301+0000","last_peered":"2024-02-02T13:32:00.572301+0000","last_clean":"2024-02-02T13:32:00.572301+0000","last_became_active":"2024-02-02T13:32:00.570646+0000","last_became_peered":"2024-02-02T13:32:00.570646+0000","last_unstale":"2024-02-02T13:32:00.572301+0000","last_undegraded":"2024-02-02T13:32:00.572301+0000","last_fullsized":"2024-02-02T13:32:00.572301+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:36:05.151351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562794+0000","last_change":"2024-02-02T13:32:00.090412+0000","last_active":"2024-02-02T13:32:00.562794+0000","last_peered":"2024-02-02T13:32:00.562794+0000","last_clean":"2024-02-02T13:32:00.562794+0000","last_became_active":"2024-02-02T13:32:00.090122+0000","last_became_peered":"2024-02-02T13:32:00.090122+0000","last_unstale":"2024-02-02T13:32:00.562794+0000","last_undegraded":"2024-02-02T13:32:00.562794+0000","last_fullsized":"2024-02-02T13:32:00.562794+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:39.901199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581234+0000","last_change":"2024-02-02T13:32:00.580995+0000","last_active":"2024-02-02T13:32:00.581234+0000","last_peered":"2024-02-02T13:32:00.581234+0000","last_clean":"2024-02-02T13:32:00.581234+0000","last_became_active":"2024-02-02T13:32:00.580767+0000","last_became_peered":"2024-02-02T13:32:00.580767+0000","last_unstale":"2024-02-02T13:32:00.581234+0000","last_undegraded":"2024-02-02T13:32:00.581234+0000","last_fullsized":"2024-02-02T13:32:00.581234+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:22:40.859966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.563399+0000","last_change":"2024-02-02T13:32:00.097067+0000","last_active":"2024-02-02T13:32:00.563399+0000","last_peered":"2024-02-02T13:32:00.563399+0000","last_clean":"2024-02-02T13:32:00.563399+0000","last_became_active":"2024-02-02T13:32:00.096803+0000","last_became_peered":"2024-02-02T13:32:00.096803+0000","last_unstale":"2024-02-02T13:32:00.563399+0000","last_undegraded":"2024-02-02T13:32:00.563399+0000","last_fullsized":"2024-02-02T13:32:00.563399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:29.156726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573961+0000","last_change":"2024-02-02T13:32:00.571908+0000","last_active":"2024-02-02T13:32:00.573961+0000","last_peered":"2024-02-02T13:32:00.573961+0000","last_clean":"2024-02-02T13:32:00.573961+0000","last_became_active":"2024-02-02T13:32:00.571350+0000","last_became_peered":"2024-02-02T13:32:00.571350+0000","last_unstale":"2024-02-02T13:32:00.573961+0000","last_undegraded":"2024-02-02T13:32:00.573961+0000","last_fullsized":"2024-02-02T13:32:00.573961+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:23:59.823167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.561475+0000","last_change":"2024-02-02T13:32:00.090227+0000","last_active":"2024-02-02T13:32:00.561475+0000","last_peered":"2024-02-02T13:32:00.561475+0000","last_clean":"2024-02-02T13:32:00.561475+0000","last_became_active":"2024-02-02T13:32:00.089426+0000","last_became_peered":"2024-02-02T13:32:00.089426+0000","last_unstale":"2024-02-02T13:32:00.561475+0000","last_undegraded":"2024-02-02T13:32:00.561475+0000","last_fullsized":"2024-02-02T13:32:00.561475+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:56:44.272405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582863+0000","last_change":"2024-02-02T13:32:00.580509+0000","last_active":"2024-02-02T13:32:00.582863+0000","last_peered":"2024-02-02T13:32:00.582863+0000","last_clean":"2024-02-02T13:32:00.582863+0000","last_became_active":"2024-02-02T13:32:00.578471+0000","last_became_peered":"2024-02-02T13:32:00.578471+0000","last_unstale":"2024-02-02T13:32:00.582863+0000","last_undegraded":"2024-02-02T13:32:00.582863+0000","last_fullsized":"2024-02-02T13:32:00.582863+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:59.311899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.552101+0000","last_change":"2024-02-02T13:32:00.091859+0000","last_active":"2024-02-02T13:32:00.552101+0000","last_peered":"2024-02-02T13:32:00.552101+0000","last_clean":"2024-02-02T13:32:00.552101+0000","last_became_active":"2024-02-02T13:32:00.091294+0000","last_became_peered":"2024-02-02T13:32:00.091294+0000","last_unstale":"2024-02-02T13:32:00.552101+0000","last_undegraded":"2024-02-02T13:32:00.552101+0000","last_fullsized":"2024-02-02T13:32:00.552101+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:47:49.142735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569968+0000","last_change":"2024-02-02T13:32:00.569903+0000","last_active":"2024-02-02T13:32:00.569968+0000","last_peered":"2024-02-02T13:32:00.569968+0000","last_clean":"2024-02-02T13:32:00.569968+0000","last_became_active":"2024-02-02T13:32:00.569700+0000","last_became_peered":"2024-02-02T13:32:00.569700+0000","last_unstale":"2024-02-02T13:32:00.569968+0000","last_undegraded":"2024-02-02T13:32:00.569968+0000","last_fullsized":"2024-02-02T13:32:00.569968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:59.245267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.581217+0000","last_change":"2024-02-02T13:32:00.089704+0000","last_active":"2024-02-02T13:32:00.581217+0000","last_peered":"2024-02-02T13:32:00.581217+0000","last_clean":"2024-02-02T13:32:00.581217+0000","last_became_active":"2024-02-02T13:32:00.089447+0000","last_became_peered":"2024-02-02T13:32:00.089447+0000","last_unstale":"2024-02-02T13:32:00.581217+0000","last_undegraded":"2024-02-02T13:32:00.581217+0000","last_fullsized":"2024-02-02T13:32:00.581217+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:59:08.318315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.570316+0000","last_change":"2024-02-02T13:32:00.569392+0000","last_active":"2024-02-02T13:32:00.570316+0000","last_peered":"2024-02-02T13:32:00.570316+0000","last_clean":"2024-02-02T13:32:00.570316+0000","last_became_active":"2024-02-02T13:32:00.568960+0000","last_became_peered":"2024-02-02T13:32:00.568960+0000","last_unstale":"2024-02-02T13:32:00.570316+0000","last_undegraded":"2024-02-02T13:32:00.570316+0000","last_fullsized":"2024-02-02T13:32:00.570316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:58:12.612972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582144+0000","last_change":"2024-02-02T13:32:00.092756+0000","last_active":"2024-02-02T13:32:00.582144+0000","last_peered":"2024-02-02T13:32:00.582144+0000","last_clean":"2024-02-02T13:32:00.582144+0000","last_became_active":"2024-02-02T13:32:00.092077+0000","last_became_peered":"2024-02-02T13:32:00.092077+0000","last_unstale":"2024-02-02T13:32:00.582144+0000","last_undegraded":"2024-02-02T13:32:00.582144+0000","last_fullsized":"2024-02-02T13:32:00.582144+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:29:58.230507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.563343+0000","last_change":"2024-02-02T13:32:00.563274+0000","last_active":"2024-02-02T13:32:00.563343+0000","last_peered":"2024-02-02T13:32:00.563343+0000","last_clean":"2024-02-02T13:32:00.563343+0000","last_became_active":"2024-02-02T13:32:00.563041+0000","last_became_peered":"2024-02-02T13:32:00.563041+0000","last_unstale":"2024-02-02T13:32:00.563343+0000","last_undegraded":"2024-02-02T13:32:00.563343+0000","last_fullsized":"2024-02-02T13:32:00.563343+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:11:30.171486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578892+0000","last_change":"2024-02-02T13:32:00.087069+0000","last_active":"2024-02-02T13:32:00.578892+0000","last_peered":"2024-02-02T13:32:00.578892+0000","last_clean":"2024-02-02T13:32:00.578892+0000","last_became_active":"2024-02-02T13:32:00.086799+0000","last_became_peered":"2024-02-02T13:32:00.086799+0000","last_unstale":"2024-02-02T13:32:00.578892+0000","last_undegraded":"2024-02-02T13:32:00.578892+0000","last_fullsized":"2024-02-02T13:32:00.578892+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:45:51.365321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.565028+0000","last_change":"2024-02-02T13:32:00.564934+0000","last_active":"2024-02-02T13:32:00.565028+0000","last_peered":"2024-02-02T13:32:00.565028+0000","last_clean":"2024-02-02T13:32:00.565028+0000","last_became_active":"2024-02-02T13:32:00.564577+0000","last_became_peered":"2024-02-02T13:32:00.564577+0000","last_unstale":"2024-02-02T13:32:00.565028+0000","last_undegraded":"2024-02-02T13:32:00.565028+0000","last_fullsized":"2024-02-02T13:32:00.565028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:47.088654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571138+0000","last_change":"2024-02-02T13:32:00.095149+0000","last_active":"2024-02-02T13:32:00.571138+0000","last_peered":"2024-02-02T13:32:00.571138+0000","last_clean":"2024-02-02T13:32:00.571138+0000","last_became_active":"2024-02-02T13:32:00.094875+0000","last_became_peered":"2024-02-02T13:32:00.094875+0000","last_unstale":"2024-02-02T13:32:00.571138+0000","last_undegraded":"2024-02-02T13:32:00.571138+0000","last_fullsized":"2024-02-02T13:32:00.571138+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:28.939106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572214+0000","last_change":"2024-02-02T13:32:00.572115+0000","last_active":"2024-02-02T13:32:00.572214+0000","last_peered":"2024-02-02T13:32:00.572214+0000","last_clean":"2024-02-02T13:32:00.572214+0000","last_became_active":"2024-02-02T13:32:00.571775+0000","last_became_peered":"2024-02-02T13:32:00.571775+0000","last_unstale":"2024-02-02T13:32:00.572214+0000","last_undegraded":"2024-02-02T13:32:00.572214+0000","last_fullsized":"2024-02-02T13:32:00.572214+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:58.765170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.579332+0000","last_change":"2024-02-02T13:32:00.095339+0000","last_active":"2024-02-02T13:32:00.579332+0000","last_peered":"2024-02-02T13:32:00.579332+0000","last_clean":"2024-02-02T13:32:00.579332+0000","last_became_active":"2024-02-02T13:32:00.095122+0000","last_became_peered":"2024-02-02T13:32:00.095122+0000","last_unstale":"2024-02-02T13:32:00.579332+0000","last_undegraded":"2024-02-02T13:32:00.579332+0000","last_fullsized":"2024-02-02T13:32:00.579332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:59.441599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.571546+0000","last_change":"2024-02-02T13:32:00.571311+0000","last_active":"2024-02-02T13:32:00.571546+0000","last_peered":"2024-02-02T13:32:00.571546+0000","last_clean":"2024-02-02T13:32:00.571546+0000","last_became_active":"2024-02-02T13:32:00.571101+0000","last_became_peered":"2024-02-02T13:32:00.571101+0000","last_unstale":"2024-02-02T13:32:00.571546+0000","last_undegraded":"2024-02-02T13:32:00.571546+0000","last_fullsized":"2024-02-02T13:32:00.571546+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:27:11.434157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574212+0000","last_change":"2024-02-02T13:32:00.092916+0000","last_active":"2024-02-02T13:32:00.574212+0000","last_peered":"2024-02-02T13:32:00.574212+0000","last_clean":"2024-02-02T13:32:00.574212+0000","last_became_active":"2024-02-02T13:32:00.092397+0000","last_became_peered":"2024-02-02T13:32:00.092397+0000","last_unstale":"2024-02-02T13:32:00.574212+0000","last_undegraded":"2024-02-02T13:32:00.574212+0000","last_fullsized":"2024-02-02T13:32:00.574212+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:56:05.207739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.566271+0000","last_change":"2024-02-02T13:32:00.565792+0000","last_active":"2024-02-02T13:32:00.566271+0000","last_peered":"2024-02-02T13:32:00.566271+0000","last_clean":"2024-02-02T13:32:00.566271+0000","last_became_active":"2024-02-02T13:32:00.565566+0000","last_became_peered":"2024-02-02T13:32:00.565566+0000","last_unstale":"2024-02-02T13:32:00.566271+0000","last_undegraded":"2024-02-02T13:32:00.566271+0000","last_fullsized":"2024-02-02T13:32:00.566271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:19:07.790844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578554+0000","last_change":"2024-02-02T13:32:00.093809+0000","last_active":"2024-02-02T13:32:00.578554+0000","last_peered":"2024-02-02T13:32:00.578554+0000","last_clean":"2024-02-02T13:32:00.578554+0000","last_became_active":"2024-02-02T13:32:00.093246+0000","last_became_peered":"2024-02-02T13:32:00.093246+0000","last_unstale":"2024-02-02T13:32:00.578554+0000","last_undegraded":"2024-02-02T13:32:00.578554+0000","last_fullsized":"2024-02-02T13:32:00.578554+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:54:08.561101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578611+0000","last_change":"2024-02-02T13:32:00.575319+0000","last_active":"2024-02-02T13:32:00.578611+0000","last_peered":"2024-02-02T13:32:00.578611+0000","last_clean":"2024-02-02T13:32:00.578611+0000","last_became_active":"2024-02-02T13:32:00.574981+0000","last_became_peered":"2024-02-02T13:32:00.574981+0000","last_unstale":"2024-02-02T13:32:00.578611+0000","last_undegraded":"2024-02-02T13:32:00.578611+0000","last_fullsized":"2024-02-02T13:32:00.578611+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:12:47.109639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578846+0000","last_change":"2024-02-02T13:32:00.092851+0000","last_active":"2024-02-02T13:32:00.578846+0000","last_peered":"2024-02-02T13:32:00.578846+0000","last_clean":"2024-02-02T13:32:00.578846+0000","last_became_active":"2024-02-02T13:32:00.090804+0000","last_became_peered":"2024-02-02T13:32:00.090804+0000","last_unstale":"2024-02-02T13:32:00.578846+0000","last_undegraded":"2024-02-02T13:32:00.578846+0000","last_fullsized":"2024-02-02T13:32:00.578846+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:15.508475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.567719+0000","last_change":"2024-02-02T13:32:00.567461+0000","last_active":"2024-02-02T13:32:00.567719+0000","last_peered":"2024-02-02T13:32:00.567719+0000","last_clean":"2024-02-02T13:32:00.567719+0000","last_became_active":"2024-02-02T13:32:00.566827+0000","last_became_peered":"2024-02-02T13:32:00.566827+0000","last_unstale":"2024-02-02T13:32:00.567719+0000","last_undegraded":"2024-02-02T13:32:00.567719+0000","last_fullsized":"2024-02-02T13:32:00.567719+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:36:10.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562859+0000","last_change":"2024-02-02T13:32:00.094242+0000","last_active":"2024-02-02T13:32:00.562859+0000","last_peered":"2024-02-02T13:32:00.562859+0000","last_clean":"2024-02-02T13:32:00.562859+0000","last_became_active":"2024-02-02T13:32:00.093786+0000","last_became_peered":"2024-02-02T13:32:00.093786+0000","last_unstale":"2024-02-02T13:32:00.562859+0000","last_undegraded":"2024-02-02T13:32:00.562859+0000","last_fullsized":"2024-02-02T13:32:00.562859+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:19.155804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578495+0000","last_change":"2024-02-02T13:32:00.572255+0000","last_active":"2024-02-02T13:32:00.578495+0000","last_peered":"2024-02-02T13:32:00.578495+0000","last_clean":"2024-02-02T13:32:00.578495+0000","last_became_active":"2024-02-02T13:32:00.571907+0000","last_became_peered":"2024-02-02T13:32:00.571907+0000","last_unstale":"2024-02-02T13:32:00.578495+0000","last_undegraded":"2024-02-02T13:32:00.578495+0000","last_fullsized":"2024-02-02T13:32:00.578495+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:33.795196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580432+0000","last_change":"2024-02-02T13:32:00.087915+0000","last_active":"2024-02-02T13:32:00.580432+0000","last_peered":"2024-02-02T13:32:00.580432+0000","last_clean":"2024-02-02T13:32:00.580432+0000","last_became_active":"2024-02-02T13:32:00.087644+0000","last_became_peered":"2024-02-02T13:32:00.087644+0000","last_unstale":"2024-02-02T13:32:00.580432+0000","last_undegraded":"2024-02-02T13:32:00.580432+0000","last_fullsized":"2024-02-02T13:32:00.580432+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:22:21.773079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.575624+0000","last_change":"2024-02-02T13:32:00.575567+0000","last_active":"2024-02-02T13:32:00.575624+0000","last_peered":"2024-02-02T13:32:00.575624+0000","last_clean":"2024-02-02T13:32:00.575624+0000","last_became_active":"2024-02-02T13:32:00.574030+0000","last_became_peered":"2024-02-02T13:32:00.574030+0000","last_unstale":"2024-02-02T13:32:00.575624+0000","last_undegraded":"2024-02-02T13:32:00.575624+0000","last_fullsized":"2024-02-02T13:32:00.575624+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:49:06.430460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560561+0000","last_change":"2024-02-02T13:32:00.088439+0000","last_active":"2024-02-02T13:32:00.560561+0000","last_peered":"2024-02-02T13:32:00.560561+0000","last_clean":"2024-02-02T13:32:00.560561+0000","last_became_active":"2024-02-02T13:32:00.087866+0000","last_became_peered":"2024-02-02T13:32:00.087866+0000","last_unstale":"2024-02-02T13:32:00.560561+0000","last_undegraded":"2024-02-02T13:32:00.560561+0000","last_fullsized":"2024-02-02T13:32:00.560561+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:44:18.471870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.578626+0000","last_change":"2024-02-02T13:32:00.575655+0000","last_active":"2024-02-02T13:32:00.578626+0000","last_peered":"2024-02-02T13:32:00.578626+0000","last_clean":"2024-02-02T13:32:00.578626+0000","last_became_active":"2024-02-02T13:32:00.575354+0000","last_became_peered":"2024-02-02T13:32:00.575354+0000","last_unstale":"2024-02-02T13:32:00.578626+0000","last_undegraded":"2024-02-02T13:32:00.578626+0000","last_fullsized":"2024-02-02T13:32:00.578626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:45.330479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.556643+0000","last_change":"2024-02-02T13:32:00.089831+0000","last_active":"2024-02-02T13:32:00.556643+0000","last_peered":"2024-02-02T13:32:00.556643+0000","last_clean":"2024-02-02T13:32:00.556643+0000","last_became_active":"2024-02-02T13:32:00.089471+0000","last_became_peered":"2024-02-02T13:32:00.089471+0000","last_unstale":"2024-02-02T13:32:00.556643+0000","last_undegraded":"2024-02-02T13:32:00.556643+0000","last_fullsized":"2024-02-02T13:32:00.556643+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:52:11.731967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573100+0000","last_change":"2024-02-02T13:32:00.573034+0000","last_active":"2024-02-02T13:32:00.573100+0000","last_peered":"2024-02-02T13:32:00.573100+0000","last_clean":"2024-02-02T13:32:00.573100+0000","last_became_active":"2024-02-02T13:32:00.572713+0000","last_became_peered":"2024-02-02T13:32:00.572713+0000","last_unstale":"2024-02-02T13:32:00.573100+0000","last_undegraded":"2024-02-02T13:32:00.573100+0000","last_fullsized":"2024-02-02T13:32:00.573100+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:40:17.955007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568999+0000","last_change":"2024-02-02T13:32:00.094234+0000","last_active":"2024-02-02T13:32:00.568999+0000","last_peered":"2024-02-02T13:32:00.568999+0000","last_clean":"2024-02-02T13:32:00.568999+0000","last_became_active":"2024-02-02T13:32:00.093955+0000","last_became_peered":"2024-02-02T13:32:00.093955+0000","last_unstale":"2024-02-02T13:32:00.568999+0000","last_undegraded":"2024-02-02T13:32:00.568999+0000","last_fullsized":"2024-02-02T13:32:00.568999+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:46.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582893+0000","last_change":"2024-02-02T13:32:00.582667+0000","last_active":"2024-02-02T13:32:00.582893+0000","last_peered":"2024-02-02T13:32:00.582893+0000","last_clean":"2024-02-02T13:32:00.582893+0000","last_became_active":"2024-02-02T13:32:00.582204+0000","last_became_peered":"2024-02-02T13:32:00.582204+0000","last_unstale":"2024-02-02T13:32:00.582893+0000","last_undegraded":"2024-02-02T13:32:00.582893+0000","last_fullsized":"2024-02-02T13:32:00.582893+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:59:50.613919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.574886+0000","last_change":"2024-02-02T13:32:00.096526+0000","last_active":"2024-02-02T13:32:00.574886+0000","last_peered":"2024-02-02T13:32:00.574886+0000","last_clean":"2024-02-02T13:32:00.574886+0000","last_became_active":"2024-02-02T13:32:00.094279+0000","last_became_peered":"2024-02-02T13:32:00.094279+0000","last_unstale":"2024-02-02T13:32:00.574886+0000","last_undegraded":"2024-02-02T13:32:00.574886+0000","last_fullsized":"2024-02-02T13:32:00.574886+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:28.939106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573776+0000","last_change":"2024-02-02T13:32:00.573199+0000","last_active":"2024-02-02T13:32:00.573776+0000","last_peered":"2024-02-02T13:32:00.573776+0000","last_clean":"2024-02-02T13:32:00.573776+0000","last_became_active":"2024-02-02T13:32:00.572964+0000","last_became_peered":"2024-02-02T13:32:00.572964+0000","last_unstale":"2024-02-02T13:32:00.573776+0000","last_undegraded":"2024-02-02T13:32:00.573776+0000","last_fullsized":"2024-02-02T13:32:00.573776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:15:49.748992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573628+0000","last_change":"2024-02-02T13:32:00.093531+0000","last_active":"2024-02-02T13:32:00.573628+0000","last_peered":"2024-02-02T13:32:00.573628+0000","last_clean":"2024-02-02T13:32:00.573628+0000","last_became_active":"2024-02-02T13:32:00.091671+0000","last_became_peered":"2024-02-02T13:32:00.091671+0000","last_unstale":"2024-02-02T13:32:00.573628+0000","last_undegraded":"2024-02-02T13:32:00.573628+0000","last_fullsized":"2024-02-02T13:32:00.573628+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:22:20.921907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.572281+0000","last_change":"2024-02-02T13:32:00.571172+0000","last_active":"2024-02-02T13:32:00.572281+0000","last_peered":"2024-02-02T13:32:00.572281+0000","last_clean":"2024-02-02T13:32:00.572281+0000","last_became_active":"2024-02-02T13:32:00.570352+0000","last_became_peered":"2024-02-02T13:32:00.570352+0000","last_unstale":"2024-02-02T13:32:00.572281+0000","last_undegraded":"2024-02-02T13:32:00.572281+0000","last_fullsized":"2024-02-02T13:32:00.572281+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:04:58.904061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562610+0000","last_change":"2024-02-02T13:32:00.092626+0000","last_active":"2024-02-02T13:32:00.562610+0000","last_peered":"2024-02-02T13:32:00.562610+0000","last_clean":"2024-02-02T13:32:00.562610+0000","last_became_active":"2024-02-02T13:32:00.091569+0000","last_became_peered":"2024-02-02T13:32:00.091569+0000","last_unstale":"2024-02-02T13:32:00.562610+0000","last_undegraded":"2024-02-02T13:32:00.562610+0000","last_fullsized":"2024-02-02T13:32:00.562610+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:12:44.315720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582907+0000","last_change":"2024-02-02T13:32:00.582776+0000","last_active":"2024-02-02T13:32:00.582907+0000","last_peered":"2024-02-02T13:32:00.582907+0000","last_clean":"2024-02-02T13:32:00.582907+0000","last_became_active":"2024-02-02T13:32:00.581988+0000","last_became_peered":"2024-02-02T13:32:00.581988+0000","last_unstale":"2024-02-02T13:32:00.582907+0000","last_undegraded":"2024-02-02T13:32:00.582907+0000","last_fullsized":"2024-02-02T13:32:00.582907+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:00:30.493159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.551947+0000","last_change":"2024-02-02T13:32:00.083671+0000","last_active":"2024-02-02T13:32:00.551947+0000","last_peered":"2024-02-02T13:32:00.551947+0000","last_clean":"2024-02-02T13:32:00.551947+0000","last_became_active":"2024-02-02T13:32:00.083395+0000","last_became_peered":"2024-02-02T13:32:00.083395+0000","last_unstale":"2024-02-02T13:32:00.551947+0000","last_undegraded":"2024-02-02T13:32:00.551947+0000","last_fullsized":"2024-02-02T13:32:00.551947+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:15.508475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.576785+0000","last_change":"2024-02-02T13:32:00.575210+0000","last_active":"2024-02-02T13:32:00.576785+0000","last_peered":"2024-02-02T13:32:00.576785+0000","last_clean":"2024-02-02T13:32:00.576785+0000","last_became_active":"2024-02-02T13:32:00.574736+0000","last_became_peered":"2024-02-02T13:32:00.574736+0000","last_unstale":"2024-02-02T13:32:00.576785+0000","last_undegraded":"2024-02-02T13:32:00.576785+0000","last_fullsized":"2024-02-02T13:32:00.576785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:20.518706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.560590+0000","last_change":"2024-02-02T13:32:00.095860+0000","last_active":"2024-02-02T13:32:00.560590+0000","last_peered":"2024-02-02T13:32:00.560590+0000","last_clean":"2024-02-02T13:32:00.560590+0000","last_became_active":"2024-02-02T13:32:00.094658+0000","last_became_peered":"2024-02-02T13:32:00.094658+0000","last_unstale":"2024-02-02T13:32:00.560590+0000","last_undegraded":"2024-02-02T13:32:00.560590+0000","last_fullsized":"2024-02-02T13:32:00.560590+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:54.543807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.573427+0000","last_change":"2024-02-02T13:32:00.573299+0000","last_active":"2024-02-02T13:32:00.573427+0000","last_peered":"2024-02-02T13:32:00.573427+0000","last_clean":"2024-02-02T13:32:00.573427+0000","last_became_active":"2024-02-02T13:32:00.572947+0000","last_became_peered":"2024-02-02T13:32:00.572947+0000","last_unstale":"2024-02-02T13:32:00.573427+0000","last_undegraded":"2024-02-02T13:32:00.573427+0000","last_fullsized":"2024-02-02T13:32:00.573427+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:53:16.523235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.568345+0000","last_change":"2024-02-02T13:32:00.093624+0000","last_active":"2024-02-02T13:32:00.568345+0000","last_peered":"2024-02-02T13:32:00.568345+0000","last_clean":"2024-02-02T13:32:00.568345+0000","last_became_active":"2024-02-02T13:32:00.093387+0000","last_became_peered":"2024-02-02T13:32:00.093387+0000","last_unstale":"2024-02-02T13:32:00.568345+0000","last_undegraded":"2024-02-02T13:32:00.568345+0000","last_fullsized":"2024-02-02T13:32:00.568345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:43:38.592631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.582348+0000","last_change":"2024-02-02T13:32:00.581946+0000","last_active":"2024-02-02T13:32:00.582348+0000","last_peered":"2024-02-02T13:32:00.582348+0000","last_clean":"2024-02-02T13:32:00.582348+0000","last_became_active":"2024-02-02T13:32:00.580825+0000","last_became_peered":"2024-02-02T13:32:00.580825+0000","last_unstale":"2024-02-02T13:32:00.582348+0000","last_undegraded":"2024-02-02T13:32:00.582348+0000","last_fullsized":"2024-02-02T13:32:00.582348+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:58.765170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562368+0000","last_change":"2024-02-02T13:32:00.089294+0000","last_active":"2024-02-02T13:32:00.562368+0000","last_peered":"2024-02-02T13:32:00.562368+0000","last_clean":"2024-02-02T13:32:00.562368+0000","last_became_active":"2024-02-02T13:32:00.088991+0000","last_became_peered":"2024-02-02T13:32:00.088991+0000","last_unstale":"2024-02-02T13:32:00.562368+0000","last_undegraded":"2024-02-02T13:32:00.562368+0000","last_fullsized":"2024-02-02T13:32:00.562368+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:20:30.325522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.580131+0000","last_change":"2024-02-02T13:32:00.580047+0000","last_active":"2024-02-02T13:32:00.580131+0000","last_peered":"2024-02-02T13:32:00.580131+0000","last_clean":"2024-02-02T13:32:00.580131+0000","last_became_active":"2024-02-02T13:32:00.577888+0000","last_became_peered":"2024-02-02T13:32:00.577888+0000","last_unstale":"2024-02-02T13:32:00.580131+0000","last_undegraded":"2024-02-02T13:32:00.580131+0000","last_fullsized":"2024-02-02T13:32:00.580131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:19.870695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562663+0000","last_change":"2024-02-02T13:32:00.090802+0000","last_active":"2024-02-02T13:32:00.562663+0000","last_peered":"2024-02-02T13:32:00.562663+0000","last_clean":"2024-02-02T13:32:00.562663+0000","last_became_active":"2024-02-02T13:32:00.090403+0000","last_became_peered":"2024-02-02T13:32:00.090403+0000","last_unstale":"2024-02-02T13:32:00.562663+0000","last_undegraded":"2024-02-02T13:32:00.562663+0000","last_fullsized":"2024-02-02T13:32:00.562663+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:17.166864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.569506+0000","last_change":"2024-02-02T13:32:00.568995+0000","last_active":"2024-02-02T13:32:00.569506+0000","last_peered":"2024-02-02T13:32:00.569506+0000","last_clean":"2024-02-02T13:32:00.569506+0000","last_became_active":"2024-02-02T13:32:00.568789+0000","last_became_peered":"2024-02-02T13:32:00.568789+0000","last_unstale":"2024-02-02T13:32:00.569506+0000","last_undegraded":"2024-02-02T13:32:00.569506+0000","last_fullsized":"2024-02-02T13:32:00.569506+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:00.049803+0000","last_clean_scrub_stamp":"2024-02-02T13:32:00.049803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:36:10.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.562621+0000","last_change":"2024-02-02T13:32:00.094070+0000","last_active":"2024-02-02T13:32:00.562621+0000","last_peered":"2024-02-02T13:32:00.562621+0000","last_clean":"2024-02-02T13:32:00.562621+0000","last_became_active":"2024-02-02T13:32:00.092626+0000","last_became_peered":"2024-02-02T13:32:00.092626+0000","last_unstale":"2024-02-02T13:32:00.562621+0000","last_undegraded":"2024-02-02T13:32:00.562621+0000","last_fullsized":"2024-02-02T13:32:00.562621+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:31:59.035044+0000","last_clean_scrub_stamp":"2024-02-02T13:31:59.035044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:21:05.945643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-02-02T13:32:00.072240+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:32:00.072240+0000","last_peered":"2024-02-02T13:32:00.072240+0000","last_clean":"2024-02-02T13:32:00.072240+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:32:00.072240+0000","last_undegraded":"2024-02-02T13:32:00.072240+0000","last_fullsized":"2024-02-02T13:32:00.072240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193968,"statfs":{"total":200000000,"available":198623334,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.103,"15min":1.0700000000000001},"min":{"1min":0.71099999999999997,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.5310000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1060000000000001,"15min":1.121},"min":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.1609999999999996},"last":0.98399999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.73899999999999999,"5min":0.68999999999999995,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":1.8839999999999999,"15min":1.893},"last":1.284},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.73499999999999999,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.232},"min":{"1min":0.90800000000000003,"5min":0.81999999999999995,"15min":0.70099999999999996},"max":{"1min":1.675,"5min":1.909,"15min":2.6669999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.2070000000000001,"15min":1.1859999999999999},"min":{"1min":0.88400000000000001,"5min":0.80000000000000004,"15min":0.623},"max":{"1min":1.7949999999999999,"5min":2.0609999999999999,"15min":3.5819999999999999},"last":1.2310000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1819999999999999,"15min":1.1799999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3819999999999999,"5min":1.6339999999999999,"15min":1.819},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.345,"15min":1.3180000000000001},"min":{"1min":0.79600000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0259999999999998,"5min":2.133,"15min":2.4620000000000002},"last":1.53}]},{"osd":4,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3120000000000001,"15min":1.21},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.619,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3100000000000001},"min":{"1min":0.88,"5min":0.77600000000000002,"15min":0.61199999999999999},"max":{"1min":1.6180000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.3879999999999999,"15min":1.3580000000000001},"min":{"1min":1.0920000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.788,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.84199999999999997,"5min":0.76000000000000001,"15min":0.56200000000000006},"max":{"1min":1.921,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.5089999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:45 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.506,"15min":1.526},"min":{"1min":0.96999999999999997,"5min":0.88700000000000001,"15min":0.64200000000000002},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.4399999999999999,"15min":1.419},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":2.0179999999999998},"last":1.7809999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194545,"statfs":{"total":200000000,"available":199214651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.0309999999999999},"min":{"1min":0.64000000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.5840000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.028},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6060000000000001,"5min":1.768,"15min":1.768},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0900000000000001,"15min":1.028},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4790000000000001,"5min":1.556,"15min":1.556},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1719999999999999,"15min":1.091},"min":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.861,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.0109999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1899999999999999,"15min":1.0900000000000001},"min":{"1min":0.747,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6659999999999999,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.345},{"interface":"front","average":{"1min":1.196,"5min":1.1919999999999999,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8169999999999999,"5min":1.849,"15min":1.849},"last":1.157}]},{"osd":3,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2529999999999999,"15min":1.2589999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8149999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1990000000000001,"15min":1.147},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.643,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.103}]},{"osd":4,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.4530000000000001,"15min":1.276},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.8640000000000001,"5min":17.686,"15min":17.686},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5920000000000001,"15min":1.4530000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.1099999999999999,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.431}]},{"osd":5,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.3009999999999999,"15min":1.2769999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6399999999999999,"5min":1.986,"15min":1.986},"last":1.232},{"interface":"front","average":{"1min":1.423,"5min":1.4370000000000001,"15min":1.3660000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9119999999999999,"5min":3.331,"15min":3.331},"last":1.556}]},{"osd":7,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.494,"15min":1.4590000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.61199999999999999},"max":{"1min":2.0089999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.4670000000000001,"15min":1.4390000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.901,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.395}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":768,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194544,"statfs":{"total":200000000,"available":199213098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.028,"15min":1.022},"min":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.081,"15min":1.048},"min":{"1min":0.74099999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.554,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.153,"15min":1.2430000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.494,"5min":1.8220000000000001,"15min":4.5899999999999999},"last":1.085},{"interface":"front","average":{"1min":1.044,"5min":1.0740000000000001,"15min":1.133},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.877,"15min":4.4500000000000002},"last":1.4159999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.286,"15min":1.2310000000000001},"min":{"1min":0.78400000000000003,"5min":0.753,"15min":0.72499999999999998},"max":{"1min":1.6379999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.335},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1890000000000001,"15min":1.1759999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5129999999999999,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.4390000000000001,"5min":1.591,"15min":1.7769999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.29,"15min":1.1879999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.5349999999999999,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":1.4690000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1890000000000001,"15min":1.2190000000000001},"min":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.64000000000000001},"max":{"1min":1.615,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.012},{"interface":"front","average":{"1min":1.272,"5min":1.296,"15min":1.2829999999999999},"min":{"1min":0.92600000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5129999999999999,"5min":1.8029999999999999,"15min":1.833},"last":1.171}]},{"osd":6,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3300000000000001,"15min":1.3069999999999999},"min":{"1min":1.01,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7669999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.345,"15min":1.3460000000000001},"min":{"1min":0.82599999999999996,"5min":0.66900000000000004,"15min":0.59099999999999997},"max":{"1min":1.724,"5min":1.7889999999999999,"15min":1.8640000000000001},"last":1.375}]},{"osd":7,"last update":"Fri Feb 2 13:31:38 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.4179999999999999,"15min":1.4039999999999999},"min":{"1min":1.111,"5min":0.71899999999999997,"15min":0.66500000000000004},"max":{"1min":1.589,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.444,"15min":1.385},"min":{"1min":1.1299999999999999,"5min":0.69799999999999995,"15min":0.61599999999999999},"max":{"1min":1.71,"5min":1.851,"15min":1.851},"last":1.2849999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.091,"15min":0.998},"min":{"1min":0.67900000000000005,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.3959999999999999,"5min":4.5949999999999998,"15min":4.5949999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.079},"min":{"1min":0.71399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3999999999999999,"5min":1.839,"15min":1.839},"last":1.4630000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.0029999999999999},"min":{"1min":0.66000000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.403,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.1459999999999999,"15min":1.0489999999999999},"min":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5740000000000001,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":1.056}]},{"osd":2,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.177,"15min":1.099},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.478,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1830000000000001,"15min":1.0900000000000001},"min":{"1min":0.70099999999999996,"5min":0.68000000000000005,"15min":0.58799999999999997},"max":{"1min":1.5620000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.274,"15min":1.1899999999999999},"min":{"1min":0.871,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5349999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.331,"15min":1.252},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.2330000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.27,"15min":1.1619999999999999},"min":{"1min":0.88100000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.829,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3660000000000001,"15min":1.29},"min":{"1min":0.89900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.722,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.6140000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.403,"15min":1.3120000000000001},"min":{"1min":0.96899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.712,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.327},{"interface":"front","average":{"1min":1.454,"5min":1.48,"15min":1.401},"min":{"1min":1.0580000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7849999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.351}]},{"osd":7,"last update":"Fri Feb 2 13:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.49,"15min":1.3979999999999999},"min":{"1min":1.155,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.671},{"interface":"front","average":{"1min":1.401,"5min":1.498,"15min":1.452},"min":{"1min":1.0089999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.869,"5min":2.698,"15min":2.698},"last":1.5509999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":769,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194543,"statfs":{"total":200000000,"available":199212068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78400000000000003,"15min":0.71599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.88700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":1.014}]},{"osd":1,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96099999999999997,"15min":0.90700000000000003},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.097,"5min":1.077,"15min":1.0229999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1299999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":0.96799999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4370000000000001,"5min":1.611,"15min":1.611},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92200000000000004,"15min":0.89600000000000002},"min":{"1min":0.443,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.9159999999999999,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.181},"min":{"1min":0.71099999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.99,"5min":1.99,"15min":3.585},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2649999999999999,"15min":1.2110000000000001},"min":{"1min":0.80000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.877},"last":1.3600000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1819999999999999,"15min":1.21},"min":{"1min":0.72099999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8129999999999999,"5min":3.738,"15min":3.738},"last":1.399},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.28,"15min":1.2589999999999999},"min":{"1min":0.83499999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":1.1759999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3640000000000001,"15min":1.2749999999999999},"min":{"1min":0.91500000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.895,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.2569999999999999,"15min":1.2889999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":2.0310000000000001,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.222}]},{"osd":7,"last update":"Fri Feb 2 13:31:44 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.276,"15min":1.26},"min":{"1min":0.75,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":1.869,"15min":3.391},"last":1.079},{"interface":"front","average":{"1min":1.351,"5min":1.3779999999999999,"15min":1.321},"min":{"1min":0.877,"5min":0.73999999999999999,"15min":0.625},"max":{"1min":1.946,"5min":1.946,"15min":2.0409999999999999},"last":1.27}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":767,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194544,"statfs":{"total":200000000,"available":199213702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":0.78700000000000003},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":1.401,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.91600000000000004,"15min":0.80200000000000005},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.2410000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.002}]},{"osd":1,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97899999999999998,"15min":0.91100000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.218,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0169999999999999,"15min":0.88100000000000001},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.35899999999999999},"max":{"1min":1.4990000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1060000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.93000000000000005,"15min":0.81200000000000006},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.145,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.091,"15min":0.995},"min":{"1min":0.435,"5min":0.435,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.84299999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.288,"15min":1.1930000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.298,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2470000000000001,"15min":1.1699999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4710000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.226,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3129999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.331},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.343,"15min":1.2789999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.635,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.89400000000000002}]},{"osd":6,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.3220000000000001,"15min":1.198},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.571,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.361},{"interface":"front","average":{"1min":1.137,"5min":1.29,"15min":1.21},"min":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":1.3799999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.446}]},{"osd":7,"last update":"Fri Feb 2 13:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.379,"15min":1.232},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6000000000000001,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.3400000000000001,"15min":1.2330000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4690000000000001,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.3939999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194545,"statfs":{"total":200000000,"available":199214597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.425,"5min":1.425,"15min":3.5680000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.93000000000000005,"15min":1.2869999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4870000000000001,"5min":1.5860000000000001,"15min":5.8099999999999996},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.875,"15min":0.90300000000000002},"min":{"1min":0.52500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3700000000000001,"5min":1.431,"15min":1.639},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0049999999999999,"15min":1.077},"min":{"1min":0.50800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6890000000000001,"5min":1.823,"15min":3.0259999999999998},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.097,"15min":1.228},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.002,"5min":1.0189999999999999,"15min":1.234},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.49299999999999999},"max":{"1min":1.774,"5min":1.774,"15min":4.8760000000000003},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.2889999999999999},"min":{"1min":0.85499999999999998,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.837,"15min":2.9449999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.2829999999999999,"15min":1.351},"min":{"1min":0.83799999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":1.032}]},{"osd":5,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2869999999999999,"15min":1.4279999999999999},"min":{"1min":0.88900000000000001,"5min":0.76100000000000001,"15min":0.753},"max":{"1min":1.5569999999999999,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2789999999999999,"15min":1.4099999999999999},"min":{"1min":0.91400000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.657,"5min":1.7090000000000001,"15min":3.0249999999999999},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.306,"15min":1.526},"min":{"1min":0.97399999999999998,"5min":0.81899999999999995,"15min":0.73599999999999999},"max":{"1min":1.966,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":1.417},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3260000000000001,"15min":1.4630000000000001},"min":{"1min":0.96399999999999997,"5min":0.92100000000000004,"15min":0.81000000000000005},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:31:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.375,"15min":1.524},"min":{"1min":0.82099999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.177},{"interface":"front","average":{"1min":1.373,"5min":1.3819999999999999,"15min":1.577},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.78000000000000003},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.286}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1345,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193967,"statfs":{"total":200000000,"available":198622677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":1.069},"min":{"1min":0.35599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2629999999999999,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.90800000000000003,"15min":1.3859999999999999},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3879999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.93500000000000005,"15min":1.125},"min":{"1min":0.48399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4319999999999999,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.93100000000000005,"15min":1.387},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6240000000000001,"5min":1.706,"15min":10.781000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":1.1739999999999999},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":5.8410000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0369999999999999,"15min":1.2649999999999999},"min":{"1min":0.51600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6459999999999999,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.3140000000000001},"min":{"1min":0.73399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.55,"5min":1.9670000000000001,"15min":3.669},"last":1.45},{"interface":"front","average":{"1min":1.256,"5min":1.2689999999999999,"15min":1.383},"min":{"1min":0.77300000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.6599999999999999,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":1.169}]},{"osd":5,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2589999999999999,"15min":1.73},"min":{"1min":0.89100000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7150000000000001,"5min":2.0150000000000001,"15min":10.651},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.236,"15min":1.395},"min":{"1min":0.69799999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.153}]},{"osd":6,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3029999999999999,"15min":1.446},"min":{"1min":0.87,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.764,"5min":2.319,"15min":4.9210000000000003},"last":1.262},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2709999999999999,"15min":1.7609999999999999},"min":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.75,"5min":2.0510000000000002,"15min":10.708},"last":1.29}]},{"osd":7,"last update":"Fri Feb 2 13:31:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.337,"15min":1.7849999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":1.7390000000000001,"5min":2.3010000000000002,"15min":10.74},"last":1.216},{"interface":"front","average":{"1min":1.296,"5min":1.2949999999999999,"15min":1.423},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":3.863},"last":1.044}]}]}],"pool_statfs":[]}} 2024-02-02T13:32:05.449 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:32:05.449 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:32:05.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.703+0000 7f5dfee00640 1 Processor -- start 2024-02-02T13:32:05.705 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.703+0000 7f5dfee00640 1 -- start start 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfee00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5df8151f90 0x7f5df815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 0x7f5df80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfee00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df8070f80 con 0x7f5df80b0c40 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfee00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df80b10d0 con 0x7f5df8151f90 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8152370 con 0x7f5df806f4f0 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5df7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5df8151f90 0x7f5df815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5df80b0c40 0x7f5df80b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45480/0 (socket says 172.21.15.91:45480) 2024-02-02T13:32:05.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.705+0000 7f5dfce00640 1 -- 172.21.15.91:0/1217932946 learned_addr learned my addr 172.21.15.91:0/1217932946 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7400640 1 -- 172.21.15.91:0/1217932946 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072824118 0 0) 0x7f5df8070f80 con 0x7f5df80b0c40 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5def200640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 0x7f5df80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7400640 1 -- 172.21.15.91:0/1217932946 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd8003520 con 0x7f5df80b0c40 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7e00640 1 -- 172.21.15.91:0/1217932946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 msgr2=0x7f5df80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7e00640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 0x7f5df80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7e00640 1 -- 172.21.15.91:0/1217932946 >> v1:172.21.15.91:6789/0 conn(0x7f5df80b0c40 legacy=0x7f5df80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5df7e00640 1 -- 172.21.15.91:0/1217932946 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80cb010 con 0x7f5df8151f90 2024-02-02T13:32:05.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.706+0000 7f5def200640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 0x7f5df80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:05.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.707+0000 7f5df7e00640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df8151f90 0x7f5df815f5a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5bed1c9cea7b41 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.707+0000 7f5df7400640 1 -- 172.21.15.91:0/1217932946 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de802d040 con 0x7f5df8151f90 2024-02-02T13:32:05.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.707+0000 7f5df7400640 1 -- 172.21.15.91:0/1217932946 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8031810 con 0x7f5df8151f90 2024-02-02T13:32:05.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.707+0000 7f5df7400640 1 -- 172.21.15.91:0/1217932946 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8031ba0 con 0x7f5df8151f90 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 -- 172.21.15.91:0/1217932946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df8151f90 msgr2=0x7f5df815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df8151f90 0x7f5df815f5a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b8b0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 -- 172.21.15.91:0/1217932946 shutdown_connections 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df806f4f0 0x7f5df80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 --2- 172.21.15.91:0/1217932946 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df8151f90 0x7f5df815f5a0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 -- 172.21.15.91:0/1217932946 >> 172.21.15.91:0/1217932946 conn(0x7f5df80cec80 msgr2=0x7f5df80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 -- 172.21.15.91:0/1217932946 shutdown_connections 2024-02-02T13:32:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.708+0000 7f5dfee00640 1 -- 172.21.15.91:0/1217932946 wait complete. 2024-02-02T13:32:05.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.710+0000 7f5dfee00640 1 Processor -- start 2024-02-02T13:32:05.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.710+0000 7f5dfee00640 1 -- start start 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 0x7f5df8156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df8151f90 0x7f5df8156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80cceb0 con 0x7f5df806f4f0 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80acd70 con 0x7f5df80b0c40 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5dfee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80ce7c0 con 0x7f5df8151f90 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58508/0 (socket says 172.21.15.91:58508) 2024-02-02T13:32:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.711+0000 7f5df7e00640 1 -- 172.21.15.91:0/2831823738 learned_addr learned my addr 172.21.15.91:0/2831823738 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 -- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df8151f90 msgr2=0x7f5df8156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5dfce00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df8151f90 0x7f5df8156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df8151f90 0x7f5df8156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 -- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 msgr2=0x7f5df8156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5def200640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 0x7f5df8156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 0x7f5df8156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 -- 172.21.15.91:0/2831823738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8009400 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5def200640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 0x7f5df8156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df7e00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5de80098c0 tx=0x7f5de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.712+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8006e10 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.713+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de803ad60 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.713+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80ce200 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.713+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8044620 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.713+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80aca00 con 0x7f5df806f4f0 2024-02-02T13:32:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.714+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df80acb90 con 0x7f5df806f4f0 2024-02-02T13:32:05.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.715+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5de804d440 con 0x7f5df806f4f0 2024-02-02T13:32:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.716+0000 7f5df4c00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 0x7f5dd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.716+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de80aabf0 con 0x7f5df806f4f0 2024-02-02T13:32:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.716+0000 7f5def200640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 0x7f5dd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.717+0000 7f5def200640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 0x7f5dd006c8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006fd0 tx=0x7f5de0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:05.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.721+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5de8076270 con 0x7f5df806f4f0 2024-02-02T13:32:05.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.968+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5df80acb90 con 0x7f5df806f4f0 2024-02-02T13:32:05.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.977+0000 7f5df4c00640 1 -- 172.21.15.91:0/2831823738 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5de80350e0 con 0x7f5df806f4f0 2024-02-02T13:32:05.978 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:32:05.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.979+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 msgr2=0x7f5dd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.979+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 0x7f5dd006c8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006fd0 tx=0x7f5de0008040 comp rx=0 tx=0).stop 2024-02-02T13:32:05.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.979+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 msgr2=0x7f5df80cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.979+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5de80098c0 tx=0x7f5de80046c0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.979+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 shutdown_connections 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5dd006a430 0x7f5dd006c8f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5df8151f90 0x7f5df8156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5df80b0c40 0x7f5df8156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 --2- 172.21.15.91:0/2831823738 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5df806f4f0 0x7f5df80cc970 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 >> 172.21.15.91:0/2831823738 conn(0x7f5df80cec80 msgr2=0x7f5df8159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 shutdown_connections 2024-02-02T13:32:05.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:05.980+0000 7f5dfee00640 1 -- 172.21.15.91:0/2831823738 wait complete. 2024-02-02T13:32:05.999 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:06.045 INFO:tasks.ceph.mds.a.smithi091.stderr:2024-02-02T13:32:06.043+0000 7fcfbb000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:32:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.201+0000 7ff335400640 1 Processor -- start 2024-02-02T13:32:06.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.201+0000 7ff335400640 1 -- start start 2024-02-02T13:32:06.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 0x7ff3300c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3300ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff3300cd470 0x7ff3300b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300cc7c0 con 0x7ff3300cd470 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300ce930 con 0x7ff33015a010 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff335400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33006c890 con 0x7ff3300ce130 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.202+0000 7ff32f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff3300cd470 0x7ff3300b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff3300cd470 0x7ff3300b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58524/0 (socket says 172.21.15.91:58524) 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 -- 172.21.15.91:0/3935964397 learned_addr learned my addr 172.21.15.91:0/3935964397 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff325c00640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3300ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32e800640 1 --2- 172.21.15.91:0/3935964397 >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 0x7ff3300c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 -- 172.21.15.91:0/3935964397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 msgr2=0x7ff3300ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3300ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 -- 172.21.15.91:0/3935964397 >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 msgr2=0x7ff3300c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 --2- 172.21.15.91:0/3935964397 >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 0x7ff3300c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32f200640 1 -- 172.21.15.91:0/3935964397 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33006cb70 con 0x7ff3300cd470 2024-02-02T13:32:06.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff325c00640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3300ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.203+0000 7ff32e800640 1 --2- 172.21.15.91:0/3935964397 >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 0x7ff3300c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.204+0000 7ff32f200640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3300cd470 0x7ff3300b0700 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff3300cfaa0 tx=0x7ff32802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d023f63894548b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.204+0000 7ff32de00640 1 -- 172.21.15.91:0/3935964397 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32802ca70 con 0x7ff3300cd470 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.204+0000 7ff32de00640 1 -- 172.21.15.91:0/3935964397 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32802cbd0 con 0x7ff3300cd470 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.204+0000 7ff32de00640 1 -- 172.21.15.91:0/3935964397 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280343f0 con 0x7ff3300cd470 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 -- 172.21.15.91:0/3935964397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3300cd470 msgr2=0x7ff3300b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3300cd470 0x7ff3300b0700 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff3300cfaa0 tx=0x7ff32802bed0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 -- 172.21.15.91:0/3935964397 shutdown_connections 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3300cd470 0x7ff3300b0700 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 --2- 172.21.15.91:0/3935964397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3300ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 --2- 172.21.15.91:0/3935964397 >> v2:172.21.15.113:3300/0 conn(0x7ff33015a010 0x7ff3300c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 -- 172.21.15.91:0/3935964397 >> 172.21.15.91:0/3935964397 conn(0x7ff3300cec80 msgr2=0x7ff3300cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:06.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 -- 172.21.15.91:0/3935964397 shutdown_connections 2024-02-02T13:32:06.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.205+0000 7ff335400640 1 -- 172.21.15.91:0/3935964397 wait complete. 2024-02-02T13:32:06.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.207+0000 7ff335400640 1 Processor -- start 2024-02-02T13:32:06.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.207+0000 7ff335400640 1 -- start start 2024-02-02T13:32:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3301f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 0x7ff33015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301f3a60 con 0x7ff3301f5fe0 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301f3090 con 0x7ff33015a010 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff335400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301f6aa0 con 0x7ff3300ce130 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58540/0 (socket says 172.21.15.91:58540) 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 -- 172.21.15.91:0/116237752 learned_addr learned my addr 172.21.15.91:0/116237752 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32e800640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3301f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff325c00640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 0x7ff33015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 -- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 msgr2=0x7ff3301f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3301f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 -- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 msgr2=0x7ff33015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 0x7ff33015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff32f200640 1 -- 172.21.15.91:0/116237752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328009b30 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.208+0000 7ff325c00640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 0x7ff33015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff32f200640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff328036040 tx=0x7ff328004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328009510 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32803e450 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3301f6d80 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32803e7e0 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.209+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3300c5f50 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.210+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330066cf0 con 0x7ff3301f5fe0 2024-02-02T13:32:06.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.211+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff32803e940 con 0x7ff3301f5fe0 2024-02-02T13:32:06.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.212+0000 7ff327400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 0x7ff30806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:06.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.212+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3280aba70 con 0x7ff3301f5fe0 2024-02-02T13:32:06.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.212+0000 7ff32e800640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 0x7ff30806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:06.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.213+0000 7ff32e800640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 0x7ff30806c8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff32000a8d0 tx=0x7ff320008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.217+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff3280770e0 con 0x7ff3301f5fe0 2024-02-02T13:32:06.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.442+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3301f31d0 con 0x7ff3301f5fe0 2024-02-02T13:32:06.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.444+0000 7ff327400640 1 -- 172.21.15.91:0/116237752 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7ff328033e80 con 0x7ff3301f5fe0 2024-02-02T13:32:06.445 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:06.445 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:06.446 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:06.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 msgr2=0x7ff30806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 0x7ff30806c8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff32000a8d0 tx=0x7ff320008040 comp rx=0 tx=0).stop 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 msgr2=0x7ff3301f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff328036040 tx=0x7ff328004a40 comp rx=0 tx=0).stop 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 shutdown_connections 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30806a430 0x7ff30806c8f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3301f5fe0 0x7ff3301f6470 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33015a010 0x7ff33015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 --2- 172.21.15.91:0/116237752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3300ce130 0x7ff3301f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:06.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.446+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 >> 172.21.15.91:0/116237752 conn(0x7ff3300cec80 msgr2=0x7ff330071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:06.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.447+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 shutdown_connections 2024-02-02T13:32:06.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:06.447+0000 7ff335400640 1 -- 172.21.15.91:0/116237752 wait complete. 2024-02-02T13:32:06.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:32:00.544990+0000', 'modified': '2024-02-02T13:32:03.563018+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7584}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7584': {'gid': 7584, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/1090230947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6834', 'nonce': 1090230947}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 1090230947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:32:06.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:32:06.871 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-02T13:32:07.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.086+0000 7fd73fc00640 1 Processor -- start 2024-02-02T13:32:07.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.086+0000 7fd73fc00640 1 -- start start 2024-02-02T13:32:07.088 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.087+0000 7fd73fc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd73815ad10 0x7fd73806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.087+0000 7fd73fc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73806d910 con 0x7fd7380cb580 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.087+0000 7fd73fc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380cc850 con 0x7fd73815b0f0 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.087+0000 7fd73fc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73806d730 con 0x7fd73815ad10 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd73dc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd7380cb580 0x7fd7380c26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45486/0 (socket says 172.21.15.91:45486) 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd73dc00640 1 -- 172.21.15.91:0/2861595 learned_addr learned my addr 172.21.15.91:0/2861595 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd73d200640 1 --2- 172.21.15.91:0/2861595 >> v2:172.21.15.113:3300/0 conn(0x7fd73815ad10 0x7fd73806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764478631 0 0) 0x7fd73806d910 con 0x7fd7380cb580 2024-02-02T13:32:07.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd718003520 con 0x7fd7380cb580 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965540199 0 0) 0x7fd718003520 con 0x7fd7380cb580 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 >> v2:172.21.15.113:3300/0 conn(0x7fd73815ad10 msgr2=0x7fd73806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 --2- 172.21.15.91:0/2861595 >> v2:172.21.15.113:3300/0 conn(0x7fd73815ad10 0x7fd73806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 >> v1:172.21.15.113:6789/0 conn(0x7fd73815b0f0 legacy=0x7fd73806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380ccb30 con 0x7fd7380cb580 2024-02-02T13:32:07.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd7300019f0 con 0x7fd7380cb580 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 >> v1:172.21.15.91:6789/0 conn(0x7fd7380cb580 legacy=0x7fd7380c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd718003970 0x7fd718003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.088+0000 7fd737e00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd718004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd737e00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7180052a0 0x7fd71800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73806d730 con 0x7fd7180047c0 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71800c170 con 0x7fd718003970 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71800c330 con 0x7fd7180052a0 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd73d200640 1 --2- 172.21.15.91:0/2861595 >> v2:172.21.15.113:3300/0 conn(0x7fd73815ad10 0x7fd73806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd718004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 -- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7180052a0 msgr2=0x7fd71800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7180052a0 0x7fd71800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 -- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd718003970 msgr2=0x7fd718003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd73d200640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd718003970 0x7fd718003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd718003970 0x7fd718003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd72fc00640 1 -- 172.21.15.91:0/2861595 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718003520 con 0x7fd7180047c0 2024-02-02T13:32:07.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.089+0000 7fd73dc00640 1 -- 172.21.15.91:0/2861595 reap_dead start 2024-02-02T13:32:07.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.090+0000 7fd72fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd718004c40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd720001720 tx=0x7fd7180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0de7d1d9a59995a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200052d0 con 0x7fd7180047c0 2024-02-02T13:32:07.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72002e050 con 0x7fd7180047c0 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd737e00640 1 -- 172.21.15.91:0/2861595 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720003830 con 0x7fd7180047c0 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 -- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 msgr2=0x7fd718004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd718004c40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd720001720 tx=0x7fd7180057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 -- 172.21.15.91:0/2861595 shutdown_connections 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 --2- 172.21.15.91:0/2861595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd718004c40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 -- 172.21.15.91:0/2861595 >> 172.21.15.91:0/2861595 conn(0x7fd7380b0700 msgr2=0x7fd7380b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 -- 172.21.15.91:0/2861595 shutdown_connections 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.091+0000 7fd73fc00640 1 -- 172.21.15.91:0/2861595 wait complete. 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.093+0000 7fd73fc00640 1 Processor -- start 2024-02-02T13:32:07.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.093+0000 7fd73fc00640 1 -- start start 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7380c8930 0x7fd7380ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7380c8db0 0x7fd738150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380af290 con 0x7fd7180047c0 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380c5e70 con 0x7fd7380c8db0 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73fc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380c6150 con 0x7fd7380c8930 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58558/0 (socket says 172.21.15.91:58558) 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.094+0000 7fd73d200640 1 -- 172.21.15.91:0/1409525323 learned_addr learned my addr 172.21.15.91:0/1409525323 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 -- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7380c8930 msgr2=0x7fd7380ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7380c8930 0x7fd7380ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 -- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7380c8db0 msgr2=0x7fd738150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7380c8db0 0x7fd738150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 -- 172.21.15.91:0/1409525323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380c6430 con 0x7fd7180047c0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73d200640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd72800f570 tx=0x7fd72800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd735600640 1 -- 172.21.15.91:0/1409525323 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728014890 con 0x7fd7180047c0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd735600640 1 -- 172.21.15.91:0/1409525323 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72800c030 con 0x7fd7180047c0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd735600640 1 -- 172.21.15.91:0/1409525323 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728014c30 con 0x7fd7180047c0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.095+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7380c6710 con 0x7fd7180047c0 2024-02-02T13:32:07.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.096+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380aed80 con 0x7fd7180047c0 2024-02-02T13:32:07.099 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.097+0000 7fd735600640 1 -- 172.21.15.91:0/1409525323 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7280133e0 con 0x7fd7180047c0 2024-02-02T13:32:07.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.098+0000 7fd735600640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 0x7fd71006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.098+0000 7fd735600640 1 -- 172.21.15.91:0/1409525323 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd728088d30 con 0x7fd7180047c0 2024-02-02T13:32:07.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.099+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 0x7fd71006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.099+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 0x7fd71006c920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd720033000 tx=0x7fd72002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.099+0000 7fd74279d740 1 Processor -- start 2024-02-02T13:32:07.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.099+0000 7fd74279d740 1 -- start start 2024-02-02T13:32:07.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162ee5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162ee2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162e90650 con 0x556162eb4ed0 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162e25000 con 0x556162eb8ad0 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.100+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x556162e22790 con 0x556162ee4f50 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58574/0 (socket says 172.21.15.91:58574) 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 -- 172.21.15.91:0/1402425813 learned_addr learned my addr 172.21.15.91:0/1402425813 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd73d200640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162ee5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd73dc00640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162ee2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 -- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 msgr2=0x556162ee5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162ee5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 -- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 msgr2=0x556162ee2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162ee2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 -- 172.21.15.91:0/1402425813 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556162de5710 con 0x556162eb4ed0 2024-02-02T13:32:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd73d200640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162ee5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.101+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd720005040 tx=0x7fd720030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b2369d0177e27 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd72d400640 1 -- 172.21.15.91:0/1402425813 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720046070 con 0x556162eb4ed0 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd72d400640 1 -- 172.21.15.91:0/1402425813 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720007510 con 0x556162eb4ed0 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd72d400640 1 -- 172.21.15.91:0/1402425813 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720007680 con 0x556162eb4ed0 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 -- 172.21.15.91:0/1402425813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 msgr2=0x556162ee21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd720005040 tx=0x7fd720030000 comp rx=0 tx=0).stop 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 -- 172.21.15.91:0/1402425813 shutdown_connections 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162ee2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162eb4ed0 0x556162ee21c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 --2- 172.21.15.91:0/1402425813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162ee5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.102+0000 7fd74279d740 1 -- 172.21.15.91:0/1402425813 >> 172.21.15.91:0/1402425813 conn(0x556162eb71e0 msgr2=0x556162effd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.103+0000 7fd74279d740 1 -- 172.21.15.91:0/1402425813 shutdown_connections 2024-02-02T13:32:07.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.103+0000 7fd74279d740 1 -- 172.21.15.91:0/1402425813 wait complete. 2024-02-02T13:32:07.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.103+0000 7fd74279d740 1 -- start start 2024-02-02T13:32:07.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.103+0000 7fd74279d740 1 Processor -- start 2024-02-02T13:32:07.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f0ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ed4770 0x556162ee9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162e1e080 con 0x556162ee4f50 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162dbf850 con 0x556162eb8ad0 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.104+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x556162dbf4f0 con 0x556162ed4770 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd74279d740 10 client.0 _ll_register_callbacks cb 0x556162e179c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58580/0 (socket says 172.21.15.91:58580) 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 -- 172.21.15.91:0/1043781250 learned_addr learned my addr 172.21.15.91:0/1043781250 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd73d200640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f0ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 -- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ed4770 msgr2=0x556162ee9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ed4770 0x556162ee9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 -- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 msgr2=0x556162f0ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f0ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 -- 172.21.15.91:0/1043781250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd718003520 con 0x556162ee4f50 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd72004c040 tx=0x7fd720001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce14b304299fd35 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.105+0000 7fd726000640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720046040 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd720007680 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556162e485d0 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd720003800 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 10 client.7722 fetch_fsmap learned FSMap version 64 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x556162e1c700 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556162e485d0 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd720003960 con 0x556162ee4f50 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 10 client.7722 fetch_fsmap finished waiting for FSMap version 64 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 10 client.7722 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 20 client.7722 populate_metadata read hostname 'smithi091' 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ebad20 0x556162ee6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 4 client.7722 mds_command: new command op to 7584 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.106+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 --> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556162e108d0 con 0x556162ebad20 2024-02-02T13:32:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.107+0000 7fd73d200640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ebad20 0x556162ee6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.107+0000 7fd73d200640 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ebad20 0x556162ee6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c68c7c794bbf4f server_cookie=2238a9a1632f11f0 in_seq=0 out_seq=0 2024-02-02T13:32:07.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.107+0000 7fd726000640 10 client.7722 ms_handle_connect on v2:172.21.15.91:6834/1090230947 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd726000640 1 -- 172.21.15.91:0/1043781250 <== mds.0 v2:172.21.15.91:6834/1090230947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556162e108d0 con 0x556162ebad20 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd726000640 10 client.7722 handle_command_reply: tid=0 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 client.7722 shutdown 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 msgr2=0x556162f12a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd72004c040 tx=0x7fd720001190 comp rx=0 tx=0).stop 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 shutdown_connections 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ebad20 0x556162ee6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ed4770 0x556162ee9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162f12a30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd726000640 0 client.7722 ms_handle_reset on v2:172.21.15.91:6834/1090230947 2024-02-02T13:32:07.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 --2- 172.21.15.91:0/1043781250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f0ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 >> 172.21.15.91:0/1043781250 conn(0x556162eb4ed0 msgr2=0x556162f03270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 shutdown_connections 2024-02-02T13:32:07.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 1 -- 172.21.15.91:0/1043781250 wait complete. 2024-02-02T13:32:07.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.108+0000 7fd74279d740 20 client.7722 trim_cache size 0 max 16384 2024-02-02T13:32:07.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.124+0000 7fd74279d740 1 Processor -- start 2024-02-02T13:32:07.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.124+0000 7fd74279d740 1 -- start start 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162eb8ad0 0x556162f5c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 0x556162eb9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162b85f20 con 0x556162ee4f50 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162e1d220 con 0x556162ed4770 2024-02-02T13:32:07.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x556162eb69e0 con 0x556162eb8ad0 2024-02-02T13:32:07.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd73dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd73dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58582/0 (socket says 172.21.15.91:58582) 2024-02-02T13:32:07.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd73dc00640 1 -- 172.21.15.91:0/1995780931 learned_addr learned my addr 172.21.15.91:0/1995780931 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.125+0000 7fd73d200640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162eb8ad0 0x556162f5c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 0x556162eb9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 -- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162eb8ad0 msgr2=0x556162f5c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162eb8ad0 0x556162f5c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 -- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 msgr2=0x556162eb9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 0x556162eb9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 -- 172.21.15.91:0/1995780931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556162e52280 con 0x556162ee4f50 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd72fc00640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 0x556162eb9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd73dc00640 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd7380b1630 tx=0x7fd730007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b975c320deeec77 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd72de00640 1 -- 172.21.15.91:0/1995780931 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000d830 con 0x556162ee4f50 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd72de00640 1 -- 172.21.15.91:0/1995780931 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73000d990 con 0x556162ee4f50 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.126+0000 7fd72de00640 1 -- 172.21.15.91:0/1995780931 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000dd40 con 0x556162ee4f50 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- 172.21.15.91:0/1995780931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 msgr2=0x556162eba250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd7380b1630 tx=0x7fd730007f70 comp rx=0 tx=0).stop 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- 172.21.15.91:0/1995780931 shutdown_connections 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ee4f50 0x556162eba250 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162ed4770 0x556162eb9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 --2- 172.21.15.91:0/1995780931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162eb8ad0 0x556162f5c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- 172.21.15.91:0/1995780931 >> 172.21.15.91:0/1995780931 conn(0x556162eb4ed0 msgr2=0x556162edeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- 172.21.15.91:0/1995780931 shutdown_connections 2024-02-02T13:32:07.130 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- 172.21.15.91:0/1995780931 wait complete. 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.127+0000 7fd74279d740 1 -- start start 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.128+0000 7fd74279d740 1 Processor -- start 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f58510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162f5f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162f65da0 con 0x556162ed4770 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162eb0c10 con 0x556162eb8ad0 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x556162f0d870 con 0x556162ee4f50 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd74279d740 10 client.0 _ll_register_callbacks cb 0x7fd718004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58594/0 (socket says 172.21.15.91:58594) 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 -- 172.21.15.91:0/235624318 learned_addr learned my addr 172.21.15.91:0/235624318 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 -- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 msgr2=0x556162f5f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162f5f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 -- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 msgr2=0x556162f58510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd73d200640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f58510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f58510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.129+0000 7fd72fc00640 1 -- 172.21.15.91:0/235624318 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380ccb30 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd72fc00640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd720005360 tx=0x7fd720009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614757cc208e70ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720046070 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd720007680 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd720007510 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd720006d30 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 10 client.7728 fetch_fsmap learned FSMap version 64 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x556162dbea60 con 0x556162ed4770 2024-02-02T13:32:07.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720007510 con 0x556162ed4770 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd72003f690 con 0x556162ed4770 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 10 client.7728 fetch_fsmap finished waiting for FSMap version 64 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 10 client.7728 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.130+0000 7fd74279d740 20 client.7728 populate_metadata read hostname 'smithi091' 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ed2cf0 0x556162eb9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd74279d740 4 client.7728 mds_command: new command op to 7584 tid=0[{"prefix": "status"}] 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 --> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556162e17100 con 0x556162ed2cf0 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd73d200640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ed2cf0 0x556162eb9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd73d200640 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ed2cf0 0x556162eb9d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31512663c81afacb server_cookie=1968a3a863fee3ec in_seq=0 out_seq=0 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd726000640 10 client.7728 ms_handle_connect on v2:172.21.15.91:6834/1090230947 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd726000640 1 -- 172.21.15.91:0/235624318 <== mds.0 v2:172.21.15.91:6834/1090230947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556162e17100 con 0x556162ed2cf0 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd726000640 10 client.7728 handle_command_reply: tid=0 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.131+0000 7fd74279d740 1 client.7728 shutdown 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 msgr2=0x556162f0d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd720005360 tx=0x7fd720009030 comp rx=0 tx=0).stop 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 shutdown_connections 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:6834/1090230947,v1:172.21.15.91:6837/1090230947] conn(0x556162ed2cf0 0x556162eb9d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556162ee4f50 0x556162f5f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556162ed4770 0x556162f0d330 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 --2- 172.21.15.91:0/235624318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556162eb8ad0 0x556162f58510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd726000640 0 client.7728 ms_handle_reset on v2:172.21.15.91:6834/1090230947 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 >> 172.21.15.91:0/235624318 conn(0x556162eb4ed0 msgr2=0x556162edf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 shutdown_connections 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 1 -- 172.21.15.91:0/235624318 wait complete. 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd74279d740 20 client.7728 trim_cache size 0 max 16384 2024-02-02T13:32:07.137 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 7584, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.5821342569999999, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 64, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 143, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 143, 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 11.920054189 2024-02-02T13:32:07.138 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 msgr2=0x7fd71006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.132+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 0x7fd71006c920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd720033000 tx=0x7fd72002f000 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 msgr2=0x7fd7381f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd72800f570 tx=0x7fd72800fa30 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 shutdown_connections 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd71006a460 0x7fd71006c920 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7380c8930 0x7fd7380ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7380c8db0 0x7fd738150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 --2- 172.21.15.91:0/1409525323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7180047c0 0x7fd7381f2350 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 >> 172.21.15.91:0/1409525323 conn(0x7fd7380b0700 msgr2=0x7fd7380c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 shutdown_connections 2024-02-02T13:32:07.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.133+0000 7fd73fc00640 1 -- 172.21.15.91:0/1409525323 wait complete. 2024-02-02T13:32:07.150 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:32:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.345+0000 7fc85fa00640 1 Processor -- start 2024-02-02T13:32:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.345+0000 7fc85fa00640 1 -- start start 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.346+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 0x7fc8580c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85fa00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580cc7c0 con 0x7fc8580b0700 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85fa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580ce930 con 0x7fc8580ce130 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580cd640 con 0x7fc85815a010 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85da00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc8580b0700 0x7fc85806ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45498/0 (socket says 172.21.15.91:45498) 2024-02-02T13:32:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85da00640 1 -- 172.21.15.91:0/2825321396 learned_addr learned my addr 172.21.15.91:0/2825321396 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964817594 0 0) 0x7fc8580cc7c0 con 0x7fc8580b0700 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc838003520 con 0x7fc8580b0700 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.347+0000 7fc85d000640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 0x7fc8580c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368670529 0 0) 0x7fc838003520 con 0x7fc8580b0700 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 msgr2=0x7fc8580c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 0x7fc8580c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 >> v1:172.21.15.113:6789/0 conn(0x7fc8580ce130 legacy=0x7fc8580ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85806c890 con 0x7fc8580b0700 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc8500019f0 con 0x7fc8580b0700 2024-02-02T13:32:07.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 >> v1:172.21.15.91:6789/0 conn(0x7fc8580b0700 legacy=0x7fc85806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc838003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8380047c0 0x7fc838004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 0x7fc838005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580cd640 con 0x7fc8380039a0 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83800ba30 con 0x7fc8380047c0 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83800de00 con 0x7fc85815a010 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc838003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 -- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 msgr2=0x7fc838005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815a010 0x7fc838005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 -- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8380047c0 msgr2=0x7fc838004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8380047c0 0x7fc838004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.348+0000 7fc85d000640 1 -- 172.21.15.91:0/2825321396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc838003520 con 0x7fc8380039a0 2024-02-02T13:32:07.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.349+0000 7fc85d000640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc838003dc0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc840009a90 tx=0x7fc8380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6710ccd763aeed1c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84002c6b0 con 0x7fc8380039a0 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84002c810 con 0x7fc8380039a0 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc857400640 1 -- 172.21.15.91:0/2825321396 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84002cba0 con 0x7fc8380039a0 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 -- 172.21.15.91:0/2825321396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 msgr2=0x7fc838003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 --2- 172.21.15.91:0/2825321396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc838003dc0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc840009a90 tx=0x7fc8380057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85da00640 1 -- 172.21.15.91:0/2825321396 reap_dead start 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 -- 172.21.15.91:0/2825321396 shutdown_connections 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 -- 172.21.15.91:0/2825321396 >> 172.21.15.91:0/2825321396 conn(0x7fc8580cec80 msgr2=0x7fc8580cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 -- 172.21.15.91:0/2825321396 shutdown_connections 2024-02-02T13:32:07.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.350+0000 7fc85fa00640 1 -- 172.21.15.91:0/2825321396 wait complete. 2024-02-02T13:32:07.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.352+0000 7fc85fa00640 1 Processor -- start 2024-02-02T13:32:07.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.352+0000 7fc85fa00640 1 -- start start 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 0x7fc8580b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 0x7fc858070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85814d6b0 con 0x7fc8380039a0 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580708a0 con 0x7fc85806fee0 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.353+0000 7fc85fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580b49c0 con 0x7fc85815d090 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58600/0 (socket says 172.21.15.91:58600) 2024-02-02T13:32:07.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 -- 172.21.15.91:0/4202421083 learned_addr learned my addr 172.21.15.91:0/4202421083 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:07.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 -- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 msgr2=0x7fc8580b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc857e00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 0x7fc8580b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85da00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 0x7fc858070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 0x7fc8580b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 -- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 msgr2=0x7fc858070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 0x7fc858070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 -- 172.21.15.91:0/4202421083 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580b1de0 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc857e00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 0x7fc8580b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85da00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 0x7fc858070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.354+0000 7fc85d000640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc84002c250 tx=0x7fc840004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.355+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840003820 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.355+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840003980 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.355+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840003d30 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.355+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8580aca70 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.355+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8580b4400 con 0x7fc8380039a0 2024-02-02T13:32:07.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.356+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc858066cf0 con 0x7fc8380039a0 2024-02-02T13:32:07.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.357+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc840034440 con 0x7fc8380039a0 2024-02-02T13:32:07.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.358+0000 7fc854c00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 0x7fc83006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:07.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.358+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8400aa900 con 0x7fc8380039a0 2024-02-02T13:32:07.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.358+0000 7fc857e00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 0x7fc83006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:07.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.358+0000 7fc857e00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 0x7fc83006c8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b15b0 tx=0x7fc848009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:07.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.362+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8400760c0 con 0x7fc8380039a0 2024-02-02T13:32:07.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.591+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8580acf90 con 0x7fc8380039a0 2024-02-02T13:32:07.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.592+0000 7fc854c00640 1 -- 172.21.15.91:0/4202421083 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc84004c070 con 0x7fc8380039a0 2024-02-02T13:32:07.593 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:32:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 msgr2=0x7fc83006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 0x7fc83006c8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b15b0 tx=0x7fc848009040 comp rx=0 tx=0).stop 2024-02-02T13:32:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 msgr2=0x7fc85814d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc84002c250 tx=0x7fc840004730 comp rx=0 tx=0).stop 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 shutdown_connections 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc83006a430 0x7fc83006c8f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc85815d090 0x7fc8580b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc85806fee0 0x7fc858070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.594+0000 7fc85fa00640 1 --2- 172.21.15.91:0/4202421083 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8380039a0 0x7fc85814d170 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.595+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 >> 172.21.15.91:0/4202421083 conn(0x7fc8580cec80 msgr2=0x7fc858071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.595+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 shutdown_connections 2024-02-02T13:32:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:07.595+0000 7fc85fa00640 1 -- 172.21.15.91:0/4202421083 wait complete. 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:32:07.615 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:32:07.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:07.616 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:32:07.669 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42198sec preferred_lft 42198sec 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:07.670 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:32:07.671 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:07.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:07.672 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:32:07.759 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:32:07.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:07.764 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:32:07.817 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:32:07.818 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:32:07.818 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:32:07.818 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:32:07.818 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:32:07.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:07.818 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:32:07.818 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:32:07.818 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:32:07.818 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:32:07.819 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:32:07.819 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:32:08.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:08.000 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:32:08.048 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:32:08.048 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:32:08.049 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:32:08.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:08.049 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:32:08.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:32:08.149 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:32:08.197 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:32:08.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:32:08.198 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:32:08.242 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:32:08.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:08.242 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:08.430 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:08.430 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:32:08.461 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:08.461 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:32:08.461 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:32:08.461 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:32:08.461 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:32:08.463 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:32:08.463 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:32:08.463 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:32:08.463 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:32:08.463 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:32:08.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:32:08.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:08.467 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:32:08.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:08.549 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:32:08.609 INFO:teuthology.orchestra.run.smithi091.stdout:client5836 2024-02-02T13:32:08.609 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:08.610 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:08.654 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:32:08.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.906+0000 7ff9afe00640 1 Processor -- start 2024-02-02T13:32:08.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.907+0000 7ff9afe00640 1 -- start start 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9afe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9a815a010 0x7ff9a80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9afe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9a80ce130 0x7ff9a80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9afe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80cc7c0 con 0x7ff9a80cd470 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9afe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80ce930 con 0x7ff9a815a010 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9afe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a806c890 con 0x7ff9a80ce130 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9ad400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9a815a010 0x7ff9a80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9ad400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff9a815a010 0x7ff9a80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58606/0 (socket says 172.21.15.91:58606) 2024-02-02T13:32:08.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.908+0000 7ff9ad400640 1 -- 172.21.15.91:0/972011592 learned_addr learned my addr 172.21.15.91:0/972011592 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:08.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.909+0000 7ff9ad400640 1 -- 172.21.15.91:0/972011592 >> v2:172.21.15.113:3300/0 conn(0x7ff9a80ce130 msgr2=0x7ff9a80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:08.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.909+0000 7ff9ad400640 1 --2- 172.21.15.91:0/972011592 >> v2:172.21.15.113:3300/0 conn(0x7ff9a80ce130 0x7ff9a80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:08.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.909+0000 7ff9ad400640 1 -- 172.21.15.91:0/972011592 >> v1:172.21.15.113:6789/0 conn(0x7ff9a80cd470 legacy=0x7ff9a80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:08.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.909+0000 7ff9ad400640 1 -- 172.21.15.91:0/972011592 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a806cb70 con 0x7ff9a815a010 2024-02-02T13:32:08.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.910+0000 7ff9ad400640 1 --2- 172.21.15.91:0/972011592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a815a010 0x7ff9a80c9670 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b1870 tx=0x7ff99802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cc42660108159d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:08.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.910+0000 7ff9a7e00640 1 -- 172.21.15.91:0/972011592 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980317e0 con 0x7ff9a815a010 2024-02-02T13:32:08.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.910+0000 7ff9a7e00640 1 -- 172.21.15.91:0/972011592 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998031940 con 0x7ff9a815a010 2024-02-02T13:32:08.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.910+0000 7ff9a7e00640 1 -- 172.21.15.91:0/972011592 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998031cd0 con 0x7ff9a815a010 2024-02-02T13:32:08.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 -- 172.21.15.91:0/972011592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a815a010 msgr2=0x7ff9a80c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 --2- 172.21.15.91:0/972011592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a815a010 0x7ff9a80c9670 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b1870 tx=0x7ff99802b880 comp rx=0 tx=0).stop 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 -- 172.21.15.91:0/972011592 shutdown_connections 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 --2- 172.21.15.91:0/972011592 >> v2:172.21.15.113:3300/0 conn(0x7ff9a80ce130 0x7ff9a80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 --2- 172.21.15.91:0/972011592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a815a010 0x7ff9a80c9670 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 -- 172.21.15.91:0/972011592 >> 172.21.15.91:0/972011592 conn(0x7ff9a80cec80 msgr2=0x7ff9a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 -- 172.21.15.91:0/972011592 shutdown_connections 2024-02-02T13:32:08.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.911+0000 7ff9afe00640 1 -- 172.21.15.91:0/972011592 wait complete. 2024-02-02T13:32:08.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.913+0000 7ff9afe00640 1 Processor -- start 2024-02-02T13:32:08.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.913+0000 7ff9afe00640 1 -- start start 2024-02-02T13:32:08.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a80ce130 0x7ff9a814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a815a010 0x7ff9a81586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81662e0 con 0x7ff9a8167380 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81665c0 con 0x7ff9a80ce130 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9afe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81668a0 con 0x7ff9a815a010 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9ade00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9ade00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58610/0 (socket says 172.21.15.91:58610) 2024-02-02T13:32:08.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9ade00640 1 -- 172.21.15.91:0/3300489292 learned_addr learned my addr 172.21.15.91:0/3300489292 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:08.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.914+0000 7ff9ad400640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a80ce130 0x7ff9a814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:08.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9aca00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a815a010 0x7ff9a81586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:08.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 -- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a815a010 msgr2=0x7ff9a81586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:08.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a815a010 0x7ff9a81586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 -- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a80ce130 msgr2=0x7ff9a814efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a80ce130 0x7ff9a814efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 -- 172.21.15.91:0/3300489292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980094d0 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9ade00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c780 tx=0x7ff9a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0003b70 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0003cd0 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00052b0 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a8166b80 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.915+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a8165f20 con 0x7ff9a8167380 2024-02-02T13:32:08.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.916+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a80ccf30 con 0x7ff9a8167380 2024-02-02T13:32:08.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.918+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff9a0005410 con 0x7ff9a8167380 2024-02-02T13:32:08.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.918+0000 7ff9a5600640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 0x7ff98006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:08.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.918+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a0087d20 con 0x7ff9a8167380 2024-02-02T13:32:08.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.919+0000 7ff9ad400640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 0x7ff98006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:08.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.919+0000 7ff9ad400640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 0x7ff98006c8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff998036000 tx=0x7ff99803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:08.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:08.923+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff9a0053270 con 0x7ff9a8167380 2024-02-02T13:32:09.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.153+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9a8066cf0 con 0x7ff9a8167380 2024-02-02T13:32:09.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.154+0000 7ff9a5600640 1 -- 172.21.15.91:0/3300489292 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9a8167380 2024-02-02T13:32:09.155 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 msgr2=0x7ff98006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 0x7ff98006c8f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff998036000 tx=0x7ff99803d040 comp rx=0 tx=0).stop 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 msgr2=0x7ff9a814bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c780 tx=0x7ff9a000cc40 comp rx=0 tx=0).stop 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 shutdown_connections 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff98006a430 0x7ff98006c8f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:09.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a815a010 0x7ff9a81586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:09.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff9a8167380 0x7ff9a814bf10 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:09.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 --2- 172.21.15.91:0/3300489292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a80ce130 0x7ff9a814efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:09.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 >> 172.21.15.91:0/3300489292 conn(0x7ff9a80cec80 msgr2=0x7ff9a815df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:09.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 shutdown_connections 2024-02-02T13:32:09.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:09.156+0000 7ff9afe00640 1 -- 172.21.15.91:0/3300489292 wait complete. 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:32:09.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:32:09.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.178 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:32:09.235 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42237sec preferred_lft 42237sec 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:32:09.237 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:32:09.238 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:32:09.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.240 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:32:09.328 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:32:09.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.332 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:32:09.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.385 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:32:09.385 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:32:09.385 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:32:09.386 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:32:09.386 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:32:09.386 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:32:09.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.565 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:32:09.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:32:09.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:32:09.617 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:32:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.618 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:32:09.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:32:09.713 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:32:09.767 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:32:09.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:32:09.767 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:32:09.818 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:32:09.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:09.951 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:09.951 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:32:09.985 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:32:09.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:32:09.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:32:09.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:32:09.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:32:09.987 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:32:09.987 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:32:09.988 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:32:09.988 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:32:09.988 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:32:09.991 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:32:09.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:09.991 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:32:10.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:10.072 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:32:10.127 INFO:teuthology.orchestra.run.smithi113.stdout:client5384 2024-02-02T13:32:10.128 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:10.128 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:10.177 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:10.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.394+0000 7f36c1400640 1 Processor -- start 2024-02-02T13:32:10.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.395+0000 7f36c1400640 1 -- start start 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36c1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f36bc0b8990 0x7f36bc0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36c1400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc0b8e00 con 0x7f36bc15cee0 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36c1400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc0b7d40 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36c1400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc0b6ed0 con 0x7f36bc0b8990 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36bbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f36bc15cee0 0x7f36bc0cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37694/0 (socket says 172.21.15.91:37694) 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36bbe00640 1 -- 172.21.15.91:0/1379504788 learned_addr learned my addr 172.21.15.91:0/1379504788 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:10.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.396+0000 7f36c0a00640 1 --2- 172.21.15.91:0/1379504788 >> v2:172.21.15.91:3300/0 conn(0x7f36bc0b8990 0x7f36bc0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.397+0000 7f36c0a00640 1 -- 172.21.15.91:0/1379504788 >> v1:172.21.15.113:6789/0 conn(0x7f36bc0ce1d0 legacy=0x7f36bc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.397+0000 7f36c0a00640 1 -- 172.21.15.91:0/1379504788 >> v1:172.21.15.91:6789/0 conn(0x7f36bc15cee0 legacy=0x7f36bc0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.397+0000 7f36c0a00640 1 -- 172.21.15.91:0/1379504788 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0b81d0 con 0x7f36bc0b8990 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.397+0000 7f36bb400640 1 -- 172.21.15.91:0/1379504788 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791074078 0 0) 0x7f36bc0b8e00 con 0x7f36bc15cee0 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.398+0000 7f36c0a00640 1 --2- 172.21.15.91:0/1379504788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0b8990 0x7f36bc0b6990 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009d90 tx=0x7f36b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f6c8e6cc265c1c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:10.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.398+0000 7f36bb400640 1 -- 172.21.15.91:0/1379504788 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b402cc60 con 0x7f36bc0b8990 2024-02-02T13:32:10.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.398+0000 7f36bb400640 1 -- 172.21.15.91:0/1379504788 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b402cdc0 con 0x7f36bc0b8990 2024-02-02T13:32:10.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.398+0000 7f36bb400640 1 -- 172.21.15.91:0/1379504788 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4034660 con 0x7f36bc0b8990 2024-02-02T13:32:10.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 -- 172.21.15.91:0/1379504788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0b8990 msgr2=0x7f36bc0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 --2- 172.21.15.91:0/1379504788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0b8990 0x7f36bc0b6990 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009d90 tx=0x7f36b402beb0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 -- 172.21.15.91:0/1379504788 shutdown_connections 2024-02-02T13:32:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 --2- 172.21.15.91:0/1379504788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0b8990 0x7f36bc0b6990 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 -- 172.21.15.91:0/1379504788 >> 172.21.15.91:0/1379504788 conn(0x7f36bc070690 msgr2=0x7f36bc070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 -- 172.21.15.91:0/1379504788 shutdown_connections 2024-02-02T13:32:10.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.399+0000 7f36c1400640 1 -- 172.21.15.91:0/1379504788 wait complete. 2024-02-02T13:32:10.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.401+0000 7f36c1400640 1 Processor -- start 2024-02-02T13:32:10.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.402+0000 7f36c1400640 1 -- start start 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36bc0b8990 0x7f36bc0a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36bc15cee0 0x7f36bc1f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0a9d30 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc14bfa0 con 0x7f36bc15cee0 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36c1400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc14c240 con 0x7f36bc0b8990 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36b3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36b3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45266/0 (socket says 172.21.15.91:45266) 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36b3200640 1 -- 172.21.15.91:0/1014151676 learned_addr learned my addr 172.21.15.91:0/1014151676 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.403+0000 7f36bbe00640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36bc0b8990 0x7f36bc0a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36c0a00640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36bc15cee0 0x7f36bc1f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:10.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 -- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36bc0b8990 msgr2=0x7f36bc0a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36bc0b8990 0x7f36bc0a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 -- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36bc15cee0 msgr2=0x7f36bc1f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36bc15cee0 0x7f36bc1f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 -- 172.21.15.91:0/1014151676 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009a70 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b3200640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c430 tx=0x7f36a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4011790 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a40118f0 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4011ce0 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc14c520 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.404+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc14cae0 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.406+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc04dff0 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.407+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f36a401c460 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.408+0000 7f36b8c00640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 0x7f369406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:10.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.408+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36a4088880 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.408+0000 7f36bbe00640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 0x7f369406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:10.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.409+0000 7f36bbe00640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 0x7f369406c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36ac000f60 tx=0x7f36ac007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:10.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.413+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f36a4053e70 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.641+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36bc0b4fa0 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.642+0000 7f36b8c00640 1 -- 172.21.15.91:0/1014151676 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f36a4057490 con 0x7f36bc0ce1d0 2024-02-02T13:32:10.644 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:10.644 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:10.644 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 msgr2=0x7f369406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 0x7f369406c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36ac000f60 tx=0x7f36ac007b50 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 msgr2=0x7f36bc14f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c430 tx=0x7f36a400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 shutdown_connections 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f369406a430 0x7f369406c8f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36bc15cee0 0x7f36bc1f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f36bc0ce1d0 0x7f36bc14f490 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 --2- 172.21.15.91:0/1014151676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36bc0b8990 0x7f36bc0a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 >> 172.21.15.91:0/1014151676 conn(0x7f36bc070690 msgr2=0x7f36bc0c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:10.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 shutdown_connections 2024-02-02T13:32:10.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:10.645+0000 7f36c1400640 1 -- 172.21.15.91:0/1014151676 wait complete. 2024-02-02T13:32:10.667 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:32:10.889 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":7584,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.322606527,"mdsmap_epoch":64,"osdmap_epoch":143,"osdmap_epoch_barrier":143,"uptime":15.660527366} 2024-02-02T13:32:10.889 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 7584, "mdsmap_epoch": 64, "osdmap_epoch": 143, "osdmap_epoch_barrier": 143, "rank_uptime": 10.322606527, "state": "up:active", "uptime": 15.660527366, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:32:10.889 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-02-02T13:32:11.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.103+0000 7fd3dc800640 1 Processor -- start 2024-02-02T13:32:11.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.103+0000 7fd3dc800640 1 -- start start 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.104+0000 7fd3dc800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3d4160a20 0x7fd3d40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.104+0000 7fd3dc800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d40b7960 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.104+0000 7fd3dc800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d40b7c40 con 0x7fd3d40b5010 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.104+0000 7fd3dc800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d40b85e0 con 0x7fd3d4160a20 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.105+0000 7fd3d9400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd3d40b4ba0 0x7fd3d40d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37706/0 (socket says 172.21.15.91:37706) 2024-02-02T13:32:11.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.105+0000 7fd3d9400640 1 -- 172.21.15.91:0/2068188102 learned_addr learned my addr 172.21.15.91:0/2068188102 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:11.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.105+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322046439 0 0) 0x7fd3d40b7960 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.105+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2068188102 >> v2:172.21.15.113:3300/0 conn(0x7fd3d4160a20 0x7fd3d40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.105+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b4003520 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933224856 0 0) 0x7fd3b4003520 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 >> v2:172.21.15.113:3300/0 conn(0x7fd3d4160a20 msgr2=0x7fd3d40b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 --2- 172.21.15.91:0/2068188102 >> v2:172.21.15.113:3300/0 conn(0x7fd3d4160a20 0x7fd3d40b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 >> v1:172.21.15.113:6789/0 conn(0x7fd3d40b5010 legacy=0x7fd3d40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40b7f20 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fd3bc001670 con 0x7fd3d40b4ba0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 >> v1:172.21.15.91:6789/0 conn(0x7fd3d40b4ba0 legacy=0x7fd3d40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b40039c0 0x7fd3b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3b40047c0 0x7fd3b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 0x7fd3b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40b85e0 con 0x7fd3b4005180 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b400c080 con 0x7fd3b40039c0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b400c240 con 0x7fd3b40047c0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3da800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 0x7fd3b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.106+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b40039c0 0x7fd3b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3d9400640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3b40047c0 0x7fd3b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 -- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3b40047c0 msgr2=0x7fd3b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3b40047c0 0x7fd3b4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 -- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b40039c0 msgr2=0x7fd3b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b40039c0 0x7fd3b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 -- 172.21.15.91:0/2068188102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4003520 con 0x7fd3b4005180 2024-02-02T13:32:11.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.107+0000 7fd3da800640 1 -- 172.21.15.91:0/2068188102 reap_dead start 2024-02-02T13:32:11.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.108+0000 7fd3da800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 0x7fd3b400b980 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc001750 tx=0x7fd3b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49913c3195ca0cde server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:11.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.108+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0052d0 con 0x7fd3b4005180 2024-02-02T13:32:11.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.108+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc02e050 con 0x7fd3b4005180 2024-02-02T13:32:11.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.108+0000 7fd3d8a00640 1 -- 172.21.15.91:0/2068188102 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003830 con 0x7fd3b4005180 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 -- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 msgr2=0x7fd3b400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 0x7fd3b400b980 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc001750 tx=0x7fd3b40057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 -- 172.21.15.91:0/2068188102 shutdown_connections 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2068188102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3b4005180 0x7fd3b400b980 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 -- 172.21.15.91:0/2068188102 >> 172.21.15.91:0/2068188102 conn(0x7fd3d4070480 msgr2=0x7fd3d4070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 -- 172.21.15.91:0/2068188102 shutdown_connections 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.109+0000 7fd3dc800640 1 -- 172.21.15.91:0/2068188102 wait complete. 2024-02-02T13:32:11.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.111+0000 7fd3dc800640 1 Processor -- start 2024-02-02T13:32:11.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.111+0000 7fd3dc800640 1 -- start start 2024-02-02T13:32:11.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 0x7fd3d414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d40d1940 0x7fd3d40d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40d2300 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d414da80 con 0x7fd3b4005180 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3dc800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40caf00 con 0x7fd3d40d1940 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45284/0 (socket says 172.21.15.91:45284) 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 -- 172.21.15.91:0/2346160952 learned_addr learned my addr 172.21.15.91:0/2346160952 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3d9400640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d40d1940 0x7fd3d40d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 0x7fd3d414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 -- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d40d1940 msgr2=0x7fd3d40d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d40d1940 0x7fd3d40d1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.112+0000 7fd3da800640 1 -- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 msgr2=0x7fd3d414d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3da800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 0x7fd3d414d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3da800640 1 -- 172.21.15.91:0/2346160952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40cb1e0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 0x7fd3d414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3da800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc001870 tx=0x7fd3cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc039070 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc007a30 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0054e0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d40cb4c0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.113+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d40cba80 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.115+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d4066cf0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.115+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3cc03c020 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.116+0000 7fd3d2000640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 0x7fd3ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:11.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.116+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3cc0a8cc0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.117+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 0x7fd3ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:11.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.117+0000 7fd3d9e00640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 0x7fd3ac06c8f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd3d415aeb0 tx=0x7fd3c400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:11.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.121+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd3cc0742b0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.313+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fd3d40cbc40 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.446+0000 7fd3d2000640 1 -- 172.21.15.91:0/2346160952 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd3cc0440c0 con 0x7fd3d40ca1f0 2024-02-02T13:32:11.448 INFO:teuthology.orchestra.run.smithi091.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 msgr2=0x7fd3ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 0x7fd3ac06c8f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd3d415aeb0 tx=0x7fd3c400b040 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 msgr2=0x7fd3d40ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc001870 tx=0x7fd3cc001120 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 shutdown_connections 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3ac06a430 0x7fd3ac06c8f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d40d1940 0x7fd3d40d1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3d40ca1f0 0x7fd3d40ca6a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 --2- 172.21.15.91:0/2346160952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3b4005180 0x7fd3d414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:11.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 >> 172.21.15.91:0/2346160952 conn(0x7fd3d4070480 msgr2=0x7fd3d40cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:11.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 shutdown_connections 2024-02-02T13:32:11.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:11.450+0000 7fd3dc800640 1 -- 172.21.15.91:0/2346160952 wait complete. 2024-02-02T13:32:11.473 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:32:11.702 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:32:11.702 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:32:11.702 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-02-02T13:32:11.702 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:11.702 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:11.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:11.754 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-02-02T13:32:11.989 INFO:teuthology.orchestra.run.smithi091.stderr:16+0 records in 2024-02-02T13:32:11.989 INFO:teuthology.orchestra.run.smithi091.stderr:16+0 records out 2024-02-02T13:32:11.989 INFO:teuthology.orchestra.run.smithi091.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.177288 s, 94.6 MB/s 2024-02-02T13:32:11.991 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:11.991 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:12.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:12.042 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-02-02T13:32:12.286 INFO:teuthology.orchestra.run.smithi091.stderr:17+0 records in 2024-02-02T13:32:12.287 INFO:teuthology.orchestra.run.smithi091.stderr:17+0 records out 2024-02-02T13:32:12.287 INFO:teuthology.orchestra.run.smithi091.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.14863 s, 120 MB/s 2024-02-02T13:32:12.288 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-02-02T13:32:12.288 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:12.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.490+0000 7f6958400640 1 Processor -- start 2024-02-02T13:32:12.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.490+0000 7f6958400640 1 -- start start 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.491+0000 7f6958400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f695006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.491+0000 7f6958400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695006d910 con 0x7f695015ad10 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.491+0000 7f6958400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69500cc850 con 0x7f69500cb580 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.491+0000 7f6958400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695006d730 con 0x7f695015b0f0 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f6955a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f695015ad10 0x7f695006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37716/0 (socket says 172.21.15.91:37716) 2024-02-02T13:32:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f6955a00640 1 -- 172.21.15.91:0/2976411229 learned_addr learned my addr 172.21.15.91:0/2976411229 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962096179 0 0) 0x7f695006d910 con 0x7f695015ad10 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6930003520 con 0x7f695015ad10 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f6955000640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f695006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839321757 0 0) 0x7f6930003520 con 0x7f695015ad10 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 msgr2=0x7f695006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f695006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.492+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 >> v1:172.21.15.113:6789/0 conn(0x7f69500cb580 legacy=0x7f69500c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500ccb30 con 0x7f695015ad10 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6940001670 con 0x7f695015ad10 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 >> v1:172.21.15.91:6789/0 conn(0x7f695015ad10 legacy=0x7f695006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69300039a0 0x7f6930003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300047c0 0x7f6930004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f6930005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695006d730 con 0x7f69300039a0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693000bb40 con 0x7f69300047c0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693000bd00 con 0x7f695015b0f0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f6955a00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69300039a0 0x7f6930003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f6955000640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300047c0 0x7f6930004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f6955a00640 1 -- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 msgr2=0x7f6930005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.493+0000 7f6956400640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f6930005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6955a00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f6930005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6955a00640 1 -- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300047c0 msgr2=0x7f6930004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6955a00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300047c0 0x7f6930004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6955a00640 1 -- 172.21.15.91:0/2976411229 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930003520 con 0x7f69300039a0 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6955000640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300047c0 0x7f6930004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:12.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.494+0000 7f6956400640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f695015b0f0 0x7f6930005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:12.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f6955a00640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69300039a0 0x7f6930003dc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f694000c0e0 tx=0x7f69300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28912bec837164c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940005ad0 con 0x7f69300039a0 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940005c50 con 0x7f69300039a0 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f694fe00640 1 -- 172.21.15.91:0/2976411229 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694002c810 con 0x7f69300039a0 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f6958400640 1 -- 172.21.15.91:0/2976411229 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69300039a0 msgr2=0x7f6930003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f6958400640 1 --2- 172.21.15.91:0/2976411229 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69300039a0 0x7f6930003dc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f694000c0e0 tx=0x7f69300057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.495+0000 7f6956400640 1 -- 172.21.15.91:0/2976411229 reap_dead start 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.496+0000 7f6958400640 1 -- 172.21.15.91:0/2976411229 shutdown_connections 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.496+0000 7f6958400640 1 -- 172.21.15.91:0/2976411229 >> 172.21.15.91:0/2976411229 conn(0x7f69500b0700 msgr2=0x7f69500b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.496+0000 7f6958400640 1 -- 172.21.15.91:0/2976411229 shutdown_connections 2024-02-02T13:32:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.496+0000 7f6958400640 1 -- 172.21.15.91:0/2976411229 wait complete. 2024-02-02T13:32:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.498+0000 7f6958400640 1 Processor -- start 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.498+0000 7f6958400640 1 -- start start 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 0x7f69501501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69501f3150 0x7f69501f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69501518e0 con 0x7f69501f3de0 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69501506f0 con 0x7f69300039a0 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6958400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6950150c60 con 0x7f69501f3150 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6956400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6956400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45302/0 (socket says 172.21.15.91:45302) 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6956400640 1 -- 172.21.15.91:0/1398094626 learned_addr learned my addr 172.21.15.91:0/1398094626 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6955000640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69501f3150 0x7f69501f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.499+0000 7f6955a00640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 0x7f69501501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 -- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69501f3150 msgr2=0x7f69501f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69501f3150 0x7f69501f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 -- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 msgr2=0x7f69501501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 0x7f69501501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 -- 172.21.15.91:0/1398094626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950150f40 con 0x7f69501f3de0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6955a00640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 0x7f69501501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6956400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6948000f80 tx=0x7f694800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948006d50 con 0x7f69501f3de0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6948003970 con 0x7f69501f3de0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6950151220 con 0x7f69501f3de0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948003d20 con 0x7f69501f3de0 2024-02-02T13:32:12.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.500+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69501f35d0 con 0x7f69501f3de0 2024-02-02T13:32:12.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.501+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6950069fa0 con 0x7f69501f3de0 2024-02-02T13:32:12.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.502+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6948005070 con 0x7f69501f3de0 2024-02-02T13:32:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.503+0000 7f694d600640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 0x7f692806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.503+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6948088560 con 0x7f69501f3de0 2024-02-02T13:32:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.503+0000 7f6955a00640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 0x7f692806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.504+0000 7f6955a00640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 0x7f692806c8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6940034000 tx=0x7f6940009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:12.509 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.507+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6948053be0 con 0x7f69501f3de0 2024-02-02T13:32:12.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.698+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6950151a20 con 0x7f69501f3de0 2024-02-02T13:32:12.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.701+0000 7f694d600640 1 -- 172.21.15.91:0/1398094626 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) v1 ==== 81+0+33502 (secure 0 0 0) 0x7f6950151a20 con 0x7f69501f3de0 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 144, 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:11.430888+0000", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:12.703 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:12.704 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:12.705 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:12.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:12.707 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:12.708 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:12.709 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:12.710 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:12.711 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:12.712 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:12.713 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:12.714 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.715 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:12.716 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:12.717 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "144", 2024-02-02T13:32:12.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:12.719 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:12.720 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.721 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:12.722 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.723 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.724 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.725 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.726 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:12.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.733 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:12.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:12.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.738 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.739 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:12.743 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.744 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.747 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:12.749 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.752 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.754 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:12.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.759 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:12.760 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.761 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:12.762 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:12.764 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.765 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.766 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.768 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:12.770 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:12.771 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:12.772 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.703+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 msgr2=0x7f692806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.703+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 0x7f692806c8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6940034000 tx=0x7f6940009030 comp rx=0 tx=0).stop 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.703+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 msgr2=0x7f6950151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.703+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6948000f80 tx=0x7f694800dc00 comp rx=0 tx=0).stop 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 shutdown_connections 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f692806a430 0x7f692806c8f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69501f3150 0x7f69501f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69501f3de0 0x7f6950151d00 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 --2- 172.21.15.91:0/1398094626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69300039a0 0x7f69501501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:12.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 >> 172.21.15.91:0/1398094626 conn(0x7f69500b0700 msgr2=0x7f6950070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:12.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 shutdown_connections 2024-02-02T13:32:12.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:12.704+0000 7f6958400640 1 -- 172.21.15.91:0/1398094626 wait complete. 2024-02-02T13:32:12.775 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:32:17.776 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:17.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.993+0000 7f3da0a00640 1 Processor -- start 2024-02-02T13:32:17.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.993+0000 7f3da0a00640 1 -- start start 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3da0a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d98159340 0x7f3d9806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3da0a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d980cd5e0 con 0x7f3d980cd150 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3da0a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d980cd8c0 con 0x7f3d9815d870 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3da0a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980cdba0 con 0x7f3d98159340 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d98159340 0x7f3d9806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d98159340 0x7f3d9806ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45316/0 (socket says 172.21.15.91:45316) 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9d600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3d980cd150 0x7f3d9815d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:37722/0 (socket says 172.21.15.91:37722) 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9e000640 1 -- 172.21.15.91:0/4161080816 learned_addr learned my addr 172.21.15.91:0/4161080816 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:17.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9e000640 1 -- 172.21.15.91:0/4161080816 >> v1:172.21.15.113:6789/0 conn(0x7f3d9815d870 legacy=0x7f3d98159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.995+0000 7f3d9e000640 1 -- 172.21.15.91:0/4161080816 >> v1:172.21.15.91:6789/0 conn(0x7f3d980cd150 legacy=0x7f3d9815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.996+0000 7f3d9e000640 1 -- 172.21.15.91:0/4161080816 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980cde80 con 0x7f3d98159340 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.996+0000 7f3d9cc00640 1 -- 172.21.15.91:0/4161080816 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181698851 0 0) 0x7f3d980cd5e0 con 0x7f3d980cd150 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3d9e000640 1 --2- 172.21.15.91:0/4161080816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d98159340 0x7f3d9806ff20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009870 tx=0x7f3d9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45d62edeca22f97 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3d9cc00640 1 -- 172.21.15.91:0/4161080816 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90030860 con 0x7f3d98159340 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3d9cc00640 1 -- 172.21.15.91:0/4161080816 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d900309c0 con 0x7f3d98159340 2024-02-02T13:32:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3d9cc00640 1 -- 172.21.15.91:0/4161080816 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90030d50 con 0x7f3d98159340 2024-02-02T13:32:17.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3da0a00640 1 -- 172.21.15.91:0/4161080816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d98159340 msgr2=0x7f3d9806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:17.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.997+0000 7f3da0a00640 1 --2- 172.21.15.91:0/4161080816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d98159340 0x7f3d9806ff20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009870 tx=0x7f3d9002b9a0 comp rx=0 tx=0).stop 2024-02-02T13:32:17.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.998+0000 7f3da0a00640 1 -- 172.21.15.91:0/4161080816 shutdown_connections 2024-02-02T13:32:17.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.998+0000 7f3da0a00640 1 --2- 172.21.15.91:0/4161080816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d98159340 0x7f3d9806ff20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.998+0000 7f3da0a00640 1 -- 172.21.15.91:0/4161080816 >> 172.21.15.91:0/4161080816 conn(0x7f3d980b0700 msgr2=0x7f3d980b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.998+0000 7f3da0a00640 1 -- 172.21.15.91:0/4161080816 shutdown_connections 2024-02-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.998+0000 7f3da0a00640 1 -- 172.21.15.91:0/4161080816 wait complete. 2024-02-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.999+0000 7f3da0a00640 1 Processor -- start 2024-02-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:17.999+0000 7f3da0a00640 1 -- start start 2024-02-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 0x7f3d9806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d9806cf00 0x7f3d98145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981589e0 con 0x7f3d9815d870 2024-02-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9806ca70 con 0x7f3d98159340 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.000+0000 7f3da0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9806be60 con 0x7f3d9806cf00 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45324/0 (socket says 172.21.15.91:45324) 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 -- 172.21.15.91:0/685416523 learned_addr learned my addr 172.21.15.91:0/685416523 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9e000640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 0x7f3d9806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 -- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d9806cf00 msgr2=0x7f3d98145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d9806cf00 0x7f3d98145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 -- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 msgr2=0x7f3d9806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 0x7f3d9806b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 -- 172.21.15.91:0/685416523 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900094d0 con 0x7f3d9815d870 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9e000640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 0x7f3d9806b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.001+0000 7f3d9d600640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c4b0 tx=0x7f3d8000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.002+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800074e0 con 0x7f3d9815d870 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.002+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d80007660 con 0x7f3d9815d870 2024-02-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.002+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80004a20 con 0x7f3d9815d870 2024-02-02T13:32:18.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.002+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9806c140 con 0x7f3d9815d870 2024-02-02T13:32:18.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.002+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9806c700 con 0x7f3d9815d870 2024-02-02T13:32:18.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.003+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9806cc20 con 0x7f3d9815d870 2024-02-02T13:32:18.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.004+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3d80004b80 con 0x7f3d9815d870 2024-02-02T13:32:18.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.005+0000 7f3d96000640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 0x7f3d7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:18.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.005+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d80087b90 con 0x7f3d9815d870 2024-02-02T13:32:18.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.005+0000 7f3d9e000640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 0x7f3d7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:18.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.006+0000 7f3d9e000640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 0x7f3d7006c8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3d90036000 tx=0x7f3d9003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:18.011 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.010+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3d9806cc20 con 0x7f3d9815d870 2024-02-02T13:32:18.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.201+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3d9806cbe0 con 0x7f3d9815d870 2024-02-02T13:32:18.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.204+0000 7f3d96000640 1 -- 172.21.15.91:0/685416523 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) v1 ==== 81+0+33502 (secure 0 0 0) 0x7f3d80018070 con 0x7f3d9815d870 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 144, 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:11.430888+0000", 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:18.207 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:18.208 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:18.209 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:18.210 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:18.211 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:18.212 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:18.213 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:18.214 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.215 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.216 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:18.217 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:18.218 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:18.219 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:18.220 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:18.221 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:18.222 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:18.223 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:18.225 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "144", 2024-02-02T13:32:18.226 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:18.227 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:18.231 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.234 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:18.237 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.238 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.239 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.240 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.241 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.242 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:18.243 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.244 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.245 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.246 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:18.247 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.248 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:18.249 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.250 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.251 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.252 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.253 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.254 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:18.255 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.256 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.257 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.258 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.259 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:18.260 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:18.261 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.262 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.263 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.264 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.265 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.266 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.267 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.269 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:18.270 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.271 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.272 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.273 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:18.274 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.275 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:18.276 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.277 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.278 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.279 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:18.280 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:18.281 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.282 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.283 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.284 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.285 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.286 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.287 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:18.288 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:18.289 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:18.290 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:18.291 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.207+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 msgr2=0x7f3d7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.207+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 0x7f3d7006c8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3d90036000 tx=0x7f3d9003b040 comp rx=0 tx=0).stop 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.207+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 msgr2=0x7f3d981584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.207+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c4b0 tx=0x7f3d8000c970 comp rx=0 tx=0).stop 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 shutdown_connections 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3d7006a430 0x7f3d7006c8f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d9806cf00 0x7f3d98145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d9815d870 0x7f3d981584a0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 --2- 172.21.15.91:0/685416523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d98159340 0x7f3d9806b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:18.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 >> 172.21.15.91:0/685416523 conn(0x7f3d980b0700 msgr2=0x7f3d980cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:18.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 shutdown_connections 2024-02-02T13:32:18.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:18.208+0000 7f3da0a00640 1 -- 172.21.15.91:0/685416523 wait complete. 2024-02-02T13:32:18.294 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:32:23.296 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:23.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.511+0000 7faddbc00640 1 Processor -- start 2024-02-02T13:32:23.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.511+0000 7faddbc00640 1 -- start start 2024-02-02T13:32:23.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7faddbc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadd40cbc90 0x7fadd415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7faddbc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 0x7fadd406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7faddbc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40ca0c0 con 0x7fadd4159de0 2024-02-02T13:32:23.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7faddbc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd40ca380 con 0x7fadd40cbc90 2024-02-02T13:32:23.514 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7faddbc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40ca660 con 0x7fadd415e170 2024-02-02T13:32:23.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadd9200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fadd4159de0 0x7fadd40cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40518/0 (socket says 172.21.15.91:40518) 2024-02-02T13:32:23.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadd9200640 1 -- 172.21.15.91:0/4232001241 learned_addr learned my addr 172.21.15.91:0/4232001241 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:23.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584147595 0 0) 0x7fadd40ca0c0 con 0x7fadd4159de0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadcbe00640 1 --2- 172.21.15.91:0/4232001241 >> v2:172.21.15.113:3300/0 conn(0x7fadd40cbc90 0x7fadd415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb4003520 con 0x7fadd4159de0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.513+0000 7fadd9c00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 0x7fadd406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57545039 0 0) 0x7fadb4003520 con 0x7fadd4159de0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 msgr2=0x7fadd406fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 0x7fadd406fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 >> v2:172.21.15.113:3300/0 conn(0x7fadd40cbc90 msgr2=0x7fadd415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 --2- 172.21.15.91:0/4232001241 >> v2:172.21.15.113:3300/0 conn(0x7fadd40cbc90 0x7fadd415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40ca940 con 0x7fadd4159de0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fadc4001690 con 0x7fadd4159de0 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 >> v1:172.21.15.91:6789/0 conn(0x7fadd4159de0 legacy=0x7fadd40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb4003970 0x7fadb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadb40047c0 0x7fadb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 0x7fadb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40ca380 con 0x7fadb40047c0 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40ca660 con 0x7fadb4003970 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb400b9d0 con 0x7fadd415e170 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadcbe00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadb40047c0 0x7fadb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadd9200640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb4003970 0x7fadb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadcbe00640 1 -- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 msgr2=0x7fadb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadcbe00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd415e170 0x7fadb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.514+0000 7fadcbe00640 1 -- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb4003970 msgr2=0x7fadb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.515+0000 7fadcbe00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb4003970 0x7fadb4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.515+0000 7fadcbe00640 1 -- 172.21.15.91:0/4232001241 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4003520 con 0x7fadb40047c0 2024-02-02T13:32:23.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.515+0000 7fadd9200640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb4003970 0x7fadb4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:23.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7fadcbe00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadb40047c0 0x7fadb4004c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009ac0 tx=0x7fadb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c26c2b158dfabe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:23.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc02c710 con 0x7fadb40047c0 2024-02-02T13:32:23.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc02c870 con 0x7fadb40047c0 2024-02-02T13:32:23.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7fadd3e00640 1 -- 172.21.15.91:0/4232001241 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc02cc00 con 0x7fadb40047c0 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7faddbc00640 1 -- 172.21.15.91:0/4232001241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadb40047c0 msgr2=0x7fadb4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7faddbc00640 1 --2- 172.21.15.91:0/4232001241 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadb40047c0 0x7fadb4004c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009ac0 tx=0x7fadb40057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.516+0000 7fadd9c00640 1 -- 172.21.15.91:0/4232001241 reap_dead start 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.517+0000 7faddbc00640 1 -- 172.21.15.91:0/4232001241 shutdown_connections 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.517+0000 7faddbc00640 1 -- 172.21.15.91:0/4232001241 >> 172.21.15.91:0/4232001241 conn(0x7fadd40b0700 msgr2=0x7fadd40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.517+0000 7faddbc00640 1 -- 172.21.15.91:0/4232001241 shutdown_connections 2024-02-02T13:32:23.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.517+0000 7faddbc00640 1 -- 172.21.15.91:0/4232001241 wait complete. 2024-02-02T13:32:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.518+0000 7faddbc00640 1 Processor -- start 2024-02-02T13:32:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.519+0000 7faddbc00640 1 -- start start 2024-02-02T13:32:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb40047c0 0x7fadd414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd40c26a0 0x7fadd40aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd414f8e0 con 0x7fadd414fac0 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4151c20 con 0x7fadb40047c0 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7faddbc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4151f00 con 0x7fadd40c26a0 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40732/0 (socket says 172.21.15.91:40732) 2024-02-02T13:32:23.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 -- 172.21.15.91:0/409617239 learned_addr learned my addr 172.21.15.91:0/409617239 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 -- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd40c26a0 msgr2=0x7fadd40aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd40c26a0 0x7fadd40aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 -- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb40047c0 msgr2=0x7fadd414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb40047c0 0x7fadd414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.520+0000 7fadcbe00640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd41521e0 con 0x7fadd414fac0 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7fadcbe00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02c2b0 tx=0x7fadbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc003810 con 0x7fadd414fac0 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc02de90 con 0x7fadd414fac0 2024-02-02T13:32:23.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc003bb0 con 0x7fadd414fac0 2024-02-02T13:32:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd41524c0 con 0x7fadd414fac0 2024-02-02T13:32:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.521+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd4150160 con 0x7fadd414fac0 2024-02-02T13:32:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.522+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd4152040 con 0x7fadd414fac0 2024-02-02T13:32:23.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.523+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fadbc003d10 con 0x7fadd414fac0 2024-02-02T13:32:23.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.524+0000 7fadd1600640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 0x7fadac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:23.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.524+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadbc0aaa10 con 0x7fadd414fac0 2024-02-02T13:32:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.524+0000 7fadd9200640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 0x7fadac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.524+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7fadac071f60 con 0x7fadd414fac0 2024-02-02T13:32:23.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.525+0000 7fadd9200640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 0x7fadac06c8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadc40017c0 tx=0x7fadc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:23.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.529+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fadbc044060 con 0x7fadd414fac0 2024-02-02T13:32:23.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.718+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fadd4152650 con 0x7fadd414fac0 2024-02-02T13:32:23.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.722+0000 7fadd1600640 1 -- 172.21.15.91:0/409617239 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33521 (secure 0 0 0) 0x7fadbc039070 con 0x7fadd414fac0 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 145, 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:19.411226+0000", 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:23.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:23.726 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:23.727 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:23.728 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:23.729 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:23.730 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:23.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:23.732 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:23.733 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:23.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:23.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:23.737 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "145", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:23.738 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:23.739 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:23.740 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:23.741 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.742 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.743 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.744 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.745 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.747 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:23.749 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.751 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.752 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:23.753 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.754 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.755 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:23.756 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.757 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.758 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.759 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.760 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:23.761 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.762 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.763 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.764 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.767 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.768 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.770 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.771 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.772 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.773 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.775 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.776 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:23.777 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:23.778 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.779 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:23.780 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:23.781 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.782 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.783 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.784 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.785 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:23.786 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:23.787 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:23.788 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:23.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.724+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 msgr2=0x7fadac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.724+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 0x7fadac06c8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadc40017c0 tx=0x7fadc4009040 comp rx=0 tx=0).stop 2024-02-02T13:32:23.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.724+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 msgr2=0x7fadd40ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:23.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.724+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02c2b0 tx=0x7fadbc004730 comp rx=0 tx=0).stop 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 shutdown_connections 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fadac06a430 0x7fadac06c8f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd40c26a0 0x7fadd40aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fadd414fac0 0x7fadd40ae640 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 --2- 172.21.15.91:0/409617239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb40047c0 0x7fadd414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 >> 172.21.15.91:0/409617239 conn(0x7fadd40b0700 msgr2=0x7fadd40cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 shutdown_connections 2024-02-02T13:32:23.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:23.725+0000 7faddbc00640 1 -- 172.21.15.91:0/409617239 wait complete. 2024-02-02T13:32:23.790 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-02-02T13:32:23.791 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:23.791 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:23.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:23.850 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-02-02T13:32:24.267 INFO:teuthology.orchestra.run.smithi091.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-02-02T13:32:24.267 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:32:24.267 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:32:24.267 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.361569 s, 145 MB/s 2024-02-02T13:32:24.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:32:24.269 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:24.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.467+0000 7f8cafa00640 1 Processor -- start 2024-02-02T13:32:24.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.467+0000 7f8cafa00640 1 -- start start 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cafa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ca8159de0 0x7f8ca80cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cafa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca815e340 0x7f8ca806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cafa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80ca0c0 con 0x7f8ca80cbac0 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cafa00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca815e790 con 0x7f8ca8159de0 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cafa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80cbf50 con 0x7f8ca815e340 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cad000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ca8159de0 0x7f8ca80cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cad000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ca8159de0 0x7f8ca80cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40740/0 (socket says 172.21.15.91:40740) 2024-02-02T13:32:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.468+0000 7f8cad000640 1 -- 172.21.15.91:0/3362230424 learned_addr learned my addr 172.21.15.91:0/3362230424 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.469+0000 7f8cad000640 1 -- 172.21.15.91:0/3362230424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca815e340 msgr2=0x7f8ca806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.469+0000 7f8cad000640 1 --2- 172.21.15.91:0/3362230424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca815e340 0x7f8ca806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.469+0000 7f8cad000640 1 -- 172.21.15.91:0/3362230424 >> v1:172.21.15.113:6789/0 conn(0x7f8ca80cbac0 legacy=0x7f8ca815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.469+0000 7f8cad000640 1 -- 172.21.15.91:0/3362230424 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80ca380 con 0x7f8ca8159de0 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.470+0000 7f8cad000640 1 --2- 172.21.15.91:0/3362230424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca8159de0 0x7f8ca80cb580 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee6adb359936e83 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.470+0000 7f8ca7400640 1 -- 172.21.15.91:0/3362230424 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98031640 con 0x7f8ca8159de0 2024-02-02T13:32:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.470+0000 7f8ca7400640 1 -- 172.21.15.91:0/3362230424 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c980317a0 con 0x7f8ca8159de0 2024-02-02T13:32:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.470+0000 7f8ca7400640 1 -- 172.21.15.91:0/3362230424 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98031b30 con 0x7f8ca8159de0 2024-02-02T13:32:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 -- 172.21.15.91:0/3362230424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca8159de0 msgr2=0x7f8ca80cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 --2- 172.21.15.91:0/3362230424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca8159de0 0x7f8ca80cb580 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c9802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 -- 172.21.15.91:0/3362230424 shutdown_connections 2024-02-02T13:32:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 --2- 172.21.15.91:0/3362230424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca815e340 0x7f8ca806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 --2- 172.21.15.91:0/3362230424 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca8159de0 0x7f8ca80cb580 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 -- 172.21.15.91:0/3362230424 >> 172.21.15.91:0/3362230424 conn(0x7f8ca80b0700 msgr2=0x7f8ca80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 -- 172.21.15.91:0/3362230424 shutdown_connections 2024-02-02T13:32:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.471+0000 7f8cafa00640 1 -- 172.21.15.91:0/3362230424 wait complete. 2024-02-02T13:32:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.472+0000 7f8cafa00640 1 Processor -- start 2024-02-02T13:32:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.473+0000 7f8cafa00640 1 -- start start 2024-02-02T13:32:24.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.473+0000 7f8cafa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca80cbac0 0x7f8ca80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cafa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 0x7f8ca80c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cafa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cafa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8158de0 con 0x7f8ca815e340 2024-02-02T13:32:24.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cafa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80c8660 con 0x7f8ca8159de0 2024-02-02T13:32:24.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cafa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80c5140 con 0x7f8ca80cbac0 2024-02-02T13:32:24.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40756/0 (socket says 172.21.15.91:40756) 2024-02-02T13:32:24.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 -- 172.21.15.91:0/1092649645 learned_addr learned my addr 172.21.15.91:0/1092649645 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cad000640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca80cbac0 0x7f8ca80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8ca7e00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 0x7f8ca80c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 -- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca80cbac0 msgr2=0x7f8ca80c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca80cbac0 0x7f8ca80c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 -- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 msgr2=0x7f8ca80c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 0x7f8ca80c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8cada00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980094d0 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.474+0000 7f8ca7e00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 0x7f8ca80c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8cada00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b970 tx=0x7f8ca000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000cb40 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000ccc0 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca80c5420 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0014680 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.475+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca8156380 con 0x7f8ca815e340 2024-02-02T13:32:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.476+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca80c5660 con 0x7f8ca815e340 2024-02-02T13:32:24.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.477+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ca001e440 con 0x7f8ca815e340 2024-02-02T13:32:24.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.478+0000 7f8ca4c00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 0x7f8c8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.478+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ca0088950 con 0x7f8ca815e340 2024-02-02T13:32:24.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.478+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7f8c80071f60 con 0x7f8ca815e340 2024-02-02T13:32:24.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.479+0000 7f8cad000640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 0x7f8c8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.479+0000 7f8cad000640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 0x7f8c8006c8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8c98036000 tx=0x7f8c9803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.483+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ca0019070 con 0x7f8ca815e340 2024-02-02T13:32:24.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.678+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8ca8158f20 con 0x7f8ca815e340 2024-02-02T13:32:24.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.680+0000 7f8ca4c00640 1 -- 172.21.15.91:0/1092649645 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33521 (secure 0 0 0) 0x7f8ca8158f20 con 0x7f8ca815e340 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 145, 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:19.411226+0000", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:24.682 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:32:24.683 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:24.684 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:24.685 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:24.686 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:24.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:24.688 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:24.689 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:24.690 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:24.691 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:24.692 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:24.693 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:24.694 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.695 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:24.696 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.697 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "145", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:24.698 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:24.699 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:24.700 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:24.701 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.702 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.703 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.704 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:24.706 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:24.707 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.708 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.709 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.710 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.711 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.712 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.713 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.714 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:24.715 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.716 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.717 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.718 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.719 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:24.720 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:24.721 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:24.722 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:24.723 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.724 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.725 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.726 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.727 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.728 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.729 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.730 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.731 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.732 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:24.733 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.734 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:24.735 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:24.736 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:24.737 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.738 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:24.739 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.740 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.741 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:24.742 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:24.743 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:24.744 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.745 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.746 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:24.747 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.748 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:24.749 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:24.750 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:24.751 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:24.752 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 msgr2=0x7f8c8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 0x7f8c8006c8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8c98036000 tx=0x7f8c9803c040 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 msgr2=0x7f8ca80c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b970 tx=0x7f8ca000be30 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 shutdown_connections 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c8006a430 0x7f8c8006c8f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.682+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ca815e340 0x7f8ca80c8120 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.683+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca8159de0 0x7f8ca80c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.683+0000 7f8cafa00640 1 --2- 172.21.15.91:0/1092649645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca80cbac0 0x7f8ca80c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.683+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 >> 172.21.15.91:0/1092649645 conn(0x7f8ca80b0700 msgr2=0x7f8ca806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.683+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 shutdown_connections 2024-02-02T13:32:24.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.683+0000 7f8cafa00640 1 -- 172.21.15.91:0/1092649645 wait complete. 2024-02-02T13:32:24.755 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:24.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.967+0000 7f437d600640 1 Processor -- start 2024-02-02T13:32:24.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.967+0000 7f437d600640 1 -- start start 2024-02-02T13:32:24.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f437d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f437815d490 0x7f437806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f437d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f43780ac8d0 0x7f43780acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f437d600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437815d870 con 0x7f43780b0700 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f437d600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780ad270 con 0x7f43780ac8d0 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f437d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437806bce0 con 0x7f437815d490 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f436de00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f43780ac8d0 0x7f43780acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f4377400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f43780b0700 0x7f43780c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40520/0 (socket says 172.21.15.91:40520) 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f4377400640 1 -- 172.21.15.91:0/3252533349 learned_addr learned my addr 172.21.15.91:0/3252533349 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:24.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.969+0000 7f4376a00640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f437815d490 0x7f437806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.970+0000 7f436de00640 1 -- 172.21.15.91:0/3252533349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f437815d490 msgr2=0x7f437806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:24.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.970+0000 7f436de00640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f437815d490 0x7f437806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.970+0000 7f436de00640 1 -- 172.21.15.91:0/3252533349 >> v1:172.21.15.91:6789/0 conn(0x7f43780b0700 legacy=0x7f43780c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.970+0000 7f436de00640 1 -- 172.21.15.91:0/3252533349 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378161220 con 0x7f43780ac8d0 2024-02-02T13:32:24.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.970+0000 7f4376000640 1 -- 172.21.15.91:0/3252533349 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516060467 0 0) 0x7f437815d870 con 0x7f43780b0700 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f436de00640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780acd30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9070e70186daeaa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f4376000640 1 -- 172.21.15.91:0/3252533349 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360031830 con 0x7f43780ac8d0 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f4376000640 1 -- 172.21.15.91:0/3252533349 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360031990 con 0x7f43780ac8d0 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f4376000640 1 -- 172.21.15.91:0/3252533349 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360031d20 con 0x7f43780ac8d0 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f437d600640 1 -- 172.21.15.91:0/3252533349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 msgr2=0x7f43780acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.971+0000 7f437d600640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780acd30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 -- 172.21.15.91:0/3252533349 shutdown_connections 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780acd30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 --2- 172.21.15.91:0/3252533349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f437815d490 0x7f437806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 -- 172.21.15.91:0/3252533349 >> 172.21.15.91:0/3252533349 conn(0x7f43780cec80 msgr2=0x7f43780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 -- 172.21.15.91:0/3252533349 shutdown_connections 2024-02-02T13:32:24.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.972+0000 7f437d600640 1 -- 172.21.15.91:0/3252533349 wait complete. 2024-02-02T13:32:24.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.973+0000 7f437d600640 1 Processor -- start 2024-02-02T13:32:24.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 -- start start 2024-02-02T13:32:24.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43780b0700 0x7f43780b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f437815d490 0x7f437815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437815ef20 con 0x7f43780ac8d0 2024-02-02T13:32:24.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437815f200 con 0x7f437815d490 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.974+0000 7f437d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437815f4e0 con 0x7f43780b0700 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40778/0 (socket says 172.21.15.91:40778) 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 -- 172.21.15.91:0/3886036837 learned_addr learned my addr 172.21.15.91:0/3886036837 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f436de00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43780b0700 0x7f43780b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 -- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43780b0700 msgr2=0x7f43780b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43780b0700 0x7f43780b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 -- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f437815d490 msgr2=0x7f437815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f437815d490 0x7f437815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009400 con 0x7f43780ac8d0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.975+0000 7f4376a00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f436800d800 tx=0x7f436800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.976+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436800b6c0 con 0x7f43780ac8d0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.976+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800b820 con 0x7f43780ac8d0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.976+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437815f7c0 con 0x7f43780ac8d0 2024-02-02T13:32:24.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.976+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436800bbb0 con 0x7f43780ac8d0 2024-02-02T13:32:24.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.976+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780c3db0 con 0x7f43780ac8d0 2024-02-02T13:32:24.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.977+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378066cf0 con 0x7f43780ac8d0 2024-02-02T13:32:24.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.978+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f436800bd10 con 0x7f43780ac8d0 2024-02-02T13:32:24.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.979+0000 7f436f400640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 0x7f435006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:24.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.980+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4368088f00 con 0x7f43780ac8d0 2024-02-02T13:32:24.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.980+0000 7f436de00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 0x7f435006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:24.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.980+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7f4350071f60 con 0x7f43780ac8d0 2024-02-02T13:32:24.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.980+0000 7f436de00640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 0x7f435006c8f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4360036000 tx=0x7f436003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:24.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:24.984+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4368054450 con 0x7f43780ac8d0 2024-02-02T13:32:25.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.210+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43780c5a60 con 0x7f43780ac8d0 2024-02-02T13:32:25.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.211+0000 7f436f400640 1 -- 172.21.15.91:0/3886036837 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f4368057a70 con 0x7f43780ac8d0 2024-02-02T13:32:25.213 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:25.213 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:25.213 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 msgr2=0x7f435006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 0x7f435006c8f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4360036000 tx=0x7f436003c040 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 msgr2=0x7f43780c3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f436800d800 tx=0x7f436800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 shutdown_connections 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f435006a430 0x7f435006c8f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43780b0700 0x7f43780b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f437815d490 0x7f437815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 --2- 172.21.15.91:0/3886036837 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43780ac8d0 0x7f43780c3870 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 >> 172.21.15.91:0/3886036837 conn(0x7f43780cec80 msgr2=0x7f43780b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 shutdown_connections 2024-02-02T13:32:25.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.214+0000 7f437d600640 1 -- 172.21.15.91:0/3886036837 wait complete. 2024-02-02T13:32:25.233 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:32:25.458 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":7584,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.891683855,"mdsmap_epoch":64,"osdmap_epoch":145,"osdmap_epoch_barrier":145,"uptime":30.229614181999999} 2024-02-02T13:32:25.458 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 7584, "mdsmap_epoch": 64, "osdmap_epoch": 145, "osdmap_epoch_barrier": 145, "rank_uptime": 24.891683855, "state": "up:active", "uptime": 30.229614182, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:32:25.458 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:32:25.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:25.458 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-02-02T13:32:25.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:25.511 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-02-02T13:32:25.560 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:25.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.749+0000 7f2099400640 1 Processor -- start 2024-02-02T13:32:25.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.750+0000 7f2099400640 1 -- start start 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2099400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 0x7f20940ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2099400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209415d3a0 0x7f20940cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2099400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940c94a0 con 0x7f20940ac960 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2099400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940ad500 con 0x7f209415d3a0 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2099400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940cbea0 con 0x7f20940cb580 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.751+0000 7f2089c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209415d3a0 0x7f20940cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209415d3a0 0x7f20940cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40788/0 (socket says 172.21.15.91:40788) 2024-02-02T13:32:25.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 -- 172.21.15.91:0/3989419136 learned_addr learned my addr 172.21.15.91:0/3989419136 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2092800640 1 --2- 172.21.15.91:0/3989419136 >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 0x7f20940ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 -- 172.21.15.91:0/3989419136 >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 msgr2=0x7f20940ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 --2- 172.21.15.91:0/3989419136 >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 0x7f20940ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 -- 172.21.15.91:0/3989419136 >> v1:172.21.15.113:6789/0 conn(0x7f20940ac960 legacy=0x7f20940acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2089c00640 1 -- 172.21.15.91:0/3989419136 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940ad6e0 con 0x7f209415d3a0 2024-02-02T13:32:25.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.752+0000 7f2092800640 1 --2- 172.21.15.91:0/3989419136 >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 0x7f20940ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:25.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.753+0000 7f2089c00640 1 --2- 172.21.15.91:0/3989419136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209415d3a0 0x7f20940cb960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f20940cc320 tx=0x7f207c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329b4afb98431668 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:25.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.753+0000 7f2091e00640 1 -- 172.21.15.91:0/3989419136 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c031870 con 0x7f209415d3a0 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.753+0000 7f2091e00640 1 -- 172.21.15.91:0/3989419136 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0319d0 con 0x7f209415d3a0 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.753+0000 7f2091e00640 1 -- 172.21.15.91:0/3989419136 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c031d60 con 0x7f209415d3a0 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 -- 172.21.15.91:0/3989419136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209415d3a0 msgr2=0x7f20940cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 --2- 172.21.15.91:0/3989419136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209415d3a0 0x7f20940cb960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f20940cc320 tx=0x7f207c02b910 comp rx=0 tx=0).stop 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 -- 172.21.15.91:0/3989419136 shutdown_connections 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 --2- 172.21.15.91:0/3989419136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209415d3a0 0x7f20940cb960 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 --2- 172.21.15.91:0/3989419136 >> v2:172.21.15.113:3300/0 conn(0x7f20940cb580 0x7f20940ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 -- 172.21.15.91:0/3989419136 >> 172.21.15.91:0/3989419136 conn(0x7f20940b0700 msgr2=0x7f20940b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 -- 172.21.15.91:0/3989419136 shutdown_connections 2024-02-02T13:32:25.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.754+0000 7f2099400640 1 -- 172.21.15.91:0/3989419136 wait complete. 2024-02-02T13:32:25.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.756+0000 7f2099400640 1 Processor -- start 2024-02-02T13:32:25.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.756+0000 7f2099400640 1 -- start start 2024-02-02T13:32:25.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20940ac960 0x7f20940b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 0x7f209415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209414f650 con 0x7f20940cb580 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209415bb90 con 0x7f209415d3a0 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2099400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209415d960 con 0x7f20940ac960 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2089c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2089c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40794/0 (socket says 172.21.15.91:40794) 2024-02-02T13:32:25.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2089c00640 1 -- 172.21.15.91:0/2875639152 learned_addr learned my addr 172.21.15.91:0/2875639152 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:25.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.757+0000 7f2093200640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 0x7f209415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2092800640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20940ac960 0x7f20940b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 -- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20940ac960 msgr2=0x7f20940b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20940ac960 0x7f20940b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 -- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 msgr2=0x7f209415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 0x7f209415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0094d0 con 0x7f20940cb580 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2093200640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 0x7f209415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2089c00640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f207c0090b0 tx=0x7f207c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c031820 con 0x7f20940cb580 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0443f0 con 0x7f20940cb580 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0447a0 con 0x7f20940cb580 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209415b2f0 con 0x7f20940cb580 2024-02-02T13:32:25.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.758+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209415b8b0 con 0x7f20940cb580 2024-02-02T13:32:25.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.760+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209415d7c0 con 0x7f20940cb580 2024-02-02T13:32:25.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.761+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f207c044900 con 0x7f20940cb580 2024-02-02T13:32:25.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.762+0000 7f208b400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 0x7f206c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:25.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.762+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f207c02d040 con 0x7f20940cb580 2024-02-02T13:32:25.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.762+0000 7f2092800640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 0x7f206c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:25.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.762+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7f206c071f90 con 0x7f20940cb580 2024-02-02T13:32:25.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.763+0000 7f2092800640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 0x7f206c06c920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2094158b80 tx=0x7f2084008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:25.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.766+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f209415d7c0 con 0x7f20940cb580 2024-02-02T13:32:25.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.960+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f209415d7c0 con 0x7f20940cb580 2024-02-02T13:32:25.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.964+0000 7f208b400640 1 -- 172.21.15.91:0/2875639152 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33521 (secure 0 0 0) 0x7f209415d7c0 con 0x7f20940cb580 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 145, 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:19.411226+0000", 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:25.966 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:25.967 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:25.968 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:25.969 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:25.970 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:25.971 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:25.972 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.973 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:25.974 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.975 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:25.976 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:25.977 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.979 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:25.980 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:25.981 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "145", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:25.982 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:25.983 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:25.984 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.985 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:25.986 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.987 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.988 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:25.989 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:25.990 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:25.991 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:25.992 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.993 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.994 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:25.995 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:25.996 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.997 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.998 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:25.999 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.000 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:26.001 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.002 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.003 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.004 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.005 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:26.006 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.007 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.008 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.009 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.010 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.011 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.012 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:26.013 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.014 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.015 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.016 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.017 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.018 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:26.020 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:26.021 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.022 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.023 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.024 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.025 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.026 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.027 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.028 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.029 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:26.030 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:26.031 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:26.032 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:26.033 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.966+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 msgr2=0x7f206c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.966+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 0x7f206c06c920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2094158b80 tx=0x7f2084008040 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.966+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 msgr2=0x7f209414f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.966+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f207c0090b0 tx=0x7f207c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.966+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 shutdown_connections 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206c06a460 0x7f206c06c920 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209415d3a0 0x7f209415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20940cb580 0x7f209414f110 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 --2- 172.21.15.91:0/2875639152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20940ac960 0x7f20940b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:26.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 >> 172.21.15.91:0/2875639152 conn(0x7f20940b0700 msgr2=0x7f20940af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:26.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 shutdown_connections 2024-02-02T13:32:26.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:25.967+0000 7f2099400640 1 -- 172.21.15.91:0/2875639152 wait complete. 2024-02-02T13:32:26.036 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:32:31.037 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:31.146 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:32:31.151 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:32:31.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.255+0000 7f080dc00640 1 Processor -- start 2024-02-02T13:32:31.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.255+0000 7f080dc00640 1 -- start start 2024-02-02T13:32:31.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0808159de0 0x7f08080cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08080cbac0 0x7f080815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 0x7f08080cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080cbf50 con 0x7f0808159de0 2024-02-02T13:32:31.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080ca0c0 con 0x7f08080cbac0 2024-02-02T13:32:31.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f080dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080ca380 con 0x7f080815e170 2024-02-02T13:32:31.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0808159de0 0x7f08080cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0808159de0 0x7f08080cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33576/0 (socket says 172.21.15.91:33576) 2024-02-02T13:32:31.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 -- 172.21.15.91:0/241191297 learned_addr learned my addr 172.21.15.91:0/241191297 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f07fde00640 1 --2- 172.21.15.91:0/241191297 >> v2:172.21.15.113:3300/0 conn(0x7f08080cbac0 0x7f080815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0807400640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 0x7f08080cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 -- 172.21.15.91:0/241191297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 msgr2=0x7f08080cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 0x7f08080cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 -- 172.21.15.91:0/241191297 >> v2:172.21.15.113:3300/0 conn(0x7f08080cbac0 msgr2=0x7f080815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.257+0000 7f0806a00640 1 --2- 172.21.15.91:0/241191297 >> v2:172.21.15.113:3300/0 conn(0x7f08080cbac0 0x7f080815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.258+0000 7f0806a00640 1 -- 172.21.15.91:0/241191297 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080ca660 con 0x7f0808159de0 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.258+0000 7f0807400640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 0x7f08080cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:31.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.259+0000 7f0806a00640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f08080cb580 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f08080b1760 tx=0x7f07f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e99dc40b453c96 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.259+0000 7f0806000640 1 -- 172.21.15.91:0/241191297 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031870 con 0x7f0808159de0 2024-02-02T13:32:31.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.259+0000 7f0806000640 1 -- 172.21.15.91:0/241191297 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f80319d0 con 0x7f0808159de0 2024-02-02T13:32:31.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.259+0000 7f0806000640 1 -- 172.21.15.91:0/241191297 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031d60 con 0x7f0808159de0 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.259+0000 7f080dc00640 1 -- 172.21.15.91:0/241191297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 msgr2=0x7f08080cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f08080cb580 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f08080b1760 tx=0x7f07f802b910 comp rx=0 tx=0).stop 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 -- 172.21.15.91:0/241191297 shutdown_connections 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f080815e170 0x7f08080cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 --2- 172.21.15.91:0/241191297 >> v2:172.21.15.113:3300/0 conn(0x7f08080cbac0 0x7f080815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 --2- 172.21.15.91:0/241191297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f08080cb580 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 -- 172.21.15.91:0/241191297 >> 172.21.15.91:0/241191297 conn(0x7f08080b0700 msgr2=0x7f08080b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 -- 172.21.15.91:0/241191297 shutdown_connections 2024-02-02T13:32:31.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.260+0000 7f080dc00640 1 -- 172.21.15.91:0/241191297 wait complete. 2024-02-02T13:32:31.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.261+0000 7f080dc00640 1 Processor -- start 2024-02-02T13:32:31.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.262+0000 7f080dc00640 1 -- start start 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.262+0000 7f080dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08080cbac0 0x7f08080b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.262+0000 7f080dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f080dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f080815e170 0x7f08081e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f080dc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080b41e0 con 0x7f0808159de0 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f080dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080814cc50 con 0x7f080815e170 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f080dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0808156fb0 con 0x7f08080cbac0 2024-02-02T13:32:31.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33578/0 (socket says 172.21.15.91:33578) 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 -- 172.21.15.91:0/4037000942 learned_addr learned my addr 172.21.15.91:0/4037000942 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 -- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08080cbac0 msgr2=0x7f08080b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f0806a00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08080cbac0 0x7f08080b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08080cbac0 0x7f08080b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f0807400640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f080815e170 0x7f08081e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 -- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f080815e170 msgr2=0x7f08081e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f080815e170 0x7f08081e7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.263+0000 7f07fde00640 1 -- 172.21.15.91:0/4037000942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80094d0 con 0x7f0808159de0 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f07fde00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f07f000d800 tx=0x7f07f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00096a0 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0009800 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0009990 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080814cea0 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.264+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080814d460 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.265+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0808066cf0 con 0x7f0808159de0 2024-02-02T13:32:31.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.266+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f07f0009af0 con 0x7f0808159de0 2024-02-02T13:32:31.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.267+0000 7f07ff400640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 0x7f07e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.267+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f07f0088ed0 con 0x7f0808159de0 2024-02-02T13:32:31.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.267+0000 7f0806a00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 0x7f07e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.268+0000 7f0806a00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 0x7f07e006c8f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f07f8036000 tx=0x7f07f803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.272+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f07f00543e0 con 0x7f0808159de0 2024-02-02T13:32:31.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.463+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f08080caef0 con 0x7f0808159de0 2024-02-02T13:32:31.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.465+0000 7f07ff400640 1 -- 172.21.15.91:0/4037000942 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v148) v1 ==== 81+0+33934 (secure 0 0 0) 0x7f07f0057a00 con 0x7f0808159de0 2024-02-02T13:32:31.466 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:31.466 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:31.466 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 148, 2024-02-02T13:32:31.466 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:29.417200+0000", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:31.467 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:31.468 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:31.469 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.470 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:31.471 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:31.472 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:31.473 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:31.474 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:31.475 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:31.476 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:31.477 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:31.478 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:31.479 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.480 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:31.481 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:31.482 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "148", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:31.483 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:31.484 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:31.485 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.6299999952316284, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.6299999952316284, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.6299999952316284, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.6299999952316284, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:31.486 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.487 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.488 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.489 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.490 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:31.491 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.492 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.493 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.494 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.495 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.496 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.497 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.498 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.499 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.500 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.501 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.502 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.503 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.504 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.505 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:31.506 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.507 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:31.508 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:31.509 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.510 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.511 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.512 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.513 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 146, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.514 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.515 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.516 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.517 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:31.518 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.519 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 146, 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.520 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.521 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.522 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:31.523 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:31.524 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.525 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.526 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.527 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:31.528 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:31.529 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.530 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.531 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.532 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:31.534 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:31.535 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "17.5", 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 0, 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 2 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "17.3c", 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 0, 2024-02-02T13:32:31.536 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 5 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:31.537 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:31.538 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:31.539 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:31.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 msgr2=0x7f07e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 0x7f07e006c8f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f07f8036000 tx=0x7f07f803c040 comp rx=0 tx=0).stop 2024-02-02T13:32:31.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 msgr2=0x7f080814d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f07f000d800 tx=0x7f07f000dcc0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 shutdown_connections 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f07e006a430 0x7f07e006c8f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f080815e170 0x7f08081e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0808159de0 0x7f080814d980 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 --2- 172.21.15.91:0/4037000942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08080cbac0 0x7f08080b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 >> 172.21.15.91:0/4037000942 conn(0x7f08080b0700 msgr2=0x7f08080b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 shutdown_connections 2024-02-02T13:32:31.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.468+0000 7f080dc00640 1 -- 172.21.15.91:0/4037000942 wait complete. 2024-02-02T13:32:31.541 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-02-02T13:32:31.542 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:32:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.785+0000 7f19a9400640 1 Processor -- start 2024-02-02T13:32:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.785+0000 7f19a9400640 1 -- start start 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a9400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19a415ad10 0x7f19a406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19a415b0f0 0x7f19a406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a9400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a406d910 con 0x7f19a40cb580 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a9400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a40cd610 con 0x7f19a415ad10 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a9400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a40cc850 con 0x7f19a415b0f0 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19a415ad10 0x7f19a406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a3200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f19a40cb580 0x7f19a406f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40458/0 (socket says 172.21.15.91:40458) 2024-02-02T13:32:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.787+0000 7f19a3200640 1 -- 172.21.15.91:0/4266040684 learned_addr learned my addr 172.21.15.91:0/4266040684 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f199be00640 1 --2- 172.21.15.91:0/4266040684 >> v2:172.21.15.113:3300/0 conn(0x7f19a415b0f0 0x7f19a406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f19a2800640 1 -- 172.21.15.91:0/4266040684 >> v2:172.21.15.113:3300/0 conn(0x7f19a415b0f0 msgr2=0x7f19a406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f19a2800640 1 --2- 172.21.15.91:0/4266040684 >> v2:172.21.15.113:3300/0 conn(0x7f19a415b0f0 0x7f19a406d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f19a2800640 1 -- 172.21.15.91:0/4266040684 >> v1:172.21.15.91:6789/0 conn(0x7f19a40cb580 legacy=0x7f19a406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f19a2800640 1 -- 172.21.15.91:0/4266040684 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40ccb30 con 0x7f19a415ad10 2024-02-02T13:32:31.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.788+0000 7f19a1e00640 1 -- 172.21.15.91:0/4266040684 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99401008 0 0) 0x7f19a406d910 con 0x7f19a40cb580 2024-02-02T13:32:31.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.789+0000 7f19a2800640 1 --2- 172.21.15.91:0/4266040684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a415ad10 0x7f19a406b7a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f19940097a0 tx=0x7f199402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d97d3307f0a659 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.790+0000 7f19a1e00640 1 -- 172.21.15.91:0/4266040684 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199402e070 con 0x7f19a415ad10 2024-02-02T13:32:31.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.790+0000 7f19a1e00640 1 -- 172.21.15.91:0/4266040684 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19940315a0 con 0x7f19a415ad10 2024-02-02T13:32:31.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.790+0000 7f19a1e00640 1 -- 172.21.15.91:0/4266040684 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994031930 con 0x7f19a415ad10 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.790+0000 7f19a9400640 1 -- 172.21.15.91:0/4266040684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a415ad10 msgr2=0x7f19a406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.790+0000 7f19a9400640 1 --2- 172.21.15.91:0/4266040684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a415ad10 0x7f19a406b7a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f19940097a0 tx=0x7f199402b840 comp rx=0 tx=0).stop 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 -- 172.21.15.91:0/4266040684 shutdown_connections 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 --2- 172.21.15.91:0/4266040684 >> v2:172.21.15.113:3300/0 conn(0x7f19a415b0f0 0x7f19a406d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 --2- 172.21.15.91:0/4266040684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a415ad10 0x7f19a406b7a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 -- 172.21.15.91:0/4266040684 >> 172.21.15.91:0/4266040684 conn(0x7f19a40b0700 msgr2=0x7f19a40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:31.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 -- 172.21.15.91:0/4266040684 shutdown_connections 2024-02-02T13:32:31.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.791+0000 7f19a9400640 1 -- 172.21.15.91:0/4266040684 wait complete. 2024-02-02T13:32:31.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.792+0000 7f19a9400640 1 Processor -- start 2024-02-02T13:32:31.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 -- start start 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 0x7f19a41f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19a415b0f0 0x7f19a41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a40c63f0 con 0x7f19a40cb580 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a40caa90 con 0x7f19a415b0f0 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.793+0000 7f19a9400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4155950 con 0x7f19a415ad10 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33602/0 (socket says 172.21.15.91:33602) 2024-02-02T13:32:31.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 -- 172.21.15.91:0/2666536204 learned_addr learned my addr 172.21.15.91:0/2666536204 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a3200640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19a415b0f0 0x7f19a41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f199be00640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 0x7f19a41f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 -- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 msgr2=0x7f19a41f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 0x7f19a41f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 -- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19a415b0f0 msgr2=0x7f19a41f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19a415b0f0 0x7f19a41f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:31.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f19a2800640 1 -- 172.21.15.91:0/2666536204 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994009400 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.794+0000 7f199be00640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 0x7f19a41f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f19a2800640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1994034040 tx=0x7f19940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994032900 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994032a80 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a4155c30 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994032e30 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.795+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a41561f0 con 0x7f19a40cb580 2024-02-02T13:32:31.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.796+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a4066cf0 con 0x7f19a40cb580 2024-02-02T13:32:31.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.797+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f199404e430 con 0x7f19a40cb580 2024-02-02T13:32:31.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.798+0000 7f199aa00640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 0x7f197c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:31.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.798+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f19940aa9b0 con 0x7f19a40cb580 2024-02-02T13:32:31.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.798+0000 7f199be00640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 0x7f197c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:31.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.799+0000 7f199be00640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 0x7f197c06c8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f198c006fd0 tx=0x7f198c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:31.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.802+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1994075f50 con 0x7f19a40cb580 2024-02-02T13:32:31.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.991+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f19a41f26a0 con 0x7f19a40cb580 2024-02-02T13:32:31.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.993+0000 7f199aa00640 1 -- 172.21.15.91:0/2666536204 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v148) v1 ==== 81+0+33934 (secure 0 0 0) 0x7f1994079570 con 0x7f19a40cb580 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 148, 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:32:29.417200+0000", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:32:31.996 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 17, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "luminous", 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:32:31.997 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:31.998 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:31.999 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:32.000 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.001 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:32:32.002 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:32.003 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 16, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:58.576657+0000", 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:32.004 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "142", 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:32.005 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:32.006 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:32.007 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.008 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.75, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.75, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.75, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.75, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.009 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 17, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:31:59.552886+0000", 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:32:32.010 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "148", 2024-02-02T13:32:32.011 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:32:32.012 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:32:32.013 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:32:32.014 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.6299999952316284, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.6299999952316284, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.6299999952316284, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.6299999952316284, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.015 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.016 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.017 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.018 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:32:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.021 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.022 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.023 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.024 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.025 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:32:32.026 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.027 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.028 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.029 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.030 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:32:32.031 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.032 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.033 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.034 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.035 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.036 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:32:32.037 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.038 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:32:32.039 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:32:32.040 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.041 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.042 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.043 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 146, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:32:32.044 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.045 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:32:32.046 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.047 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.048 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.049 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 146, 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.050 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.051 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.052 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:32:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:32:32.054 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 141, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.055 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:32:32.056 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:32:32.057 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.058 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.059 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.060 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.061 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:32:32.062 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:32:32.063 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:32:32.064 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [ 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "17.5", 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 0, 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 2 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.065 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "pgid": "17.3c", 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "mappings": [ 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "from": 0, 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "to": 5 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:32:32.066 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:32:32.067 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:32:32.068 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:32:32.068 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:32:32.068 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:32.068 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.996+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 msgr2=0x7f197c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.996+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 0x7f197c06c8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f198c006fd0 tx=0x7f198c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.996+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 msgr2=0x7f19a40c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.996+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1994034040 tx=0x7f19940046c0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 shutdown_connections 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197c06a430 0x7f197c06c8f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19a415ad10 0x7f19a41f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19a415b0f0 0x7f19a41f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 --2- 172.21.15.91:0/2666536204 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a40cb580 0x7f19a40c5eb0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 >> 172.21.15.91:0/2666536204 conn(0x7f19a40b0700 msgr2=0x7f19a4158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 shutdown_connections 2024-02-02T13:32:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:31.997+0000 7f19a9400640 1 -- 172.21.15.91:0/2666536204 wait complete. 2024-02-02T13:32:32.071 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:32.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.282+0000 7f3cde800640 1 Processor -- start 2024-02-02T13:32:32.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.282+0000 7f3cde800640 1 -- start start 2024-02-02T13:32:32.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cde800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3cd815d490 0x7f3cd806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cde800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3cd80b0e10 0x7f3cd80acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cde800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd815d870 con 0x7f3cd80ac8d0 2024-02-02T13:32:32.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cde800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd80ad2a0 con 0x7f3cd815d490 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cde800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd806bce0 con 0x7f3cd80b0e10 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3cd815d490 0x7f3cd806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3cd815d490 0x7f3cd806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33618/0 (socket says 172.21.15.91:33618) 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 -- 172.21.15.91:0/3927562517 learned_addr learned my addr 172.21.15.91:0/3927562517 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7e00640 1 --2- 172.21.15.91:0/3927562517 >> v2:172.21.15.113:3300/0 conn(0x7f3cd80b0e10 0x7f3cd80acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 -- 172.21.15.91:0/3927562517 >> v2:172.21.15.113:3300/0 conn(0x7f3cd80b0e10 msgr2=0x7f3cd80acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:32.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 --2- 172.21.15.91:0/3927562517 >> v2:172.21.15.113:3300/0 conn(0x7f3cd80b0e10 0x7f3cd80acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.284+0000 7f3cd7400640 1 -- 172.21.15.91:0/3927562517 >> v1:172.21.15.113:6789/0 conn(0x7f3cd80ac8d0 legacy=0x7f3cd80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:32.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.285+0000 7f3cd7400640 1 -- 172.21.15.91:0/3927562517 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8161220 con 0x7f3cd815d490 2024-02-02T13:32:32.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.286+0000 7f3cd7400640 1 --2- 172.21.15.91:0/3927562517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd815d490 0x7f3cd806c170 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b4bb0 tx=0x7f3cc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2d58988f7516e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:32.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.286+0000 7f3cd6a00640 1 -- 172.21.15.91:0/3927562517 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8031870 con 0x7f3cd815d490 2024-02-02T13:32:32.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.286+0000 7f3cd6a00640 1 -- 172.21.15.91:0/3927562517 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc80319d0 con 0x7f3cd815d490 2024-02-02T13:32:32.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.286+0000 7f3cd6a00640 1 -- 172.21.15.91:0/3927562517 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8031d60 con 0x7f3cd815d490 2024-02-02T13:32:32.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 -- 172.21.15.91:0/3927562517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd815d490 msgr2=0x7f3cd806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 --2- 172.21.15.91:0/3927562517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd815d490 0x7f3cd806c170 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b4bb0 tx=0x7f3cc802b910 comp rx=0 tx=0).stop 2024-02-02T13:32:32.288 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 -- 172.21.15.91:0/3927562517 shutdown_connections 2024-02-02T13:32:32.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 --2- 172.21.15.91:0/3927562517 >> v2:172.21.15.113:3300/0 conn(0x7f3cd80b0e10 0x7f3cd80acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 --2- 172.21.15.91:0/3927562517 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd815d490 0x7f3cd806c170 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 -- 172.21.15.91:0/3927562517 >> 172.21.15.91:0/3927562517 conn(0x7f3cd80cec80 msgr2=0x7f3cd80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:32.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 -- 172.21.15.91:0/3927562517 shutdown_connections 2024-02-02T13:32:32.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.287+0000 7f3cde800640 1 -- 172.21.15.91:0/3927562517 wait complete. 2024-02-02T13:32:32.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.288+0000 7f3cde800640 1 Processor -- start 2024-02-02T13:32:32.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.289+0000 7f3cde800640 1 -- start start 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.289+0000 7f3cde800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd80ac8d0 0x7f3cd814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.289+0000 7f3cde800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cde800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd815d490 0x7f3cd815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cde800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd815de80 con 0x7f3cd80b0e10 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cde800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd815f840 con 0x7f3cd80ac8d0 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cde800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd815fb20 con 0x7f3cd815d490 2024-02-02T13:32:32.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33622/0 (socket says 172.21.15.91:33622) 2024-02-02T13:32:32.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 -- 172.21.15.91:0/2263615474 learned_addr learned my addr 172.21.15.91:0/2263615474 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cd7400640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd80ac8d0 0x7f3cd814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 -- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd815d490 msgr2=0x7f3cd815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3cd7e00640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd815d490 0x7f3cd815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd815d490 0x7f3cd815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 -- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd80ac8d0 msgr2=0x7f3cd814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd80ac8d0 0x7f3cd814e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.290+0000 7f3ccfe00640 1 -- 172.21.15.91:0/2263615474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80094d0 con 0x7f3cd80b0e10 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3ccfe00640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000b3f0 tx=0x7f3cc000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011830 con 0x7f3cd80b0e10 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0011990 con 0x7f3cd80b0e10 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011d40 con 0x7f3cd80b0e10 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd815fe00 con 0x7f3cd80b0e10 2024-02-02T13:32:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.291+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd81607c0 con 0x7f3cd80b0e10 2024-02-02T13:32:32.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.292+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd806b670 con 0x7f3cd80b0e10 2024-02-02T13:32:32.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.293+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3cc001a400 con 0x7f3cd80b0e10 2024-02-02T13:32:32.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.294+0000 7f3ccf400640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 0x7f3cb006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.294+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3cc0088710 con 0x7f3cd80b0e10 2024-02-02T13:32:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.294+0000 7f3cd7400640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 0x7f3cb006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.295+0000 7f3cd7400640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 0x7f3cb006c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8036000 tx=0x7f3cc803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:32.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.298+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3cd806b670 con 0x7f3cd80b0e10 2024-02-02T13:32:32.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.527+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd815dfc0 con 0x7f3cd80b0e10 2024-02-02T13:32:32.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.528+0000 7f3ccf400640 1 -- 172.21.15.91:0/2263615474 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f3cd815dfc0 con 0x7f3cd80b0e10 2024-02-02T13:32:32.530 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:32.530 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:32.530 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:32.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 msgr2=0x7f3cb006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 0x7f3cb006c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8036000 tx=0x7f3cc803c040 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 msgr2=0x7f3cd8160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000b3f0 tx=0x7f3cc000b8b0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 shutdown_connections 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3cb006a430 0x7f3cb006c8f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd815d490 0x7f3cd815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cd80b0e10 0x7f3cd8160280 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 --2- 172.21.15.91:0/2263615474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd80ac8d0 0x7f3cd814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 >> 172.21.15.91:0/2263615474 conn(0x7f3cd80cec80 msgr2=0x7f3cd8157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 shutdown_connections 2024-02-02T13:32:32.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:32.531+0000 7f3cde800640 1 -- 172.21.15.91:0/2263615474 wait complete. 2024-02-02T13:32:32.552 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:32:32.764 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":7584,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.198765999000003,"mdsmap_epoch":64,"osdmap_epoch":148,"osdmap_epoch_barrier":145,"uptime":37.536696120999999} 2024-02-02T13:32:32.765 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 7584, "mdsmap_epoch": 64, "osdmap_epoch": 148, "osdmap_epoch_barrier": 145, "rank_uptime": 32.198765999, "state": "up:active", "uptime": 37.536696121, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:32:32.765 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:32:32.765 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:32.765 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:32.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:32.816 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-02-02T13:32:33.255 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:32:33.256 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:32:33.256 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.383224 s, 137 MB/s 2024-02-02T13:32:33.257 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:32:33.257 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:32:33.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:32:33.471 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:32:33.471 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:33.471 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:33.528 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:33.528 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:33.580 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:33.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.787+0000 7f4d4e800640 1 Processor -- start 2024-02-02T13:32:33.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.788+0000 7f4d4e800640 1 -- start start 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d480c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 0x7f4d480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4d480cd470 0x7f4d480b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d480cc7c0 con 0x7f4d480cd470 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d480ce930 con 0x7f4d480ce130 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d4e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4806c890 con 0x7f4d4815a010 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d47e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4d480cd470 0x7f4d480b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d47e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4d480cd470 0x7f4d480b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33624/0 (socket says 172.21.15.91:33624) 2024-02-02T13:32:33.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.789+0000 7f4d47e00640 1 -- 172.21.15.91:0/1402841630 learned_addr learned my addr 172.21.15.91:0/1402841630 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:33.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47e00640 1 -- 172.21.15.91:0/1402841630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 msgr2=0x7f4d480c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:33.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47400640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d480c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d3e800640 1 --2- 172.21.15.91:0/1402841630 >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 0x7f4d480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47e00640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d480c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47e00640 1 -- 172.21.15.91:0/1402841630 >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 msgr2=0x7f4d480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47e00640 1 --2- 172.21.15.91:0/1402841630 >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 0x7f4d480ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47e00640 1 -- 172.21.15.91:0/1402841630 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4806cb70 con 0x7f4d480cd470 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d47400640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d480c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.790+0000 7f4d3e800640 1 --2- 172.21.15.91:0/1402841630 >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 0x7f4d480ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:33.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.791+0000 7f4d47e00640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480cd470 0x7f4d480b0700 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4d4806d7d0 tx=0x7f4d4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd03d950db54b79c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.791+0000 7f4d46a00640 1 -- 172.21.15.91:0/1402841630 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4002ca80 con 0x7f4d480cd470 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.791+0000 7f4d46a00640 1 -- 172.21.15.91:0/1402841630 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4002cbe0 con 0x7f4d480cd470 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.791+0000 7f4d46a00640 1 -- 172.21.15.91:0/1402841630 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400343f0 con 0x7f4d480cd470 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 -- 172.21.15.91:0/1402841630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480cd470 msgr2=0x7f4d480b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480cd470 0x7f4d480b0700 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4d4806d7d0 tx=0x7f4d4002bee0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 -- 172.21.15.91:0/1402841630 shutdown_connections 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480cd470 0x7f4d480b0700 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 --2- 172.21.15.91:0/1402841630 >> v2:172.21.15.113:3300/0 conn(0x7f4d480ce130 0x7f4d480ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 --2- 172.21.15.91:0/1402841630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d480c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 -- 172.21.15.91:0/1402841630 >> 172.21.15.91:0/1402841630 conn(0x7f4d480cec80 msgr2=0x7f4d480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:33.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 -- 172.21.15.91:0/1402841630 shutdown_connections 2024-02-02T13:32:33.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.792+0000 7f4d4e800640 1 -- 172.21.15.91:0/1402841630 wait complete. 2024-02-02T13:32:33.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.794+0000 7f4d4e800640 1 Processor -- start 2024-02-02T13:32:33.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.794+0000 7f4d4e800640 1 -- start start 2024-02-02T13:32:33.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d48150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 0x7f4d480b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480c54b0 con 0x7f4d480ce130 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480c59e0 con 0x7f4d48152ca0 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d4e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480c5700 con 0x7f4d4815a010 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d47400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d47400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33638/0 (socket says 172.21.15.91:33638) 2024-02-02T13:32:33.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d47400640 1 -- 172.21.15.91:0/4051019 learned_addr learned my addr 172.21.15.91:0/4051019 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:33.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d3e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d48150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.795+0000 7f4d47400640 1 -- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 msgr2=0x7f4d48150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47400640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d48150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47400640 1 -- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 msgr2=0x7f4d480b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47e00640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 0x7f4d480b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47400640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 0x7f4d480b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47400640 1 -- 172.21.15.91:0/4051019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40009b30 con 0x7f4d480ce130 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47e00640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 0x7f4d480b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d47400640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c820 tx=0x7f4d3800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38007a30 con 0x7f4d480ce130 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d38007b90 con 0x7f4d480ce130 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800ce10 con 0x7f4d480ce130 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480b23b0 con 0x7f4d480ce130 2024-02-02T13:32:33.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.796+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480b2970 con 0x7f4d480ce130 2024-02-02T13:32:33.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.798+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4806f990 con 0x7f4d480ce130 2024-02-02T13:32:33.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.799+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4d380113f0 con 0x7f4d480ce130 2024-02-02T13:32:33.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.799+0000 7f4d3fe00640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 0x7f4d2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:33.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.800+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4d38088a40 con 0x7f4d480ce130 2024-02-02T13:32:33.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.800+0000 7f4d3e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 0x7f4d2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:33.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.800+0000 7f4d3e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 0x7f4d2006c8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4d480aeba0 tx=0x7f4d30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:33.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.804+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4d38053fe0 con 0x7f4d480ce130 2024-02-02T13:32:33.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.992+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d48066cf0 con 0x7f4d480ce130 2024-02-02T13:32:33.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.995+0000 7f4d3fe00640 1 -- 172.21.15.91:0/4051019 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16220 (secure 0 0 0) 0x7f4d38057600 con 0x7f4d480ce130 2024-02-02T13:32:33.996 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:33.996 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":148,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:29.417200+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 msgr2=0x7f4d2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 0x7f4d2006c8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4d480aeba0 tx=0x7f4d30008040 comp rx=0 tx=0).stop 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 msgr2=0x7f4d480b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c820 tx=0x7f4d3800cce0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 shutdown_connections 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4d2006a430 0x7f4d2006c8f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4815a010 0x7f4d48150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d48152ca0 0x7f4d480b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 --2- 172.21.15.91:0/4051019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d480ce130 0x7f4d480b2e90 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.997+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 >> 172.21.15.91:0/4051019 conn(0x7f4d480cec80 msgr2=0x7f4d48159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.998+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 shutdown_connections 2024-02-02T13:32:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:33.998+0000 7f4d4e800640 1 -- 172.21.15.91:0/4051019 wait complete. 2024-02-02T13:32:34.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:58.576657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:59.552886+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:32:34.017 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:32:34.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.219+0000 7f1ab4400640 1 Processor -- start 2024-02-02T13:32:34.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.219+0000 7f1ab4400640 1 -- start start 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.220+0000 7f1ab4400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 0x7f1aac06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.220+0000 7f1ab4400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1aac0cd840 0x7f1aac0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.220+0000 7f1ab4400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac15b470 con 0x7f1aac0b28b0 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.220+0000 7f1ab4400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac0ce1f0 con 0x7f1aac0cd840 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.220+0000 7f1ab4400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0b2d10 con 0x7f1aac0b62f0 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1ab1000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1aac0b28b0 0x7f1aac15bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40462/0 (socket says 172.21.15.91:40462) 2024-02-02T13:32:34.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1ab1000640 1 -- 172.21.15.91:0/2835755591 learned_addr learned my addr 172.21.15.91:0/2835755591 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:34.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 0x7f1aac06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1ab2400640 1 --2- 172.21.15.91:0/2835755591 >> v2:172.21.15.113:3300/0 conn(0x7f1aac0cd840 0x7f1aac0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327092207 0 0) 0x7f1aac15b470 con 0x7f1aac0b28b0 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.221+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a8c003520 con 0x7f1aac0b28b0 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207075171 0 0) 0x7f1a8c003520 con 0x7f1aac0b28b0 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 msgr2=0x7f1aac06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 0x7f1aac06f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 >> v2:172.21.15.113:3300/0 conn(0x7f1aac0cd840 msgr2=0x7f1aac0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 --2- 172.21.15.91:0/2835755591 >> v2:172.21.15.113:3300/0 conn(0x7f1aac0cd840 0x7f1aac0cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0b2fe0 con 0x7f1aac0b28b0 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1a94001670 con 0x7f1aac0b28b0 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 >> v1:172.21.15.91:6789/0 conn(0x7f1aac0b28b0 legacy=0x7f1aac15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a8c003970 0x7f1a8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 0x7f1a8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c004d00 0x7f1a8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0ce1f0 con 0x7f1a8c003970 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0b2d10 con 0x7f1a8c004d00 2024-02-02T13:32:34.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00b980 con 0x7f1aac0b62f0 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a8c003970 0x7f1a8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1ab1a00640 1 -- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 msgr2=0x7f1a8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.222+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac0b62f0 0x7f1a8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.223+0000 7f1ab1a00640 1 -- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c004d00 msgr2=0x7f1a8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.223+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c004d00 0x7f1a8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.223+0000 7f1ab1a00640 1 -- 172.21.15.91:0/2835755591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c003520 con 0x7f1a8c003970 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a8c003970 0x7f1a8c003d90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ac0 tx=0x7f1a8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54bd8dd97e9c5dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02c710 con 0x7f1a8c003970 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c02c870 con 0x7f1a8c003970 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1aabe00640 1 -- 172.21.15.91:0/2835755591 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02cc00 con 0x7f1a8c003970 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1ab4400640 1 -- 172.21.15.91:0/2835755591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a8c003970 msgr2=0x7f1a8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1ab4400640 1 --2- 172.21.15.91:0/2835755591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a8c003970 0x7f1a8c003d90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ac0 tx=0x7f1a8c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.224+0000 7f1ab2400640 1 -- 172.21.15.91:0/2835755591 reap_dead start 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.225+0000 7f1ab4400640 1 -- 172.21.15.91:0/2835755591 shutdown_connections 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.225+0000 7f1ab4400640 1 -- 172.21.15.91:0/2835755591 >> 172.21.15.91:0/2835755591 conn(0x7f1aac06d840 msgr2=0x7f1aac06dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.225+0000 7f1ab4400640 1 -- 172.21.15.91:0/2835755591 shutdown_connections 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.225+0000 7f1ab4400640 1 -- 172.21.15.91:0/2835755591 wait complete. 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.226+0000 7f1ab4400640 1 Processor -- start 2024-02-02T13:32:34.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.227+0000 7f1ab4400640 1 -- start start 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.227+0000 7f1ab4400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c003970 0x7f1aac069de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.227+0000 7f1ab4400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac14e3a0 0x7f1aac14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab4400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab4400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac15da40 con 0x7f1aac15d080 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab4400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac14e820 con 0x7f1a8c003970 2024-02-02T13:32:34.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab4400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac06b560 con 0x7f1aac14e3a0 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33642/0 (socket says 172.21.15.91:33642) 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 -- 172.21.15.91:0/1963308394 learned_addr learned my addr 172.21.15.91:0/1963308394 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 -- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac14e3a0 msgr2=0x7f1aac14fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac14e3a0 0x7f1aac14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 -- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c003970 msgr2=0x7f1aac069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c003970 0x7f1aac069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 -- 172.21.15.91:0/1963308394 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac150220 con 0x7f1aac15d080 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.228+0000 7f1ab2400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0cdc20 tx=0x7f1aa400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.229+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4010040 con 0x7f1aac15d080 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.229+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa400bae0 con 0x7f1aac15d080 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.229+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac06a4c0 con 0x7f1aac15d080 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.229+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400be70 con 0x7f1aac15d080 2024-02-02T13:32:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.229+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac06aa80 con 0x7f1aac15d080 2024-02-02T13:32:34.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.230+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac067400 con 0x7f1aac15d080 2024-02-02T13:32:34.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.231+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1aa401e410 con 0x7f1aac15d080 2024-02-02T13:32:34.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.232+0000 7f1aa9600640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 0x7f1a8406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.232+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1aa4088b60 con 0x7f1aac15d080 2024-02-02T13:32:34.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.232+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 0x7f1a8406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.233+0000 7f1ab1a00640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 0x7f1a8406c920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.237+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1aa4057ba0 con 0x7f1aac15d080 2024-02-02T13:32:34.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.423+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1aac0b51a0 con 0x7f1aac15d080 2024-02-02T13:32:34.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.426+0000 7f1aa9600640 1 -- 172.21.15.91:0/1963308394 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1aa4057730 con 0x7f1aac15d080 2024-02-02T13:32:34.428 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:32:34.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 msgr2=0x7f1a8406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 0x7f1a8406c920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c02e000 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 msgr2=0x7f1aac15d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0cdc20 tx=0x7f1aa400b480 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 shutdown_connections 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1a8406a460 0x7f1a8406c920 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aac14e3a0 0x7f1aac14fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1aac15d080 0x7f1aac15d500 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 --2- 172.21.15.91:0/1963308394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8c003970 0x7f1aac069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.430+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 >> 172.21.15.91:0/1963308394 conn(0x7f1aac06d840 msgr2=0x7f1aac0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.431+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 shutdown_connections 2024-02-02T13:32:34.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.431+0000 7f1ab4400640 1 -- 172.21.15.91:0/1963308394 wait complete. 2024-02-02T13:32:34.451 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:32:34.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.661+0000 7f4896000640 1 Processor -- start 2024-02-02T13:32:34.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.661+0000 7f4896000640 1 -- start start 2024-02-02T13:32:34.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f4896000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 0x7f48900b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f4896000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4890070f80 con 0x7f489015f5a0 2024-02-02T13:32:34.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f4896000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489006c600 con 0x7f4890151f90 2024-02-02T13:32:34.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f4896000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015fa30 con 0x7f48900b0e10 2024-02-02T13:32:34.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f488ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f489015f5a0 0x7f48900b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40466/0 (socket says 172.21.15.91:40466) 2024-02-02T13:32:34.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.663+0000 7f488ea00640 1 -- 172.21.15.91:0/304170154 learned_addr learned my addr 172.21.15.91:0/304170154 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488fe00640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 0x7f48900b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857797127 0 0) 0x7f4890070f80 con 0x7f489015f5a0 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4870003520 con 0x7f489015f5a0 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723025751 0 0) 0x7f4870003520 con 0x7f489015f5a0 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 msgr2=0x7f48900b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 0x7f48900b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 >> v1:172.21.15.113:6789/0 conn(0x7f4890151f90 legacy=0x7f489006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890152370 con 0x7f489015f5a0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f4878001670 con 0x7f489015f5a0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 >> v1:172.21.15.91:6789/0 conn(0x7f489015f5a0 legacy=0x7f48900b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700039a0 0x7f4870003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700047c0 0x7f4870004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 0x7f4870005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489015fa30 con 0x7f48700047c0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487000ba60 con 0x7f48700039a0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.664+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487000bc20 con 0x7f48900b0e10 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700047c0 0x7f4870004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488f400640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700039a0 0x7f4870003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 -- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 msgr2=0x7f4870005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48900b0e10 0x7f4870005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 -- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700039a0 msgr2=0x7f4870003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700039a0 0x7f4870003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488ea00640 1 -- 172.21.15.91:0/304170154 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870003520 con 0x7f48700047c0 2024-02-02T13:32:34.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.665+0000 7f488f400640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48700039a0 0x7f4870003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.666+0000 7f488ea00640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700047c0 0x7f4870004c40 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f487800c210 tx=0x7f48700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ca3d5855db720a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878005ad0 con 0x7f48700047c0 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878005c30 con 0x7f48700047c0 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f488e000640 1 -- 172.21.15.91:0/304170154 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487802c870 con 0x7f48700047c0 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 -- 172.21.15.91:0/304170154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700047c0 msgr2=0x7f4870004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 --2- 172.21.15.91:0/304170154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48700047c0 0x7f4870004c40 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f487800c210 tx=0x7f48700057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f488fe00640 1 -- 172.21.15.91:0/304170154 reap_dead start 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 -- 172.21.15.91:0/304170154 shutdown_connections 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 -- 172.21.15.91:0/304170154 >> 172.21.15.91:0/304170154 conn(0x7f48900cec80 msgr2=0x7f48900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 -- 172.21.15.91:0/304170154 shutdown_connections 2024-02-02T13:32:34.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.667+0000 7f4896000640 1 -- 172.21.15.91:0/304170154 wait complete. 2024-02-02T13:32:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.669+0000 7f4896000640 1 Processor -- start 2024-02-02T13:32:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.669+0000 7f4896000640 1 -- start start 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700047c0 0x7f48901481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 0x7f489015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f3a40 con 0x7f48900c7cd0 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900c8690 con 0x7f489015db60 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f4896000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015c560 con 0x7f48700047c0 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f488fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.670+0000 7f488fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33658/0 (socket says 172.21.15.91:33658) 2024-02-02T13:32:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 -- 172.21.15.91:0/3402214545 learned_addr learned my addr 172.21.15.91:0/3402214545 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 -- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700047c0 msgr2=0x7f48901481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700047c0 0x7f48901481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 -- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 msgr2=0x7f489015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488ea00640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 0x7f489015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 0x7f489015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 -- 172.21.15.91:0/3402214545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489015c7b0 con 0x7f48900c7cd0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488ea00640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 0x7f489015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f488fe00640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f48900ca540 tx=0x7f488800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888010040 con 0x7f48900c7cd0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.671+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488800ba50 con 0x7f48900c7cd0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.672+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489015ca90 con 0x7f48900c7cd0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.672+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800be00 con 0x7f48900c7cd0 2024-02-02T13:32:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.672+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489015d050 con 0x7f48900c7cd0 2024-02-02T13:32:34.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.673+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890066cf0 con 0x7f48900c7cd0 2024-02-02T13:32:34.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.674+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f488801d400 con 0x7f48900c7cd0 2024-02-02T13:32:34.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.674+0000 7f4887400640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 0x7f486806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:34.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.675+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4888088ec0 con 0x7f48900c7cd0 2024-02-02T13:32:34.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.675+0000 7f488f400640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 0x7f486806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:34.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.675+0000 7f488f400640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 0x7f486806c8f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f489015ef20 tx=0x7f4880009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:34.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.679+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4888054470 con 0x7f48900c7cd0 2024-02-02T13:32:34.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.869+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f489006c7b0 con 0x7f48900c7cd0 2024-02-02T13:32:34.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.870+0000 7f4887400640 1 -- 172.21.15.91:0/3402214545 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4888057a90 con 0x7f48900c7cd0 2024-02-02T13:32:34.872 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 msgr2=0x7f486806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 0x7f486806c8f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f489015ef20 tx=0x7f4880009040 comp rx=0 tx=0).stop 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 msgr2=0x7f48900c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f48900ca540 tx=0x7f488800eea0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 shutdown_connections 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f486806a430 0x7f486806c8f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48900c7cd0 0x7f48900c8150 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f489015db60 0x7f489015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 --2- 172.21.15.91:0/3402214545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48700047c0 0x7f48901481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:34.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 >> 172.21.15.91:0/3402214545 conn(0x7f48900cec80 msgr2=0x7f48900cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:34.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 shutdown_connections 2024-02-02T13:32:34.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:34.873+0000 7f4896000640 1 -- 172.21.15.91:0/3402214545 wait complete. 2024-02-02T13:32:34.892 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:32:35.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.106+0000 7f4407a00640 1 Processor -- start 2024-02-02T13:32:35.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.107+0000 7f4407a00640 1 -- start start 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.108+0000 7f4407a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 0x7f440015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.108+0000 7f4407a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44000b0e10 0x7f44000b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.108+0000 7f4407a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4400070f80 con 0x7f440006f4f0 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.108+0000 7f4407a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4400152370 con 0x7f44000b0e10 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.108+0000 7f4407a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000cb010 con 0x7f4400151f90 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ffe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f440006f4f0 0x7f44000b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40482/0 (socket says 172.21.15.91:40482) 2024-02-02T13:32:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ffe00640 1 -- 172.21.15.91:0/1566827365 learned_addr learned my addr 172.21.15.91:0/1566827365 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f4405000640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 0x7f440015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f4405a00640 1 --2- 172.21.15.91:0/1566827365 >> v2:172.21.15.113:3300/0 conn(0x7f44000b0e10 0x7f44000b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314711522 0 0) 0x7f4400070f80 con 0x7f440006f4f0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e0003520 con 0x7f440006f4f0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3145657179 0 0) 0x7f43e0003520 con 0x7f440006f4f0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 msgr2=0x7f440015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 0x7f440015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 >> v2:172.21.15.113:3300/0 conn(0x7f44000b0e10 msgr2=0x7f44000b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 --2- 172.21.15.91:0/1566827365 >> v2:172.21.15.113:3300/0 conn(0x7f44000b0e10 0x7f44000b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.109+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000cb2f0 con 0x7f440006f4f0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f43e8001670 con 0x7f440006f4f0 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 >> v1:172.21.15.91:6789/0 conn(0x7f440006f4f0 legacy=0x7f44000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e0003970 0x7f43e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 0x7f43e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e0004d00 0x7f43e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400152370 con 0x7f43e0003970 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000cb010 con 0x7f43e0004d00 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e000b980 con 0x7f4400151f90 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e0003970 0x7f43e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405a00640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e0004d00 0x7f43e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 -- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 msgr2=0x7f43e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4400151f90 0x7f43e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 -- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e0004d00 msgr2=0x7f43e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e0004d00 0x7f43e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405000640 1 -- 172.21.15.91:0/1566827365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0003520 con 0x7f43e0003970 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.110+0000 7f4405a00640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e0004d00 0x7f43e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.111+0000 7f4405000640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e0003970 0x7f43e0003d90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009ac0 tx=0x7f43e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fec8c5e8635f555 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f002c710 con 0x7f43e0003970 2024-02-02T13:32:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f002c870 con 0x7f43e0003970 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f43ff400640 1 -- 172.21.15.91:0/1566827365 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f002cc00 con 0x7f43e0003970 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 -- 172.21.15.91:0/1566827365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e0003970 msgr2=0x7f43e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 --2- 172.21.15.91:0/1566827365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f43e0003970 0x7f43e0003d90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009ac0 tx=0x7f43e00057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4405a00640 1 -- 172.21.15.91:0/1566827365 reap_dead start 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 -- 172.21.15.91:0/1566827365 shutdown_connections 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 -- 172.21.15.91:0/1566827365 >> 172.21.15.91:0/1566827365 conn(0x7f44000cec80 msgr2=0x7f44000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 -- 172.21.15.91:0/1566827365 shutdown_connections 2024-02-02T13:32:35.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.112+0000 7f4407a00640 1 -- 172.21.15.91:0/1566827365 wait complete. 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.114+0000 7f4407a00640 1 Processor -- start 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.114+0000 7f4407a00640 1 -- start start 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e0003970 0x7f440014c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44000c1eb0 0x7f44000c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000b29b0 con 0x7f44000c2c60 2024-02-02T13:32:35.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000b2c90 con 0x7f44000c1eb0 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4407a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000b2f70 con 0x7f43e0003970 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4405a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4405a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33672/0 (socket says 172.21.15.91:33672) 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.115+0000 7f4405a00640 1 -- 172.21.15.91:0/808195629 learned_addr learned my addr 172.21.15.91:0/808195629 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405000640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e0003970 0x7f440014c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f43ffe00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44000c1eb0 0x7f44000c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 -- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e0003970 msgr2=0x7f440014c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e0003970 0x7f440014c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 -- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44000c1eb0 msgr2=0x7f44000c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44000c1eb0 0x7f44000c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 -- 172.21.15.91:0/808195629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000b3250 con 0x7f44000c2c60 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4405a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f44000cec30 tx=0x7f43f800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8016420 con 0x7f44000c2c60 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8016580 con 0x7f44000c2c60 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f80168f0 con 0x7f44000c2c60 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44000b3530 con 0x7f44000c2c60 2024-02-02T13:32:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.116+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000c2330 con 0x7f44000c2c60 2024-02-02T13:32:35.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.117+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44000b3720 con 0x7f44000c2c60 2024-02-02T13:32:35.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.118+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f43f8016a50 con 0x7f44000c2c60 2024-02-02T13:32:35.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.119+0000 7f43fcc00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 0x7f43d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.119+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f43f8089600 con 0x7f44000c2c60 2024-02-02T13:32:35.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.119+0000 7f4405000640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 0x7f43d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.120+0000 7f4405000640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 0x7f43d806c8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f43f0036000 tx=0x7f43f0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.123+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f43f8054b10 con 0x7f44000c2c60 2024-02-02T13:32:35.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.314+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4400066cf0 con 0x7f44000c2c60 2024-02-02T13:32:35.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.315+0000 7f43fcc00640 1 -- 172.21.15.91:0/808195629 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43f8058130 con 0x7f44000c2c60 2024-02-02T13:32:35.316 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:32:35.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 msgr2=0x7f43d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 0x7f43d806c8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f43f0036000 tx=0x7f43f0030000 comp rx=0 tx=0).stop 2024-02-02T13:32:35.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 msgr2=0x7f44000c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f44000cec30 tx=0x7f43f800f520 comp rx=0 tx=0).stop 2024-02-02T13:32:35.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 shutdown_connections 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f43d806a430 0x7f43d806c8f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44000c2c60 0x7f44000c9790 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44000c1eb0 0x7f44000c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 --2- 172.21.15.91:0/808195629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e0003970 0x7f440014c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 >> 172.21.15.91:0/808195629 conn(0x7f44000cec80 msgr2=0x7f440006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 shutdown_connections 2024-02-02T13:32:35.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.317+0000 7f4407a00640 1 -- 172.21.15.91:0/808195629 wait complete. 2024-02-02T13:32:35.337 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:32:35.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.540+0000 7f9343c00640 1 Processor -- start 2024-02-02T13:32:35.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.540+0000 7f9343c00640 1 -- start start 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.541+0000 7f9343c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f933c0b8b20 0x7f933c0d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.541+0000 7f9343c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0d2300 con 0x7f933c0ce4c0 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.541+0000 7f9343c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c15d640 con 0x7f933c15d260 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.541+0000 7f9343c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c0b8fd0 con 0x7f933c0b8b20 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f933c0b8b20 0x7f933c0d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f933c0b8b20 0x7f933c0d1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33674/0 (socket says 172.21.15.91:33674) 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 -- 172.21.15.91:0/4167925778 learned_addr learned my addr 172.21.15.91:0/4167925778 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:35.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f933be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f933c0ce4c0 0x7f933c070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40488/0 (socket says 172.21.15.91:40488) 2024-02-02T13:32:35.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 -- 172.21.15.91:0/4167925778 >> v1:172.21.15.113:6789/0 conn(0x7f933c15d260 legacy=0x7f933c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:35.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 -- 172.21.15.91:0/4167925778 >> v1:172.21.15.91:6789/0 conn(0x7f933c0ce4c0 legacy=0x7f933c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.542+0000 7f9341c00640 1 -- 172.21.15.91:0/4167925778 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0d2530 con 0x7f933c0b8b20 2024-02-02T13:32:35.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.543+0000 7f9341c00640 1 --2- 172.21.15.91:0/4167925778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0b8b20 0x7f933c0d1dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9334009870 tx=0x7f933402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305cdc0f7f743fe3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.543+0000 7f933b400640 1 -- 172.21.15.91:0/4167925778 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334031870 con 0x7f933c0b8b20 2024-02-02T13:32:35.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f933b400640 1 -- 172.21.15.91:0/4167925778 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340319d0 con 0x7f933c0b8b20 2024-02-02T13:32:35.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f933b400640 1 -- 172.21.15.91:0/4167925778 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334031d60 con 0x7f933c0b8b20 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 -- 172.21.15.91:0/4167925778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0b8b20 msgr2=0x7f933c0d1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 --2- 172.21.15.91:0/4167925778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0b8b20 0x7f933c0d1dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9334009870 tx=0x7f933402b910 comp rx=0 tx=0).stop 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 -- 172.21.15.91:0/4167925778 shutdown_connections 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 --2- 172.21.15.91:0/4167925778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0b8b20 0x7f933c0d1dc0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 -- 172.21.15.91:0/4167925778 >> 172.21.15.91:0/4167925778 conn(0x7f933c0cfa60 msgr2=0x7f933c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 -- 172.21.15.91:0/4167925778 shutdown_connections 2024-02-02T13:32:35.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.544+0000 7f9343c00640 1 -- 172.21.15.91:0/4167925778 wait complete. 2024-02-02T13:32:35.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.546+0000 7f9343c00640 1 Processor -- start 2024-02-02T13:32:35.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.546+0000 7f9343c00640 1 -- start start 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f933c0b8b20 0x7f933c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 0x7f933c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0b58f0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c7410 con 0x7f933c15d260 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f9343c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c76b0 con 0x7f933c0b8b20 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f933be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f933be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33686/0 (socket says 172.21.15.91:33686) 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.547+0000 7f933be00640 1 -- 172.21.15.91:0/2895351796 learned_addr learned my addr 172.21.15.91:0/2895351796 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:35.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9341c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 0x7f933c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9341200640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f933c0b8b20 0x7f933c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 -- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f933c0b8b20 msgr2=0x7f933c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f933c0b8b20 0x7f933c0c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 -- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 msgr2=0x7f933c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 0x7f933c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 -- 172.21.15.91:0/2895351796 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340094d0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9341c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 0x7f933c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f933be00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f932400c7c0 tx=0x7f932400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400a640 con 0x7f933c0ce4c0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400a7a0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0c7990 con 0x7f933c0ce4c0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.548+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400ab50 con 0x7f933c0ce4c0 2024-02-02T13:32:35.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.549+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c0c62a0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.550+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c0c7eb0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.550+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f932400acb0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.551+0000 7f9338c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 0x7f931406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:35.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.551+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9324088020 con 0x7f933c0ce4c0 2024-02-02T13:32:35.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.552+0000 7f9341200640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 0x7f931406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:35.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.552+0000 7f9341200640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 0x7f931406c8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f932c006fd0 tx=0x7f932c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:35.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.556+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f933c0c7eb0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.756+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f933c0c7eb0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.757+0000 7f9338c00640 1 -- 172.21.15.91:0/2895351796 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) v1 ==== 80+0+0 (secure 0 0 0) 0x7f933c0c7eb0 con 0x7f933c0ce4c0 2024-02-02T13:32:35.759 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:32:35.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 msgr2=0x7f931406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 0x7f931406c8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f932c006fd0 tx=0x7f932c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:35.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 msgr2=0x7f933c0b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:35.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f932400c7c0 tx=0x7f932400cc80 comp rx=0 tx=0).stop 2024-02-02T13:32:35.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 shutdown_connections 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.759+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f931406a430 0x7f931406c8f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933c15d260 0x7f933c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f933c0ce4c0 0x7f933c0b2b00 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 --2- 172.21.15.91:0/2895351796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f933c0b8b20 0x7f933c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 >> 172.21.15.91:0/2895351796 conn(0x7f933c0cfa60 msgr2=0x7f933c15b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 shutdown_connections 2024-02-02T13:32:35.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:35.760+0000 7f9343c00640 1 -- 172.21.15.91:0/2895351796 wait complete. 2024-02-02T13:32:35.779 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:32:35.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:35.779 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:32:35.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:32:35.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:35.915 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:32:35.962 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:35.962 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:36.004 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:36.004 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:36.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:36.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.308+0000 7f502f400640 1 Processor -- start 2024-02-02T13:32:36.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.308+0000 7f502f400640 1 -- start start 2024-02-02T13:32:36.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5028151f90 0x7f502815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f502806f4f0 0x7f50280b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502f400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5028070f80 con 0x7f50280b0c40 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502f400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280b10d0 con 0x7f502806f4f0 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5028152370 con 0x7f5028151f90 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f502ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5028151f90 0x7f502815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.310+0000 7f501fc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f502806f4f0 0x7f50280b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f502806f4f0 0x7f50280b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:45698/0 (socket says 172.21.15.91:45698) 2024-02-02T13:32:36.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 -- 172.21.15.91:0/747765819 learned_addr learned my addr 172.21.15.91:0/747765819 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:36.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 -- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5028151f90 msgr2=0x7f502815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:36.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 --2- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5028151f90 0x7f502815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 -- 172.21.15.91:0/747765819 >> v1:172.21.15.113:6789/0 conn(0x7f50280b0c40 legacy=0x7f50280b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.311+0000 7f501fc00640 1 -- 172.21.15.91:0/747765819 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280cb010 con 0x7f502806f4f0 2024-02-02T13:32:36.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.313+0000 7f501fc00640 1 --2- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50280b0700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3607aed00cadb2d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.313+0000 7f5027e00640 1 -- 172.21.15.91:0/747765819 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010030840 con 0x7f502806f4f0 2024-02-02T13:32:36.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.313+0000 7f5027e00640 1 -- 172.21.15.91:0/747765819 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100309a0 con 0x7f502806f4f0 2024-02-02T13:32:36.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.313+0000 7f5027e00640 1 -- 172.21.15.91:0/747765819 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010030d30 con 0x7f502806f4f0 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 -- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 msgr2=0x7f50280b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 --2- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50280b0700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b980 comp rx=0 tx=0).stop 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 -- 172.21.15.91:0/747765819 shutdown_connections 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 --2- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50280b0700 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 --2- 172.21.15.91:0/747765819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5028151f90 0x7f502815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 -- 172.21.15.91:0/747765819 >> 172.21.15.91:0/747765819 conn(0x7f50280cec80 msgr2=0x7f50280cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 -- 172.21.15.91:0/747765819 shutdown_connections 2024-02-02T13:32:36.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.314+0000 7f502f400640 1 -- 172.21.15.91:0/747765819 wait complete. 2024-02-02T13:32:36.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.316+0000 7f502f400640 1 Processor -- start 2024-02-02T13:32:36.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.316+0000 7f502f400640 1 -- start start 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50281510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 0x7f50280b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50281f6b60 con 0x7f5028151f90 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50281515f0 con 0x7f502806f4f0 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280b2350 con 0x7f50280b0c40 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.317+0000 7f502d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33696/0 (socket says 172.21.15.91:33696) 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 -- 172.21.15.91:0/77040292 learned_addr learned my addr 172.21.15.91:0/77040292 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502ca00640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50281510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f501fc00640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 0x7f50280b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 -- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 msgr2=0x7f50280b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 0x7f50280b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 -- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 msgr2=0x7f50281510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50281510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 -- 172.21.15.91:0/77040292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100094d0 con 0x7f5028151f90 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f501fc00640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 0x7f50280b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502d400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f502000c730 tx=0x7f502000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020003b70 con 0x7f5028151f90 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020003cf0 con 0x7f5028151f90 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020003ea0 con 0x7f5028151f90 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.318+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50280ce790 con 0x7f5028151f90 2024-02-02T13:32:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.319+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50280c2e90 con 0x7f5028151f90 2024-02-02T13:32:36.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.320+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5028066cf0 con 0x7f5028151f90 2024-02-02T13:32:36.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.321+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5020005310 con 0x7f5028151f90 2024-02-02T13:32:36.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.322+0000 7f5025600640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.322+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f50200887d0 con 0x7f5028151f90 2024-02-02T13:32:36.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.322+0000 7f502ca00640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.323+0000 7f502ca00640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 0x7f500006c920 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5018009ad0 tx=0x7f5018008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.326+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5020053ce0 con 0x7f5028151f90 2024-02-02T13:32:36.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.517+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50280714b0 con 0x7f5028151f90 2024-02-02T13:32:36.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.519+0000 7f5025600640 1 -- 172.21.15.91:0/77040292 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16220 (secure 0 0 0) 0x7f5020057300 con 0x7f5028151f90 2024-02-02T13:32:36.521 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:36.521 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":148,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:29.417200+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 msgr2=0x7f500006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 0x7f500006c920 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5018009ad0 tx=0x7f5018008040 comp rx=0 tx=0).stop 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 msgr2=0x7f50280ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f502000c730 tx=0x7f502000cbf0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 shutdown_connections 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50280b0c40 0x7f50280b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5028151f90 0x7f50280ce250 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 --2- 172.21.15.91:0/77040292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f502806f4f0 0x7f50281510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 >> 172.21.15.91:0/77040292 conn(0x7f50280cec80 msgr2=0x7f502815a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:36.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 shutdown_connections 2024-02-02T13:32:36.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.522+0000 7f502f400640 1 -- 172.21.15.91:0/77040292 wait complete. 2024-02-02T13:32:36.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:58.576657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:59.552886+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:32:36.543 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:32:36.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.750+0000 7fcee0c00640 1 Processor -- start 2024-02-02T13:32:36.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.751+0000 7fcee0c00640 1 -- start start 2024-02-02T13:32:36.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fcee0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcedc15d330 0x7fcedc0b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fcee0c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc161090 con 0x7fcedc0d3850 2024-02-02T13:32:36.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fcee0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc161330 con 0x7fcedc0b2470 2024-02-02T13:32:36.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fcee0c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc161cd0 con 0x7fcedc15d330 2024-02-02T13:32:36.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fceda800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcedc0d3850 0x7fcedc0d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40502/0 (socket says 172.21.15.91:40502) 2024-02-02T13:32:36.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fceda800640 1 -- 172.21.15.91:0/2595659356 learned_addr learned my addr 172.21.15.91:0/2595659356 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.752+0000 7fced9e00640 1 --2- 172.21.15.91:0/2595659356 >> v2:172.21.15.113:3300/0 conn(0x7fcedc15d330 0x7fcedc0b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140976953 0 0) 0x7fcedc161090 con 0x7fcedc0d3850 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003520 con 0x7fcedc0d3850 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745172745 0 0) 0x7fcebc003520 con 0x7fcedc0d3850 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 >> v2:172.21.15.113:3300/0 conn(0x7fcedc15d330 msgr2=0x7fcedc0b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 --2- 172.21.15.91:0/2595659356 >> v2:172.21.15.113:3300/0 conn(0x7fcedc15d330 0x7fcedc0b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 >> v1:172.21.15.113:6789/0 conn(0x7fcedc0b2470 legacy=0x7fcedc0d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc161610 con 0x7fcedc0d3850 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fced4001670 con 0x7fcedc0d3850 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 >> v1:172.21.15.91:6789/0 conn(0x7fcedc0d3850 legacy=0x7fcedc0d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcebc0047c0 0x7fcebc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc0052a0 0x7fcebc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc161cd0 con 0x7fcebc003970 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00c170 con 0x7fcebc0047c0 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00c330 con 0x7fcebc0052a0 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.753+0000 7fced9e00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fced9e00640 1 -- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc0052a0 msgr2=0x7fcebc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fced9e00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc0052a0 0x7fcebc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fced9e00640 1 -- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcebc0047c0 msgr2=0x7fcebc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fced9e00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcebc0047c0 0x7fcebc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fced9e00640 1 -- 172.21.15.91:0/2595659356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc003520 con 0x7fcebc003970 2024-02-02T13:32:36.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.754+0000 7fceda800640 1 -- 172.21.15.91:0/2595659356 reap_dead start 2024-02-02T13:32:36.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.755+0000 7fced9e00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009a80 tx=0x7fcebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a84c6415bd6995a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.755+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc005990 con 0x7fcebc003970 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.755+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc005af0 con 0x7fcebc003970 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.755+0000 7fced8a00640 1 -- 172.21.15.91:0/2595659356 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc005ea0 con 0x7fcebc003970 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 -- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 msgr2=0x7fcebc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009a80 tx=0x7fcebc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 -- 172.21.15.91:0/2595659356 shutdown_connections 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2595659356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 -- 172.21.15.91:0/2595659356 >> 172.21.15.91:0/2595659356 conn(0x7fcedc070660 msgr2=0x7fcedc070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 -- 172.21.15.91:0/2595659356 shutdown_connections 2024-02-02T13:32:36.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.756+0000 7fcee0c00640 1 -- 172.21.15.91:0/2595659356 wait complete. 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.758+0000 7fcee0c00640 1 Processor -- start 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.758+0000 7fcee0c00640 1 -- start start 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 0x7fcedc0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 0x7fcedc0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b38b0 con 0x7fcedc14f740 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0ca7c0 con 0x7fcedc0c8eb0 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fcee0c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0c9870 con 0x7fcebc003970 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33716/0 (socket says 172.21.15.91:33716) 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 -- 172.21.15.91:0/2937031152 learned_addr learned my addr 172.21.15.91:0/2937031152 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9e00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 0x7fcedc0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fceda800640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 0x7fcedc0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 -- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 msgr2=0x7fcedc0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 0x7fcedc0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 -- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 msgr2=0x7fcedc0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 0x7fcedc0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.759+0000 7fced9400640 1 -- 172.21.15.91:0/2937031152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc15d890 con 0x7fcedc14f740 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fceda800640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 0x7fcedc0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fced9e00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 0x7fcedc0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fced9400640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c800 tx=0x7fcec400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4003b70 con 0x7fcedc14f740 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec4003cf0 con 0x7fcedc14f740 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec400aa90 con 0x7fcedc14f740 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc0b28a0 con 0x7fcedc14f740 2024-02-02T13:32:36.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.760+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc0b3420 con 0x7fcedc14f740 2024-02-02T13:32:36.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.761+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0b35b0 con 0x7fcedc14f740 2024-02-02T13:32:36.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.762+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcec400abf0 con 0x7fcedc14f740 2024-02-02T13:32:36.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.763+0000 7fced2000640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 0x7fceb406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:36.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.763+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcec4088880 con 0x7fcedc14f740 2024-02-02T13:32:36.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.764+0000 7fced9e00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 0x7fceb406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:36.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.764+0000 7fced9e00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 0x7fceb406c920 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcecc035000 tx=0x7fcecc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:36.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.768+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcec4053e20 con 0x7fcedc14f740 2024-02-02T13:32:36.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.954+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcedc067400 con 0x7fcedc14f740 2024-02-02T13:32:36.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.955+0000 7fced2000640 1 -- 172.21.15.91:0/2937031152 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcec4003e60 con 0x7fcedc14f740 2024-02-02T13:32:36.957 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 msgr2=0x7fceb406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 0x7fceb406c920 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcecc035000 tx=0x7fcecc030000 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 msgr2=0x7fcedc0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c800 tx=0x7fcec400ccc0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 shutdown_connections 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fceb406a460 0x7fceb406c920 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc0c8eb0 0x7fcedc0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcedc14f740 0x7fcedc0cb8a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 --2- 172.21.15.91:0/2937031152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcebc003970 0x7fcedc0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 >> 172.21.15.91:0/2937031152 conn(0x7fcedc070660 msgr2=0x7fcedc15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:36.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 shutdown_connections 2024-02-02T13:32:36.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:36.958+0000 7fcee0c00640 1 -- 172.21.15.91:0/2937031152 wait complete. 2024-02-02T13:32:36.977 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:32:37.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.190+0000 7f6dd8a00640 1 Processor -- start 2024-02-02T13:32:37.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.190+0000 7f6dd8a00640 1 -- start start 2024-02-02T13:32:37.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6dd015a010 0x7f6dd00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 0x7f6dd00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd8a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00cc7c0 con 0x7f6dd00cd470 2024-02-02T13:32:37.193 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00ce930 con 0x7f6dd015a010 2024-02-02T13:32:37.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd006c890 con 0x7f6dd00ce130 2024-02-02T13:32:37.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd6a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6dd00cd470 0x7f6dd00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40514/0 (socket says 172.21.15.91:40514) 2024-02-02T13:32:37.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd6a00640 1 -- 172.21.15.91:0/348709665 learned_addr learned my addr 172.21.15.91:0/348709665 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:37.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd5600640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 0x7f6dd00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.192+0000 7f6dd6000640 1 --2- 172.21.15.91:0/348709665 >> v2:172.21.15.113:3300/0 conn(0x7f6dd015a010 0x7f6dd00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067609246 0 0) 0x7f6dd00cc7c0 con 0x7f6dd00cd470 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db0005730 con 0x7f6dd00cd470 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962559519 0 0) 0x7f6db0005730 con 0x7f6dd00cd470 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 msgr2=0x7f6dd00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 0x7f6dd00ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 >> v2:172.21.15.113:3300/0 conn(0x7f6dd015a010 msgr2=0x7f6dd00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 --2- 172.21.15.91:0/348709665 >> v2:172.21.15.113:3300/0 conn(0x7f6dd015a010 0x7f6dd00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd006cb70 con 0x7f6dd00cd470 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6dc80019f0 con 0x7f6dd00cd470 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 >> v1:172.21.15.91:6789/0 conn(0x7f6dd00cd470 legacy=0x7f6dd00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db0005b80 0x7f6db0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6db00069d0 0x7f6db0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 0x7f6db0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00ce930 con 0x7f6db00069d0 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd006c890 con 0x7f6db0005b80 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db000db90 con 0x7f6dd00ce130 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.193+0000 7f6dd5600640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6db00069d0 0x7f6db0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.194+0000 7f6dd5600640 1 -- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 msgr2=0x7f6db0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.194+0000 7f6dd5600640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00ce130 0x7f6db0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.194+0000 7f6dd5600640 1 -- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db0005b80 msgr2=0x7f6db0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.194+0000 7f6dd5600640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db0005b80 0x7f6db0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.194+0000 7f6dd5600640 1 -- 172.21.15.91:0/348709665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0005730 con 0x7f6db00069d0 2024-02-02T13:32:37.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.195+0000 7f6dd5600640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6db00069d0 0x7f6db0006e50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009ac0 tx=0x7f6db0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b62d49c7d5de87b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.195+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db802c6d0 con 0x7f6db00069d0 2024-02-02T13:32:37.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.195+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db802c830 con 0x7f6db00069d0 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.195+0000 7f6dd4c00640 1 -- 172.21.15.91:0/348709665 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db802cbc0 con 0x7f6db00069d0 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 -- 172.21.15.91:0/348709665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6db00069d0 msgr2=0x7f6db0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/348709665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6db00069d0 0x7f6db0006e50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009ac0 tx=0x7f6db0007a00 comp rx=0 tx=0).stop 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd6a00640 1 -- 172.21.15.91:0/348709665 reap_dead start 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 -- 172.21.15.91:0/348709665 shutdown_connections 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 -- 172.21.15.91:0/348709665 >> 172.21.15.91:0/348709665 conn(0x7f6dd00cec80 msgr2=0x7f6dd00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 -- 172.21.15.91:0/348709665 shutdown_connections 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.196+0000 7f6dd8a00640 1 -- 172.21.15.91:0/348709665 wait complete. 2024-02-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.198+0000 7f6dd8a00640 1 Processor -- start 2024-02-02T13:32:37.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.198+0000 7f6dd8a00640 1 -- start start 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 0x7f6dd015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00c2da0 0x7f6dd00afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c3360 con 0x7f6dd00c83d0 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd015acb0 con 0x7f6db00069d0 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd015af90 con 0x7f6dd00c2da0 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33734/0 (socket says 172.21.15.91:33734) 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 -- 172.21.15.91:0/518005638 learned_addr learned my addr 172.21.15.91:0/518005638 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd6a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00c2da0 0x7f6dd00afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 -- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00c2da0 msgr2=0x7f6dd00afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd6000640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 0x7f6dd015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00c2da0 0x7f6dd00afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 -- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 msgr2=0x7f6dd015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 0x7f6dd015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd5600640 1 -- 172.21.15.91:0/518005638 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd015b270 con 0x7f6dd00c83d0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.199+0000 7f6dd6000640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 0x7f6dd015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dd5600640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6db802c2d0 tx=0x7f6db8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8003810 con 0x7f6dd00c83d0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8003970 con 0x7f6dd00c83d0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8003ce0 con 0x7f6dd00c83d0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd015b550 con 0x7f6dd00c83d0 2024-02-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.200+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd01ee650 con 0x7f6dd00c83d0 2024-02-02T13:32:37.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.202+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd0066cf0 con 0x7f6dd00c83d0 2024-02-02T13:32:37.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.203+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6db8040450 con 0x7f6dd00c83d0 2024-02-02T13:32:37.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.203+0000 7f6dce000640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 0x7f6da806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.204+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6db80aa700 con 0x7f6dd00c83d0 2024-02-02T13:32:37.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.204+0000 7f6dd6000640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 0x7f6da806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.204+0000 7f6dd6000640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 0x7f6da806c920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009aa0 tx=0x7f6dc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.208+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6db8075cb0 con 0x7f6dd00c83d0 2024-02-02T13:32:37.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.396+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dd00b4c00 con 0x7f6dd00c83d0 2024-02-02T13:32:37.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.397+0000 7f6dce000640 1 -- 172.21.15.91:0/518005638 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6db80044e0 con 0x7f6dd00c83d0 2024-02-02T13:32:37.399 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 msgr2=0x7f6da806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 0x7f6da806c920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009aa0 tx=0x7f6dc0008040 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 msgr2=0x7f6dd01ee110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6db802c2d0 tx=0x7f6db8004730 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 shutdown_connections 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6da806a460 0x7f6da806c920 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dd00c2da0 0x7f6dd00afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd00c83d0 0x7f6dd01ee110 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 --2- 172.21.15.91:0/518005638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db00069d0 0x7f6dd015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 >> 172.21.15.91:0/518005638 conn(0x7f6dd00cec80 msgr2=0x7f6dd015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 shutdown_connections 2024-02-02T13:32:37.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.400+0000 7f6dd8a00640 1 -- 172.21.15.91:0/518005638 wait complete. 2024-02-02T13:32:37.420 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:32:37.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.638+0000 7fa03b200640 1 Processor -- start 2024-02-02T13:32:37.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.638+0000 7fa03b200640 1 -- start start 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.639+0000 7fa03b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa034151f90 0x7fa03415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.639+0000 7fa03b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.639+0000 7fa03b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa034070f80 con 0x7fa0340b0c40 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.639+0000 7fa03b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0340b10d0 con 0x7fa034151f90 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.639+0000 7fa03b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa034152370 con 0x7fa03406f4f0 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.640+0000 7fa033e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa034151f90 0x7fa03415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.640+0000 7fa033400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.640+0000 7fa033e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa034151f90 0x7fa03415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:45756/0 (socket says 172.21.15.91:45756) 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.640+0000 7fa033e00640 1 -- 172.21.15.91:0/2898238188 learned_addr learned my addr 172.21.15.91:0/2898238188 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.640+0000 7fa033400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0340b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:34350/0 (socket says 172.21.15.91:34350) 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.641+0000 7fa033e00640 1 -- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 msgr2=0x7fa0340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.641+0000 7fa033e00640 1 --2- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.641+0000 7fa033e00640 1 -- 172.21.15.91:0/2898238188 >> v1:172.21.15.113:6789/0 conn(0x7fa0340b0c40 legacy=0x7fa0340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.641+0000 7fa033e00640 1 -- 172.21.15.91:0/2898238188 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340cb010 con 0x7fa034151f90 2024-02-02T13:32:37.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.642+0000 7fa033e00640 1 --2- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa034151f90 0x7fa03415f5a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa024009870 tx=0x7fa02402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a406b4fc588ffa48 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.642+0000 7fa032a00640 1 -- 172.21.15.91:0/2898238188 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024030840 con 0x7fa034151f90 2024-02-02T13:32:37.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.642+0000 7fa032a00640 1 -- 172.21.15.91:0/2898238188 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0240309a0 con 0x7fa034151f90 2024-02-02T13:32:37.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.642+0000 7fa032a00640 1 -- 172.21.15.91:0/2898238188 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024030d30 con 0x7fa034151f90 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 -- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa034151f90 msgr2=0x7fa03415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 --2- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa034151f90 0x7fa03415f5a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa024009870 tx=0x7fa02402b980 comp rx=0 tx=0).stop 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 -- 172.21.15.91:0/2898238188 shutdown_connections 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 --2- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 --2- 172.21.15.91:0/2898238188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa034151f90 0x7fa03415f5a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 -- 172.21.15.91:0/2898238188 >> 172.21.15.91:0/2898238188 conn(0x7fa0340cec80 msgr2=0x7fa0340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 -- 172.21.15.91:0/2898238188 shutdown_connections 2024-02-02T13:32:37.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.643+0000 7fa03b200640 1 -- 172.21.15.91:0/2898238188 wait complete. 2024-02-02T13:32:37.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.645+0000 7fa03b200640 1 Processor -- start 2024-02-02T13:32:37.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.646+0000 7fa03b200640 1 -- start start 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0341509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0340b0c40 0x7fa0340c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c8190 con 0x7fa034151f90 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c2b30 con 0x7fa0340b0c40 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa03b200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c8960 con 0x7fa03406f4f0 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33740/0 (socket says 172.21.15.91:33740) 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 -- 172.21.15.91:0/2809031783 learned_addr learned my addr 172.21.15.91:0/2809031783 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa033e00640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0341509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa033400640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0340b0c40 0x7fa0340c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 -- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 msgr2=0x7fa0341509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0341509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.647+0000 7fa039200640 1 -- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0340b0c40 msgr2=0x7fa0340c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa039200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0340b0c40 0x7fa0340c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa039200640 1 -- 172.21.15.91:0/2809031783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240094d0 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa039200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00c730 tx=0x7fa02c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c003b70 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c003cf0 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c003ea0 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0340c83a0 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.648+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0340c27c0 con 0x7fa034151f90 2024-02-02T13:32:37.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.650+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0340c2cc0 con 0x7fa034151f90 2024-02-02T13:32:37.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.650+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa02c005310 con 0x7fa034151f90 2024-02-02T13:32:37.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.651+0000 7fa02be00640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 0x7fa00c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:37.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.651+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa02c088730 con 0x7fa034151f90 2024-02-02T13:32:37.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.652+0000 7fa033e00640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 0x7fa00c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:37.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.652+0000 7fa033e00640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 0x7fa00c06c8f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa024036000 tx=0x7fa024008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:37.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.656+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa02c053ce0 con 0x7fa034151f90 2024-02-02T13:32:37.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.843+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa034066cf0 con 0x7fa034151f90 2024-02-02T13:32:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.844+0000 7fa02be00640 1 -- 172.21.15.91:0/2809031783 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa02c057300 con 0x7fa034151f90 2024-02-02T13:32:37.846 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:32:37.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 msgr2=0x7fa00c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 0x7fa00c06c8f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa024036000 tx=0x7fa024008030 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 msgr2=0x7fa034150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00c730 tx=0x7fa02c00cbf0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 shutdown_connections 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa00c06a430 0x7fa00c06c8f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa034151f90 0x7fa034150110 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0340b0c40 0x7fa0340c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 --2- 172.21.15.91:0/2809031783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa03406f4f0 0x7fa0341509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 >> 172.21.15.91:0/2809031783 conn(0x7fa0340cec80 msgr2=0x7fa0340acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 shutdown_connections 2024-02-02T13:32:37.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:37.847+0000 7fa03b200640 1 -- 172.21.15.91:0/2809031783 wait complete. 2024-02-02T13:32:37.868 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:32:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.074+0000 7f3955e00640 1 Processor -- start 2024-02-02T13:32:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.074+0000 7f3955e00640 1 -- start start 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.075+0000 7f3955e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f395015b0f0 0x7f395006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.075+0000 7f3955e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39500cb580 0x7f39500cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.075+0000 7f3955e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395006d910 con 0x7f395015ad10 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.075+0000 7f3955e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395006d730 con 0x7f395015b0f0 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.075+0000 7f3955e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39500cbf50 con 0x7f39500cb580 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f394ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f395015ad10 0x7f395006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:40530/0 (socket says 172.21.15.91:40530) 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f3945e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f395015b0f0 0x7f395006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f394ea00640 1 -- 172.21.15.91:0/1341716046 learned_addr learned my addr 172.21.15.91:0/1341716046 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f394f400640 1 --2- 172.21.15.91:0/1341716046 >> v2:172.21.15.113:3300/0 conn(0x7f39500cb580 0x7f39500cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f394e000640 1 -- 172.21.15.91:0/1341716046 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343004090 0 0) 0x7f395006d910 con 0x7f395015ad10 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f394e000640 1 -- 172.21.15.91:0/1341716046 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3930003520 con 0x7f395015ad10 2024-02-02T13:32:38.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f3945e00640 1 -- 172.21.15.91:0/1341716046 >> v2:172.21.15.113:3300/0 conn(0x7f39500cb580 msgr2=0x7f39500cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:38.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f3945e00640 1 --2- 172.21.15.91:0/1341716046 >> v2:172.21.15.113:3300/0 conn(0x7f39500cb580 0x7f39500cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f3945e00640 1 -- 172.21.15.91:0/1341716046 >> v1:172.21.15.91:6789/0 conn(0x7f395015ad10 legacy=0x7f395006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.076+0000 7f3945e00640 1 -- 172.21.15.91:0/1341716046 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500cc850 con 0x7f395015b0f0 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.077+0000 7f3945e00640 1 --2- 172.21.15.91:0/1341716046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f395006b7a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f39380097a0 tx=0x7f393802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536aabc132bb0699 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.077+0000 7f394e000640 1 -- 172.21.15.91:0/1341716046 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393802e070 con 0x7f395015b0f0 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.077+0000 7f394e000640 1 -- 172.21.15.91:0/1341716046 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39380315a0 con 0x7f395015b0f0 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f394e000640 1 -- 172.21.15.91:0/1341716046 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938031930 con 0x7f395015b0f0 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 -- 172.21.15.91:0/1341716046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 msgr2=0x7f395006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 --2- 172.21.15.91:0/1341716046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f395006b7a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f39380097a0 tx=0x7f393802b840 comp rx=0 tx=0).stop 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 -- 172.21.15.91:0/1341716046 shutdown_connections 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 --2- 172.21.15.91:0/1341716046 >> v2:172.21.15.113:3300/0 conn(0x7f39500cb580 0x7f39500cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 --2- 172.21.15.91:0/1341716046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f395006b7a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 -- 172.21.15.91:0/1341716046 >> 172.21.15.91:0/1341716046 conn(0x7f39500b0700 msgr2=0x7f39500b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 -- 172.21.15.91:0/1341716046 shutdown_connections 2024-02-02T13:32:38.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.078+0000 7f3955e00640 1 -- 172.21.15.91:0/1341716046 wait complete. 2024-02-02T13:32:38.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.080+0000 7f3955e00640 1 Processor -- start 2024-02-02T13:32:38.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.080+0000 7f3955e00640 1 -- start start 2024-02-02T13:32:38.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39500cb580 0x7f39501580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f395015ad10 0x7f39500a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c85d0 con 0x7f395015b0f0 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c88b0 con 0x7f39500cb580 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f3955e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c8b90 con 0x7f395015ad10 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f394f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f394f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33766/0 (socket says 172.21.15.91:33766) 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.081+0000 7f394f400640 1 -- 172.21.15.91:0/1403598565 learned_addr learned my addr 172.21.15.91:0/1403598565 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 -- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f395015ad10 msgr2=0x7f39500a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f395015ad10 0x7f39500a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 -- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39500cb580 msgr2=0x7f39501580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39500cb580 0x7f39501580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 -- 172.21.15.91:0/1403598565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009400 con 0x7f395015b0f0 2024-02-02T13:32:38.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f394f400640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f394800d800 tx=0x7f394800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800b6c0 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394800b840 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800bbf0 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39500c8e70 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.082+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39500c78a0 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.083+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3950066cf0 con 0x7f395015b0f0 2024-02-02T13:32:38.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.084+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f394800bd50 con 0x7f395015b0f0 2024-02-02T13:32:38.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.085+0000 7f3947e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 0x7f392806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.085+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3948088f00 con 0x7f395015b0f0 2024-02-02T13:32:38.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.085+0000 7f394ea00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 0x7f392806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.086+0000 7f394ea00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 0x7f392806c8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f39400012e0 tx=0x7f3940001650 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.091 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.090+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3948054410 con 0x7f395015b0f0 2024-02-02T13:32:38.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.291+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f395015e620 con 0x7f395015b0f0 2024-02-02T13:32:38.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.292+0000 7f3947e00640 1 -- 172.21.15.91:0/1403598565 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3948057a30 con 0x7f395015b0f0 2024-02-02T13:32:38.294 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 msgr2=0x7f392806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 0x7f392806c8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f39400012e0 tx=0x7f3940001650 comp rx=0 tx=0).stop 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 msgr2=0x7f39500a9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f394800d800 tx=0x7f394800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 shutdown_connections 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f392806a430 0x7f392806c8f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f395015ad10 0x7f39500a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f395015b0f0 0x7f39500a9380 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 --2- 172.21.15.91:0/1403598565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39500cb580 0x7f39501580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 >> 172.21.15.91:0/1403598565 conn(0x7f39500b0700 msgr2=0x7f39501e3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:38.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 shutdown_connections 2024-02-02T13:32:38.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.295+0000 7f3955e00640 1 -- 172.21.15.91:0/1403598565 wait complete. 2024-02-02T13:32:38.313 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:32:38.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:38.313 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:32:38.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:32:38.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:32:38.426 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:32:38.476 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:38.476 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:38.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:32:38.532 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:38.532 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:38.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:32:38.587 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:38.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.803+0000 7ff6ea400640 1 Processor -- start 2024-02-02T13:32:38.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.803+0000 7ff6ea400640 1 -- start start 2024-02-02T13:32:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.804+0000 7ff6ea400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6e415f5a0 0x7ff6e40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.804+0000 7ff6ea400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e40b0c40 0x7ff6e40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.804+0000 7ff6ea400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40b1070 con 0x7ff6e4151f90 2024-02-02T13:32:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.804+0000 7ff6ea400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e415fa30 con 0x7ff6e40b0c40 2024-02-02T13:32:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.804+0000 7ff6ea400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4070f80 con 0x7ff6e415f5a0 2024-02-02T13:32:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e40b0c40 0x7ff6e40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e40b0c40 0x7ff6e40b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33774/0 (socket says 172.21.15.91:33774) 2024-02-02T13:32:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 -- 172.21.15.91:0/2153237923 learned_addr learned my addr 172.21.15.91:0/2153237923 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6dbe00640 1 --2- 172.21.15.91:0/2153237923 >> v2:172.21.15.113:3300/0 conn(0x7ff6e415f5a0 0x7ff6e40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 -- 172.21.15.91:0/2153237923 >> v2:172.21.15.113:3300/0 conn(0x7ff6e415f5a0 msgr2=0x7ff6e40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 --2- 172.21.15.91:0/2153237923 >> v2:172.21.15.113:3300/0 conn(0x7ff6e415f5a0 0x7ff6e40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 -- 172.21.15.91:0/2153237923 >> v1:172.21.15.113:6789/0 conn(0x7ff6e4151f90 legacy=0x7ff6e406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.805+0000 7ff6e3e00640 1 -- 172.21.15.91:0/2153237923 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4152370 con 0x7ff6e40b0c40 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.806+0000 7ff6e3e00640 1 --2- 172.21.15.91:0/2153237923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e40b0c40 0x7ff6e40b3af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4071a00 tx=0x7ff6dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa40b23d7206f94 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.807+0000 7ff6e2a00640 1 -- 172.21.15.91:0/2153237923 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc02ca80 con 0x7ff6e40b0c40 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.807+0000 7ff6e2a00640 1 -- 172.21.15.91:0/2153237923 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc02cbe0 con 0x7ff6e40b0c40 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.807+0000 7ff6e2a00640 1 -- 172.21.15.91:0/2153237923 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0343f0 con 0x7ff6e40b0c40 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.807+0000 7ff6ea400640 1 -- 172.21.15.91:0/2153237923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e40b0c40 msgr2=0x7ff6e40b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.807+0000 7ff6ea400640 1 --2- 172.21.15.91:0/2153237923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e40b0c40 0x7ff6e40b3af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4071a00 tx=0x7ff6dc02bee0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 -- 172.21.15.91:0/2153237923 shutdown_connections 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 --2- 172.21.15.91:0/2153237923 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e40b0c40 0x7ff6e40b3af0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 --2- 172.21.15.91:0/2153237923 >> v2:172.21.15.113:3300/0 conn(0x7ff6e415f5a0 0x7ff6e40b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 -- 172.21.15.91:0/2153237923 >> 172.21.15.91:0/2153237923 conn(0x7ff6e40cec80 msgr2=0x7ff6e40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 -- 172.21.15.91:0/2153237923 shutdown_connections 2024-02-02T13:32:38.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.808+0000 7ff6ea400640 1 -- 172.21.15.91:0/2153237923 wait complete. 2024-02-02T13:32:38.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.809+0000 7ff6ea400640 1 Processor -- start 2024-02-02T13:32:38.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.810+0000 7ff6ea400640 1 -- start start 2024-02-02T13:32:38.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b0c40 0x7ff6e414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 0x7ff6e415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b2790 con 0x7ff6e4151f90 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b2a70 con 0x7ff6e415f5a0 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6ea400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b2d50 con 0x7ff6e40b0c40 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33786/0 (socket says 172.21.15.91:33786) 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 -- 172.21.15.91:0/556886756 learned_addr learned my addr 172.21.15.91:0/556886756 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6e3e00640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 0x7ff6e415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6e3400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b0c40 0x7ff6e414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 -- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b0c40 msgr2=0x7ff6e414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b0c40 0x7ff6e414c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 -- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 msgr2=0x7ff6e415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 0x7ff6e415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6dbe00640 1 -- 172.21.15.91:0/556886756 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc009b30 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.811+0000 7ff6e3e00640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 0x7ff6e415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6dbe00640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d5f0 tx=0x7ff6cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0125f0 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc012750 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40b3030 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc012b00 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.812+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e414c9c0 con 0x7ff6e4151f90 2024-02-02T13:32:38.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.813+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e4155990 con 0x7ff6e4151f90 2024-02-02T13:32:38.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.814+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff6cc012c60 con 0x7ff6e4151f90 2024-02-02T13:32:38.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.815+0000 7ff6db400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 0x7ff6bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:38.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.815+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6cc088de0 con 0x7ff6e4151f90 2024-02-02T13:32:38.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.815+0000 7ff6e3400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 0x7ff6bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:38.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.816+0000 7ff6e3400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 0x7ff6bc06c8f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400a730 tx=0x7ff6d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:38.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:38.820+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff6cc054390 con 0x7ff6e4151f90 2024-02-02T13:32:39.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.043+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e4074520 con 0x7ff6e4151f90 2024-02-02T13:32:39.046 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.044+0000 7ff6db400640 1 -- 172.21.15.91:0/556886756 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7ff6cc0579b0 con 0x7ff6e4151f90 2024-02-02T13:32:39.046 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:39.046 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:39.047 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 msgr2=0x7ff6bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 0x7ff6bc06c8f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400a730 tx=0x7ff6d4008040 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 msgr2=0x7ff6e415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d5f0 tx=0x7ff6cc00dab0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 shutdown_connections 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6bc06a430 0x7ff6bc06c8f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415f5a0 0x7ff6e415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e4151f90 0x7ff6e415a0f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 --2- 172.21.15.91:0/556886756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b0c40 0x7ff6e414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 >> 172.21.15.91:0/556886756 conn(0x7ff6e40cec80 msgr2=0x7ff6e406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:39.049 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 shutdown_connections 2024-02-02T13:32:39.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.047+0000 7ff6ea400640 1 -- 172.21.15.91:0/556886756 wait complete. 2024-02-02T13:32:39.068 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:39.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.263+0000 7f08f2600640 1 Processor -- start 2024-02-02T13:32:39.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.264+0000 7f08f2600640 1 -- start start 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08f2600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f08ec15f5a0 0x7f08ec0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08f2600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08ec0b0c40 0x7f08ec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08f2600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0b1070 con 0x7f08ec151f90 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08f2600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec15fa30 con 0x7f08ec15f5a0 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08f2600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec070f80 con 0x7f08ec0b0c40 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08e2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f08ec15f5a0 0x7f08ec0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08e2800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f08ec15f5a0 0x7f08ec0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33796/0 (socket says 172.21.15.91:33796) 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.265+0000 7f08e2800640 1 -- 172.21.15.91:0/4190643560 learned_addr learned my addr 172.21.15.91:0/4190643560 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:39.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.266+0000 7f08ebe00640 1 --2- 172.21.15.91:0/4190643560 >> v2:172.21.15.113:3300/0 conn(0x7f08ec0b0c40 0x7f08ec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.266+0000 7f08e2800640 1 -- 172.21.15.91:0/4190643560 >> v2:172.21.15.113:3300/0 conn(0x7f08ec0b0c40 msgr2=0x7f08ec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:39.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.266+0000 7f08e2800640 1 --2- 172.21.15.91:0/4190643560 >> v2:172.21.15.113:3300/0 conn(0x7f08ec0b0c40 0x7f08ec0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.266+0000 7f08e2800640 1 -- 172.21.15.91:0/4190643560 >> v1:172.21.15.113:6789/0 conn(0x7f08ec151f90 legacy=0x7f08ec06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:39.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.266+0000 7f08e2800640 1 -- 172.21.15.91:0/4190643560 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec152370 con 0x7f08ec15f5a0 2024-02-02T13:32:39.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.267+0000 7f08e2800640 1 --2- 172.21.15.91:0/4190643560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b0700 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ca520 tx=0x7f08d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2040ebac8e15d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.267+0000 7f08eaa00640 1 -- 172.21.15.91:0/4190643560 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4031870 con 0x7f08ec15f5a0 2024-02-02T13:32:39.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.267+0000 7f08eaa00640 1 -- 172.21.15.91:0/4190643560 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d40319d0 con 0x7f08ec15f5a0 2024-02-02T13:32:39.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.267+0000 7f08eaa00640 1 -- 172.21.15.91:0/4190643560 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4031d60 con 0x7f08ec15f5a0 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 -- 172.21.15.91:0/4190643560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec15f5a0 msgr2=0x7f08ec0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 --2- 172.21.15.91:0/4190643560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b0700 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ca520 tx=0x7f08d402b910 comp rx=0 tx=0).stop 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 -- 172.21.15.91:0/4190643560 shutdown_connections 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 --2- 172.21.15.91:0/4190643560 >> v2:172.21.15.113:3300/0 conn(0x7f08ec0b0c40 0x7f08ec0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 --2- 172.21.15.91:0/4190643560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b0700 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 -- 172.21.15.91:0/4190643560 >> 172.21.15.91:0/4190643560 conn(0x7f08ec0cec80 msgr2=0x7f08ec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 -- 172.21.15.91:0/4190643560 shutdown_connections 2024-02-02T13:32:39.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.268+0000 7f08f2600640 1 -- 172.21.15.91:0/4190643560 wait complete. 2024-02-02T13:32:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.270+0000 7f08f2600640 1 Processor -- start 2024-02-02T13:32:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.270+0000 7f08f2600640 1 -- start start 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08ec151f90 0x7f08ec0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1e5b30 con 0x7f08ec0b0c40 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c5390 con 0x7f08ec15f5a0 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08f2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1e51d0 con 0x7f08ec151f90 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08eb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33802/0 (socket says 172.21.15.91:33802) 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08eb400640 1 -- 172.21.15.91:0/196548650 learned_addr learned my addr 172.21.15.91:0/196548650 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08e2800640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08ec151f90 0x7f08ec0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.271+0000 7f08ebe00640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 -- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08ec151f90 msgr2=0x7f08ec0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08ec151f90 0x7f08ec0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 -- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 msgr2=0x7f08ec0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 -- 172.21.15.91:0/196548650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40094d0 con 0x7f08ec0b0c40 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08ebe00640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:39.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08eb400640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ba80 tx=0x7f08dc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00cc00 con 0x7f08ec0b0c40 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00cd60 con 0x7f08ec0b0c40 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec1e5470 con 0x7f08ec0b0c40 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc003bb0 con 0x7f08ec0b0c40 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.272+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0b24d0 con 0x7f08ec0b0c40 2024-02-02T13:32:39.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.273+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec066cf0 con 0x7f08ec0b0c40 2024-02-02T13:32:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.275+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f08dc014440 con 0x7f08ec0b0c40 2024-02-02T13:32:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.276+0000 7f08e3e00640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 0x7f08c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.276+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f08dc088790 con 0x7f08ec0b0c40 2024-02-02T13:32:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.276+0000 7f08e2800640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 0x7f08c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.277+0000 7f08e2800640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 0x7f08c406c920 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f08d4036000 tx=0x7f08d403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.282 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.280+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f08dc053d30 con 0x7f08ec0b0c40 2024-02-02T13:32:39.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.465+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08ec06c7b0 con 0x7f08ec0b0c40 2024-02-02T13:32:39.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.468+0000 7f08e3e00640 1 -- 172.21.15.91:0/196548650 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f08dc057350 con 0x7f08ec0b0c40 2024-02-02T13:32:39.470 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:39.470 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":149,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:39.421708+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:39.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 msgr2=0x7f08c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 0x7f08c406c920 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f08d4036000 tx=0x7f08d403c040 comp rx=0 tx=0).stop 2024-02-02T13:32:39.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 msgr2=0x7f08ec14b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ba80 tx=0x7f08dc00bf40 comp rx=0 tx=0).stop 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 shutdown_connections 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f08c406a460 0x7f08c406c920 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.470+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08ec151f90 0x7f08ec0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.471+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08ec15f5a0 0x7f08ec0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.471+0000 7f08f2600640 1 --2- 172.21.15.91:0/196548650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08ec0b0c40 0x7f08ec14b300 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.471+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 >> 172.21.15.91:0/196548650 conn(0x7f08ec0cec80 msgr2=0x7f08ec0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.471+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 shutdown_connections 2024-02-02T13:32:39.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.471+0000 7f08f2600640 1 -- 172.21.15.91:0/196548650 wait complete. 2024-02-02T13:32:39.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:31:58.576657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:31:59.552886+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:32:39.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:32:39.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.695+0000 7f2773800640 1 Processor -- start 2024-02-02T13:32:39.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.695+0000 7f2773800640 1 -- start start 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 0x7f276c0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 0x7f276c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f276c0d1410 0x7f276c0d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0b85e0 con 0x7f276c0d1410 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0b9140 con 0x7f276c160a20 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2773800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0b5550 con 0x7f276c0b4ba0 2024-02-02T13:32:39.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2771800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f276c0d1410 0x7f276c0d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2771800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f276c0d1410 0x7f276c0d1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33810/0 (socket says 172.21.15.91:33810) 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.697+0000 7f2771800640 1 -- 172.21.15.91:0/3768871995 learned_addr learned my addr 172.21.15.91:0/3768871995 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2770e00640 1 --2- 172.21.15.91:0/3768871995 >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 0x7f276c0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f276be00640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 0x7f276c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2771800640 1 -- 172.21.15.91:0/3768871995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 msgr2=0x7f276c0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2771800640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 0x7f276c0b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2771800640 1 -- 172.21.15.91:0/3768871995 >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 msgr2=0x7f276c0b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2771800640 1 --2- 172.21.15.91:0/3768871995 >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 0x7f276c0b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2771800640 1 -- 172.21.15.91:0/3768871995 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0b7960 con 0x7f276c0d1410 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f2770e00640 1 --2- 172.21.15.91:0/3768871995 >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 0x7f276c0b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:39.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.698+0000 7f276be00640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 0x7f276c0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.699+0000 7f2771800640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0d1410 0x7f276c0d1880 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f276c0cd970 tx=0x7f276402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc260aea5acf599 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.699+0000 7f276b400640 1 -- 172.21.15.91:0/3768871995 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764031640 con 0x7f276c0d1410 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.699+0000 7f276b400640 1 -- 172.21.15.91:0/3768871995 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27640317a0 con 0x7f276c0d1410 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.699+0000 7f276b400640 1 -- 172.21.15.91:0/3768871995 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764031b30 con 0x7f276c0d1410 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 -- 172.21.15.91:0/3768871995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0d1410 msgr2=0x7f276c0d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0d1410 0x7f276c0d1880 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f276c0cd970 tx=0x7f276402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 -- 172.21.15.91:0/3768871995 shutdown_connections 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0d1410 0x7f276c0d1880 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 --2- 172.21.15.91:0/3768871995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c0b4ba0 0x7f276c0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 --2- 172.21.15.91:0/3768871995 >> v2:172.21.15.113:3300/0 conn(0x7f276c160a20 0x7f276c0b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 -- 172.21.15.91:0/3768871995 >> 172.21.15.91:0/3768871995 conn(0x7f276c070480 msgr2=0x7f276c070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:39.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 -- 172.21.15.91:0/3768871995 shutdown_connections 2024-02-02T13:32:39.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.700+0000 7f2773800640 1 -- 172.21.15.91:0/3768871995 wait complete. 2024-02-02T13:32:39.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.702+0000 7f2773800640 1 Processor -- start 2024-02-02T13:32:39.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.702+0000 7f2773800640 1 -- start start 2024-02-02T13:32:39.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 0x7f276c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c160a20 0x7f276c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0cd110 con 0x7f276c0b4ba0 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c1f32b0 con 0x7f276c0d1410 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2773800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0ca390 con 0x7f276c160a20 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33812/0 (socket says 172.21.15.91:33812) 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 -- 172.21.15.91:0/1679663184 learned_addr learned my addr 172.21.15.91:0/1679663184 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2771800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c160a20 0x7f276c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f276be00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 0x7f276c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 -- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c160a20 msgr2=0x7f276c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c160a20 0x7f276c0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 -- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 msgr2=0x7f276c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.703+0000 7f2770e00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 0x7f276c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2770e00640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640094d0 con 0x7f276c0b4ba0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f276be00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 0x7f276c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2770e00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f275c00cf60 tx=0x7f275c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c00d840 con 0x7f276c0b4ba0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c00d9a0 con 0x7f276c0b4ba0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c0ca670 con 0x7f276c0b4ba0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c00dd30 con 0x7f276c0b4ba0 2024-02-02T13:32:39.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.704+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c0cac30 con 0x7f276c0b4ba0 2024-02-02T13:32:39.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.706+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c0cae50 con 0x7f276c0b4ba0 2024-02-02T13:32:39.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.706+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f275c015440 con 0x7f276c0b4ba0 2024-02-02T13:32:39.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.707+0000 7f2768c00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 0x7f274406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:39.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.707+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f275c089280 con 0x7f276c0b4ba0 2024-02-02T13:32:39.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.707+0000 7f276be00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 0x7f274406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:39.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.708+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f2744072020 con 0x7f276c0b4ba0 2024-02-02T13:32:39.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.708+0000 7f276be00640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 0x7f274406c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f276c0c9d00 tx=0x7f2754008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:39.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.712+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f275c01a060 con 0x7f276c0b4ba0 2024-02-02T13:32:39.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.900+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f276c066cf0 con 0x7f276c0b4ba0 2024-02-02T13:32:39.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.901+0000 7f2768c00640 1 -- 172.21.15.91:0/1679663184 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f275c054970 con 0x7f276c0b4ba0 2024-02-02T13:32:39.902 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 msgr2=0x7f274406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 0x7f274406c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f276c0c9d00 tx=0x7f2754008040 comp rx=0 tx=0).stop 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 msgr2=0x7f276c0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f275c00cf60 tx=0x7f275c00cf90 comp rx=0 tx=0).stop 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 shutdown_connections 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f274406a430 0x7f274406c8f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f276c160a20 0x7f276c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f276c0d1410 0x7f276c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 --2- 172.21.15.91:0/1679663184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f276c0b4ba0 0x7f276c0c7960 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.903+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 >> 172.21.15.91:0/1679663184 conn(0x7f276c070480 msgr2=0x7f276c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.904+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 shutdown_connections 2024-02-02T13:32:39.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:39.904+0000 7f2773800640 1 -- 172.21.15.91:0/1679663184 wait complete. 2024-02-02T13:32:39.923 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:32:40.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.135+0000 7efd91200640 1 Processor -- start 2024-02-02T13:32:40.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.136+0000 7efd91200640 1 -- start start 2024-02-02T13:32:40.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.137+0000 7efd91200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd8c151f90 0x7efd8c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.137+0000 7efd91200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd8c06f4f0 0x7efd8c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.137+0000 7efd91200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0b1070 con 0x7efd8c0b0c40 2024-02-02T13:32:40.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.137+0000 7efd91200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c070f80 con 0x7efd8c06f4f0 2024-02-02T13:32:40.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.137+0000 7efd91200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c152370 con 0x7efd8c151f90 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd8c06f4f0 0x7efd8c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd8c06f4f0 0x7efd8c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45716/0 (socket says 172.21.15.91:45716) 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 -- 172.21.15.91:0/1680793495 learned_addr learned my addr 172.21.15.91:0/1680793495 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd89e00640 1 --2- 172.21.15.91:0/1680793495 >> v2:172.21.15.113:3300/0 conn(0x7efd8c151f90 0x7efd8c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 -- 172.21.15.91:0/1680793495 >> v2:172.21.15.113:3300/0 conn(0x7efd8c151f90 msgr2=0x7efd8c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 --2- 172.21.15.91:0/1680793495 >> v2:172.21.15.113:3300/0 conn(0x7efd8c151f90 0x7efd8c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 -- 172.21.15.91:0/1680793495 >> v1:172.21.15.113:6789/0 conn(0x7efd8c0b0c40 legacy=0x7efd8c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:40.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.138+0000 7efd83e00640 1 -- 172.21.15.91:0/1680793495 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0cb010 con 0x7efd8c06f4f0 2024-02-02T13:32:40.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.140+0000 7efd83e00640 1 --2- 172.21.15.91:0/1680793495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c06f4f0 0x7efd8c0b0700 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0cc770 tx=0x7efd7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ce1330ab650f37 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.140+0000 7efd89400640 1 -- 172.21.15.91:0/1680793495 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74031870 con 0x7efd8c06f4f0 2024-02-02T13:32:40.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.140+0000 7efd89400640 1 -- 172.21.15.91:0/1680793495 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd740319d0 con 0x7efd8c06f4f0 2024-02-02T13:32:40.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.140+0000 7efd89400640 1 -- 172.21.15.91:0/1680793495 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74031d60 con 0x7efd8c06f4f0 2024-02-02T13:32:40.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 -- 172.21.15.91:0/1680793495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c06f4f0 msgr2=0x7efd8c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 --2- 172.21.15.91:0/1680793495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c06f4f0 0x7efd8c0b0700 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0cc770 tx=0x7efd7402b910 comp rx=0 tx=0).stop 2024-02-02T13:32:40.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 -- 172.21.15.91:0/1680793495 shutdown_connections 2024-02-02T13:32:40.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 --2- 172.21.15.91:0/1680793495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c06f4f0 0x7efd8c0b0700 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 --2- 172.21.15.91:0/1680793495 >> v2:172.21.15.113:3300/0 conn(0x7efd8c151f90 0x7efd8c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 -- 172.21.15.91:0/1680793495 >> 172.21.15.91:0/1680793495 conn(0x7efd8c0cec80 msgr2=0x7efd8c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:40.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 -- 172.21.15.91:0/1680793495 shutdown_connections 2024-02-02T13:32:40.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.141+0000 7efd91200640 1 -- 172.21.15.91:0/1680793495 wait complete. 2024-02-02T13:32:40.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.142+0000 7efd91200640 1 Processor -- start 2024-02-02T13:32:40.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 -- start start 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 0x7efd8c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 0x7efd8c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c1529f0 con 0x7efd8c151f90 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c152c90 con 0x7efd8c06f4f0 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.143+0000 7efd91200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c152f70 con 0x7efd8c0b0c40 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45730/0 (socket says 172.21.15.91:45730) 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 -- 172.21.15.91:0/1143198043 learned_addr learned my addr 172.21.15.91:0/1143198043 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd89e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 0x7efd8c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd83e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 0x7efd8c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 -- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 msgr2=0x7efd8c14d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 0x7efd8c14d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 -- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 msgr2=0x7efd8c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 0x7efd8c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd8a800640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740094d0 con 0x7efd8c151f90 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd89e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 0x7efd8c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.144+0000 7efd83e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 0x7efd8c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd8a800640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7efd8400b9e0 tx=0x7efd8400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8400caa0 con 0x7efd8c151f90 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8400cc00 con 0x7efd8c151f90 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c153250 con 0x7efd8c151f90 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84014440 con 0x7efd8c151f90 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.145+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c153810 con 0x7efd8c151f90 2024-02-02T13:32:40.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.146+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c153a00 con 0x7efd8c151f90 2024-02-02T13:32:40.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.147+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd8401d3b0 con 0x7efd8c151f90 2024-02-02T13:32:40.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.148+0000 7efd82000640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 0x7efd6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.148+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efd84088900 con 0x7efd8c151f90 2024-02-02T13:32:40.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.148+0000 7efd89e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 0x7efd6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.148+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7efd64072020 con 0x7efd8c151f90 2024-02-02T13:32:40.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.149+0000 7efd89e00640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 0x7efd6406c8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006fd0 tx=0x7efd7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.152+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd84019020 con 0x7efd8c151f90 2024-02-02T13:32:40.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.341+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd8c066cf0 con 0x7efd8c151f90 2024-02-02T13:32:40.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.342+0000 7efd82000640 1 -- 172.21.15.91:0/1143198043 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd84088070 con 0x7efd8c151f90 2024-02-02T13:32:40.344 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:32:40.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.345+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 msgr2=0x7efd6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.345+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 0x7efd6406c8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006fd0 tx=0x7efd7c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:40.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.345+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 msgr2=0x7efd8c1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.345+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7efd8400b9e0 tx=0x7efd8400bea0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.345+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 shutdown_connections 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd6406a430 0x7efd6406c8f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd8c0b0c40 0x7efd8c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd8c151f90 0x7efd8c1524b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 --2- 172.21.15.91:0/1143198043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c06f4f0 0x7efd8c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 >> 172.21.15.91:0/1143198043 conn(0x7efd8c0cec80 msgr2=0x7efd8c0ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 shutdown_connections 2024-02-02T13:32:40.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.346+0000 7efd91200640 1 -- 172.21.15.91:0/1143198043 wait complete. 2024-02-02T13:32:40.365 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:32:40.577 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.576+0000 7f2f17200640 1 Processor -- start 2024-02-02T13:32:40.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.576+0000 7f2f17200640 1 -- start start 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.577+0000 7f2f17200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100d0070 0x7f2f100b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.577+0000 7f2f17200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f100b2260 0x7f2f100d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.577+0000 7f2f17200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100cd930 con 0x7f2f100b6260 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.577+0000 7f2f17200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100b8f90 con 0x7f2f100b2260 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.577+0000 7f2f17200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100d1680 con 0x7f2f100d0070 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f100b2260 0x7f2f100d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f100b2260 0x7f2f100d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45744/0 (socket says 172.21.15.91:45744) 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 -- 172.21.15.91:0/4239648813 learned_addr learned my addr 172.21.15.91:0/4239648813 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f15200640 1 --1- 172.21.15.91:0/4239648813 >> v1:172.21.15.91:6789/0 conn(0x7f2f100b6260 0x7f2f100d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41644/0 (socket says 172.21.15.91:41644) 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f0fe00640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100d0070 0x7f2f100b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 -- 172.21.15.91:0/4239648813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100d0070 msgr2=0x7f2f100b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100d0070 0x7f2f100b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.580 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 -- 172.21.15.91:0/4239648813 >> v1:172.21.15.91:6789/0 conn(0x7f2f100b6260 legacy=0x7f2f100d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.578+0000 7f2f07e00640 1 -- 172.21.15.91:0/4239648813 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100d1960 con 0x7f2f100b2260 2024-02-02T13:32:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.579+0000 7f2f07e00640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f100d0450 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009870 tx=0x7f2ef802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae78b34432a25dcf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f0f400640 1 -- 172.21.15.91:0/4239648813 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8030840 con 0x7f2f100b2260 2024-02-02T13:32:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f0f400640 1 -- 172.21.15.91:0/4239648813 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80309a0 con 0x7f2f100b2260 2024-02-02T13:32:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f0f400640 1 -- 172.21.15.91:0/4239648813 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8030d30 con 0x7f2f100b2260 2024-02-02T13:32:40.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f17200640 1 -- 172.21.15.91:0/4239648813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 msgr2=0x7f2f100d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f17200640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f100d0450 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009870 tx=0x7f2ef802b980 comp rx=0 tx=0).stop 2024-02-02T13:32:40.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.580+0000 7f2f17200640 1 -- 172.21.15.91:0/4239648813 shutdown_connections 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.581+0000 7f2f17200640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f100d0450 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.581+0000 7f2f17200640 1 --2- 172.21.15.91:0/4239648813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100d0070 0x7f2f100b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.581+0000 7f2f17200640 1 -- 172.21.15.91:0/4239648813 >> 172.21.15.91:0/4239648813 conn(0x7f2f10070660 msgr2=0x7f2f10070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.581+0000 7f2f17200640 1 -- 172.21.15.91:0/4239648813 shutdown_connections 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.581+0000 7f2f17200640 1 -- 172.21.15.91:0/4239648813 wait complete. 2024-02-02T13:32:40.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.582+0000 7f2f17200640 1 Processor -- start 2024-02-02T13:32:40.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.583+0000 7f2f17200640 1 -- start start 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.583+0000 7f2f17200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.583+0000 7f2f17200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100b6260 0x7f2f100cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f17200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f100d0070 0x7f2f100d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f17200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1014ca10 con 0x7f2f100b2260 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f17200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1014ccf0 con 0x7f2f100d0070 2024-02-02T13:32:40.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f17200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100cc320 con 0x7f2f100b6260 2024-02-02T13:32:40.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45754/0 (socket says 172.21.15.91:45754) 2024-02-02T13:32:40.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 -- 172.21.15.91:0/2805871170 learned_addr learned my addr 172.21.15.91:0/2805871170 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f07e00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100b6260 0x7f2f100cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 -- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100b6260 msgr2=0x7f2f100cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100b6260 0x7f2f100cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 -- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f100d0070 msgr2=0x7f2f100d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f100d0070 0x7f2f100d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.584+0000 7f2f0fe00640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80094d0 con 0x7f2f100b2260 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f0fe00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d5f0 tx=0x7f2f0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000125f0 con 0x7f2f100b2260 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00012750 con 0x7f2f100b2260 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00012b00 con 0x7f2f100b2260 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f100cc600 con 0x7f2f100b2260 2024-02-02T13:32:40.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.585+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f101f2530 con 0x7f2f100b2260 2024-02-02T13:32:40.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.586+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100cc180 con 0x7f2f100b2260 2024-02-02T13:32:40.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.587+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f00012c60 con 0x7f2f100b2260 2024-02-02T13:32:40.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.588+0000 7f2f0cc00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 0x7f2ee806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:40.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.588+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2f00088df0 con 0x7f2f100b2260 2024-02-02T13:32:40.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.588+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f2ee8072020 con 0x7f2f100b2260 2024-02-02T13:32:40.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.588+0000 7f2f07e00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 0x7f2ee806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:40.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.589+0000 7f2f07e00640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 0x7f2ee806c8f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8036000 tx=0x7f2ef803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:40.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.592+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f100cc180 con 0x7f2f100b2260 2024-02-02T13:32:40.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.782+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f100ccb20 con 0x7f2f100b2260 2024-02-02T13:32:40.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.783+0000 7f2f0cc00640 1 -- 172.21.15.91:0/2805871170 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f100ccb20 con 0x7f2f100b2260 2024-02-02T13:32:40.785 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:32:40.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 msgr2=0x7f2ee806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 0x7f2ee806c8f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8036000 tx=0x7f2ef803b040 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 msgr2=0x7f2f101f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d5f0 tx=0x7f2f0000dab0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 shutdown_connections 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2ee806a430 0x7f2ee806c8f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f100b6260 0x7f2f100cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f100d0070 0x7f2f100d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 --2- 172.21.15.91:0/2805871170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f100b2260 0x7f2f101f1ff0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 >> 172.21.15.91:0/2805871170 conn(0x7f2f10070660 msgr2=0x7f2f100b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 shutdown_connections 2024-02-02T13:32:40.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:40.786+0000 7f2f17200640 1 -- 172.21.15.91:0/2805871170 wait complete. 2024-02-02T13:32:40.808 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:41.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.013+0000 7f6fc2000640 1 Processor -- start 2024-02-02T13:32:41.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.013+0000 7f6fc2000640 1 -- start start 2024-02-02T13:32:41.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6fbc0b0770 0x7f6fbc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc06b5f0 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc06c450 con 0x7f6fbc0b0770 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fc2000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc159b80 con 0x7f6fbc1597a0 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fbbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fbbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45770/0 (socket says 172.21.15.91:45770) 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fbbe00640 1 -- 172.21.15.91:0/2020918369 learned_addr learned my addr 172.21.15.91:0/2020918369 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fbb400640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.015+0000 7f6fb2800640 1 --2- 172.21.15.91:0/2020918369 >> v2:172.21.15.113:3300/0 conn(0x7f6fbc0b0770 0x7f6fbc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.016+0000 7f6fbbe00640 1 -- 172.21.15.91:0/2020918369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 msgr2=0x7f6fbc06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.016+0000 7f6fbbe00640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.016+0000 7f6fbbe00640 1 -- 172.21.15.91:0/2020918369 >> v2:172.21.15.113:3300/0 conn(0x7f6fbc0b0770 msgr2=0x7f6fbc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.016+0000 7f6fbbe00640 1 --2- 172.21.15.91:0/2020918369 >> v2:172.21.15.113:3300/0 conn(0x7f6fbc0b0770 0x7f6fbc0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.016+0000 7f6fbbe00640 1 -- 172.21.15.91:0/2020918369 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc06b840 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.017+0000 7f6fbbe00640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc06db00 tx=0x7f6fb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372fe45b75b390db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.017+0000 7f6fbaa00640 1 -- 172.21.15.91:0/2020918369 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb402ca70 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.017+0000 7f6fbaa00640 1 -- 172.21.15.91:0/2020918369 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb402cbd0 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.017+0000 7f6fbaa00640 1 -- 172.21.15.91:0/2020918369 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40343f0 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 -- 172.21.15.91:0/2020918369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0bd0 msgr2=0x7f6fbc0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc06db00 tx=0x7f6fb402bed0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 -- 172.21.15.91:0/2020918369 shutdown_connections 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0cdbd0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 --2- 172.21.15.91:0/2020918369 >> v2:172.21.15.113:3300/0 conn(0x7f6fbc0b0770 0x7f6fbc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 --2- 172.21.15.91:0/2020918369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 -- 172.21.15.91:0/2020918369 >> 172.21.15.91:0/2020918369 conn(0x7f6fbc0cec80 msgr2=0x7f6fbc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 -- 172.21.15.91:0/2020918369 shutdown_connections 2024-02-02T13:32:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.018+0000 7f6fc2000640 1 -- 172.21.15.91:0/2020918369 wait complete. 2024-02-02T13:32:41.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.020+0000 7f6fc2000640 1 Processor -- start 2024-02-02T13:32:41.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.020+0000 7f6fc2000640 1 -- start start 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc0c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c5630 con 0x7f6fbc0b0770 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc150210 con 0x7f6fbc0b0bd0 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fc2000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc1504f0 con 0x7f6fbc1597a0 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45778/0 (socket says 172.21.15.91:45778) 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 -- 172.21.15.91:0/1775744361 learned_addr learned my addr 172.21.15.91:0/1775744361 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:41.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fb2800640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbbe00640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc0c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 -- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 msgr2=0x7f6fbc0c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc0c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 -- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fbc0b0bd0 msgr2=0x7f6fbc0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.021+0000 7f6fbb400640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009b30 con 0x7f6fbc0b0770 2024-02-02T13:32:41.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fbb400640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d800 tx=0x7f6fac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00b6c0 con 0x7f6fbc0b0770 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac00b820 con 0x7f6fbc0b0770 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00bbb0 con 0x7f6fbc0b0770 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc1507d0 con 0x7f6fbc0b0770 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.022+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0b1f40 con 0x7f6fbc0b0770 2024-02-02T13:32:41.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.023+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc066cf0 con 0x7f6fbc0b0770 2024-02-02T13:32:41.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.024+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6fac00bd10 con 0x7f6fbc0b0770 2024-02-02T13:32:41.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.025+0000 7f6fb3e00640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 0x7f6f9406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.025+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6fac088e10 con 0x7f6fbc0b0770 2024-02-02T13:32:41.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.025+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f6f94072020 con 0x7f6fbc0b0770 2024-02-02T13:32:41.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.025+0000 7f6fb2800640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 0x7f6f9406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.026+0000 7f6fb2800640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 0x7f6f9406c8f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4006fd0 tx=0x7f6fa4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.030+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6fac054450 con 0x7f6fbc0b0770 2024-02-02T13:32:41.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.257+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fbc0cd830 con 0x7f6fbc0b0770 2024-02-02T13:32:41.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.258+0000 7f6fb3e00640 1 -- 172.21.15.91:0/1775744361 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f6fac057a70 con 0x7f6fbc0b0770 2024-02-02T13:32:41.259 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:41.259 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:41.260 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 msgr2=0x7f6f9406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 0x7f6f9406c8f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4006fd0 tx=0x7f6fa4008040 comp rx=0 tx=0).stop 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 msgr2=0x7f6fbc0c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d800 tx=0x7f6fac00dcc0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 shutdown_connections 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6f9406a430 0x7f6f9406c8f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fbc1597a0 0x7f6fbc0c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fbc0b0bd0 0x7f6fbc0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 --2- 172.21.15.91:0/1775744361 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6fbc0b0770 0x7f6fbc0c50f0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 >> 172.21.15.91:0/1775744361 conn(0x7f6fbc0cec80 msgr2=0x7f6fbc0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:41.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 shutdown_connections 2024-02-02T13:32:41.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.260+0000 7f6fc2000640 1 -- 172.21.15.91:0/1775744361 wait complete. 2024-02-02T13:32:41.281 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.487+0000 7f92d3000640 1 Processor -- start 2024-02-02T13:32:41.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.487+0000 7f92d3000640 1 -- start start 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.488+0000 7f92d3000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 0x7f92cc06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.488+0000 7f92d3000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92cc0cd690 0x7f92cc0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.488+0000 7f92d3000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc06b5f0 con 0x7f92cc0c9240 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.488+0000 7f92d3000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0ce030 con 0x7f92cc0cd690 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92d3000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc06c450 con 0x7f92cc1597a0 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92cc0cd690 0x7f92cc0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f92cc0cd690 0x7f92cc0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45790/0 (socket says 172.21.15.91:45790) 2024-02-02T13:32:41.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 -- 172.21.15.91:0/2556988165 learned_addr learned my addr 172.21.15.91:0/2556988165 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:41.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92d1000640 1 --1- 172.21.15.91:0/2556988165 >> v1:172.21.15.91:6789/0 conn(0x7f92cc0c9240 0x7f92cc06f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41646/0 (socket says 172.21.15.91:41646) 2024-02-02T13:32:41.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 -- 172.21.15.91:0/2556988165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 msgr2=0x7f92cc06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cbe00640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 0x7f92cc06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 0x7f92cc06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 -- 172.21.15.91:0/2556988165 >> v1:172.21.15.91:6789/0 conn(0x7f92cc0c9240 legacy=0x7f92cc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cb400640 1 -- 172.21.15.91:0/2556988165 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc159b80 con 0x7f92cc0cd690 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92caa00640 1 -- 172.21.15.91:0/2556988165 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877919227 0 0) 0x7f92cc06b5f0 con 0x7f92cc0c9240 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.489+0000 7f92cbe00640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 0x7f92cc06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:41.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.490+0000 7f92cb400640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0cd690 0x7f92cc0cdaf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009e50 tx=0x7f92b402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0217c028c2d104 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92caa00640 1 -- 172.21.15.91:0/2556988165 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b402cb00 con 0x7f92cc0cd690 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92caa00640 1 -- 172.21.15.91:0/2556988165 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b402cc60 con 0x7f92cc0cd690 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92caa00640 1 -- 172.21.15.91:0/2556988165 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40343f0 con 0x7f92cc0cd690 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 -- 172.21.15.91:0/2556988165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0cd690 msgr2=0x7f92cc0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0cd690 0x7f92cc0cdaf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009e50 tx=0x7f92b402bf60 comp rx=0 tx=0).stop 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 -- 172.21.15.91:0/2556988165 shutdown_connections 2024-02-02T13:32:41.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0cd690 0x7f92cc0cdaf0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 --2- 172.21.15.91:0/2556988165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc1597a0 0x7f92cc06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.491+0000 7f92d3000640 1 -- 172.21.15.91:0/2556988165 >> 172.21.15.91:0/2556988165 conn(0x7f92cc0cec80 msgr2=0x7f92cc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:41.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.492+0000 7f92d3000640 1 -- 172.21.15.91:0/2556988165 shutdown_connections 2024-02-02T13:32:41.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.492+0000 7f92d3000640 1 -- 172.21.15.91:0/2556988165 wait complete. 2024-02-02T13:32:41.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.493+0000 7f92d3000640 1 Processor -- start 2024-02-02T13:32:41.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.493+0000 7f92d3000640 1 -- start start 2024-02-02T13:32:41.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc0cd690 0x7f92cc15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92cc1597a0 0x7f92cc15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0c38a0 con 0x7f92cc0c9240 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc1475e0 con 0x7f92cc1597a0 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.494+0000 7f92d3000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0b03b0 con 0x7f92cc0cd690 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45804/0 (socket says 172.21.15.91:45804) 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 -- 172.21.15.91:0/1519685203 learned_addr learned my addr 172.21.15.91:0/1519685203 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 -- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc0cd690 msgr2=0x7f92cc15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc0cd690 0x7f92cc15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 -- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92cc1597a0 msgr2=0x7f92cc15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92cc1597a0 0x7f92cc15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4009b30 con 0x7f92cc0c9240 2024-02-02T13:32:41.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92cbe00640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00c820 tx=0x7f92bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.495+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc007a30 con 0x7f92cc0c9240 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.496+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc007b90 con 0x7f92cc0c9240 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.496+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc00ce70 con 0x7f92cc0c9240 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.496+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc0b0690 con 0x7f92cc0c9240 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.496+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc0c3dd0 con 0x7f92cc0c9240 2024-02-02T13:32:41.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.497+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc0b0820 con 0x7f92cc0c9240 2024-02-02T13:32:41.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.498+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92bc0113f0 con 0x7f92cc0c9240 2024-02-02T13:32:41.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.499+0000 7f92c3e00640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 0x7f92a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.499+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f92bc088be0 con 0x7f92cc0c9240 2024-02-02T13:32:41.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.499+0000 7f92cb400640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 0x7f92a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.499+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f92a4072020 con 0x7f92cc0c9240 2024-02-02T13:32:41.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.500+0000 7f92cb400640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 0x7f92a406c8f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f92b4035000 tx=0x7f92b402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.503+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f92bc01f060 con 0x7f92cc0c9240 2024-02-02T13:32:41.694 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.693+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92cc0c3fc0 con 0x7f92cc0c9240 2024-02-02T13:32:41.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.696+0000 7f92c3e00640 1 -- 172.21.15.91:0/1519685203 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f92bc085f80 con 0x7f92cc0c9240 2024-02-02T13:32:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:41.698 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":149,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:39.421708+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:41.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.698+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 msgr2=0x7f92a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.698+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 0x7f92a406c8f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f92b4035000 tx=0x7f92b402e000 comp rx=0 tx=0).stop 2024-02-02T13:32:41.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.698+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 msgr2=0x7f92cc1470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.698+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00c820 tx=0x7f92bc00cc20 comp rx=0 tx=0).stop 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 shutdown_connections 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f92a406a430 0x7f92a406c8f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92cc0cd690 0x7f92cc15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92cc1597a0 0x7f92cc15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 --2- 172.21.15.91:0/1519685203 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f92cc0c9240 0x7f92cc1470a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 >> 172.21.15.91:0/1519685203 conn(0x7f92cc0cec80 msgr2=0x7f92cc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 shutdown_connections 2024-02-02T13:32:41.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.699+0000 7f92d3000640 1 -- 172.21.15.91:0/1519685203 wait complete. 2024-02-02T13:32:41.719 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:32:41.719 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:41.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.928+0000 7f734e200640 1 Processor -- start 2024-02-02T13:32:41.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.928+0000 7f734e200640 1 -- start start 2024-02-02T13:32:41.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f734e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f734806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f734e200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734815d870 con 0x7f73480b0700 2024-02-02T13:32:41.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f734e200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734806bce0 con 0x7f734815d490 2024-02-02T13:32:41.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f734e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7348161220 con 0x7f73480acfe0 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f7347e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f73480b0700 0x7f73480c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41652/0 (socket says 172.21.15.91:41652) 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.930+0000 7f7347e00640 1 -- 172.21.15.91:0/1649501982 learned_addr learned my addr 172.21.15.91:0/1649501982 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f733e800640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f734806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610615322 0 0) 0x7f734815d870 con 0x7f73480b0700 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7328003520 con 0x7f73480b0700 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746069370 0 0) 0x7f7328003520 con 0x7f73480b0700 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 msgr2=0x7f734806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f734806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 >> v1:172.21.15.113:6789/0 conn(0x7f734815d490 legacy=0x7f73480ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480ad490 con 0x7f73480b0700 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f73400019f0 con 0x7f73480b0700 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 >> v1:172.21.15.91:6789/0 conn(0x7f73480b0700 legacy=0x7f73480c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f73280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7328003f20 0x7f73280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7328005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348161220 con 0x7f7328004d00 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732800ba60 con 0x7f7328003f20 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.931+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732800bc20 con 0x7f73480acfe0 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7328005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347400640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f73280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 -- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 msgr2=0x7f73280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f733e800640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7328003f20 0x7f73280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f73280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 -- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7328003f20 msgr2=0x7f73280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7328003f20 0x7f73280047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347e00640 1 -- 172.21.15.91:0/1649501982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328003520 con 0x7f7328004d00 2024-02-02T13:32:41.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.932+0000 7f7347400640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480acfe0 0x7f73280039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:41.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.933+0000 7f7347e00640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7328005180 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f734000a830 tx=0x7f73280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0b632b2d0b031b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.933+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400072d0 con 0x7f7328004d00 2024-02-02T13:32:41.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.933+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734002f070 con 0x7f7328004d00 2024-02-02T13:32:41.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.933+0000 7f7346a00640 1 -- 172.21.15.91:0/1649501982 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400076f0 con 0x7f7328004d00 2024-02-02T13:32:41.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 -- 172.21.15.91:0/1649501982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 msgr2=0x7f7328005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 --2- 172.21.15.91:0/1649501982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7328005180 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f734000a830 tx=0x7f73280057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f7347e00640 1 -- 172.21.15.91:0/1649501982 reap_dead start 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 -- 172.21.15.91:0/1649501982 shutdown_connections 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 -- 172.21.15.91:0/1649501982 >> 172.21.15.91:0/1649501982 conn(0x7f73480cec80 msgr2=0x7f73480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 -- 172.21.15.91:0/1649501982 shutdown_connections 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.934+0000 7f734e200640 1 -- 172.21.15.91:0/1649501982 wait complete. 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.936+0000 7f734e200640 1 Processor -- start 2024-02-02T13:32:41.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.936+0000 7f734e200640 1 -- start start 2024-02-02T13:32:41.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7348156d50 0x7f73480b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 0x7f73480b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815f260 con 0x7f7328004d00 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480c3150 con 0x7f7348156d50 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f734e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7348157790 con 0x7f73480b41b0 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f7347400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f7347400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45828/0 (socket says 172.21.15.91:45828) 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.937+0000 7f7347400640 1 -- 172.21.15.91:0/3849212591 learned_addr learned my addr 172.21.15.91:0/3849212591 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 -- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 msgr2=0x7f73480b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f733e800640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7348156d50 0x7f73480b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347e00640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 0x7f73480b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 0x7f73480b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 -- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7348156d50 msgr2=0x7f73480b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7348156d50 0x7f73480b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480b4770 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347e00640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 0x7f73480b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f7347400640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f733800d7d0 tx=0x7f733800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338017070 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.938+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338005070 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.939+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734815e8c0 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.939+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005400 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.939+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734815ee80 con 0x7f7328004d00 2024-02-02T13:32:41.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.940+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7348073830 con 0x7f7328004d00 2024-02-02T13:32:41.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.941+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7338007030 con 0x7f7328004d00 2024-02-02T13:32:41.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.942+0000 7f733fe00640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 0x7f732006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:41.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.942+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7338088190 con 0x7f7328004d00 2024-02-02T13:32:41.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.942+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f7320072050 con 0x7f7328004d00 2024-02-02T13:32:41.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.942+0000 7f733e800640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 0x7f732006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:41.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.943+0000 7f733e800640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 0x7f732006c920 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f734806f840 tx=0x7f7330008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:41.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:41.946+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7338053730 con 0x7f7328004d00 2024-02-02T13:32:42.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.173+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f734815f4d0 con 0x7f7328004d00 2024-02-02T13:32:42.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.174+0000 7f733fe00640 1 -- 172.21.15.91:0/3849212591 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f7338056d50 con 0x7f7328004d00 2024-02-02T13:32:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:42.176 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:42.176 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:42.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 msgr2=0x7f732006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 0x7f732006c920 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f734806f840 tx=0x7f7330008040 comp rx=0 tx=0).stop 2024-02-02T13:32:42.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 msgr2=0x7f7348157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f733800d7d0 tx=0x7f733800dc90 comp rx=0 tx=0).stop 2024-02-02T13:32:42.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 shutdown_connections 2024-02-02T13:32:42.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f732006a460 0x7f732006c920 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73480b41b0 0x7f73480b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7348156d50 0x7f73480b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 --2- 172.21.15.91:0/3849212591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7328004d00 0x7f7348157250 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 >> 172.21.15.91:0/3849212591 conn(0x7f73480cec80 msgr2=0x7f73480c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.177+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 shutdown_connections 2024-02-02T13:32:42.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.178+0000 7f734e200640 1 -- 172.21.15.91:0/3849212591 wait complete. 2024-02-02T13:32:42.198 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:42.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.406+0000 7f9abc800640 1 Processor -- start 2024-02-02T13:32:42.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.407+0000 7f9abc800640 1 -- start start 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9abc800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ab40aeea0 0x7f9ab4168240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9abc800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40b1980 con 0x7f9ab415d590 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9abc800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40b1700 con 0x7f9ab40b1ba0 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9abc800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40d07b0 con 0x7f9ab40aeea0 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9ab9e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9ab415d590 0x7f9ab40b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41666/0 (socket says 172.21.15.91:41666) 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.408+0000 7f9ab9e00640 1 -- 172.21.15.91:0/1781069530 learned_addr learned my addr 172.21.15.91:0/1781069530 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9aba800640 1 --2- 172.21.15.91:0/1781069530 >> v2:172.21.15.113:3300/0 conn(0x7f9ab40aeea0 0x7f9ab4168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563897136 0 0) 0x7f9ab40b1980 con 0x7f9ab415d590 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a94003520 con 0x7f9ab415d590 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1708092481 0 0) 0x7f9a94003520 con 0x7f9ab415d590 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 >> v2:172.21.15.113:3300/0 conn(0x7f9ab40aeea0 msgr2=0x7f9ab4168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:42.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 --2- 172.21.15.91:0/1781069530 >> v2:172.21.15.113:3300/0 conn(0x7f9ab40aeea0 0x7f9ab4168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 >> v1:172.21.15.113:6789/0 conn(0x7f9ab40b1ba0 legacy=0x7f9ab40b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40cff10 con 0x7f9ab415d590 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9aa4001670 con 0x7f9ab415d590 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 >> v1:172.21.15.91:6789/0 conn(0x7f9ab415d590 legacy=0x7f9ab40b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940039a0 0x7f9a94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 0x7f9a94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a940052a0 0x7f9a9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40d07b0 con 0x7f9a940047c0 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400c1a0 con 0x7f9a940052a0 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.409+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400c360 con 0x7f9a940039a0 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 0x7f9a94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9aba800640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a940052a0 0x7f9a9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9e00640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940039a0 0x7f9a94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 -- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940039a0 msgr2=0x7f9a94003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940039a0 0x7f9a94003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 -- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a940052a0 msgr2=0x7f9a9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a940052a0 0x7f9a9400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9ab9400640 1 -- 172.21.15.91:0/1781069530 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94003520 con 0x7f9a940047c0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.410+0000 7f9aba800640 1 -- 172.21.15.91:0/1781069530 reap_dead start 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.411+0000 7f9ab9400640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 0x7f9a94004c40 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4070eb0 tx=0x7f9a940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593ff55f870e2e40 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.411+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0052d0 con 0x7f9a940047c0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.411+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c02e050 con 0x7f9a940047c0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.411+0000 7f9ab8a00640 1 -- 172.21.15.91:0/1781069530 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c003830 con 0x7f9a940047c0 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 -- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 msgr2=0x7f9a94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 0x7f9a94004c40 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4070eb0 tx=0x7f9a940057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 -- 172.21.15.91:0/1781069530 shutdown_connections 2024-02-02T13:32:42.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 --2- 172.21.15.91:0/1781069530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a940047c0 0x7f9a94004c40 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 -- 172.21.15.91:0/1781069530 >> 172.21.15.91:0/1781069530 conn(0x7f9ab40d0fa0 msgr2=0x7f9ab40d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:42.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 -- 172.21.15.91:0/1781069530 shutdown_connections 2024-02-02T13:32:42.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.412+0000 7f9abc800640 1 -- 172.21.15.91:0/1781069530 wait complete. 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.414+0000 7f9abc800640 1 Processor -- start 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.414+0000 7f9abc800640 1 -- start start 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940047c0 0x7f9ab41f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab415d970 0x7f9ab415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab415ea80 con 0x7f9ab415bab0 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab415c480 con 0x7f9ab415d970 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9abc800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab41f6d50 con 0x7f9a940047c0 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9aba800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9aba800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45840/0 (socket says 172.21.15.91:45840) 2024-02-02T13:32:42.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9aba800640 1 -- 172.21.15.91:0/1146271941 learned_addr learned my addr 172.21.15.91:0/1146271941 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.415+0000 7f9ab9e00640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940047c0 0x7f9ab41f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9ab9400640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab415d970 0x7f9ab415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 -- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940047c0 msgr2=0x7f9ab41f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940047c0 0x7f9ab41f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 -- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab415d970 msgr2=0x7f9ab415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab415d970 0x7f9ab415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab415de00 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9aba800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4071dd0 tx=0x7f9aac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00e070 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac011db0 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac003990 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.416+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab415e0e0 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.417+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab415e6a0 con 0x7f9ab415bab0 2024-02-02T13:32:42.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.418+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab415ebc0 con 0x7f9ab415bab0 2024-02-02T13:32:42.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.419+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9aac003af0 con 0x7f9ab415bab0 2024-02-02T13:32:42.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.420+0000 7f9ab2000640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 0x7f9a8c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.420+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9aac089090 con 0x7f9ab415bab0 2024-02-02T13:32:42.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.421+0000 7f9ab9e00640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 0x7f9a8c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.421+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f9a8c072020 con 0x7f9ab415bab0 2024-02-02T13:32:42.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.421+0000 7f9ab9e00640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 0x7f9a8c06c8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ab404f8f0 tx=0x7f9aa4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.425+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9ab415ebc0 con 0x7f9ab415bab0 2024-02-02T13:32:42.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.649+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab415ebc0 con 0x7f9ab415bab0 2024-02-02T13:32:42.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.650+0000 7f9ab2000640 1 -- 172.21.15.91:0/1146271941 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f9ab415ebc0 con 0x7f9ab415bab0 2024-02-02T13:32:42.651 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:42.651 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:00.544990+0000","modified":"2024-02-02T13:32:03.563018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7584":{"gid":7584,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/1090230947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1090230947},{"type":"v1","addr":"172.21.15.91:6837","nonce":1090230947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-02-02T13:32:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 64 2024-02-02T13:32:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 msgr2=0x7f9a8c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 0x7f9a8c06c8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ab404f8f0 tx=0x7f9aa4009040 comp rx=0 tx=0).stop 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 msgr2=0x7f9ab415bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4071dd0 tx=0x7f9aac0115a0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 shutdown_connections 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9a8c06a430 0x7f9a8c06c8f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9ab415bab0 0x7f9ab415bf40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab415d970 0x7f9ab415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 --2- 172.21.15.91:0/1146271941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a940047c0 0x7f9ab41f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.652+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 >> 172.21.15.91:0/1146271941 conn(0x7f9ab40d0fa0 msgr2=0x7f9ab406d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:42.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.653+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 shutdown_connections 2024-02-02T13:32:42.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.653+0000 7f9abc800640 1 -- 172.21.15.91:0/1146271941 wait complete. 2024-02-02T13:32:42.671 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.879+0000 7f398a800640 1 Processor -- start 2024-02-02T13:32:42.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.880+0000 7f398a800640 1 -- start start 2024-02-02T13:32:42.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f398a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f398415a010 0x7f39840c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f398a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f398a800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39840cc7c0 con 0x7f39840ce130 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f398a800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39840b10d0 con 0x7f398415a010 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f398a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840ce930 con 0x7f39840b0700 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f3983400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f398415a010 0x7f39840c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f397a800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f39840ce130 0x7f39840ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41676/0 (socket says 172.21.15.91:41676) 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f3983400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f398415a010 0x7f39840c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45850/0 (socket says 172.21.15.91:45850) 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.881+0000 7f397a800640 1 -- 172.21.15.91:0/511685105 learned_addr learned my addr 172.21.15.91:0/511685105 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.882+0000 7f3983400640 1 -- 172.21.15.91:0/511685105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 msgr2=0x7f39840b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.882+0000 7f3983400640 1 --2- 172.21.15.91:0/511685105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.882+0000 7f3983400640 1 -- 172.21.15.91:0/511685105 >> v1:172.21.15.91:6789/0 conn(0x7f39840ce130 legacy=0x7f39840ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.882+0000 7f3983400640 1 -- 172.21.15.91:0/511685105 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840cd640 con 0x7f398415a010 2024-02-02T13:32:42.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.882+0000 7f3982a00640 1 -- 172.21.15.91:0/511685105 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484817549 0 0) 0x7f39840cc7c0 con 0x7f39840ce130 2024-02-02T13:32:42.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.883+0000 7f3983400640 1 --2- 172.21.15.91:0/511685105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f39840c9670 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f397400ba40 tx=0x7f397402e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e8f8434e65c741 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.883+0000 7f3982a00640 1 -- 172.21.15.91:0/511685105 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397402ee30 con 0x7f398415a010 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.883+0000 7f3982a00640 1 -- 172.21.15.91:0/511685105 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3974036440 con 0x7f398415a010 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f3982a00640 1 -- 172.21.15.91:0/511685105 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39740367d0 con 0x7f398415a010 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 -- 172.21.15.91:0/511685105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 msgr2=0x7f39840c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 --2- 172.21.15.91:0/511685105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f39840c9670 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f397400ba40 tx=0x7f397402e080 comp rx=0 tx=0).stop 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 -- 172.21.15.91:0/511685105 shutdown_connections 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 --2- 172.21.15.91:0/511685105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 --2- 172.21.15.91:0/511685105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f39840c9670 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 -- 172.21.15.91:0/511685105 >> 172.21.15.91:0/511685105 conn(0x7f39840cec80 msgr2=0x7f39840cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 -- 172.21.15.91:0/511685105 shutdown_connections 2024-02-02T13:32:42.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.884+0000 7f398a800640 1 -- 172.21.15.91:0/511685105 wait complete. 2024-02-02T13:32:42.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.886+0000 7f398a800640 1 Processor -- start 2024-02-02T13:32:42.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.886+0000 7f398a800640 1 -- start start 2024-02-02T13:32:42.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 0x7f39841f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39841f3770 con 0x7f398415a010 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398415b5e0 con 0x7f39840ce130 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f398a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398415b8c0 con 0x7f39840b0700 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f3983e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f3983e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45854/0 (socket says 172.21.15.91:45854) 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.887+0000 7f3983e00640 1 -- 172.21.15.91:0/101897883 learned_addr learned my addr 172.21.15.91:0/101897883 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983400640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f397a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 0x7f39841f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 -- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 msgr2=0x7f39840b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 -- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 msgr2=0x7f39841f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 0x7f39841f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397400b6a0 con 0x7f398415a010 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f397a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 0x7f39841f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:42.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f3983e00640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b9e0 tx=0x7f397c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00caa0 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c00cc20 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c014440 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.888+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398415bba0 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.889+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39841f3430 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.890+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398415c0c0 con 0x7f398415a010 2024-02-02T13:32:42.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.891+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f397c01d3b0 con 0x7f398415a010 2024-02-02T13:32:42.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.891+0000 7f397be00640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 0x7f395c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:42.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.892+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f397c088870 con 0x7f398415a010 2024-02-02T13:32:42.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.892+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f395c072020 con 0x7f398415a010 2024-02-02T13:32:42.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.892+0000 7f3983400640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 0x7f395c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:42.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.892+0000 7f3983400640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 0x7f395c06c8f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3974038000 tx=0x7f3974008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:42.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:42.896+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f397c019020 con 0x7f398415a010 2024-02-02T13:32:43.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.070+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3984066cf0 con 0x7f398415a010 2024-02-02T13:32:43.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.073+0000 7f397be00640 1 -- 172.21.15.91:0/101897883 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f398415c0c0 con 0x7f398415a010 2024-02-02T13:32:43.075 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:43.075 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":149,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:39.421708+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:31:58.576657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-02-02T13:31:59.552886+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 msgr2=0x7f395c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 0x7f395c06c8f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3974038000 tx=0x7f3974008040 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 msgr2=0x7f3984156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b9e0 tx=0x7f397c00bea0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 shutdown_connections 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f395c06a430 0x7f395c06c8f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f398415a010 0x7f3984156060 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39840ce130 0x7f39841f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 --2- 172.21.15.91:0/101897883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39840b0700 0x7f39840b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 >> 172.21.15.91:0/101897883 conn(0x7f39840cec80 msgr2=0x7f39840cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 shutdown_connections 2024-02-02T13:32:43.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.075+0000 7f398a800640 1 -- 172.21.15.91:0/101897883 wait complete. 2024-02-02T13:32:43.096 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:32:43.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.288+0000 7f6afa000640 1 Processor -- start 2024-02-02T13:32:43.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.288+0000 7f6afa000640 1 -- start start 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6afa000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40c9a80 0x7f6af40c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6afa000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6af415d8a0 0x7f6af415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6afa000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af406faa0 con 0x7f6af40ce4d0 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6afa000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af415e800 con 0x7f6af415d8a0 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6afa000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4070ec0 con 0x7f6af40c9a80 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6af8c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6af40ce4d0 0x7f6af40707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41680/0 (socket says 172.21.15.91:41680) 2024-02-02T13:32:43.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.289+0000 7f6af8c00640 1 -- 172.21.15.91:0/2341399670 learned_addr learned my addr 172.21.15.91:0/2341399670 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:43.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808204855 0 0) 0x7f6af406faa0 con 0x7f6af40ce4d0 2024-02-02T13:32:43.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad4003520 con 0x7f6af40ce4d0 2024-02-02T13:32:43.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af9600640 1 --2- 172.21.15.91:0/2341399670 >> v2:172.21.15.113:3300/0 conn(0x7f6af415d8a0 0x7f6af415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40c9a80 0x7f6af40c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370529738 0 0) 0x7f6ad4003520 con 0x7f6af40ce4d0 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40c9a80 msgr2=0x7f6af40c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40c9a80 0x7f6af40c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 >> v2:172.21.15.113:3300/0 conn(0x7f6af415d8a0 msgr2=0x7f6af415dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 --2- 172.21.15.91:0/2341399670 >> v2:172.21.15.113:3300/0 conn(0x7f6af415d8a0 0x7f6af415dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40ca420 con 0x7f6af40ce4d0 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6ae4001670 con 0x7f6af40ce4d0 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 >> v1:172.21.15.91:6789/0 conn(0x7f6af40ce4d0 legacy=0x7f6af40707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4003970 0x7f6ad4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ad40047c0 0x7f6ad4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ad4005180 0x7f6ad400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af415e800 con 0x7f6ad40047c0 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4070ec0 con 0x7f6ad4003970 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad400e280 con 0x7f6ad4005180 2024-02-02T13:32:43.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.290+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40c9a80 0x7f6af40c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ad40047c0 0x7f6ad4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 -- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ad4005180 msgr2=0x7f6ad400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ad4005180 0x7f6ad400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af8c00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4003970 0x7f6ad4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 -- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4003970 msgr2=0x7f6ad4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4003970 0x7f6ad4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.291+0000 7f6af3e00640 1 -- 172.21.15.91:0/2341399670 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4003520 con 0x7f6ad40047c0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.292+0000 7f6af3e00640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ad40047c0 0x7f6ad4004c40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6af40711b0 tx=0x7f6ad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31a80f935ac6e10 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.292+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc032620 con 0x7f6ad40047c0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.292+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0327a0 con 0x7f6ad40047c0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.292+0000 7f6af3400640 1 -- 172.21.15.91:0/2341399670 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc032b50 con 0x7f6ad40047c0 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 -- 172.21.15.91:0/2341399670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ad40047c0 msgr2=0x7f6ad4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 --2- 172.21.15.91:0/2341399670 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ad40047c0 0x7f6ad4004c40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6af40711b0 tx=0x7f6ad40057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6af9600640 1 -- 172.21.15.91:0/2341399670 reap_dead start 2024-02-02T13:32:43.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 -- 172.21.15.91:0/2341399670 shutdown_connections 2024-02-02T13:32:43.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 -- 172.21.15.91:0/2341399670 >> 172.21.15.91:0/2341399670 conn(0x7f6af40cb4a0 msgr2=0x7f6af40cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:43.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 -- 172.21.15.91:0/2341399670 shutdown_connections 2024-02-02T13:32:43.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.293+0000 7f6afa000640 1 -- 172.21.15.91:0/2341399670 wait complete. 2024-02-02T13:32:43.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.294+0000 7f6afa000640 1 Processor -- start 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 -- start start 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad40047c0 0x7f6af40c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40acad0 0x7f6af40ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c28d0 con 0x7f6af414f150 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c6e70 con 0x7f6ad40047c0 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.295+0000 7f6afa000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c7150 con 0x7f6af40acad0 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45866/0 (socket says 172.21.15.91:45866) 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 -- 172.21.15.91:0/2083983875 learned_addr learned my addr 172.21.15.91:0/2083983875 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 -- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40acad0 msgr2=0x7f6af40ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40acad0 0x7f6af40ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 -- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad40047c0 msgr2=0x7f6af40c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad40047c0 0x7f6af40c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40c7430 con 0x7f6af414f150 2024-02-02T13:32:43.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.296+0000 7f6af9600640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6af415dc80 tx=0x7f6aec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.297+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00ed90 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.297+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec014420 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.297+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0147d0 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.297+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40c7710 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.297+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af40c6c20 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.298+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af404dff0 con 0x7f6af414f150 2024-02-02T13:32:43.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.299+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6aec014930 con 0x7f6af414f150 2024-02-02T13:32:43.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.299+0000 7f6af0c00640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 0x7f6acc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.300+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6aec088e40 con 0x7f6af414f150 2024-02-02T13:32:43.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.300+0000 7f6af8c00640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 0x7f6acc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.300+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f6acc072020 con 0x7f6af414f150 2024-02-02T13:32:43.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.300+0000 7f6af8c00640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 0x7f6acc06c8f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000ff0 tx=0x7f6ae4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.304+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6aec054480 con 0x7f6af414f150 2024-02-02T13:32:43.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.516+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6af40c2a10 con 0x7f6af414f150 2024-02-02T13:32:43.519 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:32:43.517+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:32:43.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.521+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(150..150 src has 1..150) v4 ==== 374+0+0 (secure 0 0 0) 0x7f6aec058140 con 0x7f6af414f150 2024-02-02T13:32:43.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.521+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f6acc0728d0 con 0x7f6af414f150 2024-02-02T13:32:43.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.536+0000 7f6af0c00640 1 -- 172.21.15.91:0/2083983875 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6aec057aa0 con 0x7f6af414f150 2024-02-02T13:32:43.537 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 msgr2=0x7f6acc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 0x7f6acc06c8f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000ff0 tx=0x7f6ae4009040 comp rx=0 tx=0).stop 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 msgr2=0x7f6af40cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6af415dc80 tx=0x7f6aec00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 shutdown_connections 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6acc06a430 0x7f6acc06c8f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6af40acad0 0x7f6af40ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6af414f150 0x7f6af40cd2a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 --2- 172.21.15.91:0/2083983875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad40047c0 0x7f6af40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 >> 172.21.15.91:0/2083983875 conn(0x7f6af40cb4a0 msgr2=0x7f6af415cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:43.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 shutdown_connections 2024-02-02T13:32:43.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.539+0000 7f6afa000640 1 -- 172.21.15.91:0/2083983875 wait complete. 2024-02-02T13:32:43.547 INFO:tasks.ceph.mds.a.smithi091.stderr: -9099> 2024-02-02T13:32:06.043+0000 7fcfbb000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:32:43.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:32:43.598 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:32:43.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.732+0000 7f5d50000640 1 Processor -- start 2024-02-02T13:32:43.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.732+0000 7f5d50000640 1 -- start start 2024-02-02T13:32:43.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d50000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 0x7f5d480b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d50000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5d480b0bd0 0x7f5d480c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d50000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480b1e60 con 0x7f5d4815b2f0 2024-02-02T13:32:43.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d50000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4815fdc0 con 0x7f5d480b0bd0 2024-02-02T13:32:43.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d50000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4815b6d0 con 0x7f5d480b1a00 2024-02-02T13:32:43.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d4d600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5d4815b2f0 0x7f5d480711f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41698/0 (socket says 172.21.15.91:41698) 2024-02-02T13:32:43.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d4d600640 1 -- 172.21.15.91:0/3484989563 learned_addr learned my addr 172.21.15.91:0/3484989563 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:43.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d4e000640 1 --2- 172.21.15.91:0/3484989563 >> v2:172.21.15.91:3300/0 conn(0x7f5d480b0bd0 0x7f5d480c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 0x7f5d480b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.734+0000 7f5d47e00640 1 -- 172.21.15.91:0/3484989563 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061319388 0 0) 0x7f5d480b1e60 con 0x7f5d4815b2f0 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d47e00640 1 -- 172.21.15.91:0/3484989563 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d28003520 con 0x7f5d4815b2f0 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d4e000640 1 -- 172.21.15.91:0/3484989563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 msgr2=0x7f5d480b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d4e000640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 0x7f5d480b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d4e000640 1 -- 172.21.15.91:0/3484989563 >> v1:172.21.15.91:6789/0 conn(0x7f5d4815b2f0 legacy=0x7f5d480711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d4e000640 1 -- 172.21.15.91:0/3484989563 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480b1030 con 0x7f5d480b0bd0 2024-02-02T13:32:43.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.735+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 0x7f5d480b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:32:43.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.736+0000 7f5d4e000640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d480c9480 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5d400097d0 tx=0x7f5d4002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5918db0a79471286 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.736+0000 7f5d47e00640 1 -- 172.21.15.91:0/3484989563 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40031770 con 0x7f5d480b0bd0 2024-02-02T13:32:43.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.736+0000 7f5d47e00640 1 -- 172.21.15.91:0/3484989563 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d400318d0 con 0x7f5d480b0bd0 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.736+0000 7f5d47e00640 1 -- 172.21.15.91:0/3484989563 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40031c60 con 0x7f5d480b0bd0 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 -- 172.21.15.91:0/3484989563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 msgr2=0x7f5d480c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d480c9480 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5d400097d0 tx=0x7f5d4002b810 comp rx=0 tx=0).stop 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 -- 172.21.15.91:0/3484989563 shutdown_connections 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d480c9480 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 --2- 172.21.15.91:0/3484989563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d480b1a00 0x7f5d480b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 -- 172.21.15.91:0/3484989563 >> 172.21.15.91:0/3484989563 conn(0x7f5d480ceb90 msgr2=0x7f5d480cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 -- 172.21.15.91:0/3484989563 shutdown_connections 2024-02-02T13:32:43.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.737+0000 7f5d50000640 1 -- 172.21.15.91:0/3484989563 wait complete. 2024-02-02T13:32:43.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.739+0000 7f5d50000640 1 Processor -- start 2024-02-02T13:32:43.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.739+0000 7f5d50000640 1 -- start start 2024-02-02T13:32:43.742 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 0x7f5d48155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 0x7f5d480add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480ae290 con 0x7f5d480b0bd0 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480accb0 con 0x7f5d480b1a00 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d50000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480acf90 con 0x7f5d4815b2f0 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45906/0 (socket says 172.21.15.91:45906) 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 -- 172.21.15.91:0/3381259734 learned_addr learned my addr 172.21.15.91:0/3381259734 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 0x7f5d48155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4e000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 0x7f5d480add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 -- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 msgr2=0x7f5d480add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 0x7f5d480add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 -- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 msgr2=0x7f5d48155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:43.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 0x7f5d48155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4d600640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40009400 con 0x7f5d480b0bd0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 0x7f5d48155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.740+0000 7f5d4e000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 0x7f5d480add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d4d600640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c500 tx=0x7f5d3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005ce0 con 0x7f5d480b0bd0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38005e40 con 0x7f5d480b0bd0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007330 con 0x7f5d480b0bd0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480ad270 con 0x7f5d480b0bd0 2024-02-02T13:32:43.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.741+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480ad830 con 0x7f5d480b0bd0 2024-02-02T13:32:43.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.742+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d480ada50 con 0x7f5d480b0bd0 2024-02-02T13:32:43.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.743+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5d38007490 con 0x7f5d480b0bd0 2024-02-02T13:32:43.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.744+0000 7f5d45600640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 0x7f5d2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:43.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.744+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5d38087f70 con 0x7f5d480b0bd0 2024-02-02T13:32:43.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.744+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 0x7f5d2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:43.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.744+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f5d20072130 con 0x7f5d480b0bd0 2024-02-02T13:32:43.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.745+0000 7f5d4cc00640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 0x7f5d2006c8f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5d300094c0 tx=0x7f5d3000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:43.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.748+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5d380baca0 con 0x7f5d480b0bd0 2024-02-02T13:32:43.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:43.977+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5d48066cf0 con 0x7f5d480b0bd0 2024-02-02T13:32:44.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.539+0000 7f5d45600640 1 -- 172.21.15.91:0/3381259734 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5d38017060 con 0x7f5d480b0bd0 2024-02-02T13:32:44.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.542+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 msgr2=0x7f5d2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 0x7f5d2006c8f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5d300094c0 tx=0x7f5d3000b040 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 msgr2=0x7f5d4815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c500 tx=0x7f5d3800c9c0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 shutdown_connections 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2006a430 0x7f5d2006c8f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d4815b2f0 0x7f5d480add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d480b1a00 0x7f5d48155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 --2- 172.21.15.91:0/3381259734 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d480b0bd0 0x7f5d4815d8a0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 >> 172.21.15.91:0/3381259734 conn(0x7f5d480ceb90 msgr2=0x7f5d48158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 shutdown_connections 2024-02-02T13:32:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.543+0000 7f5d50000640 1 -- 172.21.15.91:0/3381259734 wait complete. 2024-02-02T13:32:44.565 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:32:44.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.818+0000 7f16b1a00640 1 Processor -- start 2024-02-02T13:32:44.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.818+0000 7f16b1a00640 1 -- start start 2024-02-02T13:32:44.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.819+0000 7f16b1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16ac06ca20 0x7f16ac0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16b1a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0c9910 con 0x7f16ac06ce00 2024-02-02T13:32:44.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16b1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0cc180 con 0x7f16ac0cbc80 2024-02-02T13:32:44.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16b1a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac0cc460 con 0x7f16ac06ca20 2024-02-02T13:32:44.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16ab400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f16ac06ce00 0x7f16ac0cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41704/0 (socket says 172.21.15.91:41704) 2024-02-02T13:32:44.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16ab400640 1 -- 172.21.15.91:0/288354387 learned_addr learned my addr 172.21.15.91:0/288354387 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:44.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16abe00640 1 --2- 172.21.15.91:0/288354387 >> v2:172.21.15.113:3300/0 conn(0x7f16ac06ca20 0x7f16ac0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191819997 0 0) 0x7f16ac0c9910 con 0x7f16ac06ce00 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.820+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c003520 con 0x7f16ac06ce00 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1802319316 0 0) 0x7f168c003520 con 0x7f16ac06ce00 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 >> v2:172.21.15.113:3300/0 conn(0x7f16ac06ca20 msgr2=0x7f16ac0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 --2- 172.21.15.91:0/288354387 >> v2:172.21.15.113:3300/0 conn(0x7f16ac06ca20 0x7f16ac0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 >> v1:172.21.15.113:6789/0 conn(0x7f16ac0cbc80 legacy=0x7f16ac159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0cd420 con 0x7f16ac06ce00 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1694001670 con 0x7f16ac06ce00 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 >> v1:172.21.15.91:6789/0 conn(0x7f16ac06ce00 legacy=0x7f16ac0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168c0039c0 0x7f168c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f168c0047c0 0x7f168c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f168c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0cc460 con 0x7f168c005180 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00c080 con 0x7f168c0039c0 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00c240 con 0x7f168c0047c0 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.821+0000 7f16b1000640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f168c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 -- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f168c0047c0 msgr2=0x7f168c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f168c0047c0 0x7f168c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 -- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168c0039c0 msgr2=0x7f168c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16abe00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168c0039c0 0x7f168c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168c0039c0 0x7f168c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 -- 172.21.15.91:0/288354387 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c003520 con 0x7f168c005180 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.822+0000 7f16b1000640 1 -- 172.21.15.91:0/288354387 reap_dead start 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1000640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f168c00b980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f16a4001a30 tx=0x7f168c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fedc3f3261859d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4003b70 con 0x7f168c005180 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4003cd0 con 0x7f168c005180 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16aaa00640 1 -- 172.21.15.91:0/288354387 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40052f0 con 0x7f168c005180 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1a00640 1 -- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 msgr2=0x7f168c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1a00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f168c00b980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f16a4001a30 tx=0x7f168c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1a00640 1 -- 172.21.15.91:0/288354387 shutdown_connections 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1a00640 1 --2- 172.21.15.91:0/288354387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f168c00b980 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.823+0000 7f16b1a00640 1 -- 172.21.15.91:0/288354387 >> 172.21.15.91:0/288354387 conn(0x7f16ac0b08e0 msgr2=0x7f16ac0b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.824+0000 7f16b1a00640 1 -- 172.21.15.91:0/288354387 shutdown_connections 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.824+0000 7f16b1a00640 1 -- 172.21.15.91:0/288354387 wait complete. 2024-02-02T13:32:44.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.825+0000 7f16b1a00640 1 Processor -- start 2024-02-02T13:32:44.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.825+0000 7f16b1a00640 1 -- start start 2024-02-02T13:32:44.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ac1595d0 0x7f16ac14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ac15b870 0x7f16ac15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac15c260 con 0x7f168c005180 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac159d90 con 0x7f16ac15b870 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.826+0000 7f16b1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac15a070 con 0x7f16ac1595d0 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45922/0 (socket says 172.21.15.91:45922) 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 -- 172.21.15.91:0/2113715886 learned_addr learned my addr 172.21.15.91:0/2113715886 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16ab400640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ac1595d0 0x7f16ac14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 -- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ac1595d0 msgr2=0x7f16ac14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ac1595d0 0x7f16ac14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 -- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ac15b870 msgr2=0x7f16ac15bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ac15b870 0x7f16ac15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.827+0000 7f16abe00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac15a350 con 0x7f168c005180 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16abe00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f169c010860 tx=0x7f169c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c009610 con 0x7f168c005180 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c00eb60 con 0x7f168c005180 2024-02-02T13:32:44.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac15a630 con 0x7f168c005180 2024-02-02T13:32:44.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c01c400 con 0x7f168c005180 2024-02-02T13:32:44.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.828+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac159a50 con 0x7f168c005180 2024-02-02T13:32:44.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.829+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac04dff0 con 0x7f168c005180 2024-02-02T13:32:44.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.830+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f169c01c560 con 0x7f168c005180 2024-02-02T13:32:44.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.831+0000 7f16a3e00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 0x7f168406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:44.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.831+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f169c088fa0 con 0x7f168c005180 2024-02-02T13:32:44.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.831+0000 7f16ab400640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 0x7f168406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:44.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.831+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f1684072160 con 0x7f168c005180 2024-02-02T13:32:44.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.832+0000 7f16ab400640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 0x7f168406c920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0c5970 tx=0x7f1694009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:44.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:44.836+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f169c054510 con 0x7f168c005180 2024-02-02T13:32:45.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.026+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16ac0cd080 con 0x7f168c005180 2024-02-02T13:32:45.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.549+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(151..151 src has 1..151) v4 ==== 296+0+0 (secure 0 0 0) 0x7f169c0581d0 con 0x7f168c005180 2024-02-02T13:32:45.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.550+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7f1684070590 con 0x7f168c005180 2024-02-02T13:32:45.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.555+0000 7f16a3e00640 1 -- 172.21.15.91:0/2113715886 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) v1 ==== 181+0+0 (secure 0 0 0) 0x7f169c057b30 con 0x7f168c005180 2024-02-02T13:32:45.563 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:32:45.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.564+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 msgr2=0x7f168406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.564+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 0x7f168406c920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0c5970 tx=0x7f1694009040 comp rx=0 tx=0).stop 2024-02-02T13:32:45.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.564+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 msgr2=0x7f16ac0c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.564+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f169c010860 tx=0x7f169c010d20 comp rx=0 tx=0).stop 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 shutdown_connections 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f168406a460 0x7f168406c920 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ac1595d0 0x7f16ac14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ac15b870 0x7f16ac15bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 --2- 172.21.15.91:0/2113715886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f168c005180 0x7f16ac0c8090 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.565+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 >> 172.21.15.91:0/2113715886 conn(0x7f16ac0b08e0 msgr2=0x7f16ac06fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:45.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.566+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 shutdown_connections 2024-02-02T13:32:45.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.566+0000 7f16b1a00640 1 -- 172.21.15.91:0/2113715886 wait complete. 2024-02-02T13:32:45.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:32:45.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.794+0000 7f390ae00640 1 Processor -- start 2024-02-02T13:32:45.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.795+0000 7f390ae00640 1 -- start start 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.796+0000 7f390ae00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f390415d2a0 0x7f39040cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.796+0000 7f390ae00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39040cfb90 con 0x7f3904066020 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.796+0000 7f390ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39040af2a0 con 0x7f3904070e30 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.796+0000 7f390ae00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39040af580 con 0x7f390415d2a0 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.796+0000 7f3908e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3904066020 0x7f390406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41706/0 (socket says 172.21.15.91:41706) 2024-02-02T13:32:45.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3908e00640 1 -- 172.21.15.91:0/44202289 learned_addr learned my addr 172.21.15.91:0/44202289 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903e00640 1 --2- 172.21.15.91:0/44202289 >> v2:172.21.15.113:3300/0 conn(0x7f390415d2a0 0x7f39040cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900490569 0 0) 0x7f39040cfb90 con 0x7f3904066020 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003520 con 0x7f3904066020 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 339486410 0 0) 0x7f38e4003520 con 0x7f3904066020 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 >> v2:172.21.15.113:3300/0 conn(0x7f390415d2a0 msgr2=0x7f39040cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 --2- 172.21.15.91:0/44202289 >> v2:172.21.15.113:3300/0 conn(0x7f390415d2a0 0x7f39040cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.797+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 >> v1:172.21.15.113:6789/0 conn(0x7f3904070e30 legacy=0x7f3904065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040cf8b0 con 0x7f3904066020 2024-02-02T13:32:45.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f38fc0019f0 con 0x7f3904066020 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 >> v1:172.21.15.91:6789/0 conn(0x7f3904066020 legacy=0x7f390406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f38e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38e40047c0 0x7f38e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38e40052a0 0x7f38e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040af580 con 0x7f38e40039c0 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c1a0 con 0x7f38e40052a0 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c360 con 0x7f38e40047c0 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f38e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 -- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38e40047c0 msgr2=0x7f38e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38e40047c0 0x7f38e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 -- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38e40052a0 msgr2=0x7f38e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:45.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38e40052a0 0x7f38e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3903e00640 1 -- 172.21.15.91:0/44202289 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4003520 con 0x7f38e40039c0 2024-02-02T13:32:45.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.798+0000 7f3908e00640 1 -- 172.21.15.91:0/44202289 reap_dead start 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.799+0000 7f3903e00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f38e4003de0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009ac0 tx=0x7f38e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6f2efbbb3053ce server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f402c710 con 0x7f38e40039c0 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f402c870 con 0x7f38e40039c0 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f3903400640 1 -- 172.21.15.91:0/44202289 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f402cc00 con 0x7f38e40039c0 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f390ae00640 1 -- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 msgr2=0x7f38e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f390ae00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f38e4003de0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009ac0 tx=0x7f38e40057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f390ae00640 1 -- 172.21.15.91:0/44202289 shutdown_connections 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f390ae00640 1 --2- 172.21.15.91:0/44202289 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f38e4003de0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.800+0000 7f390ae00640 1 -- 172.21.15.91:0/44202289 >> 172.21.15.91:0/44202289 conn(0x7f39040cfda0 msgr2=0x7f39040d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.801+0000 7f390ae00640 1 -- 172.21.15.91:0/44202289 shutdown_connections 2024-02-02T13:32:45.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.801+0000 7f390ae00640 1 -- 172.21.15.91:0/44202289 wait complete. 2024-02-02T13:32:45.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.802+0000 7f390ae00640 1 Processor -- start 2024-02-02T13:32:45.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.803+0000 7f390ae00640 1 -- start start 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.803+0000 7f390ae00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.803+0000 7f390ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3904153440 0x7f390415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f390ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39040b47b0 0x7f39040b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f390ae00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040b5180 con 0x7f38e40039c0 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f390ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39041e6500 con 0x7f3904153440 2024-02-02T13:32:45.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f390ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39041e6710 con 0x7f39040b47b0 2024-02-02T13:32:45.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:45.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45942/0 (socket says 172.21.15.91:45942) 2024-02-02T13:32:45.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 -- 172.21.15.91:0/522114412 learned_addr learned my addr 172.21.15.91:0/522114412 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 -- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39040b47b0 msgr2=0x7f39040b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39040b47b0 0x7f39040b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 -- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3904153440 msgr2=0x7f390415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3904153440 0x7f390415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39041e69f0 con 0x7f38e40039c0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.804+0000 7f3903e00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f38f402c2b0 tx=0x7f38f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.805+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003810 con 0x7f38e40039c0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.805+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f402de90 con 0x7f38e40039c0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.805+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003b70 con 0x7f38e40039c0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.805+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39041e6cd0 con 0x7f38e40039c0 2024-02-02T13:32:45.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.805+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39041e74d0 con 0x7f38e40039c0 2024-02-02T13:32:45.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.806+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3904066cf0 con 0x7f38e40039c0 2024-02-02T13:32:45.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.807+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f38f4003cd0 con 0x7f38e40039c0 2024-02-02T13:32:45.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.808+0000 7f3900c00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 0x7f38dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:45.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.808+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f38f40aa710 con 0x7f38e40039c0 2024-02-02T13:32:45.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.808+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7f38dc0712c0 con 0x7f38e40039c0 2024-02-02T13:32:45.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.808+0000 7f38fbe00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 0x7f38dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:45.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.809+0000 7f38fbe00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 0x7f38dc06c8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f39041e7790 tx=0x7f38ec007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:45.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:45.812+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f38f4075d90 con 0x7f38e40039c0 2024-02-02T13:32:46.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.001+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39041e77c0 con 0x7f38e40039c0 2024-02-02T13:32:46.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.561+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(152..152 src has 1..152) v4 ==== 330+0+0 (secure 0 0 0) 0x7f38f407a150 con 0x7f38e40039c0 2024-02-02T13:32:46.563 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.561+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7f38dc070050 con 0x7f38e40039c0 2024-02-02T13:32:46.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.575+0000 7f3900c00640 1 -- 172.21.15.91:0/522114412 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38f4039070 con 0x7f38e40039c0 2024-02-02T13:32:46.579 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:32:46.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 msgr2=0x7f38dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 0x7f38dc06c8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f39041e7790 tx=0x7f38ec007990 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 msgr2=0x7f3904152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f38f402c2b0 tx=0x7f38f4004730 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 shutdown_connections 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f38dc06a430 0x7f38dc06c8f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39040b47b0 0x7f39040b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3904153440 0x7f390415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 --2- 172.21.15.91:0/522114412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38e40039c0 0x7f3904152d80 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 >> 172.21.15.91:0/522114412 conn(0x7f39040cfda0 msgr2=0x7f39040ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:46.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.580+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 shutdown_connections 2024-02-02T13:32:46.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.581+0000 7f390ae00640 1 -- 172.21.15.91:0/522114412 wait complete. 2024-02-02T13:32:46.603 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:32:46.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.791+0000 7fb68f600640 1 Processor -- start 2024-02-02T13:32:46.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.792+0000 7fb68f600640 1 -- start start 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb68f600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb68815d430 0x7fb688160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb68f600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6880adfe0 0x7fb68815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb68f600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880c9260 con 0x7fb68815e610 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb68f600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68815dc40 con 0x7fb6880adfe0 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb68f600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb688161460 con 0x7fb68815d430 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb67fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6880adfe0 0x7fb68815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb67fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb6880adfe0 0x7fb68815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45956/0 (socket says 172.21.15.91:45956) 2024-02-02T13:32:46.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.793+0000 7fb67fe00640 1 -- 172.21.15.91:0/3258310596 learned_addr learned my addr 172.21.15.91:0/3258310596 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:46.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.794+0000 7fb68cc00640 1 --2- 172.21.15.91:0/3258310596 >> v2:172.21.15.113:3300/0 conn(0x7fb68815d430 0x7fb688160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.794+0000 7fb67fe00640 1 -- 172.21.15.91:0/3258310596 >> v2:172.21.15.113:3300/0 conn(0x7fb68815d430 msgr2=0x7fb688160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:46.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.794+0000 7fb67fe00640 1 --2- 172.21.15.91:0/3258310596 >> v2:172.21.15.113:3300/0 conn(0x7fb68815d430 0x7fb688160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.794+0000 7fb67fe00640 1 -- 172.21.15.91:0/3258310596 >> v1:172.21.15.113:6789/0 conn(0x7fb68815e610 legacy=0x7fb6880ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:46.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.794+0000 7fb67fe00640 1 -- 172.21.15.91:0/3258310596 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688161700 con 0x7fb6880adfe0 2024-02-02T13:32:46.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.795+0000 7fb67fe00640 1 --2- 172.21.15.91:0/3258310596 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6880adfe0 0x7fb68815e0d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb670009870 tx=0x7fb67002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d3c52d652a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:46.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.795+0000 7fb687e00640 1 -- 172.21.15.91:0/3258310596 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670030850 con 0x7fb6880adfe0 2024-02-02T13:32:46.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb687e00640 1 -- 172.21.15.91:0/3258310596 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6700309b0 con 0x7fb6880adfe0 2024-02-02T13:32:46.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb687e00640 1 -- 172.21.15.91:0/3258310596 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670030d40 con 0x7fb6880adfe0 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 -- 172.21.15.91:0/3258310596 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6880adfe0 msgr2=0x7fb68815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 --2- 172.21.15.91:0/3258310596 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6880adfe0 0x7fb68815e0d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb670009870 tx=0x7fb67002b990 comp rx=0 tx=0).stop 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 -- 172.21.15.91:0/3258310596 shutdown_connections 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 --2- 172.21.15.91:0/3258310596 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb6880adfe0 0x7fb68815e0d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 --2- 172.21.15.91:0/3258310596 >> v2:172.21.15.113:3300/0 conn(0x7fb68815d430 0x7fb688160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 -- 172.21.15.91:0/3258310596 >> 172.21.15.91:0/3258310596 conn(0x7fb6880ceb90 msgr2=0x7fb6880cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.796+0000 7fb68f600640 1 -- 172.21.15.91:0/3258310596 shutdown_connections 2024-02-02T13:32:46.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.797+0000 7fb68f600640 1 -- 172.21.15.91:0/3258310596 wait complete. 2024-02-02T13:32:46.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.799+0000 7fb68f600640 1 Processor -- start 2024-02-02T13:32:46.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.799+0000 7fb68f600640 1 -- start start 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880adfe0 0x7fb688155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 0x7fb6880c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b2f60 con 0x7fb68815e610 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881f2ce0 con 0x7fb6880adfe0 2024-02-02T13:32:46.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68f600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b2350 con 0x7fb68815d430 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45958/0 (socket says 172.21.15.91:45958) 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68d600640 1 -- 172.21.15.91:0/2557352598 learned_addr learned my addr 172.21.15.91:0/2557352598 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb67fe00640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 0x7fb6880c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.800+0000 7fb68cc00640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880adfe0 0x7fb688155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 -- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 msgr2=0x7fb6880c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 0x7fb6880c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 -- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880adfe0 msgr2=0x7fb688155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880adfe0 0x7fb688155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 -- 172.21.15.91:0/2557352598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700094d0 con 0x7fb68815e610 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb67fe00640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 0x7fb6880c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:46.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68d600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb68000b990 tx=0x7fb68000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:46.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000cba0 con 0x7fb68815e610 2024-02-02T13:32:46.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68000cd00 con 0x7fb68815e610 2024-02-02T13:32:46.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680014680 con 0x7fb68815e610 2024-02-02T13:32:46.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.801+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6880b2630 con 0x7fb68815e610 2024-02-02T13:32:46.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.802+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880b2bf0 con 0x7fb68815e610 2024-02-02T13:32:46.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.803+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6880ade50 con 0x7fb68815e610 2024-02-02T13:32:46.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.804+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6800147e0 con 0x7fb68815e610 2024-02-02T13:32:46.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.805+0000 7fb685600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 0x7fb66006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:46.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.805+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6800885d0 con 0x7fb68815e610 2024-02-02T13:32:46.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.805+0000 7fb68cc00640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 0x7fb66006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:46.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.806+0000 7fb68cc00640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 0x7fb66006c8f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb678006fd0 tx=0x7fb678008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:46.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.809+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb680053eb0 con 0x7fb68815e610 2024-02-02T13:32:46.999 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:46.997+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb6881f2e70 con 0x7fb68815e610 2024-02-02T13:32:47.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.005+0000 7fb685600640 1 -- 172.21.15.91:0/2557352598 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6800574d0 con 0x7fb68815e610 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 msgr2=0x7fb66006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 0x7fb66006c8f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb678006fd0 tx=0x7fb678008040 comp rx=0 tx=0).stop 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 msgr2=0x7fb6880c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb68000b990 tx=0x7fb68000be50 comp rx=0 tx=0).stop 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 shutdown_connections 2024-02-02T13:32:47.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb66006a430 0x7fb66006c8f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb68815d430 0x7fb6880c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb68815e610 0x7fb6880c6110 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 --2- 172.21.15.91:0/2557352598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880adfe0 0x7fb688155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.007+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 >> 172.21.15.91:0/2557352598 conn(0x7fb6880ceb90 msgr2=0x7fb68806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.008+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 shutdown_connections 2024-02-02T13:32:47.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.008+0000 7fb68f600640 1 -- 172.21.15.91:0/2557352598 wait complete. 2024-02-02T13:32:47.028 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-02-02T13:32:47.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.231+0000 7f3e4f200640 1 Processor -- start 2024-02-02T13:32:47.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.232+0000 7f3e4f200640 1 -- start start 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e4f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4815f5a0 0x7f3e480b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e4f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e480b0c40 0x7f3e480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e4f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e48070f80 con 0x7f3e48151f90 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e4f200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4815fa30 con 0x7f3e4815f5a0 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e4f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480b10b0 con 0x7f3e480b0c40 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e3f200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4815f5a0 0x7f3e480b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e47e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3e48151f90 0x7f3e4806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41722/0 (socket says 172.21.15.91:41722) 2024-02-02T13:32:47.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.233+0000 7f3e47e00640 1 -- 172.21.15.91:0/3373799270 learned_addr learned my addr 172.21.15.91:0/3373799270 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:47.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.234+0000 7f3e3f200640 1 -- 172.21.15.91:0/3373799270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e480b0c40 msgr2=0x7f3e480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:47.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.234+0000 7f3e3f200640 1 --2- 172.21.15.91:0/3373799270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e480b0c40 0x7f3e480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.234+0000 7f3e3f200640 1 -- 172.21.15.91:0/3373799270 >> v1:172.21.15.91:6789/0 conn(0x7f3e48151f90 legacy=0x7f3e4806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.234+0000 7f3e3f200640 1 -- 172.21.15.91:0/3373799270 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48152370 con 0x7f3e4815f5a0 2024-02-02T13:32:47.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.234+0000 7f3e47400640 1 -- 172.21.15.91:0/3373799270 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979339934 0 0) 0x7f3e48070f80 con 0x7f3e48151f90 2024-02-02T13:32:47.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.235+0000 7f3e3f200640 1 --2- 172.21.15.91:0/3373799270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4815f5a0 0x7f3e480b0700 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3e300097a0 tx=0x7f3e3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826556758c8b6fc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.235+0000 7f3e47400640 1 -- 172.21.15.91:0/3373799270 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30031830 con 0x7f3e4815f5a0 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.235+0000 7f3e47400640 1 -- 172.21.15.91:0/3373799270 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30031990 con 0x7f3e4815f5a0 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.235+0000 7f3e47400640 1 -- 172.21.15.91:0/3373799270 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30031d20 con 0x7f3e4815f5a0 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 -- 172.21.15.91:0/3373799270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4815f5a0 msgr2=0x7f3e480b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 --2- 172.21.15.91:0/3373799270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4815f5a0 0x7f3e480b0700 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3e300097a0 tx=0x7f3e3002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 -- 172.21.15.91:0/3373799270 shutdown_connections 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 --2- 172.21.15.91:0/3373799270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e480b0c40 0x7f3e480b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 --2- 172.21.15.91:0/3373799270 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4815f5a0 0x7f3e480b0700 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 -- 172.21.15.91:0/3373799270 >> 172.21.15.91:0/3373799270 conn(0x7f3e480cec80 msgr2=0x7f3e480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 -- 172.21.15.91:0/3373799270 shutdown_connections 2024-02-02T13:32:47.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.236+0000 7f3e4f200640 1 -- 172.21.15.91:0/3373799270 wait complete. 2024-02-02T13:32:47.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.238+0000 7f3e4f200640 1 Processor -- start 2024-02-02T13:32:47.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.238+0000 7f3e4f200640 1 -- start start 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e48151f90 0x7f3e480cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4815f5a0 0x7f3e480cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480c5630 con 0x7f3e480b0c40 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480ccfe0 con 0x7f3e4815f5a0 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e4f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480ae2b0 con 0x7f3e48151f90 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45978/0 (socket says 172.21.15.91:45978) 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 -- 172.21.15.91:0/778364118 learned_addr learned my addr 172.21.15.91:0/778364118 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 -- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e48151f90 msgr2=0x7f3e480cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e48151f90 0x7f3e480cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 -- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4815f5a0 msgr2=0x7f3e480cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4815f5a0 0x7f3e480cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.239+0000 7f3e47e00640 1 -- 172.21.15.91:0/778364118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e30009400 con 0x7f3e480b0c40 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e47e00640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800c430 tx=0x7f3e3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e380074e0 con 0x7f3e480b0c40 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38007660 con 0x7f3e480b0c40 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e480cc960 con 0x7f3e480b0c40 2024-02-02T13:32:47.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38004a20 con 0x7f3e480b0c40 2024-02-02T13:32:47.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.240+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e480adfd0 con 0x7f3e480b0c40 2024-02-02T13:32:47.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.241+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e48066cf0 con 0x7f3e480b0c40 2024-02-02T13:32:47.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.243+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e38004b80 con 0x7f3e480b0c40 2024-02-02T13:32:47.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.243+0000 7f3e44c00640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 0x7f3e2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.244+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e380884c0 con 0x7f3e480b0c40 2024-02-02T13:32:47.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.244+0000 7f3e3f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 0x7f3e2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.244+0000 7f3e3f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 0x7f3e2006c8f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3e30036000 tx=0x7f3e30030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.248+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e38053db0 con 0x7f3e480b0c40 2024-02-02T13:32:47.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.506+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f3e4806c7b0 con 0x7f3e480b0c40 2024-02-02T13:32:47.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.568+0000 7f3e44c00640 1 -- 172.21.15.91:0/778364118 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v424) v1 ==== 140+0+0 (secure 0 0 0) 0x7f3e380573d0 con 0x7f3e480b0c40 2024-02-02T13:32:47.572 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 msgr2=0x7f3e2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 0x7f3e2006c8f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3e30036000 tx=0x7f3e30030000 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 msgr2=0x7f3e480ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800c430 tx=0x7f3e3800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 shutdown_connections 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2006a430 0x7f3e2006c8f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e48151f90 0x7f3e480cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4815f5a0 0x7f3e480cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 --2- 172.21.15.91:0/778364118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e480b0c40 0x7f3e480ad7c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 >> 172.21.15.91:0/778364118 conn(0x7f3e480cec80 msgr2=0x7f3e4806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 shutdown_connections 2024-02-02T13:32:47.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.571+0000 7f3e4f200640 1 -- 172.21.15.91:0/778364118 wait complete. 2024-02-02T13:32:47.591 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-02-02T13:32:47.591 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-02-02T13:32:47.591 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-02-02T13:32:47.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.791+0000 7fc0dc600640 1 Processor -- start 2024-02-02T13:32:47.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.791+0000 7fc0dc600640 1 -- start start 2024-02-02T13:32:47.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.792+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.792+0000 7fc0dc600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0d406f4f0 0x7fc0d40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.792+0000 7fc0dc600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d4070f80 con 0x7fc0d40b0c40 2024-02-02T13:32:47.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.792+0000 7fc0dc600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d40b10d0 con 0x7fc0d406f4f0 2024-02-02T13:32:47.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.792+0000 7fc0dc600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4152370 con 0x7fc0d4151f90 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0d406f4f0 0x7fc0d40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0da600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc0d40b0c40 0x7fc0d40b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41736/0 (socket says 172.21.15.91:41736) 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0da600640 1 -- 172.21.15.91:0/3458251032 learned_addr learned my addr 172.21.15.91:0/3458251032 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9200640 1 -- 172.21.15.91:0/3458251032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 msgr2=0x7fc0d415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9c00640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d3e00640 1 -- 172.21.15.91:0/3458251032 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830780807 0 0) 0x7fc0d4070f80 con 0x7fc0d40b0c40 2024-02-02T13:32:47.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9200640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9200640 1 -- 172.21.15.91:0/3458251032 >> v1:172.21.15.91:6789/0 conn(0x7fc0d40b0c40 legacy=0x7fc0d40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9200640 1 -- 172.21.15.91:0/3458251032 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d40cb010 con 0x7fc0d406f4f0 2024-02-02T13:32:47.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.793+0000 7fc0d9c00640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.794+0000 7fc0d9200640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d40b0700 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0097a0 tx=0x7fc0bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b490316ffb1705a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0d3e00640 1 -- 172.21.15.91:0/3458251032 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031830 con 0x7fc0d406f4f0 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0d3e00640 1 -- 172.21.15.91:0/3458251032 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc031990 con 0x7fc0d406f4f0 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0d3e00640 1 -- 172.21.15.91:0/3458251032 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031d20 con 0x7fc0d406f4f0 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 -- 172.21.15.91:0/3458251032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 msgr2=0x7fc0d40b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d40b0700 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0097a0 tx=0x7fc0bc02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 -- 172.21.15.91:0/3458251032 shutdown_connections 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d40b0700 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 --2- 172.21.15.91:0/3458251032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.795+0000 7fc0dc600640 1 -- 172.21.15.91:0/3458251032 >> 172.21.15.91:0/3458251032 conn(0x7fc0d40cec80 msgr2=0x7fc0d40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:47.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.796+0000 7fc0dc600640 1 -- 172.21.15.91:0/3458251032 shutdown_connections 2024-02-02T13:32:47.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.796+0000 7fc0dc600640 1 -- 172.21.15.91:0/3458251032 wait complete. 2024-02-02T13:32:47.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.797+0000 7fc0dc600640 1 Processor -- start 2024-02-02T13:32:47.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.797+0000 7fc0dc600640 1 -- start start 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0d40b0c40 0x7fc0d40adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d40cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41511a0 con 0x7fc0d406f4f0 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40ad850 con 0x7fc0d40b0c40 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.798+0000 7fc0dc600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415d550 con 0x7fc0d4151f90 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:45996/0 (socket says 172.21.15.91:45996) 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 -- 172.21.15.91:0/2048715768 learned_addr learned my addr 172.21.15.91:0/2048715768 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 -- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 msgr2=0x7fc0d40cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d40cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 -- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0d40b0c40 msgr2=0x7fc0d40adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0d40b0c40 0x7fc0d40adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:47.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 -- 172.21.15.91:0/2048715768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009400 con 0x7fc0d406f4f0 2024-02-02T13:32:47.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.799+0000 7fc0d9c00640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c820 tx=0x7fc0c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.800+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4007a30 con 0x7fc0d406f4f0 2024-02-02T13:32:47.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.800+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c4007b90 con 0x7fc0d406f4f0 2024-02-02T13:32:47.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.800+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40ae220 con 0x7fc0d406f4f0 2024-02-02T13:32:47.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.800+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400ce70 con 0x7fc0d406f4f0 2024-02-02T13:32:47.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.800+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d40acf20 con 0x7fc0d406f4f0 2024-02-02T13:32:47.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.801+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d4066cf0 con 0x7fc0d406f4f0 2024-02-02T13:32:47.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.802+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0c40113f0 con 0x7fc0d406f4f0 2024-02-02T13:32:47.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.803+0000 7fc0d1600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 0x7fc0ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:47.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.803+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0c4088800 con 0x7fc0d406f4f0 2024-02-02T13:32:47.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.804+0000 7fc0d9200640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 0x7fc0ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:47.806 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.804+0000 7fc0d9200640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 0x7fc0ac06c8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc036000 tx=0x7fc0bc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:47.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:47.808+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0c40540f0 con 0x7fc0d406f4f0 2024-02-02T13:32:48.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.068+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fc0d406c7b0 con 0x7fc0d406f4f0 2024-02-02T13:32:48.573 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.572+0000 7fc0d1600640 1 -- 172.21.15.91:0/2048715768 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v425) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc0c401f0c0 con 0x7fc0d406f4f0 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.575+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 msgr2=0x7fc0ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 0x7fc0ac06c8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc036000 tx=0x7fc0bc030000 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 msgr2=0x7fc0d4150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c820 tx=0x7fc0c400cc20 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 shutdown_connections 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc0ac06a430 0x7fc0ac06c8f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0d4151f90 0x7fc0d40cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0d40b0c40 0x7fc0d40adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 --2- 172.21.15.91:0/2048715768 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0d406f4f0 0x7fc0d4150c60 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 >> 172.21.15.91:0/2048715768 conn(0x7fc0d40cec80 msgr2=0x7fc0d40b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 shutdown_connections 2024-02-02T13:32:48.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.576+0000 7fc0dc600640 1 -- 172.21.15.91:0/2048715768 wait complete. 2024-02-02T13:32:48.599 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:32:48.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.811+0000 7f12d6a00640 1 Processor -- start 2024-02-02T13:32:48.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.811+0000 7f12d6a00640 1 -- start start 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.812+0000 7f12d6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12d00cad30 0x7f12d00cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.812+0000 7f12d6a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d006c1c0 con 0x7f12d015aec0 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.812+0000 7f12d6a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d006b5f0 con 0x7f12d015b2a0 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.812+0000 7f12d6a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d006cd60 con 0x7f12d00cad30 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12d5600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f12d015aec0 0x7f12d006c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41740/0 (socket says 172.21.15.91:41740) 2024-02-02T13:32:48.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12d5600640 1 -- 172.21.15.91:0/656011745 learned_addr learned my addr 172.21.15.91:0/656011745 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:48.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991808239 0 0) 0x7f12d006c1c0 con 0x7f12d015aec0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12d6000640 1 --2- 172.21.15.91:0/656011745 >> v2:172.21.15.113:3300/0 conn(0x7f12d00cad30 0x7f12d00cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b0003520 con 0x7f12d015aec0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812427440 0 0) 0x7f12b0003520 con 0x7f12d015aec0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 >> v2:172.21.15.113:3300/0 conn(0x7f12d00cad30 msgr2=0x7f12d00cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 --2- 172.21.15.91:0/656011745 >> v2:172.21.15.113:3300/0 conn(0x7f12d00cad30 0x7f12d00cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.813+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 >> v1:172.21.15.113:6789/0 conn(0x7f12d015b2a0 legacy=0x7f12d00ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d006b890 con 0x7f12d015aec0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f12c0001670 con 0x7f12d015aec0 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 >> v1:172.21.15.91:6789/0 conn(0x7f12d015aec0 legacy=0x7f12d006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 0x7f12b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12b00047c0 0x7f12b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0005180 0x7f12b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d006cd60 con 0x7f12b0003970 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b000c030 con 0x7f12b0005180 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b000c1f0 con 0x7f12b00047c0 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12d5600640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 0x7f12b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12d4c00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12b00047c0 0x7f12b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.814+0000 7f12d5600640 1 -- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12b00047c0 msgr2=0x7f12b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d5600640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12b00047c0 0x7f12b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d5600640 1 -- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0005180 msgr2=0x7f12b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d5600640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0005180 0x7f12b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d5600640 1 -- 172.21.15.91:0/656011745 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0003520 con 0x7f12b0003970 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d6000640 1 -- 172.21.15.91:0/656011745 reap_dead start 2024-02-02T13:32:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.815+0000 7f12d4c00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12b00047c0 0x7f12b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12d5600640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 0x7f12b0003d90 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c240 tx=0x7f12b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefc8ff952b211e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0005ad0 con 0x7f12b0003970 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0005c30 con 0x7f12b0003970 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12cfe00640 1 -- 172.21.15.91:0/656011745 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c002caf0 con 0x7f12b0003970 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12d6a00640 1 -- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 msgr2=0x7f12b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.816+0000 7f12d6a00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 0x7f12b0003d90 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c240 tx=0x7f12b00057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.817+0000 7f12d6a00640 1 -- 172.21.15.91:0/656011745 shutdown_connections 2024-02-02T13:32:48.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.817+0000 7f12d6a00640 1 --2- 172.21.15.91:0/656011745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12b0003970 0x7f12b0003d90 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.817+0000 7f12d6a00640 1 -- 172.21.15.91:0/656011745 >> 172.21.15.91:0/656011745 conn(0x7f12d00cea90 msgr2=0x7f12d00ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.817+0000 7f12d6a00640 1 -- 172.21.15.91:0/656011745 shutdown_connections 2024-02-02T13:32:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.817+0000 7f12d6a00640 1 -- 172.21.15.91:0/656011745 wait complete. 2024-02-02T13:32:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.818+0000 7f12d6a00640 1 Processor -- start 2024-02-02T13:32:48.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 -- start start 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0003970 0x7f12d015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d00ae510 0x7f12d00ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00af870 con 0x7f12d014a830 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d014b390 con 0x7f12b0003970 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.819+0000 7f12d6a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00af2b0 con 0x7f12d00ae510 2024-02-02T13:32:48.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46014/0 (socket says 172.21.15.91:46014) 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 -- 172.21.15.91:0/882227186 learned_addr learned my addr 172.21.15.91:0/882227186 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d5600640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0003970 0x7f12d015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d6000640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d00ae510 0x7f12d00ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 -- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d00ae510 msgr2=0x7f12d00ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d00ae510 0x7f12d00ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 -- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0003970 msgr2=0x7f12d015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0003970 0x7f12d015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.820+0000 7f12d4c00640 1 -- 172.21.15.91:0/882227186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00af590 con 0x7f12d014a830 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12d4c00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d430 tx=0x7f12b800d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800b070 con 0x7f12d014a830 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80054e0 con 0x7f12d014a830 2024-02-02T13:32:48.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8003830 con 0x7f12d014a830 2024-02-02T13:32:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d014afb0 con 0x7f12d014a830 2024-02-02T13:32:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.821+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00aeed0 con 0x7f12d014a830 2024-02-02T13:32:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.822+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d014ae10 con 0x7f12d014a830 2024-02-02T13:32:48.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.823+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f12b8003990 con 0x7f12d014a830 2024-02-02T13:32:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.824+0000 7f12cd600640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 0x7f12a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.824+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f12b8087e60 con 0x7f12d014a830 2024-02-02T13:32:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.825+0000 7f12d5600640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 0x7f12a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:48.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.825+0000 7f12d5600640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 0x7f12a806c8f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f12c0034000 tx=0x7f12c0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:48.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:48.829+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f12d014ae10 con 0x7f12d014a830 2024-02-02T13:32:49.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.024+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f12d014ae10 con 0x7f12d014a830 2024-02-02T13:32:49.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.031+0000 7f12cd600640 1 -- 172.21.15.91:0/882227186 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12d014ae10 con 0x7f12d014a830 2024-02-02T13:32:49.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 msgr2=0x7f12a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 0x7f12a806c8f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f12c0034000 tx=0x7f12c0009030 comp rx=0 tx=0).stop 2024-02-02T13:32:49.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 msgr2=0x7f12d015e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d430 tx=0x7f12b800d8f0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 shutdown_connections 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f12a806a430 0x7f12a806c8f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d00ae510 0x7f12d00ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12d014a830 0x7f12d015e4e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 --2- 172.21.15.91:0/882227186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b0003970 0x7f12d015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.033+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 >> 172.21.15.91:0/882227186 conn(0x7f12d00cea90 msgr2=0x7f12d00c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.034+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 shutdown_connections 2024-02-02T13:32:49.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.034+0000 7f12d6a00640 1 -- 172.21.15.91:0/882227186 wait complete. 2024-02-02T13:32:49.054 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:32:49.055 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:32:49.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:32:49.286 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:32:49.286 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:49.286 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:49.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:32:49.341 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:49.341 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:49.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:32:49.397 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:32:49.397 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:49.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:32:49.452 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:32:49.452 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:32:49.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:32:49.506 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:32:49.507 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:49.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.722+0000 7ff593e00640 1 Processor -- start 2024-02-02T13:32:49.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.722+0000 7ff593e00640 1 -- start start 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.723+0000 7ff593e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff58c15d490 0x7ff58c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.723+0000 7ff593e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff593e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c15d870 con 0x7ff58c0ac8d0 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff593e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0ad2a0 con 0x7ff58c15d490 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff593e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c06bce0 con 0x7ff58c0b0e10 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff591400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff58c15d490 0x7ff58c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff590a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff58c0ac8d0 0x7ff58c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41750/0 (socket says 172.21.15.91:41750) 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff590a00640 1 -- 172.21.15.91:0/2657739974 learned_addr learned my addr 172.21.15.91:0/2657739974 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff591400640 1 -- 172.21.15.91:0/2657739974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 msgr2=0x7ff58c0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff591400640 1 --2- 172.21.15.91:0/2657739974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff591400640 1 -- 172.21.15.91:0/2657739974 >> v1:172.21.15.91:6789/0 conn(0x7ff58c0ac8d0 legacy=0x7ff58c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff591400640 1 -- 172.21.15.91:0/2657739974 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c161220 con 0x7ff58c15d490 2024-02-02T13:32:49.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.724+0000 7ff58be00640 1 -- 172.21.15.91:0/2657739974 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358256886 0 0) 0x7ff58c15d870 con 0x7ff58c0ac8d0 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.725+0000 7ff591400640 1 --2- 172.21.15.91:0/2657739974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c06c170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097a0 tx=0x7ff57c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3a2bb81930cbdc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff58be00640 1 -- 172.21.15.91:0/2657739974 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c031830 con 0x7ff58c15d490 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff58be00640 1 -- 172.21.15.91:0/2657739974 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c031990 con 0x7ff58c15d490 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff58be00640 1 -- 172.21.15.91:0/2657739974 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c031d20 con 0x7ff58c15d490 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 -- 172.21.15.91:0/2657739974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 msgr2=0x7ff58c06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 --2- 172.21.15.91:0/2657739974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c06c170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097a0 tx=0x7ff57c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 -- 172.21.15.91:0/2657739974 shutdown_connections 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 --2- 172.21.15.91:0/2657739974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 --2- 172.21.15.91:0/2657739974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c06c170 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 -- 172.21.15.91:0/2657739974 >> 172.21.15.91:0/2657739974 conn(0x7ff58c0cec80 msgr2=0x7ff58c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 -- 172.21.15.91:0/2657739974 shutdown_connections 2024-02-02T13:32:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.726+0000 7ff593e00640 1 -- 172.21.15.91:0/2657739974 wait complete. 2024-02-02T13:32:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.728+0000 7ff593e00640 1 Processor -- start 2024-02-02T13:32:49.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.728+0000 7ff593e00640 1 -- start start 2024-02-02T13:32:49.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 0x7ff58c0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0b4770 con 0x7ff58c15d490 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f6b70 con 0x7ff58c0ac8d0 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff593e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0b2590 con 0x7ff58c0b0e10 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.729+0000 7ff591e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46036/0 (socket says 172.21.15.91:46036) 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 -- 172.21.15.91:0/162461396 learned_addr learned my addr 172.21.15.91:0/162461396 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591400640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 0x7ff58c0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 -- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 msgr2=0x7ff58c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff590a00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 -- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 msgr2=0x7ff58c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 0x7ff58c0b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 -- 172.21.15.91:0/162461396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c009400 con 0x7ff58c15d490 2024-02-02T13:32:49.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591400640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 0x7ff58c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:49.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.730+0000 7ff591e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b9e0 tx=0x7ff58400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:49.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.731+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400ca20 con 0x7ff58c15d490 2024-02-02T13:32:49.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.731+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400cb80 con 0x7ff58c15d490 2024-02-02T13:32:49.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.731+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c1f7000 con 0x7ff58c15d490 2024-02-02T13:32:49.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.731+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584014440 con 0x7ff58c15d490 2024-02-02T13:32:49.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.731+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c1f6520 con 0x7ff58c15d490 2024-02-02T13:32:49.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.732+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c06a010 con 0x7ff58c15d490 2024-02-02T13:32:49.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.733+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff58401d3b0 con 0x7ff58c15d490 2024-02-02T13:32:49.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.734+0000 7ff589600640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 0x7ff56406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:49.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.734+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff5840884d0 con 0x7ff58c15d490 2024-02-02T13:32:49.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.734+0000 7ff591400640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 0x7ff56406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:49.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.735+0000 7ff591400640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 0x7ff56406c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff57c036000 tx=0x7ff57c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:49.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.739+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff584053d20 con 0x7ff58c15d490 2024-02-02T13:32:49.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.964+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff58c066cf0 con 0x7ff58c15d490 2024-02-02T13:32:49.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.965+0000 7ff589600640 1 -- 172.21.15.91:0/162461396 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3532 (secure 0 0 0) 0x7ff584019090 con 0x7ff58c15d490 2024-02-02T13:32:49.967 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:49.967 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2969257370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2969257370},{"type":"v1","addr":"172.21.15.91:6836","nonce":2969257370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/2161079321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2161079321},{"type":"v1","addr":"172.21.15.113:6834","nonce":2161079321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5788,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/821980817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":821980817},{"type":"v1","addr":"172.21.15.113:6835","nonce":821980817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2263756245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2263756245},{"type":"v1","addr":"172.21.15.91:6837","nonce":2263756245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[]} 2024-02-02T13:32:49.967 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 66 2024-02-02T13:32:49.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 msgr2=0x7ff56406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 0x7ff56406c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff57c036000 tx=0x7ff57c008030 comp rx=0 tx=0).stop 2024-02-02T13:32:49.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 msgr2=0x7ff58c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:49.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b9e0 tx=0x7ff58400bea0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 shutdown_connections 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff56406a460 0x7ff56406c920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff58c0b0e10 0x7ff58c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c15d490 0x7ff58c158110 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 --2- 172.21.15.91:0/162461396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff58c0ac8d0 0x7ff58c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 >> 172.21.15.91:0/162461396 conn(0x7ff58c0cec80 msgr2=0x7ff58c1572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 shutdown_connections 2024-02-02T13:32:49.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:49.968+0000 7ff593e00640 1 -- 172.21.15.91:0/162461396 wait complete. 2024-02-02T13:32:49.988 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:32:49.988 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:32:49.989 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:32:49.989 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:32:49.989 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:32:49.989 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:32:49.989 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:32:49.989 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:32:49.990 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:32:49.990 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:32:49.990 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:32:49.990 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:32:49.990 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:32:49.990 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:32:49.991 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:32:49.991 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:32:50.007 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:32:50.007 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:32:50.010 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:32:50.010 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:32:50.017 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:32:50.018 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:32:50.021 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:32:50.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:32:50.057 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:32:50.057 DEBUG:teuthology.parallel:result is None 2024-02-02T13:32:50.063 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:32:50.064 DEBUG:teuthology.parallel:result is None 2024-02-02T13:32:50.068 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:32:50.068 DEBUG:teuthology.parallel:result is None 2024-02-02T13:32:50.074 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:32:50.075 DEBUG:teuthology.parallel:result is None 2024-02-02T13:32:50.075 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:32:50.210 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:32:50.211 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:32:50.229 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:32:50.232 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:32:50.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.261+0000 7faf62000640 1 Processor -- start 2024-02-02T13:32:50.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.262+0000 7faf62000640 1 -- start start 2024-02-02T13:32:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf62000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf5c0ca1d0 0x7faf5c0ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf62000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0c94a0 con 0x7faf5c0cb580 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf62000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0ad6e0 con 0x7faf5c15d3a0 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf62000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0c96f0 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf5c0ca1d0 0x7faf5c0ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faf5c0cb580 0x7faf5c0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49874/0 (socket says 172.21.15.91:49874) 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5b400640 1 -- 172.21.15.91:0/4171828832 learned_addr learned my addr 172.21.15.91:0/4171828832 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5be00640 1 -- 172.21.15.91:0/4171828832 >> v1:172.21.15.113:6789/0 conn(0x7faf5c15d3a0 legacy=0x7faf5c0ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5be00640 1 -- 172.21.15.91:0/4171828832 >> v1:172.21.15.91:6789/0 conn(0x7faf5c0cb580 legacy=0x7faf5c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.264+0000 7faf5be00640 1 -- 172.21.15.91:0/4171828832 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0c99d0 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.263+0000 7faf5aa00640 1 -- 172.21.15.91:0/4171828832 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7312377 0 0) 0x7faf5c0c94a0 con 0x7faf5c0cb580 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf5be00640 1 --2- 172.21.15.91:0/4171828832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c0ad070 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf5402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64fb1183aac26e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf5aa00640 1 -- 172.21.15.91:0/4171828832 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5402cc60 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf5aa00640 1 -- 172.21.15.91:0/4171828832 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5402cdc0 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf5aa00640 1 -- 172.21.15.91:0/4171828832 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54034660 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf62000640 1 -- 172.21.15.91:0/4171828832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0ca1d0 msgr2=0x7faf5c0ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:50.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf62000640 1 --2- 172.21.15.91:0/4171828832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c0ad070 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf5402beb0 comp rx=0 tx=0).stop 2024-02-02T13:32:50.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf62000640 1 -- 172.21.15.91:0/4171828832 shutdown_connections 2024-02-02T13:32:50.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf62000640 1 --2- 172.21.15.91:0/4171828832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c0ad070 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:50.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.265+0000 7faf62000640 1 -- 172.21.15.91:0/4171828832 >> 172.21.15.91:0/4171828832 conn(0x7faf5c0b0700 msgr2=0x7faf5c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:50.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.266+0000 7faf62000640 1 -- 172.21.15.91:0/4171828832 shutdown_connections 2024-02-02T13:32:50.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.266+0000 7faf62000640 1 -- 172.21.15.91:0/4171828832 wait complete. 2024-02-02T13:32:50.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.267+0000 7faf62000640 1 Processor -- start 2024-02-02T13:32:50.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.267+0000 7faf62000640 1 -- start start 2024-02-02T13:32:50.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c0cb580 0x7faf5c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c153d80 con 0x7faf5c15d3a0 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c06c350 con 0x7faf5c0ca1d0 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf62000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c14f7e0 con 0x7faf5c0cb580 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.268+0000 7faf5be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58170/0 (socket says 172.21.15.91:58170) 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 -- 172.21.15.91:0/1409187337 learned_addr learned my addr 172.21.15.91:0/1409187337 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5b400640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf52800640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c0cb580 0x7faf5c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 -- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c0cb580 msgr2=0x7faf5c06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c0cb580 0x7faf5c06d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 -- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c0ca1d0 msgr2=0x7faf5c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:50.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:50.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 -- 172.21.15.91:0/1409187337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54009a70 con 0x7faf5c15d3a0 2024-02-02T13:32:50.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.269+0000 7faf5be00640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:50.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.270+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5403e610 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.270+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5403e790 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.270+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5403eb40 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.270+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c06b7d0 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.270+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c06bd90 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.271+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c153ef0 con 0x7faf5c15d3a0 2024-02-02T13:32:50.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.272+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf5403b050 con 0x7faf5c15d3a0 2024-02-02T13:32:50.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.273+0000 7faf53e00640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:50.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.273+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faf540ab1e0 con 0x7faf5c15d3a0 2024-02-02T13:32:50.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.273+0000 7faf5b400640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:50.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.274+0000 7faf5b400640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faf4c007f70 tx=0x7faf4c007f00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:50.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.277+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf5c153ef0 con 0x7faf5c15d3a0 2024-02-02T13:32:50.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:50.471+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf5c0c8fc0 con 0x7faf5c15d3a0 2024-02-02T13:32:51.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.272+0000 7faf53e00640 1 -- 172.21.15.91:0/1409187337 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v157) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf54076c00 con 0x7faf5c15d3a0 2024-02-02T13:32:51.274 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 msgr2=0x7faf3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faf4c007f70 tx=0x7faf4c007f00 comp rx=0 tx=0).stop 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 msgr2=0x7faf5c15b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf54004a40 comp rx=0 tx=0).stop 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 shutdown_connections 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7faf3406a430 0x7faf3406c8f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf5c0cb580 0x7faf5c06d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf5c15d3a0 0x7faf5c15b9e0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 --2- 172.21.15.91:0/1409187337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf5c0ca1d0 0x7faf5c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 >> 172.21.15.91:0/1409187337 conn(0x7faf5c0b0700 msgr2=0x7faf5c0ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:51.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.276+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 shutdown_connections 2024-02-02T13:32:51.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.277+0000 7faf62000640 1 -- 172.21.15.91:0/1409187337 wait complete. 2024-02-02T13:32:51.297 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:32:51.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.504+0000 7f9d11800640 1 Processor -- start 2024-02-02T13:32:51.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.504+0000 7f9d11800640 1 -- start start 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.505+0000 7f9d11800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9d0c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.505+0000 7f9d11800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0ca0c0 con 0x7f9d0c159de0 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.505+0000 7f9d11800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0ca380 con 0x7f9d0c15e170 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.505+0000 7f9d11800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0ca660 con 0x7f9d0c0cbc90 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d0aa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9d0c159de0 0x7f9d0c0cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49888/0 (socket says 172.21.15.91:49888) 2024-02-02T13:32:51.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d0aa00640 1 -- 172.21.15.91:0/3686629445 learned_addr learned my addr 172.21.15.91:0/3686629445 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807088419 0 0) 0x7f9d0c0ca0c0 con 0x7f9d0c159de0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d0a000640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9d0c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cec003520 con 0x7f9d0c159de0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546376497 0 0) 0x7f9cec003520 con 0x7f9d0c159de0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 msgr2=0x7f9d0c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9d0c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 >> v1:172.21.15.113:6789/0 conn(0x7f9d0c15e170 legacy=0x7f9d0c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0ca940 con 0x7f9d0c159de0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9cfc001670 con 0x7f9d0c159de0 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.506+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 >> v1:172.21.15.91:6789/0 conn(0x7f9d0c159de0 legacy=0x7f9d0c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9cec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004020 0x7f9cec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cec004d00 0x7f9cec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0ca660 con 0x7f9cec004d00 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec00ddd0 con 0x7f9cec004020 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec00df90 con 0x7f9d0c0cbc90 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cec004d00 0x7f9cec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0aa00640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9cec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0a000640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004020 0x7f9cec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 -- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 msgr2=0x7f9cec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c0cbc90 0x7f9cec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 -- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004020 msgr2=0x7f9cec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004020 0x7f9cec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0b400640 1 -- 172.21.15.91:0/3686629445 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec003520 con 0x7f9cec004d00 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.507+0000 7f9d0a000640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004020 0x7f9cec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:51.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.508+0000 7f9d0b400640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cec004d00 0x7f9cec005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001e50 tx=0x7f9cec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a158cb7f93cfc40c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04003b70 con 0x7f9cec004d00 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04003cd0 con 0x7f9cec004d00 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d09600640 1 -- 172.21.15.91:0/3686629445 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04005310 con 0x7f9cec004d00 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 -- 172.21.15.91:0/3686629445 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cec004d00 msgr2=0x7f9cec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 --2- 172.21.15.91:0/3686629445 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9cec004d00 0x7f9cec005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001e50 tx=0x7f9cec0057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d0b400640 1 -- 172.21.15.91:0/3686629445 reap_dead start 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 -- 172.21.15.91:0/3686629445 shutdown_connections 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 -- 172.21.15.91:0/3686629445 >> 172.21.15.91:0/3686629445 conn(0x7f9d0c0b0700 msgr2=0x7f9d0c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 -- 172.21.15.91:0/3686629445 shutdown_connections 2024-02-02T13:32:51.513 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.509+0000 7f9d11800640 1 -- 172.21.15.91:0/3686629445 wait complete. 2024-02-02T13:32:51.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.511+0000 7f9d11800640 1 Processor -- start 2024-02-02T13:32:51.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.511+0000 7f9d11800640 1 -- start start 2024-02-02T13:32:51.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 0x7f9d0c0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c150100 0x7f9d0c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c15cc70 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b1790 con 0x7f9cec004d00 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d11800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b46f0 con 0x7f9d0c150100 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d0b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d0b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58186/0 (socket says 172.21.15.91:58186) 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.512+0000 7f9d0b400640 1 -- 172.21.15.91:0/819913134 learned_addr learned my addr 172.21.15.91:0/819913134 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0aa00640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 0x7f9d0c0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 -- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c150100 msgr2=0x7f9d0c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c150100 0x7f9d0c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 -- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 msgr2=0x7f9d0c0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 0x7f9d0c0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 -- 172.21.15.91:0/819913134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c14b910 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.517 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0aa00640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 0x7f9d0c0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d0b400640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001f70 tx=0x7f9d040014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0403e070 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04003e40 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04006d90 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.513+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c15c0f0 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.514+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c15c6b0 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.518 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.515+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c069fa0 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.515+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9d04007030 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.516+0000 7f9d02a00640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 0x7f9ce406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.516+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d040a9770 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.517+0000 7f9d0aa00640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 0x7f9ce406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.517+0000 7f9d0aa00640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 0x7f9ce406c920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001eb0 tx=0x7f9cfc000f50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.521+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9d04075010 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.763+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9d0c14b8f0 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.764+0000 7f9d02a00640 1 -- 172.21.15.91:0/819913134 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9d04048090 con 0x7f9d0c0b3d30 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.766 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.767 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.768 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.769 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.770 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.771 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.772 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.773 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.774 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.775 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.776 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.777 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:32:51.777 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:32:51.777 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.777 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.778 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:32:51.779 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:32:51.779 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.779 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:32:51.779 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:32:51.780 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:32:51.781 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:32:51.782 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:32:51.783 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:32:51.784 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 msgr2=0x7f9ce406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 0x7f9ce406c920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001eb0 tx=0x7f9cfc000f50 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 msgr2=0x7f9d0c0b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001f70 tx=0x7f9d040014a0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 shutdown_connections 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9ce406a460 0x7f9ce406c920 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d0c150100 0x7f9d0c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9d0c0b3d30 0x7f9d0c0b41b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 --2- 172.21.15.91:0/819913134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cec004d00 0x7f9d0c0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 >> 172.21.15.91:0/819913134 conn(0x7f9d0c0b0700 msgr2=0x7f9d0c06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 shutdown_connections 2024-02-02T13:32:51.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.767+0000 7f9d11800640 1 -- 172.21.15.91:0/819913134 wait complete. 2024-02-02T13:32:51.787 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:51.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.968+0000 7f6727200640 1 Processor -- start 2024-02-02T13:32:51.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.968+0000 7f6727200640 1 -- start start 2024-02-02T13:32:51.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.969+0000 7f6727200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.969+0000 7f6727200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f67200b0700 0x7f67200b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.969+0000 7f6727200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67200cc7c0 con 0x7f672015a010 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.969+0000 7f6727200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67200b10d0 con 0x7f67200b0700 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.969+0000 7f6727200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200ce930 con 0x7f67200ce130 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f67200b0700 0x7f67200b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f67200b0700 0x7f67200b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58188/0 (socket says 172.21.15.91:58188) 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 -- 172.21.15.91:0/3067454510 learned_addr learned my addr 172.21.15.91:0/3067454510 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6717200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 -- 172.21.15.91:0/3067454510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 msgr2=0x7f67200c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 -- 172.21.15.91:0/3067454510 >> v1:172.21.15.113:6789/0 conn(0x7f672015a010 legacy=0x7f67200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.970+0000 7f6725200640 1 -- 172.21.15.91:0/3067454510 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200cd640 con 0x7f67200b0700 2024-02-02T13:32:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.971+0000 7f6725200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67200b0b90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6718009e50 tx=0x7f671802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2e9062b95926a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.971+0000 7f671f400640 1 -- 172.21.15.91:0/3067454510 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671802ca70 con 0x7f67200b0700 2024-02-02T13:32:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f671f400640 1 -- 172.21.15.91:0/3067454510 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671802cbd0 con 0x7f67200b0700 2024-02-02T13:32:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f671f400640 1 -- 172.21.15.91:0/3067454510 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180343f0 con 0x7f67200b0700 2024-02-02T13:32:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 -- 172.21.15.91:0/3067454510 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 msgr2=0x7f67200b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67200b0b90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6718009e50 tx=0x7f671802bed0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 -- 172.21.15.91:0/3067454510 shutdown_connections 2024-02-02T13:32:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67200b0b90 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 --2- 172.21.15.91:0/3067454510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.972+0000 7f6727200640 1 -- 172.21.15.91:0/3067454510 >> 172.21.15.91:0/3067454510 conn(0x7f67200cec80 msgr2=0x7f67200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.973+0000 7f6727200640 1 -- 172.21.15.91:0/3067454510 shutdown_connections 2024-02-02T13:32:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.973+0000 7f6727200640 1 -- 172.21.15.91:0/3067454510 wait complete. 2024-02-02T13:32:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.974+0000 7f6727200640 1 Processor -- start 2024-02-02T13:32:51.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.974+0000 7f6727200640 1 -- start start 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672015a010 0x7f672015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c8010 con 0x7f67200b0700 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200ad160 con 0x7f672015a010 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f6727200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200ad440 con 0x7f67200ce130 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.975+0000 7f671fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58202/0 (socket says 172.21.15.91:58202) 2024-02-02T13:32:51.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 -- 172.21.15.91:0/1549319011 learned_addr learned my addr 172.21.15.91:0/1549319011 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 -- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 msgr2=0x7f67200adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 -- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672015a010 msgr2=0x7f672015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672015a010 0x7f672015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 -- 172.21.15.91:0/1549319011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718009b30 con 0x7f67200b0700 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671fe00640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f671000b3f0 tx=0x7f671000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710011830 con 0x7f67200b0700 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710011990 con 0x7f67200b0700 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710011d40 con 0x7f67200b0700 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.976+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67200ad720 con 0x7f67200b0700 2024-02-02T13:32:51.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.977+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200c82f0 con 0x7f67200b0700 2024-02-02T13:32:51.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.978+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67200ad910 con 0x7f67200b0700 2024-02-02T13:32:51.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.979+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f671001a400 con 0x7f67200b0700 2024-02-02T13:32:51.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.980+0000 7f671cc00640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 0x7f66f806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:51.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.980+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6710088480 con 0x7f67200b0700 2024-02-02T13:32:51.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.980+0000 7f6717200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 0x7f66f806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:51.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.981+0000 7f6717200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 0x7f66f806c920 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f67201f60f0 tx=0x7f6708008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:51.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:51.984+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6710053d20 con 0x7f67200b0700 2024-02-02T13:32:52.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.158+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6720066cf0 con 0x7f67200b0700 2024-02-02T13:32:52.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.159+0000 7f671cc00640 1 -- 172.21.15.91:0/1549319011 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f6710057340 con 0x7f67200b0700 2024-02-02T13:32:52.161 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:52.161 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":157,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:51.256516+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:52.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 msgr2=0x7f66f806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 0x7f66f806c920 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f67201f60f0 tx=0x7f6708008040 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 msgr2=0x7f67201506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f671000b3f0 tx=0x7f671000b8b0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 shutdown_connections 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f66f806a460 0x7f66f806c920 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67200ce130 0x7f67200adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672015a010 0x7f672015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 --2- 172.21.15.91:0/1549319011 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67200b0700 0x7f67201506d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 >> 172.21.15.91:0/1549319011 conn(0x7f67200cec80 msgr2=0x7f6720073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 shutdown_connections 2024-02-02T13:32:52.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.162+0000 7f6727200640 1 -- 172.21.15.91:0/1549319011 wait complete. 2024-02-02T13:32:52.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:32:52.182 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:32:52.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.381+0000 7f3ea3400640 1 Processor -- start 2024-02-02T13:32:52.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.381+0000 7f3ea3400640 1 -- start start 2024-02-02T13:32:52.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.382+0000 7f3ea3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e9c15d490 0x7f3e9c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.382+0000 7f3ea3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.382+0000 7f3ea3400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c15d870 con 0x7f3e9c0b0700 2024-02-02T13:32:52.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3ea3400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0ad2a0 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3ea3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c06bce0 con 0x7f3e9c15d490 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58214/0 (socket says 172.21.15.91:58214) 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 -- 172.21.15.91:0/296783687 learned_addr learned my addr 172.21.15.91:0/296783687 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 -- 172.21.15.91:0/296783687 >> v2:172.21.15.113:3300/0 conn(0x7f3e9c15d490 msgr2=0x7f3e9c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 --2- 172.21.15.91:0/296783687 >> v2:172.21.15.113:3300/0 conn(0x7f3e9c15d490 0x7f3e9c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 -- 172.21.15.91:0/296783687 >> v1:172.21.15.113:6789/0 conn(0x7f3e9c0b0700 legacy=0x7f3e9c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:52.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.383+0000 7f3e9be00640 1 -- 172.21.15.91:0/296783687 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c161220 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.384+0000 7f3e9be00640 1 --2- 172.21.15.91:0/296783687 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c1613b0 tx=0x7f3e8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a8df03b8efe963 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.385+0000 7f3e9b400640 1 -- 172.21.15.91:0/296783687 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84031870 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.385+0000 7f3e9b400640 1 -- 172.21.15.91:0/296783687 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840319d0 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.385+0000 7f3e9b400640 1 -- 172.21.15.91:0/296783687 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84031d60 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.385+0000 7f3ea3400640 1 -- 172.21.15.91:0/296783687 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0ac8d0 msgr2=0x7f3e9c0acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.385+0000 7f3ea3400640 1 --2- 172.21.15.91:0/296783687 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c1613b0 tx=0x7f3e8402b910 comp rx=0 tx=0).stop 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 -- 172.21.15.91:0/296783687 shutdown_connections 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 --2- 172.21.15.91:0/296783687 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c0acd60 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 --2- 172.21.15.91:0/296783687 >> v2:172.21.15.113:3300/0 conn(0x7f3e9c15d490 0x7f3e9c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 -- 172.21.15.91:0/296783687 >> 172.21.15.91:0/296783687 conn(0x7f3e9c0cec80 msgr2=0x7f3e9c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:52.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 -- 172.21.15.91:0/296783687 shutdown_connections 2024-02-02T13:32:52.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.386+0000 7f3ea3400640 1 -- 172.21.15.91:0/296783687 wait complete. 2024-02-02T13:32:52.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.387+0000 7f3ea3400640 1 Processor -- start 2024-02-02T13:32:52.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.387+0000 7f3ea3400640 1 -- start start 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 0x7f3e9c070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c14e750 con 0x7f3e9c0b0700 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c5040 con 0x7f3e9c0ac8d0 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.388+0000 7f3ea3400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c4430 con 0x7f3e9c15d490 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58226/0 (socket says 172.21.15.91:58226) 2024-02-02T13:32:52.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 -- 172.21.15.91:0/3564475573 learned_addr learned my addr 172.21.15.91:0/3564475573 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 -- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 msgr2=0x7f3e9c070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3ea0a00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3ea1400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 0x7f3e9c070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 0x7f3e9c070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 -- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c0ac8d0 msgr2=0x7f3e9c14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3e9be00640 1 -- 172.21.15.91:0/3564475573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840094d0 con 0x7f3e9c0b0700 2024-02-02T13:32:52.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.389+0000 7f3ea1400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 0x7f3e9c070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3e9be00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3e84000c00 tx=0x7f3e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8403ad10 con 0x7f3e9c0b0700 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8403ae70 con 0x7f3e9c0b0700 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84044610 con 0x7f3e9c0b0700 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c0c4710 con 0x7f3e9c0b0700 2024-02-02T13:32:52.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.390+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c0c4cd0 con 0x7f3e9c0b0700 2024-02-02T13:32:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.391+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c066cf0 con 0x7f3e9c0b0700 2024-02-02T13:32:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.392+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e8404d450 con 0x7f3e9c0b0700 2024-02-02T13:32:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.393+0000 7f3e98c00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 0x7f3e7406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.393+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e8402d040 con 0x7f3e9c0b0700 2024-02-02T13:32:52.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.393+0000 7f3ea0a00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 0x7f3e7406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.394+0000 7f3ea0a00640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 0x7f3e7406c920 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00a810 tx=0x7f3e8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.397+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e840766f0 con 0x7f3e9c0b0700 2024-02-02T13:32:52.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.588+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e9c0b4c00 con 0x7f3e9c0b0700 2024-02-02T13:32:52.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.589+0000 7f3e98c00640 1 -- 172.21.15.91:0/3564475573 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e84079d10 con 0x7f3e9c0b0700 2024-02-02T13:32:52.590 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:32:52.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 msgr2=0x7f3e7406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 0x7f3e7406c920 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00a810 tx=0x7f3e8c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 msgr2=0x7f3e9c0c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3e84000c00 tx=0x7f3e840046c0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 shutdown_connections 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.591+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e7406a460 0x7f3e7406c920 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e9c15d490 0x7f3e9c070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c0b0700 0x7f3e9c0c3a40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 --2- 172.21.15.91:0/3564475573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e9c0ac8d0 0x7f3e9c14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 >> 172.21.15.91:0/3564475573 conn(0x7f3e9c0cec80 msgr2=0x7f3e9c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 shutdown_connections 2024-02-02T13:32:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.592+0000 7f3ea3400640 1 -- 172.21.15.91:0/3564475573 wait complete. 2024-02-02T13:32:52.611 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:32:52.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.808+0000 7f8c43e00640 1 Processor -- start 2024-02-02T13:32:52.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.808+0000 7f8c43e00640 1 -- start start 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15ad10 0x7f8c3c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c3c15b0f0 0x7f8c3c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8c3c0cb580 0x7f8c3c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cbf50 con 0x7f8c3c0cb580 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cd610 con 0x7f8c3c15b0f0 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.809+0000 7f8c43e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c06d910 con 0x7f8c3c15ad10 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8c3c0cb580 0x7f8c3c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8c3c0cb580 0x7f8c3c0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58234/0 (socket says 172.21.15.91:58234) 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 -- 172.21.15.91:0/2259519731 learned_addr learned my addr 172.21.15.91:0/2259519731 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:52.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41400640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15ad10 0x7f8c3c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c40a00640 1 --2- 172.21.15.91:0/2259519731 >> v2:172.21.15.113:3300/0 conn(0x7f8c3c15b0f0 0x7f8c3c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 -- 172.21.15.91:0/2259519731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15ad10 msgr2=0x7f8c3c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15ad10 0x7f8c3c06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 -- 172.21.15.91:0/2259519731 >> v2:172.21.15.113:3300/0 conn(0x7f8c3c15b0f0 msgr2=0x7f8c3c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 --2- 172.21.15.91:0/2259519731 >> v2:172.21.15.113:3300/0 conn(0x7f8c3c15b0f0 0x7f8c3c06d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.810+0000 7f8c41e00640 1 -- 172.21.15.91:0/2259519731 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0cc850 con 0x7f8c3c0cb580 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c41e00640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c0cba10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0cb480 tx=0x7f8c3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148d212228c0508f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c3be00640 1 -- 172.21.15.91:0/2259519731 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3402ca70 con 0x7f8c3c0cb580 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c3be00640 1 -- 172.21.15.91:0/2259519731 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3402cbd0 con 0x7f8c3c0cb580 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c3be00640 1 -- 172.21.15.91:0/2259519731 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340343f0 con 0x7f8c3c0cb580 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c43e00640 1 -- 172.21.15.91:0/2259519731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c0cb580 msgr2=0x7f8c3c0cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.812+0000 7f8c43e00640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c0cba10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0cb480 tx=0x7f8c3402bed0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 -- 172.21.15.91:0/2259519731 shutdown_connections 2024-02-02T13:32:52.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c0cba10 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 --2- 172.21.15.91:0/2259519731 >> v2:172.21.15.113:3300/0 conn(0x7f8c3c15b0f0 0x7f8c3c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 --2- 172.21.15.91:0/2259519731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15ad10 0x7f8c3c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 -- 172.21.15.91:0/2259519731 >> 172.21.15.91:0/2259519731 conn(0x7f8c3c0b0700 msgr2=0x7f8c3c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:52.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 -- 172.21.15.91:0/2259519731 shutdown_connections 2024-02-02T13:32:52.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.813+0000 7f8c43e00640 1 -- 172.21.15.91:0/2259519731 wait complete. 2024-02-02T13:32:52.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.814+0000 7f8c43e00640 1 Processor -- start 2024-02-02T13:32:52.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.814+0000 7f8c43e00640 1 -- start start 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15b0f0 0x7f8c3c0ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0a8760 con 0x7f8c3c15ad10 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0aade0 con 0x7f8c3c0cb580 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.815+0000 7f8c43e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0c7bf0 con 0x7f8c3c15b0f0 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58240/0 (socket says 172.21.15.91:58240) 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 -- 172.21.15.91:0/1701315118 learned_addr learned my addr 172.21.15.91:0/1701315118 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c41e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15b0f0 0x7f8c3c0ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c41400640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 -- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15b0f0 msgr2=0x7f8c3c0ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15b0f0 0x7f8c3c0ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 -- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 msgr2=0x7f8c3c14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 -- 172.21.15.91:0/1701315118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34009b30 con 0x7f8c3c15ad10 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c41400640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c40a00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d5f0 tx=0x7f8c2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c240125f0 con 0x7f8c3c15ad10 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c24012750 con 0x7f8c3c15ad10 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.816+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24012b00 con 0x7f8c3c15ad10 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.817+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c0c7ed0 con 0x7f8c3c15ad10 2024-02-02T13:32:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.817+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c0c8490 con 0x7f8c3c15ad10 2024-02-02T13:32:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.818+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c066cf0 con 0x7f8c3c15ad10 2024-02-02T13:32:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.818+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8c24012c60 con 0x7f8c3c15ad10 2024-02-02T13:32:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.819+0000 7f8c39600640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 0x7f8c1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.819+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c24088a70 con 0x7f8c3c15ad10 2024-02-02T13:32:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.820+0000 7f8c41400640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 0x7f8c1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:52.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.820+0000 7f8c41400640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 0x7f8c1406c8f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006fd0 tx=0x7f8c2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:52.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:52.824+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8c24054310 con 0x7f8c3c15ad10 2024-02-02T13:32:53.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.014+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8c3c070dc0 con 0x7f8c3c15ad10 2024-02-02T13:32:53.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.015+0000 7f8c39600640 1 -- 172.21.15.91:0/1701315118 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v157) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8c24057930 con 0x7f8c3c15ad10 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:32:53.017 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.017+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 msgr2=0x7f8c1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.017+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 0x7f8c1406c8f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006fd0 tx=0x7f8c2c008040 comp rx=0 tx=0).stop 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.017+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 msgr2=0x7f8c3c0ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.017+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d5f0 tx=0x7f8c2400dab0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 shutdown_connections 2024-02-02T13:32:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8c1406a430 0x7f8c1406c8f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c3c15b0f0 0x7f8c3c0ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c3c15ad10 0x7f8c3c0ab0b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 --2- 172.21.15.91:0/1701315118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c3c0cb580 0x7f8c3c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 >> 172.21.15.91:0/1701315118 conn(0x7f8c3c0b0700 msgr2=0x7f8c3c0c1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 shutdown_connections 2024-02-02T13:32:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.018+0000 7f8c43e00640 1 -- 172.21.15.91:0/1701315118 wait complete. 2024-02-02T13:32:53.037 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:32:53.038 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:32:53.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.248+0000 7fe07e400640 1 Processor -- start 2024-02-02T13:32:53.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.248+0000 7fe07e400640 1 -- start start 2024-02-02T13:32:53.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe07e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0780b0b90 0x7fe0780cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe07e400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0780ccdd0 con 0x7fe078161660 2024-02-02T13:32:53.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe07e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0780cc7c0 con 0x7fe0780b0700 2024-02-02T13:32:53.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe07e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780c9570 con 0x7fe0780b0b90 2024-02-02T13:32:53.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe077400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe078161660 0x7fe07806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49892/0 (socket says 172.21.15.91:49892) 2024-02-02T13:32:53.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.250+0000 7fe077400640 1 -- 172.21.15.91:0/592210442 learned_addr learned my addr 172.21.15.91:0/592210442 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe077e00640 1 --2- 172.21.15.91:0/592210442 >> v2:172.21.15.113:3300/0 conn(0x7fe0780b0b90 0x7fe0780cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520547128 0 0) 0x7fe0780ccdd0 con 0x7fe078161660 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe058003520 con 0x7fe078161660 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141916476 0 0) 0x7fe058003520 con 0x7fe078161660 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 >> v2:172.21.15.113:3300/0 conn(0x7fe0780b0b90 msgr2=0x7fe0780cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 --2- 172.21.15.91:0/592210442 >> v2:172.21.15.113:3300/0 conn(0x7fe0780b0b90 0x7fe0780cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 >> v1:172.21.15.113:6789/0 conn(0x7fe0780b0700 legacy=0x7fe0780cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780c9850 con 0x7fe078161660 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe068001670 con 0x7fe078161660 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 >> v1:172.21.15.91:6789/0 conn(0x7fe078161660 legacy=0x7fe07806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe058003970 0x7fe058003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 0x7fe058004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe058005180 0x7fe05800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780c9570 con 0x7fe0580047c0 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05800c120 con 0x7fe058005180 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.251+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05800c2e0 con 0x7fe058003970 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 0x7fe058004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe077400640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe058003970 0x7fe058003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 -- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe058003970 msgr2=0x7fe058003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe058003970 0x7fe058003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 -- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe058005180 msgr2=0x7fe05800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe058005180 0x7fe05800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe06fe00640 1 -- 172.21.15.91:0/592210442 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058003520 con 0x7fe0580047c0 2024-02-02T13:32:53.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.252+0000 7fe077e00640 1 -- 172.21.15.91:0/592210442 reap_dead start 2024-02-02T13:32:53.256 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.253+0000 7fe06fe00640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 0x7fe058004c40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe060001750 tx=0x7fe0580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7944602d0afc98a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.253+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600052d0 con 0x7fe0580047c0 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.253+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06002e050 con 0x7fe0580047c0 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.253+0000 7fe076a00640 1 -- 172.21.15.91:0/592210442 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060003830 con 0x7fe0580047c0 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 -- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 msgr2=0x7fe058004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 0x7fe058004c40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe060001750 tx=0x7fe0580057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 -- 172.21.15.91:0/592210442 shutdown_connections 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 --2- 172.21.15.91:0/592210442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0580047c0 0x7fe058004c40 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 -- 172.21.15.91:0/592210442 >> 172.21.15.91:0/592210442 conn(0x7fe0780cec80 msgr2=0x7fe0780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 -- 172.21.15.91:0/592210442 shutdown_connections 2024-02-02T13:32:53.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.254+0000 7fe07e400640 1 -- 172.21.15.91:0/592210442 wait complete. 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.256+0000 7fe07e400640 1 Processor -- start 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.256+0000 7fe07e400640 1 -- start start 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 0x7fe0780c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0780b3c00 0x7fe078070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781fa560 con 0x7fe078070c60 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe078160b40 con 0x7fe0780b3c00 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe07e400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780c5d60 con 0x7fe0580047c0 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58270/0 (socket says 172.21.15.91:58270) 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077e00640 1 -- 172.21.15.91:0/1219463783 learned_addr learned my addr 172.21.15.91:0/1219463783 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 0x7fe0780c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe06fe00640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0780b3c00 0x7fe078070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077e00640 1 -- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 msgr2=0x7fe0780c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.257+0000 7fe077e00640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 0x7fe0780c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe077e00640 1 -- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0780b3c00 msgr2=0x7fe078070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe077e00640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0780b3c00 0x7fe078070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe077e00640 1 -- 172.21.15.91:0/1219463783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780c6040 con 0x7fe078070c60 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe077400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 0x7fe0780c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:53.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe077e00640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0780c26f0 tx=0x7fe07000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070010040 con 0x7fe078070c60 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07000ba50 con 0x7fe078070c60 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0780c6320 con 0x7fe078070c60 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07000be00 con 0x7fe078070c60 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.258+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0780c3ef0 con 0x7fe078070c60 2024-02-02T13:32:53.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.259+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0781fa6a0 con 0x7fe078070c60 2024-02-02T13:32:53.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.260+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe07001d400 con 0x7fe078070c60 2024-02-02T13:32:53.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.261+0000 7fe06f400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 0x7fe05006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.261+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe070088c80 con 0x7fe078070c60 2024-02-02T13:32:53.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.262+0000 7fe077400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 0x7fe05006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.262+0000 7fe077400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 0x7fe05006c8f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe068007950 tx=0x7fe0680014b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.266+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe0781fa6a0 con 0x7fe078070c60 2024-02-02T13:32:53.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.458+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe0781fa6a0 con 0x7fe078070c60 2024-02-02T13:32:53.619 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.617+0000 7fe06f400640 1 -- 172.21.15.91:0/1219463783 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v158) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe0781fa6a0 con 0x7fe078070c60 2024-02-02T13:32:53.619 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.623+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 msgr2=0x7fe05006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.623+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 0x7fe05006c8f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe068007950 tx=0x7fe0680014b0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.623+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 msgr2=0x7fe0781fa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.623+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0780c26f0 tx=0x7fe07000eea0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 shutdown_connections 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe05006a430 0x7fe05006c8f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe078070c60 0x7fe0781fa980 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0780b3c00 0x7fe078070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 --2- 172.21.15.91:0/1219463783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580047c0 0x7fe0780c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 >> 172.21.15.91:0/1219463783 conn(0x7fe0780cec80 msgr2=0x7fe078165140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 shutdown_connections 2024-02-02T13:32:53.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.624+0000 7fe07e400640 1 -- 172.21.15.91:0/1219463783 wait complete. 2024-02-02T13:32:53.645 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:32:53.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.854+0000 7f04c8400640 1 Processor -- start 2024-02-02T13:32:53.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.854+0000 7f04c8400640 1 -- start start 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.855+0000 7f04c8400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04c00b8a50 0x7f04c00d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.855+0000 7f04c8400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00cd930 con 0x7f04c00b2260 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04c8400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00d1680 con 0x7f04c00d0070 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04c8400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c00d1960 con 0x7f04c00b8a50 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04c5000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f04c00b2260 0x7f04c00b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49898/0 (socket says 172.21.15.91:49898) 2024-02-02T13:32:53.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04c5000640 1 -- 172.21.15.91:0/893537419 learned_addr learned my addr 172.21.15.91:0/893537419 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412573111 0 0) 0x7f04b0001110 con 0x7f04c00b2260 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0003520 con 0x7f04c00b2260 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.856+0000 7f04c6400640 1 --2- 172.21.15.91:0/893537419 >> v2:172.21.15.113:3300/0 conn(0x7f04c00b8a50 0x7f04c00d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410701142 0 0) 0x7f04a0003520 con 0x7f04c00b2260 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 >> v2:172.21.15.113:3300/0 conn(0x7f04c00b8a50 msgr2=0x7f04c00d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 --2- 172.21.15.91:0/893537419 >> v2:172.21.15.113:3300/0 conn(0x7f04c00b8a50 0x7f04c00d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 >> v1:172.21.15.113:6789/0 conn(0x7f04c00d0070 legacy=0x7f04c00d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00d1c40 con 0x7f04c00b2260 2024-02-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f04b0001800 con 0x7f04c00b2260 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 >> v1:172.21.15.91:6789/0 conn(0x7f04c00b2260 legacy=0x7f04c00b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0003970 0x7f04a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04a00047c0 0x7f04a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 0x7f04a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00d1960 con 0x7f04a0005180 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a000c020 con 0x7f04a0003970 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a000c1e0 con 0x7f04a00047c0 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.857+0000 7f04c6400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 0x7f04a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c5000640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04a00047c0 0x7f04a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c5a00640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0003970 0x7f04a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c6400640 1 -- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04a00047c0 msgr2=0x7f04a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c6400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04a00047c0 0x7f04a0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c6400640 1 -- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0003970 msgr2=0x7f04a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c6400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0003970 0x7f04a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c6400640 1 -- 172.21.15.91:0/893537419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0003520 con 0x7f04a0005180 2024-02-02T13:32:53.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.858+0000 7f04c5a00640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0003970 0x7f04a0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:53.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.859+0000 7f04c6400640 1 -- 172.21.15.91:0/893537419 reap_dead start 2024-02-02T13:32:53.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.859+0000 7f04c6400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 0x7f04a000b980 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009fc0 tx=0x7f04a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d25ec52951f3df server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.859+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b802cc60 con 0x7f04a0005180 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.859+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800f040 con 0x7f04a0005180 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.859+0000 7f04bfe00640 1 -- 172.21.15.91:0/893537419 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8003990 con 0x7f04a0005180 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 -- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 msgr2=0x7f04a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 0x7f04a000b980 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009fc0 tx=0x7f04a00057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 -- 172.21.15.91:0/893537419 shutdown_connections 2024-02-02T13:32:53.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 --2- 172.21.15.91:0/893537419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04a0005180 0x7f04a000b980 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 -- 172.21.15.91:0/893537419 >> 172.21.15.91:0/893537419 conn(0x7f04c0070660 msgr2=0x7f04c0070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:53.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 -- 172.21.15.91:0/893537419 shutdown_connections 2024-02-02T13:32:53.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.860+0000 7f04c8400640 1 -- 172.21.15.91:0/893537419 wait complete. 2024-02-02T13:32:53.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.861+0000 7f04c8400640 1 Processor -- start 2024-02-02T13:32:53.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.862+0000 7f04c8400640 1 -- start start 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.862+0000 7f04c8400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 0x7f04c00c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.862+0000 7f04c8400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04c014e810 0x7f04c00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.862+0000 7f04c8400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c8400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00c9470 con 0x7f04c015aa10 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c8400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00c7c80 con 0x7f04a0005180 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c8400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00c7f60 con 0x7f04c014e810 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58274/0 (socket says 172.21.15.91:58274) 2024-02-02T13:32:53.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 -- 172.21.15.91:0/3536321909 learned_addr learned my addr 172.21.15.91:0/3536321909 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c5a00640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 0x7f04c00c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c5000640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04c014e810 0x7f04c00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 -- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04c014e810 msgr2=0x7f04c00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04c014e810 0x7f04c00cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 -- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 msgr2=0x7f04c00c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 0x7f04c00c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c6400640 1 -- 172.21.15.91:0/3536321909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00c8240 con 0x7f04c015aa10 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.863+0000 7f04c5a00640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 0x7f04c00c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04c6400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f04b802c6d0 tx=0x7f04b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8035e40 con 0x7f04c015aa10 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8003990 con 0x7f04c015aa10 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c00cb6c0 con 0x7f04c015aa10 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8003d80 con 0x7f04c015aa10 2024-02-02T13:32:53.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.864+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00ca920 con 0x7f04c015aa10 2024-02-02T13:32:53.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.865+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c0066cf0 con 0x7f04c015aa10 2024-02-02T13:32:53.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.866+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f04b8004030 con 0x7f04c015aa10 2024-02-02T13:32:53.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.867+0000 7f04bd600640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 0x7f049806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:53.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.867+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f04b80abda0 con 0x7f04c015aa10 2024-02-02T13:32:53.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.867+0000 7f04c5a00640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 0x7f049806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:53.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.868+0000 7f04c5a00640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 0x7f049806c8f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001690 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:53.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:53.871+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f04b8077690 con 0x7f04c015aa10 2024-02-02T13:32:54.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.061+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f04c00c80a0 con 0x7f04c015aa10 2024-02-02T13:32:54.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.631+0000 7f04bd600640 1 -- 172.21.15.91:0/3536321909 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v159) v1 ==== 170+0+0 (secure 0 0 0) 0x7f04c00c80a0 con 0x7f04c015aa10 2024-02-02T13:32:54.638 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:32:54.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 msgr2=0x7f049806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 0x7f049806c8f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001690 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-02-02T13:32:54.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 msgr2=0x7f04c00ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f04b802c6d0 tx=0x7f04b8009750 comp rx=0 tx=0).stop 2024-02-02T13:32:54.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 shutdown_connections 2024-02-02T13:32:54.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f049806a430 0x7f049806c8f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04c014e810 0x7f04c00cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04c015aa10 0x7f04c00ca200 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 --2- 172.21.15.91:0/3536321909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04a0005180 0x7f04c00c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.641+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 >> 172.21.15.91:0/3536321909 conn(0x7f04c0070660 msgr2=0x7f04c00b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:54.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.644+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 shutdown_connections 2024-02-02T13:32:54.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.644+0000 7f04c8400640 1 -- 172.21.15.91:0/3536321909 wait complete. 2024-02-02T13:32:54.669 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:32:54.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.853+0000 7f2978200640 1 Processor -- start 2024-02-02T13:32:54.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.853+0000 7f2978200640 1 -- start start 2024-02-02T13:32:54.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2978200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2970151f90 0x7f297015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2978200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f297006f4f0 0x7f29700b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2978200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2970070f80 con 0x7f29700b0c40 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2978200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29700b10d0 con 0x7f2970151f90 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2978200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2970152370 con 0x7f297006f4f0 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2970151f90 0x7f297015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2970151f90 0x7f297015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58288/0 (socket says 172.21.15.91:58288) 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 -- 172.21.15.91:0/2002000821 learned_addr learned my addr 172.21.15.91:0/2002000821 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2976200640 1 --1- 172.21.15.91:0/2002000821 >> v1:172.21.15.91:6789/0 conn(0x7f29700b0c40 0x7f29700b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49914/0 (socket says 172.21.15.91:49914) 2024-02-02T13:32:54.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 -- 172.21.15.91:0/2002000821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f297006f4f0 msgr2=0x7f29700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:54.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 --2- 172.21.15.91:0/2002000821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f297006f4f0 0x7f29700b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 -- 172.21.15.91:0/2002000821 >> v1:172.21.15.91:6789/0 conn(0x7f29700b0c40 legacy=0x7f29700b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.855+0000 7f2975800640 1 -- 172.21.15.91:0/2002000821 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700cb010 con 0x7f2970151f90 2024-02-02T13:32:54.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.857+0000 7f2975800640 1 --2- 172.21.15.91:0/2002000821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2970151f90 0x7f297015f5a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2960009870 tx=0x7f296002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2767507945e180 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:54.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.857+0000 7f296fe00640 1 -- 172.21.15.91:0/2002000821 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960030840 con 0x7f2970151f90 2024-02-02T13:32:54.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.857+0000 7f296fe00640 1 -- 172.21.15.91:0/2002000821 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29600309a0 con 0x7f2970151f90 2024-02-02T13:32:54.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.857+0000 7f296fe00640 1 -- 172.21.15.91:0/2002000821 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960030d30 con 0x7f2970151f90 2024-02-02T13:32:54.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 -- 172.21.15.91:0/2002000821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2970151f90 msgr2=0x7f297015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 --2- 172.21.15.91:0/2002000821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2970151f90 0x7f297015f5a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2960009870 tx=0x7f296002b980 comp rx=0 tx=0).stop 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 -- 172.21.15.91:0/2002000821 shutdown_connections 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 --2- 172.21.15.91:0/2002000821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f297006f4f0 0x7f29700b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 --2- 172.21.15.91:0/2002000821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2970151f90 0x7f297015f5a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 -- 172.21.15.91:0/2002000821 >> 172.21.15.91:0/2002000821 conn(0x7f29700cec80 msgr2=0x7f29700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 -- 172.21.15.91:0/2002000821 shutdown_connections 2024-02-02T13:32:54.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.858+0000 7f2978200640 1 -- 172.21.15.91:0/2002000821 wait complete. 2024-02-02T13:32:54.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.860+0000 7f2978200640 1 Processor -- start 2024-02-02T13:32:54.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.860+0000 7f2978200640 1 -- start start 2024-02-02T13:32:54.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 0x7f297014d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970151f90 0x7f29700c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297014db20 con 0x7f29700b0c40 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c4b80 con 0x7f297006f4f0 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2978200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c35a0 con 0x7f2970151f90 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2974e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2974e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58304/0 (socket says 172.21.15.91:58304) 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2974e00640 1 -- 172.21.15.91:0/160561987 learned_addr learned my addr 172.21.15.91:0/160561987 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2975800640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 0x7f297014d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2974e00640 1 -- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970151f90 msgr2=0x7f29700c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2976200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970151f90 0x7f29700c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.861+0000 7f2974e00640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970151f90 0x7f29700c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2974e00640 1 -- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 msgr2=0x7f297014d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2974e00640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 0x7f297014d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:54.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2974e00640 1 -- 172.21.15.91:0/160561987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600094d0 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2975800640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 0x7f297014d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2974e00640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f295800b410 tx=0x7f295800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958011830 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29580119b0 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958011d60 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29700c3880 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.862+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29700c3e40 con 0x7f29700b0c40 2024-02-02T13:32:54.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.863+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970066cf0 con 0x7f29700b0c40 2024-02-02T13:32:54.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.864+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f295801a400 con 0x7f29700b0c40 2024-02-02T13:32:54.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.865+0000 7f296d600640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 0x7f294806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:54.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.865+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2958088a30 con 0x7f29700b0c40 2024-02-02T13:32:54.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.866+0000 7f2975800640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 0x7f294806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:54.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.866+0000 7f2975800640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 0x7f294806c8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2960036000 tx=0x7f296003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:54.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:54.876+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2958054070 con 0x7f29700b0c40 2024-02-02T13:32:55.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.112+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f29700b0a20 con 0x7f29700b0c40 2024-02-02T13:32:55.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.125+0000 7f296d600640 1 -- 172.21.15.91:0/160561987 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2958015020 con 0x7f29700b0c40 2024-02-02T13:32:55.127 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 18 and data pool 19 2024-02-02T13:32:55.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 msgr2=0x7f294806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 0x7f294806c8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2960036000 tx=0x7f296003b040 comp rx=0 tx=0).stop 2024-02-02T13:32:55.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 msgr2=0x7f29700c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f295800b410 tx=0x7f295800b8d0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 shutdown_connections 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f294806a430 0x7f294806c8f0 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2960036000 tx=0x7f296003b040 comp rx=0 tx=0).stop 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970151f90 0x7f29700c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29700b0c40 0x7f29700c4640 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 --2- 172.21.15.91:0/160561987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297006f4f0 0x7f297014d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.135+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 >> 172.21.15.91:0/160561987 conn(0x7f29700cec80 msgr2=0x7f29700c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:55.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.142+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 shutdown_connections 2024-02-02T13:32:55.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.143+0000 7f2978200640 1 -- 172.21.15.91:0/160561987 wait complete. 2024-02-02T13:32:55.167 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.345+0000 7f8187000640 1 Processor -- start 2024-02-02T13:32:55.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.345+0000 7f8187000640 1 -- start start 2024-02-02T13:32:55.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.346+0000 7f8187000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 0x7f81800aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8187000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81800af040 0x7f8180070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8187000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800717b0 con 0x7f81800cbbf0 2024-02-02T13:32:55.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8187000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800af4b0 con 0x7f81800af040 2024-02-02T13:32:55.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8187000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81800714d0 con 0x7f81800cd6e0 2024-02-02T13:32:55.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8185c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f81800cbbf0 0x7f8180160d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49920/0 (socket says 172.21.15.91:49920) 2024-02-02T13:32:55.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8185c00640 1 -- 172.21.15.91:0/3993906412 learned_addr learned my addr 172.21.15.91:0/3993906412 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:55.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8185200640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 0x7f81800aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863644340 0 0) 0x7f81800717b0 con 0x7f81800cbbf0 2024-02-02T13:32:55.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8160003520 con 0x7f81800cbbf0 2024-02-02T13:32:55.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.347+0000 7f8186600640 1 --2- 172.21.15.91:0/3993906412 >> v2:172.21.15.113:3300/0 conn(0x7f81800af040 0x7f8180070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963044930 0 0) 0x7f8160003520 con 0x7f81800cbbf0 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 msgr2=0x7f81800aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 0x7f81800aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 >> v2:172.21.15.113:3300/0 conn(0x7f81800af040 msgr2=0x7f8180070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 --2- 172.21.15.91:0/3993906412 >> v2:172.21.15.113:3300/0 conn(0x7f81800af040 0x7f8180070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800b1740 con 0x7f81800cbbf0 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8170001670 con 0x7f81800cbbf0 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 >> v1:172.21.15.91:6789/0 conn(0x7f81800cbbf0 legacy=0x7f8180160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8160003970 0x7f8160003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81600047c0 0x7f8160004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 0x7f8160005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800af4b0 con 0x7f81600047c0 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800714d0 con 0x7f8160003970 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816000ba30 con 0x7f81800cd6e0 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81600047c0 0x7f8160004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 -- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 msgr2=0x7f8160005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81800cd6e0 0x7f8160005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 -- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8160003970 msgr2=0x7f8160003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8160003970 0x7f8160003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.348+0000 7f8185200640 1 -- 172.21.15.91:0/3993906412 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160003520 con 0x7f81600047c0 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.349+0000 7f8185200640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81600047c0 0x7f8160004c40 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8168009a90 tx=0x7f81600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e586c75dd0fe3719 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816802c600 con 0x7f81600047c0 2024-02-02T13:32:55.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816802c760 con 0x7f81600047c0 2024-02-02T13:32:55.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f817fe00640 1 -- 172.21.15.91:0/3993906412 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816802caf0 con 0x7f81600047c0 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8187000640 1 -- 172.21.15.91:0/3993906412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81600047c0 msgr2=0x7f8160004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8187000640 1 --2- 172.21.15.91:0/3993906412 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81600047c0 0x7f8160004c40 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8168009a90 tx=0x7f81600057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8186600640 1 -- 172.21.15.91:0/3993906412 reap_dead start 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8187000640 1 -- 172.21.15.91:0/3993906412 shutdown_connections 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8187000640 1 -- 172.21.15.91:0/3993906412 >> 172.21.15.91:0/3993906412 conn(0x7f81800cfba0 msgr2=0x7f81800cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.350+0000 7f8187000640 1 -- 172.21.15.91:0/3993906412 shutdown_connections 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.351+0000 7f8187000640 1 -- 172.21.15.91:0/3993906412 wait complete. 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.352+0000 7f8187000640 1 Processor -- start 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.352+0000 7f8187000640 1 -- start start 2024-02-02T13:32:55.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81600047c0 0x7f81801f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8180151260 0x7f81800c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81801f27f0 con 0x7f8180150de0 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800c5a90 con 0x7f81600047c0 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8187000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81801508f0 con 0x7f8180151260 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.353+0000 7f8185200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58324/0 (socket says 172.21.15.91:58324) 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 -- 172.21.15.91:0/357306855 learned_addr learned my addr 172.21.15.91:0/357306855 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 -- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8180151260 msgr2=0x7f81800c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8180151260 0x7f81800c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 -- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81600047c0 msgr2=0x7f81801f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81600047c0 0x7f81801f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 -- 172.21.15.91:0/357306855 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800c69a0 con 0x7f8180150de0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.354+0000 7f8185200640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f816802c1a0 tx=0x7f8168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.355+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168003810 con 0x7f8180150de0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.355+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8168003970 con 0x7f8180150de0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.355+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168003ce0 con 0x7f8180150de0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.355+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8180066520 con 0x7f8180150de0 2024-02-02T13:32:55.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.355+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81800c5440 con 0x7f8180150de0 2024-02-02T13:32:55.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.356+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818004dff0 con 0x7f8180150de0 2024-02-02T13:32:55.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.357+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8168040450 con 0x7f8180150de0 2024-02-02T13:32:55.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.358+0000 7f817d600640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 0x7f815806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.358+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81680aa710 con 0x7f8180150de0 2024-02-02T13:32:55.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.358+0000 7f8185c00640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 0x7f815806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.359+0000 7f8185c00640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 0x7f815806c8f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8170007a50 tx=0x7f81700015e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.363+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8168075d00 con 0x7f8180150de0 2024-02-02T13:32:55.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.548+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8180074480 con 0x7f8180150de0 2024-02-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.551+0000 7f817d600640 1 -- 172.21.15.91:0/357306855 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f81680044e0 con 0x7f8180150de0 2024-02-02T13:32:55.553 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:55.553 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":160,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:55.114428+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:55.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 msgr2=0x7f815806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 0x7f815806c8f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8170007a50 tx=0x7f81700015e0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 msgr2=0x7f81800c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f816802c1a0 tx=0x7f8168004730 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 shutdown_connections 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f815806a430 0x7f815806c8f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8180151260 0x7f81800c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8180150de0 0x7f81800c5f20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 --2- 172.21.15.91:0/357306855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81600047c0 0x7f81801f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 >> 172.21.15.91:0/357306855 conn(0x7f81800cfba0 msgr2=0x7f8180160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 shutdown_connections 2024-02-02T13:32:55.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.553+0000 7f8187000640 1 -- 172.21.15.91:0/357306855 wait complete. 2024-02-02T13:32:55.572 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.765+0000 7f4827200640 1 Processor -- start 2024-02-02T13:32:55.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.766+0000 7f4827200640 1 -- start start 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f4827200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48200b0e10 0x7f48200c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f4827200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200cc7c0 con 0x7f482015a010 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f4827200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200ce930 con 0x7f48200ce130 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f4827200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48200cd640 con 0x7f48200b0e10 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f481fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f482015a010 0x7f48200ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49932/0 (socket says 172.21.15.91:49932) 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f481fe00640 1 -- 172.21.15.91:0/3827669423 learned_addr learned my addr 172.21.15.91:0/3827669423 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:55.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.767+0000 7f4825200640 1 --2- 172.21.15.91:0/3827669423 >> v2:172.21.15.91:3300/0 conn(0x7f48200b0e10 0x7f48200c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.768+0000 7f481f400640 1 -- 172.21.15.91:0/3827669423 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913460826 0 0) 0x7f48200cc7c0 con 0x7f482015a010 2024-02-02T13:32:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.768+0000 7f481f400640 1 -- 172.21.15.91:0/3827669423 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4800005730 con 0x7f482015a010 2024-02-02T13:32:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.768+0000 7f4825200640 1 -- 172.21.15.91:0/3827669423 >> v1:172.21.15.113:6789/0 conn(0x7f48200ce130 legacy=0x7f48200b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.768+0000 7f4825200640 1 -- 172.21.15.91:0/3827669423 >> v1:172.21.15.91:6789/0 conn(0x7f482015a010 legacy=0x7f48200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.768+0000 7f4825200640 1 -- 172.21.15.91:0/3827669423 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482006c890 con 0x7f48200b0e10 2024-02-02T13:32:55.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.769+0000 7f4825200640 1 --2- 172.21.15.91:0/3827669423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200c9670 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f481802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583c74827119d654 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.769+0000 7f481f400640 1 -- 172.21.15.91:0/3827669423 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481802e070 con 0x7f48200b0e10 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.769+0000 7f481f400640 1 -- 172.21.15.91:0/3827669423 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48180315a0 con 0x7f48200b0e10 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.769+0000 7f481f400640 1 -- 172.21.15.91:0/3827669423 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818031930 con 0x7f48200b0e10 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 -- 172.21.15.91:0/3827669423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 msgr2=0x7f48200c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 --2- 172.21.15.91:0/3827669423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200c9670 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f48180097a0 tx=0x7f481802b840 comp rx=0 tx=0).stop 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 -- 172.21.15.91:0/3827669423 shutdown_connections 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 --2- 172.21.15.91:0/3827669423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200c9670 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 -- 172.21.15.91:0/3827669423 >> 172.21.15.91:0/3827669423 conn(0x7f48200cec80 msgr2=0x7f48200cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 -- 172.21.15.91:0/3827669423 shutdown_connections 2024-02-02T13:32:55.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.770+0000 7f4827200640 1 -- 172.21.15.91:0/3827669423 wait complete. 2024-02-02T13:32:55.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.772+0000 7f4827200640 1 Processor -- start 2024-02-02T13:32:55.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.772+0000 7f4827200640 1 -- start start 2024-02-02T13:32:55.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200ce130 0x7f48201f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 0x7f48201f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c5cc0 con 0x7f48200b0e10 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200b47b0 con 0x7f48200ce130 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4827200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48201f4000 con 0x7f482015a010 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58334/0 (socket says 172.21.15.91:58334) 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 -- 172.21.15.91:0/2765770165 learned_addr learned my addr 172.21.15.91:0/2765770165 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4817200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200ce130 0x7f48201f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f4825200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 0x7f48201f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 -- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 msgr2=0x7f48201f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 0x7f48201f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 -- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200ce130 msgr2=0x7f48201f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.773+0000 7f481fe00640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200ce130 0x7f48201f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f481fe00640 1 -- 172.21.15.91:0/2765770165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009400 con 0x7f48200b0e10 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f4825200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 0x7f48201f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f481fe00640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f481000cbf0 tx=0x7f481000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000ad40 con 0x7f48200b0e10 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810012400 con 0x7f48200b0e10 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000aea0 con 0x7f48200b0e10 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48201f3150 con 0x7f48200b0e10 2024-02-02T13:32:55.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.774+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48201f3710 con 0x7f48200b0e10 2024-02-02T13:32:55.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.775+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820073830 con 0x7f48200b0e10 2024-02-02T13:32:55.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.776+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4810007070 con 0x7f48200b0e10 2024-02-02T13:32:55.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.777+0000 7f481cc00640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 0x7f47f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:55.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.777+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48100889f0 con 0x7f48200b0e10 2024-02-02T13:32:55.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.777+0000 7f4817200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 0x7f47f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:55.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.778+0000 7f4817200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 0x7f47f806c8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f48200add00 tx=0x7f4808009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:55.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.781+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4810053f40 con 0x7f48200b0e10 2024-02-02T13:32:55.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.969+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f482015a3f0 con 0x7f48200b0e10 2024-02-02T13:32:55.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.972+0000 7f481cc00640 1 -- 172.21.15.91:0/2765770165 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f4810017020 con 0x7f48200b0e10 2024-02-02T13:32:55.974 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:55.974 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":160,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:55.114428+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:55.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.974+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 msgr2=0x7f47f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.974+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 0x7f47f806c8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f48200add00 tx=0x7f4808009040 comp rx=0 tx=0).stop 2024-02-02T13:32:55.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.974+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 msgr2=0x7f48200b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:55.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.974+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f481000cbf0 tx=0x7f481000a430 comp rx=0 tx=0).stop 2024-02-02T13:32:55.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 shutdown_connections 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f47f806a430 0x7f47f806c8f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015a010 0x7f48201f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200ce130 0x7f48201f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 --2- 172.21.15.91:0/2765770165 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48200b0e10 0x7f48200b4270 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 >> 172.21.15.91:0/2765770165 conn(0x7f48200cec80 msgr2=0x7f48200c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 shutdown_connections 2024-02-02T13:32:55.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:55.975+0000 7f4827200640 1 -- 172.21.15.91:0/2765770165 wait complete. 2024-02-02T13:32:55.995 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:32:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.202+0000 7f024e600640 1 Processor -- start 2024-02-02T13:32:56.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.202+0000 7f024e600640 1 -- start start 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f024815b2f0 0x7f02480b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02480711f0 0x7f02480b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0248071650 0x7f02480b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024815fdc0 con 0x7f02480711f0 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024815b6d0 con 0x7f024815b2f0 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f024e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024815f1b0 con 0x7f0248071650 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f023e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02480711f0 0x7f02480b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f023e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02480711f0 0x7f02480b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58344/0 (socket says 172.21.15.91:58344) 2024-02-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.204+0000 7f023e800640 1 -- 172.21.15.91:0/1548259190 learned_addr learned my addr 172.21.15.91:0/1548259190 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:56.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.205+0000 7f023e800640 1 -- 172.21.15.91:0/1548259190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0248071650 msgr2=0x7f02480b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:56.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.205+0000 7f023e800640 1 --2- 172.21.15.91:0/1548259190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0248071650 0x7f02480b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.205+0000 7f023e800640 1 -- 172.21.15.91:0/1548259190 >> v2:172.21.15.113:3300/0 conn(0x7f024815b2f0 msgr2=0x7f02480b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:56.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.205+0000 7f023e800640 1 --2- 172.21.15.91:0/1548259190 >> v2:172.21.15.113:3300/0 conn(0x7f024815b2f0 0x7f02480b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.205+0000 7f023e800640 1 -- 172.21.15.91:0/1548259190 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024815f490 con 0x7f02480711f0 2024-02-02T13:32:56.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.206+0000 7f023e800640 1 --2- 172.21.15.91:0/1548259190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f02480b0690 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f02480b4890 tx=0x7f023002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3a54a1802ee2b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:56.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.206+0000 7f0246a00640 1 -- 172.21.15.91:0/1548259190 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300317c0 con 0x7f02480711f0 2024-02-02T13:32:56.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.206+0000 7f0246a00640 1 -- 172.21.15.91:0/1548259190 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230031920 con 0x7f02480711f0 2024-02-02T13:32:56.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.206+0000 7f0246a00640 1 -- 172.21.15.91:0/1548259190 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230031cb0 con 0x7f02480711f0 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 -- 172.21.15.91:0/1548259190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 msgr2=0x7f02480b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 --2- 172.21.15.91:0/1548259190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f02480b0690 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f02480b4890 tx=0x7f023002b860 comp rx=0 tx=0).stop 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 -- 172.21.15.91:0/1548259190 shutdown_connections 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 --2- 172.21.15.91:0/1548259190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0248071650 0x7f02480b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 --2- 172.21.15.91:0/1548259190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f02480b0690 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 --2- 172.21.15.91:0/1548259190 >> v2:172.21.15.113:3300/0 conn(0x7f024815b2f0 0x7f02480b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 -- 172.21.15.91:0/1548259190 >> 172.21.15.91:0/1548259190 conn(0x7f02480ceb90 msgr2=0x7f02480cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 -- 172.21.15.91:0/1548259190 shutdown_connections 2024-02-02T13:32:56.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.207+0000 7f024e600640 1 -- 172.21.15.91:0/1548259190 wait complete. 2024-02-02T13:32:56.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.209+0000 7f024e600640 1 Processor -- start 2024-02-02T13:32:56.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.209+0000 7f024e600640 1 -- start start 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0248071650 0x7f0248150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f024815b2f0 0x7f02481527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480c6bc0 con 0x7f02480711f0 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480c6ea0 con 0x7f0248071650 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f024e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480c7180 con 0x7f024815b2f0 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f0247400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f0247400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58352/0 (socket says 172.21.15.91:58352) 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.210+0000 7f0247400640 1 -- 172.21.15.91:0/2137952026 learned_addr learned my addr 172.21.15.91:0/2137952026 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:56.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247e00640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f024815b2f0 0x7f02481527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f023e800640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0248071650 0x7f0248150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 -- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f024815b2f0 msgr2=0x7f02481527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f024815b2f0 0x7f02481527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 -- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0248071650 msgr2=0x7f0248150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0248071650 0x7f0248150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 -- 172.21.15.91:0/2137952026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300094d0 con 0x7f02480711f0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f0247400640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f023800ba50 tx=0x7f023800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800cbe0 con 0x7f02480711f0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023800cd60 con 0x7f02480711f0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238014680 con 0x7f02480711f0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.211+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02480c7460 con 0x7f02480711f0 2024-02-02T13:32:56.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.212+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248153540 con 0x7f02480711f0 2024-02-02T13:32:56.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.213+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02480c7980 con 0x7f02480711f0 2024-02-02T13:32:56.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.213+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f023801d3b0 con 0x7f02480711f0 2024-02-02T13:32:56.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.214+0000 7f023fe00640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 0x7f022006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:56.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.214+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0238088920 con 0x7f02480711f0 2024-02-02T13:32:56.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.215+0000 7f023e800640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 0x7f022006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:56.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.215+0000 7f023e800640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 0x7f022006c8f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0230036000 tx=0x7f023003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:56.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.219+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f02480c7980 con 0x7f02480711f0 2024-02-02T13:32:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:56.431+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f02480c7980 con 0x7f02480711f0 2024-02-02T13:32:57.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.148+0000 7f023fe00640 1 -- 172.21.15.91:0/2137952026 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7f02480c7980 con 0x7f02480711f0 2024-02-02T13:32:57.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 msgr2=0x7f022006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 0x7f022006c8f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0230036000 tx=0x7f023003d040 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 msgr2=0x7f0248150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f023800ba50 tx=0x7f023800bf10 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 shutdown_connections 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f022006a430 0x7f022006c8f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f024815b2f0 0x7f02481527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0248071650 0x7f0248150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 --2- 172.21.15.91:0/2137952026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02480711f0 0x7f0248150950 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 >> 172.21.15.91:0/2137952026 conn(0x7f02480ceb90 msgr2=0x7f024814f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 shutdown_connections 2024-02-02T13:32:57.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.152+0000 7f024e600640 1 -- 172.21.15.91:0/2137952026 wait complete. 2024-02-02T13:32:57.176 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:32:57.177 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:32:57.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.385+0000 7fbdbfe00640 1 Processor -- start 2024-02-02T13:32:57.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.386+0000 7fbdbfe00640 1 -- start start 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdc0160610 0x7fbdc00d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbfe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbdc00d34b0 0x7fbdc00b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbfe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc00b4930 con 0x7fbdc00d2890 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbfe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc00b4c10 con 0x7fbdc00d34b0 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbfe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc00b4ef0 con 0x7fbdc0160610 2024-02-02T13:32:57.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbf400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbdc00d34b0 0x7fbdc00b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbf400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbdc00d34b0 0x7fbdc00b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58368/0 (socket says 172.21.15.91:58368) 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbf400640 1 -- 172.21.15.91:0/421820945 learned_addr learned my addr 172.21.15.91:0/421820945 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.387+0000 7fbdbe000640 1 --1- 172.21.15.91:0/421820945 >> v1:172.21.15.91:6789/0 conn(0x7fbdc00d2890 0x7fbdc00d2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49940/0 (socket says 172.21.15.91:49940) 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbea00640 1 --2- 172.21.15.91:0/421820945 >> v2:172.21.15.113:3300/0 conn(0x7fbdc0160610 0x7fbdc00d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbf400640 1 -- 172.21.15.91:0/421820945 >> v2:172.21.15.113:3300/0 conn(0x7fbdc0160610 msgr2=0x7fbdc00d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbf400640 1 --2- 172.21.15.91:0/421820945 >> v2:172.21.15.113:3300/0 conn(0x7fbdc0160610 0x7fbdc00d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbf400640 1 -- 172.21.15.91:0/421820945 >> v1:172.21.15.91:6789/0 conn(0x7fbdc00d2890 legacy=0x7fbdc00d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbf400640 1 -- 172.21.15.91:0/421820945 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00b51d0 con 0x7fbdc00d34b0 2024-02-02T13:32:57.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.388+0000 7fbdbd600640 1 -- 172.21.15.91:0/421820945 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470458286 0 0) 0x7fbdc00b4930 con 0x7fbdc00d2890 2024-02-02T13:32:57.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.389+0000 7fbdbf400640 1 --2- 172.21.15.91:0/421820945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00b60a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00d3960 tx=0x7fbdb802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fb578ed68c271 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:57.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.389+0000 7fbdbd600640 1 -- 172.21.15.91:0/421820945 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb802cb20 con 0x7fbdc00d34b0 2024-02-02T13:32:57.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.389+0000 7fbdbd600640 1 -- 172.21.15.91:0/421820945 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb802cc80 con 0x7fbdc00d34b0 2024-02-02T13:32:57.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.389+0000 7fbdbd600640 1 -- 172.21.15.91:0/421820945 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80343f0 con 0x7fbdc00d34b0 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 -- 172.21.15.91:0/421820945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d34b0 msgr2=0x7fbdc00b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/421820945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00b60a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00d3960 tx=0x7fbdb802bf80 comp rx=0 tx=0).stop 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 -- 172.21.15.91:0/421820945 shutdown_connections 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/421820945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00b60a0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/421820945 >> v2:172.21.15.113:3300/0 conn(0x7fbdc0160610 0x7fbdc00d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 -- 172.21.15.91:0/421820945 >> 172.21.15.91:0/421820945 conn(0x7fbdc0070690 msgr2=0x7fbdc0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:57.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 -- 172.21.15.91:0/421820945 shutdown_connections 2024-02-02T13:32:57.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.390+0000 7fbdbfe00640 1 -- 172.21.15.91:0/421820945 wait complete. 2024-02-02T13:32:57.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.392+0000 7fbdbfe00640 1 Processor -- start 2024-02-02T13:32:57.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.392+0000 7fbdbfe00640 1 -- start start 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc0160610 0x7fbdc015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00ab590 con 0x7fbdc00d2890 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00a9fb0 con 0x7fbdc00d34b0 2024-02-02T13:32:57.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbfe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00aa290 con 0x7fbdc0160610 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58382/0 (socket says 172.21.15.91:58382) 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.393+0000 7fbdbea00640 1 -- 172.21.15.91:0/4051022587 learned_addr learned my addr 172.21.15.91:0/4051022587 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 -- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc0160610 msgr2=0x7fbdc015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc0160610 0x7fbdc015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 -- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc00d34b0 msgr2=0x7fbdc00ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:57.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 -- 172.21.15.91:0/4051022587 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009b30 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbea00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c7f0 tx=0x7fbdb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0004220 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0004380 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0007840 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.394+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc00aa570 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.395+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc00aab30 con 0x7fbdc00d2890 2024-02-02T13:32:57.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.396+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc00aad50 con 0x7fbdc00d2890 2024-02-02T13:32:57.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.397+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbdb0007bb0 con 0x7fbdc00d2890 2024-02-02T13:32:57.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.398+0000 7fbdb6a00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 0x7fbd9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:57.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.398+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdb0088880 con 0x7fbdc00d2890 2024-02-02T13:32:57.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.398+0000 7fbdbe000640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 0x7fbd9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:57.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.399+0000 7fbdbe000640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 0x7fbd9806c8f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007fd0 tx=0x7fbda8007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:57.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.402+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbdb0053e70 con 0x7fbdc00d2890 2024-02-02T13:32:57.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:57.633+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbdc00cf210 con 0x7fbdc00d2890 2024-02-02T13:32:58.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.646+0000 7fbdb6a00640 1 -- 172.21.15.91:0/4051022587 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbdb0007e60 con 0x7fbdc00d2890 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 msgr2=0x7fbd9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 0x7fbd9806c8f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007fd0 tx=0x7fbda8007920 comp rx=0 tx=0).stop 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 msgr2=0x7fbdc015aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c7f0 tx=0x7fbdb000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 shutdown_connections 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd9806a430 0x7fbd9806c8f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc0160610 0x7fbdc015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc00d34b0 0x7fbdc00ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 --2- 172.21.15.91:0/4051022587 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdc00d2890 0x7fbdc015aee0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 >> 172.21.15.91:0/4051022587 conn(0x7fbdc0070690 msgr2=0x7fbdc00b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:58.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 shutdown_connections 2024-02-02T13:32:58.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.649+0000 7fbdbfe00640 1 -- 172.21.15.91:0/4051022587 wait complete. 2024-02-02T13:32:58.670 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:32:58.670 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:32:58.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.871+0000 7f789fe00640 1 Processor -- start 2024-02-02T13:32:58.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.871+0000 7f789fe00640 1 -- start start 2024-02-02T13:32:58.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.872+0000 7f789fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 0x7f78a006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.872+0000 7f789fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78a006c820 0x7f78a00cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.872+0000 7f789fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a00ad2e0 con 0x7f78a006cd50 2024-02-02T13:32:58.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.872+0000 7f789fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a00ad5c0 con 0x7f78a006c820 2024-02-02T13:32:58.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.872+0000 7f789fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a006b4f0 con 0x7f78a015d2c0 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78a006c820 0x7f78a00cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78a006c820 0x7f78a00cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58396/0 (socket says 172.21.15.91:58396) 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 -- 172.21.15.91:0/114901363 learned_addr learned my addr 172.21.15.91:0/114901363 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f789ea00640 1 --2- 172.21.15.91:0/114901363 >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 0x7f78a006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 -- 172.21.15.91:0/114901363 >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 msgr2=0x7f78a006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 --2- 172.21.15.91:0/114901363 >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 0x7f78a006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 -- 172.21.15.91:0/114901363 >> v1:172.21.15.91:6789/0 conn(0x7f78a006cd50 legacy=0x7f78a00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f7895e00640 1 -- 172.21.15.91:0/114901363 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a006b7d0 con 0x7f78a006c820 2024-02-02T13:32:58.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f789e000640 1 -- 172.21.15.91:0/114901363 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618643839 0 0) 0x7f78a00ad2e0 con 0x7f78a006cd50 2024-02-02T13:32:58.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.873+0000 7f789ea00640 1 --2- 172.21.15.91:0/114901363 >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 0x7f78a006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.874+0000 7f7895e00640 1 --2- 172.21.15.91:0/114901363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00cc660 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f78a006d600 tx=0x7f788802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33795c06fb2ce71f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789e000640 1 -- 172.21.15.91:0/114901363 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888030860 con 0x7f78a006c820 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789e000640 1 -- 172.21.15.91:0/114901363 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880309c0 con 0x7f78a006c820 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789e000640 1 -- 172.21.15.91:0/114901363 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888030d50 con 0x7f78a006c820 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 -- 172.21.15.91:0/114901363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 msgr2=0x7f78a00cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 --2- 172.21.15.91:0/114901363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00cc660 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f78a006d600 tx=0x7f788802b9a0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 -- 172.21.15.91:0/114901363 shutdown_connections 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 --2- 172.21.15.91:0/114901363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00cc660 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 --2- 172.21.15.91:0/114901363 >> v2:172.21.15.113:3300/0 conn(0x7f78a015d2c0 0x7f78a006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 -- 172.21.15.91:0/114901363 >> 172.21.15.91:0/114901363 conn(0x7f78a00cea90 msgr2=0x7f78a00ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 -- 172.21.15.91:0/114901363 shutdown_connections 2024-02-02T13:32:58.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.875+0000 7f789fe00640 1 -- 172.21.15.91:0/114901363 wait complete. 2024-02-02T13:32:58.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.877+0000 7f789fe00640 1 Processor -- start 2024-02-02T13:32:58.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.877+0000 7f789fe00640 1 -- start start 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a006cd50 0x7f78a01f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78a015d2c0 0x7f78a0156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00af9e0 con 0x7f78a006c820 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00af6c0 con 0x7f78a006cd50 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.878+0000 7f789fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00af3e0 con 0x7f78a015d2c0 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58398/0 (socket says 172.21.15.91:58398) 2024-02-02T13:32:58.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 -- 172.21.15.91:0/1930701242 learned_addr learned my addr 172.21.15.91:0/1930701242 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 -- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78a015d2c0 msgr2=0x7f78a0156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789f400640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78a015d2c0 0x7f78a0156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78a015d2c0 0x7f78a0156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f7895e00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a006cd50 0x7f78a01f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 -- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a006cd50 msgr2=0x7f78a01f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a006cd50 0x7f78a01f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 -- 172.21.15.91:0/1930701242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880094d0 con 0x7f78a006c820 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.879+0000 7f789ea00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f789000c7f0 tx=0x7f789000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.880+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890007820 con 0x7f78a006c820 2024-02-02T13:32:58.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.880+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7890007980 con 0x7f78a006c820 2024-02-02T13:32:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.880+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890007cf0 con 0x7f78a006c820 2024-02-02T13:32:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.880+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a00afbc0 con 0x7f78a006c820 2024-02-02T13:32:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.880+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a00b0180 con 0x7f78a006c820 2024-02-02T13:32:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.881+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a004dff0 con 0x7f78a006c820 2024-02-02T13:32:58.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.882+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f78900113f0 con 0x7f78a006c820 2024-02-02T13:32:58.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.883+0000 7f7897400640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 0x7f787806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:58.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.883+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7890088c50 con 0x7f78a006c820 2024-02-02T13:32:58.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.883+0000 7f7895e00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 0x7f787806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:58.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.884+0000 7f7895e00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 0x7f787806c8f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7888036000 tx=0x7f788803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:58.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:58.887+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7890054230 con 0x7f78a006c820 2024-02-02T13:32:59.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.114+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a00ad1f0 con 0x7f78a006c820 2024-02-02T13:32:59.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.114+0000 7f7897400640 1 -- 172.21.15.91:0/1930701242 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f789001f090 con 0x7f78a006c820 2024-02-02T13:32:59.116 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:59.116 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:32:59.116 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:32:59.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 msgr2=0x7f787806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 0x7f787806c8f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7888036000 tx=0x7f788803b040 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 msgr2=0x7f78a00b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f789000c7f0 tx=0x7f789000ccb0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 shutdown_connections 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f787806a430 0x7f787806c8f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78a015d2c0 0x7f78a0156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a006cd50 0x7f78a01f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 --2- 172.21.15.91:0/1930701242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78a006c820 0x7f78a00b1680 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 >> 172.21.15.91:0/1930701242 conn(0x7f78a00cea90 msgr2=0x7f78a00cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 shutdown_connections 2024-02-02T13:32:59.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.117+0000 7f789fe00640 1 -- 172.21.15.91:0/1930701242 wait complete. 2024-02-02T13:32:59.137 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:32:59.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.344+0000 7f3100400640 1 Processor -- start 2024-02-02T13:32:59.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.345+0000 7f3100400640 1 -- start start 2024-02-02T13:32:59.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f3100400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f30f815a010 0x7f30f80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f3100400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80cc7c0 con 0x7f30f80b0700 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f3100400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80ce930 con 0x7f30f80ce130 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f3100400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f80cd640 con 0x7f30f815a010 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f30fda00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f30f815a010 0x7f30f80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f30fda00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f30f815a010 0x7f30f80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58406/0 (socket says 172.21.15.91:58406) 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f30fda00640 1 -- 172.21.15.91:0/1746192743 learned_addr learned my addr 172.21.15.91:0/1746192743 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.346+0000 7f30fe400640 1 --1- 172.21.15.91:0/1746192743 >> v1:172.21.15.91:6789/0 conn(0x7f30f80b0700 0x7f30f806ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49956/0 (socket says 172.21.15.91:49956) 2024-02-02T13:32:59.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.347+0000 7f30fda00640 1 -- 172.21.15.91:0/1746192743 >> v1:172.21.15.113:6789/0 conn(0x7f30f80ce130 legacy=0x7f30f80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:59.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.347+0000 7f30fda00640 1 -- 172.21.15.91:0/1746192743 >> v1:172.21.15.91:6789/0 conn(0x7f30f80b0700 legacy=0x7f30f806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.347+0000 7f30fda00640 1 -- 172.21.15.91:0/1746192743 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f806c890 con 0x7f30f815a010 2024-02-02T13:32:59.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f30fda00640 1 --2- 172.21.15.91:0/1746192743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c9670 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f30e800bb10 tx=0x7f30e802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6744b5a102cf95b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f30f7e00640 1 -- 172.21.15.91:0/1746192743 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8036440 con 0x7f30f815a010 2024-02-02T13:32:59.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f30f7e00640 1 -- 172.21.15.91:0/1746192743 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e80365a0 con 0x7f30f815a010 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f30f7e00640 1 -- 172.21.15.91:0/1746192743 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8036930 con 0x7f30f815a010 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f3100400640 1 -- 172.21.15.91:0/1746192743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 msgr2=0x7f30f80c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.348+0000 7f3100400640 1 --2- 172.21.15.91:0/1746192743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c9670 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f30e800bb10 tx=0x7f30e802e160 comp rx=0 tx=0).stop 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.349+0000 7f3100400640 1 -- 172.21.15.91:0/1746192743 shutdown_connections 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.349+0000 7f3100400640 1 --2- 172.21.15.91:0/1746192743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c9670 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.349+0000 7f3100400640 1 -- 172.21.15.91:0/1746192743 >> 172.21.15.91:0/1746192743 conn(0x7f30f80cec80 msgr2=0x7f30f80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:59.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.349+0000 7f3100400640 1 -- 172.21.15.91:0/1746192743 shutdown_connections 2024-02-02T13:32:59.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.349+0000 7f3100400640 1 -- 172.21.15.91:0/1746192743 wait complete. 2024-02-02T13:32:59.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.350+0000 7f3100400640 1 Processor -- start 2024-02-02T13:32:59.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.350+0000 7f3100400640 1 -- start start 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f80b0700 0x7f30f81531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 0x7f30f80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80b19f0 con 0x7f30f815a010 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80b1cb0 con 0x7f30f80ce130 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.351+0000 7f3100400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80b1f90 con 0x7f30f80b0700 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58408/0 (socket says 172.21.15.91:58408) 2024-02-02T13:32:59.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 -- 172.21.15.91:0/2176371182 learned_addr learned my addr 172.21.15.91:0/2176371182 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:59.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fd000640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 0x7f30f80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 -- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f80b0700 msgr2=0x7f30f81531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f80b0700 0x7f30f81531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 -- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 msgr2=0x7f30f80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 0x7f30f80c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 -- 172.21.15.91:0/2176371182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e800b770 con 0x7f30f815a010 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fd000640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 0x7f30f80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.352+0000 7f30fe400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c730 tx=0x7f30f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.353+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ce40 con 0x7f30f815a010 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.353+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f000a5e0 con 0x7f30f815a010 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.353+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000a990 con 0x7f30f815a010 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.353+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f80b2270 con 0x7f30f815a010 2024-02-02T13:32:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.353+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f8153730 con 0x7f30f815a010 2024-02-02T13:32:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.354+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f8066cf0 con 0x7f30f815a010 2024-02-02T13:32:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.355+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f30f000aaf0 con 0x7f30f815a010 2024-02-02T13:32:59.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.356+0000 7f30f5600640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 0x7f30d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.356+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30f0087f60 con 0x7f30f815a010 2024-02-02T13:32:59.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.356+0000 7f30fda00640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 0x7f30d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.357+0000 7f30fda00640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 0x7f30d006c8f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f30e8038000 tx=0x7f30e8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.361+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30f0057090 con 0x7f30f815a010 2024-02-02T13:32:59.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.551+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30f80c5a80 con 0x7f30f815a010 2024-02-02T13:32:59.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.555+0000 7f30f5600640 1 -- 172.21.15.91:0/2176371182 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16112 (secure 0 0 0) 0x7f30f0017090 con 0x7f30f815a010 2024-02-02T13:32:59.556 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:32:59.557 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":161,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:32:56.118137+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 msgr2=0x7f30d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 0x7f30d006c8f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f30e8038000 tx=0x7f30e8030000 comp rx=0 tx=0).stop 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 msgr2=0x7f30f80c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c730 tx=0x7f30f000cbf0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 shutdown_connections 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f30d006a430 0x7f30d006c8f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.559 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f30f815a010 0x7f30f80c7de0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f80ce130 0x7f30f80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 --2- 172.21.15.91:0/2176371182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30f80b0700 0x7f30f81531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.557+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 >> 172.21.15.91:0/2176371182 conn(0x7f30f80cec80 msgr2=0x7f30f815e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:59.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.558+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 shutdown_connections 2024-02-02T13:32:59.560 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.558+0000 7f3100400640 1 -- 172.21.15.91:0/2176371182 wait complete. 2024-02-02T13:32:59.578 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:32:59.578 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:32:59.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.789+0000 7fdfcec00640 1 Processor -- start 2024-02-02T13:32:59.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.789+0000 7fdfcec00640 1 -- start start 2024-02-02T13:32:59.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfcec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfc80ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfcec00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc815dec0 con 0x7fdfc80b16d0 2024-02-02T13:32:59.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfcec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc80c9d30 con 0x7fdfc80cd1f0 2024-02-02T13:32:59.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfcec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815e1a0 con 0x7fdfc81598a0 2024-02-02T13:32:59.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfccc00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdfc80b16d0 0x7fdfc80cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49958/0 (socket says 172.21.15.91:49958) 2024-02-02T13:32:59.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.791+0000 7fdfccc00640 1 -- 172.21.15.91:0/1018833158 learned_addr learned my addr 172.21.15.91:0/1018833158 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfc80ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185461390 0 0) 0x7fdfc815dec0 con 0x7fdfc80b16d0 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa8003520 con 0x7fdfc80b16d0 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3855127175 0 0) 0x7fdfa8003520 con 0x7fdfc80b16d0 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 msgr2=0x7fdfc80ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfc80ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 >> v1:172.21.15.113:6789/0 conn(0x7fdfc80cd1f0 legacy=0x7fdfc80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80ca010 con 0x7fdfc80b16d0 2024-02-02T13:32:59.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fdfc00019f0 con 0x7fdfc80b16d0 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 >> v1:172.21.15.91:6789/0 conn(0x7fdfc80b16d0 legacy=0x7fdfc80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfa80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfa8003f70 0x7fdfa80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.792+0000 7fdfc7400640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfa8004d00 0x7fdfa8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815e1a0 con 0x7fdfa8004d00 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa800bb70 con 0x7fdfa8003f70 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa800bd30 con 0x7fdfc81598a0 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfa8004d00 0x7fdfa8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfa80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfbf200640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfa8003f70 0x7fdfa80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 -- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 msgr2=0x7fdfa80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfc81598a0 0x7fdfa80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 -- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfa8003f70 msgr2=0x7fdfa80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfa8003f70 0x7fdfa80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfccc00640 1 -- 172.21.15.91:0/1018833158 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8003520 con 0x7fdfa8004d00 2024-02-02T13:32:59.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.793+0000 7fdfbf200640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfa8003f70 0x7fdfa80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:59.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.794+0000 7fdfccc00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfa8004d00 0x7fdfa8005180 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806fee0 tx=0x7fdfa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fbf2f33627dc7f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.794+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0034630 con 0x7fdfa8004d00 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0034790 con 0x7fdfa8004d00 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfc7400640 1 -- 172.21.15.91:0/1018833158 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0034bc0 con 0x7fdfa8004d00 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 -- 172.21.15.91:0/1018833158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfa8004d00 msgr2=0x7fdfa8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 --2- 172.21.15.91:0/1018833158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfa8004d00 0x7fdfa8005180 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806fee0 tx=0x7fdfa80057f0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfccc00640 1 -- 172.21.15.91:0/1018833158 reap_dead start 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 -- 172.21.15.91:0/1018833158 shutdown_connections 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 -- 172.21.15.91:0/1018833158 >> 172.21.15.91:0/1018833158 conn(0x7fdfc80cbfc0 msgr2=0x7fdfc80cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 -- 172.21.15.91:0/1018833158 shutdown_connections 2024-02-02T13:32:59.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.795+0000 7fdfcec00640 1 -- 172.21.15.91:0/1018833158 wait complete. 2024-02-02T13:32:59.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.797+0000 7fdfcec00640 1 Processor -- start 2024-02-02T13:32:59.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.797+0000 7fdfcec00640 1 -- start start 2024-02-02T13:32:59.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 0x7fdfc81506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 0x7fdfc80acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815c760 con 0x7fdfc80ae190 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815bb50 con 0x7fdfc80add10 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.798+0000 7fdfcec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815be30 con 0x7fdfa8004d00 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:58422/0 (socket says 172.21.15.91:58422) 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 -- 172.21.15.91:0/2222857499 learned_addr learned my addr 172.21.15.91:0/2222857499 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 0x7fdfc81506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfbf200640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 0x7fdfc80acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 -- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 msgr2=0x7fdfc81506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 0x7fdfc81506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 -- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 msgr2=0x7fdfc80acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 0x7fdfc80acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 -- 172.21.15.91:0/2222857499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc815c110 con 0x7fdfc80ae190 2024-02-02T13:32:59.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 0x7fdfc81506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfbf200640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 0x7fdfc80acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.799+0000 7fdfccc00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0037040 tx=0x7fdfc0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.800+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0034d10 con 0x7fdfc80ae190 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.800+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0034e90 con 0x7fdfc80ae190 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.800+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc815c3f0 con 0x7fdfc80ae190 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.800+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc003f660 con 0x7fdfc80ae190 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.800+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc815b770 con 0x7fdfc80ae190 2024-02-02T13:32:59.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.801+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8066980 con 0x7fdfc80ae190 2024-02-02T13:32:59.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.802+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdfc003f7c0 con 0x7fdfc80ae190 2024-02-02T13:32:59.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.803+0000 7fdfc4c00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 0x7fdfa006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:32:59.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.803+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfc00aae50 con 0x7fdfc80ae190 2024-02-02T13:32:59.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.803+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 0x7fdfa006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:32:59.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.804+0000 7fdfc7e00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 0x7fdfa006c920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009a00 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:32:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.807+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdfc00764e0 con 0x7fdfc80ae190 2024-02-02T13:32:59.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:32:59.996+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfc815dbd0 con 0x7fdfa006a460 2024-02-02T13:33:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.016+0000 7fdfc4c00640 1 -- 172.21.15.91:0/2222857499 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283120 (secure 0 0 0) 0x7fdfb8012950 con 0x7fdfa006a460 2024-02-02T13:33:00.022 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:33:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:33:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 msgr2=0x7fdfa006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 0x7fdfa006c920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009a00 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-02-02T13:33:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 msgr2=0x7fdfc80ad4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0037040 tx=0x7fdfc0001cb0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 shutdown_connections 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdfa006a460 0x7fdfa006c920 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfc80ae190 0x7fdfc80ad4e0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc80add10 0x7fdfc80acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 --2- 172.21.15.91:0/2222857499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfa8004d00 0x7fdfc81506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 >> 172.21.15.91:0/2222857499 conn(0x7fdfc80cbfc0 msgr2=0x7fdfc8158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 shutdown_connections 2024-02-02T13:33:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.024+0000 7fdfcec00640 1 -- 172.21.15.91:0/2222857499 wait complete. 2024-02-02T13:33:00.046 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-02-02T13:32:58.399667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554375,"statfs":{"total":1600000000,"available":1591680320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.786674"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143778+0000","last_change":"2024-02-02T13:32:55.143657+0000","last_active":"2024-02-02T13:32:55.143778+0000","last_peered":"2024-02-02T13:32:55.143778+0000","last_clean":"2024-02-02T13:32:55.143778+0000","last_became_active":"2024-02-02T13:32:55.143445+0000","last_became_peered":"2024-02-02T13:32:55.143445+0000","last_unstale":"2024-02-02T13:32:55.143778+0000","last_undegraded":"2024-02-02T13:32:55.143778+0000","last_fullsized":"2024-02-02T13:32:55.143778+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.490307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123546+0000","last_change":"2024-02-02T13:32:55.138559+0000","last_active":"2024-02-02T13:32:56.123546+0000","last_peered":"2024-02-02T13:32:56.123546+0000","last_clean":"2024-02-02T13:32:56.123546+0000","last_became_active":"2024-02-02T13:32:55.137573+0000","last_became_peered":"2024-02-02T13:32:55.137573+0000","last_unstale":"2024-02-02T13:32:56.123546+0000","last_undegraded":"2024-02-02T13:32:56.123546+0000","last_fullsized":"2024-02-02T13:32:56.123546+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:35.416020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209073+0000","last_change":"2024-02-02T13:32:55.154334+0000","last_active":"2024-02-02T13:32:56.209073+0000","last_peered":"2024-02-02T13:32:56.209073+0000","last_clean":"2024-02-02T13:32:56.209073+0000","last_became_active":"2024-02-02T13:32:55.151602+0000","last_became_peered":"2024-02-02T13:32:55.151602+0000","last_unstale":"2024-02-02T13:32:56.209073+0000","last_undegraded":"2024-02-02T13:32:56.209073+0000","last_fullsized":"2024-02-02T13:32:56.209073+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:06.197241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145658+0000","last_change":"2024-02-02T13:32:55.145611+0000","last_active":"2024-02-02T13:32:55.145658+0000","last_peered":"2024-02-02T13:32:55.145658+0000","last_clean":"2024-02-02T13:32:55.145658+0000","last_became_active":"2024-02-02T13:32:55.145441+0000","last_became_peered":"2024-02-02T13:32:55.145441+0000","last_unstale":"2024-02-02T13:32:55.145658+0000","last_undegraded":"2024-02-02T13:32:55.145658+0000","last_fullsized":"2024-02-02T13:32:55.145658+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:30.684351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126850+0000","last_change":"2024-02-02T13:32:55.140235+0000","last_active":"2024-02-02T13:32:56.126850+0000","last_peered":"2024-02-02T13:32:56.126850+0000","last_clean":"2024-02-02T13:32:56.126850+0000","last_became_active":"2024-02-02T13:32:55.139735+0000","last_became_peered":"2024-02-02T13:32:55.139735+0000","last_unstale":"2024-02-02T13:32:56.126850+0000","last_undegraded":"2024-02-02T13:32:56.126850+0000","last_fullsized":"2024-02-02T13:32:56.126850+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:16.191248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146080+0000","last_change":"2024-02-02T13:32:55.143321+0000","last_active":"2024-02-02T13:32:55.146080+0000","last_peered":"2024-02-02T13:32:55.146080+0000","last_clean":"2024-02-02T13:32:55.146080+0000","last_became_active":"2024-02-02T13:32:55.142970+0000","last_became_peered":"2024-02-02T13:32:55.142970+0000","last_unstale":"2024-02-02T13:32:55.146080+0000","last_undegraded":"2024-02-02T13:32:55.146080+0000","last_fullsized":"2024-02-02T13:32:55.146080+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:15:37.403990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152839+0000","last_change":"2024-02-02T13:32:55.152190+0000","last_active":"2024-02-02T13:32:55.152839+0000","last_peered":"2024-02-02T13:32:55.152839+0000","last_clean":"2024-02-02T13:32:55.152839+0000","last_became_active":"2024-02-02T13:32:55.151939+0000","last_became_peered":"2024-02-02T13:32:55.151939+0000","last_unstale":"2024-02-02T13:32:55.152839+0000","last_undegraded":"2024-02-02T13:32:55.152839+0000","last_fullsized":"2024-02-02T13:32:55.152839+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:15.437339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209321+0000","last_change":"2024-02-02T13:32:55.154020+0000","last_active":"2024-02-02T13:32:56.209321+0000","last_peered":"2024-02-02T13:32:56.209321+0000","last_clean":"2024-02-02T13:32:56.209321+0000","last_became_active":"2024-02-02T13:32:55.151296+0000","last_became_peered":"2024-02-02T13:32:55.151296+0000","last_unstale":"2024-02-02T13:32:56.209321+0000","last_undegraded":"2024-02-02T13:32:56.209321+0000","last_fullsized":"2024-02-02T13:32:56.209321+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:56:24.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.208785+0000","last_change":"2024-02-02T13:32:55.153602+0000","last_active":"2024-02-02T13:32:56.208785+0000","last_peered":"2024-02-02T13:32:56.208785+0000","last_clean":"2024-02-02T13:32:56.208785+0000","last_became_active":"2024-02-02T13:32:55.153276+0000","last_became_peered":"2024-02-02T13:32:55.153276+0000","last_unstale":"2024-02-02T13:32:56.208785+0000","last_undegraded":"2024-02-02T13:32:56.208785+0000","last_fullsized":"2024-02-02T13:32:56.208785+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:35.160953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124444+0000","last_change":"2024-02-02T13:32:55.147189+0000","last_active":"2024-02-02T13:32:56.124444+0000","last_peered":"2024-02-02T13:32:56.124444+0000","last_clean":"2024-02-02T13:32:56.124444+0000","last_became_active":"2024-02-02T13:32:55.139691+0000","last_became_peered":"2024-02-02T13:32:55.139691+0000","last_unstale":"2024-02-02T13:32:56.124444+0000","last_undegraded":"2024-02-02T13:32:56.124444+0000","last_fullsized":"2024-02-02T13:32:56.124444+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:17:55.490328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206777+0000","last_change":"2024-02-02T13:32:55.151979+0000","last_active":"2024-02-02T13:32:56.206777+0000","last_peered":"2024-02-02T13:32:56.206777+0000","last_clean":"2024-02-02T13:32:56.206777+0000","last_became_active":"2024-02-02T13:32:55.151627+0000","last_became_peered":"2024-02-02T13:32:55.151627+0000","last_unstale":"2024-02-02T13:32:56.206777+0000","last_undegraded":"2024-02-02T13:32:56.206777+0000","last_fullsized":"2024-02-02T13:32:56.206777+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:09:18.206248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209180+0000","last_change":"2024-02-02T13:32:55.156418+0000","last_active":"2024-02-02T13:32:56.209180+0000","last_peered":"2024-02-02T13:32:56.209180+0000","last_clean":"2024-02-02T13:32:56.209180+0000","last_became_active":"2024-02-02T13:32:55.154919+0000","last_became_peered":"2024-02-02T13:32:55.154919+0000","last_unstale":"2024-02-02T13:32:56.209180+0000","last_undegraded":"2024-02-02T13:32:56.209180+0000","last_fullsized":"2024-02-02T13:32:56.209180+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:08:43.064057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.140404+0000","last_change":"2024-02-02T13:32:55.139471+0000","last_active":"2024-02-02T13:32:55.140404+0000","last_peered":"2024-02-02T13:32:55.140404+0000","last_clean":"2024-02-02T13:32:55.140404+0000","last_became_active":"2024-02-02T13:32:55.139332+0000","last_became_peered":"2024-02-02T13:32:55.139332+0000","last_unstale":"2024-02-02T13:32:55.140404+0000","last_undegraded":"2024-02-02T13:32:55.140404+0000","last_fullsized":"2024-02-02T13:32:55.140404+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:51.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123531+0000","last_change":"2024-02-02T13:32:55.146689+0000","last_active":"2024-02-02T13:32:56.123531+0000","last_peered":"2024-02-02T13:32:56.123531+0000","last_clean":"2024-02-02T13:32:56.123531+0000","last_became_active":"2024-02-02T13:32:55.146472+0000","last_became_peered":"2024-02-02T13:32:55.146472+0000","last_unstale":"2024-02-02T13:32:56.123531+0000","last_undegraded":"2024-02-02T13:32:56.123531+0000","last_fullsized":"2024-02-02T13:32:56.123531+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:19.433416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.155933+0000","last_change":"2024-02-02T13:32:55.155855+0000","last_active":"2024-02-02T13:32:55.155933+0000","last_peered":"2024-02-02T13:32:55.155933+0000","last_clean":"2024-02-02T13:32:55.155933+0000","last_became_active":"2024-02-02T13:32:55.155461+0000","last_became_peered":"2024-02-02T13:32:55.155461+0000","last_unstale":"2024-02-02T13:32:55.155933+0000","last_undegraded":"2024-02-02T13:32:55.155933+0000","last_fullsized":"2024-02-02T13:32:55.155933+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:25.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.142918+0000","last_change":"2024-02-02T13:32:55.142402+0000","last_active":"2024-02-02T13:32:55.142918+0000","last_peered":"2024-02-02T13:32:55.142918+0000","last_clean":"2024-02-02T13:32:55.142918+0000","last_became_active":"2024-02-02T13:32:55.142233+0000","last_became_peered":"2024-02-02T13:32:55.142233+0000","last_unstale":"2024-02-02T13:32:55.142918+0000","last_undegraded":"2024-02-02T13:32:55.142918+0000","last_fullsized":"2024-02-02T13:32:55.142918+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:24.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152934+0000","last_change":"2024-02-02T13:32:55.152826+0000","last_active":"2024-02-02T13:32:55.152934+0000","last_peered":"2024-02-02T13:32:55.152934+0000","last_clean":"2024-02-02T13:32:55.152934+0000","last_became_active":"2024-02-02T13:32:55.152562+0000","last_became_peered":"2024-02-02T13:32:55.152562+0000","last_unstale":"2024-02-02T13:32:55.152934+0000","last_undegraded":"2024-02-02T13:32:55.152934+0000","last_fullsized":"2024-02-02T13:32:55.152934+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:03.978723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.137124+0000","last_change":"2024-02-02T13:32:55.136928+0000","last_active":"2024-02-02T13:32:55.137124+0000","last_peered":"2024-02-02T13:32:55.137124+0000","last_clean":"2024-02-02T13:32:55.137124+0000","last_became_active":"2024-02-02T13:32:55.136228+0000","last_became_peered":"2024-02-02T13:32:55.136228+0000","last_unstale":"2024-02-02T13:32:55.137124+0000","last_undegraded":"2024-02-02T13:32:55.137124+0000","last_fullsized":"2024-02-02T13:32:55.137124+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:25:03.384036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206511+0000","last_change":"2024-02-02T13:32:55.145455+0000","last_active":"2024-02-02T13:32:56.206511+0000","last_peered":"2024-02-02T13:32:56.206511+0000","last_clean":"2024-02-02T13:32:56.206511+0000","last_became_active":"2024-02-02T13:32:55.145222+0000","last_became_peered":"2024-02-02T13:32:55.145222+0000","last_unstale":"2024-02-02T13:32:56.206511+0000","last_undegraded":"2024-02-02T13:32:56.206511+0000","last_fullsized":"2024-02-02T13:32:56.206511+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:57:07.862777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.129823+0000","last_change":"2024-02-02T13:32:55.129695+0000","last_active":"2024-02-02T13:32:55.129823+0000","last_peered":"2024-02-02T13:32:55.129823+0000","last_clean":"2024-02-02T13:32:55.129823+0000","last_became_active":"2024-02-02T13:32:55.129104+0000","last_became_peered":"2024-02-02T13:32:55.129104+0000","last_unstale":"2024-02-02T13:32:55.129823+0000","last_undegraded":"2024-02-02T13:32:55.129823+0000","last_fullsized":"2024-02-02T13:32:55.129823+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:33:26.571878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128547+0000","last_change":"2024-02-02T13:32:55.151447+0000","last_active":"2024-02-02T13:32:56.128547+0000","last_peered":"2024-02-02T13:32:56.128547+0000","last_clean":"2024-02-02T13:32:56.128547+0000","last_became_active":"2024-02-02T13:32:55.151171+0000","last_became_peered":"2024-02-02T13:32:55.151171+0000","last_unstale":"2024-02-02T13:32:56.128547+0000","last_undegraded":"2024-02-02T13:32:56.128547+0000","last_fullsized":"2024-02-02T13:32:56.128547+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:16.532202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206774+0000","last_change":"2024-02-02T13:32:55.147175+0000","last_active":"2024-02-02T13:32:56.206774+0000","last_peered":"2024-02-02T13:32:56.206774+0000","last_clean":"2024-02-02T13:32:56.206774+0000","last_became_active":"2024-02-02T13:32:55.146718+0000","last_became_peered":"2024-02-02T13:32:55.146718+0000","last_unstale":"2024-02-02T13:32:56.206774+0000","last_undegraded":"2024-02-02T13:32:56.206774+0000","last_fullsized":"2024-02-02T13:32:56.206774+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:28.642640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209960+0000","last_change":"2024-02-02T13:32:55.153382+0000","last_active":"2024-02-02T13:32:56.209960+0000","last_peered":"2024-02-02T13:32:56.209960+0000","last_clean":"2024-02-02T13:32:56.209960+0000","last_became_active":"2024-02-02T13:32:55.150560+0000","last_became_peered":"2024-02-02T13:32:55.150560+0000","last_unstale":"2024-02-02T13:32:56.209960+0000","last_undegraded":"2024-02-02T13:32:56.209960+0000","last_fullsized":"2024-02-02T13:32:56.209960+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:31.019555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206604+0000","last_change":"2024-02-02T13:32:55.143309+0000","last_active":"2024-02-02T13:32:56.206604+0000","last_peered":"2024-02-02T13:32:56.206604+0000","last_clean":"2024-02-02T13:32:56.206604+0000","last_became_active":"2024-02-02T13:32:55.142952+0000","last_became_peered":"2024-02-02T13:32:55.142952+0000","last_unstale":"2024-02-02T13:32:56.206604+0000","last_undegraded":"2024-02-02T13:32:56.206604+0000","last_fullsized":"2024-02-02T13:32:56.206604+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:28.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124535+0000","last_change":"2024-02-02T13:32:55.139419+0000","last_active":"2024-02-02T13:32:56.124535+0000","last_peered":"2024-02-02T13:32:56.124535+0000","last_clean":"2024-02-02T13:32:56.124535+0000","last_became_active":"2024-02-02T13:32:55.138383+0000","last_became_peered":"2024-02-02T13:32:55.138383+0000","last_unstale":"2024-02-02T13:32:56.124535+0000","last_undegraded":"2024-02-02T13:32:56.124535+0000","last_fullsized":"2024-02-02T13:32:56.124535+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:01:33.246916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.141818+0000","last_change":"2024-02-02T13:32:55.141765+0000","last_active":"2024-02-02T13:32:55.141818+0000","last_peered":"2024-02-02T13:32:55.141818+0000","last_clean":"2024-02-02T13:32:55.141818+0000","last_became_active":"2024-02-02T13:32:55.141633+0000","last_became_peered":"2024-02-02T13:32:55.141633+0000","last_unstale":"2024-02-02T13:32:55.141818+0000","last_undegraded":"2024-02-02T13:32:55.141818+0000","last_fullsized":"2024-02-02T13:32:55.141818+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:38.566112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143168+0000","last_change":"2024-02-02T13:32:55.142804+0000","last_active":"2024-02-02T13:32:55.143168+0000","last_peered":"2024-02-02T13:32:55.143168+0000","last_clean":"2024-02-02T13:32:55.143168+0000","last_became_active":"2024-02-02T13:32:55.142605+0000","last_became_peered":"2024-02-02T13:32:55.142605+0000","last_unstale":"2024-02-02T13:32:55.143168+0000","last_undegraded":"2024-02-02T13:32:55.143168+0000","last_fullsized":"2024-02-02T13:32:55.143168+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:07:43.379816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127062+0000","last_change":"2024-02-02T13:32:55.138553+0000","last_active":"2024-02-02T13:32:56.127062+0000","last_peered":"2024-02-02T13:32:56.127062+0000","last_clean":"2024-02-02T13:32:56.127062+0000","last_became_active":"2024-02-02T13:32:55.133675+0000","last_became_peered":"2024-02-02T13:32:55.133675+0000","last_unstale":"2024-02-02T13:32:56.127062+0000","last_undegraded":"2024-02-02T13:32:56.127062+0000","last_fullsized":"2024-02-02T13:32:56.127062+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:29.793178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.154016+0000","last_change":"2024-02-02T13:32:55.153950+0000","last_active":"2024-02-02T13:32:55.154016+0000","last_peered":"2024-02-02T13:32:55.154016+0000","last_clean":"2024-02-02T13:32:55.154016+0000","last_became_active":"2024-02-02T13:32:55.153725+0000","last_became_peered":"2024-02-02T13:32:55.153725+0000","last_unstale":"2024-02-02T13:32:55.154016+0000","last_undegraded":"2024-02-02T13:32:55.154016+0000","last_fullsized":"2024-02-02T13:32:55.154016+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:57:01.762993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.155785+0000","last_change":"2024-02-02T13:32:55.155720+0000","last_active":"2024-02-02T13:32:55.155785+0000","last_peered":"2024-02-02T13:32:55.155785+0000","last_clean":"2024-02-02T13:32:55.155785+0000","last_became_active":"2024-02-02T13:32:55.155142+0000","last_became_peered":"2024-02-02T13:32:55.155142+0000","last_unstale":"2024-02-02T13:32:55.155785+0000","last_undegraded":"2024-02-02T13:32:55.155785+0000","last_fullsized":"2024-02-02T13:32:55.155785+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:21.131397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127689+0000","last_change":"2024-02-02T13:32:55.146091+0000","last_active":"2024-02-02T13:32:56.127689+0000","last_peered":"2024-02-02T13:32:56.127689+0000","last_clean":"2024-02-02T13:32:56.127689+0000","last_became_active":"2024-02-02T13:32:55.145576+0000","last_became_peered":"2024-02-02T13:32:55.145576+0000","last_unstale":"2024-02-02T13:32:56.127689+0000","last_undegraded":"2024-02-02T13:32:56.127689+0000","last_fullsized":"2024-02-02T13:32:56.127689+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:26:58.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145171+0000","last_change":"2024-02-02T13:32:55.141878+0000","last_active":"2024-02-02T13:32:55.145171+0000","last_peered":"2024-02-02T13:32:55.145171+0000","last_clean":"2024-02-02T13:32:55.145171+0000","last_became_active":"2024-02-02T13:32:55.141186+0000","last_became_peered":"2024-02-02T13:32:55.141186+0000","last_unstale":"2024-02-02T13:32:55.145171+0000","last_undegraded":"2024-02-02T13:32:55.145171+0000","last_fullsized":"2024-02-02T13:32:55.145171+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:19:17.566619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.132491+0000","last_change":"2024-02-02T13:32:55.132442+0000","last_active":"2024-02-02T13:32:55.132491+0000","last_peered":"2024-02-02T13:32:55.132491+0000","last_clean":"2024-02-02T13:32:55.132491+0000","last_became_active":"2024-02-02T13:32:55.132275+0000","last_became_peered":"2024-02-02T13:32:55.132275+0000","last_unstale":"2024-02-02T13:32:55.132491+0000","last_undegraded":"2024-02-02T13:32:55.132491+0000","last_fullsized":"2024-02-02T13:32:55.132491+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:50.996447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"160'4","reported_seq":15,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125705+0000","last_change":"2024-02-02T13:32:54.648778+0000","last_active":"2024-02-02T13:32:56.125705+0000","last_peered":"2024-02-02T13:32:56.125705+0000","last_clean":"2024-02-02T13:32:56.125705+0000","last_became_active":"2024-02-02T13:32:54.648456+0000","last_became_peered":"2024-02-02T13:32:54.648456+0000","last_unstale":"2024-02-02T13:32:56.125705+0000","last_undegraded":"2024-02-02T13:32:56.125705+0000","last_fullsized":"2024-02-02T13:32:56.125705+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:08:42.046996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209570+0000","last_change":"2024-02-02T13:32:55.153729+0000","last_active":"2024-02-02T13:32:56.209570+0000","last_peered":"2024-02-02T13:32:56.209570+0000","last_clean":"2024-02-02T13:32:56.209570+0000","last_became_active":"2024-02-02T13:32:55.150951+0000","last_became_peered":"2024-02-02T13:32:55.150951+0000","last_unstale":"2024-02-02T13:32:56.209570+0000","last_undegraded":"2024-02-02T13:32:56.209570+0000","last_fullsized":"2024-02-02T13:32:56.209570+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:50.996447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126309+0000","last_change":"2024-02-02T13:32:54.646654+0000","last_active":"2024-02-02T13:32:56.126309+0000","last_peered":"2024-02-02T13:32:56.126309+0000","last_clean":"2024-02-02T13:32:56.126309+0000","last_became_active":"2024-02-02T13:32:54.642723+0000","last_became_peered":"2024-02-02T13:32:54.642723+0000","last_unstale":"2024-02-02T13:32:56.126309+0000","last_undegraded":"2024-02-02T13:32:56.126309+0000","last_fullsized":"2024-02-02T13:32:56.126309+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:18.416354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127920+0000","last_change":"2024-02-02T13:32:55.138581+0000","last_active":"2024-02-02T13:32:56.127920+0000","last_peered":"2024-02-02T13:32:56.127920+0000","last_clean":"2024-02-02T13:32:56.127920+0000","last_became_active":"2024-02-02T13:32:55.137825+0000","last_became_peered":"2024-02-02T13:32:55.137825+0000","last_unstale":"2024-02-02T13:32:56.127920+0000","last_undegraded":"2024-02-02T13:32:56.127920+0000","last_fullsized":"2024-02-02T13:32:56.127920+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:05.203908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206630+0000","last_change":"2024-02-02T13:32:54.645982+0000","last_active":"2024-02-02T13:32:56.206630+0000","last_peered":"2024-02-02T13:32:56.206630+0000","last_clean":"2024-02-02T13:32:56.206630+0000","last_became_active":"2024-02-02T13:32:54.645765+0000","last_became_peered":"2024-02-02T13:32:54.645765+0000","last_unstale":"2024-02-02T13:32:56.206630+0000","last_undegraded":"2024-02-02T13:32:56.206630+0000","last_fullsized":"2024-02-02T13:32:56.206630+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:20:30.036447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127036+0000","last_change":"2024-02-02T13:32:55.134699+0000","last_active":"2024-02-02T13:32:56.127036+0000","last_peered":"2024-02-02T13:32:56.127036+0000","last_clean":"2024-02-02T13:32:56.127036+0000","last_became_active":"2024-02-02T13:32:55.133962+0000","last_became_peered":"2024-02-02T13:32:55.133962+0000","last_unstale":"2024-02-02T13:32:56.127036+0000","last_undegraded":"2024-02-02T13:32:56.127036+0000","last_fullsized":"2024-02-02T13:32:56.127036+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:48.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123986+0000","last_change":"2024-02-02T13:32:54.644347+0000","last_active":"2024-02-02T13:32:56.123986+0000","last_peered":"2024-02-02T13:32:56.123986+0000","last_clean":"2024-02-02T13:32:56.123986+0000","last_became_active":"2024-02-02T13:32:54.644125+0000","last_became_peered":"2024-02-02T13:32:54.644125+0000","last_unstale":"2024-02-02T13:32:56.123986+0000","last_undegraded":"2024-02-02T13:32:56.123986+0000","last_fullsized":"2024-02-02T13:32:56.123986+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:50.462189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.136853+0000","last_change":"2024-02-02T13:32:55.136685+0000","last_active":"2024-02-02T13:32:55.136853+0000","last_peered":"2024-02-02T13:32:55.136853+0000","last_clean":"2024-02-02T13:32:55.136853+0000","last_became_active":"2024-02-02T13:32:55.135933+0000","last_became_peered":"2024-02-02T13:32:55.135933+0000","last_unstale":"2024-02-02T13:32:55.136853+0000","last_undegraded":"2024-02-02T13:32:55.136853+0000","last_fullsized":"2024-02-02T13:32:55.136853+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:53.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123648+0000","last_change":"2024-02-02T13:32:54.646641+0000","last_active":"2024-02-02T13:32:56.123648+0000","last_peered":"2024-02-02T13:32:56.123648+0000","last_clean":"2024-02-02T13:32:56.123648+0000","last_became_active":"2024-02-02T13:32:54.646404+0000","last_became_peered":"2024-02-02T13:32:54.646404+0000","last_unstale":"2024-02-02T13:32:56.123648+0000","last_undegraded":"2024-02-02T13:32:56.123648+0000","last_fullsized":"2024-02-02T13:32:56.123648+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:24.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.151249+0000","last_change":"2024-02-02T13:32:55.151143+0000","last_active":"2024-02-02T13:32:55.151249+0000","last_peered":"2024-02-02T13:32:55.151249+0000","last_clean":"2024-02-02T13:32:55.151249+0000","last_became_active":"2024-02-02T13:32:55.150868+0000","last_became_peered":"2024-02-02T13:32:55.150868+0000","last_unstale":"2024-02-02T13:32:55.151249+0000","last_undegraded":"2024-02-02T13:32:55.151249+0000","last_fullsized":"2024-02-02T13:32:55.151249+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:11:26.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.148844+0000","last_change":"2024-02-02T13:32:54.646008+0000","last_active":"2024-02-02T13:32:55.148844+0000","last_peered":"2024-02-02T13:32:55.148844+0000","last_clean":"2024-02-02T13:32:55.148844+0000","last_became_active":"2024-02-02T13:32:54.644468+0000","last_became_peered":"2024-02-02T13:32:54.644468+0000","last_unstale":"2024-02-02T13:32:55.148844+0000","last_undegraded":"2024-02-02T13:32:55.148844+0000","last_fullsized":"2024-02-02T13:32:55.148844+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:11:25.059474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209525+0000","last_change":"2024-02-02T13:32:55.156225+0000","last_active":"2024-02-02T13:32:56.209525+0000","last_peered":"2024-02-02T13:32:56.209525+0000","last_clean":"2024-02-02T13:32:56.209525+0000","last_became_active":"2024-02-02T13:32:55.154599+0000","last_became_peered":"2024-02-02T13:32:55.154599+0000","last_unstale":"2024-02-02T13:32:56.209525+0000","last_undegraded":"2024-02-02T13:32:56.209525+0000","last_fullsized":"2024-02-02T13:32:56.209525+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:43.943303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.132167+0000","last_change":"2024-02-02T13:32:54.649908+0000","last_active":"2024-02-02T13:32:55.132167+0000","last_peered":"2024-02-02T13:32:55.132167+0000","last_clean":"2024-02-02T13:32:55.132167+0000","last_became_active":"2024-02-02T13:32:54.649462+0000","last_became_peered":"2024-02-02T13:32:54.649462+0000","last_unstale":"2024-02-02T13:32:55.132167+0000","last_undegraded":"2024-02-02T13:32:55.132167+0000","last_fullsized":"2024-02-02T13:32:55.132167+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:01:32.229854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123747+0000","last_change":"2024-02-02T13:32:55.132217+0000","last_active":"2024-02-02T13:32:56.123747+0000","last_peered":"2024-02-02T13:32:56.123747+0000","last_clean":"2024-02-02T13:32:56.123747+0000","last_became_active":"2024-02-02T13:32:55.131598+0000","last_became_peered":"2024-02-02T13:32:55.131598+0000","last_unstale":"2024-02-02T13:32:56.123747+0000","last_undegraded":"2024-02-02T13:32:56.123747+0000","last_fullsized":"2024-02-02T13:32:56.123747+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:19:59.978075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124701+0000","last_change":"2024-02-02T13:32:54.647904+0000","last_active":"2024-02-02T13:32:56.124701+0000","last_peered":"2024-02-02T13:32:56.124701+0000","last_clean":"2024-02-02T13:32:56.124701+0000","last_became_active":"2024-02-02T13:32:54.647617+0000","last_became_peered":"2024-02-02T13:32:54.647617+0000","last_unstale":"2024-02-02T13:32:56.124701+0000","last_undegraded":"2024-02-02T13:32:56.124701+0000","last_fullsized":"2024-02-02T13:32:56.124701+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:20.114336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.149048+0000","last_change":"2024-02-02T13:32:55.149000+0000","last_active":"2024-02-02T13:32:55.149048+0000","last_peered":"2024-02-02T13:32:55.149048+0000","last_clean":"2024-02-02T13:32:55.149048+0000","last_became_active":"2024-02-02T13:32:55.148740+0000","last_became_peered":"2024-02-02T13:32:55.148740+0000","last_unstale":"2024-02-02T13:32:55.149048+0000","last_undegraded":"2024-02-02T13:32:55.149048+0000","last_fullsized":"2024-02-02T13:32:55.149048+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:57:28.874057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128536+0000","last_change":"2024-02-02T13:32:54.636230+0000","last_active":"2024-02-02T13:32:56.128536+0000","last_peered":"2024-02-02T13:32:56.128536+0000","last_clean":"2024-02-02T13:32:56.128536+0000","last_became_active":"2024-02-02T13:32:54.635684+0000","last_became_peered":"2024-02-02T13:32:54.635684+0000","last_unstale":"2024-02-02T13:32:56.128536+0000","last_undegraded":"2024-02-02T13:32:56.128536+0000","last_fullsized":"2024-02-02T13:32:56.128536+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:34.398958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127906+0000","last_change":"2024-02-02T13:32:55.142411+0000","last_active":"2024-02-02T13:32:56.127906+0000","last_peered":"2024-02-02T13:32:56.127906+0000","last_clean":"2024-02-02T13:32:56.127906+0000","last_became_active":"2024-02-02T13:32:55.134978+0000","last_became_peered":"2024-02-02T13:32:55.134978+0000","last_unstale":"2024-02-02T13:32:56.127906+0000","last_undegraded":"2024-02-02T13:32:56.127906+0000","last_fullsized":"2024-02-02T13:32:56.127906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:18.561892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121623+0000","last_change":"2024-02-02T13:32:54.645555+0000","last_active":"2024-02-02T13:32:55.121623+0000","last_peered":"2024-02-02T13:32:55.121623+0000","last_clean":"2024-02-02T13:32:55.121623+0000","last_became_active":"2024-02-02T13:32:54.644068+0000","last_became_peered":"2024-02-02T13:32:54.644068+0000","last_unstale":"2024-02-02T13:32:55.121623+0000","last_undegraded":"2024-02-02T13:32:55.121623+0000","last_fullsized":"2024-02-02T13:32:55.121623+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:15:36.386928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125703+0000","last_change":"2024-02-02T13:32:55.134684+0000","last_active":"2024-02-02T13:32:56.125703+0000","last_peered":"2024-02-02T13:32:56.125703+0000","last_clean":"2024-02-02T13:32:56.125703+0000","last_became_active":"2024-02-02T13:32:55.133986+0000","last_became_peered":"2024-02-02T13:32:55.133986+0000","last_unstale":"2024-02-02T13:32:56.125703+0000","last_undegraded":"2024-02-02T13:32:56.125703+0000","last_fullsized":"2024-02-02T13:32:56.125703+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:37.017486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"160'3","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127269+0000","last_change":"2024-02-02T13:32:54.642706+0000","last_active":"2024-02-02T13:32:56.127269+0000","last_peered":"2024-02-02T13:32:56.127269+0000","last_clean":"2024-02-02T13:32:56.127269+0000","last_became_active":"2024-02-02T13:32:54.634322+0000","last_became_peered":"2024-02-02T13:32:54.634322+0000","last_unstale":"2024-02-02T13:32:56.127269+0000","last_undegraded":"2024-02-02T13:32:56.127269+0000","last_fullsized":"2024-02-02T13:32:56.127269+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:24:04.062561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125424+0000","last_change":"2024-02-02T13:32:55.153402+0000","last_active":"2024-02-02T13:32:56.125424+0000","last_peered":"2024-02-02T13:32:56.125424+0000","last_clean":"2024-02-02T13:32:56.125424+0000","last_became_active":"2024-02-02T13:32:55.152737+0000","last_became_peered":"2024-02-02T13:32:55.152737+0000","last_unstale":"2024-02-02T13:32:56.125424+0000","last_undegraded":"2024-02-02T13:32:56.125424+0000","last_fullsized":"2024-02-02T13:32:56.125424+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:24.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"160'6","reported_seq":17,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209117+0000","last_change":"2024-02-02T13:32:54.650801+0000","last_active":"2024-02-02T13:32:56.209117+0000","last_peered":"2024-02-02T13:32:56.209117+0000","last_clean":"2024-02-02T13:32:56.209117+0000","last_became_active":"2024-02-02T13:32:54.650540+0000","last_became_peered":"2024-02-02T13:32:54.650540+0000","last_unstale":"2024-02-02T13:32:56.209117+0000","last_undegraded":"2024-02-02T13:32:56.209117+0000","last_fullsized":"2024-02-02T13:32:56.209117+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:20:07.268407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143761+0000","last_change":"2024-02-02T13:32:55.143327+0000","last_active":"2024-02-02T13:32:55.143761+0000","last_peered":"2024-02-02T13:32:55.143761+0000","last_clean":"2024-02-02T13:32:55.143761+0000","last_became_active":"2024-02-02T13:32:55.143189+0000","last_became_peered":"2024-02-02T13:32:55.143189+0000","last_unstale":"2024-02-02T13:32:55.143761+0000","last_undegraded":"2024-02-02T13:32:55.143761+0000","last_fullsized":"2024-02-02T13:32:55.143761+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:22:36.659330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.140551+0000","last_change":"2024-02-02T13:32:54.647125+0000","last_active":"2024-02-02T13:32:55.140551+0000","last_peered":"2024-02-02T13:32:55.140551+0000","last_clean":"2024-02-02T13:32:55.140551+0000","last_became_active":"2024-02-02T13:32:54.646820+0000","last_became_peered":"2024-02-02T13:32:54.646820+0000","last_unstale":"2024-02-02T13:32:55.140551+0000","last_undegraded":"2024-02-02T13:32:55.140551+0000","last_fullsized":"2024-02-02T13:32:55.140551+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:24:59.236773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123546+0000","last_change":"2024-02-02T13:32:55.154103+0000","last_active":"2024-02-02T13:32:56.123546+0000","last_peered":"2024-02-02T13:32:56.123546+0000","last_clean":"2024-02-02T13:32:56.123546+0000","last_became_active":"2024-02-02T13:32:55.153861+0000","last_became_peered":"2024-02-02T13:32:55.153861+0000","last_unstale":"2024-02-02T13:32:56.123546+0000","last_undegraded":"2024-02-02T13:32:56.123546+0000","last_fullsized":"2024-02-02T13:32:56.123546+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123739+0000","last_change":"2024-02-02T13:32:54.651216+0000","last_active":"2024-02-02T13:32:56.123739+0000","last_peered":"2024-02-02T13:32:56.123739+0000","last_clean":"2024-02-02T13:32:56.123739+0000","last_became_active":"2024-02-02T13:32:54.650939+0000","last_became_peered":"2024-02-02T13:32:54.650939+0000","last_unstale":"2024-02-02T13:32:56.123739+0000","last_undegraded":"2024-02-02T13:32:56.123739+0000","last_fullsized":"2024-02-02T13:32:56.123739+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:26.473245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209411+0000","last_change":"2024-02-02T13:32:55.156265+0000","last_active":"2024-02-02T13:32:56.209411+0000","last_peered":"2024-02-02T13:32:56.209411+0000","last_clean":"2024-02-02T13:32:56.209411+0000","last_became_active":"2024-02-02T13:32:55.154309+0000","last_became_peered":"2024-02-02T13:32:55.154309+0000","last_unstale":"2024-02-02T13:32:56.209411+0000","last_undegraded":"2024-02-02T13:32:56.209411+0000","last_fullsized":"2024-02-02T13:32:56.209411+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:55:40.229185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"160'8","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.210307+0000","last_change":"2024-02-02T13:32:54.645174+0000","last_active":"2024-02-02T13:32:56.210307+0000","last_peered":"2024-02-02T13:32:56.210307+0000","last_clean":"2024-02-02T13:32:56.210307+0000","last_became_active":"2024-02-02T13:32:54.642994+0000","last_became_peered":"2024-02-02T13:32:54.642994+0000","last_unstale":"2024-02-02T13:32:56.210307+0000","last_undegraded":"2024-02-02T13:32:56.210307+0000","last_fullsized":"2024-02-02T13:32:56.210307+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:39:00.577504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209705+0000","last_change":"2024-02-02T13:32:55.155071+0000","last_active":"2024-02-02T13:32:56.209705+0000","last_peered":"2024-02-02T13:32:56.209705+0000","last_clean":"2024-02-02T13:32:56.209705+0000","last_became_active":"2024-02-02T13:32:55.151417+0000","last_became_peered":"2024-02-02T13:32:55.151417+0000","last_unstale":"2024-02-02T13:32:56.209705+0000","last_undegraded":"2024-02-02T13:32:56.209705+0000","last_fullsized":"2024-02-02T13:32:56.209705+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:52:40.527926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126579+0000","last_change":"2024-02-02T13:32:54.650068+0000","last_active":"2024-02-02T13:32:56.126579+0000","last_peered":"2024-02-02T13:32:56.126579+0000","last_clean":"2024-02-02T13:32:56.126579+0000","last_became_active":"2024-02-02T13:32:54.649800+0000","last_became_peered":"2024-02-02T13:32:54.649800+0000","last_unstale":"2024-02-02T13:32:56.126579+0000","last_undegraded":"2024-02-02T13:32:56.126579+0000","last_fullsized":"2024-02-02T13:32:56.126579+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:39.140143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126048+0000","last_change":"2024-02-02T13:32:55.140617+0000","last_active":"2024-02-02T13:32:56.126048+0000","last_peered":"2024-02-02T13:32:56.126048+0000","last_clean":"2024-02-02T13:32:56.126048+0000","last_became_active":"2024-02-02T13:32:55.140189+0000","last_became_peered":"2024-02-02T13:32:55.140189+0000","last_unstale":"2024-02-02T13:32:56.126048+0000","last_undegraded":"2024-02-02T13:32:56.126048+0000","last_fullsized":"2024-02-02T13:32:56.126048+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:24.159578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206714+0000","last_change":"2024-02-02T13:32:54.648911+0000","last_active":"2024-02-02T13:32:56.206714+0000","last_peered":"2024-02-02T13:32:56.206714+0000","last_clean":"2024-02-02T13:32:56.206714+0000","last_became_active":"2024-02-02T13:32:54.648684+0000","last_became_peered":"2024-02-02T13:32:54.648684+0000","last_unstale":"2024-02-02T13:32:56.206714+0000","last_undegraded":"2024-02-02T13:32:56.206714+0000","last_fullsized":"2024-02-02T13:32:56.206714+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:46:32.135281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.133352+0000","last_change":"2024-02-02T13:32:55.133104+0000","last_active":"2024-02-02T13:32:55.133352+0000","last_peered":"2024-02-02T13:32:55.133352+0000","last_clean":"2024-02-02T13:32:55.133352+0000","last_became_active":"2024-02-02T13:32:55.132887+0000","last_became_peered":"2024-02-02T13:32:55.132887+0000","last_unstale":"2024-02-02T13:32:55.133352+0000","last_undegraded":"2024-02-02T13:32:55.133352+0000","last_fullsized":"2024-02-02T13:32:55.133352+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:56:24.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.150094+0000","last_change":"2024-02-02T13:32:54.648682+0000","last_active":"2024-02-02T13:32:55.150094+0000","last_peered":"2024-02-02T13:32:55.150094+0000","last_clean":"2024-02-02T13:32:55.150094+0000","last_became_active":"2024-02-02T13:32:54.647044+0000","last_became_peered":"2024-02-02T13:32:54.647044+0000","last_unstale":"2024-02-02T13:32:55.150094+0000","last_undegraded":"2024-02-02T13:32:55.150094+0000","last_fullsized":"2024-02-02T13:32:55.150094+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:32:19.129795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124176+0000","last_change":"2024-02-02T13:32:55.131494+0000","last_active":"2024-02-02T13:32:56.124176+0000","last_peered":"2024-02-02T13:32:56.124176+0000","last_clean":"2024-02-02T13:32:56.124176+0000","last_became_active":"2024-02-02T13:32:55.131282+0000","last_became_peered":"2024-02-02T13:32:55.131282+0000","last_unstale":"2024-02-02T13:32:56.124176+0000","last_undegraded":"2024-02-02T13:32:56.124176+0000","last_fullsized":"2024-02-02T13:32:56.124176+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:20.579158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127347+0000","last_change":"2024-02-02T13:32:54.644664+0000","last_active":"2024-02-02T13:32:56.127347+0000","last_peered":"2024-02-02T13:32:56.127347+0000","last_clean":"2024-02-02T13:32:56.127347+0000","last_became_active":"2024-02-02T13:32:54.644244+0000","last_became_peered":"2024-02-02T13:32:54.644244+0000","last_unstale":"2024-02-02T13:32:56.127347+0000","last_undegraded":"2024-02-02T13:32:56.127347+0000","last_fullsized":"2024-02-02T13:32:56.127347+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:19:58.961014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146528+0000","last_change":"2024-02-02T13:32:55.146375+0000","last_active":"2024-02-02T13:32:55.146528+0000","last_peered":"2024-02-02T13:32:55.146528+0000","last_clean":"2024-02-02T13:32:55.146528+0000","last_became_active":"2024-02-02T13:32:55.146189+0000","last_became_peered":"2024-02-02T13:32:55.146189+0000","last_unstale":"2024-02-02T13:32:55.146528+0000","last_undegraded":"2024-02-02T13:32:55.146528+0000","last_fullsized":"2024-02-02T13:32:55.146528+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:51:44.334699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127502+0000","last_change":"2024-02-02T13:32:54.643327+0000","last_active":"2024-02-02T13:32:56.127502+0000","last_peered":"2024-02-02T13:32:56.127502+0000","last_clean":"2024-02-02T13:32:56.127502+0000","last_became_active":"2024-02-02T13:32:54.635222+0000","last_became_peered":"2024-02-02T13:32:54.635222+0000","last_unstale":"2024-02-02T13:32:56.127502+0000","last_undegraded":"2024-02-02T13:32:56.127502+0000","last_fullsized":"2024-02-02T13:32:56.127502+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:17:54.473266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.154752+0000","last_change":"2024-02-02T13:32:55.154569+0000","last_active":"2024-02-02T13:32:55.154752+0000","last_peered":"2024-02-02T13:32:55.154752+0000","last_clean":"2024-02-02T13:32:55.154752+0000","last_became_active":"2024-02-02T13:32:55.154324+0000","last_became_peered":"2024-02-02T13:32:55.154324+0000","last_unstale":"2024-02-02T13:32:55.154752+0000","last_undegraded":"2024-02-02T13:32:55.154752+0000","last_fullsized":"2024-02-02T13:32:55.154752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:57.056137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121613+0000","last_change":"2024-02-02T13:32:54.644902+0000","last_active":"2024-02-02T13:32:55.121613+0000","last_peered":"2024-02-02T13:32:55.121613+0000","last_clean":"2024-02-02T13:32:55.121613+0000","last_became_active":"2024-02-02T13:32:54.644298+0000","last_became_peered":"2024-02-02T13:32:54.644298+0000","last_unstale":"2024-02-02T13:32:55.121613+0000","last_undegraded":"2024-02-02T13:32:55.121613+0000","last_fullsized":"2024-02-02T13:32:55.121613+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:56.039076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123819+0000","last_change":"2024-02-02T13:32:55.144909+0000","last_active":"2024-02-02T13:32:56.123819+0000","last_peered":"2024-02-02T13:32:56.123819+0000","last_clean":"2024-02-02T13:32:56.123819+0000","last_became_active":"2024-02-02T13:32:55.144616+0000","last_became_peered":"2024-02-02T13:32:55.144616+0000","last_unstale":"2024-02-02T13:32:56.123819+0000","last_undegraded":"2024-02-02T13:32:56.123819+0000","last_fullsized":"2024-02-02T13:32:56.123819+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:40.157204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.122965+0000","last_change":"2024-02-02T13:32:54.633822+0000","last_active":"2024-02-02T13:32:55.122965+0000","last_peered":"2024-02-02T13:32:55.122965+0000","last_clean":"2024-02-02T13:32:55.122965+0000","last_became_active":"2024-02-02T13:32:54.632989+0000","last_became_peered":"2024-02-02T13:32:54.632989+0000","last_unstale":"2024-02-02T13:32:55.122965+0000","last_undegraded":"2024-02-02T13:32:55.122965+0000","last_fullsized":"2024-02-02T13:32:55.122965+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:52.750896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126647+0000","last_change":"2024-02-02T13:32:55.141594+0000","last_active":"2024-02-02T13:32:56.126647+0000","last_peered":"2024-02-02T13:32:56.126647+0000","last_clean":"2024-02-02T13:32:56.126647+0000","last_became_active":"2024-02-02T13:32:55.141104+0000","last_became_peered":"2024-02-02T13:32:55.141104+0000","last_unstale":"2024-02-02T13:32:56.126647+0000","last_undegraded":"2024-02-02T13:32:56.126647+0000","last_fullsized":"2024-02-02T13:32:56.126647+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:30.707307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206857+0000","last_change":"2024-02-02T13:32:54.648728+0000","last_active":"2024-02-02T13:32:56.206857+0000","last_peered":"2024-02-02T13:32:56.206857+0000","last_clean":"2024-02-02T13:32:56.206857+0000","last_became_active":"2024-02-02T13:32:54.647937+0000","last_became_peered":"2024-02-02T13:32:54.647937+0000","last_unstale":"2024-02-02T13:32:56.206857+0000","last_undegraded":"2024-02-02T13:32:56.206857+0000","last_fullsized":"2024-02-02T13:32:56.206857+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:47.243559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145209+0000","last_change":"2024-02-02T13:32:55.143132+0000","last_active":"2024-02-02T13:32:55.145209+0000","last_peered":"2024-02-02T13:32:55.145209+0000","last_clean":"2024-02-02T13:32:55.145209+0000","last_became_active":"2024-02-02T13:32:55.142066+0000","last_became_peered":"2024-02-02T13:32:55.142066+0000","last_unstale":"2024-02-02T13:32:55.145209+0000","last_undegraded":"2024-02-02T13:32:55.145209+0000","last_fullsized":"2024-02-02T13:32:55.145209+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:53.767957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146636+0000","last_change":"2024-02-02T13:32:54.644159+0000","last_active":"2024-02-02T13:32:55.146636+0000","last_peered":"2024-02-02T13:32:55.146636+0000","last_clean":"2024-02-02T13:32:55.146636+0000","last_became_active":"2024-02-02T13:32:54.643873+0000","last_became_peered":"2024-02-02T13:32:54.643873+0000","last_unstale":"2024-02-02T13:32:55.146636+0000","last_undegraded":"2024-02-02T13:32:55.146636+0000","last_fullsized":"2024-02-02T13:32:55.146636+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:52.729134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152347+0000","last_change":"2024-02-02T13:32:55.151801+0000","last_active":"2024-02-02T13:32:55.152347+0000","last_peered":"2024-02-02T13:32:55.152347+0000","last_clean":"2024-02-02T13:32:55.152347+0000","last_became_active":"2024-02-02T13:32:55.151592+0000","last_became_peered":"2024-02-02T13:32:55.151592+0000","last_unstale":"2024-02-02T13:32:55.152347+0000","last_undegraded":"2024-02-02T13:32:55.152347+0000","last_fullsized":"2024-02-02T13:32:55.152347+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:57:51.157364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125918+0000","last_change":"2024-02-02T13:32:54.652191+0000","last_active":"2024-02-02T13:32:56.125918+0000","last_peered":"2024-02-02T13:32:56.125918+0000","last_clean":"2024-02-02T13:32:56.125918+0000","last_became_active":"2024-02-02T13:32:54.651906+0000","last_became_peered":"2024-02-02T13:32:54.651906+0000","last_unstale":"2024-02-02T13:32:56.125918+0000","last_undegraded":"2024-02-02T13:32:56.125918+0000","last_fullsized":"2024-02-02T13:32:56.125918+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:34.143891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124623+0000","last_change":"2024-02-02T13:32:55.139935+0000","last_active":"2024-02-02T13:32:56.124623+0000","last_peered":"2024-02-02T13:32:56.124623+0000","last_clean":"2024-02-02T13:32:56.124623+0000","last_became_active":"2024-02-02T13:32:55.138936+0000","last_became_peered":"2024-02-02T13:32:55.138936+0000","last_unstale":"2024-02-02T13:32:56.124623+0000","last_undegraded":"2024-02-02T13:32:56.124623+0000","last_fullsized":"2024-02-02T13:32:56.124623+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:17.483117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123867+0000","last_change":"2024-02-02T13:32:54.649257+0000","last_active":"2024-02-02T13:32:56.123867+0000","last_peered":"2024-02-02T13:32:56.123867+0000","last_clean":"2024-02-02T13:32:56.123867+0000","last_became_active":"2024-02-02T13:32:54.649024+0000","last_became_peered":"2024-02-02T13:32:54.649024+0000","last_unstale":"2024-02-02T13:32:56.123867+0000","last_undegraded":"2024-02-02T13:32:56.123867+0000","last_fullsized":"2024-02-02T13:32:56.123867+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:07:42.362754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145500+0000","last_change":"2024-02-02T13:32:55.145367+0000","last_active":"2024-02-02T13:32:55.145500+0000","last_peered":"2024-02-02T13:32:55.145500+0000","last_clean":"2024-02-02T13:32:55.145500+0000","last_became_active":"2024-02-02T13:32:55.144753+0000","last_became_peered":"2024-02-02T13:32:55.144753+0000","last_unstale":"2024-02-02T13:32:55.145500+0000","last_undegraded":"2024-02-02T13:32:55.145500+0000","last_fullsized":"2024-02-02T13:32:55.145500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:37.667814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206812+0000","last_change":"2024-02-02T13:32:54.645010+0000","last_active":"2024-02-02T13:32:56.206812+0000","last_peered":"2024-02-02T13:32:56.206812+0000","last_clean":"2024-02-02T13:32:56.206812+0000","last_became_active":"2024-02-02T13:32:54.644796+0000","last_became_peered":"2024-02-02T13:32:54.644796+0000","last_unstale":"2024-02-02T13:32:56.206812+0000","last_undegraded":"2024-02-02T13:32:56.206812+0000","last_fullsized":"2024-02-02T13:32:56.206812+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:58.955942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125338+0000","last_change":"2024-02-02T13:32:55.142394+0000","last_active":"2024-02-02T13:32:56.125338+0000","last_peered":"2024-02-02T13:32:56.125338+0000","last_clean":"2024-02-02T13:32:56.125338+0000","last_became_active":"2024-02-02T13:32:55.142115+0000","last_became_peered":"2024-02-02T13:32:55.142115+0000","last_unstale":"2024-02-02T13:32:56.125338+0000","last_undegraded":"2024-02-02T13:32:56.125338+0000","last_fullsized":"2024-02-02T13:32:56.125338+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:56.282353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124329+0000","last_change":"2024-02-02T13:32:54.651676+0000","last_active":"2024-02-02T13:32:56.124329+0000","last_peered":"2024-02-02T13:32:56.124329+0000","last_clean":"2024-02-02T13:32:56.124329+0000","last_became_active":"2024-02-02T13:32:54.651371+0000","last_became_peered":"2024-02-02T13:32:54.651371+0000","last_unstale":"2024-02-02T13:32:56.124329+0000","last_undegraded":"2024-02-02T13:32:56.124329+0000","last_fullsized":"2024-02-02T13:32:56.124329+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:51:43.317637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128151+0000","last_change":"2024-02-02T13:32:55.147216+0000","last_active":"2024-02-02T13:32:56.128151+0000","last_peered":"2024-02-02T13:32:56.128151+0000","last_clean":"2024-02-02T13:32:56.128151+0000","last_became_active":"2024-02-02T13:32:55.146917+0000","last_became_peered":"2024-02-02T13:32:55.146917+0000","last_unstale":"2024-02-02T13:32:56.128151+0000","last_undegraded":"2024-02-02T13:32:56.128151+0000","last_fullsized":"2024-02-02T13:32:56.128151+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:28.715360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126027+0000","last_change":"2024-02-02T13:32:54.651755+0000","last_active":"2024-02-02T13:32:56.126027+0000","last_peered":"2024-02-02T13:32:56.126027+0000","last_clean":"2024-02-02T13:32:56.126027+0000","last_became_active":"2024-02-02T13:32:54.651473+0000","last_became_peered":"2024-02-02T13:32:54.651473+0000","last_unstale":"2024-02-02T13:32:56.126027+0000","last_undegraded":"2024-02-02T13:32:56.126027+0000","last_fullsized":"2024-02-02T13:32:56.126027+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:41:48.597809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152684+0000","last_change":"2024-02-02T13:32:55.152464+0000","last_active":"2024-02-02T13:32:55.152684+0000","last_peered":"2024-02-02T13:32:55.152684+0000","last_clean":"2024-02-02T13:32:55.152684+0000","last_became_active":"2024-02-02T13:32:55.152250+0000","last_became_peered":"2024-02-02T13:32:55.152250+0000","last_unstale":"2024-02-02T13:32:55.152684+0000","last_undegraded":"2024-02-02T13:32:55.152684+0000","last_fullsized":"2024-02-02T13:32:55.152684+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:16.532202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121774+0000","last_change":"2024-02-02T13:32:54.649669+0000","last_active":"2024-02-02T13:32:55.121774+0000","last_peered":"2024-02-02T13:32:55.121774+0000","last_clean":"2024-02-02T13:32:55.121774+0000","last_became_active":"2024-02-02T13:32:54.649330+0000","last_became_peered":"2024-02-02T13:32:54.649330+0000","last_unstale":"2024-02-02T13:32:55.121774+0000","last_undegraded":"2024-02-02T13:32:55.121774+0000","last_fullsized":"2024-02-02T13:32:55.121774+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:19:16.549558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123738+0000","last_change":"2024-02-02T13:32:55.146077+0000","last_active":"2024-02-02T13:32:56.123738+0000","last_peered":"2024-02-02T13:32:56.123738+0000","last_clean":"2024-02-02T13:32:56.123738+0000","last_became_active":"2024-02-02T13:32:55.145750+0000","last_became_peered":"2024-02-02T13:32:55.145750+0000","last_unstale":"2024-02-02T13:32:56.123738+0000","last_undegraded":"2024-02-02T13:32:56.123738+0000","last_fullsized":"2024-02-02T13:32:56.123738+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:24:05.079623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206740+0000","last_change":"2024-02-02T13:32:54.647837+0000","last_active":"2024-02-02T13:32:56.206740+0000","last_peered":"2024-02-02T13:32:56.206740+0000","last_clean":"2024-02-02T13:32:56.206740+0000","last_became_active":"2024-02-02T13:32:54.647623+0000","last_became_peered":"2024-02-02T13:32:54.647623+0000","last_unstale":"2024-02-02T13:32:56.206740+0000","last_undegraded":"2024-02-02T13:32:56.206740+0000","last_fullsized":"2024-02-02T13:32:56.206740+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:36.000425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128145+0000","last_change":"2024-02-02T13:32:55.145926+0000","last_active":"2024-02-02T13:32:56.128145+0000","last_peered":"2024-02-02T13:32:56.128145+0000","last_clean":"2024-02-02T13:32:56.128145+0000","last_became_active":"2024-02-02T13:32:55.144036+0000","last_became_peered":"2024-02-02T13:32:55.144036+0000","last_unstale":"2024-02-02T13:32:56.128145+0000","last_undegraded":"2024-02-02T13:32:56.128145+0000","last_fullsized":"2024-02-02T13:32:56.128145+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:02.516881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127219+0000","last_change":"2024-02-02T13:32:54.651920+0000","last_active":"2024-02-02T13:32:56.127219+0000","last_peered":"2024-02-02T13:32:56.127219+0000","last_clean":"2024-02-02T13:32:56.127219+0000","last_became_active":"2024-02-02T13:32:54.651571+0000","last_became_peered":"2024-02-02T13:32:54.651571+0000","last_unstale":"2024-02-02T13:32:56.127219+0000","last_undegraded":"2024-02-02T13:32:56.127219+0000","last_fullsized":"2024-02-02T13:32:56.127219+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:55:39.212123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":303,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.127081+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:32:55.127081+0000","last_peered":"2024-02-02T13:32:55.127081+0000","last_clean":"2024-02-02T13:32:55.127081+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:32:55.127081+0000","last_undegraded":"2024-02-02T13:32:55.127081+0000","last_fullsized":"2024-02-02T13:32:55.127081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1450,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193862,"statfs":{"total":200000000,"available":198515143,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.087,"15min":1.0680000000000001},"min":{"1min":0.72899999999999998,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.514,"5min":1.7390000000000001,"15min":2.5310000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.071,"15min":1.113},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3120000000000001,"5min":1.6459999999999999,"15min":4.1609999999999996},"last":0.94699999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1140000000000001,"15min":1.093},"min":{"1min":0.746,"5min":0.68999999999999995,"15min":0.66300000000000003},"max":{"1min":1.506,"5min":1.8839999999999999,"15min":1.893},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1919999999999999,"15min":1.133},"min":{"1min":0.80200000000000005,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.54,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.266,"15min":1.228},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70099999999999996},"max":{"1min":1.5109999999999999,"5min":1.909,"15min":2.6669999999999998},"last":1.304},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1879999999999999,"15min":1.1830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.623},"max":{"1min":1.583,"5min":1.7949999999999999,"15min":3.5819999999999999},"last":1.075}]},{"osd":3,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.179,"15min":1.177},"min":{"1min":0.76400000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5449999999999999,"5min":1.6339999999999999,"15min":1.819},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.3480000000000001,"15min":1.3169999999999999},"min":{"1min":0.94299999999999995,"5min":0.752,"15min":0.48999999999999999},"max":{"1min":1.738,"5min":2.133,"15min":2.4620000000000002},"last":1.399}]},{"osd":4,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.292,"15min":1.216},"min":{"1min":0.72899999999999998,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.8169999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3160000000000001,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.61199999999999999},"max":{"1min":1.5609999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.98499999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.397,"15min":1.3620000000000001},"min":{"1min":1.2010000000000001,"5min":0.80200000000000005,"15min":0.54000000000000004},"max":{"1min":1.641,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.403,"15min":1.4390000000000001},"min":{"1min":0.91800000000000004,"5min":0.76000000000000001,"15min":0.56200000000000006},"max":{"1min":1.772,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.4399999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.4910000000000001,"15min":1.526},"min":{"1min":1.085,"5min":0.88700000000000001,"15min":0.64200000000000002},"max":{"1min":1.8979999999999999,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4470000000000001,"15min":1.4239999999999999},"min":{"1min":1.1279999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7809999999999999,"5min":1.821,"15min":2.0179999999999998},"last":1.1539999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107168,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0960000000000001,"15min":1.036},"min":{"1min":0.63200000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.4910000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.379},{"interface":"front","average":{"1min":1.034,"5min":1.099,"15min":1.0309999999999999},"min":{"1min":0.68799999999999994,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.462,"5min":1.768,"15min":1.768},"last":1.02}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0309999999999999},"min":{"1min":0.68799999999999994,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.421,"5min":1.5229999999999999,"15min":1.556},"last":1.466},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1619999999999999,"15min":1.0940000000000001},"min":{"1min":0.66000000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5369999999999999,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.333}]},{"osd":2,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1799999999999999,"15min":1.1000000000000001},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5820000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.151},{"interface":"front","average":{"1min":1.113,"5min":1.1739999999999999,"15min":1.1579999999999999},"min":{"1min":0.80400000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.5760000000000001,"5min":1.849,"15min":1.849},"last":1.121}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.254},"min":{"1min":0.82199999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5549999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1839999999999999,"15min":1.147},"min":{"1min":0.84899999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3400000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.5580000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4359999999999999,"15min":1.2849999999999999},"min":{"1min":0.71399999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.728,"5min":17.686,"15min":17.686},"last":1.244},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.5700000000000001,"15min":1.452},"min":{"1min":0.76000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.661,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.401}]},{"osd":5,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.2729999999999999},"min":{"1min":0.56399999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6950000000000001,"5min":1.986,"15min":1.986},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.415,"15min":1.369},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6259999999999999,"5min":3.331,"15min":3.331},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4590000000000001,"15min":1.4550000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":1.7230000000000001,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.196},{"interface":"front","average":{"1min":1.353,"5min":1.4370000000000001,"15min":1.4339999999999999},"min":{"1min":0.80200000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8939999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.617}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194441,"statfs":{"total":200000000,"available":199107601,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.64400000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.28,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.054,"15min":1.0449999999999999},"min":{"1min":0.68700000000000006,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4279999999999999,"5min":1.554,"15min":1.7150000000000001},"last":1.0389999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1439999999999999,"15min":1.2310000000000001},"min":{"1min":0.68999999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.585,"5min":1.6830000000000001,"15min":4.5899999999999999},"last":1.151},{"interface":"front","average":{"1min":1.107,"5min":1.0980000000000001,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.4159999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.196}]},{"osd":2,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.302,"15min":1.236},"min":{"1min":0.85199999999999998,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.7210000000000001,"5min":1.8899999999999999,"15min":3.0339999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.76000000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.641,"5min":1.706,"15min":3.0720000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.82599999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5009999999999999,"5min":1.591,"15min":1.7769999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2569999999999999,"15min":1.196},"min":{"1min":0.94699999999999995,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":5.4610000000000003},"last":0.89900000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2010000000000001,"15min":1.216},"min":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.64000000000000001},"max":{"1min":1.675,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3049999999999999,"15min":1.284},"min":{"1min":0.89300000000000002,"5min":0.622,"15min":0.55100000000000005},"max":{"1min":1.748,"5min":1.8029999999999999,"15min":1.833},"last":1.4419999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.357,"15min":1.3140000000000001},"min":{"1min":0.98899999999999999,"5min":0.88600000000000001,"15min":0.64700000000000002},"max":{"1min":1.78,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.359,"15min":1.3480000000000001},"min":{"1min":1.0369999999999999,"5min":0.82599999999999996,"15min":0.59099999999999997},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8640000000000001},"last":1.538}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.423,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":1.012,"15min":0.66500000000000004},"max":{"1min":1.841,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.651},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4810000000000001,"15min":1.3979999999999999},"min":{"1min":1.083,"5min":1.036,"15min":0.61599999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.5920000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":870,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194442,"statfs":{"total":200000000,"available":199108662,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0589999999999999,"15min":1.004},"min":{"1min":0.71499999999999997,"5min":0.60099999999999998,"15min":0.57999999999999996},"max":{"1min":1.5309999999999999,"5min":1.6439999999999999,"15min":4.5949999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.099,"15min":1.0820000000000001},"min":{"1min":0.73799999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7609999999999999,"5min":1.839,"15min":1.839},"last":1.149}]},{"osd":1,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.069,"15min":1.0069999999999999},"min":{"1min":0.75700000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.379,"5min":2.1659999999999999,"15min":2.4809999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.103,"5min":1.119,"15min":1.0549999999999999},"min":{"1min":0.82099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6240000000000001,"5min":1.7,"15min":4.5060000000000002},"last":1.2370000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.1439999999999999,"15min":1.1000000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.8580000000000001,"5min":1.9399999999999999,"15min":3.1200000000000001},"last":1.478},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1850000000000001,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.524,"5min":2.077,"15min":2.6000000000000001},"last":1.091}]},{"osd":3,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.198},"min":{"1min":0.82099999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.661,"5min":2.0470000000000002,"15min":2.9079999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3200000000000001,"15min":1.2589999999999999},"min":{"1min":0.94799999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5760000000000001,"5min":2.1110000000000002,"15min":3.9540000000000002},"last":1.1990000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.258,"15min":1.1719999999999999},"min":{"1min":0.83799999999999997,"5min":0.63700000000000001,"15min":0.60899999999999999},"max":{"1min":1.645,"5min":1.958,"15min":4.3940000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.357,"15min":1.296},"min":{"1min":0.98899999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.944,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.603}]},{"osd":6,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4039999999999999,"15min":1.3220000000000001},"min":{"1min":0.86799999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0920000000000001,"5min":2.2869999999999999,"15min":2.7989999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4670000000000001,"15min":1.4079999999999999},"min":{"1min":1.024,"5min":0.70399999999999996,"15min":0.67800000000000005},"max":{"1min":1.9610000000000001,"5min":2.1240000000000001,"15min":4.2809999999999997},"last":1.4470000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.498,"15min":1.407},"min":{"1min":1.081,"5min":0.76000000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":1.022,"5min":0.74099999999999999,"15min":0.73599999999999999},"max":{"1min":2.0680000000000001,"5min":2.4900000000000002,"15min":2.698},"last":1.2949999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107553,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.72099999999999997},"min":{"1min":0.50900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0449999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92300000000000004,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.361,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.95299999999999996,"15min":0.90800000000000003},"min":{"1min":0.58899999999999997,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":1.234,"5min":2.403,"15min":2.403},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.998,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63500000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.474,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.139}]},{"osd":2,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.02,"15min":0.96999999999999997},"min":{"1min":0.57599999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.266,"5min":1.611,"15min":1.611},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.1719999999999999,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1830000000000001,"15min":1.181},"min":{"1min":0.90500000000000003,"5min":0.70299999999999996,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.99,"15min":3.585},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2549999999999999,"15min":1.214},"min":{"1min":0.72499999999999998,"5min":0.68100000000000005,"15min":0.61299999999999999},"max":{"1min":1.5489999999999999,"5min":1.7749999999999999,"15min":1.877},"last":1.597}]},{"osd":5,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.169,"15min":1.202},"min":{"1min":0.73799999999999999,"5min":0.64400000000000002,"15min":0.57099999999999995},"max":{"1min":1.444,"5min":3.738,"15min":3.738},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.262,"15min":1.2529999999999999},"min":{"1min":0.85699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.603,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3520000000000001,"15min":1.28},"min":{"1min":0.89000000000000001,"5min":0.72599999999999998,"15min":0.58299999999999996},"max":{"1min":1.663,"5min":1.895,"15min":1.8979999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2809999999999999},"min":{"1min":0.82199999999999995,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":1.3939999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.155}]},{"osd":7,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2729999999999999,"15min":1.258},"min":{"1min":0.81599999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6910000000000001,"5min":1.869,"15min":3.391},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.357,"15min":1.3220000000000001},"min":{"1min":0.83699999999999997,"5min":0.753,"15min":0.625},"max":{"1min":1.613,"5min":1.946,"15min":2.0409999999999999},"last":1.6579999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107097,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.879,"15min":0.80500000000000005},"min":{"1min":0.435,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":7.1840000000000002,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.4550000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0109999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3720000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.085},{"interface":"front","average":{"1min":1.034,"5min":1.028,"15min":0.89600000000000002},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.35899999999999999},"max":{"1min":1.6279999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93100000000000005,"15min":0.82399999999999995},"min":{"1min":0.56699999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0880000000000001,"15min":1},"min":{"1min":0.56399999999999995,"5min":0.435,"15min":0.432},"max":{"1min":1.458,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2829999999999999,"15min":1.1990000000000001},"min":{"1min":0.77000000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.605,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.2609999999999999,"15min":1.181},"min":{"1min":0.86699999999999999,"5min":0.503,"15min":0.503},"max":{"1min":2.02,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.3360000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2330000000000001,"15min":1.1200000000000001},"min":{"1min":0.877,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.304},{"interface":"front","average":{"1min":1.282,"5min":1.335,"15min":1.2809999999999999},"min":{"1min":0.85599999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.9770000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.22}]},{"osd":6,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.343,"15min":1.2150000000000001},"min":{"1min":0.83899999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.919,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3160000000000001,"15min":1.2250000000000001},"min":{"1min":0.80700000000000005,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":2.6920000000000002,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.6200000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.395,"15min":1.252},"min":{"1min":0.96799999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.976,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.667},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.2450000000000001},"min":{"1min":0.85899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.0179999999999998,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.8300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":870,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194442,"statfs":{"total":200000000,"available":199108732,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.95099999999999996},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.34200000000000003},"max":{"1min":1.304,"5min":1.425,"15min":3.5680000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":1.2390000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.256,"5min":1.4870000000000001,"15min":5.8099999999999996},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.639},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.0669999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.4410000000000001,"5min":1.823,"15min":3.0259999999999998},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.119,"15min":1.208},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6699999999999999,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.2030000000000001},"min":{"1min":0.505,"5min":0.501,"15min":0.49299999999999999},"max":{"1min":1.377,"5min":1.774,"15min":4.8760000000000003},"last":1.29}]},{"osd":4,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.125,"15min":1.264},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.625},"max":{"1min":1.6220000000000001,"5min":1.837,"15min":2.9449999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2869999999999999,"15min":1.339},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.9490000000000001,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":0.996}]},{"osd":5,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.296,"15min":1.409},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.579,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.454},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3899999999999999},"min":{"1min":0.81599999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.6970000000000001,"5min":1.7090000000000001,"15min":3.0249999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.321,"15min":1.5},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.5529999999999999,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.3200000000000001,"15min":1.4410000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.81000000000000005},"max":{"1min":1.77,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3839999999999999,"15min":1.5029999999999999},"min":{"1min":0.89200000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.008,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3779999999999999,"15min":1.548},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":2.0790000000000002,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.6259999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1446,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193865,"statfs":{"total":200000000,"available":198518364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":1.0369999999999999},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.194,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":1.331},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.29199999999999998},"max":{"1min":1.6459999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94899999999999995,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32700000000000001},"max":{"1min":1.349,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93999999999999995,"15min":1.337},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.7,"5min":1.706,"15min":10.781000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":1.1439999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.998,"5min":1.04,"15min":1.236},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.36699999999999999},"max":{"1min":1.4059999999999999,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.2969999999999999},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.66800000000000004},"max":{"1min":1.52,"5min":1.9670000000000001,"15min":3.669},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.2529999999999999,"15min":1.3620000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4790000000000001,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2669999999999999,"15min":1.6779999999999999},"min":{"1min":0.79300000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6899999999999999,"5min":2.0150000000000001,"15min":10.651},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.232,"15min":1.3720000000000001},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.5069999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3049999999999999,"15min":1.427},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5349999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.76},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.708},"min":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7290000000000001,"5min":2.0510000000000002,"15min":10.708},"last":1.0349999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.347,"15min":1.7350000000000001},"min":{"1min":0.77700000000000002,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":1.833,"5min":2.3010000000000002,"15min":10.74},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.298,"15min":1.405},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.633,"5min":1.9159999999999999,"15min":3.863},"last":1.1000000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:33:00.051 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:33:00.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.300+0000 7f0ca8a00640 1 Processor -- start 2024-02-02T13:33:00.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.300+0000 7f0ca8a00640 1 -- start start 2024-02-02T13:33:00.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca8a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0ca015a010 0x7f0ca00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 0x7f0ca00b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca00cc7c0 con 0x7f0ca00ce130 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca8a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00b10b0 con 0x7f0ca015a010 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00ce930 con 0x7f0ca00b0700 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.301+0000 7f0ca6000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0ca015a010 0x7f0ca00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0ca015a010 0x7f0ca00c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59388/0 (socket says 172.21.15.91:59388) 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 -- 172.21.15.91:0/1013343796 learned_addr learned my addr 172.21.15.91:0/1013343796 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6a00640 1 --2- 172.21.15.91:0/1013343796 >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 0x7f0ca00b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 -- 172.21.15.91:0/1013343796 >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 msgr2=0x7f0ca00b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 --2- 172.21.15.91:0/1013343796 >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 0x7f0ca00b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 -- 172.21.15.91:0/1013343796 >> v1:172.21.15.113:6789/0 conn(0x7f0ca00ce130 legacy=0x7f0ca00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6000640 1 -- 172.21.15.91:0/1013343796 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca00cd640 con 0x7f0ca015a010 2024-02-02T13:33:00.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.302+0000 7f0ca6a00640 1 --2- 172.21.15.91:0/1013343796 >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 0x7f0ca00b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.303+0000 7f0ca6000640 1 --2- 172.21.15.91:0/1013343796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca015a010 0x7f0ca00c9670 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00cec10 tx=0x7f0c9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84f8facbd887d61 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.303+0000 7f0ca4c00640 1 -- 172.21.15.91:0/1013343796 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90031870 con 0x7f0ca015a010 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.303+0000 7f0ca4c00640 1 -- 172.21.15.91:0/1013343796 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c900319d0 con 0x7f0ca015a010 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.303+0000 7f0ca4c00640 1 -- 172.21.15.91:0/1013343796 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90031d60 con 0x7f0ca015a010 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 -- 172.21.15.91:0/1013343796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca015a010 msgr2=0x7f0ca00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/1013343796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca015a010 0x7f0ca00c9670 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00cec10 tx=0x7f0c9002b910 comp rx=0 tx=0).stop 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 -- 172.21.15.91:0/1013343796 shutdown_connections 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/1013343796 >> v2:172.21.15.113:3300/0 conn(0x7f0ca00b0700 0x7f0ca00b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/1013343796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca015a010 0x7f0ca00c9670 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 -- 172.21.15.91:0/1013343796 >> 172.21.15.91:0/1013343796 conn(0x7f0ca00cec80 msgr2=0x7f0ca00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:00.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 -- 172.21.15.91:0/1013343796 shutdown_connections 2024-02-02T13:33:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.304+0000 7f0ca8a00640 1 -- 172.21.15.91:0/1013343796 wait complete. 2024-02-02T13:33:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.306+0000 7f0ca8a00640 1 Processor -- start 2024-02-02T13:33:00.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.306+0000 7f0ca8a00640 1 -- start start 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ca00b0700 0x7f0ca014b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ca015a010 0x7f0ca015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00c2390 con 0x7f0ca00ce130 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00c25a0 con 0x7f0ca015a010 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00c2880 con 0x7f0ca00b0700 2024-02-02T13:33:00.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca5600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59394/0 (socket says 172.21.15.91:59394) 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.307+0000 7f0ca5600640 1 -- 172.21.15.91:0/3635456635 learned_addr learned my addr 172.21.15.91:0/3635456635 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca6000640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ca00b0700 0x7f0ca014b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca6a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ca015a010 0x7f0ca015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 -- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ca00b0700 msgr2=0x7f0ca014b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ca00b0700 0x7f0ca014b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 -- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ca015a010 msgr2=0x7f0ca015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ca015a010 0x7f0ca015d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 -- 172.21.15.91:0/3635456635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900094d0 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca5600640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c430 tx=0x7f0c8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88011790 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c880118f0 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88011ce0 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.308+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca00c2b60 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.309+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca014b590 con 0x7f0ca00ce130 2024-02-02T13:33:00.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.310+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0066cf0 con 0x7f0ca00ce130 2024-02-02T13:33:00.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.311+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c8801c460 con 0x7f0ca00ce130 2024-02-02T13:33:00.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.311+0000 7f0c9e000640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 0x7f0c7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.312+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c88088880 con 0x7f0ca00ce130 2024-02-02T13:33:00.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.312+0000 7f0ca6000640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 0x7f0c7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.312+0000 7f0ca6000640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 0x7f0c7806c8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0c90036000 tx=0x7f0c9003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.316+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c88053e70 con 0x7f0ca00ce130 2024-02-02T13:33:00.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.498+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ca006cb70 con 0x7f0c7806a430 2024-02-02T13:33:00.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.520+0000 7f0c9e000640 1 -- 172.21.15.91:0/3635456635 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283120 (secure 0 0 0) 0x7f0c90031810 con 0x7f0c7806a430 2024-02-02T13:33:00.523 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:33:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 msgr2=0x7f0c7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 0x7f0c7806c8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0c90036000 tx=0x7f0c9003c040 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 msgr2=0x7f0ca00c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c430 tx=0x7f0c8800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 shutdown_connections 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0c7806a430 0x7f0c7806c8f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ca015a010 0x7f0ca015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca00ce130 0x7f0ca00c3360 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 --2- 172.21.15.91:0/3635456635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ca00b0700 0x7f0ca014b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 >> 172.21.15.91:0/3635456635 conn(0x7f0ca00cec80 msgr2=0x7f0ca006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 shutdown_connections 2024-02-02T13:33:00.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.528+0000 7f0ca8a00640 1 -- 172.21.15.91:0/3635456635 wait complete. 2024-02-02T13:33:00.547 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-02-02T13:33:00.400285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554375,"statfs":{"total":1600000000,"available":1591680320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002137"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143778+0000","last_change":"2024-02-02T13:32:55.143657+0000","last_active":"2024-02-02T13:32:55.143778+0000","last_peered":"2024-02-02T13:32:55.143778+0000","last_clean":"2024-02-02T13:32:55.143778+0000","last_became_active":"2024-02-02T13:32:55.143445+0000","last_became_peered":"2024-02-02T13:32:55.143445+0000","last_unstale":"2024-02-02T13:32:55.143778+0000","last_undegraded":"2024-02-02T13:32:55.143778+0000","last_fullsized":"2024-02-02T13:32:55.143778+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:27.490307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123546+0000","last_change":"2024-02-02T13:32:55.138559+0000","last_active":"2024-02-02T13:32:56.123546+0000","last_peered":"2024-02-02T13:32:56.123546+0000","last_clean":"2024-02-02T13:32:56.123546+0000","last_became_active":"2024-02-02T13:32:55.137573+0000","last_became_peered":"2024-02-02T13:32:55.137573+0000","last_unstale":"2024-02-02T13:32:56.123546+0000","last_undegraded":"2024-02-02T13:32:56.123546+0000","last_fullsized":"2024-02-02T13:32:56.123546+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:35.416020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209073+0000","last_change":"2024-02-02T13:32:55.154334+0000","last_active":"2024-02-02T13:32:56.209073+0000","last_peered":"2024-02-02T13:32:56.209073+0000","last_clean":"2024-02-02T13:32:56.209073+0000","last_became_active":"2024-02-02T13:32:55.151602+0000","last_became_peered":"2024-02-02T13:32:55.151602+0000","last_unstale":"2024-02-02T13:32:56.209073+0000","last_undegraded":"2024-02-02T13:32:56.209073+0000","last_fullsized":"2024-02-02T13:32:56.209073+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:56:06.197241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145658+0000","last_change":"2024-02-02T13:32:55.145611+0000","last_active":"2024-02-02T13:32:55.145658+0000","last_peered":"2024-02-02T13:32:55.145658+0000","last_clean":"2024-02-02T13:32:55.145658+0000","last_became_active":"2024-02-02T13:32:55.145441+0000","last_became_peered":"2024-02-02T13:32:55.145441+0000","last_unstale":"2024-02-02T13:32:55.145658+0000","last_undegraded":"2024-02-02T13:32:55.145658+0000","last_fullsized":"2024-02-02T13:32:55.145658+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:30.684351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126850+0000","last_change":"2024-02-02T13:32:55.140235+0000","last_active":"2024-02-02T13:32:56.126850+0000","last_peered":"2024-02-02T13:32:56.126850+0000","last_clean":"2024-02-02T13:32:56.126850+0000","last_became_active":"2024-02-02T13:32:55.139735+0000","last_became_peered":"2024-02-02T13:32:55.139735+0000","last_unstale":"2024-02-02T13:32:56.126850+0000","last_undegraded":"2024-02-02T13:32:56.126850+0000","last_fullsized":"2024-02-02T13:32:56.126850+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:34:16.191248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146080+0000","last_change":"2024-02-02T13:32:55.143321+0000","last_active":"2024-02-02T13:32:55.146080+0000","last_peered":"2024-02-02T13:32:55.146080+0000","last_clean":"2024-02-02T13:32:55.146080+0000","last_became_active":"2024-02-02T13:32:55.142970+0000","last_became_peered":"2024-02-02T13:32:55.142970+0000","last_unstale":"2024-02-02T13:32:55.146080+0000","last_undegraded":"2024-02-02T13:32:55.146080+0000","last_fullsized":"2024-02-02T13:32:55.146080+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:15:37.403990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152839+0000","last_change":"2024-02-02T13:32:55.152190+0000","last_active":"2024-02-02T13:32:55.152839+0000","last_peered":"2024-02-02T13:32:55.152839+0000","last_clean":"2024-02-02T13:32:55.152839+0000","last_became_active":"2024-02-02T13:32:55.151939+0000","last_became_peered":"2024-02-02T13:32:55.151939+0000","last_unstale":"2024-02-02T13:32:55.152839+0000","last_undegraded":"2024-02-02T13:32:55.152839+0000","last_fullsized":"2024-02-02T13:32:55.152839+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:44:15.437339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209321+0000","last_change":"2024-02-02T13:32:55.154020+0000","last_active":"2024-02-02T13:32:56.209321+0000","last_peered":"2024-02-02T13:32:56.209321+0000","last_clean":"2024-02-02T13:32:56.209321+0000","last_became_active":"2024-02-02T13:32:55.151296+0000","last_became_peered":"2024-02-02T13:32:55.151296+0000","last_unstale":"2024-02-02T13:32:56.209321+0000","last_undegraded":"2024-02-02T13:32:56.209321+0000","last_fullsized":"2024-02-02T13:32:56.209321+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:56:24.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.208785+0000","last_change":"2024-02-02T13:32:55.153602+0000","last_active":"2024-02-02T13:32:56.208785+0000","last_peered":"2024-02-02T13:32:56.208785+0000","last_clean":"2024-02-02T13:32:56.208785+0000","last_became_active":"2024-02-02T13:32:55.153276+0000","last_became_peered":"2024-02-02T13:32:55.153276+0000","last_unstale":"2024-02-02T13:32:56.208785+0000","last_undegraded":"2024-02-02T13:32:56.208785+0000","last_fullsized":"2024-02-02T13:32:56.208785+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:35.160953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124444+0000","last_change":"2024-02-02T13:32:55.147189+0000","last_active":"2024-02-02T13:32:56.124444+0000","last_peered":"2024-02-02T13:32:56.124444+0000","last_clean":"2024-02-02T13:32:56.124444+0000","last_became_active":"2024-02-02T13:32:55.139691+0000","last_became_peered":"2024-02-02T13:32:55.139691+0000","last_unstale":"2024-02-02T13:32:56.124444+0000","last_undegraded":"2024-02-02T13:32:56.124444+0000","last_fullsized":"2024-02-02T13:32:56.124444+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:17:55.490328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206777+0000","last_change":"2024-02-02T13:32:55.151979+0000","last_active":"2024-02-02T13:32:56.206777+0000","last_peered":"2024-02-02T13:32:56.206777+0000","last_clean":"2024-02-02T13:32:56.206777+0000","last_became_active":"2024-02-02T13:32:55.151627+0000","last_became_peered":"2024-02-02T13:32:55.151627+0000","last_unstale":"2024-02-02T13:32:56.206777+0000","last_undegraded":"2024-02-02T13:32:56.206777+0000","last_fullsized":"2024-02-02T13:32:56.206777+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:09:18.206248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209180+0000","last_change":"2024-02-02T13:32:55.156418+0000","last_active":"2024-02-02T13:32:56.209180+0000","last_peered":"2024-02-02T13:32:56.209180+0000","last_clean":"2024-02-02T13:32:56.209180+0000","last_became_active":"2024-02-02T13:32:55.154919+0000","last_became_peered":"2024-02-02T13:32:55.154919+0000","last_unstale":"2024-02-02T13:32:56.209180+0000","last_undegraded":"2024-02-02T13:32:56.209180+0000","last_fullsized":"2024-02-02T13:32:56.209180+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:08:43.064057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.140404+0000","last_change":"2024-02-02T13:32:55.139471+0000","last_active":"2024-02-02T13:32:55.140404+0000","last_peered":"2024-02-02T13:32:55.140404+0000","last_clean":"2024-02-02T13:32:55.140404+0000","last_became_active":"2024-02-02T13:32:55.139332+0000","last_became_peered":"2024-02-02T13:32:55.139332+0000","last_unstale":"2024-02-02T13:32:55.140404+0000","last_undegraded":"2024-02-02T13:32:55.140404+0000","last_fullsized":"2024-02-02T13:32:55.140404+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:51.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123531+0000","last_change":"2024-02-02T13:32:55.146689+0000","last_active":"2024-02-02T13:32:56.123531+0000","last_peered":"2024-02-02T13:32:56.123531+0000","last_clean":"2024-02-02T13:32:56.123531+0000","last_became_active":"2024-02-02T13:32:55.146472+0000","last_became_peered":"2024-02-02T13:32:55.146472+0000","last_unstale":"2024-02-02T13:32:56.123531+0000","last_undegraded":"2024-02-02T13:32:56.123531+0000","last_fullsized":"2024-02-02T13:32:56.123531+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:19.433416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.155933+0000","last_change":"2024-02-02T13:32:55.155855+0000","last_active":"2024-02-02T13:32:55.155933+0000","last_peered":"2024-02-02T13:32:55.155933+0000","last_clean":"2024-02-02T13:32:55.155933+0000","last_became_active":"2024-02-02T13:32:55.155461+0000","last_became_peered":"2024-02-02T13:32:55.155461+0000","last_unstale":"2024-02-02T13:32:55.155933+0000","last_undegraded":"2024-02-02T13:32:55.155933+0000","last_fullsized":"2024-02-02T13:32:55.155933+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:25.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.142918+0000","last_change":"2024-02-02T13:32:55.142402+0000","last_active":"2024-02-02T13:32:55.142918+0000","last_peered":"2024-02-02T13:32:55.142918+0000","last_clean":"2024-02-02T13:32:55.142918+0000","last_became_active":"2024-02-02T13:32:55.142233+0000","last_became_peered":"2024-02-02T13:32:55.142233+0000","last_unstale":"2024-02-02T13:32:55.142918+0000","last_undegraded":"2024-02-02T13:32:55.142918+0000","last_fullsized":"2024-02-02T13:32:55.142918+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:24.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152934+0000","last_change":"2024-02-02T13:32:55.152826+0000","last_active":"2024-02-02T13:32:55.152934+0000","last_peered":"2024-02-02T13:32:55.152934+0000","last_clean":"2024-02-02T13:32:55.152934+0000","last_became_active":"2024-02-02T13:32:55.152562+0000","last_became_peered":"2024-02-02T13:32:55.152562+0000","last_unstale":"2024-02-02T13:32:55.152934+0000","last_undegraded":"2024-02-02T13:32:55.152934+0000","last_fullsized":"2024-02-02T13:32:55.152934+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:03.978723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.137124+0000","last_change":"2024-02-02T13:32:55.136928+0000","last_active":"2024-02-02T13:32:55.137124+0000","last_peered":"2024-02-02T13:32:55.137124+0000","last_clean":"2024-02-02T13:32:55.137124+0000","last_became_active":"2024-02-02T13:32:55.136228+0000","last_became_peered":"2024-02-02T13:32:55.136228+0000","last_unstale":"2024-02-02T13:32:55.137124+0000","last_undegraded":"2024-02-02T13:32:55.137124+0000","last_fullsized":"2024-02-02T13:32:55.137124+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:25:03.384036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206511+0000","last_change":"2024-02-02T13:32:55.145455+0000","last_active":"2024-02-02T13:32:56.206511+0000","last_peered":"2024-02-02T13:32:56.206511+0000","last_clean":"2024-02-02T13:32:56.206511+0000","last_became_active":"2024-02-02T13:32:55.145222+0000","last_became_peered":"2024-02-02T13:32:55.145222+0000","last_unstale":"2024-02-02T13:32:56.206511+0000","last_undegraded":"2024-02-02T13:32:56.206511+0000","last_fullsized":"2024-02-02T13:32:56.206511+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:57:07.862777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.129823+0000","last_change":"2024-02-02T13:32:55.129695+0000","last_active":"2024-02-02T13:32:55.129823+0000","last_peered":"2024-02-02T13:32:55.129823+0000","last_clean":"2024-02-02T13:32:55.129823+0000","last_became_active":"2024-02-02T13:32:55.129104+0000","last_became_peered":"2024-02-02T13:32:55.129104+0000","last_unstale":"2024-02-02T13:32:55.129823+0000","last_undegraded":"2024-02-02T13:32:55.129823+0000","last_fullsized":"2024-02-02T13:32:55.129823+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:33:26.571878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128547+0000","last_change":"2024-02-02T13:32:55.151447+0000","last_active":"2024-02-02T13:32:56.128547+0000","last_peered":"2024-02-02T13:32:56.128547+0000","last_clean":"2024-02-02T13:32:56.128547+0000","last_became_active":"2024-02-02T13:32:55.151171+0000","last_became_peered":"2024-02-02T13:32:55.151171+0000","last_unstale":"2024-02-02T13:32:56.128547+0000","last_undegraded":"2024-02-02T13:32:56.128547+0000","last_fullsized":"2024-02-02T13:32:56.128547+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:16.532202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206774+0000","last_change":"2024-02-02T13:32:55.147175+0000","last_active":"2024-02-02T13:32:56.206774+0000","last_peered":"2024-02-02T13:32:56.206774+0000","last_clean":"2024-02-02T13:32:56.206774+0000","last_became_active":"2024-02-02T13:32:55.146718+0000","last_became_peered":"2024-02-02T13:32:55.146718+0000","last_unstale":"2024-02-02T13:32:56.206774+0000","last_undegraded":"2024-02-02T13:32:56.206774+0000","last_fullsized":"2024-02-02T13:32:56.206774+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:56:28.642640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209960+0000","last_change":"2024-02-02T13:32:55.153382+0000","last_active":"2024-02-02T13:32:56.209960+0000","last_peered":"2024-02-02T13:32:56.209960+0000","last_clean":"2024-02-02T13:32:56.209960+0000","last_became_active":"2024-02-02T13:32:55.150560+0000","last_became_peered":"2024-02-02T13:32:55.150560+0000","last_unstale":"2024-02-02T13:32:56.209960+0000","last_undegraded":"2024-02-02T13:32:56.209960+0000","last_fullsized":"2024-02-02T13:32:56.209960+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:29:31.019555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206604+0000","last_change":"2024-02-02T13:32:55.143309+0000","last_active":"2024-02-02T13:32:56.206604+0000","last_peered":"2024-02-02T13:32:56.206604+0000","last_clean":"2024-02-02T13:32:56.206604+0000","last_became_active":"2024-02-02T13:32:55.142952+0000","last_became_peered":"2024-02-02T13:32:55.142952+0000","last_unstale":"2024-02-02T13:32:56.206604+0000","last_undegraded":"2024-02-02T13:32:56.206604+0000","last_fullsized":"2024-02-02T13:32:56.206604+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:28.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124535+0000","last_change":"2024-02-02T13:32:55.139419+0000","last_active":"2024-02-02T13:32:56.124535+0000","last_peered":"2024-02-02T13:32:56.124535+0000","last_clean":"2024-02-02T13:32:56.124535+0000","last_became_active":"2024-02-02T13:32:55.138383+0000","last_became_peered":"2024-02-02T13:32:55.138383+0000","last_unstale":"2024-02-02T13:32:56.124535+0000","last_undegraded":"2024-02-02T13:32:56.124535+0000","last_fullsized":"2024-02-02T13:32:56.124535+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:01:33.246916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.141818+0000","last_change":"2024-02-02T13:32:55.141765+0000","last_active":"2024-02-02T13:32:55.141818+0000","last_peered":"2024-02-02T13:32:55.141818+0000","last_clean":"2024-02-02T13:32:55.141818+0000","last_became_active":"2024-02-02T13:32:55.141633+0000","last_became_peered":"2024-02-02T13:32:55.141633+0000","last_unstale":"2024-02-02T13:32:55.141818+0000","last_undegraded":"2024-02-02T13:32:55.141818+0000","last_fullsized":"2024-02-02T13:32:55.141818+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:18:38.566112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143168+0000","last_change":"2024-02-02T13:32:55.142804+0000","last_active":"2024-02-02T13:32:55.143168+0000","last_peered":"2024-02-02T13:32:55.143168+0000","last_clean":"2024-02-02T13:32:55.143168+0000","last_became_active":"2024-02-02T13:32:55.142605+0000","last_became_peered":"2024-02-02T13:32:55.142605+0000","last_unstale":"2024-02-02T13:32:55.143168+0000","last_undegraded":"2024-02-02T13:32:55.143168+0000","last_fullsized":"2024-02-02T13:32:55.143168+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:07:43.379816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127062+0000","last_change":"2024-02-02T13:32:55.138553+0000","last_active":"2024-02-02T13:32:56.127062+0000","last_peered":"2024-02-02T13:32:56.127062+0000","last_clean":"2024-02-02T13:32:56.127062+0000","last_became_active":"2024-02-02T13:32:55.133675+0000","last_became_peered":"2024-02-02T13:32:55.133675+0000","last_unstale":"2024-02-02T13:32:56.127062+0000","last_undegraded":"2024-02-02T13:32:56.127062+0000","last_fullsized":"2024-02-02T13:32:56.127062+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:29.793178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.154016+0000","last_change":"2024-02-02T13:32:55.153950+0000","last_active":"2024-02-02T13:32:55.154016+0000","last_peered":"2024-02-02T13:32:55.154016+0000","last_clean":"2024-02-02T13:32:55.154016+0000","last_became_active":"2024-02-02T13:32:55.153725+0000","last_became_peered":"2024-02-02T13:32:55.153725+0000","last_unstale":"2024-02-02T13:32:55.154016+0000","last_undegraded":"2024-02-02T13:32:55.154016+0000","last_fullsized":"2024-02-02T13:32:55.154016+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:57:01.762993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.155785+0000","last_change":"2024-02-02T13:32:55.155720+0000","last_active":"2024-02-02T13:32:55.155785+0000","last_peered":"2024-02-02T13:32:55.155785+0000","last_clean":"2024-02-02T13:32:55.155785+0000","last_became_active":"2024-02-02T13:32:55.155142+0000","last_became_peered":"2024-02-02T13:32:55.155142+0000","last_unstale":"2024-02-02T13:32:55.155785+0000","last_undegraded":"2024-02-02T13:32:55.155785+0000","last_fullsized":"2024-02-02T13:32:55.155785+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:21.131397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127689+0000","last_change":"2024-02-02T13:32:55.146091+0000","last_active":"2024-02-02T13:32:56.127689+0000","last_peered":"2024-02-02T13:32:56.127689+0000","last_clean":"2024-02-02T13:32:56.127689+0000","last_became_active":"2024-02-02T13:32:55.145576+0000","last_became_peered":"2024-02-02T13:32:55.145576+0000","last_unstale":"2024-02-02T13:32:56.127689+0000","last_undegraded":"2024-02-02T13:32:56.127689+0000","last_fullsized":"2024-02-02T13:32:56.127689+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:26:58.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145171+0000","last_change":"2024-02-02T13:32:55.141878+0000","last_active":"2024-02-02T13:32:55.145171+0000","last_peered":"2024-02-02T13:32:55.145171+0000","last_clean":"2024-02-02T13:32:55.145171+0000","last_became_active":"2024-02-02T13:32:55.141186+0000","last_became_peered":"2024-02-02T13:32:55.141186+0000","last_unstale":"2024-02-02T13:32:55.145171+0000","last_undegraded":"2024-02-02T13:32:55.145171+0000","last_fullsized":"2024-02-02T13:32:55.145171+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:19:17.566619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.132491+0000","last_change":"2024-02-02T13:32:55.132442+0000","last_active":"2024-02-02T13:32:55.132491+0000","last_peered":"2024-02-02T13:32:55.132491+0000","last_clean":"2024-02-02T13:32:55.132491+0000","last_became_active":"2024-02-02T13:32:55.132275+0000","last_became_peered":"2024-02-02T13:32:55.132275+0000","last_unstale":"2024-02-02T13:32:55.132491+0000","last_undegraded":"2024-02-02T13:32:55.132491+0000","last_fullsized":"2024-02-02T13:32:55.132491+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:50.996447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"160'4","reported_seq":15,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125705+0000","last_change":"2024-02-02T13:32:54.648778+0000","last_active":"2024-02-02T13:32:56.125705+0000","last_peered":"2024-02-02T13:32:56.125705+0000","last_clean":"2024-02-02T13:32:56.125705+0000","last_became_active":"2024-02-02T13:32:54.648456+0000","last_became_peered":"2024-02-02T13:32:54.648456+0000","last_unstale":"2024-02-02T13:32:56.125705+0000","last_undegraded":"2024-02-02T13:32:56.125705+0000","last_fullsized":"2024-02-02T13:32:56.125705+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:08:42.046996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209570+0000","last_change":"2024-02-02T13:32:55.153729+0000","last_active":"2024-02-02T13:32:56.209570+0000","last_peered":"2024-02-02T13:32:56.209570+0000","last_clean":"2024-02-02T13:32:56.209570+0000","last_became_active":"2024-02-02T13:32:55.150951+0000","last_became_peered":"2024-02-02T13:32:55.150951+0000","last_unstale":"2024-02-02T13:32:56.209570+0000","last_undegraded":"2024-02-02T13:32:56.209570+0000","last_fullsized":"2024-02-02T13:32:56.209570+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:50.996447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126309+0000","last_change":"2024-02-02T13:32:54.646654+0000","last_active":"2024-02-02T13:32:56.126309+0000","last_peered":"2024-02-02T13:32:56.126309+0000","last_clean":"2024-02-02T13:32:56.126309+0000","last_became_active":"2024-02-02T13:32:54.642723+0000","last_became_peered":"2024-02-02T13:32:54.642723+0000","last_unstale":"2024-02-02T13:32:56.126309+0000","last_undegraded":"2024-02-02T13:32:56.126309+0000","last_fullsized":"2024-02-02T13:32:56.126309+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:06:18.416354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127920+0000","last_change":"2024-02-02T13:32:55.138581+0000","last_active":"2024-02-02T13:32:56.127920+0000","last_peered":"2024-02-02T13:32:56.127920+0000","last_clean":"2024-02-02T13:32:56.127920+0000","last_became_active":"2024-02-02T13:32:55.137825+0000","last_became_peered":"2024-02-02T13:32:55.137825+0000","last_unstale":"2024-02-02T13:32:56.127920+0000","last_undegraded":"2024-02-02T13:32:56.127920+0000","last_fullsized":"2024-02-02T13:32:56.127920+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:33:05.203908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206630+0000","last_change":"2024-02-02T13:32:54.645982+0000","last_active":"2024-02-02T13:32:56.206630+0000","last_peered":"2024-02-02T13:32:56.206630+0000","last_clean":"2024-02-02T13:32:56.206630+0000","last_became_active":"2024-02-02T13:32:54.645765+0000","last_became_peered":"2024-02-02T13:32:54.645765+0000","last_unstale":"2024-02-02T13:32:56.206630+0000","last_undegraded":"2024-02-02T13:32:56.206630+0000","last_fullsized":"2024-02-02T13:32:56.206630+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:20:30.036447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127036+0000","last_change":"2024-02-02T13:32:55.134699+0000","last_active":"2024-02-02T13:32:56.127036+0000","last_peered":"2024-02-02T13:32:56.127036+0000","last_clean":"2024-02-02T13:32:56.127036+0000","last_became_active":"2024-02-02T13:32:55.133962+0000","last_became_peered":"2024-02-02T13:32:55.133962+0000","last_unstale":"2024-02-02T13:32:56.127036+0000","last_undegraded":"2024-02-02T13:32:56.127036+0000","last_fullsized":"2024-02-02T13:32:56.127036+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:48.260620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123986+0000","last_change":"2024-02-02T13:32:54.644347+0000","last_active":"2024-02-02T13:32:56.123986+0000","last_peered":"2024-02-02T13:32:56.123986+0000","last_clean":"2024-02-02T13:32:56.123986+0000","last_became_active":"2024-02-02T13:32:54.644125+0000","last_became_peered":"2024-02-02T13:32:54.644125+0000","last_unstale":"2024-02-02T13:32:56.123986+0000","last_undegraded":"2024-02-02T13:32:56.123986+0000","last_fullsized":"2024-02-02T13:32:56.123986+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:08:50.462189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.136853+0000","last_change":"2024-02-02T13:32:55.136685+0000","last_active":"2024-02-02T13:32:55.136853+0000","last_peered":"2024-02-02T13:32:55.136853+0000","last_clean":"2024-02-02T13:32:55.136853+0000","last_became_active":"2024-02-02T13:32:55.135933+0000","last_became_peered":"2024-02-02T13:32:55.135933+0000","last_unstale":"2024-02-02T13:32:55.136853+0000","last_undegraded":"2024-02-02T13:32:55.136853+0000","last_fullsized":"2024-02-02T13:32:55.136853+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:53.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123648+0000","last_change":"2024-02-02T13:32:54.646641+0000","last_active":"2024-02-02T13:32:56.123648+0000","last_peered":"2024-02-02T13:32:56.123648+0000","last_clean":"2024-02-02T13:32:56.123648+0000","last_became_active":"2024-02-02T13:32:54.646404+0000","last_became_peered":"2024-02-02T13:32:54.646404+0000","last_unstale":"2024-02-02T13:32:56.123648+0000","last_undegraded":"2024-02-02T13:32:56.123648+0000","last_fullsized":"2024-02-02T13:32:56.123648+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:24.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.151249+0000","last_change":"2024-02-02T13:32:55.151143+0000","last_active":"2024-02-02T13:32:55.151249+0000","last_peered":"2024-02-02T13:32:55.151249+0000","last_clean":"2024-02-02T13:32:55.151249+0000","last_became_active":"2024-02-02T13:32:55.150868+0000","last_became_peered":"2024-02-02T13:32:55.150868+0000","last_unstale":"2024-02-02T13:32:55.151249+0000","last_undegraded":"2024-02-02T13:32:55.151249+0000","last_fullsized":"2024-02-02T13:32:55.151249+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:11:26.076536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.148844+0000","last_change":"2024-02-02T13:32:54.646008+0000","last_active":"2024-02-02T13:32:55.148844+0000","last_peered":"2024-02-02T13:32:55.148844+0000","last_clean":"2024-02-02T13:32:55.148844+0000","last_became_active":"2024-02-02T13:32:54.644468+0000","last_became_peered":"2024-02-02T13:32:54.644468+0000","last_unstale":"2024-02-02T13:32:55.148844+0000","last_undegraded":"2024-02-02T13:32:55.148844+0000","last_fullsized":"2024-02-02T13:32:55.148844+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:11:25.059474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209525+0000","last_change":"2024-02-02T13:32:55.156225+0000","last_active":"2024-02-02T13:32:56.209525+0000","last_peered":"2024-02-02T13:32:56.209525+0000","last_clean":"2024-02-02T13:32:56.209525+0000","last_became_active":"2024-02-02T13:32:55.154599+0000","last_became_peered":"2024-02-02T13:32:55.154599+0000","last_unstale":"2024-02-02T13:32:56.209525+0000","last_undegraded":"2024-02-02T13:32:56.209525+0000","last_fullsized":"2024-02-02T13:32:56.209525+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:43.943303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.132167+0000","last_change":"2024-02-02T13:32:54.649908+0000","last_active":"2024-02-02T13:32:55.132167+0000","last_peered":"2024-02-02T13:32:55.132167+0000","last_clean":"2024-02-02T13:32:55.132167+0000","last_became_active":"2024-02-02T13:32:54.649462+0000","last_became_peered":"2024-02-02T13:32:54.649462+0000","last_unstale":"2024-02-02T13:32:55.132167+0000","last_undegraded":"2024-02-02T13:32:55.132167+0000","last_fullsized":"2024-02-02T13:32:55.132167+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:01:32.229854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123747+0000","last_change":"2024-02-02T13:32:55.132217+0000","last_active":"2024-02-02T13:32:56.123747+0000","last_peered":"2024-02-02T13:32:56.123747+0000","last_clean":"2024-02-02T13:32:56.123747+0000","last_became_active":"2024-02-02T13:32:55.131598+0000","last_became_peered":"2024-02-02T13:32:55.131598+0000","last_unstale":"2024-02-02T13:32:56.123747+0000","last_undegraded":"2024-02-02T13:32:56.123747+0000","last_fullsized":"2024-02-02T13:32:56.123747+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:19:59.978075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124701+0000","last_change":"2024-02-02T13:32:54.647904+0000","last_active":"2024-02-02T13:32:56.124701+0000","last_peered":"2024-02-02T13:32:56.124701+0000","last_clean":"2024-02-02T13:32:56.124701+0000","last_became_active":"2024-02-02T13:32:54.647617+0000","last_became_peered":"2024-02-02T13:32:54.647617+0000","last_unstale":"2024-02-02T13:32:56.124701+0000","last_undegraded":"2024-02-02T13:32:56.124701+0000","last_fullsized":"2024-02-02T13:32:56.124701+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:38:20.114336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.149048+0000","last_change":"2024-02-02T13:32:55.149000+0000","last_active":"2024-02-02T13:32:55.149048+0000","last_peered":"2024-02-02T13:32:55.149048+0000","last_clean":"2024-02-02T13:32:55.149048+0000","last_became_active":"2024-02-02T13:32:55.148740+0000","last_became_peered":"2024-02-02T13:32:55.148740+0000","last_unstale":"2024-02-02T13:32:55.149048+0000","last_undegraded":"2024-02-02T13:32:55.149048+0000","last_fullsized":"2024-02-02T13:32:55.149048+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:57:28.874057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128536+0000","last_change":"2024-02-02T13:32:54.636230+0000","last_active":"2024-02-02T13:32:56.128536+0000","last_peered":"2024-02-02T13:32:56.128536+0000","last_clean":"2024-02-02T13:32:56.128536+0000","last_became_active":"2024-02-02T13:32:54.635684+0000","last_became_peered":"2024-02-02T13:32:54.635684+0000","last_unstale":"2024-02-02T13:32:56.128536+0000","last_undegraded":"2024-02-02T13:32:56.128536+0000","last_fullsized":"2024-02-02T13:32:56.128536+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:34.398958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127906+0000","last_change":"2024-02-02T13:32:55.142411+0000","last_active":"2024-02-02T13:32:56.127906+0000","last_peered":"2024-02-02T13:32:56.127906+0000","last_clean":"2024-02-02T13:32:56.127906+0000","last_became_active":"2024-02-02T13:32:55.134978+0000","last_became_peered":"2024-02-02T13:32:55.134978+0000","last_unstale":"2024-02-02T13:32:56.127906+0000","last_undegraded":"2024-02-02T13:32:56.127906+0000","last_fullsized":"2024-02-02T13:32:56.127906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:18.561892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121623+0000","last_change":"2024-02-02T13:32:54.645555+0000","last_active":"2024-02-02T13:32:55.121623+0000","last_peered":"2024-02-02T13:32:55.121623+0000","last_clean":"2024-02-02T13:32:55.121623+0000","last_became_active":"2024-02-02T13:32:54.644068+0000","last_became_peered":"2024-02-02T13:32:54.644068+0000","last_unstale":"2024-02-02T13:32:55.121623+0000","last_undegraded":"2024-02-02T13:32:55.121623+0000","last_fullsized":"2024-02-02T13:32:55.121623+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:15:36.386928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125703+0000","last_change":"2024-02-02T13:32:55.134684+0000","last_active":"2024-02-02T13:32:56.125703+0000","last_peered":"2024-02-02T13:32:56.125703+0000","last_clean":"2024-02-02T13:32:56.125703+0000","last_became_active":"2024-02-02T13:32:55.133986+0000","last_became_peered":"2024-02-02T13:32:55.133986+0000","last_unstale":"2024-02-02T13:32:56.125703+0000","last_undegraded":"2024-02-02T13:32:56.125703+0000","last_fullsized":"2024-02-02T13:32:56.125703+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:37.017486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"160'3","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127269+0000","last_change":"2024-02-02T13:32:54.642706+0000","last_active":"2024-02-02T13:32:56.127269+0000","last_peered":"2024-02-02T13:32:56.127269+0000","last_clean":"2024-02-02T13:32:56.127269+0000","last_became_active":"2024-02-02T13:32:54.634322+0000","last_became_peered":"2024-02-02T13:32:54.634322+0000","last_unstale":"2024-02-02T13:32:56.127269+0000","last_undegraded":"2024-02-02T13:32:56.127269+0000","last_fullsized":"2024-02-02T13:32:56.127269+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:24:04.062561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125424+0000","last_change":"2024-02-02T13:32:55.153402+0000","last_active":"2024-02-02T13:32:56.125424+0000","last_peered":"2024-02-02T13:32:56.125424+0000","last_clean":"2024-02-02T13:32:56.125424+0000","last_became_active":"2024-02-02T13:32:55.152737+0000","last_became_peered":"2024-02-02T13:32:55.152737+0000","last_unstale":"2024-02-02T13:32:56.125424+0000","last_undegraded":"2024-02-02T13:32:56.125424+0000","last_fullsized":"2024-02-02T13:32:56.125424+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:32:24.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"160'6","reported_seq":17,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209117+0000","last_change":"2024-02-02T13:32:54.650801+0000","last_active":"2024-02-02T13:32:56.209117+0000","last_peered":"2024-02-02T13:32:56.209117+0000","last_clean":"2024-02-02T13:32:56.209117+0000","last_became_active":"2024-02-02T13:32:54.650540+0000","last_became_peered":"2024-02-02T13:32:54.650540+0000","last_unstale":"2024-02-02T13:32:56.209117+0000","last_undegraded":"2024-02-02T13:32:56.209117+0000","last_fullsized":"2024-02-02T13:32:56.209117+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:20:07.268407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.143761+0000","last_change":"2024-02-02T13:32:55.143327+0000","last_active":"2024-02-02T13:32:55.143761+0000","last_peered":"2024-02-02T13:32:55.143761+0000","last_clean":"2024-02-02T13:32:55.143761+0000","last_became_active":"2024-02-02T13:32:55.143189+0000","last_became_peered":"2024-02-02T13:32:55.143189+0000","last_unstale":"2024-02-02T13:32:55.143761+0000","last_undegraded":"2024-02-02T13:32:55.143761+0000","last_fullsized":"2024-02-02T13:32:55.143761+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:22:36.659330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.140551+0000","last_change":"2024-02-02T13:32:54.647125+0000","last_active":"2024-02-02T13:32:55.140551+0000","last_peered":"2024-02-02T13:32:55.140551+0000","last_clean":"2024-02-02T13:32:55.140551+0000","last_became_active":"2024-02-02T13:32:54.646820+0000","last_became_peered":"2024-02-02T13:32:54.646820+0000","last_unstale":"2024-02-02T13:32:55.140551+0000","last_undegraded":"2024-02-02T13:32:55.140551+0000","last_fullsized":"2024-02-02T13:32:55.140551+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:24:59.236773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123546+0000","last_change":"2024-02-02T13:32:55.154103+0000","last_active":"2024-02-02T13:32:56.123546+0000","last_peered":"2024-02-02T13:32:56.123546+0000","last_clean":"2024-02-02T13:32:56.123546+0000","last_became_active":"2024-02-02T13:32:55.153861+0000","last_became_peered":"2024-02-02T13:32:55.153861+0000","last_unstale":"2024-02-02T13:32:56.123546+0000","last_undegraded":"2024-02-02T13:32:56.123546+0000","last_fullsized":"2024-02-02T13:32:56.123546+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:16:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123739+0000","last_change":"2024-02-02T13:32:54.651216+0000","last_active":"2024-02-02T13:32:56.123739+0000","last_peered":"2024-02-02T13:32:56.123739+0000","last_clean":"2024-02-02T13:32:56.123739+0000","last_became_active":"2024-02-02T13:32:54.650939+0000","last_became_peered":"2024-02-02T13:32:54.650939+0000","last_unstale":"2024-02-02T13:32:56.123739+0000","last_undegraded":"2024-02-02T13:32:56.123739+0000","last_fullsized":"2024-02-02T13:32:56.123739+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:26.473245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209411+0000","last_change":"2024-02-02T13:32:55.156265+0000","last_active":"2024-02-02T13:32:56.209411+0000","last_peered":"2024-02-02T13:32:56.209411+0000","last_clean":"2024-02-02T13:32:56.209411+0000","last_became_active":"2024-02-02T13:32:55.154309+0000","last_became_peered":"2024-02-02T13:32:55.154309+0000","last_unstale":"2024-02-02T13:32:56.209411+0000","last_undegraded":"2024-02-02T13:32:56.209411+0000","last_fullsized":"2024-02-02T13:32:56.209411+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:55:40.229185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"160'8","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.210307+0000","last_change":"2024-02-02T13:32:54.645174+0000","last_active":"2024-02-02T13:32:56.210307+0000","last_peered":"2024-02-02T13:32:56.210307+0000","last_clean":"2024-02-02T13:32:56.210307+0000","last_became_active":"2024-02-02T13:32:54.642994+0000","last_became_peered":"2024-02-02T13:32:54.642994+0000","last_unstale":"2024-02-02T13:32:56.210307+0000","last_undegraded":"2024-02-02T13:32:56.210307+0000","last_fullsized":"2024-02-02T13:32:56.210307+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:39:00.577504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.209705+0000","last_change":"2024-02-02T13:32:55.155071+0000","last_active":"2024-02-02T13:32:56.209705+0000","last_peered":"2024-02-02T13:32:56.209705+0000","last_clean":"2024-02-02T13:32:56.209705+0000","last_became_active":"2024-02-02T13:32:55.151417+0000","last_became_peered":"2024-02-02T13:32:55.151417+0000","last_unstale":"2024-02-02T13:32:56.209705+0000","last_undegraded":"2024-02-02T13:32:56.209705+0000","last_fullsized":"2024-02-02T13:32:56.209705+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:52:40.527926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"160'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126579+0000","last_change":"2024-02-02T13:32:54.650068+0000","last_active":"2024-02-02T13:32:56.126579+0000","last_peered":"2024-02-02T13:32:56.126579+0000","last_clean":"2024-02-02T13:32:56.126579+0000","last_became_active":"2024-02-02T13:32:54.649800+0000","last_became_peered":"2024-02-02T13:32:54.649800+0000","last_unstale":"2024-02-02T13:32:56.126579+0000","last_undegraded":"2024-02-02T13:32:56.126579+0000","last_fullsized":"2024-02-02T13:32:56.126579+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:39.140143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126048+0000","last_change":"2024-02-02T13:32:55.140617+0000","last_active":"2024-02-02T13:32:56.126048+0000","last_peered":"2024-02-02T13:32:56.126048+0000","last_clean":"2024-02-02T13:32:56.126048+0000","last_became_active":"2024-02-02T13:32:55.140189+0000","last_became_peered":"2024-02-02T13:32:55.140189+0000","last_unstale":"2024-02-02T13:32:56.126048+0000","last_undegraded":"2024-02-02T13:32:56.126048+0000","last_fullsized":"2024-02-02T13:32:56.126048+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:08:24.159578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206714+0000","last_change":"2024-02-02T13:32:54.648911+0000","last_active":"2024-02-02T13:32:56.206714+0000","last_peered":"2024-02-02T13:32:56.206714+0000","last_clean":"2024-02-02T13:32:56.206714+0000","last_became_active":"2024-02-02T13:32:54.648684+0000","last_became_peered":"2024-02-02T13:32:54.648684+0000","last_unstale":"2024-02-02T13:32:56.206714+0000","last_undegraded":"2024-02-02T13:32:56.206714+0000","last_fullsized":"2024-02-02T13:32:56.206714+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:46:32.135281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.133352+0000","last_change":"2024-02-02T13:32:55.133104+0000","last_active":"2024-02-02T13:32:55.133352+0000","last_peered":"2024-02-02T13:32:55.133352+0000","last_clean":"2024-02-02T13:32:55.133352+0000","last_became_active":"2024-02-02T13:32:55.132887+0000","last_became_peered":"2024-02-02T13:32:55.132887+0000","last_unstale":"2024-02-02T13:32:55.133352+0000","last_undegraded":"2024-02-02T13:32:55.133352+0000","last_fullsized":"2024-02-02T13:32:55.133352+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:56:24.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.150094+0000","last_change":"2024-02-02T13:32:54.648682+0000","last_active":"2024-02-02T13:32:55.150094+0000","last_peered":"2024-02-02T13:32:55.150094+0000","last_clean":"2024-02-02T13:32:55.150094+0000","last_became_active":"2024-02-02T13:32:54.647044+0000","last_became_peered":"2024-02-02T13:32:54.647044+0000","last_unstale":"2024-02-02T13:32:55.150094+0000","last_undegraded":"2024-02-02T13:32:55.150094+0000","last_fullsized":"2024-02-02T13:32:55.150094+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:32:19.129795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124176+0000","last_change":"2024-02-02T13:32:55.131494+0000","last_active":"2024-02-02T13:32:56.124176+0000","last_peered":"2024-02-02T13:32:56.124176+0000","last_clean":"2024-02-02T13:32:56.124176+0000","last_became_active":"2024-02-02T13:32:55.131282+0000","last_became_peered":"2024-02-02T13:32:55.131282+0000","last_unstale":"2024-02-02T13:32:56.124176+0000","last_undegraded":"2024-02-02T13:32:56.124176+0000","last_fullsized":"2024-02-02T13:32:56.124176+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:20.579158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127347+0000","last_change":"2024-02-02T13:32:54.644664+0000","last_active":"2024-02-02T13:32:56.127347+0000","last_peered":"2024-02-02T13:32:56.127347+0000","last_clean":"2024-02-02T13:32:56.127347+0000","last_became_active":"2024-02-02T13:32:54.644244+0000","last_became_peered":"2024-02-02T13:32:54.644244+0000","last_unstale":"2024-02-02T13:32:56.127347+0000","last_undegraded":"2024-02-02T13:32:56.127347+0000","last_fullsized":"2024-02-02T13:32:56.127347+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:19:58.961014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146528+0000","last_change":"2024-02-02T13:32:55.146375+0000","last_active":"2024-02-02T13:32:55.146528+0000","last_peered":"2024-02-02T13:32:55.146528+0000","last_clean":"2024-02-02T13:32:55.146528+0000","last_became_active":"2024-02-02T13:32:55.146189+0000","last_became_peered":"2024-02-02T13:32:55.146189+0000","last_unstale":"2024-02-02T13:32:55.146528+0000","last_undegraded":"2024-02-02T13:32:55.146528+0000","last_fullsized":"2024-02-02T13:32:55.146528+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:51:44.334699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127502+0000","last_change":"2024-02-02T13:32:54.643327+0000","last_active":"2024-02-02T13:32:56.127502+0000","last_peered":"2024-02-02T13:32:56.127502+0000","last_clean":"2024-02-02T13:32:56.127502+0000","last_became_active":"2024-02-02T13:32:54.635222+0000","last_became_peered":"2024-02-02T13:32:54.635222+0000","last_unstale":"2024-02-02T13:32:56.127502+0000","last_undegraded":"2024-02-02T13:32:56.127502+0000","last_fullsized":"2024-02-02T13:32:56.127502+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:17:54.473266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.154752+0000","last_change":"2024-02-02T13:32:55.154569+0000","last_active":"2024-02-02T13:32:55.154752+0000","last_peered":"2024-02-02T13:32:55.154752+0000","last_clean":"2024-02-02T13:32:55.154752+0000","last_became_active":"2024-02-02T13:32:55.154324+0000","last_became_peered":"2024-02-02T13:32:55.154324+0000","last_unstale":"2024-02-02T13:32:55.154752+0000","last_undegraded":"2024-02-02T13:32:55.154752+0000","last_fullsized":"2024-02-02T13:32:55.154752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:57.056137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121613+0000","last_change":"2024-02-02T13:32:54.644902+0000","last_active":"2024-02-02T13:32:55.121613+0000","last_peered":"2024-02-02T13:32:55.121613+0000","last_clean":"2024-02-02T13:32:55.121613+0000","last_became_active":"2024-02-02T13:32:54.644298+0000","last_became_peered":"2024-02-02T13:32:54.644298+0000","last_unstale":"2024-02-02T13:32:55.121613+0000","last_undegraded":"2024-02-02T13:32:55.121613+0000","last_fullsized":"2024-02-02T13:32:55.121613+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:37:56.039076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123819+0000","last_change":"2024-02-02T13:32:55.144909+0000","last_active":"2024-02-02T13:32:56.123819+0000","last_peered":"2024-02-02T13:32:56.123819+0000","last_clean":"2024-02-02T13:32:56.123819+0000","last_became_active":"2024-02-02T13:32:55.144616+0000","last_became_peered":"2024-02-02T13:32:55.144616+0000","last_unstale":"2024-02-02T13:32:56.123819+0000","last_undegraded":"2024-02-02T13:32:56.123819+0000","last_fullsized":"2024-02-02T13:32:56.123819+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:56:40.157204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.122965+0000","last_change":"2024-02-02T13:32:54.633822+0000","last_active":"2024-02-02T13:32:55.122965+0000","last_peered":"2024-02-02T13:32:55.122965+0000","last_clean":"2024-02-02T13:32:55.122965+0000","last_became_active":"2024-02-02T13:32:54.632989+0000","last_became_peered":"2024-02-02T13:32:54.632989+0000","last_unstale":"2024-02-02T13:32:55.122965+0000","last_undegraded":"2024-02-02T13:32:55.122965+0000","last_fullsized":"2024-02-02T13:32:55.122965+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:52.750896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126647+0000","last_change":"2024-02-02T13:32:55.141594+0000","last_active":"2024-02-02T13:32:56.126647+0000","last_peered":"2024-02-02T13:32:56.126647+0000","last_clean":"2024-02-02T13:32:56.126647+0000","last_became_active":"2024-02-02T13:32:55.141104+0000","last_became_peered":"2024-02-02T13:32:55.141104+0000","last_unstale":"2024-02-02T13:32:56.126647+0000","last_undegraded":"2024-02-02T13:32:56.126647+0000","last_fullsized":"2024-02-02T13:32:56.126647+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:41:30.707307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206857+0000","last_change":"2024-02-02T13:32:54.648728+0000","last_active":"2024-02-02T13:32:56.206857+0000","last_peered":"2024-02-02T13:32:56.206857+0000","last_clean":"2024-02-02T13:32:56.206857+0000","last_became_active":"2024-02-02T13:32:54.647937+0000","last_became_peered":"2024-02-02T13:32:54.647937+0000","last_unstale":"2024-02-02T13:32:56.206857+0000","last_undegraded":"2024-02-02T13:32:56.206857+0000","last_fullsized":"2024-02-02T13:32:56.206857+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:47.243559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145209+0000","last_change":"2024-02-02T13:32:55.143132+0000","last_active":"2024-02-02T13:32:55.145209+0000","last_peered":"2024-02-02T13:32:55.145209+0000","last_clean":"2024-02-02T13:32:55.145209+0000","last_became_active":"2024-02-02T13:32:55.142066+0000","last_became_peered":"2024-02-02T13:32:55.142066+0000","last_unstale":"2024-02-02T13:32:55.145209+0000","last_undegraded":"2024-02-02T13:32:55.145209+0000","last_fullsized":"2024-02-02T13:32:55.145209+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:53.767957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.146636+0000","last_change":"2024-02-02T13:32:54.644159+0000","last_active":"2024-02-02T13:32:55.146636+0000","last_peered":"2024-02-02T13:32:55.146636+0000","last_clean":"2024-02-02T13:32:55.146636+0000","last_became_active":"2024-02-02T13:32:54.643873+0000","last_became_peered":"2024-02-02T13:32:54.643873+0000","last_unstale":"2024-02-02T13:32:55.146636+0000","last_undegraded":"2024-02-02T13:32:55.146636+0000","last_fullsized":"2024-02-02T13:32:55.146636+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:52.729134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152347+0000","last_change":"2024-02-02T13:32:55.151801+0000","last_active":"2024-02-02T13:32:55.152347+0000","last_peered":"2024-02-02T13:32:55.152347+0000","last_clean":"2024-02-02T13:32:55.152347+0000","last_became_active":"2024-02-02T13:32:55.151592+0000","last_became_peered":"2024-02-02T13:32:55.151592+0000","last_unstale":"2024-02-02T13:32:55.152347+0000","last_undegraded":"2024-02-02T13:32:55.152347+0000","last_fullsized":"2024-02-02T13:32:55.152347+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:57:51.157364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125918+0000","last_change":"2024-02-02T13:32:54.652191+0000","last_active":"2024-02-02T13:32:56.125918+0000","last_peered":"2024-02-02T13:32:56.125918+0000","last_clean":"2024-02-02T13:32:56.125918+0000","last_became_active":"2024-02-02T13:32:54.651906+0000","last_became_peered":"2024-02-02T13:32:54.651906+0000","last_unstale":"2024-02-02T13:32:56.125918+0000","last_undegraded":"2024-02-02T13:32:56.125918+0000","last_fullsized":"2024-02-02T13:32:56.125918+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:34.143891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124623+0000","last_change":"2024-02-02T13:32:55.139935+0000","last_active":"2024-02-02T13:32:56.124623+0000","last_peered":"2024-02-02T13:32:56.124623+0000","last_clean":"2024-02-02T13:32:56.124623+0000","last_became_active":"2024-02-02T13:32:55.138936+0000","last_became_peered":"2024-02-02T13:32:55.138936+0000","last_unstale":"2024-02-02T13:32:56.124623+0000","last_undegraded":"2024-02-02T13:32:56.124623+0000","last_fullsized":"2024-02-02T13:32:56.124623+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:37:17.483117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123867+0000","last_change":"2024-02-02T13:32:54.649257+0000","last_active":"2024-02-02T13:32:56.123867+0000","last_peered":"2024-02-02T13:32:56.123867+0000","last_clean":"2024-02-02T13:32:56.123867+0000","last_became_active":"2024-02-02T13:32:54.649024+0000","last_became_peered":"2024-02-02T13:32:54.649024+0000","last_unstale":"2024-02-02T13:32:56.123867+0000","last_undegraded":"2024-02-02T13:32:56.123867+0000","last_fullsized":"2024-02-02T13:32:56.123867+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:07:42.362754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.145500+0000","last_change":"2024-02-02T13:32:55.145367+0000","last_active":"2024-02-02T13:32:55.145500+0000","last_peered":"2024-02-02T13:32:55.145500+0000","last_clean":"2024-02-02T13:32:55.145500+0000","last_became_active":"2024-02-02T13:32:55.144753+0000","last_became_peered":"2024-02-02T13:32:55.144753+0000","last_unstale":"2024-02-02T13:32:55.145500+0000","last_undegraded":"2024-02-02T13:32:55.145500+0000","last_fullsized":"2024-02-02T13:32:55.145500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:37.667814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206812+0000","last_change":"2024-02-02T13:32:54.645010+0000","last_active":"2024-02-02T13:32:56.206812+0000","last_peered":"2024-02-02T13:32:56.206812+0000","last_clean":"2024-02-02T13:32:56.206812+0000","last_became_active":"2024-02-02T13:32:54.644796+0000","last_became_peered":"2024-02-02T13:32:54.644796+0000","last_unstale":"2024-02-02T13:32:56.206812+0000","last_undegraded":"2024-02-02T13:32:56.206812+0000","last_fullsized":"2024-02-02T13:32:56.206812+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:31:58.955942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.125338+0000","last_change":"2024-02-02T13:32:55.142394+0000","last_active":"2024-02-02T13:32:56.125338+0000","last_peered":"2024-02-02T13:32:56.125338+0000","last_clean":"2024-02-02T13:32:56.125338+0000","last_became_active":"2024-02-02T13:32:55.142115+0000","last_became_peered":"2024-02-02T13:32:55.142115+0000","last_unstale":"2024-02-02T13:32:56.125338+0000","last_undegraded":"2024-02-02T13:32:56.125338+0000","last_fullsized":"2024-02-02T13:32:56.125338+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:56.282353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.124329+0000","last_change":"2024-02-02T13:32:54.651676+0000","last_active":"2024-02-02T13:32:56.124329+0000","last_peered":"2024-02-02T13:32:56.124329+0000","last_clean":"2024-02-02T13:32:56.124329+0000","last_became_active":"2024-02-02T13:32:54.651371+0000","last_became_peered":"2024-02-02T13:32:54.651371+0000","last_unstale":"2024-02-02T13:32:56.124329+0000","last_undegraded":"2024-02-02T13:32:56.124329+0000","last_fullsized":"2024-02-02T13:32:56.124329+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:51:43.317637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128151+0000","last_change":"2024-02-02T13:32:55.147216+0000","last_active":"2024-02-02T13:32:56.128151+0000","last_peered":"2024-02-02T13:32:56.128151+0000","last_clean":"2024-02-02T13:32:56.128151+0000","last_became_active":"2024-02-02T13:32:55.146917+0000","last_became_peered":"2024-02-02T13:32:55.146917+0000","last_unstale":"2024-02-02T13:32:56.128151+0000","last_undegraded":"2024-02-02T13:32:56.128151+0000","last_fullsized":"2024-02-02T13:32:56.128151+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:27:28.715360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.126027+0000","last_change":"2024-02-02T13:32:54.651755+0000","last_active":"2024-02-02T13:32:56.126027+0000","last_peered":"2024-02-02T13:32:56.126027+0000","last_clean":"2024-02-02T13:32:56.126027+0000","last_became_active":"2024-02-02T13:32:54.651473+0000","last_became_peered":"2024-02-02T13:32:54.651473+0000","last_unstale":"2024-02-02T13:32:56.126027+0000","last_undegraded":"2024-02-02T13:32:56.126027+0000","last_fullsized":"2024-02-02T13:32:56.126027+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:41:48.597809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.152684+0000","last_change":"2024-02-02T13:32:55.152464+0000","last_active":"2024-02-02T13:32:55.152684+0000","last_peered":"2024-02-02T13:32:55.152684+0000","last_clean":"2024-02-02T13:32:55.152684+0000","last_became_active":"2024-02-02T13:32:55.152250+0000","last_became_peered":"2024-02-02T13:32:55.152250+0000","last_unstale":"2024-02-02T13:32:55.152684+0000","last_undegraded":"2024-02-02T13:32:55.152684+0000","last_fullsized":"2024-02-02T13:32:55.152684+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:16.532202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.121774+0000","last_change":"2024-02-02T13:32:54.649669+0000","last_active":"2024-02-02T13:32:55.121774+0000","last_peered":"2024-02-02T13:32:55.121774+0000","last_clean":"2024-02-02T13:32:55.121774+0000","last_became_active":"2024-02-02T13:32:54.649330+0000","last_became_peered":"2024-02-02T13:32:54.649330+0000","last_unstale":"2024-02-02T13:32:55.121774+0000","last_undegraded":"2024-02-02T13:32:55.121774+0000","last_fullsized":"2024-02-02T13:32:55.121774+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:19:16.549558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.123738+0000","last_change":"2024-02-02T13:32:55.146077+0000","last_active":"2024-02-02T13:32:56.123738+0000","last_peered":"2024-02-02T13:32:56.123738+0000","last_clean":"2024-02-02T13:32:56.123738+0000","last_became_active":"2024-02-02T13:32:55.145750+0000","last_became_peered":"2024-02-02T13:32:55.145750+0000","last_unstale":"2024-02-02T13:32:56.123738+0000","last_undegraded":"2024-02-02T13:32:56.123738+0000","last_fullsized":"2024-02-02T13:32:56.123738+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:24:05.079623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.206740+0000","last_change":"2024-02-02T13:32:54.647837+0000","last_active":"2024-02-02T13:32:56.206740+0000","last_peered":"2024-02-02T13:32:56.206740+0000","last_clean":"2024-02-02T13:32:56.206740+0000","last_became_active":"2024-02-02T13:32:54.647623+0000","last_became_peered":"2024-02-02T13:32:54.647623+0000","last_unstale":"2024-02-02T13:32:56.206740+0000","last_undegraded":"2024-02-02T13:32:56.206740+0000","last_fullsized":"2024-02-02T13:32:56.206740+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:45:36.000425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.128145+0000","last_change":"2024-02-02T13:32:55.145926+0000","last_active":"2024-02-02T13:32:56.128145+0000","last_peered":"2024-02-02T13:32:56.128145+0000","last_clean":"2024-02-02T13:32:56.128145+0000","last_became_active":"2024-02-02T13:32:55.144036+0000","last_became_peered":"2024-02-02T13:32:55.144036+0000","last_unstale":"2024-02-02T13:32:56.128145+0000","last_undegraded":"2024-02-02T13:32:56.128145+0000","last_fullsized":"2024-02-02T13:32:56.128145+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:54.611102+0000","last_clean_scrub_stamp":"2024-02-02T13:32:54.611102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:02.516881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"160'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-02T13:32:56.127219+0000","last_change":"2024-02-02T13:32:54.651920+0000","last_active":"2024-02-02T13:32:56.127219+0000","last_peered":"2024-02-02T13:32:56.127219+0000","last_clean":"2024-02-02T13:32:56.127219+0000","last_became_active":"2024-02-02T13:32:54.651571+0000","last_became_peered":"2024-02-02T13:32:54.651571+0000","last_unstale":"2024-02-02T13:32:56.127219+0000","last_undegraded":"2024-02-02T13:32:56.127219+0000","last_fullsized":"2024-02-02T13:32:56.127219+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:32:53.594040+0000","last_clean_scrub_stamp":"2024-02-02T13:32:53.594040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:55:39.212123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":303,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-02-02T13:32:55.127081+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:32:55.127081+0000","last_peered":"2024-02-02T13:32:55.127081+0000","last_clean":"2024-02-02T13:32:55.127081+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:32:55.127081+0000","last_undegraded":"2024-02-02T13:32:55.127081+0000","last_fullsized":"2024-02-02T13:32:55.127081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1450,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193862,"statfs":{"total":200000000,"available":198515143,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.087,"15min":1.0680000000000001},"min":{"1min":0.72899999999999998,"5min":0.66600000000000004,"15min":0.61499999999999999},"max":{"1min":1.514,"5min":1.7390000000000001,"15min":2.5310000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.071,"15min":1.113},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3120000000000001,"5min":1.6459999999999999,"15min":4.1609999999999996},"last":0.94699999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1140000000000001,"15min":1.093},"min":{"1min":0.746,"5min":0.68999999999999995,"15min":0.66300000000000003},"max":{"1min":1.506,"5min":1.8839999999999999,"15min":1.893},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1919999999999999,"15min":1.133},"min":{"1min":0.80200000000000005,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.54,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.2050000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.266,"15min":1.228},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70099999999999996},"max":{"1min":1.5109999999999999,"5min":1.909,"15min":2.6669999999999998},"last":1.304},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1879999999999999,"15min":1.1830000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.623},"max":{"1min":1.583,"5min":1.7949999999999999,"15min":3.5819999999999999},"last":1.075}]},{"osd":3,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.179,"15min":1.177},"min":{"1min":0.76400000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5449999999999999,"5min":1.6339999999999999,"15min":1.819},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.3480000000000001,"15min":1.3169999999999999},"min":{"1min":0.94299999999999995,"5min":0.752,"15min":0.48999999999999999},"max":{"1min":1.738,"5min":2.133,"15min":2.4620000000000002},"last":1.399}]},{"osd":4,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.292,"15min":1.216},"min":{"1min":0.72899999999999998,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.8169999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3160000000000001,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.61199999999999999},"max":{"1min":1.5609999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.98499999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.397,"15min":1.3620000000000001},"min":{"1min":1.2010000000000001,"5min":0.80200000000000005,"15min":0.54000000000000004},"max":{"1min":1.641,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.403,"15min":1.4390000000000001},"min":{"1min":0.91800000000000004,"5min":0.76000000000000001,"15min":0.56200000000000006},"max":{"1min":1.772,"5min":2.0750000000000002,"15min":3.7949999999999999},"last":1.4399999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:48 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.4910000000000001,"15min":1.526},"min":{"1min":1.085,"5min":0.88700000000000001,"15min":0.64200000000000002},"max":{"1min":1.8979999999999999,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4470000000000001,"15min":1.4239999999999999},"min":{"1min":1.1279999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7809999999999999,"5min":1.821,"15min":2.0179999999999998},"last":1.1539999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107168,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0960000000000001,"15min":1.036},"min":{"1min":0.63200000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.4910000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.379},{"interface":"front","average":{"1min":1.034,"5min":1.099,"15min":1.0309999999999999},"min":{"1min":0.68799999999999994,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.462,"5min":1.768,"15min":1.768},"last":1.02}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0309999999999999},"min":{"1min":0.68799999999999994,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.421,"5min":1.5229999999999999,"15min":1.556},"last":1.466},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1619999999999999,"15min":1.0940000000000001},"min":{"1min":0.66000000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5369999999999999,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.333}]},{"osd":2,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1799999999999999,"15min":1.1000000000000001},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5820000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.151},{"interface":"front","average":{"1min":1.113,"5min":1.1739999999999999,"15min":1.1579999999999999},"min":{"1min":0.80400000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.5760000000000001,"5min":1.849,"15min":1.849},"last":1.121}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.254},"min":{"1min":0.82199999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5549999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1839999999999999,"15min":1.147},"min":{"1min":0.84899999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3400000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.5580000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4359999999999999,"15min":1.2849999999999999},"min":{"1min":0.71399999999999997,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.728,"5min":17.686,"15min":17.686},"last":1.244},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.5700000000000001,"15min":1.452},"min":{"1min":0.76000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.661,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.401}]},{"osd":5,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.2729999999999999},"min":{"1min":0.56399999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6950000000000001,"5min":1.986,"15min":1.986},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.415,"15min":1.369},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6259999999999999,"5min":3.331,"15min":3.331},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4590000000000001,"15min":1.4550000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":1.7230000000000001,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.196},{"interface":"front","average":{"1min":1.353,"5min":1.4370000000000001,"15min":1.4339999999999999},"min":{"1min":0.80200000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8939999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.617}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194441,"statfs":{"total":200000000,"available":199107601,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.64400000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.28,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.054,"15min":1.0449999999999999},"min":{"1min":0.68700000000000006,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4279999999999999,"5min":1.554,"15min":1.7150000000000001},"last":1.0389999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1439999999999999,"15min":1.2310000000000001},"min":{"1min":0.68999999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.585,"5min":1.6830000000000001,"15min":4.5899999999999999},"last":1.151},{"interface":"front","average":{"1min":1.107,"5min":1.0980000000000001,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.4159999999999999,"5min":1.877,"15min":4.4500000000000002},"last":1.196}]},{"osd":2,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.302,"15min":1.236},"min":{"1min":0.85199999999999998,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.7210000000000001,"5min":1.8899999999999999,"15min":3.0339999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.76000000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.641,"5min":1.706,"15min":3.0720000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.82599999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5009999999999999,"5min":1.591,"15min":1.7769999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2569999999999999,"15min":1.196},"min":{"1min":0.94699999999999995,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":5.4610000000000003},"last":0.89900000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2010000000000001,"15min":1.216},"min":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.64000000000000001},"max":{"1min":1.675,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3049999999999999,"15min":1.284},"min":{"1min":0.89300000000000002,"5min":0.622,"15min":0.55100000000000005},"max":{"1min":1.748,"5min":1.8029999999999999,"15min":1.833},"last":1.4419999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.357,"15min":1.3140000000000001},"min":{"1min":0.98899999999999999,"5min":0.88600000000000001,"15min":0.64700000000000002},"max":{"1min":1.78,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.359,"15min":1.3480000000000001},"min":{"1min":1.0369999999999999,"5min":0.82599999999999996,"15min":0.59099999999999997},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8640000000000001},"last":1.538}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.423,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":1.012,"15min":0.66500000000000004},"max":{"1min":1.841,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.651},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4810000000000001,"15min":1.3979999999999999},"min":{"1min":1.083,"5min":1.036,"15min":0.61599999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.5920000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":870,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194442,"statfs":{"total":200000000,"available":199108662,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0589999999999999,"15min":1.004},"min":{"1min":0.71499999999999997,"5min":0.60099999999999998,"15min":0.57999999999999996},"max":{"1min":1.5309999999999999,"5min":1.6439999999999999,"15min":4.5949999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.099,"15min":1.0820000000000001},"min":{"1min":0.73799999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7609999999999999,"5min":1.839,"15min":1.839},"last":1.149}]},{"osd":1,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.069,"15min":1.0069999999999999},"min":{"1min":0.75700000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.379,"5min":2.1659999999999999,"15min":2.4809999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.103,"5min":1.119,"15min":1.0549999999999999},"min":{"1min":0.82099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6240000000000001,"5min":1.7,"15min":4.5060000000000002},"last":1.2370000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.1439999999999999,"15min":1.1000000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.8580000000000001,"5min":1.9399999999999999,"15min":3.1200000000000001},"last":1.478},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1850000000000001,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.524,"5min":2.077,"15min":2.6000000000000001},"last":1.091}]},{"osd":3,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.198},"min":{"1min":0.82099999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.661,"5min":2.0470000000000002,"15min":2.9079999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3200000000000001,"15min":1.2589999999999999},"min":{"1min":0.94799999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5760000000000001,"5min":2.1110000000000002,"15min":3.9540000000000002},"last":1.1990000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.258,"15min":1.1719999999999999},"min":{"1min":0.83799999999999997,"5min":0.63700000000000001,"15min":0.60899999999999999},"max":{"1min":1.645,"5min":1.958,"15min":4.3940000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.357,"15min":1.296},"min":{"1min":0.98899999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.944,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.603}]},{"osd":6,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4039999999999999,"15min":1.3220000000000001},"min":{"1min":0.86799999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0920000000000001,"5min":2.2869999999999999,"15min":2.7989999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4670000000000001,"15min":1.4079999999999999},"min":{"1min":1.024,"5min":0.70399999999999996,"15min":0.67800000000000005},"max":{"1min":1.9610000000000001,"5min":2.1240000000000001,"15min":4.2809999999999997},"last":1.4470000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:32:37 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.498,"15min":1.407},"min":{"1min":1.081,"5min":0.76000000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.1669999999999998,"15min":2.3479999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":1.022,"5min":0.74099999999999999,"15min":0.73599999999999999},"max":{"1min":2.0680000000000001,"5min":2.4900000000000002,"15min":2.698},"last":1.2949999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107553,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.72099999999999997},"min":{"1min":0.50900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0449999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92300000000000004,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.361,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.95299999999999996,"15min":0.90800000000000003},"min":{"1min":0.58899999999999997,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":1.234,"5min":2.403,"15min":2.403},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.998,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63500000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.474,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.139}]},{"osd":2,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.02,"15min":0.96999999999999997},"min":{"1min":0.57599999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.266,"5min":1.611,"15min":1.611},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.1719999999999999,"5min":3.4510000000000001,"15min":3.4510000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1830000000000001,"15min":1.181},"min":{"1min":0.90500000000000003,"5min":0.70299999999999996,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.99,"15min":3.585},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2549999999999999,"15min":1.214},"min":{"1min":0.72499999999999998,"5min":0.68100000000000005,"15min":0.61299999999999999},"max":{"1min":1.5489999999999999,"5min":1.7749999999999999,"15min":1.877},"last":1.597}]},{"osd":5,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.169,"15min":1.202},"min":{"1min":0.73799999999999999,"5min":0.64400000000000002,"15min":0.57099999999999995},"max":{"1min":1.444,"5min":3.738,"15min":3.738},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.262,"15min":1.2529999999999999},"min":{"1min":0.85699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.603,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3520000000000001,"15min":1.28},"min":{"1min":0.89000000000000001,"5min":0.72599999999999998,"15min":0.58299999999999996},"max":{"1min":1.663,"5min":1.895,"15min":1.8979999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2809999999999999},"min":{"1min":0.82199999999999995,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":1.3939999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":1.155}]},{"osd":7,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2729999999999999,"15min":1.258},"min":{"1min":0.81599999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6910000000000001,"5min":1.869,"15min":3.391},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.357,"15min":1.3220000000000001},"min":{"1min":0.83699999999999997,"5min":0.753,"15min":0.625},"max":{"1min":1.613,"5min":1.946,"15min":2.0409999999999999},"last":1.6579999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":871,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194440,"statfs":{"total":200000000,"available":199107097,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.879,"15min":0.80500000000000005},"min":{"1min":0.435,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":7.1840000000000002,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.4550000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0109999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3720000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.085},{"interface":"front","average":{"1min":1.034,"5min":1.028,"15min":0.89600000000000002},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.35899999999999999},"max":{"1min":1.6279999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93100000000000005,"15min":0.82399999999999995},"min":{"1min":0.56699999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0880000000000001,"15min":1},"min":{"1min":0.56399999999999995,"5min":0.435,"15min":0.432},"max":{"1min":1.458,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2829999999999999,"15min":1.1990000000000001},"min":{"1min":0.77000000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.605,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.2609999999999999,"15min":1.181},"min":{"1min":0.86699999999999999,"5min":0.503,"15min":0.503},"max":{"1min":2.02,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.3360000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2330000000000001,"15min":1.1200000000000001},"min":{"1min":0.877,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.304},{"interface":"front","average":{"1min":1.282,"5min":1.335,"15min":1.2809999999999999},"min":{"1min":0.85599999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.9770000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.22}]},{"osd":6,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.343,"15min":1.2150000000000001},"min":{"1min":0.83899999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.919,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3160000000000001,"15min":1.2250000000000001},"min":{"1min":0.80700000000000005,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":2.6920000000000002,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.6200000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.395,"15min":1.252},"min":{"1min":0.96799999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.976,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.667},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.2450000000000001},"min":{"1min":0.85899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.0179999999999998,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.8300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":870,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194442,"statfs":{"total":200000000,"available":199108732,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.95099999999999996},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.34200000000000003},"max":{"1min":1.304,"5min":1.425,"15min":3.5680000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":1.2390000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.256,"5min":1.4870000000000001,"15min":5.8099999999999996},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.639},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":1.0669999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.4410000000000001,"5min":1.823,"15min":3.0259999999999998},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.119,"15min":1.208},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6699999999999999,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.2030000000000001},"min":{"1min":0.505,"5min":0.501,"15min":0.49299999999999999},"max":{"1min":1.377,"5min":1.774,"15min":4.8760000000000003},"last":1.29}]},{"osd":4,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.125,"15min":1.264},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.625},"max":{"1min":1.6220000000000001,"5min":1.837,"15min":2.9449999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2869999999999999,"15min":1.339},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.9490000000000001,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":0.996}]},{"osd":5,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.296,"15min":1.409},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.579,"5min":1.9970000000000001,"15min":4.1849999999999996},"last":1.454},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3899999999999999},"min":{"1min":0.81599999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.6970000000000001,"5min":1.7090000000000001,"15min":3.0249999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.321,"15min":1.5},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.5529999999999999,"5min":2.1539999999999999,"15min":4.3819999999999997},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.3200000000000001,"15min":1.4410000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.81000000000000005},"max":{"1min":1.77,"5min":1.8280000000000001,"15min":2.9910000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3839999999999999,"15min":1.5029999999999999},"min":{"1min":0.89200000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.008,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3779999999999999,"15min":1.548},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":2.0790000000000002,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.6259999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1446,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193865,"statfs":{"total":200000000,"available":198518364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":1.0369999999999999},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.194,"5min":1.8680000000000001,"15min":4.8070000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":1.331},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.29199999999999998},"max":{"1min":1.6459999999999999,"5min":1.6870000000000001,"15min":10.816000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94899999999999995,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32700000000000001},"max":{"1min":1.349,"5min":2.0369999999999999,"15min":4.8490000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93999999999999995,"15min":1.337},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.7,"5min":1.706,"15min":10.781000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":1.1439999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.998,"5min":1.04,"15min":1.236},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.36699999999999999},"max":{"1min":1.4059999999999999,"5min":2.2170000000000001,"15min":4.9530000000000003},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.2969999999999999},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.66800000000000004},"max":{"1min":1.52,"5min":1.9670000000000001,"15min":3.669},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.2529999999999999,"15min":1.3620000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4790000000000001,"5min":2.2719999999999998,"15min":4.8780000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2669999999999999,"15min":1.6779999999999999},"min":{"1min":0.79300000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6899999999999999,"5min":2.0150000000000001,"15min":10.651},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.232,"15min":1.3720000000000001},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.5069999999999999,"5min":2.0009999999999999,"15min":3.7869999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3049999999999999,"15min":1.427},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5349999999999999,"5min":2.319,"15min":4.9210000000000003},"last":1.76},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.708},"min":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7290000000000001,"5min":2.0510000000000002,"15min":10.708},"last":1.0349999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:32:46 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.347,"15min":1.7350000000000001},"min":{"1min":0.77700000000000002,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":1.833,"5min":2.3010000000000002,"15min":10.74},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.298,"15min":1.405},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.633,"5min":1.9159999999999999,"15min":3.863},"last":1.1000000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:33:00.552 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:33:00.552 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:33:00.589 INFO:tasks.ceph.mds.c.smithi091.stderr:2024-02-02T13:33:00.587+0000 7f2777c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:33:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.804+0000 7fcd14c00640 1 Processor -- start 2024-02-02T13:33:00.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.804+0000 7fcd14c00640 1 -- start start 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.805+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 0x7fcd1006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd14c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1015d870 con 0x7fcd100b0700 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd14c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1006bce0 con 0x7fcd1015d490 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd14c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10161220 con 0x7fcd100acfe0 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd0e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcd100b0700 0x7fcd100c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:36408/0 (socket says 172.21.15.91:36408) 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd0e800640 1 -- 172.21.15.91:0/58245026 learned_addr learned my addr 172.21.15.91:0/58245026 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.806+0000 7fcd0d400640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 0x7fcd1006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102691949 0 0) 0x7fcd1015d870 con 0x7fcd100b0700 2024-02-02T13:33:00.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf0003520 con 0x7fcd100b0700 2024-02-02T13:33:00.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2513430165 0 0) 0x7fccf0003520 con 0x7fcd100b0700 2024-02-02T13:33:00.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 msgr2=0x7fcd1006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:00.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 0x7fcd1006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 >> v1:172.21.15.113:6789/0 conn(0x7fcd1015d490 legacy=0x7fcd100ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100ad490 con 0x7fcd100b0700 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fcd080019f0 con 0x7fcd100b0700 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 >> v1:172.21.15.91:6789/0 conn(0x7fcd100b0700 legacy=0x7fcd100c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 0x7fccf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10161220 con 0x7fccf00047c0 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf000ba60 con 0x7fccf00039a0 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf000bc20 con 0x7fcd100acfe0 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.807+0000 7fcd0d400640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0d400640 1 -- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 msgr2=0x7fccf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0d400640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd100acfe0 0x7fccf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0d400640 1 -- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccf00039a0 msgr2=0x7fccf0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0de00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0d400640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0d400640 1 -- 172.21.15.91:0/58245026 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf0003520 con 0x7fccf00047c0 2024-02-02T13:33:00.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.808+0000 7fcd0de00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:33:00.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.809+0000 7fcd0d400640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcd100ad3c0 tx=0x7fccf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba8bf1d9896665b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.809+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf802c680 con 0x7fccf00047c0 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.809+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf802c7e0 con 0x7fccf00047c0 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.809+0000 7fcd0ca00640 1 -- 172.21.15.91:0/58245026 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf802cb70 con 0x7fccf00047c0 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 -- 172.21.15.91:0/58245026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 msgr2=0x7fccf0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 --2- 172.21.15.91:0/58245026 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcd100ad3c0 tx=0x7fccf00057f0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd0e800640 1 -- 172.21.15.91:0/58245026 reap_dead start 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 -- 172.21.15.91:0/58245026 shutdown_connections 2024-02-02T13:33:00.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 -- 172.21.15.91:0/58245026 >> 172.21.15.91:0/58245026 conn(0x7fcd100cec80 msgr2=0x7fcd100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:00.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 -- 172.21.15.91:0/58245026 shutdown_connections 2024-02-02T13:33:00.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.810+0000 7fcd14c00640 1 -- 172.21.15.91:0/58245026 wait complete. 2024-02-02T13:33:00.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.812+0000 7fcd14c00640 1 Processor -- start 2024-02-02T13:33:00.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.812+0000 7fcd14c00640 1 -- start start 2024-02-02T13:33:00.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd100c7380 0x7fcd101568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd10156df0 0x7fcd101f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c69b0 con 0x7fccf00047c0 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c5e10 con 0x7fcd100c7380 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd14c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c6080 con 0x7fcd10156df0 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59414/0 (socket says 172.21.15.91:59414) 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 -- 172.21.15.91:0/1748307032 learned_addr learned my addr 172.21.15.91:0/1748307032 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 -- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd10156df0 msgr2=0x7fcd101f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd10156df0 0x7fcd101f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 -- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd100c7380 msgr2=0x7fcd101568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd100c7380 0x7fcd101568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.813+0000 7fcd0de00640 1 -- 172.21.15.91:0/1748307032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100c6360 con 0x7fccf00047c0 2024-02-02T13:33:00.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd0de00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d7a0 tx=0x7fcd0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00017070 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00005070 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00005460 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd100c6640 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.814+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd101f6510 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.815+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100b1370 con 0x7fccf00047c0 2024-02-02T13:33:00.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.816+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd00007030 con 0x7fccf00047c0 2024-02-02T13:33:00.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.817+0000 7fcd06000640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 0x7fcce806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:00.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.817+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd000881f0 con 0x7fccf00047c0 2024-02-02T13:33:00.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.817+0000 7fcd0d400640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 0x7fcce806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:00.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.818+0000 7fcd0d400640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 0x7fcce806c8f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fccf8036000 tx=0x7fccf802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:00.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:00.821+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcd000537e0 con 0x7fccf00047c0 2024-02-02T13:33:01.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.074+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcd1006b940 con 0x7fccf00047c0 2024-02-02T13:33:01.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.081+0000 7fcd06000640 1 -- 172.21.15.91:0/1748307032 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcd00056e00 con 0x7fccf00047c0 2024-02-02T13:33:01.083 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:33:01.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.083+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 msgr2=0x7fcce806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.083+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 0x7fcce806c8f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fccf8036000 tx=0x7fccf802e000 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.083+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 msgr2=0x7fcd100c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.083+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d7a0 tx=0x7fcd0000dc60 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 shutdown_connections 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcce806a430 0x7fcce806c8f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd10156df0 0x7fcd101f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd100c7380 0x7fcd101568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 --2- 172.21.15.91:0/1748307032 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fccf00047c0 0x7fcd100c6e40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 >> 172.21.15.91:0/1748307032 conn(0x7fcd100cec80 msgr2=0x7fcd10073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 shutdown_connections 2024-02-02T13:33:01.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.084+0000 7fcd14c00640 1 -- 172.21.15.91:0/1748307032 wait complete. 2024-02-02T13:33:01.101 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:33:01.274 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:33:01.282 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:33:01.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.297+0000 7effe9200640 1 Processor -- start 2024-02-02T13:33:01.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.297+0000 7effe9200640 1 -- start start 2024-02-02T13:33:01.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe40d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe9200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7effe40d3680 0x7effe40d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe9200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe4161090 con 0x7effe415d330 2024-02-02T13:33:01.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe9200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe4161cd0 con 0x7effe40d3680 2024-02-02T13:33:01.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40d3af0 con 0x7effe40b2470 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe1e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7effe415d330 0x7effe40b8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:36420/0 (socket says 172.21.15.91:36420) 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe1e00640 1 -- 172.21.15.91:0/9735965 learned_addr learned my addr 172.21.15.91:0/9735965 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.298+0000 7effe2800640 1 --2- 172.21.15.91:0/9735965 >> v2:172.21.15.91:3300/0 conn(0x7effe40d3680 0x7effe40d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effd9200640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe40d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe1400640 1 -- 172.21.15.91:0/9735965 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163437531 0 0) 0x7effe4161090 con 0x7effe415d330 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe1400640 1 -- 172.21.15.91:0/9735965 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc4003520 con 0x7effe415d330 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe2800640 1 -- 172.21.15.91:0/9735965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 msgr2=0x7effe40d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe2800640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe40d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe2800640 1 -- 172.21.15.91:0/9735965 >> v1:172.21.15.91:6789/0 conn(0x7effe415d330 legacy=0x7effe40b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effe2800640 1 -- 172.21.15.91:0/9735965 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4161330 con 0x7effe40d3680 2024-02-02T13:33:01.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.299+0000 7effd9200640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe40d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:33:01.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.300+0000 7effe2800640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40d1610 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7effdc0097a0 tx=0x7effdc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c7f50f2a342b56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.300+0000 7effe1400640 1 -- 172.21.15.91:0/9735965 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc02f050 con 0x7effe40d3680 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.300+0000 7effe1400640 1 -- 172.21.15.91:0/9735965 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc031700 con 0x7effe40d3680 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.300+0000 7effe1400640 1 -- 172.21.15.91:0/9735965 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc031a70 con 0x7effe40d3680 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 -- 172.21.15.91:0/9735965 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 msgr2=0x7effe40d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40d1610 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7effdc0097a0 tx=0x7effdc02b7a0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 -- 172.21.15.91:0/9735965 shutdown_connections 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40d1610 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 --2- 172.21.15.91:0/9735965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe40d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 -- 172.21.15.91:0/9735965 >> 172.21.15.91:0/9735965 conn(0x7effe4070660 msgr2=0x7effe4070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 -- 172.21.15.91:0/9735965 shutdown_connections 2024-02-02T13:33:01.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.301+0000 7effe9200640 1 -- 172.21.15.91:0/9735965 wait complete. 2024-02-02T13:33:01.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.303+0000 7effe9200640 1 Processor -- start 2024-02-02T13:33:01.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.303+0000 7effe9200640 1 -- start start 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe415d330 0x7effe40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe415fe80 con 0x7effe40d3680 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b1c10 con 0x7effe415d330 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe415df80 con 0x7effe40b2470 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effd9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effd9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59426/0 (socket says 172.21.15.91:59426) 2024-02-02T13:33:01.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effd9200640 1 -- 172.21.15.91:0/2561256417 learned_addr learned my addr 172.21.15.91:0/2561256417 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:01.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.304+0000 7effe1e00640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effe2800640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe415d330 0x7effe40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 -- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 msgr2=0x7effe415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe415b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 -- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe415d330 msgr2=0x7effe40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe415d330 0x7effe40cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 -- 172.21.15.91:0/2561256417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009400 con 0x7effe40d3680 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.305+0000 7effd9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c7f0 tx=0x7effcc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.306+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc007820 con 0x7effe40d3680 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.306+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc007980 con 0x7effe40d3680 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.306+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc007d10 con 0x7effe40d3680 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.306+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe415e260 con 0x7effe40d3680 2024-02-02T13:33:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.306+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe415e820 con 0x7effe40d3680 2024-02-02T13:33:01.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.307+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe4067400 con 0x7effe40d3680 2024-02-02T13:33:01.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.308+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7effcc0113f0 con 0x7effe40d3680 2024-02-02T13:33:01.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.309+0000 7effdaa00640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 0x7effbc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:01.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.309+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effcc088b60 con 0x7effe40d3680 2024-02-02T13:33:01.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.309+0000 7effe1e00640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 0x7effbc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:01.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.310+0000 7effe1e00640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 0x7effbc06c8f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7effd40016e0 tx=0x7effd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:01.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.313+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7effcc054160 con 0x7effe40d3680 2024-02-02T13:33:01.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.544+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe40b5ac0 con 0x7effe40d3680 2024-02-02T13:33:01.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.545+0000 7effdaa00640 1 -- 172.21.15.91:0/2561256417 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7effcc01f090 con 0x7effe40d3680 2024-02-02T13:33:01.547 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:33:01.547 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:33:01.547 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 msgr2=0x7effbc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 0x7effbc06c8f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7effd40016e0 tx=0x7effd4009040 comp rx=0 tx=0).stop 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 msgr2=0x7effe40cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c7f0 tx=0x7effcc00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 shutdown_connections 2024-02-02T13:33:01.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7effbc06a430 0x7effbc06c8f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.547+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe415d330 0x7effe40cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.548+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7effe40d3680 0x7effe40cc450 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.548+0000 7effe9200640 1 --2- 172.21.15.91:0/2561256417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe40b2470 0x7effe415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.548+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 >> 172.21.15.91:0/2561256417 conn(0x7effe4070660 msgr2=0x7effe40d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.548+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 shutdown_connections 2024-02-02T13:33:01.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:01.548+0000 7effe9200640 1 -- 172.21.15.91:0/2561256417 wait complete. 2024-02-02T13:33:01.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:32:55.114755+0000', 'modified': '2024-02-02T13:32:58.630163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8034}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8034': {'gid': 8034, 'name': 'c', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/2641930599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6834', 'nonce': 2641930599}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 2641930599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:33:01.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:33:01.970 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T13:33:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.185+0000 7fd3ccc00640 1 Processor -- start 2024-02-02T13:33:02.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.185+0000 7fd3ccc00640 1 -- start start 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3c815b0f0 0x7fd3c806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3c80cb580 0x7fd3c80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c80cbf50 con 0x7fd3c815b0f0 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c80cd610 con 0x7fd3c80cb580 2024-02-02T13:33:02.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.186+0000 7fd3ccc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c806d910 con 0x7fd3c815ad10 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3c815b0f0 0x7fd3c806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd3c815b0f0 0x7fd3c806d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59430/0 (socket says 172.21.15.91:59430) 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 -- 172.21.15.91:0/3070576135 learned_addr learned my addr 172.21.15.91:0/3070576135 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 -- 172.21.15.91:0/3070576135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 msgr2=0x7fd3c806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3070576135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 -- 172.21.15.91:0/3070576135 >> v2:172.21.15.113:3300/0 conn(0x7fd3c80cb580 msgr2=0x7fd3c80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3070576135 >> v2:172.21.15.113:3300/0 conn(0x7fd3c80cb580 0x7fd3c80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.187+0000 7fd3bd200640 1 -- 172.21.15.91:0/3070576135 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80cc850 con 0x7fd3c815b0f0 2024-02-02T13:33:02.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.188+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3070576135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c815b0f0 0x7fd3c806d020 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80cc740 tx=0x7fd3b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f5c7770a22814 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.188+0000 7fd3c5400640 1 -- 172.21.15.91:0/3070576135 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0031640 con 0x7fd3c815b0f0 2024-02-02T13:33:02.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.188+0000 7fd3c5400640 1 -- 172.21.15.91:0/3070576135 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b00317a0 con 0x7fd3c815b0f0 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.188+0000 7fd3c5400640 1 -- 172.21.15.91:0/3070576135 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0031b30 con 0x7fd3c815b0f0 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3070576135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c815b0f0 msgr2=0x7fd3c806d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3070576135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c815b0f0 0x7fd3c806d020 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80cc740 tx=0x7fd3b002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3070576135 shutdown_connections 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3070576135 >> v2:172.21.15.113:3300/0 conn(0x7fd3c80cb580 0x7fd3c80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3070576135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c815b0f0 0x7fd3c806d020 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3070576135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3070576135 >> 172.21.15.91:0/3070576135 conn(0x7fd3c80b0700 msgr2=0x7fd3c80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3070576135 shutdown_connections 2024-02-02T13:33:02.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.189+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3070576135 wait complete. 2024-02-02T13:33:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.190+0000 7fd3ccc00640 1 Processor -- start 2024-02-02T13:33:02.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.191+0000 7fd3ccc00640 1 -- start start 2024-02-02T13:33:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.191+0000 7fd3ccc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.191+0000 7fd3ccc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c80c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3ccc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c815b0f0 0x7fd3c815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3ccc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80c7d10 con 0x7fd3c80cb580 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3ccc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80ae9e0 con 0x7fd3c815b0f0 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3ccc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80c71d0 con 0x7fd3c815ad10 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59442/0 (socket says 172.21.15.91:59442) 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 -- 172.21.15.91:0/3623451936 learned_addr learned my addr 172.21.15.91:0/3623451936 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 -- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 msgr2=0x7fd3c80c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c80c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 -- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c815b0f0 msgr2=0x7fd3c815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c815b0f0 0x7fd3c815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 -- 172.21.15.91:0/3623451936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00094d0 con 0x7fd3c80cb580 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.192+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b4a0 tx=0x7fd3b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.193+0000 7fd3bea00640 1 -- 172.21.15.91:0/3623451936 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8010610 con 0x7fd3c80cb580 2024-02-02T13:33:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.193+0000 7fd3bea00640 1 -- 172.21.15.91:0/3623451936 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b8010770 con 0x7fd3c80cb580 2024-02-02T13:33:02.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.193+0000 7fd3bea00640 1 -- 172.21.15.91:0/3623451936 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8010b20 con 0x7fd3c80cb580 2024-02-02T13:33:02.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.193+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c80c7470 con 0x7fd3c80cb580 2024-02-02T13:33:02.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.193+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c80c7a30 con 0x7fd3c80cb580 2024-02-02T13:33:02.196 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.195+0000 7fd3bea00640 1 -- 172.21.15.91:0/3623451936 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3b8010c80 con 0x7fd3c80cb580 2024-02-02T13:33:02.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.196+0000 7fd3bea00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 0x7fd3a006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.197+0000 7fd3bea00640 1 -- 172.21.15.91:0/3623451936 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3b8088530 con 0x7fd3c80cb580 2024-02-02T13:33:02.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.197+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 0x7fd3a006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.197+0000 7fd3cf87a740 1 Processor -- start 2024-02-02T13:33:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.197+0000 7fd3cf87a740 1 -- start start 2024-02-02T13:33:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.197+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 0x7fd3a006c920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0036000 tx=0x7fd3b0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc940330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc90fed0 0x564efc93d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc8eb650 con 0x564efc913ad0 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc880000 con 0x564efc90fed0 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564efc87d790 con 0x564efc93ff50 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.198+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59452/0 (socket says 172.21.15.91:59452) 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 -- 172.21.15.91:0/3694468115 learned_addr learned my addr 172.21.15.91:0/3694468115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc940330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3bd200640 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc90fed0 0x564efc93d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 -- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 msgr2=0x564efc940330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc940330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 -- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc90fed0 msgr2=0x564efc93d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc90fed0 0x564efc93d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 -- 172.21.15.91:0/3694468115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564efc840710 con 0x564efc913ad0 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3c6800640 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009cd0 tx=0x7fd3c000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d37cae11ef279b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.199+0000 7fd3b6000640 1 -- 172.21.15.91:0/3694468115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000e070 con 0x564efc913ad0 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3b6000640 1 -- 172.21.15.91:0/3694468115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c000bd40 con 0x564efc913ad0 2024-02-02T13:33:02.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3b6000640 1 -- 172.21.15.91:0/3694468115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0012650 con 0x564efc913ad0 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 -- 172.21.15.91:0/3694468115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 msgr2=0x564efc93d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009cd0 tx=0x7fd3c000b420 comp rx=0 tx=0).stop 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 -- 172.21.15.91:0/3694468115 shutdown_connections 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc913ad0 0x564efc93d700 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc90fed0 0x564efc93d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/3694468115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc940330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 -- 172.21.15.91:0/3694468115 >> 172.21.15.91:0/3694468115 conn(0x564efc9121e0 msgr2=0x564efc95ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 -- 172.21.15.91:0/3694468115 shutdown_connections 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.200+0000 7fd3cf87a740 1 -- 172.21.15.91:0/3694468115 wait complete. 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.201+0000 7fd3cf87a740 1 -- start start 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.201+0000 7fd3cf87a740 1 Processor -- start 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc913ad0 0x564efc960850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc95b370 0x564efc929a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc879080 con 0x564efc93ff50 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc81a850 con 0x564efc95b370 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564efc81a4f0 con 0x564efc913ad0 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3cf87a740 10 client.0 _ll_register_callbacks cb 0x564efc8729c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3bd200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3bd200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59458/0 (socket says 172.21.15.91:59458) 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.202+0000 7fd3bd200640 1 -- 172.21.15.91:0/181708930 learned_addr learned my addr 172.21.15.91:0/181708930 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc913ad0 0x564efc960850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 -- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc913ad0 msgr2=0x564efc960850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc913ad0 0x564efc960850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 -- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc95b370 msgr2=0x564efc929a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc95b370 0x564efc929a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 -- 172.21.15.91:0/181708930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c0009980 con 0x564efc93ff50 2024-02-02T13:33:02.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3bd200640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd3b002b930 tx=0x7fd3b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3491e9f9dd482c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3aea00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b004f440 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3b0050c50 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.203+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564efc8a35d0 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b0050e10 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 10 client.8190 fetch_fsmap learned FSMap version 72 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x564efc877700 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564efc8a35d0 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 72) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd3b004f5a0 con 0x564efc93ff50 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 10 client.8190 fetch_fsmap finished waiting for FSMap version 72 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 10 client.8190 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 20 client.8190 populate_metadata read hostname 'smithi091' 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc96a5f0 0x564efc96a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 4 client.8190 mds_command: new command op to 8034 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:33:02.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 --> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564efc86b8d0 con 0x564efc96a5f0 2024-02-02T13:33:02.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.204+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc96a5f0 0x564efc96a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.205+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc96a5f0 0x564efc96a9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e5b184aa3e70b server_cookie=2c9585b166a32ad7 in_seq=0 out_seq=0 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.205+0000 7fd3aea00640 10 client.8190 ms_handle_connect on v2:172.21.15.91:6834/2641930599 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3aea00640 1 -- 172.21.15.91:0/181708930 <== mds.0 v2:172.21.15.91:6834/2641930599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564efc86b8d0 con 0x564efc96a5f0 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3aea00640 10 client.8190 handle_command_reply: tid=0 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 client.8190 shutdown 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 msgr2=0x564efc96bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd3b002b930 tx=0x7fd3b0004730 comp rx=0 tx=0).stop 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 shutdown_connections 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc95b370 0x564efc929a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc93ff50 0x564efc96bac0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc96a5f0 0x564efc96a9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/181708930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc913ad0 0x564efc960850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 >> 172.21.15.91:0/181708930 conn(0x564efc90fed0 msgr2=0x564efc914280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3aea00640 0 client.8190 ms_handle_reset on v2:172.21.15.91:6834/2641930599 2024-02-02T13:33:02.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 shutdown_connections 2024-02-02T13:33:02.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.206+0000 7fd3cf87a740 1 -- 172.21.15.91:0/181708930 wait complete. 2024-02-02T13:33:02.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.207+0000 7fd3cf87a740 20 client.8190 trim_cache size 0 max 16384 2024-02-02T13:33:02.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.222+0000 7fd3cf87a740 1 Processor -- start 2024-02-02T13:33:02.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.222+0000 7fd3cf87a740 1 -- start start 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc914070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc9145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc5e0f20 con 0x564efc95b370 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc878220 con 0x564efc913ad0 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564efc942f50 con 0x564efc93ff50 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59470/0 (socket says 172.21.15.91:59470) 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 -- 172.21.15.91:0/1809812069 learned_addr learned my addr 172.21.15.91:0/1809812069 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 -- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 msgr2=0x564efc9145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc9145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 -- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 msgr2=0x564efc914070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc914070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.223+0000 7fd3c6800640 1 -- 172.21.15.91:0/1809812069 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564efc8ad280 con 0x564efc95b370 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.224+0000 7fd3c6800640 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0013000 tx=0x7fd3c000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59a3dffcc7689e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.224+0000 7fd3b6a00640 1 -- 172.21.15.91:0/1809812069 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00126b0 con 0x564efc95b370 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.224+0000 7fd3b6a00640 1 -- 172.21.15.91:0/1809812069 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c0012810 con 0x564efc95b370 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.224+0000 7fd3b6a00640 1 -- 172.21.15.91:0/1809812069 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0012bc0 con 0x564efc95b370 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- 172.21.15.91:0/1809812069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 msgr2=0x564efc96a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0013000 tx=0x7fd3c000e000 comp rx=0 tx=0).stop 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- 172.21.15.91:0/1809812069 shutdown_connections 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc9145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc96a9d0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/1809812069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc914070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- 172.21.15.91:0/1809812069 >> 172.21.15.91:0/1809812069 conn(0x564efc90fed0 msgr2=0x564efc960830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- 172.21.15.91:0/1809812069 shutdown_connections 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- 172.21.15.91:0/1809812069 wait complete. 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 -- start start 2024-02-02T13:33:02.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.225+0000 7fd3cf87a740 1 Processor -- start 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc95cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc958420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc96b840 con 0x564efc95b370 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efc9591b0 con 0x564efc913ad0 2024-02-02T13:33:02.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564efc93a030 con 0x564efc93ff50 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.226+0000 7fd3cf87a740 10 client.0 _ll_register_callbacks cb 0x7fd3b00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59486/0 (socket says 172.21.15.91:59486) 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 -- 172.21.15.91:0/520525864 learned_addr learned my addr 172.21.15.91:0/520525864 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 -- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 msgr2=0x564efc958420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc958420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 -- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 msgr2=0x564efc95cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc95cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 -- 172.21.15.91:0/520525864 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564efc840710 con 0x564efc95b370 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3c6800640 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009c70 tx=0x7fd3c0004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b8a498ed9b2e5e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3aea00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.227+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000e040 con 0x564efc95b370 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3c0012e40 con 0x564efc95b370 2024-02-02T13:33:02.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3c000bd40 con 0x564efc95b370 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3c0025600 con 0x564efc95b370 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 10 client.8196 fetch_fsmap learned FSMap version 72 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x564efc819a60 con 0x564efc95b370 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c000bd40 con 0x564efc95b370 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 72) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd3c0036020 con 0x564efc95b370 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 10 client.8196 fetch_fsmap finished waiting for FSMap version 72 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 10 client.8196 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 20 client.8196 populate_metadata read hostname 'smithi091' 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc9435f0 0x564efc9439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 4 client.8196 mds_command: new command op to 8034 tid=0[{"prefix": "status"}] 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 --> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564efc872100 con 0x564efc9435f0 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.228+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc9435f0 0x564efc9439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3c5e00640 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc9435f0 0x564efc9439d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76cdc65ba3fdeb0e server_cookie=24dbe08d2b77b8dd in_seq=0 out_seq=0 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3aea00640 10 client.8196 ms_handle_connect on v2:172.21.15.91:6834/2641930599 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3aea00640 1 -- 172.21.15.91:0/520525864 <== mds.0 v2:172.21.15.91:6834/2641930599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x564efc872100 con 0x564efc9435f0 2024-02-02T13:33:02.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3aea00640 10 client.8196 handle_command_reply: tid=0 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 client.8196 shutdown 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 msgr2=0x564efc958960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009c70 tx=0x7fd3c0004bc0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 shutdown_connections 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:6834/2641930599,v1:172.21.15.91:6837/2641930599] conn(0x564efc9435f0 0x564efc9439d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564efc93ff50 0x564efc958420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564efc95b370 0x564efc958960 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 --2- 172.21.15.91:0/520525864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564efc913ad0 0x564efc95cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3aea00640 0 client.8196 ms_handle_reset on v2:172.21.15.91:6834/2641930599 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.229+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 >> 172.21.15.91:0/520525864 conn(0x564efc90fed0 msgr2=0x564efc914070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 shutdown_connections 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3cf87a740 1 -- 172.21.15.91:0/520525864 wait complete. 2024-02-02T13:33:02.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3cf87a740 20 client.8196 trim_cache size 0 max 16384 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 8034, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 7.1102782690000002, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 72, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 161, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 161, 2024-02-02T13:33:02.235 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 12.015278499000001 2024-02-02T13:33:02.236 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:33:02.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 msgr2=0x7fd3a006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 0x7fd3a006c920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0036000 tx=0x7fd3b0030000 comp rx=0 tx=0).stop 2024-02-02T13:33:02.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 msgr2=0x7fd3c80c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b4a0 tx=0x7fd3b800b960 comp rx=0 tx=0).stop 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 shutdown_connections 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd3a006a460 0x7fd3a006c920 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c815ad10 0x7fd3c80c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c815b0f0 0x7fd3c815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 --2- 172.21.15.91:0/3623451936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3c80cb580 0x7fd3c80c8700 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 >> 172.21.15.91:0/3623451936 conn(0x7fd3c80b0700 msgr2=0x7fd3c815a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 shutdown_connections 2024-02-02T13:33:02.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.230+0000 7fd3ccc00640 1 -- 172.21.15.91:0/3623451936 wait complete. 2024-02-02T13:33:02.247 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.435+0000 7ff309200640 1 Processor -- start 2024-02-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.435+0000 7ff309200640 1 -- start start 2024-02-02T13:33:02.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.436+0000 7ff309200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.436+0000 7ff309200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff30406f4f0 0x7ff3040b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff309200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff304070f80 con 0x7ff3040b0c40 2024-02-02T13:33:02.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff309200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3040b10d0 con 0x7ff30406f4f0 2024-02-02T13:33:02.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff309200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff304152370 con 0x7ff304151f90 2024-02-02T13:33:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff301e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff301e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:40524/0 (socket says 172.21.15.91:40524) 2024-02-02T13:33:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.437+0000 7ff301e00640 1 -- 172.21.15.91:0/715922136 learned_addr learned my addr 172.21.15.91:0/715922136 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.438+0000 7ff2f9200640 1 --2- 172.21.15.91:0/715922136 >> v2:172.21.15.113:3300/0 conn(0x7ff30406f4f0 0x7ff3040b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.438+0000 7ff301e00640 1 -- 172.21.15.91:0/715922136 >> v2:172.21.15.113:3300/0 conn(0x7ff30406f4f0 msgr2=0x7ff3040b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.438+0000 7ff301e00640 1 --2- 172.21.15.91:0/715922136 >> v2:172.21.15.113:3300/0 conn(0x7ff30406f4f0 0x7ff3040b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.438+0000 7ff301e00640 1 -- 172.21.15.91:0/715922136 >> v1:172.21.15.113:6789/0 conn(0x7ff3040b0c40 legacy=0x7ff3040b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.438+0000 7ff301e00640 1 -- 172.21.15.91:0/715922136 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040cb010 con 0x7ff304151f90 2024-02-02T13:33:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.439+0000 7ff301e00640 1 --2- 172.21.15.91:0/715922136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e50 tx=0x7ff2f402bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f737d7c30a6b32 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.440+0000 7ff301400640 1 -- 172.21.15.91:0/715922136 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f402caf0 con 0x7ff304151f90 2024-02-02T13:33:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.440+0000 7ff301400640 1 -- 172.21.15.91:0/715922136 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f402cc50 con 0x7ff304151f90 2024-02-02T13:33:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.440+0000 7ff301400640 1 -- 172.21.15.91:0/715922136 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40343f0 con 0x7ff304151f90 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.440+0000 7ff309200640 1 -- 172.21.15.91:0/715922136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 msgr2=0x7ff30415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.440+0000 7ff309200640 1 --2- 172.21.15.91:0/715922136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e50 tx=0x7ff2f402bf50 comp rx=0 tx=0).stop 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 -- 172.21.15.91:0/715922136 shutdown_connections 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 --2- 172.21.15.91:0/715922136 >> v2:172.21.15.113:3300/0 conn(0x7ff30406f4f0 0x7ff3040b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 --2- 172.21.15.91:0/715922136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff304151f90 0x7ff30415f5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 -- 172.21.15.91:0/715922136 >> 172.21.15.91:0/715922136 conn(0x7ff3040cec80 msgr2=0x7ff3040cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 -- 172.21.15.91:0/715922136 shutdown_connections 2024-02-02T13:33:02.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.441+0000 7ff309200640 1 -- 172.21.15.91:0/715922136 wait complete. 2024-02-02T13:33:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.443+0000 7ff309200640 1 Processor -- start 2024-02-02T13:33:02.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.443+0000 7ff309200640 1 -- start start 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30406f4f0 0x7ff3040c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 0x7ff30415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c6a60 con 0x7ff304151f90 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c7990 con 0x7ff3040b0c40 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.444+0000 7ff309200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c7c70 con 0x7ff30406f4f0 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59498/0 (socket says 172.21.15.91:59498) 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 -- 172.21.15.91:0/1418116356 learned_addr learned my addr 172.21.15.91:0/1418116356 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff301e00640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30406f4f0 0x7ff3040c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff2f9200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 0x7ff30415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 -- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30406f4f0 msgr2=0x7ff3040c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30406f4f0 0x7ff3040c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 -- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 msgr2=0x7ff30415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 0x7ff30415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 -- 172.21.15.91:0/1418116356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009b30 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff2f9200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 0x7ff30415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.445+0000 7ff302800640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00c3c0 tx=0x7ff2fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.446+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0052d0 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.446+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc005430 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.446+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc003830 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.446+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040c7f50 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.446+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041f04e0 con 0x7ff304151f90 2024-02-02T13:33:02.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.447+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3040c77f0 con 0x7ff304151f90 2024-02-02T13:33:02.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.448+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff2fc003990 con 0x7ff304151f90 2024-02-02T13:33:02.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.449+0000 7ff2faa00640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 0x7ff2dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:02.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.449+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2fc087d60 con 0x7ff304151f90 2024-02-02T13:33:02.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.449+0000 7ff301e00640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 0x7ff2dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:02.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.450+0000 7ff301e00640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 0x7ff2dc06c8f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4035000 tx=0x7ff2f4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:02.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.453+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff3040c77f0 con 0x7ff304151f90 2024-02-02T13:33:02.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.686+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3040c77f0 con 0x7ff304151f90 2024-02-02T13:33:02.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.687+0000 7ff2faa00640 1 -- 172.21.15.91:0/1418116356 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff2fc017060 con 0x7ff304151f90 2024-02-02T13:33:02.689 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:33:02.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 msgr2=0x7ff2dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 0x7ff2dc06c8f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4035000 tx=0x7ff2f4030000 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 msgr2=0x7ff30415a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00c3c0 tx=0x7ff2fc00c880 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 shutdown_connections 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff2dc06a430 0x7ff2dc06c8f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff304151f90 0x7ff30415a610 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3040b0c40 0x7ff30415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 --2- 172.21.15.91:0/1418116356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30406f4f0 0x7ff3040c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 >> 172.21.15.91:0/1418116356 conn(0x7ff3040cec80 msgr2=0x7ff3040c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 shutdown_connections 2024-02-02T13:33:02.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:02.690+0000 7ff309200640 1 -- 172.21.15.91:0/1418116356 wait complete. 2024-02-02T13:33:02.708 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:33:02.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:33:02.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:02.709 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:02.757 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42143sec preferred_lft 42143sec 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:33:02.758 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:33:02.759 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:02.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:02.760 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:33:02.843 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:33:02.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:02.847 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:33:02.894 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:33:02.894 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:33:02.894 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:33:02.894 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:33:02.895 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:33:02.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:02.895 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:33:02.896 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:33:02.896 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:33:02.896 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:33:02.896 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:33:02.896 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:33:03.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:03.075 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:33:03.126 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:33:03.126 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:33:03.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:33:03.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:03.127 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:33:03.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:33:03.223 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:33:03.274 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:33:03.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:33:03.274 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:33:03.321 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:33:03.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:03.321 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:03.502 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:03.502 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:33:03.534 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:03.535 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:33:03.535 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:33:03.535 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:33:03.535 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:33:03.537 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:33:03.538 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:33:03.538 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:33:03.538 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:33:03.538 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:33:03.542 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:33:03.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:03.542 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:33:03.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:03.615 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:33:03.677 INFO:teuthology.orchestra.run.smithi091.stdout:client6013 2024-02-02T13:33:03.677 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:33:03.678 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:33:03.722 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:33:03.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.974+0000 7efd42400640 1 Processor -- start 2024-02-02T13:33:03.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.974+0000 7efd42400640 1 -- start start 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.975+0000 7efd42400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd3c0b0e10 0x7efd3c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd42400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c070f80 con 0x7efd3c151f90 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd42400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c06c600 con 0x7efd3c15f5a0 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd42400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c15fa30 con 0x7efd3c0b0e10 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd3b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7efd3c151f90 0x7efd3c06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:36432/0 (socket says 172.21.15.91:36432) 2024-02-02T13:33:03.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd3b400640 1 -- 172.21.15.91:0/3137677072 learned_addr learned my addr 172.21.15.91:0/3137677072 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:33:03.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429336724 0 0) 0x7efd3c070f80 con 0x7efd3c151f90 2024-02-02T13:33:03.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c003520 con 0x7efd3c151f90 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.976+0000 7efd3be00640 1 --2- 172.21.15.91:0/3137677072 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0b0e10 0x7efd3c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1914628729 0 0) 0x7efd1c003520 con 0x7efd3c151f90 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0b0e10 msgr2=0x7efd3c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 --2- 172.21.15.91:0/3137677072 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0b0e10 0x7efd3c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 >> v1:172.21.15.113:6789/0 conn(0x7efd3c15f5a0 legacy=0x7efd3c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c152370 con 0x7efd3c151f90 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7efd2c001670 con 0x7efd3c151f90 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 >> v1:172.21.15.91:6789/0 conn(0x7efd3c151f90 legacy=0x7efd3c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd1c0047c0 0x7efd1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1c0051d0 0x7efd1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15fa30 con 0x7efd1c0039c0 2024-02-02T13:33:03.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00c120 con 0x7efd1c0047c0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00c2e0 con 0x7efd1c0051d0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd3b400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.977+0000 7efd33e00640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd1c0047c0 0x7efd1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3be00640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1c0051d0 0x7efd1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3b400640 1 -- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1c0051d0 msgr2=0x7efd1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3b400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1c0051d0 0x7efd1c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3b400640 1 -- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd1c0047c0 msgr2=0x7efd1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3b400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd1c0047c0 0x7efd1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3b400640 1 -- 172.21.15.91:0/3137677072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c003520 con 0x7efd1c0039c0 2024-02-02T13:33:03.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.978+0000 7efd3be00640 1 -- 172.21.15.91:0/3137677072 reap_dead start 2024-02-02T13:33:03.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.979+0000 7efd3b400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd1c003de0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c0e0 tx=0x7efd1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a5d0fde7e2745b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:03.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.979+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c005ad0 con 0x7efd1c0039c0 2024-02-02T13:33:03.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.979+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c005c30 con 0x7efd1c0039c0 2024-02-02T13:33:03.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.979+0000 7efd3aa00640 1 -- 172.21.15.91:0/3137677072 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c02ca00 con 0x7efd1c0039c0 2024-02-02T13:33:03.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 -- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 msgr2=0x7efd1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd1c003de0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c0e0 tx=0x7efd1c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 -- 172.21.15.91:0/3137677072 shutdown_connections 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 --2- 172.21.15.91:0/3137677072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd1c003de0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 -- 172.21.15.91:0/3137677072 >> 172.21.15.91:0/3137677072 conn(0x7efd3c0cec80 msgr2=0x7efd3c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 -- 172.21.15.91:0/3137677072 shutdown_connections 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.980+0000 7efd42400640 1 -- 172.21.15.91:0/3137677072 wait complete. 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.982+0000 7efd42400640 1 Processor -- start 2024-02-02T13:33:03.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.982+0000 7efd42400640 1 -- start start 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adb80 0x7efd3c0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0ae000 0x7efd3c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0ac8d0 con 0x7efd1c0039c0 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0acae0 con 0x7efd3c0adb80 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd42400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0acdc0 con 0x7efd3c0ae000 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd3b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd3b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59502/0 (socket says 172.21.15.91:59502) 2024-02-02T13:33:03.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.983+0000 7efd3b400640 1 -- 172.21.15.91:0/1906290950 learned_addr learned my addr 172.21.15.91:0/1906290950 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:03.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3be00640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0ae000 0x7efd3c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 -- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0ae000 msgr2=0x7efd3c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0ae000 0x7efd3c0cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 -- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adb80 msgr2=0x7efd3c0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adb80 0x7efd3c0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 -- 172.21.15.91:0/1906290950 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0ad0a0 con 0x7efd1c0039c0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd3b400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c03c040 con 0x7efd1c0039c0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c02cd30 con 0x7efd1c0039c0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c02cea0 con 0x7efd1c0039c0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c0ad380 con 0x7efd1c0039c0 2024-02-02T13:33:03.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.984+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c14e690 con 0x7efd1c0039c0 2024-02-02T13:33:03.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.986+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c066cf0 con 0x7efd1c0039c0 2024-02-02T13:33:03.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.986+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd2c032070 con 0x7efd1c0039c0 2024-02-02T13:33:03.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.987+0000 7efd33400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 0x7efd1406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:03.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.987+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd2c0a9db0 con 0x7efd1c0039c0 2024-02-02T13:33:03.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.987+0000 7efd33e00640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 0x7efd1406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:03.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.988+0000 7efd33e00640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 0x7efd1406c920 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:03.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:03.991+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd2c075390 con 0x7efd1c0039c0 2024-02-02T13:33:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.218+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd3c1516b0 con 0x7efd1c0039c0 2024-02-02T13:33:04.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.219+0000 7efd33400640 1 -- 172.21.15.91:0/1906290950 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd2c0789b0 con 0x7efd1c0039c0 2024-02-02T13:33:04.221 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 msgr2=0x7efd1406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 0x7efd1406c920 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24009040 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 msgr2=0x7efd3c14dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c02e070 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 shutdown_connections 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd1406a460 0x7efd1406c920 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0ae000 0x7efd3c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adb80 0x7efd3c0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 --2- 172.21.15.91:0/1906290950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0039c0 0x7efd3c14dde0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 >> 172.21.15.91:0/1906290950 conn(0x7efd3c0cec80 msgr2=0x7efd3c0cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:04.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 shutdown_connections 2024-02-02T13:33:04.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:04.222+0000 7efd42400640 1 -- 172.21.15.91:0/1906290950 wait complete. 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:33:04.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:33:04.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.244 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:04.303 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.304 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:33:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:33:04.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.306 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:33:04.397 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:33:04.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.401 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:33:04.455 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:33:04.455 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:33:04.455 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:33:04.455 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:33:04.456 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:33:04.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:33:04.456 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:33:04.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.631 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:33:04.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:33:04.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:33:04.681 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:33:04.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:33:04.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:33:04.770 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:33:04.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:33:04.821 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:33:04.871 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:33:04.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:04.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:05.002 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:05.002 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:33:05.038 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:33:05.038 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:33:05.038 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:33:05.038 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:33:05.038 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:33:05.042 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:33:05.042 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:33:05.042 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:33:05.042 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:33:05.042 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:33:05.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:33:05.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:05.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:33:05.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:33:05.124 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:33:05.178 INFO:teuthology.orchestra.run.smithi113.stdout:client8208 2024-02-02T13:33:05.179 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:33:05.179 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:33:05.230 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:33:05.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.450+0000 7fb793000640 1 Processor -- start 2024-02-02T13:33:05.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.450+0000 7fb793000640 1 -- start start 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.451+0000 7fb793000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb78c0b61d0 0x7fb78c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.451+0000 7fb793000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c0ce210 con 0x7fb78c070980 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.451+0000 7fb793000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c0ce4f0 con 0x7fb78c160830 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.451+0000 7fb793000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78c0ce7d0 con 0x7fb78c0b61d0 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb783200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb78c070980 0x7fb78c0d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:36434/0 (socket says 172.21.15.91:36434) 2024-02-02T13:33:05.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb783200640 1 -- 172.21.15.91:0/3248564242 learned_addr learned my addr 172.21.15.91:0/3248564242 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb791000640 1 --2- 172.21.15.91:0/3248564242 >> v2:172.21.15.113:3300/0 conn(0x7fb78c0b61d0 0x7fb78c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183878789 0 0) 0x7fb78c0ce210 con 0x7fb78c070980 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb76c003520 con 0x7fb78c070980 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591769498 0 0) 0x7fb76c003520 con 0x7fb78c070980 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 >> v2:172.21.15.113:3300/0 conn(0x7fb78c0b61d0 msgr2=0x7fb78c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 --2- 172.21.15.91:0/3248564242 >> v2:172.21.15.113:3300/0 conn(0x7fb78c0b61d0 0x7fb78c0b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.452+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 >> v1:172.21.15.113:6789/0 conn(0x7fb78c160830 legacy=0x7fb78c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0d17b0 con 0x7fb78c070980 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb774001670 con 0x7fb78c070980 2024-02-02T13:33:05.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 >> v1:172.21.15.91:6789/0 conn(0x7fb78c070980 legacy=0x7fb78c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb76c005180 0x7fb76c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0ce7d0 con 0x7fb76c003970 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c00c120 con 0x7fb76c0047c0 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c00c2e0 con 0x7fb76c005180 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb783200640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 -- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb76c005180 msgr2=0x7fb76c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb76c005180 0x7fb76c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 -- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76c0047c0 msgr2=0x7fb76c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb78be00640 1 -- 172.21.15.91:0/3248564242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c003520 con 0x7fb76c003970 2024-02-02T13:33:05.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.453+0000 7fb791000640 1 -- 172.21.15.91:0/3248564242 reap_dead start 2024-02-02T13:33:05.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.454+0000 7fb78be00640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001750 tx=0x7fb76c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999dedbcb850d8c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0052d0 con 0x7fb76c003970 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c02e050 con 0x7fb76c003970 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb78b400640 1 -- 172.21.15.91:0/3248564242 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003830 con 0x7fb76c003970 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 -- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 msgr2=0x7fb76c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001750 tx=0x7fb76c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 -- 172.21.15.91:0/3248564242 shutdown_connections 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 --2- 172.21.15.91:0/3248564242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 -- 172.21.15.91:0/3248564242 >> 172.21.15.91:0/3248564242 conn(0x7fb78c0d3030 msgr2=0x7fb78c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 -- 172.21.15.91:0/3248564242 shutdown_connections 2024-02-02T13:33:05.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.455+0000 7fb793000640 1 -- 172.21.15.91:0/3248564242 wait complete. 2024-02-02T13:33:05.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.457+0000 7fb793000640 1 Processor -- start 2024-02-02T13:33:05.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.457+0000 7fb793000640 1 -- start start 2024-02-02T13:33:05.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb78c1f3750 0x7fb78c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb78c1f5830 0x7fb78c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c156590 con 0x7fb76c003970 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0c80d0 con 0x7fb78c1f3750 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.458+0000 7fb793000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0c83a0 con 0x7fb78c1f5830 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59530/0 (socket says 172.21.15.91:59530) 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 -- 172.21.15.91:0/2607135261 learned_addr learned my addr 172.21.15.91:0/2607135261 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb783200640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb78c1f3750 0x7fb78c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 -- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb78c1f5830 msgr2=0x7fb78c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb78c1f5830 0x7fb78c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 -- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb78c1f3750 msgr2=0x7fb78c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb78c1f3750 0x7fb78c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 -- 172.21.15.91:0/2607135261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0c8680 con 0x7fb76c003970 2024-02-02T13:33:05.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.459+0000 7fb78be00640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02bc00 tx=0x7fb77c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:05.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.460+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007a30 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.460+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c007640 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.460+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c006be0 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.460+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c0c8960 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.460+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1f5cb0 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.461+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c066cf0 con 0x7fb76c003970 2024-02-02T13:33:05.463 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.462+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb77c006d40 con 0x7fb76c003970 2024-02-02T13:33:05.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.463+0000 7fb788c00640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 0x7fb76406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:05.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.463+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb77c0aa530 con 0x7fb76c003970 2024-02-02T13:33:05.464 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.463+0000 7fb783200640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 0x7fb76406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:05.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.464+0000 7fb783200640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 0x7fb76406c920 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb774001200 tx=0x7fb774000f70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:05.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.468+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb77c075b30 con 0x7fb76c003970 2024-02-02T13:33:05.696 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.694+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb78c0ceb40 con 0x7fb76c003970 2024-02-02T13:33:05.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.696+0000 7fb788c00640 1 -- 172.21.15.91:0/2607135261 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fb77c079150 con 0x7fb76c003970 2024-02-02T13:33:05.698 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:33:05.698 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:33:05.698 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:33:05.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.698+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 msgr2=0x7fb76406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.698+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 0x7fb76406c920 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb774001200 tx=0x7fb774000f70 comp rx=0 tx=0).stop 2024-02-02T13:33:05.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 msgr2=0x7fb78c1f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:05.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02bc00 tx=0x7fb77c001120 comp rx=0 tx=0).stop 2024-02-02T13:33:05.701 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 shutdown_connections 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb76406a460 0x7fb76406c920 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb78c1f5830 0x7fb78c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb78c1f3750 0x7fb78c0c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 --2- 172.21.15.91:0/2607135261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb76c003970 0x7fb78c1f3c90 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 >> 172.21.15.91:0/2607135261 conn(0x7fb78c0d3030 msgr2=0x7fb78c0b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 shutdown_connections 2024-02-02T13:33:05.702 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:05.699+0000 7fb793000640 1 -- 172.21.15.91:0/2607135261 wait complete. 2024-02-02T13:33:05.718 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-02-02T13:33:05.933 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":8034,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.798161908999999,"mdsmap_epoch":72,"osdmap_epoch":161,"osdmap_epoch_barrier":161,"uptime":15.703176673} 2024-02-02T13:33:05.934 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 8034, "mdsmap_epoch": 72, "osdmap_epoch": 161, "osdmap_epoch_barrier": 161, "rank_uptime": 10.798161909, "state": "up:active", "uptime": 15.703176673, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:33:05.934 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-02-02T13:33:06.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.147+0000 7f5d59000640 1 Processor -- start 2024-02-02T13:33:06.149 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.148+0000 7f5d59000640 1 -- start start 2024-02-02T13:33:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.149+0000 7f5d59000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5415d650 0x7f5d540ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.149+0000 7f5d59000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5d540b0220 0x7f5d5406a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.149+0000 7f5d59000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540d1db0 con 0x7f5d5406a650 2024-02-02T13:33:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.149+0000 7f5d59000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5406aae0 con 0x7f5d540b0220 2024-02-02T13:33:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.149+0000 7f5d59000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540d0690 con 0x7f5d5415d650 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5d540b0220 0x7f5d5406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d51e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5415d650 0x7f5d540ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5d540b0220 0x7f5d5406a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59542/0 (socket says 172.21.15.91:59542) 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 -- 172.21.15.91:0/569769352 learned_addr learned my addr 172.21.15.91:0/569769352 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 -- 172.21.15.91:0/569769352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5415d650 msgr2=0x7f5d540ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 --2- 172.21.15.91:0/569769352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5415d650 0x7f5d540ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 -- 172.21.15.91:0/569769352 >> v1:172.21.15.113:6789/0 conn(0x7f5d5406a650 legacy=0x7f5d540cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.150+0000 7f5d4be00640 1 -- 172.21.15.91:0/569769352 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540d17f0 con 0x7f5d540b0220 2024-02-02T13:33:06.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.153+0000 7f5d4be00640 1 --2- 172.21.15.91:0/569769352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d5406a110 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c79e4cc1b51277a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:06.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.153+0000 7f5d51400640 1 -- 172.21.15.91:0/569769352 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c030840 con 0x7f5d540b0220 2024-02-02T13:33:06.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.153+0000 7f5d51400640 1 -- 172.21.15.91:0/569769352 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c0309a0 con 0x7f5d540b0220 2024-02-02T13:33:06.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.153+0000 7f5d51400640 1 -- 172.21.15.91:0/569769352 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c030d30 con 0x7f5d540b0220 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 -- 172.21.15.91:0/569769352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 msgr2=0x7f5d5406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 --2- 172.21.15.91:0/569769352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d5406a110 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c02b980 comp rx=0 tx=0).stop 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 -- 172.21.15.91:0/569769352 shutdown_connections 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 --2- 172.21.15.91:0/569769352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d5406a110 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 --2- 172.21.15.91:0/569769352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5415d650 0x7f5d540ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 -- 172.21.15.91:0/569769352 >> 172.21.15.91:0/569769352 conn(0x7f5d5409a1a0 msgr2=0x7f5d540d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 -- 172.21.15.91:0/569769352 shutdown_connections 2024-02-02T13:33:06.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.154+0000 7f5d59000640 1 -- 172.21.15.91:0/569769352 wait complete. 2024-02-02T13:33:06.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.156+0000 7f5d59000640 1 Processor -- start 2024-02-02T13:33:06.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.156+0000 7f5d59000640 1 -- start start 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5406a650 0x7f5d540c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5415d650 0x7f5d540c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541547d0 con 0x7f5d540b0220 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c6cf0 con 0x7f5d5415d650 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d59000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540b3c90 con 0x7f5d5406a650 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59556/0 (socket says 172.21.15.91:59556) 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.157+0000 7f5d4be00640 1 -- 172.21.15.91:0/1787257606 learned_addr learned my addr 172.21.15.91:0/1787257606 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d52800640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5415d650 0x7f5d540c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:06.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 -- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5406a650 msgr2=0x7f5d540c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5406a650 0x7f5d540c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 -- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5415d650 msgr2=0x7f5d540c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5415d650 0x7f5d540c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 -- 172.21.15.91:0/1787257606 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0094d0 con 0x7f5d540b0220 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4be00640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c039c90 con 0x7f5d540b0220 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c039df0 con 0x7f5d540b0220 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c043640 con 0x7f5d540b0220 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.158+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540b3f70 con 0x7f5d540b0220 2024-02-02T13:33:06.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.159+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d54153bd0 con 0x7f5d540b0220 2024-02-02T13:33:06.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.160+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540b3af0 con 0x7f5d540b0220 2024-02-02T13:33:06.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.160+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5d3c0437a0 con 0x7f5d540b0220 2024-02-02T13:33:06.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.161+0000 7f5d4a000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 0x7f5d2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:33:06.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.161+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c03e070 con 0x7f5d540b0220 2024-02-02T13:33:06.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.162+0000 7f5d51e00640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 0x7f5d2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:33:06.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.162+0000 7f5d51e00640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 0x7f5d2c06c8f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006a90 tx=0x7f5d44006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:33:06.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.166+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5d540b3af0 con 0x7f5d540b0220 2024-02-02T13:33:06.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.357+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f5d540b3af0 con 0x7f5d540b0220 2024-02-02T13:33:06.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.764+0000 7f5d4a000640 1 -- 172.21.15.91:0/1787257606 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v162) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5d540b3af0 con 0x7f5d540b0220 2024-02-02T13:33:06.766 INFO:teuthology.orchestra.run.smithi091.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 msgr2=0x7f5d2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 0x7f5d2c06c8f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006a90 tx=0x7f5d44006a20 comp rx=0 tx=0).stop 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 msgr2=0x7f5d54153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c004730 comp rx=0 tx=0).stop 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 shutdown_connections 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5d2c06a430 0x7f5d2c06c8f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5415d650 0x7f5d540c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d540b0220 0x7f5d54153690 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 --2- 172.21.15.91:0/1787257606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5406a650 0x7f5d540c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:33:06.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.768+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 >> 172.21.15.91:0/1787257606 conn(0x7f5d5409a1a0 msgr2=0x7f5d5406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:33:06.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.769+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 shutdown_connections 2024-02-02T13:33:06.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:33:06.769+0000 7f5d59000640 1 -- 172.21.15.91:0/1787257606 wait complete. 2024-02-02T13:33:06.793 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:33:07.015 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:33:07.015 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:33:07.016 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-02-02T13:33:07.235 INFO:teuthology.orchestra.run.smithi091.stdout:{"mon_tick_interval":"5"} 2024-02-02T13:33:07.235 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-02-02T13:33:07.235 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-02-02T13:33:07.235 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:33:07.235 DEBUG:teuthology.orchestra.run.smithi091:> import time 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> import datetime 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> # Write some buffered data through before going full, all should be well 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> print("writing some data through which we expect to succeed") 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> bytes = 0 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-02-02T13:33:07.236 DEBUG:teuthology.orchestra.run.smithi091:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # buffered IO, the error should always only appear from fsync and not 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # from write 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> full = False 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(32 * 0.9)): 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> print("done writing {0} bytes".format(bytes)) 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # OK, now we should sneak in under the full condition 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # due to the time it takes the OSDs to report to the 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # mons, and get a successful fsync on our full-making data 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> # buffered write, add more dirty data to the buffer 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> print("starting buffered write") 2024-02-02T13:33:07.237 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(32 * 0.2)): 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(15.0) 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> except OSError: 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> pass; 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> # Wait long enough for a background flush that should fail 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(30) 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> if False: 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> # ...and check that the failed background flush is reflected in fclose 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:33:07.238 DEBUG:teuthology.orchestra.run.smithi091:> except OSError: 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> print("close() returned an error as expected") 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("close() failed to raise error") 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> # The kernel cephfs client does not raise errors on fclose 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-02-02T13:33:07.239 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:33:31.413 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:33:31.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:34:01.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:34:01.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:34:31.703 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:34:31.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:35:01.842 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:35:01.850 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:writing some data through which we expect to succeed 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-02-02T13:35:10.563 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-02-02T13:35:10.564 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:done writing 29884416 bytes 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:successfully fsync'ed prior to getting full state reported 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:starting buffered write 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.565 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.566 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:35:10.566 INFO:teuthology.orchestra.run.smithi091.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-02-02T13:35:10.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:35:10.759 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:35:10.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:35:11.002 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:35:11.002 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:11.002 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:11.060 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:11.060 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:11.112 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:11.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.316+0000 7ff57a200640 1 Processor -- start 2024-02-02T13:35:11.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.316+0000 7ff57a200640 1 -- start start 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.317+0000 7ff57a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5740b0e10 0x7ff5740c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff57a200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740cc7c0 con 0x7ff57415a010 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff57a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740ce930 con 0x7ff5740ce130 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff57a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740cd640 con 0x7ff5740b0e10 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff573400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff57415a010 0x7ff5740ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41382/0 (socket says 172.21.15.91:41382) 2024-02-02T13:35:11.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff573400640 1 -- 172.21.15.91:0/3360160120 learned_addr learned my addr 172.21.15.91:0/3360160120 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:11.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff573e00640 1 --2- 172.21.15.91:0/3360160120 >> v2:172.21.15.113:3300/0 conn(0x7ff5740b0e10 0x7ff5740c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261306041 0 0) 0x7ff5740cc7c0 con 0x7ff57415a010 2024-02-02T13:35:11.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.318+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff554003520 con 0x7ff57415a010 2024-02-02T13:35:11.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823218884 0 0) 0x7ff554003520 con 0x7ff57415a010 2024-02-02T13:35:11.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 >> v2:172.21.15.113:3300/0 conn(0x7ff5740b0e10 msgr2=0x7ff5740c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 --2- 172.21.15.91:0/3360160120 >> v2:172.21.15.113:3300/0 conn(0x7ff5740b0e10 0x7ff5740c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 >> v1:172.21.15.113:6789/0 conn(0x7ff5740ce130 legacy=0x7ff5740b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57406c890 con 0x7ff57415a010 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff564001670 con 0x7ff57415a010 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 >> v1:172.21.15.91:6789/0 conn(0x7ff57415a010 legacy=0x7ff5740ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 0x7ff554003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5540047c0 0x7ff554004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554005180 0x7ff55400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cd640 con 0x7ff554003970 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55400c120 con 0x7ff554005180 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55400c2e0 con 0x7ff5540047c0 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.319+0000 7ff573400640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 0x7ff554003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573e00640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554005180 0x7ff55400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573400640 1 -- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5540047c0 msgr2=0x7ff554004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:11.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573400640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5540047c0 0x7ff554004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573400640 1 -- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554005180 msgr2=0x7ff55400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:11.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573400640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554005180 0x7ff55400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573e00640 1 -- 172.21.15.91:0/3360160120 reap_dead start 2024-02-02T13:35:11.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.320+0000 7ff573400640 1 -- 172.21.15.91:0/3360160120 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554003520 con 0x7ff554003970 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.321+0000 7ff573400640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 0x7ff554003d90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c240 tx=0x7ff5540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b30b0158650806 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.321+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564005ad0 con 0x7ff554003970 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.321+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff564005c50 con 0x7ff554003970 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.321+0000 7ff572a00640 1 -- 172.21.15.91:0/3360160120 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56402c850 con 0x7ff554003970 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 -- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 msgr2=0x7ff554003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 0x7ff554003d90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c240 tx=0x7ff5540057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 -- 172.21.15.91:0/3360160120 shutdown_connections 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 --2- 172.21.15.91:0/3360160120 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff554003970 0x7ff554003d90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 -- 172.21.15.91:0/3360160120 >> 172.21.15.91:0/3360160120 conn(0x7ff5740cec80 msgr2=0x7ff5740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:11.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 -- 172.21.15.91:0/3360160120 shutdown_connections 2024-02-02T13:35:11.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.322+0000 7ff57a200640 1 -- 172.21.15.91:0/3360160120 wait complete. 2024-02-02T13:35:11.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.324+0000 7ff57a200640 1 Processor -- start 2024-02-02T13:35:11.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.324+0000 7ff57a200640 1 -- start start 2024-02-02T13:35:11.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554003970 0x7ff5741f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5740b4360 0x7ff5740c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740c5cc0 con 0x7ff5740c6980 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b47e0 con 0x7ff554003970 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff57a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740c64f0 con 0x7ff5740b4360 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46624/0 (socket says 172.21.15.91:46624) 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 -- 172.21.15.91:0/1740029250 learned_addr learned my addr 172.21.15.91:0/1740029250 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff573400640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554003970 0x7ff5741f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff573e00640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5740b4360 0x7ff5740c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 -- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5740b4360 msgr2=0x7ff5740c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5740b4360 0x7ff5740c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 -- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554003970 msgr2=0x7ff5741f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554003970 0x7ff5741f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.325+0000 7ff56a800640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740c3d50 con 0x7ff5740c6980 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff56a800640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00c3c0 tx=0x7ff55c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0052d0 con 0x7ff5740c6980 2024-02-02T13:35:11.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c005430 con 0x7ff5740c6980 2024-02-02T13:35:11.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c003830 con 0x7ff5740c6980 2024-02-02T13:35:11.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5741f2880 con 0x7ff5740c6980 2024-02-02T13:35:11.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.326+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5740c5ea0 con 0x7ff5740c6980 2024-02-02T13:35:11.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.327+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff574066cf0 con 0x7ff5740c6980 2024-02-02T13:35:11.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.328+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff55c003990 con 0x7ff5740c6980 2024-02-02T13:35:11.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.329+0000 7ff56be00640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 0x7ff54c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.329+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff55c087e70 con 0x7ff5740c6980 2024-02-02T13:35:11.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.329+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7ff54c072050 con 0x7ff5740c6980 2024-02-02T13:35:11.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.329+0000 7ff573400640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 0x7ff54c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.330+0000 7ff573400640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 0x7ff54c06c920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff564034000 tx=0x7ff564009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.334+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff55c053410 con 0x7ff5740c6980 2024-02-02T13:35:11.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.527+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5740b4c00 con 0x7ff5740c6980 2024-02-02T13:35:11.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.533+0000 7ff56be00640 1 -- 172.21.15.91:0/1740029250 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16235 (secure 0 0 0) 0x7ff55c017020 con 0x7ff5740c6980 2024-02-02T13:35:11.535 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:11.535 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":165,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:34:29.471448+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.535+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 msgr2=0x7ff54c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.535+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 0x7ff54c06c920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff564034000 tx=0x7ff564009030 comp rx=0 tx=0).stop 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 msgr2=0x7ff5740b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00c3c0 tx=0x7ff55c001e70 comp rx=0 tx=0).stop 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 shutdown_connections 2024-02-02T13:35:11.537 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff54c06a460 0x7ff54c06c920 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5740b4360 0x7ff5740c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5740c6980 0x7ff5740b3e20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 --2- 172.21.15.91:0/1740029250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff554003970 0x7ff5741f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 >> 172.21.15.91:0/1740029250 conn(0x7ff5740cec80 msgr2=0x7ff57406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 shutdown_connections 2024-02-02T13:35:11.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.536+0000 7ff57a200640 1 -- 172.21.15.91:0/1740029250 wait complete. 2024-02-02T13:35:11.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:32:53.511243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:32:54.111607+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:35:11.556 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:35:11.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.766+0000 7fdcf7a00640 1 Processor -- start 2024-02-02T13:35:11.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.766+0000 7fdcf7a00640 1 -- start start 2024-02-02T13:35:11.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdcf7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdcf0160a20 0x7fdcf00b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.769 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 0x7fdcf00b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdcf7a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf00b7730 con 0x7fdcf00b4ba0 2024-02-02T13:35:11.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdcf7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf00b85e0 con 0x7fdcf0160a20 2024-02-02T13:35:11.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00b9140 con 0x7fdcf00b5010 2024-02-02T13:35:11.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdce7c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdcf00b4ba0 0x7fdcf00d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41384/0 (socket says 172.21.15.91:41384) 2024-02-02T13:35:11.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.768+0000 7fdce7c00640 1 -- 172.21.15.91:0/1330332784 learned_addr learned my addr 172.21.15.91:0/1330332784 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5000640 1 --2- 172.21.15.91:0/1330332784 >> v2:172.21.15.113:3300/0 conn(0x7fdcf0160a20 0x7fdcf00b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5a00640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 0x7fdcf00b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674545794 0 0) 0x7fdcf00b7730 con 0x7fdcf00b4ba0 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd0003520 con 0x7fdcf00b4ba0 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5000640 1 -- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 msgr2=0x7fdcf00b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5000640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 0x7fdcf00b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5000640 1 -- 172.21.15.91:0/1330332784 >> v1:172.21.15.91:6789/0 conn(0x7fdcf00b4ba0 legacy=0x7fdcf00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755697149 0 0) 0x7fdcd0003520 con 0x7fdcf00b4ba0 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5000640 1 -- 172.21.15.91:0/1330332784 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00b7960 con 0x7fdcf0160a20 2024-02-02T13:35:11.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.769+0000 7fdcf5a00640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 0x7fdcf00b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:35:11.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.771+0000 7fdcf5000640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b8c00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdce00097d0 tx=0x7fdce002b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575f5bbc8d835a35 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.771+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80019f0 con 0x7fdcf0160a20 2024-02-02T13:35:11.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.771+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce002e050 con 0x7fdcf0160a20 2024-02-02T13:35:11.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.771+0000 7fdcefe00640 1 -- 172.21.15.91:0/1330332784 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0030730 con 0x7fdcf0160a20 2024-02-02T13:35:11.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 -- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf0160a20 msgr2=0x7fdcf00b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b8c00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdce00097d0 tx=0x7fdce002b770 comp rx=0 tx=0).stop 2024-02-02T13:35:11.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 -- 172.21.15.91:0/1330332784 shutdown_connections 2024-02-02T13:35:11.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b5010 0x7fdcf00b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/1330332784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b8c00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 -- 172.21.15.91:0/1330332784 >> 172.21.15.91:0/1330332784 conn(0x7fdcf0070480 msgr2=0x7fdcf0070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:11.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 -- 172.21.15.91:0/1330332784 shutdown_connections 2024-02-02T13:35:11.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.772+0000 7fdcf7a00640 1 -- 172.21.15.91:0/1330332784 wait complete. 2024-02-02T13:35:11.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.774+0000 7fdcf7a00640 1 Processor -- start 2024-02-02T13:35:11.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.774+0000 7fdcf7a00640 1 -- start start 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 0x7fdcf015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00b5010 0x7fdcf00ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00cb830 con 0x7fdcf0160a20 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00cacb0 con 0x7fdcf00b5010 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00caf90 con 0x7fdcf00b4ba0 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf5a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf5a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46634/0 (socket says 172.21.15.91:46634) 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf5a00640 1 -- 172.21.15.91:0/649530132 learned_addr learned my addr 172.21.15.91:0/649530132 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:11.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.775+0000 7fdcf5000640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 0x7fdcf015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdce7c00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00b5010 0x7fdcf00ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 -- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 msgr2=0x7fdcf015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 0x7fdcf015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 -- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00b5010 msgr2=0x7fdcf00ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00b5010 0x7fdcf00ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009400 con 0x7fdcf0160a20 2024-02-02T13:35:11.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5000640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 0x7fdcf015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf5a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e930 tx=0x7fdce800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8009700 con 0x7fdcf0160a20 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce800ca00 con 0x7fdcf0160a20 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800cd90 con 0x7fdcf0160a20 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.776+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce0009400 con 0x7fdcf0160a20 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.777+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf00cb550 con 0x7fdcf0160a20 2024-02-02T13:35:11.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.778+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf01f26e0 con 0x7fdcf0160a20 2024-02-02T13:35:11.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.779+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdce8013470 con 0x7fdcf0160a20 2024-02-02T13:35:11.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.780+0000 7fdced600640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:11.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.780+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdce8088680 con 0x7fdcf0160a20 2024-02-02T13:35:11.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.780+0000 7fdcf5000640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:11.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.780+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fdcc8072020 con 0x7fdcf0160a20 2024-02-02T13:35:11.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.781+0000 7fdcf5000640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdce0034000 tx=0x7fdce0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:11.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.785+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdcf01f26e0 con 0x7fdcf0160a20 2024-02-02T13:35:11.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.976+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcf01f26e0 con 0x7fdcf0160a20 2024-02-02T13:35:11.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.977+0000 7fdced600640 1 -- 172.21.15.91:0/649530132 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcf01f26e0 con 0x7fdcf0160a20 2024-02-02T13:35:11.978 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.979+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 msgr2=0x7fdcc806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.979+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdce0034000 tx=0x7fdce0008030 comp rx=0 tx=0).stop 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.979+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 msgr2=0x7fdcf00b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.979+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e930 tx=0x7fdce800edf0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 shutdown_connections 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdcc806a430 0x7fdcc806c8f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcf0160a20 0x7fdcf00b3ad0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00b5010 0x7fdcf00ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 --2- 172.21.15.91:0/649530132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00b4ba0 0x7fdcf015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:11.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 >> 172.21.15.91:0/649530132 conn(0x7fdcf0070480 msgr2=0x7fdcf00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:11.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 shutdown_connections 2024-02-02T13:35:11.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:11.980+0000 7fdcf7a00640 1 -- 172.21.15.91:0/649530132 wait complete. 2024-02-02T13:35:12.001 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:35:12.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.211+0000 7f78c5200640 1 Processor -- start 2024-02-02T13:35:12.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.211+0000 7f78c5200640 1 -- start start 2024-02-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78c5200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78c00b0e10 0x7f78c006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78c5200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c015d870 con 0x7f78c015d490 2024-02-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78c5200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c006bce0 con 0x7f78c00acfe0 2024-02-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78c5200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c0161220 con 0x7f78c00b0e10 2024-02-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78bde00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f78c015d490 0x7f78c00ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41392/0 (socket says 172.21.15.91:41392) 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78bde00640 1 -- 172.21.15.91:0/1608774214 learned_addr learned my addr 172.21.15.91:0/1608774214 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.213+0000 7f78be800640 1 --2- 172.21.15.91:0/1608774214 >> v2:172.21.15.91:3300/0 conn(0x7f78c00b0e10 0x7f78c006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.214+0000 7f78be800640 1 -- 172.21.15.91:0/1608774214 >> v1:172.21.15.113:6789/0 conn(0x7f78c00acfe0 legacy=0x7f78c00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.214+0000 7f78be800640 1 -- 172.21.15.91:0/1608774214 >> v1:172.21.15.91:6789/0 conn(0x7f78c015d490 legacy=0x7f78c00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.214+0000 7f78be800640 1 -- 172.21.15.91:0/1608774214 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c00ad490 con 0x7f78c00b0e10 2024-02-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.214+0000 7f78bd400640 1 -- 172.21.15.91:0/1608774214 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476368145 0 0) 0x7f78c015d870 con 0x7f78c015d490 2024-02-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.215+0000 7f78be800640 1 --2- 172.21.15.91:0/1608774214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c006c170 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd9f39ea8ad366c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.215+0000 7f78bd400640 1 -- 172.21.15.91:0/1608774214 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b802cd00 con 0x7f78c00b0e10 2024-02-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.215+0000 7f78bd400640 1 -- 172.21.15.91:0/1608774214 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b802ce60 con 0x7f78c00b0e10 2024-02-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.215+0000 7f78bd400640 1 -- 172.21.15.91:0/1608774214 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8034660 con 0x7f78c00b0e10 2024-02-02T13:35:12.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 -- 172.21.15.91:0/1608774214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 msgr2=0x7f78c006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 --2- 172.21.15.91:0/1608774214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c006c170 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009d90 tx=0x7f78b802bf50 comp rx=0 tx=0).stop 2024-02-02T13:35:12.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 -- 172.21.15.91:0/1608774214 shutdown_connections 2024-02-02T13:35:12.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 --2- 172.21.15.91:0/1608774214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c006c170 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 -- 172.21.15.91:0/1608774214 >> 172.21.15.91:0/1608774214 conn(0x7f78c00cec80 msgr2=0x7f78c00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:12.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 -- 172.21.15.91:0/1608774214 shutdown_connections 2024-02-02T13:35:12.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.216+0000 7f78c5200640 1 -- 172.21.15.91:0/1608774214 wait complete. 2024-02-02T13:35:12.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.217+0000 7f78c5200640 1 Processor -- start 2024-02-02T13:35:12.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 -- start start 2024-02-02T13:35:12.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78c00acfe0 0x7f78c0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78c015d490 0x7f78c00b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00b3a70 con 0x7f78c00b0e10 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00b4680 con 0x7f78c00acfe0 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.218+0000 7f78c5200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00b2e60 con 0x7f78c015d490 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46638/0 (socket says 172.21.15.91:46638) 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 -- 172.21.15.91:0/1567785094 learned_addr learned my addr 172.21.15.91:0/1567785094 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78bde00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78c00acfe0 0x7f78c0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78be800640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78c015d490 0x7f78c00b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 -- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78c015d490 msgr2=0x7f78c00b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78c015d490 0x7f78c00b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 -- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78c00acfe0 msgr2=0x7f78c0157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78c00acfe0 0x7f78c0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.219+0000 7f78b7e00640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8009a70 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78b7e00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f78a800c430 tx=0x7f78a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80052d0 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8005430 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8003830 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c00b3140 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.220+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c00b3700 con 0x7f78c00b0e10 2024-02-02T13:35:12.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.221+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c00b38c0 con 0x7f78c00b0e10 2024-02-02T13:35:12.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.222+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f78a8003990 con 0x7f78c00b0e10 2024-02-02T13:35:12.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.223+0000 7f78b6000640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 0x7f789806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.223+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f78a8087ff0 con 0x7f78c00b0e10 2024-02-02T13:35:12.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.223+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f7898072020 con 0x7f78c00b0e10 2024-02-02T13:35:12.224 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.223+0000 7f78bde00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 0x7f789806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.225 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.224+0000 7f78bde00640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 0x7f789806c8f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f78b00011e0 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.228+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f78a80baca0 con 0x7f78c00b0e10 2024-02-02T13:35:12.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.409+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78c0066cf0 con 0x7f78c00b0e10 2024-02-02T13:35:12.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.410+0000 7f78b6000640 1 -- 172.21.15.91:0/1567785094 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78a8017020 con 0x7f78c00b0e10 2024-02-02T13:35:12.412 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:35:12.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 msgr2=0x7f789806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 0x7f789806c8f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f78b00011e0 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-02-02T13:35:12.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 msgr2=0x7f78c00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f78a800c430 tx=0x7f78a800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 shutdown_connections 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f789806a430 0x7f789806c8f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78c015d490 0x7f78c00b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78c00b0e10 0x7f78c00b2280 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 --2- 172.21.15.91:0/1567785094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78c00acfe0 0x7f78c0157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 >> 172.21.15.91:0/1567785094 conn(0x7f78c00cec80 msgr2=0x7f78c00b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 shutdown_connections 2024-02-02T13:35:12.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.413+0000 7f78c5200640 1 -- 172.21.15.91:0/1567785094 wait complete. 2024-02-02T13:35:12.432 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:35:12.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.637+0000 7f80f8600640 1 Processor -- start 2024-02-02T13:35:12.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.637+0000 7f80f8600640 1 -- start start 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.638+0000 7f80f8600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f80f015f5a0 0x7f80f00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.638+0000 7f80f8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f00b0c40 0x7f80f00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f8600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f0070f80 con 0x7f80f0151f90 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f8600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f015fa30 con 0x7f80f015f5a0 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b10b0 con 0x7f80f00b0c40 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f80f015f5a0 0x7f80f00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f80f0151f90 0x7f80f006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41394/0 (socket says 172.21.15.91:41394) 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5c00640 1 -- 172.21.15.91:0/865508595 learned_addr learned my addr 172.21.15.91:0/865508595 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f80f015f5a0 0x7f80f00b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46642/0 (socket says 172.21.15.91:46642) 2024-02-02T13:35:12.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f6600640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f00b0c40 0x7f80f00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80efe00640 1 -- 172.21.15.91:0/865508595 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479572248 0 0) 0x7f80f0070f80 con 0x7f80f0151f90 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80efe00640 1 -- 172.21.15.91:0/865508595 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80d0003520 con 0x7f80f0151f90 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 -- 172.21.15.91:0/865508595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f00b0c40 msgr2=0x7f80f00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f00b0c40 0x7f80f00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 -- 172.21.15.91:0/865508595 >> v1:172.21.15.91:6789/0 conn(0x7f80f0151f90 legacy=0x7f80f006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.639+0000 7f80f5200640 1 -- 172.21.15.91:0/865508595 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f0152370 con 0x7f80f015f5a0 2024-02-02T13:35:12.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80f5200640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f015f5a0 0x7f80f00b0700 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f80d80097a0 tx=0x7f80d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907e295dd3a56eee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80efe00640 1 -- 172.21.15.91:0/865508595 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d802d040 con 0x7f80f015f5a0 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80efe00640 1 -- 172.21.15.91:0/865508595 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8031810 con 0x7f80f015f5a0 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80efe00640 1 -- 172.21.15.91:0/865508595 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8031ba0 con 0x7f80f015f5a0 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80f8600640 1 -- 172.21.15.91:0/865508595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f015f5a0 msgr2=0x7f80f00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.641+0000 7f80f8600640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f015f5a0 0x7f80f00b0700 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f80d80097a0 tx=0x7f80d802b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 -- 172.21.15.91:0/865508595 shutdown_connections 2024-02-02T13:35:12.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f00b0c40 0x7f80f00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 --2- 172.21.15.91:0/865508595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f015f5a0 0x7f80f00b0700 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 -- 172.21.15.91:0/865508595 >> 172.21.15.91:0/865508595 conn(0x7f80f00cec80 msgr2=0x7f80f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:12.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 -- 172.21.15.91:0/865508595 shutdown_connections 2024-02-02T13:35:12.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.642+0000 7f80f8600640 1 -- 172.21.15.91:0/865508595 wait complete. 2024-02-02T13:35:12.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.643+0000 7f80f8600640 1 Processor -- start 2024-02-02T13:35:12.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.644+0000 7f80f8600640 1 -- start start 2024-02-02T13:35:12.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.644+0000 7f80f8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f00b0c40 0x7f80f00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.644+0000 7f80f8600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f015f5a0 0x7f80f01f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f8600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b1e10 con 0x7f80f0151f90 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b20f0 con 0x7f80f00b0c40 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b23d0 con 0x7f80f015f5a0 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46644/0 (socket says 172.21.15.91:46644) 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 -- 172.21.15.91:0/4219110311 learned_addr learned my addr 172.21.15.91:0/4219110311 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 -- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f015f5a0 msgr2=0x7f80f01f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f015f5a0 0x7f80f01f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 -- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f00b0c40 msgr2=0x7f80f00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f00b0c40 0x7f80f00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8009400 con 0x7f80f0151f90 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.645+0000 7f80f5200640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f80d8000c00 tx=0x7f80d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.646+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8006e10 con 0x7f80f0151f90 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.646+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d803ad60 con 0x7f80f0151f90 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.646+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8044630 con 0x7f80f0151f90 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.646+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f00b26b0 con 0x7f80f0151f90 2024-02-02T13:35:12.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.646+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f00b2eb0 con 0x7f80f0151f90 2024-02-02T13:35:12.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.647+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00b2bd0 con 0x7f80f0151f90 2024-02-02T13:35:12.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.648+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f80d804d450 con 0x7f80f0151f90 2024-02-02T13:35:12.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.649+0000 7f80ed600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 0x7f80c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:12.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.649+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f80d80aac30 con 0x7f80f0151f90 2024-02-02T13:35:12.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.649+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f80c8072020 con 0x7f80f0151f90 2024-02-02T13:35:12.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.649+0000 7f80f5c00640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 0x7f80c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:12.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.649+0000 7f80f5c00640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 0x7f80c806c8f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80f00c4390 tx=0x7f80e0007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:12.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.653+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f80f00b2bd0 con 0x7f80f0151f90 2024-02-02T13:35:12.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.840+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80f00b2bd0 con 0x7f80f0151f90 2024-02-02T13:35:12.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.841+0000 7f80ed600640 1 -- 172.21.15.91:0/4219110311 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80f00b2bd0 con 0x7f80f0151f90 2024-02-02T13:35:12.843 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:35:12.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 msgr2=0x7f80c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 0x7f80c806c8f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80f00c4390 tx=0x7f80e0007930 comp rx=0 tx=0).stop 2024-02-02T13:35:12.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 msgr2=0x7f80f01f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f80d8000c00 tx=0x7f80d80046c0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 shutdown_connections 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f80c806a430 0x7f80c806c8f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f015f5a0 0x7f80f01f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80f0151f90 0x7f80f01f3b30 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 --2- 172.21.15.91:0/4219110311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f00b0c40 0x7f80f00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 >> 172.21.15.91:0/4219110311 conn(0x7f80f00cec80 msgr2=0x7f80f006f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 shutdown_connections 2024-02-02T13:35:12.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:12.844+0000 7f80f8600640 1 -- 172.21.15.91:0/4219110311 wait complete. 2024-02-02T13:35:12.862 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:35:13.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.056+0000 7ff8f9600640 1 Processor -- start 2024-02-02T13:35:13.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.056+0000 7ff8f9600640 1 -- start start 2024-02-02T13:35:13.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.057+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.057+0000 7ff8f9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8f40cb580 0x7ff8f40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.057+0000 7ff8f9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f406d910 con 0x7ff8f415b0f0 2024-02-02T13:35:13.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.057+0000 7ff8f9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f406d730 con 0x7ff8f40cb580 2024-02-02T13:35:13.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.057+0000 7ff8f9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40cbf50 con 0x7ff8f415ad10 2024-02-02T13:35:13.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.058+0000 7ff8f2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.058+0000 7ff8f3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8f40cb580 0x7ff8f40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.058+0000 7ff8f2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:51224/0 (socket says 172.21.15.91:51224) 2024-02-02T13:35:13.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.058+0000 7ff8f2a00640 1 -- 172.21.15.91:0/2765377384 learned_addr learned my addr 172.21.15.91:0/2765377384 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:13.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.059+0000 7ff8f2a00640 1 -- 172.21.15.91:0/2765377384 >> v2:172.21.15.113:3300/0 conn(0x7ff8f40cb580 msgr2=0x7ff8f40cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:13.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.059+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/2765377384 >> v2:172.21.15.113:3300/0 conn(0x7ff8f40cb580 0x7ff8f40cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.059+0000 7ff8f2a00640 1 -- 172.21.15.91:0/2765377384 >> v1:172.21.15.113:6789/0 conn(0x7ff8f415b0f0 legacy=0x7ff8f406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:13.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.059+0000 7ff8f2a00640 1 -- 172.21.15.91:0/2765377384 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f40cc850 con 0x7ff8f415ad10 2024-02-02T13:35:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.060+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/2765377384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf0ad9b9c6b1c89 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.061+0000 7ff8f2000640 1 -- 172.21.15.91:0/2765377384 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4031640 con 0x7ff8f415ad10 2024-02-02T13:35:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.061+0000 7ff8f2000640 1 -- 172.21.15.91:0/2765377384 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40317a0 con 0x7ff8f415ad10 2024-02-02T13:35:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.061+0000 7ff8f2000640 1 -- 172.21.15.91:0/2765377384 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4031b30 con 0x7ff8f415ad10 2024-02-02T13:35:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 -- 172.21.15.91:0/2765377384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 msgr2=0x7ff8f406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 --2- 172.21.15.91:0/2765377384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 -- 172.21.15.91:0/2765377384 shutdown_connections 2024-02-02T13:35:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 --2- 172.21.15.91:0/2765377384 >> v2:172.21.15.113:3300/0 conn(0x7ff8f40cb580 0x7ff8f40cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 --2- 172.21.15.91:0/2765377384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415ad10 0x7ff8f406b7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 -- 172.21.15.91:0/2765377384 >> 172.21.15.91:0/2765377384 conn(0x7ff8f40b0700 msgr2=0x7ff8f40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 -- 172.21.15.91:0/2765377384 shutdown_connections 2024-02-02T13:35:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.062+0000 7ff8f9600640 1 -- 172.21.15.91:0/2765377384 wait complete. 2024-02-02T13:35:13.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.064+0000 7ff8f9600640 1 Processor -- start 2024-02-02T13:35:13.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.064+0000 7ff8f9600640 1 -- start start 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 0x7ff8f41f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415b0f0 0x7ff8f40c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40af200 con 0x7ff8f415ad10 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41f24d0 con 0x7ff8f40cb580 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8f9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40ae330 con 0x7ff8f415b0f0 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.065+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46656/0 (socket says 172.21.15.91:46656) 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 -- 172.21.15.91:0/3639010235 learned_addr learned my addr 172.21.15.91:0/3639010235 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:13.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 0x7ff8f41f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8f3400640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415b0f0 0x7ff8f40c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 -- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415b0f0 msgr2=0x7ff8f40c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415b0f0 0x7ff8f40c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 -- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 msgr2=0x7ff8f41f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 0x7ff8f41f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40094d0 con 0x7ff8f415ad10 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 0x7ff8f41f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8ebe00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b3f0 tx=0x7ff8dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc011830 con 0x7ff8f415ad10 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc011990 con 0x7ff8f415ad10 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.066+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc011d40 con 0x7ff8f415ad10 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.067+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f40caae0 con 0x7ff8f415ad10 2024-02-02T13:35:13.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.067+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f40ae860 con 0x7ff8f415ad10 2024-02-02T13:35:13.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.068+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f4066cf0 con 0x7ff8f415ad10 2024-02-02T13:35:13.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.068+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff8dc01a400 con 0x7ff8f415ad10 2024-02-02T13:35:13.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.069+0000 7ff8eaa00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 0x7ff8cc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.069+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff8dc088820 con 0x7ff8f415ad10 2024-02-02T13:35:13.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.070+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 0x7ff8cc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.070+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7ff8cc072050 con 0x7ff8f415ad10 2024-02-02T13:35:13.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.071+0000 7ff8f2a00640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 0x7ff8cc06c920 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036000 tx=0x7ff8e403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.075+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8dc053d40 con 0x7ff8f415ad10 2024-02-02T13:35:13.272 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.271+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8f40c3b00 con 0x7ff8f415ad10 2024-02-02T13:35:13.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.272+0000 7ff8eaa00640 1 -- 172.21.15.91:0/3639010235 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v165) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff8dc015080 con 0x7ff8f415ad10 2024-02-02T13:35:13.273 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 msgr2=0x7ff8cc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 0x7ff8cc06c920 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036000 tx=0x7ff8e403c040 comp rx=0 tx=0).stop 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 msgr2=0x7ff8f40ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b3f0 tx=0x7ff8dc00b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 shutdown_connections 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff8cc06a460 0x7ff8cc06c920 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8f415b0f0 0x7ff8f40c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8f415ad10 0x7ff8f40ca5a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 --2- 172.21.15.91:0/3639010235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8f40cb580 0x7ff8f41f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 >> 172.21.15.91:0/3639010235 conn(0x7ff8f40b0700 msgr2=0x7ff8f4070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:13.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 shutdown_connections 2024-02-02T13:35:13.277 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.274+0000 7ff8f9600640 1 -- 172.21.15.91:0/3639010235 wait complete. 2024-02-02T13:35:13.293 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:35:13.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:13.294 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:35:13.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:35:13.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:13.388 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:35:13.433 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:13.434 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:13.490 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:13.490 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:13.543 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:13.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.752+0000 7f1b1c400640 1 Processor -- start 2024-02-02T13:35:13.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.752+0000 7f1b1c400640 1 -- start start 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.753+0000 7f1b1c400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b1415d490 0x7f1b1406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.753+0000 7f1b1c400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1415d870 con 0x7f1b140ac8d0 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.753+0000 7f1b1c400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1406bce0 con 0x7f1b140b0e10 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.753+0000 7f1b1c400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b14161220 con 0x7f1b1415d490 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b19000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1b140ac8d0 0x7f1b140b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41398/0 (socket says 172.21.15.91:41398) 2024-02-02T13:35:13.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b19000640 1 -- 172.21.15.91:0/1471739077 learned_addr learned my addr 172.21.15.91:0/1471739077 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:13.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1471739077 >> v2:172.21.15.113:3300/0 conn(0x7f1b1415d490 0x7f1b1406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761849660 0 0) 0x7f1b1415d870 con 0x7f1b140ac8d0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4003520 con 0x7f1b140ac8d0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1322024457 0 0) 0x7f1af4003520 con 0x7f1b140ac8d0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 >> v2:172.21.15.113:3300/0 conn(0x7f1b1415d490 msgr2=0x7f1b1406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 --2- 172.21.15.91:0/1471739077 >> v2:172.21.15.113:3300/0 conn(0x7f1b1415d490 0x7f1b1406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 >> v1:172.21.15.113:6789/0 conn(0x7f1b140b0e10 legacy=0x7f1b140c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140ad490 con 0x7f1b140ac8d0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.754+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1afc001670 con 0x7f1b140ac8d0 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 >> v1:172.21.15.91:6789/0 conn(0x7f1b140ac8d0 legacy=0x7f1b140b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1af4005180 0x7f1af400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14161220 con 0x7f1af40039c0 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400c080 con 0x7f1af4005180 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400c240 con 0x7f1af40047c0 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b19000640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b1a400640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1af4005180 0x7f1af400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b19a00640 1 -- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40047c0 msgr2=0x7f1af4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.755+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.756+0000 7f1b19a00640 1 -- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1af4005180 msgr2=0x7f1af400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.756+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1af4005180 0x7f1af400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.756+0000 7f1b1a400640 1 -- 172.21.15.91:0/1471739077 reap_dead start 2024-02-02T13:35:13.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.756+0000 7f1b19a00640 1 -- 172.21.15.91:0/1471739077 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4003520 con 0x7f1af40039c0 2024-02-02T13:35:13.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.757+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009ac0 tx=0x7f1af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40da69f08384cd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.757+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0402c6b0 con 0x7f1af40039c0 2024-02-02T13:35:13.760 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.757+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0402c810 con 0x7f1af40039c0 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.757+0000 7f1b13e00640 1 -- 172.21.15.91:0/1471739077 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0402cba0 con 0x7f1af40039c0 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 -- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 msgr2=0x7f1af4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009ac0 tx=0x7f1af40057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 -- 172.21.15.91:0/1471739077 shutdown_connections 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1471739077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 -- 172.21.15.91:0/1471739077 >> 172.21.15.91:0/1471739077 conn(0x7f1b140cec80 msgr2=0x7f1b140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 -- 172.21.15.91:0/1471739077 shutdown_connections 2024-02-02T13:35:13.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.758+0000 7f1b1c400640 1 -- 172.21.15.91:0/1471739077 wait complete. 2024-02-02T13:35:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.760+0000 7f1b1c400640 1 Processor -- start 2024-02-02T13:35:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.760+0000 7f1b1c400640 1 -- start start 2024-02-02T13:35:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40039c0 0x7f1b1415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b140c5ff0 0x7f1b140af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140c7410 con 0x7f1b140c6a20 2024-02-02T13:35:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141f61d0 con 0x7f1b140c5ff0 2024-02-02T13:35:13.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1c400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141f64b0 con 0x7f1af40039c0 2024-02-02T13:35:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.761+0000 7f1b1a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46672/0 (socket says 172.21.15.91:46672) 2024-02-02T13:35:13.764 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 -- 172.21.15.91:0/1997103719 learned_addr learned my addr 172.21.15.91:0/1997103719 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b19000640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b140c5ff0 0x7f1b140af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40039c0 0x7f1b1415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 -- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40039c0 msgr2=0x7f1b1415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40039c0 0x7f1b1415c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 -- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b140c5ff0 msgr2=0x7f1b140af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b140c5ff0 0x7f1b140af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b141f6790 con 0x7f1b140c6a20 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b1a400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1b1406d7a0 tx=0x7f1b0c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.762+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c006d60 con 0x7f1b140c6a20 2024-02-02T13:35:13.765 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.763+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c003970 con 0x7f1b140c6a20 2024-02-02T13:35:13.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.763+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c003d20 con 0x7f1b140c6a20 2024-02-02T13:35:13.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.763+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b141f6a70 con 0x7f1b140c6a20 2024-02-02T13:35:13.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.763+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1415c9e0 con 0x7f1b140c6a20 2024-02-02T13:35:13.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.764+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b141f6f90 con 0x7f1b140c6a20 2024-02-02T13:35:13.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.765+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1b0c005110 con 0x7f1b140c6a20 2024-02-02T13:35:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.766+0000 7f1b11600640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 0x7f1aec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.766+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1b0c087dc0 con 0x7f1b140c6a20 2024-02-02T13:35:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.766+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 0x7f1aec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:13.767 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.766+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f1aec072020 con 0x7f1b140c6a20 2024-02-02T13:35:13.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.766+0000 7f1b19a00640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 0x7f1aec06c8f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1b04036000 tx=0x7f1b04030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:13.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.770+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1b0c019060 con 0x7f1b140c6a20 2024-02-02T13:35:13.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.961+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b14066cf0 con 0x7f1b140c6a20 2024-02-02T13:35:13.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.963+0000 7f1b11600640 1 -- 172.21.15.91:0/1997103719 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16235 (secure 0 0 0) 0x7f1b141f6f90 con 0x7f1b140c6a20 2024-02-02T13:35:13.965 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:13.965 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":165,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:34:29.471448+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 msgr2=0x7f1aec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 0x7f1aec06c8f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1b04036000 tx=0x7f1b04030000 comp rx=0 tx=0).stop 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 msgr2=0x7f1b140c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1b1406d7a0 tx=0x7f1b0c00e910 comp rx=0 tx=0).stop 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 shutdown_connections 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1aec06a430 0x7f1aec06c8f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1b140c6a20 0x7f1b140c6ed0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b140c5ff0 0x7f1b140af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 --2- 172.21.15.91:0/1997103719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1af40039c0 0x7f1b1415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:13.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.965+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 >> 172.21.15.91:0/1997103719 conn(0x7f1b140cec80 msgr2=0x7f1b14157160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:13.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.966+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 shutdown_connections 2024-02-02T13:35:13.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:13.966+0000 7f1b1c400640 1 -- 172.21.15.91:0/1997103719 wait complete. 2024-02-02T13:35:13.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:32:53.511243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:32:54.111607+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:35:13.986 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:35:14.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.198+0000 7f208dc00640 1 Processor -- start 2024-02-02T13:35:14.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.198+0000 7f208dc00640 1 -- start start 2024-02-02T13:35:14.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f208dc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f208815cee0 0x7f20880b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f208dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20880b6990 0x7f20880cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f208dc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880b7d40 con 0x7f20880ce000 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f208dc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20880ce400 con 0x7f208815cee0 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f208dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20880b81d0 con 0x7f20880b6990 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f2087e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f208815cee0 0x7f20880b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f2087e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f208815cee0 0x7f20880b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46684/0 (socket says 172.21.15.91:46684) 2024-02-02T13:35:14.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.200+0000 7f2087e00640 1 -- 172.21.15.91:0/439805582 learned_addr learned my addr 172.21.15.91:0/439805582 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.201+0000 7f207f200640 1 --2- 172.21.15.91:0/439805582 >> v2:172.21.15.113:3300/0 conn(0x7f20880b6990 0x7f20880cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.201+0000 7f2087e00640 1 -- 172.21.15.91:0/439805582 >> v2:172.21.15.113:3300/0 conn(0x7f20880b6990 msgr2=0x7f20880cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.201+0000 7f2087e00640 1 --2- 172.21.15.91:0/439805582 >> v2:172.21.15.113:3300/0 conn(0x7f20880b6990 0x7f20880cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.201+0000 7f2087e00640 1 -- 172.21.15.91:0/439805582 >> v1:172.21.15.91:6789/0 conn(0x7f20880ce000 legacy=0x7f20880b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.201+0000 7f2087e00640 1 -- 172.21.15.91:0/439805582 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880b7130 con 0x7f208815cee0 2024-02-02T13:35:14.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.202+0000 7f2087e00640 1 --2- 172.21.15.91:0/439805582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880b8990 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f20880d1080 tx=0x7f207802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2765b4566f85b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.202+0000 7f2087400640 1 -- 172.21.15.91:0/439805582 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030860 con 0x7f208815cee0 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.202+0000 7f2087400640 1 -- 172.21.15.91:0/439805582 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20780309c0 con 0x7f208815cee0 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.202+0000 7f2087400640 1 -- 172.21.15.91:0/439805582 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030d50 con 0x7f208815cee0 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 -- 172.21.15.91:0/439805582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 msgr2=0x7f20880b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 --2- 172.21.15.91:0/439805582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880b8990 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f20880d1080 tx=0x7f207802b9a0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 -- 172.21.15.91:0/439805582 shutdown_connections 2024-02-02T13:35:14.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 --2- 172.21.15.91:0/439805582 >> v2:172.21.15.113:3300/0 conn(0x7f20880b6990 0x7f20880cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 --2- 172.21.15.91:0/439805582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880b8990 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 -- 172.21.15.91:0/439805582 >> 172.21.15.91:0/439805582 conn(0x7f2088070690 msgr2=0x7f2088070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:14.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 -- 172.21.15.91:0/439805582 shutdown_connections 2024-02-02T13:35:14.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.203+0000 7f208dc00640 1 -- 172.21.15.91:0/439805582 wait complete. 2024-02-02T13:35:14.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.205+0000 7f208dc00640 1 Processor -- start 2024-02-02T13:35:14.206 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.205+0000 7f208dc00640 1 -- start start 2024-02-02T13:35:14.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b6990 0x7f208814c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 0x7f20880c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208815ff70 con 0x7f208815cee0 2024-02-02T13:35:14.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208814e230 con 0x7f20880ce000 2024-02-02T13:35:14.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208814d620 con 0x7f20880b6990 2024-02-02T13:35:14.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46698/0 (socket says 172.21.15.91:46698) 2024-02-02T13:35:14.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 -- 172.21.15.91:0/1529561325 learned_addr learned my addr 172.21.15.91:0/1529561325 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 -- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b6990 msgr2=0x7f208814c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b6990 0x7f208814c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 -- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 msgr2=0x7f20880c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f207f200640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 0x7f20880c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 0x7f20880c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f208d200640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780094d0 con 0x7f208815cee0 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.206+0000 7f207f200640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 0x7f20880c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f208d200640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7a0 tx=0x7f208000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080003b70 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080003cf0 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080003ea0 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208814d900 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.207+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208814dec0 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.208+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208804dff0 con 0x7f208815cee0 2024-02-02T13:35:14.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.209+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f208000aa90 con 0x7f208815cee0 2024-02-02T13:35:14.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.210+0000 7f2084c00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 0x7f206006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.210+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2080088840 con 0x7f208815cee0 2024-02-02T13:35:14.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.210+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f2060072050 con 0x7f208815cee0 2024-02-02T13:35:14.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.210+0000 7f2087e00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 0x7f206006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.211+0000 7f2087e00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 0x7f206006c920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f207803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.215+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2080053d50 con 0x7f208815cee0 2024-02-02T13:35:14.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.420+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f208814e3a0 con 0x7f208815cee0 2024-02-02T13:35:14.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.421+0000 7f2084c00640 1 -- 172.21.15.91:0/1529561325 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2080057370 con 0x7f208815cee0 2024-02-02T13:35:14.423 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 msgr2=0x7f206006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 0x7f206006c920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f207803b040 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 msgr2=0x7f20880c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7a0 tx=0x7f208000cc60 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 shutdown_connections 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206006a460 0x7f206006c920 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f208815cee0 0x7f20880c6e60 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20880ce000 0x7f20880c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 --2- 172.21.15.91:0/1529561325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b6990 0x7f208814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 >> 172.21.15.91:0/1529561325 conn(0x7f2088070690 msgr2=0x7f20880c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:14.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 shutdown_connections 2024-02-02T13:35:14.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.424+0000 7f208dc00640 1 -- 172.21.15.91:0/1529561325 wait complete. 2024-02-02T13:35:14.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:35:14.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.649+0000 7fddc6200640 1 Processor -- start 2024-02-02T13:35:14.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.649+0000 7fddc6200640 1 -- start start 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.650+0000 7fddc6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 0x7fddc00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.650+0000 7fddc6200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fddc00ce130 0x7fddc00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.650+0000 7fddc6200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc00cc7c0 con 0x7fddc00cd470 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.650+0000 7fddc6200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc00ce930 con 0x7fddc00ce130 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.650+0000 7fddc6200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc006c890 con 0x7fddc015a010 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbfe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fddc00cd470 0x7fddc00b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41426/0 (socket says 172.21.15.91:41426) 2024-02-02T13:35:14.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbfe00640 1 -- 172.21.15.91:0/4052999048 learned_addr learned my addr 172.21.15.91:0/4052999048 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:14.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbea00640 1 --2- 172.21.15.91:0/4052999048 >> v2:172.21.15.113:3300/0 conn(0x7fddc00ce130 0x7fddc00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbf400640 1 --2- 172.21.15.91:0/4052999048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 0x7fddc00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852878877 0 0) 0x7fddc00cc7c0 con 0x7fddc00cd470 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.651+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda0003520 con 0x7fddc00cd470 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.652+0000 7fddbf400640 1 -- 172.21.15.91:0/4052999048 >> v2:172.21.15.113:3300/0 conn(0x7fddc00ce130 msgr2=0x7fddc00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.652+0000 7fddbf400640 1 --2- 172.21.15.91:0/4052999048 >> v2:172.21.15.113:3300/0 conn(0x7fddc00ce130 0x7fddc00ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.652+0000 7fddbf400640 1 -- 172.21.15.91:0/4052999048 >> v1:172.21.15.91:6789/0 conn(0x7fddc00cd470 legacy=0x7fddc00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.652+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3976155844 0 0) 0x7fdda0003520 con 0x7fddc00cd470 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.652+0000 7fddbf400640 1 -- 172.21.15.91:0/4052999048 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc006cb70 con 0x7fddc015a010 2024-02-02T13:35:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.653+0000 7fddbf400640 1 --2- 172.21.15.91:0/4052999048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 0x7fddc00c9670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009d90 tx=0x7fddb002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceece44e09514d4a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8001670 con 0x7fddc015a010 2024-02-02T13:35:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb002c8a0 con 0x7fddc015a010 2024-02-02T13:35:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddbe000640 1 -- 172.21.15.91:0/4052999048 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb002cc10 con 0x7fddc015a010 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 -- 172.21.15.91:0/4052999048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 msgr2=0x7fddc00c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 --2- 172.21.15.91:0/4052999048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 0x7fddc00c9670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009d90 tx=0x7fddb002bd70 comp rx=0 tx=0).stop 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 -- 172.21.15.91:0/4052999048 shutdown_connections 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 --2- 172.21.15.91:0/4052999048 >> v2:172.21.15.113:3300/0 conn(0x7fddc00ce130 0x7fddc00ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 --2- 172.21.15.91:0/4052999048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc015a010 0x7fddc00c9670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 -- 172.21.15.91:0/4052999048 >> 172.21.15.91:0/4052999048 conn(0x7fddc00cec80 msgr2=0x7fddc00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:14.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.654+0000 7fddc6200640 1 -- 172.21.15.91:0/4052999048 shutdown_connections 2024-02-02T13:35:14.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.655+0000 7fddc6200640 1 -- 172.21.15.91:0/4052999048 wait complete. 2024-02-02T13:35:14.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.656+0000 7fddc6200640 1 Processor -- start 2024-02-02T13:35:14.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.656+0000 7fddc6200640 1 -- start start 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc00ce130 0x7fddc0166230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 0x7fddc0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01f2480 con 0x7fddc00c7650 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00c83d0 con 0x7fddc015a010 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.657+0000 7fddc6200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0166770 con 0x7fddc00ce130 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46710/0 (socket says 172.21.15.91:46710) 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 -- 172.21.15.91:0/3216461619 learned_addr learned my addr 172.21.15.91:0/3216461619 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbf400640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc00ce130 0x7fddc0166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbea00640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 0x7fddc0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 -- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc00ce130 msgr2=0x7fddc0166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc00ce130 0x7fddc0166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 -- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 msgr2=0x7fddc0150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 0x7fddc0150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009a70 con 0x7fddc00c7650 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbea00640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 0x7fddc0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:14.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.658+0000 7fddbfe00640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c4e0 tx=0x7fddb800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.659+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005ce0 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.659+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8005e60 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.659+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007290 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.659+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc00c7b30 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.659+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc00c80f0 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.660+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc0066cf0 con 0x7fddc00c7650 2024-02-02T13:35:14.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.661+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fddb80073f0 con 0x7fddc00c7650 2024-02-02T13:35:14.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.662+0000 7fddb7400640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 0x7fdd9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:14.663 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.662+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fddb8088550 con 0x7fddc00c7650 2024-02-02T13:35:14.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.662+0000 7fddbf400640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 0x7fdd9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:14.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.662+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fdd98072020 con 0x7fddc00c7650 2024-02-02T13:35:14.664 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.663+0000 7fddbf400640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 0x7fdd9806c8f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fddb0036000 tx=0x7fddb0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:14.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.667+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fddb8086e60 con 0x7fddc00c7650 2024-02-02T13:35:14.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.854+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fddc00b4c00 con 0x7fddc00c7650 2024-02-02T13:35:14.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.855+0000 7fddb7400640 1 -- 172.21.15.91:0/3216461619 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddb8057620 con 0x7fddc00c7650 2024-02-02T13:35:14.857 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 msgr2=0x7fdd9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 0x7fdd9806c8f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fddb0036000 tx=0x7fddb0030000 comp rx=0 tx=0).stop 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 msgr2=0x7fddc01f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c4e0 tx=0x7fddb800c9a0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 shutdown_connections 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdd9806a430 0x7fdd9806c8f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddc00c7650 0x7fddc01f2bb0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc015a010 0x7fddc0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 --2- 172.21.15.91:0/3216461619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc00ce130 0x7fddc0166230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 >> 172.21.15.91:0/3216461619 conn(0x7fddc00cec80 msgr2=0x7fddc015c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 shutdown_connections 2024-02-02T13:35:14.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:14.858+0000 7fddc6200640 1 -- 172.21.15.91:0/3216461619 wait complete. 2024-02-02T13:35:14.877 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:35:15.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.081+0000 7f3e53400640 1 Processor -- start 2024-02-02T13:35:15.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.082+0000 7f3e53400640 1 -- start start 2024-02-02T13:35:15.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e53400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e53400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c0cc7c0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e53400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c0ce930 con 0x7f3e4c15a010 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e53400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c0cd640 con 0x7f3e4c0ce130 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46724/0 (socket says 172.21.15.91:46724) 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 -- 172.21.15.91:0/3786953950 learned_addr learned my addr 172.21.15.91:0/3786953950 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e51400640 1 --1- 172.21.15.91:0/3786953950 >> v1:172.21.15.91:6789/0 conn(0x7f3e4c0b0700 0x7f3e4c06ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41430/0 (socket says 172.21.15.91:41430) 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 -- 172.21.15.91:0/3786953950 >> v1:172.21.15.113:6789/0 conn(0x7f3e4c15a010 legacy=0x7f3e4c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 -- 172.21.15.91:0/3786953950 >> v1:172.21.15.91:6789/0 conn(0x7f3e4c0b0700 legacy=0x7f3e4c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4be00640 1 -- 172.21.15.91:0/3786953950 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c06c890 con 0x7f3e4c0ce130 2024-02-02T13:35:15.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.083+0000 7f3e4b400640 1 -- 172.21.15.91:0/3786953950 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573631665 0 0) 0x7f3e4c0cc7c0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e4be00640 1 --2- 172.21.15.91:0/3786953950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bb10 tx=0x7f3e3402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90afecb058fb1c7a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e4b400640 1 -- 172.21.15.91:0/3786953950 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34036440 con 0x7f3e4c0ce130 2024-02-02T13:35:15.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e4b400640 1 -- 172.21.15.91:0/3786953950 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e340365a0 con 0x7f3e4c0ce130 2024-02-02T13:35:15.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e4b400640 1 -- 172.21.15.91:0/3786953950 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34036930 con 0x7f3e4c0ce130 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e53400640 1 -- 172.21.15.91:0/3786953950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 msgr2=0x7f3e4c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.085+0000 7f3e53400640 1 --2- 172.21.15.91:0/3786953950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bb10 tx=0x7f3e3402e170 comp rx=0 tx=0).stop 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.086+0000 7f3e53400640 1 -- 172.21.15.91:0/3786953950 shutdown_connections 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.086+0000 7f3e53400640 1 --2- 172.21.15.91:0/3786953950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0c9670 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.086+0000 7f3e53400640 1 -- 172.21.15.91:0/3786953950 >> 172.21.15.91:0/3786953950 conn(0x7f3e4c0cec80 msgr2=0x7f3e4c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.086+0000 7f3e53400640 1 -- 172.21.15.91:0/3786953950 shutdown_connections 2024-02-02T13:35:15.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.086+0000 7f3e53400640 1 -- 172.21.15.91:0/3786953950 wait complete. 2024-02-02T13:35:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.087+0000 7f3e53400640 1 Processor -- start 2024-02-02T13:35:15.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.087+0000 7f3e53400640 1 -- start start 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 0x7f3e4c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1558e0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0ad200 con 0x7f3e4c0ce130 2024-02-02T13:35:15.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.088+0000 7f3e53400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0ad4e0 con 0x7f3e4c15a010 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46738/0 (socket says 172.21.15.91:46738) 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 -- 172.21.15.91:0/1084656329 learned_addr learned my addr 172.21.15.91:0/1084656329 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e51400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 0x7f3e4c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 -- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 msgr2=0x7f3e4c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e4be00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 0x7f3e4c0af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 -- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 msgr2=0x7f3e4c0ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3400b770 con 0x7f3e4c0b0700 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e51400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 0x7f3e4c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e4be00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:15.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.089+0000 7f3e50a00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00ba50 tx=0x7f3e3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.090+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00cbe0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.090+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c00cd80 con 0x7f3e4c0b0700 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.090+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c0ad7c0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.090+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c014680 con 0x7f3e4c0b0700 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.090+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c0add80 con 0x7f3e4c0b0700 2024-02-02T13:35:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.091+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c066cf0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.092+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e3c0147e0 con 0x7f3e4c0b0700 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.093+0000 7f3e48c00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.093+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3e3c088800 con 0x7f3e4c0b0700 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.093+0000 7f3e4be00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.093+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f3e24072020 con 0x7f3e4c0b0700 2024-02-02T13:35:15.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.094+0000 7f3e4be00640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3e34038000 tx=0x7f3e34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.097+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e3c019020 con 0x7f3e4c0b0700 2024-02-02T13:35:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.289+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e4c0b4c00 con 0x7f3e4c0b0700 2024-02-02T13:35:15.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.290+0000 7f3e48c00640 1 -- 172.21.15.91:0/1084656329 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e3c053e50 con 0x7f3e4c0b0700 2024-02-02T13:35:15.291 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 msgr2=0x7f3e2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3e34038000 tx=0x7f3e34008040 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 msgr2=0x7f3e4c1553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00ba50 tx=0x7f3e3c00bf10 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 shutdown_connections 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e2406a430 0x7f3e2406c8f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e4c15a010 0x7f3e4c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e4c0ce130 0x7f3e4c0ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 --2- 172.21.15.91:0/1084656329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e4c0b0700 0x7f3e4c1553a0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 >> 172.21.15.91:0/1084656329 conn(0x7f3e4c0cec80 msgr2=0x7f3e4c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 shutdown_connections 2024-02-02T13:35:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.292+0000 7f3e53400640 1 -- 172.21.15.91:0/1084656329 wait complete. 2024-02-02T13:35:15.312 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:35:15.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.521+0000 7fd77fe00640 1 Processor -- start 2024-02-02T13:35:15.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.521+0000 7fd77fe00640 1 -- start start 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd77815ad10 0x7fd77806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7780cb580 0x7fd7780cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77806d910 con 0x7fd77815b0f0 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77806d730 con 0x7fd77815ad10 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780cbf50 con 0x7fd7780cb580 2024-02-02T13:35:15.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77ca00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd77815b0f0 0x7fd77806d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41436/0 (socket says 172.21.15.91:41436) 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77ca00640 1 -- 172.21.15.91:0/2105870976 learned_addr learned my addr 172.21.15.91:0/2105870976 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77d400640 1 --2- 172.21.15.91:0/2105870976 >> v2:172.21.15.91:3300/0 conn(0x7fd77815ad10 0x7fd77806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.523+0000 7fd77de00640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7780cb580 0x7fd7780cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd777e00640 1 -- 172.21.15.91:0/2105870976 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859153387 0 0) 0x7fd77806d910 con 0x7fd77815b0f0 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd777e00640 1 -- 172.21.15.91:0/2105870976 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd758003520 con 0x7fd77815b0f0 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd77d400640 1 -- 172.21.15.91:0/2105870976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7780cb580 msgr2=0x7fd7780cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd77d400640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7780cb580 0x7fd7780cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd77d400640 1 -- 172.21.15.91:0/2105870976 >> v1:172.21.15.91:6789/0 conn(0x7fd77815b0f0 legacy=0x7fd77806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.524+0000 7fd77d400640 1 -- 172.21.15.91:0/2105870976 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780cc850 con 0x7fd77815ad10 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.525+0000 7fd77d400640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd77815ad10 0x7fd77806b7a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd768009dc0 tx=0x7fd76802be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bb446e54296901 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.525+0000 7fd777e00640 1 -- 172.21.15.91:0/2105870976 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76802cbb0 con 0x7fd77815ad10 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd777e00640 1 -- 172.21.15.91:0/2105870976 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76802cd10 con 0x7fd77815ad10 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd777e00640 1 -- 172.21.15.91:0/2105870976 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768034660 con 0x7fd77815ad10 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 -- 172.21.15.91:0/2105870976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd77815ad10 msgr2=0x7fd77806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd77815ad10 0x7fd77806b7a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd768009dc0 tx=0x7fd76802be00 comp rx=0 tx=0).stop 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 -- 172.21.15.91:0/2105870976 shutdown_connections 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7780cb580 0x7fd7780cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 --2- 172.21.15.91:0/2105870976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd77815ad10 0x7fd77806b7a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 -- 172.21.15.91:0/2105870976 >> 172.21.15.91:0/2105870976 conn(0x7fd7780b0700 msgr2=0x7fd7780b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:15.528 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 -- 172.21.15.91:0/2105870976 shutdown_connections 2024-02-02T13:35:15.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.526+0000 7fd77fe00640 1 -- 172.21.15.91:0/2105870976 wait complete. 2024-02-02T13:35:15.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.528+0000 7fd77fe00640 1 Processor -- start 2024-02-02T13:35:15.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.528+0000 7fd77fe00640 1 -- start start 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd77815ad10 0x7fd7780c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd77815b0f0 0x7fd7780c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780aeab0 con 0x7fd7780cb580 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780aed90 con 0x7fd77815ad10 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780af070 con 0x7fd77815b0f0 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46760/0 (socket says 172.21.15.91:46760) 2024-02-02T13:35:15.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.529+0000 7fd77d400640 1 -- 172.21.15.91:0/69603879 learned_addr learned my addr 172.21.15.91:0/69603879 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:15.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 -- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd77815b0f0 msgr2=0x7fd7780c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd77815b0f0 0x7fd7780c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 -- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd77815ad10 msgr2=0x7fd7780c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd77815ad10 0x7fd7780c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768009a70 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd77d400640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ee0 tx=0x7fd768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.530+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76803e610 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.531+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76803e770 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.531+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7780af350 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.531+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76803eae0 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.531+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd778159a10 con 0x7fd7780cb580 2024-02-02T13:35:15.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.532+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd778066cf0 con 0x7fd7780cb580 2024-02-02T13:35:15.534 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.533+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd76803b050 con 0x7fd7780cb580 2024-02-02T13:35:15.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.534+0000 7fd775600640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 0x7fd75006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:15.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.534+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7680ab3d0 con 0x7fd7780cb580 2024-02-02T13:35:15.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.534+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fd750072020 con 0x7fd7780cb580 2024-02-02T13:35:15.535 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.534+0000 7fd77ca00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 0x7fd75006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:15.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.535+0000 7fd77ca00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 0x7fd75006c8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd760000f60 tx=0x7fd7600078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:15.540 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.539+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd768039060 con 0x7fd7780cb580 2024-02-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.742+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd7780713f0 con 0x7fd7780cb580 2024-02-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.742+0000 7fd775600640 1 -- 172.21.15.91:0/69603879 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v165) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd76804e030 con 0x7fd7780cb580 2024-02-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 msgr2=0x7fd75006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 0x7fd75006c8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd760000f60 tx=0x7fd7600078f0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 msgr2=0x7fd7781594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ee0 tx=0x7fd768004a40 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 shutdown_connections 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd75006a430 0x7fd75006c8f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd77815b0f0 0x7fd7780c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd77815ad10 0x7fd7780c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 --2- 172.21.15.91:0/69603879 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7780cb580 0x7fd7781594d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 >> 172.21.15.91:0/69603879 conn(0x7fd7780b0700 msgr2=0x7fd77815dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 shutdown_connections 2024-02-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:15.745+0000 7fd77fe00640 1 -- 172.21.15.91:0/69603879 wait complete. 2024-02-02T13:35:15.766 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:35:15.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:15.766 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:35:15.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:35:15.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:15.878 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:35:15.930 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:15.930 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:15.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:35:15.988 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:15.988 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:16.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:35:16.043 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:16.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.256+0000 7fd5a1a00640 1 Processor -- start 2024-02-02T13:35:16.257 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.256+0000 7fd5a1a00640 1 -- start start 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.257+0000 7fd5a1a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd59c15a010 0x7fd59c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.257+0000 7fd5a1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.257+0000 7fd5a1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c0cc7c0 con 0x7fd59c0ce130 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.257+0000 7fd5a1a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0b10d0 con 0x7fd59c15a010 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.257+0000 7fd5a1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0ce930 con 0x7fd59c0b0700 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd59c15a010 0x7fd59c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd59c15a010 0x7fd59c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46774/0 (socket says 172.21.15.91:46774) 2024-02-02T13:35:16.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 -- 172.21.15.91:0/2558489893 learned_addr learned my addr 172.21.15.91:0/2558489893 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:16.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 -- 172.21.15.91:0/2558489893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 msgr2=0x7fd59c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:16.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 --2- 172.21.15.91:0/2558489893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 -- 172.21.15.91:0/2558489893 >> v1:172.21.15.113:6789/0 conn(0x7fd59c0ce130 legacy=0x7fd59c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:16.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.258+0000 7fd59aa00640 1 -- 172.21.15.91:0/2558489893 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0cd640 con 0x7fd59c15a010 2024-02-02T13:35:16.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.259+0000 7fd59aa00640 1 --2- 172.21.15.91:0/2558489893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c0c9670 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f298e3dedcd0c671 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd59a000640 1 -- 172.21.15.91:0/2558489893 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c031640 con 0x7fd59c15a010 2024-02-02T13:35:16.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd59a000640 1 -- 172.21.15.91:0/2558489893 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c0317a0 con 0x7fd59c15a010 2024-02-02T13:35:16.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd59a000640 1 -- 172.21.15.91:0/2558489893 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c031b30 con 0x7fd59c15a010 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 -- 172.21.15.91:0/2558489893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 msgr2=0x7fd59c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/2558489893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c0c9670 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 -- 172.21.15.91:0/2558489893 shutdown_connections 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/2558489893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/2558489893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c0c9670 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 -- 172.21.15.91:0/2558489893 >> 172.21.15.91:0/2558489893 conn(0x7fd59c0cec80 msgr2=0x7fd59c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.260+0000 7fd5a1a00640 1 -- 172.21.15.91:0/2558489893 shutdown_connections 2024-02-02T13:35:16.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.261+0000 7fd5a1a00640 1 -- 172.21.15.91:0/2558489893 wait complete. 2024-02-02T13:35:16.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.263+0000 7fd5a1a00640 1 Processor -- start 2024-02-02T13:35:16.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.263+0000 7fd5a1a00640 1 -- start start 2024-02-02T13:35:16.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59c0ce130 0x7fd59c0c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c1431d0 con 0x7fd59c15a010 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0c5920 con 0x7fd59c0ce130 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd5a1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0c3910 con 0x7fd59c0b0700 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd59b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd59b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46778/0 (socket says 172.21.15.91:46778) 2024-02-02T13:35:16.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd59b400640 1 -- 172.21.15.91:0/3053271739 learned_addr learned my addr 172.21.15.91:0/3053271739 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.264+0000 7fd59aa00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 -- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 msgr2=0x7fd59c0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 -- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59c0ce130 msgr2=0x7fd59c0c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59c0ce130 0x7fd59c0c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0094d0 con 0x7fd59c15a010 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd59b400640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b9e0 tx=0x7fd59400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400caa0 con 0x7fd59c15a010 2024-02-02T13:35:16.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59400cc20 con 0x7fd59c15a010 2024-02-02T13:35:16.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594014440 con 0x7fd59c15a010 2024-02-02T13:35:16.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c142550 con 0x7fd59c15a010 2024-02-02T13:35:16.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.265+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c142b10 con 0x7fd59c15a010 2024-02-02T13:35:16.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.266+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c143310 con 0x7fd59c15a010 2024-02-02T13:35:16.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.268+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd59401d3b0 con 0x7fd59c15a010 2024-02-02T13:35:16.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.269+0000 7fd593400640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 0x7fd57406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.269+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd594088ac0 con 0x7fd59c15a010 2024-02-02T13:35:16.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.269+0000 7fd59aa00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 0x7fd57406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.269+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fd574072020 con 0x7fd59c15a010 2024-02-02T13:35:16.271 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.270+0000 7fd59aa00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 0x7fd57406c8f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd58c036000 tx=0x7fd58c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.272+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd594019070 con 0x7fd59c15a010 2024-02-02T13:35:16.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.499+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd59c143310 con 0x7fd59c15a010 2024-02-02T13:35:16.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.500+0000 7fd593400640 1 -- 172.21.15.91:0/3053271739 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fd59c143310 con 0x7fd59c15a010 2024-02-02T13:35:16.502 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:16.502 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:35:16.502 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 msgr2=0x7fd57406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 0x7fd57406c8f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd58c036000 tx=0x7fd58c03c040 comp rx=0 tx=0).stop 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 msgr2=0x7fd59c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b9e0 tx=0x7fd59400bea0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 shutdown_connections 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd57406a430 0x7fd57406c8f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.503+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd59c15a010 0x7fd59c15be20 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.504+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59c0ce130 0x7fd59c0c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.504+0000 7fd5a1a00640 1 --2- 172.21.15.91:0/3053271739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd59c0b0700 0x7fd59c0c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.504+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 >> 172.21.15.91:0/3053271739 conn(0x7fd59c0cec80 msgr2=0x7fd59c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:16.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.504+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 shutdown_connections 2024-02-02T13:35:16.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.504+0000 7fd5a1a00640 1 -- 172.21.15.91:0/3053271739 wait complete. 2024-02-02T13:35:16.525 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:16.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.711+0000 7f9098200640 1 Processor -- start 2024-02-02T13:35:16.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.711+0000 7f9098200640 1 -- start start 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.712+0000 7f9098200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f909015a010 0x7f90900c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.712+0000 7f9098200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90900b0700 0x7f90900b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.712+0000 7f9098200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900cc7c0 con 0x7f90900ce130 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.712+0000 7f9098200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900b10b0 con 0x7f909015a010 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.712+0000 7f9098200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900ce930 con 0x7f90900b0700 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f909015a010 0x7f90900c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f909015a010 0x7f90900c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46790/0 (socket says 172.21.15.91:46790) 2024-02-02T13:35:16.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 -- 172.21.15.91:0/55902022 learned_addr learned my addr 172.21.15.91:0/55902022 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:16.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9096200640 1 --2- 172.21.15.91:0/55902022 >> v2:172.21.15.113:3300/0 conn(0x7f90900b0700 0x7f90900b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 -- 172.21.15.91:0/55902022 >> v2:172.21.15.113:3300/0 conn(0x7f90900b0700 msgr2=0x7f90900b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:16.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 --2- 172.21.15.91:0/55902022 >> v2:172.21.15.113:3300/0 conn(0x7f90900b0700 0x7f90900b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 -- 172.21.15.91:0/55902022 >> v1:172.21.15.91:6789/0 conn(0x7f90900ce130 legacy=0x7f90900ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f9095800640 1 -- 172.21.15.91:0/55902022 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900cd640 con 0x7f909015a010 2024-02-02T13:35:16.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.713+0000 7f908fe00640 1 -- 172.21.15.91:0/55902022 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419166518 0 0) 0x7f90900cc7c0 con 0x7f90900ce130 2024-02-02T13:35:16.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.714+0000 7f9095800640 1 --2- 172.21.15.91:0/55902022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f909015a010 0x7f90900c9670 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f90900cec10 tx=0x7f908002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de7223aeb57489 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.714+0000 7f908fe00640 1 -- 172.21.15.91:0/55902022 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080030860 con 0x7f909015a010 2024-02-02T13:35:16.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.714+0000 7f908fe00640 1 -- 172.21.15.91:0/55902022 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90800309c0 con 0x7f909015a010 2024-02-02T13:35:16.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f908fe00640 1 -- 172.21.15.91:0/55902022 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080030d50 con 0x7f909015a010 2024-02-02T13:35:16.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 -- 172.21.15.91:0/55902022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f909015a010 msgr2=0x7f90900c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 --2- 172.21.15.91:0/55902022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f909015a010 0x7f90900c9670 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f90900cec10 tx=0x7f908002b9a0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 -- 172.21.15.91:0/55902022 shutdown_connections 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 --2- 172.21.15.91:0/55902022 >> v2:172.21.15.113:3300/0 conn(0x7f90900b0700 0x7f90900b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 --2- 172.21.15.91:0/55902022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f909015a010 0x7f90900c9670 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 -- 172.21.15.91:0/55902022 >> 172.21.15.91:0/55902022 conn(0x7f90900cec80 msgr2=0x7f90900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 -- 172.21.15.91:0/55902022 shutdown_connections 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.715+0000 7f9098200640 1 -- 172.21.15.91:0/55902022 wait complete. 2024-02-02T13:35:16.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.717+0000 7f9098200640 1 Processor -- start 2024-02-02T13:35:16.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.717+0000 7f9098200640 1 -- start start 2024-02-02T13:35:16.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90900b0700 0x7f90901f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f909015a010 0x7f9090157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909015d240 con 0x7f90900ce130 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901f3110 con 0x7f90900b0700 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9098200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90901f33f0 con 0x7f909015a010 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9094e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9094e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46802/0 (socket says 172.21.15.91:46802) 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.718+0000 7f9094e00640 1 -- 172.21.15.91:0/1264993535 learned_addr learned my addr 172.21.15.91:0/1264993535 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9096200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f909015a010 0x7f9090157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 -- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f909015a010 msgr2=0x7f9090157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f909015a010 0x7f9090157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 -- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90900b0700 msgr2=0x7f90901f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90900b0700 0x7f90901f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800094d0 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9094e00640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f907800c4b0 tx=0x7f907800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078005480 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780035f0 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780039e0 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901f36d0 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.719+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9090157a70 con 0x7f90900ce130 2024-02-02T13:35:16.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.720+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90900acbb0 con 0x7f90900ce130 2024-02-02T13:35:16.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.721+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9078003b40 con 0x7f90900ce130 2024-02-02T13:35:16.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.722+0000 7f908d600640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 0x7f906806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:16.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.722+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9078087fe0 con 0x7f90900ce130 2024-02-02T13:35:16.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.722+0000 7f9095800640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 0x7f906806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:16.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.723+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f9068072020 con 0x7f90900ce130 2024-02-02T13:35:16.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.723+0000 7f9095800640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 0x7f906806c8f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9080036000 tx=0x7f908003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:16.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.727+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9078017070 con 0x7f90900ce130 2024-02-02T13:35:16.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.921+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9090066cf0 con 0x7f90900ce130 2024-02-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.924+0000 7f908d600640 1 -- 172.21.15.91:0/1264993535 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16235 (secure 0 0 0) 0x7f90900acbb0 con 0x7f90900ce130 2024-02-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":165,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:34:29.471448+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 msgr2=0x7f906806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 0x7f906806c8f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9080036000 tx=0x7f908003b040 comp rx=0 tx=0).stop 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 msgr2=0x7f909015cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f907800c4b0 tx=0x7f907800c970 comp rx=0 tx=0).stop 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 shutdown_connections 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f906806a430 0x7f906806c8f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f909015a010 0x7f9090157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90900ce130 0x7f909015cd00 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 --2- 172.21.15.91:0/1264993535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90900b0700 0x7f90901f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:16.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 >> 172.21.15.91:0/1264993535 conn(0x7f90900cec80 msgr2=0x7f909015de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:16.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.928+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 shutdown_connections 2024-02-02T13:35:16.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:16.929+0000 7f9098200640 1 -- 172.21.15.91:0/1264993535 wait complete. 2024-02-02T13:35:16.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:32:53.511243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:32:54.111607+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:35:16.949 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:35:17.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.158+0000 7fcd5f400640 1 Processor -- start 2024-02-02T13:35:17.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.159+0000 7fcd5f400640 1 -- start start 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd580b2260 0x7fcd580d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5f400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580cd930 con 0x7fcd580b6260 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5f400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd580b8f90 con 0x7fcd580b2260 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580d1680 con 0x7fcd580d0070 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd57e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd580b2260 0x7fcd580d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd57e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd580b2260 0x7fcd580d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46806/0 (socket says 172.21.15.91:46806) 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd57e00640 1 -- 172.21.15.91:0/1095622481 learned_addr learned my addr 172.21.15.91:0/1095622481 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:17.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.160+0000 7fcd5d400640 1 --1- 172.21.15.91:0/1095622481 >> v1:172.21.15.91:6789/0 conn(0x7fcd580b6260 0x7fcd580d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41466/0 (socket says 172.21.15.91:41466) 2024-02-02T13:35:17.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd5ca00640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd57e00640 1 -- 172.21.15.91:0/1095622481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 msgr2=0x7fcd580b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:17.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd57e00640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd57e00640 1 -- 172.21.15.91:0/1095622481 >> v1:172.21.15.91:6789/0 conn(0x7fcd580b6260 legacy=0x7fcd580d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd57e00640 1 -- 172.21.15.91:0/1095622481 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580d1960 con 0x7fcd580b2260 2024-02-02T13:35:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd5ca00640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:35:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.161+0000 7fcd57400640 1 -- 172.21.15.91:0/1095622481 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651971785 0 0) 0x7fcd50001110 con 0x7fcd580b6260 2024-02-02T13:35:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.162+0000 7fcd57e00640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd580d0450 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78820e7763366fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.162+0000 7fcd57400640 1 -- 172.21.15.91:0/1095622481 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40030840 con 0x7fcd580b2260 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.162+0000 7fcd57400640 1 -- 172.21.15.91:0/1095622481 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd400309a0 con 0x7fcd580b2260 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd57400640 1 -- 172.21.15.91:0/1095622481 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40030d30 con 0x7fcd580b2260 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 -- 172.21.15.91:0/1095622481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 msgr2=0x7fcd580d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd580d0450 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4002b980 comp rx=0 tx=0).stop 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 -- 172.21.15.91:0/1095622481 shutdown_connections 2024-02-02T13:35:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd580d0450 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 --2- 172.21.15.91:0/1095622481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 -- 172.21.15.91:0/1095622481 >> 172.21.15.91:0/1095622481 conn(0x7fcd58070660 msgr2=0x7fcd58070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 -- 172.21.15.91:0/1095622481 shutdown_connections 2024-02-02T13:35:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.163+0000 7fcd5f400640 1 -- 172.21.15.91:0/1095622481 wait complete. 2024-02-02T13:35:17.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.165+0000 7fcd5f400640 1 Processor -- start 2024-02-02T13:35:17.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.165+0000 7fcd5f400640 1 -- start start 2024-02-02T13:35:17.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd580b6260 0x7fcd580d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580ccf90 con 0x7fcd580b2260 2024-02-02T13:35:17.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580cca20 con 0x7fcd580b6260 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.166+0000 7fcd5f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580cb9c0 con 0x7fcd580d0070 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46814/0 (socket says 172.21.15.91:46814) 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 -- 172.21.15.91:0/2126643482 learned_addr learned my addr 172.21.15.91:0/2126643482 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 -- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 msgr2=0x7fcd580d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 -- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd580b6260 msgr2=0x7fcd580d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd580b6260 0x7fcd580d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400094d0 con 0x7fcd580b2260 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.167+0000 7fcd5ca00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fcd48007f20 tx=0x7fcd4800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.168+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800dee0 con 0x7fcd580b2260 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.168+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4800f040 con 0x7fcd580b2260 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.168+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48003dd0 con 0x7fcd580b2260 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.168+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd580cbc60 con 0x7fcd580b2260 2024-02-02T13:35:17.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.168+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd580cc220 con 0x7fcd580b2260 2024-02-02T13:35:17.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.169+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58066cf0 con 0x7fcd580b2260 2024-02-02T13:35:17.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.170+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd48004030 con 0x7fcd580b2260 2024-02-02T13:35:17.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.171+0000 7fcd54c00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 0x7fcd3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.171+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd48088b60 con 0x7fcd580b2260 2024-02-02T13:35:17.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.171+0000 7fcd57e00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 0x7fcd3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.171+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fcd30072020 con 0x7fcd580b2260 2024-02-02T13:35:17.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.172+0000 7fcd57e00640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 0x7fcd3006c8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcd40036000 tx=0x7fcd4003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.175+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcd4801a060 con 0x7fcd580b2260 2024-02-02T13:35:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.364+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd580b5900 con 0x7fcd580b2260 2024-02-02T13:35:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.365+0000 7fcd54c00640 1 -- 172.21.15.91:0/2126643482 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd480541b0 con 0x7fcd580b2260 2024-02-02T13:35:17.367 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.367+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 msgr2=0x7fcd3006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.367+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 0x7fcd3006c8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcd40036000 tx=0x7fcd4003b040 comp rx=0 tx=0).stop 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.367+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 msgr2=0x7fcd581599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.367+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fcd48007f20 tx=0x7fcd4800d560 comp rx=0 tx=0).stop 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 shutdown_connections 2024-02-02T13:35:17.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd3006a430 0x7fcd3006c8f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd580d0070 0x7fcd580d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd580b6260 0x7fcd580d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 --2- 172.21.15.91:0/2126643482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd580b2260 0x7fcd581599c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 >> 172.21.15.91:0/2126643482 conn(0x7fcd58070660 msgr2=0x7fcd580b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 shutdown_connections 2024-02-02T13:35:17.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.368+0000 7fcd5f400640 1 -- 172.21.15.91:0/2126643482 wait complete. 2024-02-02T13:35:17.386 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:35:17.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.588+0000 7f9006400640 1 Processor -- start 2024-02-02T13:35:17.590 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.588+0000 7f9006400640 1 -- start start 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.589+0000 7f9006400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f90000b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.589+0000 7f9006400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90000b0c40 0x7f90000b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.589+0000 7f9006400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900015fa00 con 0x7f9000151f90 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.589+0000 7f9006400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90000b10d0 con 0x7f90000b0c40 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.589+0000 7f9006400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9000070f80 con 0x7f900015f5a0 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8fff400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9000151f90 0x7f900006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41480/0 (socket says 172.21.15.91:41480) 2024-02-02T13:35:17.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8fff400640 1 -- 172.21.15.91:0/4207732821 learned_addr learned my addr 172.21.15.91:0/4207732821 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:17.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267320492 0 0) 0x7f8ff0001110 con 0x7f9000151f90 2024-02-02T13:35:17.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ff7e00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f90000b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe0003520 con 0x7f9000151f90 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8fffe00640 1 --2- 172.21.15.91:0/4207732821 >> v2:172.21.15.113:3300/0 conn(0x7f90000b0c40 0x7f90000b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195436345 0 0) 0x7f8fe0003520 con 0x7f9000151f90 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 msgr2=0x7f90000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f90000b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 >> v2:172.21.15.113:3300/0 conn(0x7f90000b0c40 msgr2=0x7f90000b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 --2- 172.21.15.91:0/4207732821 >> v2:172.21.15.113:3300/0 conn(0x7f90000b0c40 0x7f90000b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000152370 con 0x7f9000151f90 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.590+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8ff0001800 con 0x7f9000151f90 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 >> v1:172.21.15.91:6789/0 conn(0x7f9000151f90 legacy=0x7f900006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f8fe0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f8fe00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0004d00 0x7f8fe0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b10d0 con 0x7f8fe0003970 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000070f80 con 0x7f8fe0004d00 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe000ba30 con 0x7f900015f5a0 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f8fe0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8ff7e00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f8fe00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fffe00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0004d00 0x7f8fe0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 -- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 msgr2=0x7f8fe00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f900015f5a0 0x7f8fe00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 -- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0004d00 msgr2=0x7f8fe0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0004d00 0x7f8fe0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fff400640 1 -- 172.21.15.91:0/4207732821 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0003520 con 0x7f8fe0003970 2024-02-02T13:35:17.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.591+0000 7f8fffe00640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0004d00 0x7f8fe0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.592+0000 7f8fff400640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f8fe0003d90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c110 tx=0x7f8fe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc483aaa0315f40 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0005ad0 con 0x7f8fe0003970 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0005c50 con 0x7f8fe0003970 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f8ffea00640 1 -- 172.21.15.91:0/4207732821 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff002c810 con 0x7f8fe0003970 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f9006400640 1 -- 172.21.15.91:0/4207732821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 msgr2=0x7f8fe0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f9006400640 1 --2- 172.21.15.91:0/4207732821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f8fe0003d90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c110 tx=0x7f8fe00057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f8fffe00640 1 -- 172.21.15.91:0/4207732821 reap_dead start 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f9006400640 1 -- 172.21.15.91:0/4207732821 shutdown_connections 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.593+0000 7f9006400640 1 -- 172.21.15.91:0/4207732821 >> 172.21.15.91:0/4207732821 conn(0x7f90000cec80 msgr2=0x7f90000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.594+0000 7f9006400640 1 -- 172.21.15.91:0/4207732821 shutdown_connections 2024-02-02T13:35:17.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.594+0000 7f9006400640 1 -- 172.21.15.91:0/4207732821 wait complete. 2024-02-02T13:35:17.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.595+0000 7f9006400640 1 Processor -- start 2024-02-02T13:35:17.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.595+0000 7f9006400640 1 -- start start 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000150f80 0x7f90001506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90000add60 0x7f90000cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000ad8d0 con 0x7f8fe0003970 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000accc0 con 0x7f9000150f80 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.596+0000 7f9006400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90000acfa0 con 0x7f90000add60 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46830/0 (socket says 172.21.15.91:46830) 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 -- 172.21.15.91:0/1463756184 learned_addr learned my addr 172.21.15.91:0/1463756184 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 -- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90000add60 msgr2=0x7f90000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90000add60 0x7f90000cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 -- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000150f80 msgr2=0x7f90001506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000150f80 0x7f90001506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000ad280 con 0x7f8fe0003970 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.597+0000 7f8fff400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002c3b0 tx=0x7f8ff0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.598+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0039070 con 0x7f8fe0003970 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.598+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff002cc20 con 0x7f8fe0003970 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.598+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000ad560 con 0x7f8fe0003970 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.598+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff003c040 con 0x7f8fe0003970 2024-02-02T13:35:17.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.598+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90000ac900 con 0x7f8fe0003970 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.599+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000155990 con 0x7f8fe0003970 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.600+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ff0032070 con 0x7f8fe0003970 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.601+0000 7f8ff7400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 0x7f8fd806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.601+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ff00a9d90 con 0x7f8fe0003970 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.601+0000 7f8ff7e00640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 0x7f8fd806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:17.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.601+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f8fd8072050 con 0x7f8fe0003970 2024-02-02T13:35:17.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.601+0000 7f8ff7e00640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 0x7f8fd806c920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90000cfd80 tx=0x7f8fe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:17.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.605+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ff00753c0 con 0x7f8fe0003970 2024-02-02T13:35:17.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.797+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f900015fa00 con 0x7f8fe0003970 2024-02-02T13:35:17.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.798+0000 7f8ff7400640 1 -- 172.21.15.91:0/1463756184 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7f900015fa00 con 0x7f8fe0003970 2024-02-02T13:35:17.799 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:35:17.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 msgr2=0x7f8fd806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 0x7f8fd806c920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90000cfd80 tx=0x7f8fe8008040 comp rx=0 tx=0).stop 2024-02-02T13:35:17.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 msgr2=0x7f90000c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:17.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002c3b0 tx=0x7f8ff0001ac0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 shutdown_connections 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8fd806a460 0x7f8fd806c920 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90000add60 0x7f90000cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000150f80 0x7f90001506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 --2- 172.21.15.91:0/1463756184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8fe0003970 0x7f90000c20d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.800+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 >> 172.21.15.91:0/1463756184 conn(0x7f90000cec80 msgr2=0x7f900006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.801+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 shutdown_connections 2024-02-02T13:35:17.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:17.801+0000 7f9006400640 1 -- 172.21.15.91:0/1463756184 wait complete. 2024-02-02T13:35:17.822 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:35:18.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.032+0000 7f1e17a00640 1 Processor -- start 2024-02-02T13:35:18.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.032+0000 7f1e17a00640 1 -- start start 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.033+0000 7f1e17a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e1015a010 0x7f1e100c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.033+0000 7f1e17a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1e100ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e17a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100cc7c0 con 0x7f1e100cd470 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e17a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100ce930 con 0x7f1e1015a010 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e17a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1006c890 con 0x7f1e100ce130 2024-02-02T13:35:18.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e15a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1e100cd470 0x7f1e100b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41488/0 (socket says 172.21.15.91:41488) 2024-02-02T13:35:18.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e15a00640 1 -- 172.21.15.91:0/992122305 learned_addr learned my addr 172.21.15.91:0/992122305 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1e100ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e15000640 1 --2- 172.21.15.91:0/992122305 >> v2:172.21.15.113:3300/0 conn(0x7f1e1015a010 0x7f1e100c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842988322 0 0) 0x7f1e100cc7c0 con 0x7f1e100cd470 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.034+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003520 con 0x7f1e100cd470 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774566519 0 0) 0x7f1df0003520 con 0x7f1e100cd470 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 msgr2=0x7f1e100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1e100ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 >> v2:172.21.15.113:3300/0 conn(0x7f1e1015a010 msgr2=0x7f1e100c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 --2- 172.21.15.91:0/992122305 >> v2:172.21.15.113:3300/0 conn(0x7f1e1015a010 0x7f1e100c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1006cb70 con 0x7f1e100cd470 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1e080019f0 con 0x7f1e100cd470 2024-02-02T13:35:18.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 >> v1:172.21.15.91:6789/0 conn(0x7f1e100cd470 legacy=0x7f1e100b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1df0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1df00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df0004d00 0x7f1df0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100ce930 con 0x7f1df0003f40 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1006c890 con 0x7f1df0004d00 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df000b980 con 0x7f1e100ce130 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.035+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1df00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e15000640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1df0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e0fe00640 1 -- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 msgr2=0x7f1df0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e15a00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df0004d00 0x7f1df0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e100ce130 0x7f1df0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e0fe00640 1 -- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df0004d00 msgr2=0x7f1df0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df0004d00 0x7f1df0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e0fe00640 1 -- 172.21.15.91:0/992122305 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0003520 con 0x7f1df0003f40 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.036+0000 7f1e15a00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df0004d00 0x7f1df0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1df00047c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009af0 tx=0x7f1df00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a92bfb966b5ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df802c6b0 con 0x7f1df0003f40 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df802c810 con 0x7f1df0003f40 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e0f400640 1 -- 172.21.15.91:0/992122305 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df802cba0 con 0x7f1df0003f40 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e17a00640 1 -- 172.21.15.91:0/992122305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 msgr2=0x7f1df00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.037+0000 7f1e17a00640 1 --2- 172.21.15.91:0/992122305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1df00047c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009af0 tx=0x7f1df00057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.038+0000 7f1e15a00640 1 -- 172.21.15.91:0/992122305 reap_dead start 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.038+0000 7f1e17a00640 1 -- 172.21.15.91:0/992122305 shutdown_connections 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.038+0000 7f1e17a00640 1 -- 172.21.15.91:0/992122305 >> 172.21.15.91:0/992122305 conn(0x7f1e100cec80 msgr2=0x7f1e100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.038+0000 7f1e17a00640 1 -- 172.21.15.91:0/992122305 shutdown_connections 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.038+0000 7f1e17a00640 1 -- 172.21.15.91:0/992122305 wait complete. 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.039+0000 7f1e17a00640 1 Processor -- start 2024-02-02T13:35:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.040+0000 7f1e17a00640 1 -- start start 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.040+0000 7f1e17a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.040+0000 7f1e17a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e10158340 0x7f1e100c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e17a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e101f0a60 0x7f1e1015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e17a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c5250 con 0x7f1df0003f40 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e17a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101587a0 con 0x7f1e101f0a60 2024-02-02T13:35:18.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e17a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c6d50 con 0x7f1e10158340 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46860/0 (socket says 172.21.15.91:46860) 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 -- 172.21.15.91:0/4156288179 learned_addr learned my addr 172.21.15.91:0/4156288179 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 -- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e10158340 msgr2=0x7f1e100c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e10158340 0x7f1e100c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 -- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e101f0a60 msgr2=0x7f1e1015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e101f0a60 0x7f1e1015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100c7030 con 0x7f1df0003f40 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.041+0000 7f1e15000640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1e000116d0 tx=0x7f1e00011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.042+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0000f9c0 con 0x7f1df0003f40 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.042+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0000fb20 con 0x7f1df0003f40 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.042+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0000fe90 con 0x7f1df0003f40 2024-02-02T13:35:18.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.042+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100c7310 con 0x7f1df0003f40 2024-02-02T13:35:18.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.042+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100c78d0 con 0x7f1df0003f40 2024-02-02T13:35:18.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.043+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e101588e0 con 0x7f1df0003f40 2024-02-02T13:35:18.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.044+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1e00015460 con 0x7f1df0003f40 2024-02-02T13:35:18.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.045+0000 7f1e0cc00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 0x7f1de806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.045+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e00088ff0 con 0x7f1df0003f40 2024-02-02T13:35:18.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.045+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 0x7f1de806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.046+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f1de8072020 con 0x7f1df0003f40 2024-02-02T13:35:18.047 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.046+0000 7f1e0fe00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 0x7f1de806c8f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1df8036000 tx=0x7f1df802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.050+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1e00054500 con 0x7f1df0003f40 2024-02-02T13:35:18.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.242+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e10066cf0 con 0x7f1df0003f40 2024-02-02T13:35:18.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.243+0000 7f1e0cc00640 1 -- 172.21.15.91:0/4156288179 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e00022070 con 0x7f1df0003f40 2024-02-02T13:35:18.244 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 msgr2=0x7f1de806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 0x7f1de806c8f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1df8036000 tx=0x7f1df802e000 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 msgr2=0x7f1e100c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1e000116d0 tx=0x7f1e00011b90 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 shutdown_connections 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1de806a430 0x7f1de806c8f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e10158340 0x7f1e100c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e101f0a60 0x7f1e1015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 --2- 172.21.15.91:0/4156288179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1df0003f40 0x7f1e100c6690 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 >> 172.21.15.91:0/4156288179 conn(0x7f1e100cec80 msgr2=0x7f1e100ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:18.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 shutdown_connections 2024-02-02T13:35:18.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.246+0000 7f1e17a00640 1 -- 172.21.15.91:0/4156288179 wait complete. 2024-02-02T13:35:18.268 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:18.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.472+0000 7f2518e00640 1 Processor -- start 2024-02-02T13:35:18.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.472+0000 7f2518e00640 1 -- start start 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.473+0000 7f2518e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f251415f5a0 0x7f25140b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.473+0000 7f2518e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140b1070 con 0x7f25140b0c40 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.473+0000 7f2518e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251406c600 con 0x7f2514151f90 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.473+0000 7f2518e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251415fa30 con 0x7f251415f5a0 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2512800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f25140b0c40 0x7f25140b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41498/0 (socket says 172.21.15.91:41498) 2024-02-02T13:35:18.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2512800640 1 -- 172.21.15.91:0/2023872947 learned_addr learned my addr 172.21.15.91:0/2023872947 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2509200640 1 --2- 172.21.15.91:0/2023872947 >> v2:172.21.15.113:3300/0 conn(0x7f251415f5a0 0x7f25140b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339508566 0 0) 0x7f25140b1070 con 0x7f25140b0c40 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f4003520 con 0x7f25140b0c40 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.474+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695789077 0 0) 0x7f24f4003520 con 0x7f25140b0c40 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 >> v2:172.21.15.113:3300/0 conn(0x7f251415f5a0 msgr2=0x7f25140b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 --2- 172.21.15.91:0/2023872947 >> v2:172.21.15.113:3300/0 conn(0x7f251415f5a0 0x7f25140b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 >> v1:172.21.15.113:6789/0 conn(0x7f2514151f90 legacy=0x7f251406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514070f80 con 0x7f25140b0c40 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f250c0019f0 con 0x7f25140b0c40 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 >> v1:172.21.15.91:6789/0 conn(0x7f25140b0c40 legacy=0x7f25140b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f24f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f4005180 0x7f24f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251415fa30 con 0x7f24f40039c0 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f400c080 con 0x7f24f40047c0 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f400c240 con 0x7f24f4005180 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f24f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2509200640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 -- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f4005180 msgr2=0x7f24f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f4005180 0x7f24f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 -- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 msgr2=0x7f24f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f40047c0 0x7f24f4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2511e00640 1 -- 172.21.15.91:0/2023872947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4003520 con 0x7f24f40039c0 2024-02-02T13:35:18.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.475+0000 7f2512800640 1 -- 172.21.15.91:0/2023872947 reap_dead start 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.476+0000 7f2511e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f24f4003de0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2504001750 tx=0x7f24f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abd434114221e0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040052d0 con 0x7f24f40039c0 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250402e050 con 0x7f24f40039c0 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2511400640 1 -- 172.21.15.91:0/2023872947 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504003830 con 0x7f24f40039c0 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2518e00640 1 -- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 msgr2=0x7f24f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2518e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f24f4003de0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2504001750 tx=0x7f24f40057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2518e00640 1 -- 172.21.15.91:0/2023872947 shutdown_connections 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2518e00640 1 --2- 172.21.15.91:0/2023872947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f24f4003de0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.477+0000 7f2518e00640 1 -- 172.21.15.91:0/2023872947 >> 172.21.15.91:0/2023872947 conn(0x7f25140cec80 msgr2=0x7f25140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.478+0000 7f2518e00640 1 -- 172.21.15.91:0/2023872947 shutdown_connections 2024-02-02T13:35:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.478+0000 7f2518e00640 1 -- 172.21.15.91:0/2023872947 wait complete. 2024-02-02T13:35:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.479+0000 7f2518e00640 1 Processor -- start 2024-02-02T13:35:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.479+0000 7f2518e00640 1 -- start start 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140c36d0 0x7f25140c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f251414ca40 0x7f251414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140ce6e0 con 0x7f24f40039c0 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251414d410 con 0x7f251414ca40 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.480+0000 7f2518e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140cda60 con 0x7f25140c36d0 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46866/0 (socket says 172.21.15.91:46866) 2024-02-02T13:35:18.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 -- 172.21.15.91:0/1478619302 learned_addr learned my addr 172.21.15.91:0/1478619302 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:18.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 -- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140c36d0 msgr2=0x7f25140c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2509200640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140c36d0 0x7f25140c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2512800640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f251414ca40 0x7f251414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140c36d0 0x7f25140c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 -- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f251414ca40 msgr2=0x7f251414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f251414ca40 0x7f251414ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140cdd40 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.481+0000 7f2511e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2504005040 tx=0x7f2504001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.482+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040076c0 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.482+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504007a30 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.482+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504006c20 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.482+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140ce020 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.482+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140cd6e0 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.483+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514066cf0 con 0x7f24f40039c0 2024-02-02T13:35:18.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.484+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f25040035f0 con 0x7f24f40039c0 2024-02-02T13:35:18.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.485+0000 7f250aa00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.485+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f25040aa620 con 0x7f24f40039c0 2024-02-02T13:35:18.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.485+0000 7f2509200640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.485+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f24ec072020 con 0x7f24f40039c0 2024-02-02T13:35:18.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.486+0000 7f2509200640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f25140ad900 tx=0x7f24fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.489+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2504041060 con 0x7f24f40039c0 2024-02-02T13:35:18.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.722+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2514155900 con 0x7f24f40039c0 2024-02-02T13:35:18.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.723+0000 7f250aa00640 1 -- 172.21.15.91:0/1478619302 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f25040796e0 con 0x7f24f40039c0 2024-02-02T13:35:18.724 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:18.725 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:35:18.725 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:35:18.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.725+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 msgr2=0x7f24ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.725+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f25140ad900 tx=0x7f24fc008040 comp rx=0 tx=0).stop 2024-02-02T13:35:18.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.725+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 msgr2=0x7f25140af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.725+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2504005040 tx=0x7f2504001120 comp rx=0 tx=0).stop 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 shutdown_connections 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24ec06a430 0x7f24ec06c8f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25140c36d0 0x7f25140c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f251414ca40 0x7f251414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 --2- 172.21.15.91:0/1478619302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24f40039c0 0x7f25140af750 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 >> 172.21.15.91:0/1478619302 conn(0x7f25140cec80 msgr2=0x7f251415a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 shutdown_connections 2024-02-02T13:35:18.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.726+0000 7f2518e00640 1 -- 172.21.15.91:0/1478619302 wait complete. 2024-02-02T13:35:18.746 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:18.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.951+0000 7fd817a00640 1 Processor -- start 2024-02-02T13:35:18.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.951+0000 7fd817a00640 1 -- start start 2024-02-02T13:35:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.952+0000 7fd817a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd810151f90 0x7fd81015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.952+0000 7fd817a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd81006f4f0 0x7fd8100b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.952+0000 7fd817a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8100b1070 con 0x7fd8100b0c40 2024-02-02T13:35:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.952+0000 7fd817a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd810070f80 con 0x7fd810151f90 2024-02-02T13:35:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.952+0000 7fd817a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd810152370 con 0x7fd81006f4f0 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd810151f90 0x7fd81015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd810151f90 0x7fd81015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46870/0 (socket says 172.21.15.91:46870) 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 -- 172.21.15.91:0/3235027679 learned_addr learned my addr 172.21.15.91:0/3235027679 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd80fe00640 1 --2- 172.21.15.91:0/3235027679 >> v2:172.21.15.113:3300/0 conn(0x7fd81006f4f0 0x7fd8100b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 -- 172.21.15.91:0/3235027679 >> v2:172.21.15.113:3300/0 conn(0x7fd81006f4f0 msgr2=0x7fd8100b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 --2- 172.21.15.91:0/3235027679 >> v2:172.21.15.113:3300/0 conn(0x7fd81006f4f0 0x7fd8100b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 -- 172.21.15.91:0/3235027679 >> v1:172.21.15.113:6789/0 conn(0x7fd8100b0c40 legacy=0x7fd8100b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.953+0000 7fd815000640 1 -- 172.21.15.91:0/3235027679 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100cb010 con 0x7fd810151f90 2024-02-02T13:35:18.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.955+0000 7fd815000640 1 --2- 172.21.15.91:0/3235027679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd810151f90 0x7fd81015f5a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd810071a00 tx=0x7fd80002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378cf4452fbfade server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.955+0000 7fd80f400640 1 -- 172.21.15.91:0/3235027679 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800031870 con 0x7fd810151f90 2024-02-02T13:35:18.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.955+0000 7fd80f400640 1 -- 172.21.15.91:0/3235027679 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000319d0 con 0x7fd810151f90 2024-02-02T13:35:18.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.955+0000 7fd80f400640 1 -- 172.21.15.91:0/3235027679 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800031d60 con 0x7fd810151f90 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 -- 172.21.15.91:0/3235027679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd810151f90 msgr2=0x7fd81015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 --2- 172.21.15.91:0/3235027679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd810151f90 0x7fd81015f5a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd810071a00 tx=0x7fd80002b910 comp rx=0 tx=0).stop 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 -- 172.21.15.91:0/3235027679 shutdown_connections 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 --2- 172.21.15.91:0/3235027679 >> v2:172.21.15.113:3300/0 conn(0x7fd81006f4f0 0x7fd8100b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 --2- 172.21.15.91:0/3235027679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd810151f90 0x7fd81015f5a0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 -- 172.21.15.91:0/3235027679 >> 172.21.15.91:0/3235027679 conn(0x7fd8100cec80 msgr2=0x7fd8100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 -- 172.21.15.91:0/3235027679 shutdown_connections 2024-02-02T13:35:18.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.956+0000 7fd817a00640 1 -- 172.21.15.91:0/3235027679 wait complete. 2024-02-02T13:35:18.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.959+0000 7fd817a00640 1 Processor -- start 2024-02-02T13:35:18.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.960+0000 7fd817a00640 1 -- start start 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.961+0000 7fd817a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 0x7fd81014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.961+0000 7fd817a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.961+0000 7fd817a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd810151f90 0x7fd81015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.961+0000 7fd817a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81014e200 con 0x7fd8100b0c40 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd817a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100c6510 con 0x7fd81006f4f0 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd817a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100c67f0 con 0x7fd810151f90 2024-02-02T13:35:18.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46878/0 (socket says 172.21.15.91:46878) 2024-02-02T13:35:18.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 -- 172.21.15.91:0/3035718005 learned_addr learned my addr 172.21.15.91:0/3035718005 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd815000640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 0x7fd81014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd815a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd810151f90 0x7fd81015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 -- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd810151f90 msgr2=0x7fd81015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd810151f90 0x7fd81015a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 -- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 msgr2=0x7fd81014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 0x7fd81014dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd80fe00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000094d0 con 0x7fd8100b0c40 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.962+0000 7fd815000640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 0x7fd81014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:18.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd80fe00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c7f0 tx=0x7fd7f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8004220 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f8004380 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007840 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8100c6ad0 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.963+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8100c6200 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.964+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd810066cf0 con 0x7fd8100b0c40 2024-02-02T13:35:18.966 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.965+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7f8007bb0 con 0x7fd8100b0c40 2024-02-02T13:35:18.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.966+0000 7fd80cc00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 0x7fd7e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:18.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.966+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7f8088930 con 0x7fd8100b0c40 2024-02-02T13:35:18.967 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.966+0000 7fd815000640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 0x7fd7e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:18.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.966+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7fd7e8072020 con 0x7fd8100b0c40 2024-02-02T13:35:18.968 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.967+0000 7fd815000640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 0x7fd7e806c8f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd800036000 tx=0x7fd80003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:18.971 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:18.970+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7f8053e40 con 0x7fd8100b0c40 2024-02-02T13:35:19.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.164+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd81006c7b0 con 0x7fd8100b0c40 2024-02-02T13:35:19.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.165+0000 7fd80cc00640 1 -- 172.21.15.91:0/3035718005 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16235 (secure 0 0 0) 0x7fd7f801e020 con 0x7fd8100b0c40 2024-02-02T13:35:19.167 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:19.167 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":165,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:34:29.471448+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:19.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 msgr2=0x7fd7e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 0x7fd7e806c8f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd800036000 tx=0x7fd80003c040 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 msgr2=0x7fd8100c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c7f0 tx=0x7fd7f800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 shutdown_connections 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e806a430 0x7fd7e806c8f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd810151f90 0x7fd81015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd8100b0c40 0x7fd8100c5cc0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 --2- 172.21.15.91:0/3035718005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd81006f4f0 0x7fd81014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 >> 172.21.15.91:0/3035718005 conn(0x7fd8100cec80 msgr2=0x7fd8100acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 shutdown_connections 2024-02-02T13:35:19.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.168+0000 7fd817a00640 1 -- 172.21.15.91:0/3035718005 wait complete. 2024-02-02T13:35:19.187 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:35:19.188 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:19.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.390+0000 7f537e800640 1 Processor -- start 2024-02-02T13:35:19.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.390+0000 7f537e800640 1 -- start start 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5378151f90 0x7f537815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 0x7f53780b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53780b0c40 0x7f53780b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53780b10d0 con 0x7f53780b0c40 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5378070f80 con 0x7f5378151f90 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.391+0000 7f537e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5378152370 con 0x7f537806f4f0 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53780b0c40 0x7f53780b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53780b0c40 0x7f53780b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46888/0 (socket says 172.21.15.91:46888) 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 -- 172.21.15.91:0/1105764619 learned_addr learned my addr 172.21.15.91:0/1105764619 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377400640 1 --2- 172.21.15.91:0/1105764619 >> v2:172.21.15.113:3300/0 conn(0x7f5378151f90 0x7f537815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5376a00640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 0x7f53780b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 -- 172.21.15.91:0/1105764619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 msgr2=0x7f53780b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:19.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 0x7f53780b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 -- 172.21.15.91:0/1105764619 >> v2:172.21.15.113:3300/0 conn(0x7f5378151f90 msgr2=0x7f537815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:19.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 --2- 172.21.15.91:0/1105764619 >> v2:172.21.15.113:3300/0 conn(0x7f5378151f90 0x7f537815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5377e00640 1 -- 172.21.15.91:0/1105764619 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780cb010 con 0x7f53780b0c40 2024-02-02T13:35:19.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.392+0000 7f5376a00640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 0x7f53780b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:19.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.393+0000 7f5377e00640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53780b0c40 0x7f53780b3af0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f53780cc770 tx=0x7f537002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9b77719110f390 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.394+0000 7f5376000640 1 -- 172.21.15.91:0/1105764619 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370031870 con 0x7f53780b0c40 2024-02-02T13:35:19.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.394+0000 7f5376000640 1 -- 172.21.15.91:0/1105764619 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53700319d0 con 0x7f53780b0c40 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.394+0000 7f5376000640 1 -- 172.21.15.91:0/1105764619 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370031d60 con 0x7f53780b0c40 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.394+0000 7f537e800640 1 -- 172.21.15.91:0/1105764619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53780b0c40 msgr2=0x7f53780b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.394+0000 7f537e800640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53780b0c40 0x7f53780b3af0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f53780cc770 tx=0x7f537002b910 comp rx=0 tx=0).stop 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 -- 172.21.15.91:0/1105764619 shutdown_connections 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53780b0c40 0x7f53780b3af0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 --2- 172.21.15.91:0/1105764619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537806f4f0 0x7f53780b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 --2- 172.21.15.91:0/1105764619 >> v2:172.21.15.113:3300/0 conn(0x7f5378151f90 0x7f537815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 -- 172.21.15.91:0/1105764619 >> 172.21.15.91:0/1105764619 conn(0x7f53780cec80 msgr2=0x7f53780cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 -- 172.21.15.91:0/1105764619 shutdown_connections 2024-02-02T13:35:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.395+0000 7f537e800640 1 -- 172.21.15.91:0/1105764619 wait complete. 2024-02-02T13:35:19.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.397+0000 7f537e800640 1 Processor -- start 2024-02-02T13:35:19.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.397+0000 7f537e800640 1 -- start start 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.398+0000 7f537e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.398+0000 7f537e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780b0c40 0x7f5378157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f537e800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5378151f90 0x7f53780b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f537e800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53781f0470 con 0x7f537806f4f0 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f537e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53780b1de0 con 0x7f53780b0c40 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f537e800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53780b2010 con 0x7f5378151f90 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46894/0 (socket says 172.21.15.91:46894) 2024-02-02T13:35:19.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 -- 172.21.15.91:0/2981768113 learned_addr learned my addr 172.21.15.91:0/2981768113 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5376a00640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780b0c40 0x7f5378157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377e00640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5378151f90 0x7f53780b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 -- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5378151f90 msgr2=0x7f53780b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5378151f90 0x7f53780b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 -- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780b0c40 msgr2=0x7f5378157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780b0c40 0x7f5378157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.399+0000 7f5377400640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700094d0 con 0x7f537806f4f0 2024-02-02T13:35:19.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f5377400640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f536000d800 tx=0x7f536000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000b6c0 con 0x7f537806f4f0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536000b820 con 0x7f537806f4f0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000bbb0 con 0x7f537806f4f0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53780b22f0 con 0x7f537806f4f0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.400+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53780b28b0 con 0x7f537806f4f0 2024-02-02T13:35:19.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.401+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5378066cf0 con 0x7f537806f4f0 2024-02-02T13:35:19.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.402+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f536000bd10 con 0x7f537806f4f0 2024-02-02T13:35:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.403+0000 7f536f400640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 0x7f535006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.403+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5360088dd0 con 0x7f537806f4f0 2024-02-02T13:35:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.403+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f5350072020 con 0x7f537806f4f0 2024-02-02T13:35:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.403+0000 7f5376a00640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 0x7f535006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.404+0000 7f5376a00640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 0x7f535006c8f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5378151000 tx=0x7f5368008430 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.411 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.410+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5360054370 con 0x7f537806f4f0 2024-02-02T13:35:19.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.493+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(166..166 src has 1..166) v4 ==== 679+0+0 (secure 0 0 0) 0x7f536008d070 con 0x7f537806f4f0 2024-02-02T13:35:19.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.493+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f5350072bb0 con 0x7f537806f4f0 2024-02-02T13:35:19.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.648+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53780711d0 con 0x7f537806f4f0 2024-02-02T13:35:19.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.649+0000 7f536f400640 1 -- 172.21.15.91:0/2981768113 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f5360057990 con 0x7f537806f4f0 2024-02-02T13:35:19.651 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:19.651 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:35:19.652 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 msgr2=0x7f535006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 0x7f535006c8f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5378151000 tx=0x7f5368008430 comp rx=0 tx=0).stop 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 msgr2=0x7f5378150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f536000d800 tx=0x7f536000dcc0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 shutdown_connections 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f535006a430 0x7f535006c8f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5378151f90 0x7f53780b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780b0c40 0x7f5378157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 --2- 172.21.15.91:0/2981768113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f537806f4f0 0x7f5378150ac0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.652+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 >> 172.21.15.91:0/2981768113 conn(0x7f53780cec80 msgr2=0x7f53780c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.653+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 shutdown_connections 2024-02-02T13:35:19.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.653+0000 7f537e800640 1 -- 172.21.15.91:0/2981768113 wait complete. 2024-02-02T13:35:19.672 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:19.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.872+0000 7f1e5a600640 1 Processor -- start 2024-02-02T13:35:19.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.872+0000 7f1e5a600640 1 -- start start 2024-02-02T13:35:19.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e5a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e540b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e5a600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5415fa00 con 0x7f1e540b0c40 2024-02-02T13:35:19.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e5a600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5406c600 con 0x7f1e54151f90 2024-02-02T13:35:19.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e5a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b10d0 con 0x7f1e5415f5a0 2024-02-02T13:35:19.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e53e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1e540b0c40 0x7f1e540b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45982/0 (socket says 172.21.15.91:45982) 2024-02-02T13:35:19.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e53e00640 1 -- 172.21.15.91:0/1507374216 learned_addr learned my addr 172.21.15.91:0/1507374216 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:19.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e4be00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e540b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743754001 0 0) 0x7f1e4c001490 con 0x7f1e540b0c40 2024-02-02T13:35:19.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.874+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e34003520 con 0x7f1e540b0c40 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1457168887 0 0) 0x7f1e34003520 con 0x7f1e540b0c40 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 msgr2=0x7f1e540b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e540b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 >> v1:172.21.15.113:6789/0 conn(0x7f1e54151f90 legacy=0x7f1e5406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54070f80 con 0x7f1e540b0c40 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1e4c001b80 con 0x7f1e540b0c40 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 >> v1:172.21.15.91:6789/0 conn(0x7f1e540b0c40 legacy=0x7f1e540b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e34004d00 0x7f1e34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b10d0 con 0x7f1e340039a0 2024-02-02T13:35:19.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3400ba90 con 0x7f1e34004d00 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3400bc50 con 0x7f1e5415f5a0 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.875+0000 7f1e53400640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e4be00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53e00640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e34004d00 0x7f1e34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53400640 1 -- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 msgr2=0x7f1e340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53400640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5415f5a0 0x7f1e340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53400640 1 -- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e34004d00 msgr2=0x7f1e34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53400640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e34004d00 0x7f1e34005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.876+0000 7f1e53400640 1 -- 172.21.15.91:0/1507374216 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34003520 con 0x7f1e340039a0 2024-02-02T13:35:19.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.877+0000 7f1e53400640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e34003dc0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1e44001720 tx=0x7f1e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e4a959e0c5d167 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.877+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440052d0 con 0x7f1e340039a0 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.877+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4402e050 con 0x7f1e340039a0 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.877+0000 7f1e52a00640 1 -- 172.21.15.91:0/1507374216 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440035f0 con 0x7f1e340039a0 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 -- 172.21.15.91:0/1507374216 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 msgr2=0x7f1e34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 --2- 172.21.15.91:0/1507374216 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e34003dc0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1e44001720 tx=0x7f1e340057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e53e00640 1 -- 172.21.15.91:0/1507374216 reap_dead start 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 -- 172.21.15.91:0/1507374216 shutdown_connections 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 -- 172.21.15.91:0/1507374216 >> 172.21.15.91:0/1507374216 conn(0x7f1e540cec80 msgr2=0x7f1e540cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 -- 172.21.15.91:0/1507374216 shutdown_connections 2024-02-02T13:35:19.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.878+0000 7f1e5a600640 1 -- 172.21.15.91:0/1507374216 wait complete. 2024-02-02T13:35:19.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.880+0000 7f1e5a600640 1 Processor -- start 2024-02-02T13:35:19.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.880+0000 7f1e5a600640 1 -- start start 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e540cd450 0x7f1e540c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e540ad950 0x7f1e540addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ad4c0 con 0x7f1e340039a0 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ac8d0 con 0x7f1e540cd450 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e5a600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540acb90 con 0x7f1e540ad950 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e53400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e53400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47632/0 (socket says 172.21.15.91:47632) 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.881+0000 7f1e53400640 1 -- 172.21.15.91:0/431782115 learned_addr learned my addr 172.21.15.91:0/431782115 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e4be00640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e540cd450 0x7f1e540c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 -- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e540ad950 msgr2=0x7f1e540addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e540ad950 0x7f1e540addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.884 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 -- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e540cd450 msgr2=0x7f1e540c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e540cd450 0x7f1e540c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 -- 172.21.15.91:0/431782115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540ace70 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e53400640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1e44008fd0 tx=0x7f1e44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440076c0 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e440069e0 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540ad150 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.882+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44006d50 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.883+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e540c6df0 con 0x7f1e340039a0 2024-02-02T13:35:19.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.884+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e54066cf0 con 0x7f1e340039a0 2024-02-02T13:35:19.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.885+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1e440035f0 con 0x7f1e340039a0 2024-02-02T13:35:19.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.886+0000 7f1e4b400640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 0x7f1e2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:19.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.886+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e440aa7d0 con 0x7f1e340039a0 2024-02-02T13:35:19.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.886+0000 7f1e4be00640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 0x7f1e2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:19.888 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.887+0000 7f1e4be00640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 0x7f1e2c06c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cc990 tx=0x7f1e3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:19.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:19.891+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1e44075e00 con 0x7f1e340039a0 2024-02-02T13:35:20.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.116+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e5406c7b0 con 0x7f1e340039a0 2024-02-02T13:35:20.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.117+0000 7f1e4b400640 1 -- 172.21.15.91:0/431782115 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f1e44079420 con 0x7f1e340039a0 2024-02-02T13:35:20.119 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:20.119 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:32:55.114755+0000","modified":"2024-02-02T13:32:58.630163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2641930599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2641930599},{"type":"v1","addr":"172.21.15.91:6837","nonce":2641930599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-02-02T13:35:20.119 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 72 2024-02-02T13:35:20.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.119+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 msgr2=0x7f1e2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.119+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 0x7f1e2c06c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cc990 tx=0x7f1e3c008040 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 msgr2=0x7f1e5414b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1e44008fd0 tx=0x7f1e44001120 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 shutdown_connections 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1e2c06a430 0x7f1e2c06c8f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e540ad950 0x7f1e540addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e540cd450 0x7f1e540c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 --2- 172.21.15.91:0/431782115 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e340039a0 0x7f1e5414b390 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 >> 172.21.15.91:0/431782115 conn(0x7f1e540cec80 msgr2=0x7f1e54159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 shutdown_connections 2024-02-02T13:35:20.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.120+0000 7f1e5a600640 1 -- 172.21.15.91:0/431782115 wait complete. 2024-02-02T13:35:20.140 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:20.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.349+0000 7f854cc00640 1 Processor -- start 2024-02-02T13:35:20.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.349+0000 7f854cc00640 1 -- start start 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.350+0000 7f854cc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f854815a010 0x7f85480c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.350+0000 7f854cc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 0x7f85480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.350+0000 7f854cc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85480cc7c0 con 0x7f85480cd470 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.350+0000 7f854cc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85480ce930 con 0x7f854815a010 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.350+0000 7f854cc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854806c890 con 0x7f85480ce130 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8546800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f85480cd470 0x7f85480b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45984/0 (socket says 172.21.15.91:45984) 2024-02-02T13:35:20.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8546800640 1 -- 172.21.15.91:0/3793875325 learned_addr learned my addr 172.21.15.91:0/3793875325 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:20.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8545e00640 1 --2- 172.21.15.91:0/3793875325 >> v2:172.21.15.113:3300/0 conn(0x7f854815a010 0x7f85480c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8545400640 1 --2- 172.21.15.91:0/3793875325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 0x7f85480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8544a00640 1 -- 172.21.15.91:0/3793875325 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156335209 0 0) 0x7f85480cc7c0 con 0x7f85480cd470 2024-02-02T13:35:20.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.351+0000 7f8544a00640 1 -- 172.21.15.91:0/3793875325 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8528003520 con 0x7f85480cd470 2024-02-02T13:35:20.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.352+0000 7f8545400640 1 -- 172.21.15.91:0/3793875325 >> v2:172.21.15.113:3300/0 conn(0x7f854815a010 msgr2=0x7f85480c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:20.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.352+0000 7f8545400640 1 --2- 172.21.15.91:0/3793875325 >> v2:172.21.15.113:3300/0 conn(0x7f854815a010 0x7f85480c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.352+0000 7f8545400640 1 -- 172.21.15.91:0/3793875325 >> v1:172.21.15.91:6789/0 conn(0x7f85480cd470 legacy=0x7f85480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.352+0000 7f8545400640 1 -- 172.21.15.91:0/3793875325 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854806cb70 con 0x7f85480ce130 2024-02-02T13:35:20.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.353+0000 7f8545400640 1 --2- 172.21.15.91:0/3793875325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 0x7f85480ccf30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85300097a0 tx=0x7f853002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c20c6d5e7395a5d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.354+0000 7f8544a00640 1 -- 172.21.15.91:0/3793875325 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853002d040 con 0x7f85480ce130 2024-02-02T13:35:20.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.354+0000 7f8544a00640 1 -- 172.21.15.91:0/3793875325 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530031810 con 0x7f85480ce130 2024-02-02T13:35:20.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.354+0000 7f8544a00640 1 -- 172.21.15.91:0/3793875325 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530031ba0 con 0x7f85480ce130 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 -- 172.21.15.91:0/3793875325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 msgr2=0x7f85480ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 --2- 172.21.15.91:0/3793875325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 0x7f85480ccf30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85300097a0 tx=0x7f853002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 -- 172.21.15.91:0/3793875325 shutdown_connections 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 --2- 172.21.15.91:0/3793875325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480ce130 0x7f85480ccf30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 --2- 172.21.15.91:0/3793875325 >> v2:172.21.15.113:3300/0 conn(0x7f854815a010 0x7f85480c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 -- 172.21.15.91:0/3793875325 >> 172.21.15.91:0/3793875325 conn(0x7f85480cec80 msgr2=0x7f85480cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 -- 172.21.15.91:0/3793875325 shutdown_connections 2024-02-02T13:35:20.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.355+0000 7f854cc00640 1 -- 172.21.15.91:0/3793875325 wait complete. 2024-02-02T13:35:20.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.357+0000 7f854cc00640 1 Processor -- start 2024-02-02T13:35:20.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.357+0000 7f854cc00640 1 -- start start 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85480ce130 0x7f8548158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480c6200 0x7f854815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85481ee080 con 0x7f854815a010 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85480c6710 con 0x7f85480ce130 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f854cc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480c69f0 con 0x7f85480c6200 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f8545400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f8545400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47636/0 (socket says 172.21.15.91:47636) 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.358+0000 7f8545400640 1 -- 172.21.15.91:0/2153395363 learned_addr learned my addr 172.21.15.91:0/2153395363 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545e00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85480ce130 0x7f8548158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8546800640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480c6200 0x7f854815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 -- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480c6200 msgr2=0x7f854815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480c6200 0x7f854815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 -- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85480ce130 msgr2=0x7f8548158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85480ce130 0x7f8548158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 -- 172.21.15.91:0/2153395363 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009400 con 0x7f854815a010 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f8545400640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f85300098c0 tx=0x7f85300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853003ace0 con 0x7f854815a010 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853003ae60 con 0x7f854815a010 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85480c6cd0 con 0x7f854815a010 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.359+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530044640 con 0x7f854815a010 2024-02-02T13:35:20.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.360+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85480c7290 con 0x7f854815a010 2024-02-02T13:35:20.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.361+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548066cf0 con 0x7f854815a010 2024-02-02T13:35:20.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.362+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f853004d440 con 0x7f854815a010 2024-02-02T13:35:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.362+0000 7f853e000640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 0x7f852006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.363+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f85300aab20 con 0x7f854815a010 2024-02-02T13:35:20.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.363+0000 7f8545e00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 0x7f852006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.363+0000 7f8545e00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 0x7f852006c920 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8538009970 tx=0x7f8538008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.367+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f85300760d0 con 0x7f854815a010 2024-02-02T13:35:20.561 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.559+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85480b4c00 con 0x7f854815a010 2024-02-02T13:35:20.562 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.561+0000 7f853e000640 1 -- 172.21.15.91:0/2153395363 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16216 (secure 0 0 0) 0x7f8530035090 con 0x7f854815a010 2024-02-02T13:35:20.563 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:20.563 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":166,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:35:19.489697+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:32:53.511243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-02-02T13:32:54.111607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:20.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 msgr2=0x7f852006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 0x7f852006c920 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8538009970 tx=0x7f8538008040 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 msgr2=0x7f85480c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f85300098c0 tx=0x7f85300046c0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 shutdown_connections 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f852006a460 0x7f852006c920 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85480c6200 0x7f854815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f854815a010 0x7f85480c5cc0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 --2- 172.21.15.91:0/2153395363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85480ce130 0x7f8548158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 >> 172.21.15.91:0/2153395363 conn(0x7f85480cec80 msgr2=0x7f85480b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 shutdown_connections 2024-02-02T13:35:20.566 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.564+0000 7f854cc00640 1 -- 172.21.15.91:0/2153395363 wait complete. 2024-02-02T13:35:20.584 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:35:20.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.783+0000 7f250ac00640 1 Processor -- start 2024-02-02T13:35:20.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.783+0000 7f250ac00640 1 -- start start 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.784+0000 7f250ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f250415d0d0 0x7f25040cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f25040cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250ac00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25040cde20 con 0x7f25040b8bb0 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040d2a20 con 0x7f250415d0d0 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250415d4b0 con 0x7f25040b8750 2024-02-02T13:35:20.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250a200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f25040b8bb0 0x7f25040d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:45998/0 (socket says 172.21.15.91:45998) 2024-02-02T13:35:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f250a200640 1 -- 172.21.15.91:0/3208531384 learned_addr learned my addr 172.21.15.91:0/3208531384 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f2508e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f25040cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184783531 0 0) 0x7f25040cde20 con 0x7f25040b8bb0 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003520 con 0x7f25040b8bb0 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.785+0000 7f2509800640 1 --2- 172.21.15.91:0/3208531384 >> v2:172.21.15.113:3300/0 conn(0x7f250415d0d0 0x7f25040cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3596031558 0 0) 0x7f24e4003520 con 0x7f25040b8bb0 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 msgr2=0x7f25040cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f25040cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 >> v2:172.21.15.113:3300/0 conn(0x7f250415d0d0 msgr2=0x7f25040cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 --2- 172.21.15.91:0/3208531384 >> v2:172.21.15.113:3300/0 conn(0x7f250415d0d0 0x7f25040cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040d23f0 con 0x7f25040b8bb0 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f24fc001670 con 0x7f25040b8bb0 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2509800640 1 --2- 172.21.15.91:0/3208531384 >> v2:172.21.15.113:3300/0 conn(0x7f250415d0d0 0x7f25040cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 >> v1:172.21.15.91:6789/0 conn(0x7f25040b8bb0 legacy=0x7f25040d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f24e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24e4004d00 0x7f24e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25040d2a20 con 0x7f24e4004d00 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e400ba30 con 0x7f24e4003970 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e400bb70 con 0x7f25040b8750 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f250a200640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24e4004d00 0x7f24e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2509800640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.786+0000 7f2508e00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f24e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.787+0000 7f250a200640 1 -- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 msgr2=0x7f24e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.787+0000 7f250a200640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b8750 0x7f24e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.787+0000 7f250a200640 1 -- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4003970 msgr2=0x7f24e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.787+0000 7f250a200640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.787+0000 7f250a200640 1 -- 172.21.15.91:0/3208531384 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4003520 con 0x7f24e4004d00 2024-02-02T13:35:20.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.788+0000 7f250a200640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24e4004d00 0x7f24e4005180 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00c240 tx=0x7f24e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db407813de18632b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.788+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc005ad0 con 0x7f24e4004d00 2024-02-02T13:35:20.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.788+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc005c30 con 0x7f24e4004d00 2024-02-02T13:35:20.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.788+0000 7f2503e00640 1 -- 172.21.15.91:0/3208531384 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc02c8b0 con 0x7f24e4004d00 2024-02-02T13:35:20.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 -- 172.21.15.91:0/3208531384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24e4004d00 msgr2=0x7f24e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 --2- 172.21.15.91:0/3208531384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f24e4004d00 0x7f24e4005180 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00c240 tx=0x7f24e40057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250a200640 1 -- 172.21.15.91:0/3208531384 reap_dead start 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 -- 172.21.15.91:0/3208531384 shutdown_connections 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 -- 172.21.15.91:0/3208531384 >> 172.21.15.91:0/3208531384 conn(0x7f2504065910 msgr2=0x7f2504065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 -- 172.21.15.91:0/3208531384 shutdown_connections 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.789+0000 7f250ac00640 1 -- 172.21.15.91:0/3208531384 wait complete. 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.790+0000 7f250ac00640 1 Processor -- start 2024-02-02T13:35:20.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 -- start start 2024-02-02T13:35:20.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 0x7f250415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 0x7f25040c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25040b4b50 con 0x7f250415ec50 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250415dbb0 con 0x7f24e4004d00 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.791+0000 7f250ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250415de90 con 0x7f25040b5600 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47650/0 (socket says 172.21.15.91:47650) 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 -- 172.21.15.91:0/290669002 learned_addr learned my addr 172.21.15.91:0/290669002 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f250a200640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 0x7f25040c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2509800640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 0x7f250415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 -- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 msgr2=0x7f25040c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 0x7f25040c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 -- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 msgr2=0x7f250415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 0x7f250415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:20.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2508e00640 1 -- 172.21.15.91:0/290669002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250415e170 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f250a200640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 0x7f25040c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.792+0000 7f2509800640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 0x7f250415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f2508e00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00f580 tx=0x7f24ec00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec013810 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec013970 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec013ce0 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250415e450 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.793+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2504158730 con 0x7f250415ec50 2024-02-02T13:35:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.794+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250404dff0 con 0x7f250415ec50 2024-02-02T13:35:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.795+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f24ec01f460 con 0x7f250415ec50 2024-02-02T13:35:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.796+0000 7f2501600640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 0x7f24dc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.796+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f24ec088d90 con 0x7f250415ec50 2024-02-02T13:35:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.796+0000 7f2509800640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 0x7f24dc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.797+0000 7f2509800640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 0x7f24dc06c920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f250415abd0 tx=0x7f24f400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:20.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:20.800+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f24ec0542b0 con 0x7f250415ec50 2024-02-02T13:35:21.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.026+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f25040c66e0 con 0x7f250415ec50 2024-02-02T13:35:21.029 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:35:21.028+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:35:21.050 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.048+0000 7f2501600640 1 -- 172.21.15.91:0/290669002 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7f24ec0578d0 con 0x7f250415ec50 2024-02-02T13:35:21.050 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 msgr2=0x7f24dc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 0x7f24dc06c920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f250415abd0 tx=0x7f24f400b040 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 msgr2=0x7f25041581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00f580 tx=0x7f24ec00fa40 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 shutdown_connections 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f24dc06a460 0x7f24dc06c920 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25040b5600 0x7f25040c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f250415ec50 0x7f25041581f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 --2- 172.21.15.91:0/290669002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e4004d00 0x7f250415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 >> 172.21.15.91:0/290669002 conn(0x7f2504065910 msgr2=0x7f25040cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 shutdown_connections 2024-02-02T13:35:21.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.052+0000 7f250ac00640 1 -- 172.21.15.91:0/290669002 wait complete. 2024-02-02T13:35:21.075 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:35:21.118 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:35:21.254 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.253+0000 7fa15de00640 1 Processor -- start 2024-02-02T13:35:21.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.253+0000 7fa15de00640 1 -- start start 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.254+0000 7fa15de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa158164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.254+0000 7fa15de00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580cca70 con 0x7fa1580c9f90 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.254+0000 7fa15de00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580cc770 con 0x7fa15815a1e0 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.254+0000 7fa15de00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ca570 con 0x7fa1580b0da0 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa14de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa1580c9f90 0x7fa1580b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46030/0 (socket says 172.21.15.91:46030) 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa14de00640 1 -- 172.21.15.91:0/2597880863 learned_addr learned my addr 172.21.15.91:0/2597880863 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa157400640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa158164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131703431 0 0) 0x7fa1580cca70 con 0x7fa1580c9f90 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa138003520 con 0x7fa1580c9f90 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495574367 0 0) 0x7fa138003520 con 0x7fa1580c9f90 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 msgr2=0x7fa158164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa158164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 >> v1:172.21.15.113:6789/0 conn(0x7fa15815a1e0 legacy=0x7fa158070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580ca850 con 0x7fa1580c9f90 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa140001670 con 0x7fa1580c9f90 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 >> v1:172.21.15.91:6789/0 conn(0x7fa1580c9f90 legacy=0x7fa1580b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa1380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138003f70 0x7fa1380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.255+0000 7fa156000640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa138005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ca570 con 0x7fa138004d00 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13800bac0 con 0x7fa138003f70 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13800bc80 con 0x7fa1580b0da0 2024-02-02T13:35:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa138005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa156a00640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa1380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 -- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 msgr2=0x7fa1380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa14de00640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138003f70 0x7fa1380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa1380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 -- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138003f70 msgr2=0x7fa1380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138003f70 0x7fa1380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa157400640 1 -- 172.21.15.91:0/2597880863 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138003520 con 0x7fa138004d00 2024-02-02T13:35:21.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.256+0000 7fa156a00640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580b0da0 0x7fa1380039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:21.261 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.257+0000 7fa157400640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa138005180 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa158071970 tx=0x7fa1380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e700d8d436a977 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.257+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15002cb90 con 0x7fa138004d00 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.257+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000f040 con 0x7fa138004d00 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.257+0000 7fa156000640 1 -- 172.21.15.91:0/2597880863 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150034440 con 0x7fa138004d00 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 -- 172.21.15.91:0/2597880863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 msgr2=0x7fa138005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 --2- 172.21.15.91:0/2597880863 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa138005180 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa158071970 tx=0x7fa1380057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa157400640 1 -- 172.21.15.91:0/2597880863 reap_dead start 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 -- 172.21.15.91:0/2597880863 shutdown_connections 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 -- 172.21.15.91:0/2597880863 >> 172.21.15.91:0/2597880863 conn(0x7fa1580ceb90 msgr2=0x7fa1580cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 -- 172.21.15.91:0/2597880863 shutdown_connections 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.258+0000 7fa15de00640 1 -- 172.21.15.91:0/2597880863 wait complete. 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.260+0000 7fa15de00640 1 Processor -- start 2024-02-02T13:35:21.262 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.260+0000 7fa15de00640 1 -- start start 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 0x7fa1580b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1580b3f00 0x7fa1581f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580b4380 con 0x7fa138004d00 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580afec0 con 0x7fa1580b3f00 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa15de00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580c7b70 con 0x7fa1580c51f0 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47670/0 (socket says 172.21.15.91:47670) 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 -- 172.21.15.91:0/2147946072 learned_addr learned my addr 172.21.15.91:0/2147946072 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa14de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 0x7fa1580b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 -- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 msgr2=0x7fa1580b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 0x7fa1580b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 -- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1580b3f00 msgr2=0x7fa1581f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1580b3f00 0x7fa1581f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa156a00640 1 -- 172.21.15.91:0/2147946072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580c7e50 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.261+0000 7fa14de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 0x7fa1580b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa156a00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d7a0 tx=0x7fa14800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148017070 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148005070 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148005420 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1580c8130 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.262+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1580c86f0 con 0x7fa138004d00 2024-02-02T13:35:21.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.263+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158066cf0 con 0x7fa138004d00 2024-02-02T13:35:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.264+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa148007030 con 0x7fa138004d00 2024-02-02T13:35:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.265+0000 7fa14fe00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 0x7fa13006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.265+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa148088220 con 0x7fa138004d00 2024-02-02T13:35:21.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.265+0000 7fa14de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 0x7fa13006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:21.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.266+0000 7fa14de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 0x7fa13006c920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa140001eb0 tx=0x7fa140009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:21.270 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.269+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa148053770 con 0x7fa138004d00 2024-02-02T13:35:21.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:21.497+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa1580c9920 con 0x7fa138004d00 2024-02-02T13:35:22.053 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.051+0000 7fa14fe00640 1 -- 172.21.15.91:0/2147946072 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa148056d90 con 0x7fa138004d00 2024-02-02T13:35:22.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 msgr2=0x7fa13006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:22.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 0x7fa13006c920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa140001eb0 tx=0x7fa140009040 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 msgr2=0x7fa1580c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d7a0 tx=0x7fa14800dc60 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 shutdown_connections 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa13006a460 0x7fa13006c920 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1580c51f0 0x7fa1580b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1580b3f00 0x7fa1581f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 --2- 172.21.15.91:0/2147946072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa138004d00 0x7fa1580c65e0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 >> 172.21.15.91:0/2147946072 conn(0x7fa1580ceb90 msgr2=0x7fa15806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 shutdown_connections 2024-02-02T13:35:22.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.055+0000 7fa15de00640 1 -- 172.21.15.91:0/2147946072 wait complete. 2024-02-02T13:35:22.079 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:35:22.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.337+0000 7f79dee00640 1 Processor -- start 2024-02-02T13:35:22.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.338+0000 7f79dee00640 1 -- start start 2024-02-02T13:35:22.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79d815d590 0x7f79d80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79d80b1ba0 0x7f79d80b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d80b1360 0x7f79d8069e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80b1980 con 0x7f79d815d590 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80d07b0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79dee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cff10 con 0x7f79d80b1360 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.339+0000 7f79d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79d815d590 0x7f79d80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79d815d590 0x7f79d80b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47686/0 (socket says 172.21.15.91:47686) 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 -- 172.21.15.91:0/1977942567 learned_addr learned my addr 172.21.15.91:0/1977942567 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 -- 172.21.15.91:0/1977942567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d80b1360 msgr2=0x7f79d8069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 --2- 172.21.15.91:0/1977942567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d80b1360 0x7f79d8069e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 -- 172.21.15.91:0/1977942567 >> v2:172.21.15.113:3300/0 conn(0x7f79d80b1ba0 msgr2=0x7f79d80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 --2- 172.21.15.91:0/1977942567 >> v2:172.21.15.113:3300/0 conn(0x7f79d80b1ba0 0x7f79d80b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.340+0000 7f79d7e00640 1 -- 172.21.15.91:0/1977942567 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80d01f0 con 0x7f79d815d590 2024-02-02T13:35:22.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.341+0000 7f79d7e00640 1 --2- 172.21.15.91:0/1977942567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d815d590 0x7f79d80b08e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f79d80c4aa0 tx=0x7f79c802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa72159ea60d1757 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:22.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.341+0000 7f79d6a00640 1 -- 172.21.15.91:0/1977942567 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8031750 con 0x7f79d815d590 2024-02-02T13:35:22.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.341+0000 7f79d6a00640 1 -- 172.21.15.91:0/1977942567 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c80318b0 con 0x7f79d815d590 2024-02-02T13:35:22.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.341+0000 7f79d6a00640 1 -- 172.21.15.91:0/1977942567 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8031c40 con 0x7f79d815d590 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 -- 172.21.15.91:0/1977942567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d815d590 msgr2=0x7f79d80b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 --2- 172.21.15.91:0/1977942567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d815d590 0x7f79d80b08e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f79d80c4aa0 tx=0x7f79c802b7f0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 -- 172.21.15.91:0/1977942567 shutdown_connections 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 --2- 172.21.15.91:0/1977942567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d80b1360 0x7f79d8069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 --2- 172.21.15.91:0/1977942567 >> v2:172.21.15.113:3300/0 conn(0x7f79d80b1ba0 0x7f79d80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 --2- 172.21.15.91:0/1977942567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d815d590 0x7f79d80b08e0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 -- 172.21.15.91:0/1977942567 >> 172.21.15.91:0/1977942567 conn(0x7f79d80d0fa0 msgr2=0x7f79d80d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:22.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 -- 172.21.15.91:0/1977942567 shutdown_connections 2024-02-02T13:35:22.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.342+0000 7f79dee00640 1 -- 172.21.15.91:0/1977942567 wait complete. 2024-02-02T13:35:22.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.344+0000 7f79dee00640 1 Processor -- start 2024-02-02T13:35:22.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.344+0000 7f79dee00640 1 -- start start 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79d80b1360 0x7f79d8160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d815d590 0x7f79d80b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d815f770 con 0x7f79d80b1ba0 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81f73b0 con 0x7f79d80b1360 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79dee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81602d0 con 0x7f79d815d590 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79d7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79d7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47700/0 (socket says 172.21.15.91:47700) 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.345+0000 7f79d7400640 1 -- 172.21.15.91:0/2860596062 learned_addr learned my addr 172.21.15.91:0/2860596062 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 -- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d815d590 msgr2=0x7f79d80b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:22.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d815d590 0x7f79d80b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 -- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79d80b1360 msgr2=0x7f79d8160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79d80b1360 0x7f79d8160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 -- 172.21.15.91:0/2860596062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c80094d0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79d7400640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b480 tx=0x7f79c000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0010870 con 0x7f79d80b1ba0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c00109d0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d815f9a0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0010d80 con 0x7f79d80b1ba0 2024-02-02T13:35:22.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.346+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d815ff60 con 0x7f79d80b1ba0 2024-02-02T13:35:22.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.347+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8160ca0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.348+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f79c000f3c0 con 0x7f79d80b1ba0 2024-02-02T13:35:22.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.349+0000 7f79cfe00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 0x7f79b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:22.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.349+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f79c0088a00 con 0x7f79d80b1ba0 2024-02-02T13:35:22.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.349+0000 7f79d7e00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 0x7f79b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:22.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.350+0000 7f79d7e00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 0x7f79b006c8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f79c8035000 tx=0x7f79c8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:22.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.353+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f79c0053f50 con 0x7f79d80b1ba0 2024-02-02T13:35:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:22.541+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79d8066cf0 con 0x7f79d80b1ba0 2024-02-02T13:35:23.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.062+0000 7f79cfe00640 1 -- 172.21.15.91:0/2860596062 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v168) v1 ==== 181+0+0 (secure 0 0 0) 0x7f79c00140d0 con 0x7f79d80b1ba0 2024-02-02T13:35:23.064 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.066+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 msgr2=0x7f79b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.066+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 0x7f79b006c8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f79c8035000 tx=0x7f79c8030000 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.066+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 msgr2=0x7f79d80ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.066+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b480 tx=0x7f79c000b940 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 shutdown_connections 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f79b006a430 0x7f79b006c8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79d815d590 0x7f79d80b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79d80b1ba0 0x7f79d80ae100 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 --2- 172.21.15.91:0/2860596062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79d80b1360 0x7f79d8160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 >> 172.21.15.91:0/2860596062 conn(0x7f79d80d0fa0 msgr2=0x7f79d815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 shutdown_connections 2024-02-02T13:35:23.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.067+0000 7f79dee00640 1 -- 172.21.15.91:0/2860596062 wait complete. 2024-02-02T13:35:23.091 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:35:23.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.299+0000 7fef20000640 1 Processor -- start 2024-02-02T13:35:23.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.299+0000 7fef20000640 1 -- start start 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef20000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fef180ce130 0x7fef180c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef20000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 0x7fef180b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef20000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180cc7c0 con 0x7fef1815a010 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef20000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef180b10d0 con 0x7fef180ce130 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef20000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef180ce930 con 0x7fef180b0700 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef1d600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fef1815a010 0x7fef180ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46034/0 (socket says 172.21.15.91:46034) 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef1d600640 1 -- 172.21.15.91:0/1884834563 learned_addr learned my addr 172.21.15.91:0/1884834563 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.300+0000 7fef1cc00640 1 --2- 172.21.15.91:0/1884834563 >> v2:172.21.15.91:3300/0 conn(0x7fef180ce130 0x7fef180c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1e000640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 0x7fef180b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef17e00640 1 -- 172.21.15.91:0/1884834563 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844605158 0 0) 0x7fef180cc7c0 con 0x7fef1815a010 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1cc00640 1 -- 172.21.15.91:0/1884834563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 msgr2=0x7fef180b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1cc00640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 0x7fef180b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1cc00640 1 -- 172.21.15.91:0/1884834563 >> v1:172.21.15.91:6789/0 conn(0x7fef1815a010 legacy=0x7fef180ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1cc00640 1 -- 172.21.15.91:0/1884834563 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180cd640 con 0x7fef180ce130 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.301+0000 7fef1e000640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 0x7fef180b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.302+0000 7fef1cc00640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180ce130 0x7fef180c9670 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fef000097a0 tx=0x7fef0002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590f83052e89120b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.302+0000 7fef17e00640 1 -- 172.21.15.91:0/1884834563 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0002d040 con 0x7fef180ce130 2024-02-02T13:35:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.302+0000 7fef17e00640 1 -- 172.21.15.91:0/1884834563 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00031810 con 0x7fef180ce130 2024-02-02T13:35:23.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.302+0000 7fef17e00640 1 -- 172.21.15.91:0/1884834563 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00031ba0 con 0x7fef180ce130 2024-02-02T13:35:23.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 -- 172.21.15.91:0/1884834563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180ce130 msgr2=0x7fef180c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180ce130 0x7fef180c9670 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fef000097a0 tx=0x7fef0002b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 -- 172.21.15.91:0/1884834563 shutdown_connections 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180b0700 0x7fef180b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 --2- 172.21.15.91:0/1884834563 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180ce130 0x7fef180c9670 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 -- 172.21.15.91:0/1884834563 >> 172.21.15.91:0/1884834563 conn(0x7fef180cec80 msgr2=0x7fef180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 -- 172.21.15.91:0/1884834563 shutdown_connections 2024-02-02T13:35:23.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.303+0000 7fef20000640 1 -- 172.21.15.91:0/1884834563 wait complete. 2024-02-02T13:35:23.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.305+0000 7fef20000640 1 Processor -- start 2024-02-02T13:35:23.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.305+0000 7fef20000640 1 -- start start 2024-02-02T13:35:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180ce130 0x7fef18146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef1815a010 0x7fef181f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181474b0 con 0x7fef180b0700 2024-02-02T13:35:23.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181f0f50 con 0x7fef1815a010 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef20000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef181f1230 con 0x7fef180ce130 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef1d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef1d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47724/0 (socket says 172.21.15.91:47724) 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.306+0000 7fef1d600640 1 -- 172.21.15.91:0/2084589016 learned_addr learned my addr 172.21.15.91:0/2084589016 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1cc00640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180ce130 0x7fef18146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1e000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef1815a010 0x7fef181f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 -- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180ce130 msgr2=0x7fef18146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180ce130 0x7fef18146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 -- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef1815a010 msgr2=0x7fef181f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef1815a010 0x7fef181f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:23.310 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 -- 172.21.15.91:0/2084589016 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009400 con 0x7fef180b0700 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef1d600640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c430 tx=0x7fef0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080074e0 con 0x7fef180b0700 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08007660 con 0x7fef180b0700 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08004a20 con 0x7fef180b0700 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef18149870 con 0x7fef180b0700 2024-02-02T13:35:23.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.307+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef18149e30 con 0x7fef180b0700 2024-02-02T13:35:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.308+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef18066cf0 con 0x7fef180b0700 2024-02-02T13:35:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.309+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fef08004b80 con 0x7fef180b0700 2024-02-02T13:35:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.310+0000 7fef15600640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 0x7feef006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.310+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fef080886d0 con 0x7fef180b0700 2024-02-02T13:35:23.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.310+0000 7fef1cc00640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 0x7feef006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:23.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.311+0000 7fef1cc00640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 0x7feef006c8f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fef00036000 tx=0x7fef0003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:23.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.315+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fef08053d40 con 0x7fef180b0700 2024-02-02T13:35:23.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:23.506+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef180cca00 con 0x7fef180b0700 2024-02-02T13:35:24.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.077+0000 7fef15600640 1 -- 172.21.15.91:0/2084589016 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v169) v1 ==== 169+0+0 (secure 0 0 0) 0x7fef08057360 con 0x7fef180b0700 2024-02-02T13:35:24.079 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:35:24.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 msgr2=0x7feef006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 0x7feef006c8f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fef00036000 tx=0x7fef0003c040 comp rx=0 tx=0).stop 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 msgr2=0x7fef1814a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c430 tx=0x7fef0800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 shutdown_connections 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feef006a430 0x7feef006c8f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef180ce130 0x7fef18146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef1815a010 0x7fef181f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.081+0000 7fef20000640 1 --2- 172.21.15.91:0/2084589016 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef180b0700 0x7fef1814a350 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.082+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 >> 172.21.15.91:0/2084589016 conn(0x7fef180cec80 msgr2=0x7fef1806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:24.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.082+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 shutdown_connections 2024-02-02T13:35:24.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.082+0000 7fef20000640 1 -- 172.21.15.91:0/2084589016 wait complete. 2024-02-02T13:35:24.106 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:35:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.325+0000 7f19cea00640 1 Processor -- start 2024-02-02T13:35:24.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.325+0000 7f19cea00640 1 -- start start 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19c815d420 0x7f19c806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19c8065910 0x7f19c8065da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c80cf3c0 con 0x7f19c80af170 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80662e0 con 0x7f19c8065910 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c815d800 con 0x7f19c815d420 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19c8065910 0x7f19c8065da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19c8065910 0x7f19c8065da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47736/0 (socket says 172.21.15.91:47736) 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.327+0000 7f19cca00640 1 -- 172.21.15.91:0/1292715028 learned_addr learned my addr 172.21.15.91:0/1292715028 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.328+0000 7f19cca00640 1 -- 172.21.15.91:0/1292715028 >> v2:172.21.15.113:3300/0 conn(0x7f19c815d420 msgr2=0x7f19c806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.328+0000 7f19cca00640 1 --2- 172.21.15.91:0/1292715028 >> v2:172.21.15.113:3300/0 conn(0x7f19c815d420 0x7f19c806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.328+0000 7f19cca00640 1 -- 172.21.15.91:0/1292715028 >> v1:172.21.15.113:6789/0 conn(0x7f19c80af170 legacy=0x7f19c80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:24.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.328+0000 7f19cca00640 1 -- 172.21.15.91:0/1292715028 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c8071540 con 0x7f19c8065910 2024-02-02T13:35:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.329+0000 7f19cca00640 1 --2- 172.21.15.91:0/1292715028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c8065da0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f19c8072390 tx=0x7f19c002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2554958b13347ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.329+0000 7f19c6a00640 1 -- 172.21.15.91:0/1292715028 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0031750 con 0x7f19c8065910 2024-02-02T13:35:24.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.329+0000 7f19c6a00640 1 -- 172.21.15.91:0/1292715028 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c00318b0 con 0x7f19c8065910 2024-02-02T13:35:24.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.329+0000 7f19c6a00640 1 -- 172.21.15.91:0/1292715028 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0031c40 con 0x7f19c8065910 2024-02-02T13:35:24.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 -- 172.21.15.91:0/1292715028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 msgr2=0x7f19c8065da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 --2- 172.21.15.91:0/1292715028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c8065da0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f19c8072390 tx=0x7f19c002b7f0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 -- 172.21.15.91:0/1292715028 shutdown_connections 2024-02-02T13:35:24.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 --2- 172.21.15.91:0/1292715028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c8065da0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 --2- 172.21.15.91:0/1292715028 >> v2:172.21.15.113:3300/0 conn(0x7f19c815d420 0x7f19c806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 -- 172.21.15.91:0/1292715028 >> 172.21.15.91:0/1292715028 conn(0x7f19c80cc7a0 msgr2=0x7f19c80ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 -- 172.21.15.91:0/1292715028 shutdown_connections 2024-02-02T13:35:24.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.330+0000 7f19cea00640 1 -- 172.21.15.91:0/1292715028 wait complete. 2024-02-02T13:35:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.332+0000 7f19cea00640 1 Processor -- start 2024-02-02T13:35:24.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.332+0000 7f19cea00640 1 -- start start 2024-02-02T13:35:24.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19c80af170 0x7f19c80c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19c815d420 0x7f19c8156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c8770 con 0x7f19c8065910 2024-02-02T13:35:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c4b40 con 0x7f19c815d420 2024-02-02T13:35:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c4e20 con 0x7f19c80af170 2024-02-02T13:35:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47742/0 (socket says 172.21.15.91:47742) 2024-02-02T13:35:24.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 -- 172.21.15.91:0/4233732740 learned_addr learned my addr 172.21.15.91:0/4233732740 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 -- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19c80af170 msgr2=0x7f19c80c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19c80af170 0x7f19c80c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19cca00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19c815d420 0x7f19c8156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 -- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19c815d420 msgr2=0x7f19c8156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.333+0000 7f19c7e00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19c815d420 0x7f19c8156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19c7e00640 1 -- 172.21.15.91:0/4233732740 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00094d0 con 0x7f19c8065910 2024-02-02T13:35:24.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19c7e00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f19b800ba50 tx=0x7f19b800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800cbe0 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800cd60 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8014680 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80c89d0 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.334+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c80c6370 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.335+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c80c5040 con 0x7f19c8065910 2024-02-02T13:35:24.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.336+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f19b801d3b0 con 0x7f19c8065910 2024-02-02T13:35:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.337+0000 7f19bfe00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 0x7f19a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.337+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19b8088580 con 0x7f19c8065910 2024-02-02T13:35:24.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.337+0000 7f19c7400640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 0x7f19a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.338+0000 7f19c7400640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 0x7f19a006c8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f19c81f6160 tx=0x7f19b0006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.342+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f19b8053f00 con 0x7f19c8065910 2024-02-02T13:35:24.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.531+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f19c8066cf0 con 0x7f19c8065910 2024-02-02T13:35:24.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.540+0000 7f19bfe00640 1 -- 172.21.15.91:0/4233732740 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19b8057520 con 0x7f19c8065910 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 msgr2=0x7f19a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 0x7f19a006c8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f19c81f6160 tx=0x7f19b0006ba0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 msgr2=0x7f19c80c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f19b800ba50 tx=0x7f19b800bf10 comp rx=0 tx=0).stop 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 shutdown_connections 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f19a006a430 0x7f19a006c8f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19c80af170 0x7f19c80c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19c815d420 0x7f19c8156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 --2- 172.21.15.91:0/4233732740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19c8065910 0x7f19c80c5e30 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 >> 172.21.15.91:0/4233732740 conn(0x7f19c80cc7a0 msgr2=0x7f19c80cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:24.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.542+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 shutdown_connections 2024-02-02T13:35:24.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.543+0000 7f19cea00640 1 -- 172.21.15.91:0/4233732740 wait complete. 2024-02-02T13:35:24.562 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-02-02T13:35:24.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.764+0000 7efd79400640 1 Processor -- start 2024-02-02T13:35:24.766 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.765+0000 7efd79400640 1 -- start start 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd79400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd740b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd79400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740b7d40 con 0x7efd7415cee0 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd79400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740b81d0 con 0x7efd740b6990 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd79400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b7130 con 0x7efd740ce1d0 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd73e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7efd7415cee0 0x7efd740cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46044/0 (socket says 172.21.15.91:46044) 2024-02-02T13:35:24.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.766+0000 7efd73e00640 1 -- 172.21.15.91:0/3145522488 learned_addr learned my addr 172.21.15.91:0/3145522488 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:24.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd6b200640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd740b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303573291 0 0) 0x7efd740b7d40 con 0x7efd7415cee0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd54003520 con 0x7efd7415cee0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189499647 0 0) 0x7efd54003520 con 0x7efd7415cee0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 msgr2=0x7efd740b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd740b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 >> v1:172.21.15.113:6789/0 conn(0x7efd740b6990 legacy=0x7efd740d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740b7410 con 0x7efd7415cee0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.767+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7efd64001670 con 0x7efd7415cee0 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 >> v1:172.21.15.91:6789/0 conn(0x7efd7415cee0 legacy=0x7efd740cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd54003f20 0x7efd540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54004d00 0x7efd54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b7130 con 0x7efd54003f20 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5400ba60 con 0x7efd54004d00 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5400bc20 con 0x7efd740ce1d0 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd54003f20 0x7efd540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd73e00640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 -- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 msgr2=0x7efd540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740ce1d0 0x7efd540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 -- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54004d00 msgr2=0x7efd54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd78a00640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54004d00 0x7efd54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54004d00 0x7efd54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd6b200640 1 -- 172.21.15.91:0/3145522488 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54003520 con 0x7efd54003f20 2024-02-02T13:35:24.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.768+0000 7efd78a00640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54004d00 0x7efd54005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:24.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.769+0000 7efd6b200640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd54003f20 0x7efd540047c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009ac0 tx=0x7efd540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1bab481ea60a54 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c02c6b0 con 0x7efd54003f20 2024-02-02T13:35:24.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c02c810 con 0x7efd54003f20 2024-02-02T13:35:24.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd73400640 1 -- 172.21.15.91:0/3145522488 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c02cba0 con 0x7efd54003f20 2024-02-02T13:35:24.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd79400640 1 -- 172.21.15.91:0/3145522488 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd54003f20 msgr2=0x7efd540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd79400640 1 --2- 172.21.15.91:0/3145522488 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd54003f20 0x7efd540047c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009ac0 tx=0x7efd540057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.770+0000 7efd78a00640 1 -- 172.21.15.91:0/3145522488 reap_dead start 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.771+0000 7efd79400640 1 -- 172.21.15.91:0/3145522488 shutdown_connections 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.771+0000 7efd79400640 1 -- 172.21.15.91:0/3145522488 >> 172.21.15.91:0/3145522488 conn(0x7efd74070690 msgr2=0x7efd74070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.771+0000 7efd79400640 1 -- 172.21.15.91:0/3145522488 shutdown_connections 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.771+0000 7efd79400640 1 -- 172.21.15.91:0/3145522488 wait complete. 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.772+0000 7efd79400640 1 Processor -- start 2024-02-02T13:35:24.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.773+0000 7efd79400640 1 -- start start 2024-02-02T13:35:24.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.773+0000 7efd79400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54003f20 0x7efd740ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.773+0000 7efd79400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.773+0000 7efd79400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740b37e0 0x7efd740b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.773+0000 7efd79400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b41a0 con 0x7efd741594f0 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd79400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740cab30 con 0x7efd54003f20 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd79400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c9f60 con 0x7efd740b37e0 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47760/0 (socket says 172.21.15.91:47760) 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 -- 172.21.15.91:0/3710261795 learned_addr learned my addr 172.21.15.91:0/3710261795 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd73e00640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54003f20 0x7efd740ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd78a00640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740b37e0 0x7efd740b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 -- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740b37e0 msgr2=0x7efd740b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740b37e0 0x7efd740b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 -- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54003f20 msgr2=0x7efd740ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54003f20 0x7efd740ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.774+0000 7efd6b200640 1 -- 172.21.15.91:0/3710261795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740cafc0 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd6b200640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efd5c037040 tx=0x7efd5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c003810 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c003970 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd740ca200 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c003d00 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.775+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd740ca7c0 con 0x7efd741594f0 2024-02-02T13:35:24.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.776+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7404dff0 con 0x7efd741594f0 2024-02-02T13:35:24.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.777+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd5c040450 con 0x7efd741594f0 2024-02-02T13:35:24.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.778+0000 7efd70c00640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 0x7efd4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:24.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.778+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efd5c0aa5e0 con 0x7efd741594f0 2024-02-02T13:35:24.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.778+0000 7efd73e00640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 0x7efd4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:24.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.779+0000 7efd73e00640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 0x7efd4c06c8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd640059d0 tx=0x7efd64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:24.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:24.782+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd5c075e30 con 0x7efd741594f0 2024-02-02T13:35:25.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.039+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7efd740d0830 con 0x7efd741594f0 2024-02-02T13:35:25.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.084+0000 7efd70c00640 1 -- 172.21.15.91:0/3710261795 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v520) v1 ==== 132+0+0 (secure 0 0 0) 0x7efd5c003e60 con 0x7efd741594f0 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 msgr2=0x7efd4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 0x7efd4c06c8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd640059d0 tx=0x7efd64009040 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 msgr2=0x7efd740cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efd5c037040 tx=0x7efd5c004730 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 shutdown_connections 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efd4c06a430 0x7efd4c06c8f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd740b37e0 0x7efd740b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd741594f0 0x7efd740cc670 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 --2- 172.21.15.91:0/3710261795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd54003f20 0x7efd740ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 >> 172.21.15.91:0/3710261795 conn(0x7efd74070690 msgr2=0x7efd740d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 shutdown_connections 2024-02-02T13:35:25.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.087+0000 7efd79400640 1 -- 172.21.15.91:0/3710261795 wait complete. 2024-02-02T13:35:25.108 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-02-02T13:35:25.108 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-02-02T13:35:25.109 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-02-02T13:35:25.109 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-02-02T13:35:25.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.319+0000 7f79bd600640 1 Processor -- start 2024-02-02T13:35:25.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.319+0000 7f79bd600640 1 -- start start 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79bd600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79b815a010 0x7f79b80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79bd600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80b0700 0x7f79b80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79bd600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b80cc7c0 con 0x7f79b80ce130 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79bd600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b80b10d0 con 0x7f79b815a010 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79bd600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80ce930 con 0x7f79b80b0700 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79b6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79b815a010 0x7f79b80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79b6a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79b815a010 0x7f79b80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47776/0 (socket says 172.21.15.91:47776) 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.321+0000 7f79b6a00640 1 -- 172.21.15.91:0/3591519720 learned_addr learned my addr 172.21.15.91:0/3591519720 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:25.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.322+0000 7f79b7400640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80b0700 0x7f79b80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.322+0000 7f79b6a00640 1 -- 172.21.15.91:0/3591519720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80b0700 msgr2=0x7f79b80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:25.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.322+0000 7f79b6a00640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80b0700 0x7f79b80b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.322+0000 7f79b6a00640 1 -- 172.21.15.91:0/3591519720 >> v1:172.21.15.113:6789/0 conn(0x7f79b80ce130 legacy=0x7f79b80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.322+0000 7f79b6a00640 1 -- 172.21.15.91:0/3591519720 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80cd640 con 0x7f79b815a010 2024-02-02T13:35:25.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.323+0000 7f79b6a00640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b815a010 0x7f79b80c9670 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009870 tx=0x7f79a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befb7e55e65b57f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:25.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.323+0000 7f79b6000640 1 -- 172.21.15.91:0/3591519720 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8030840 con 0x7f79b815a010 2024-02-02T13:35:25.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.323+0000 7f79b6000640 1 -- 172.21.15.91:0/3591519720 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a80309a0 con 0x7f79b815a010 2024-02-02T13:35:25.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.323+0000 7f79b6000640 1 -- 172.21.15.91:0/3591519720 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8030d30 con 0x7f79b815a010 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 -- 172.21.15.91:0/3591519720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b815a010 msgr2=0x7f79b80c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b815a010 0x7f79b80c9670 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009870 tx=0x7f79a802b980 comp rx=0 tx=0).stop 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 -- 172.21.15.91:0/3591519720 shutdown_connections 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80b0700 0x7f79b80b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 --2- 172.21.15.91:0/3591519720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b815a010 0x7f79b80c9670 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 -- 172.21.15.91:0/3591519720 >> 172.21.15.91:0/3591519720 conn(0x7f79b80cec80 msgr2=0x7f79b80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 -- 172.21.15.91:0/3591519720 shutdown_connections 2024-02-02T13:35:25.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.324+0000 7f79bd600640 1 -- 172.21.15.91:0/3591519720 wait complete. 2024-02-02T13:35:25.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.326+0000 7f79bd600640 1 Processor -- start 2024-02-02T13:35:25.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.326+0000 7f79bd600640 1 -- start start 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80ce130 0x7f79b8147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79b815a010 0x7f79b815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b81482a0 con 0x7f79b80b0700 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80c5ed0 con 0x7f79b815a010 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79bd600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b815b2f0 con 0x7f79b80ce130 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79b6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79b6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47784/0 (socket says 172.21.15.91:47784) 2024-02-02T13:35:25.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.327+0000 7f79b6a00640 1 -- 172.21.15.91:0/1335608156 learned_addr learned my addr 172.21.15.91:0/1335608156 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:25.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79afe00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80ce130 0x7f79b8147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 -- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80ce130 msgr2=0x7f79b8147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80ce130 0x7f79b8147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 -- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79b815a010 msgr2=0x7f79b815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b7400640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79b815a010 0x7f79b815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79b815a010 0x7f79b815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 -- 172.21.15.91:0/1335608156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80094d0 con 0x7f79b80b0700 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.328+0000 7f79b6a00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000c00 tx=0x7f79a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.329+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8039a90 con 0x7f79b80b0700 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.329+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a8039bf0 con 0x7f79b80b0700 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.329+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8043410 con 0x7f79b80b0700 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.329+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b815b570 con 0x7f79b80b0700 2024-02-02T13:35:25.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.329+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b815bb30 con 0x7f79b80b0700 2024-02-02T13:35:25.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.330+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b80c5cc0 con 0x7f79b80b0700 2024-02-02T13:35:25.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.331+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f79a802f390 con 0x7f79b80b0700 2024-02-02T13:35:25.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.332+0000 7f79aea00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 0x7f799006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:25.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.332+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79a80aa800 con 0x7f79b80b0700 2024-02-02T13:35:25.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.332+0000 7f79afe00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 0x7f799006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:25.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.333+0000 7f79afe00640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 0x7f799006c8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f79b80af440 tx=0x7f79a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:25.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.337+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f79a80760f0 con 0x7f79b80b0700 2024-02-02T13:35:25.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:25.597+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f79b8066cf0 con 0x7f79b80b0700 2024-02-02T13:35:26.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.091+0000 7f79aea00640 1 -- 172.21.15.91:0/1335608156 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v521) v1 ==== 134+0+0 (secure 0 0 0) 0x7f79a80340c0 con 0x7f79b80b0700 2024-02-02T13:35:26.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 msgr2=0x7f799006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 0x7f799006c8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f79b80af440 tx=0x7f79a0009040 comp rx=0 tx=0).stop 2024-02-02T13:35:26.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 msgr2=0x7f79b815d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000c00 tx=0x7f79a8004730 comp rx=0 tx=0).stop 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 shutdown_connections 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f799006a430 0x7f799006c8f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79b80ce130 0x7f79b8147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79b815a010 0x7f79b815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.094+0000 7f79bd600640 1 --2- 172.21.15.91:0/1335608156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79b80b0700 0x7f79b815d0e0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.095+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 >> 172.21.15.91:0/1335608156 conn(0x7f79b80cec80 msgr2=0x7f79b80c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.095+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 shutdown_connections 2024-02-02T13:35:26.097 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.095+0000 7f79bd600640 1 -- 172.21.15.91:0/1335608156 wait complete. 2024-02-02T13:35:26.116 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:35:26.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.326+0000 7f1781400640 1 Processor -- start 2024-02-02T13:35:26.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.326+0000 7f1781400640 1 -- start start 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.327+0000 7f1781400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f177c0cad30 0x7f177c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.327+0000 7f1781400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177c06c1c0 con 0x7f177c15b2a0 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.327+0000 7f1781400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177c06b5f0 con 0x7f177c15aec0 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.327+0000 7f1781400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177c06cd60 con 0x7f177c0cad30 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f1780a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f177c0cad30 0x7f177c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f177b400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f177c15b2a0 0x7f177c0ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46048/0 (socket says 172.21.15.91:46048) 2024-02-02T13:35:26.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f177b400640 1 -- 172.21.15.91:0/3586407506 learned_addr learned my addr 172.21.15.91:0/3586407506 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:26.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f1780a00640 1 -- 172.21.15.91:0/3586407506 >> v1:172.21.15.113:6789/0 conn(0x7f177c15aec0 legacy=0x7f177c06c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f1780a00640 1 -- 172.21.15.91:0/3586407506 >> v1:172.21.15.91:6789/0 conn(0x7f177c15b2a0 legacy=0x7f177c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.328+0000 7f1780a00640 1 -- 172.21.15.91:0/3586407506 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c06b890 con 0x7f177c0cad30 2024-02-02T13:35:26.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.329+0000 7f1780a00640 1 --2- 172.21.15.91:0/3586407506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c0cad30 0x7f177c0cf5b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1774009d90 tx=0x7f177402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2fc0112b178ba2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:26.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f177aa00640 1 -- 172.21.15.91:0/3586407506 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177402cd80 con 0x7f177c0cad30 2024-02-02T13:35:26.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f177aa00640 1 -- 172.21.15.91:0/3586407506 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1774034440 con 0x7f177c0cad30 2024-02-02T13:35:26.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f177aa00640 1 -- 172.21.15.91:0/3586407506 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740347d0 con 0x7f177c0cad30 2024-02-02T13:35:26.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f1781400640 1 -- 172.21.15.91:0/3586407506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c0cad30 msgr2=0x7f177c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f1781400640 1 --2- 172.21.15.91:0/3586407506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c0cad30 0x7f177c0cf5b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1774009d90 tx=0x7f177402bfd0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f1781400640 1 -- 172.21.15.91:0/3586407506 shutdown_connections 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f1781400640 1 --2- 172.21.15.91:0/3586407506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c0cad30 0x7f177c0cf5b0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.330+0000 7f1781400640 1 -- 172.21.15.91:0/3586407506 >> 172.21.15.91:0/3586407506 conn(0x7f177c0cea90 msgr2=0x7f177c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.331+0000 7f1781400640 1 -- 172.21.15.91:0/3586407506 shutdown_connections 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.331+0000 7f1781400640 1 -- 172.21.15.91:0/3586407506 wait complete. 2024-02-02T13:35:26.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.332+0000 7f1781400640 1 Processor -- start 2024-02-02T13:35:26.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.332+0000 7f1781400640 1 -- start start 2024-02-02T13:35:26.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 0x7f177c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:26.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:26.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f177c15b2a0 0x7f177c14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:26.336 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c14c930 con 0x7f177c15aec0 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c14b350 con 0x7f177c0cad30 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f1781400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177c14b630 con 0x7f177c15b2a0 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47804/0 (socket says 172.21.15.91:47804) 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 -- 172.21.15.91:0/2607859364 learned_addr learned my addr 172.21.15.91:0/2607859364 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177be00640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 0x7f177c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 -- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f177c15b2a0 msgr2=0x7f177c14ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f177c15b2a0 0x7f177c14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 -- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 msgr2=0x7f177c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 0x7f177c15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177b400640 1 -- 172.21.15.91:0/2607859364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774009a70 con 0x7f177c15aec0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.334+0000 7f177be00640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 0x7f177c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f177b400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f176400c480 tx=0x7f176400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764005ce0 con 0x7f177c15aec0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764005e40 con 0x7f177c15aec0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c14b910 con 0x7f177c15aec0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640072f0 con 0x7f177c15aec0 2024-02-02T13:35:26.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.335+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177c14bed0 con 0x7f177c15aec0 2024-02-02T13:35:26.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.336+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177c14c0c0 con 0x7f177c15aec0 2024-02-02T13:35:26.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.337+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1764007510 con 0x7f177c15aec0 2024-02-02T13:35:26.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.338+0000 7f1773e00640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 0x7f175406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:26.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.338+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1764087b30 con 0x7f177c15aec0 2024-02-02T13:35:26.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.338+0000 7f177be00640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 0x7f175406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:26.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.339+0000 7f177be00640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 0x7f175406c8f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f176c000f60 tx=0x7f176c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:26.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.342+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f17640534c0 con 0x7f177c15aec0 2024-02-02T13:35:26.533 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.531+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f177c04dff0 con 0x7f177c15aec0 2024-02-02T13:35:26.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.537+0000 7f1773e00640 1 -- 172.21.15.91:0/2607859364 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17640170c0 con 0x7f177c15aec0 2024-02-02T13:35:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.539+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 msgr2=0x7f175406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.539+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 0x7f175406c8f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f176c000f60 tx=0x7f176c009040 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.539+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 msgr2=0x7f177c14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.539+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f176400c480 tx=0x7f176400c940 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 shutdown_connections 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f175406a430 0x7f175406c8f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f177c15b2a0 0x7f177c14ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f177c15aec0 0x7f177c14c3f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 --2- 172.21.15.91:0/2607859364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f177c0cad30 0x7f177c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 >> 172.21.15.91:0/2607859364 conn(0x7f177c0cea90 msgr2=0x7f177c0c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 shutdown_connections 2024-02-02T13:35:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:26.540+0000 7f1781400640 1 -- 172.21.15.91:0/2607859364 wait complete. 2024-02-02T13:35:26.559 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:35:26.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:35:26.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:35:26.789 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:35:26.789 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:26.789 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:26.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:35:26.845 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:26.846 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:26.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:35:26.903 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:26.903 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:26.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:35:26.960 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:26.960 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:27.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:35:27.017 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:35:27.018 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:27.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.230+0000 7fd1c2c00640 1 Processor -- start 2024-02-02T13:35:27.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.230+0000 7fd1c2c00640 1 -- start start 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.231+0000 7fd1c2c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 0x7fd1bc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.231+0000 7fd1c2c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.231+0000 7fd1c2c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc0cc7c0 con 0x7fd1bc0cd470 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.231+0000 7fd1c2c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc0ce930 con 0x7fd1bc0ce130 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.231+0000 7fd1c2c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc06c890 con 0x7fd1bc15a010 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47818/0 (socket says 172.21.15.91:47818) 2024-02-02T13:35:27.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 -- 172.21.15.91:0/1018429296 learned_addr learned my addr 172.21.15.91:0/1018429296 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bbe00640 1 --2- 172.21.15.91:0/1018429296 >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 0x7fd1bc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 -- 172.21.15.91:0/1018429296 >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 msgr2=0x7fd1bc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 --2- 172.21.15.91:0/1018429296 >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 0x7fd1bc0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 -- 172.21.15.91:0/1018429296 >> v1:172.21.15.91:6789/0 conn(0x7fd1bc0cd470 legacy=0x7fd1bc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bb400640 1 -- 172.21.15.91:0/1018429296 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc06cb70 con 0x7fd1bc0ce130 2024-02-02T13:35:27.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1baa00640 1 -- 172.21.15.91:0/1018429296 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807907498 0 0) 0x7fd1bc0cc7c0 con 0x7fd1bc0cd470 2024-02-02T13:35:27.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.232+0000 7fd1bbe00640 1 --2- 172.21.15.91:0/1018429296 >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 0x7fd1bc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:27.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.233+0000 7fd1bb400640 1 --2- 172.21.15.91:0/1018429296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0cfaa0 tx=0x7fd1a402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc2bfc7d671373b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1baa00640 1 -- 172.21.15.91:0/1018429296 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4031890 con 0x7fd1bc0ce130 2024-02-02T13:35:27.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1baa00640 1 -- 172.21.15.91:0/1018429296 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a40319f0 con 0x7fd1bc0ce130 2024-02-02T13:35:27.235 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1baa00640 1 -- 172.21.15.91:0/1018429296 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4031d80 con 0x7fd1bc0ce130 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 -- 172.21.15.91:0/1018429296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 msgr2=0x7fd1bc0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/1018429296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0cfaa0 tx=0x7fd1a402b930 comp rx=0 tx=0).stop 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 -- 172.21.15.91:0/1018429296 shutdown_connections 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/1018429296 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc0ccf30 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/1018429296 >> v2:172.21.15.113:3300/0 conn(0x7fd1bc15a010 0x7fd1bc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.234+0000 7fd1c2c00640 1 -- 172.21.15.91:0/1018429296 >> 172.21.15.91:0/1018429296 conn(0x7fd1bc0cec80 msgr2=0x7fd1bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:27.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.235+0000 7fd1c2c00640 1 -- 172.21.15.91:0/1018429296 shutdown_connections 2024-02-02T13:35:27.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.235+0000 7fd1c2c00640 1 -- 172.21.15.91:0/1018429296 wait complete. 2024-02-02T13:35:27.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.236+0000 7fd1c2c00640 1 Processor -- start 2024-02-02T13:35:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.236+0000 7fd1c2c00640 1 -- start start 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1bc15a010 0x7fd1bc0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3470 0x7fd1bc0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15d060 con 0x7fd1bc0ce130 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc157ae0 con 0x7fd1bc0b3470 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.237+0000 7fd1c2c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c6110 con 0x7fd1bc15a010 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47830/0 (socket says 172.21.15.91:47830) 2024-02-02T13:35:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 -- 172.21.15.91:0/6144028 learned_addr learned my addr 172.21.15.91:0/6144028 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:27.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 -- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1bc15a010 msgr2=0x7fd1bc0b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1bc15a010 0x7fd1bc0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 -- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3470 msgr2=0x7fd1bc0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3470 0x7fd1bc0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 -- 172.21.15.91:0/6144028 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40094d0 con 0x7fd1bc0ce130 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.238+0000 7fd1bbe00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c7f0 tx=0x7fd1ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.239+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac007820 con 0x7fd1bc0ce130 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.239+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac007980 con 0x7fd1bc0ce130 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.239+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac007cf0 con 0x7fd1bc0ce130 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.239+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc15cd80 con 0x7fd1bc0ce130 2024-02-02T13:35:27.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.239+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc0c6690 con 0x7fd1bc0ce130 2024-02-02T13:35:27.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.240+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1bc066cf0 con 0x7fd1bc0ce130 2024-02-02T13:35:27.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.241+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd1ac0113f0 con 0x7fd1bc0ce130 2024-02-02T13:35:27.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.242+0000 7fd1b3e00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 0x7fd19406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.242+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd1ac088950 con 0x7fd1bc0ce130 2024-02-02T13:35:27.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.242+0000 7fd1bb400640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 0x7fd19406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.243+0000 7fd1bb400640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 0x7fd19406c920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036000 tx=0x7fd1a4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.247+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd1ac054230 con 0x7fd1bc0ce130 2024-02-02T13:35:27.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.473+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1bc0b4c00 con 0x7fd1bc0ce130 2024-02-02T13:35:27.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.474+0000 7fd1b3e00640 1 -- 172.21.15.91:0/6144028 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fd1ac01f090 con 0x7fd1bc0ce130 2024-02-02T13:35:27.475 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:27.476 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1591102937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1591102937},{"type":"v1","addr":"172.21.15.113:6833","nonce":1591102937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4035242176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4035242176},{"type":"v1","addr":"172.21.15.113:6835","nonce":4035242176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3332456374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3332456374},{"type":"v1","addr":"172.21.15.91:6836","nonce":3332456374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8373,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/224715174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":224715174},{"type":"v1","addr":"172.21.15.91:6837","nonce":224715174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2024-02-02T13:35:27.476 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 74 2024-02-02T13:35:27.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 msgr2=0x7fd19406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 0x7fd19406c920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036000 tx=0x7fd1a4030000 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 msgr2=0x7fd1bc1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c7f0 tx=0x7fd1ac00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 shutdown_connections 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd19406a460 0x7fd19406c920 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1bc15a010 0x7fd1bc0b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3470 0x7fd1bc0b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 --2- 172.21.15.91:0/6144028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1bc0ce130 0x7fd1bc1575a0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 >> 172.21.15.91:0/6144028 conn(0x7fd1bc0cec80 msgr2=0x7fd1bc0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 shutdown_connections 2024-02-02T13:35:27.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.476+0000 7fd1c2c00640 1 -- 172.21.15.91:0/6144028 wait complete. 2024-02-02T13:35:27.496 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:35:27.496 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:35:27.496 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:35:27.496 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:35:27.497 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:35:27.497 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:35:27.497 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:35:27.497 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:35:27.497 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:35:27.497 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:35:27.497 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:35:27.498 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:35:27.498 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:35:27.498 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:35:27.498 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:35:27.498 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:35:27.515 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:35:27.516 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:35:27.520 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:35:27.520 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:35:27.528 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:35:27.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:35:27.538 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:35:27.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:35:27.567 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:35:27.568 DEBUG:teuthology.parallel:result is None 2024-02-02T13:35:27.574 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:35:27.574 DEBUG:teuthology.parallel:result is None 2024-02-02T13:35:27.584 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:35:27.584 DEBUG:teuthology.parallel:result is None 2024-02-02T13:35:27.589 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:35:27.589 DEBUG:teuthology.parallel:result is None 2024-02-02T13:35:27.589 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:35:27.724 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:35:27.725 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:35:27.750 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:35:27.750 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:35:27.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.774+0000 7fa29bc00640 1 Processor -- start 2024-02-02T13:35:27.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.775+0000 7fa29bc00640 1 -- start start 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa29bc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa29415d3e0 0x7fa2940b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa29bc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2940d1a60 0x7fa2940cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa29bc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940704a0 con 0x7fa2940ce2e0 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa29bc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940d2800 con 0x7fa29415d3e0 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa29bc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940ce770 con 0x7fa2940d1a60 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa29415d3e0 0x7fa2940b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa2940ce2e0 0x7fa294160db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46056/0 (socket says 172.21.15.91:46056) 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa29415d3e0 0x7fa2940b88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47866/0 (socket says 172.21.15.91:47866) 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299c00640 1 -- 172.21.15.91:0/3645397 learned_addr learned my addr 172.21.15.91:0/3645397 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa293e00640 1 --2- 172.21.15.91:0/3645397 >> v2:172.21.15.113:3300/0 conn(0x7fa2940d1a60 0x7fa2940cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 -- 172.21.15.91:0/3645397 >> v2:172.21.15.113:3300/0 conn(0x7fa2940d1a60 msgr2=0x7fa2940cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 --2- 172.21.15.91:0/3645397 >> v2:172.21.15.113:3300/0 conn(0x7fa2940d1a60 0x7fa2940cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 -- 172.21.15.91:0/3645397 >> v1:172.21.15.91:6789/0 conn(0x7fa2940ce2e0 legacy=0x7fa294160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa299200640 1 -- 172.21.15.91:0/3645397 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2940d1f60 con 0x7fa29415d3e0 2024-02-02T13:35:27.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.776+0000 7fa293400640 1 -- 172.21.15.91:0/3645397 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549055152 0 0) 0x7fa2940704a0 con 0x7fa2940ce2e0 2024-02-02T13:35:27.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa299200640 1 --2- 172.21.15.91:0/3645397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa29415d3e0 0x7fa2940b88f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa2940b5b90 tx=0x7fa28402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54a7b06e47fa0b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa293400640 1 -- 172.21.15.91:0/3645397 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284031830 con 0x7fa29415d3e0 2024-02-02T13:35:27.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa293400640 1 -- 172.21.15.91:0/3645397 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284031990 con 0x7fa29415d3e0 2024-02-02T13:35:27.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa293400640 1 -- 172.21.15.91:0/3645397 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284031d20 con 0x7fa29415d3e0 2024-02-02T13:35:27.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa29bc00640 1 -- 172.21.15.91:0/3645397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa29415d3e0 msgr2=0x7fa2940b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.778+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3645397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa29415d3e0 0x7fa2940b88f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa2940b5b90 tx=0x7fa28402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 -- 172.21.15.91:0/3645397 shutdown_connections 2024-02-02T13:35:27.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3645397 >> v2:172.21.15.113:3300/0 conn(0x7fa2940d1a60 0x7fa2940cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3645397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa29415d3e0 0x7fa2940b88f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 -- 172.21.15.91:0/3645397 >> 172.21.15.91:0/3645397 conn(0x7fa2940cfa50 msgr2=0x7fa2940cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:27.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 -- 172.21.15.91:0/3645397 shutdown_connections 2024-02-02T13:35:27.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.779+0000 7fa29bc00640 1 -- 172.21.15.91:0/3645397 wait complete. 2024-02-02T13:35:27.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.780+0000 7fa29bc00640 1 Processor -- start 2024-02-02T13:35:27.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.781+0000 7fa29bc00640 1 -- start start 2024-02-02T13:35:27.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.781+0000 7fa29bc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940ce2e0 0x7fa2940cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa29bc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa29bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29415d3e0 0x7fa2940b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa29bc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940cb790 con 0x7fa2940d1a60 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa29bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940cba70 con 0x7fa29415d3e0 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa29bc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29415e8b0 con 0x7fa2940ce2e0 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47868/0 (socket says 172.21.15.91:47868) 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 -- 172.21.15.91:0/3857394520 learned_addr learned my addr 172.21.15.91:0/3857394520 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa299200640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940ce2e0 0x7fa2940cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa299c00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29415d3e0 0x7fa2940b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 -- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940ce2e0 msgr2=0x7fa2940cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940ce2e0 0x7fa2940cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 -- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29415d3e0 msgr2=0x7fa2940b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29415d3e0 0x7fa2940b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.782+0000 7fa293e00640 1 -- 172.21.15.91:0/3857394520 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840093a0 con 0x7fa2940d1a60 2024-02-02T13:35:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa293e00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e840 tx=0x7fa27c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0094f0 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00c940 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29415eb90 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00ccd0 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2940cce80 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.784+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa294066cf0 con 0x7fa2940d1a60 2024-02-02T13:35:27.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.785+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa27c013470 con 0x7fa2940d1a60 2024-02-02T13:35:27.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.786+0000 7fa290c00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 0x7fa26c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:27.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.786+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa27c088630 con 0x7fa2940d1a60 2024-02-02T13:35:27.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.786+0000 7fa299200640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 0x7fa26c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:27.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.787+0000 7fa299200640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 0x7fa26c06c8f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa28403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:27.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.791+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa27c053fd0 con 0x7fa2940d1a60 2024-02-02T13:35:27.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:27.997+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa2940ce9b0 con 0x7fa2940d1a60 2024-02-02T13:35:28.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.779+0000 7fa290c00640 1 -- 172.21.15.91:0/3857394520 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v174) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa27c0575f0 con 0x7fa2940d1a60 2024-02-02T13:35:28.781 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.782+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 msgr2=0x7fa26c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.782+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 0x7fa26c06c8f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa28403c040 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.782+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 msgr2=0x7fa2940cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.782+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e840 tx=0x7fa27c00ed00 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 shutdown_connections 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa26c06a430 0x7fa26c06c8f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29415d3e0 0x7fa2940b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2940d1a60 0x7fa2940cace0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 --2- 172.21.15.91:0/3857394520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940ce2e0 0x7fa2940cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 >> 172.21.15.91:0/3857394520 conn(0x7fa2940cfa50 msgr2=0x7fa2940b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 shutdown_connections 2024-02-02T13:35:28.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:28.783+0000 7fa29bc00640 1 -- 172.21.15.91:0/3857394520 wait complete. 2024-02-02T13:35:28.804 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:35:29.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.011+0000 7f08c8200640 1 Processor -- start 2024-02-02T13:35:29.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.011+0000 7f08c8200640 1 -- start start 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.012+0000 7f08c8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 0x7f08c006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.012+0000 7f08c8200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c006d910 con 0x7f08c015b0f0 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.012+0000 7f08c8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00cc850 con 0x7f08c015ad10 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.012+0000 7f08c8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c006d730 con 0x7f08c00cbc90 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.013+0000 7f08c4e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f08c015b0f0 0x7f08c00cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46072/0 (socket says 172.21.15.91:46072) 2024-02-02T13:35:29.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.013+0000 7f08c4e00640 1 -- 172.21.15.91:0/1346347184 learned_addr learned my addr 172.21.15.91:0/1346347184 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:29.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.013+0000 7f08c6200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 0x7f08c006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.013+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533723570 0 0) 0x7f08c006d910 con 0x7f08c015b0f0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.013+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a0003520 con 0x7f08c015b0f0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024611843 0 0) 0x7f08a0003520 con 0x7f08c015b0f0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 msgr2=0x7f08c006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 0x7f08c006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 >> v1:172.21.15.113:6789/0 conn(0x7f08c015ad10 legacy=0x7f08c006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00ccb30 con 0x7f08c015b0f0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f08a8001670 con 0x7f08c015b0f0 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 >> v1:172.21.15.91:6789/0 conn(0x7f08c015b0f0 legacy=0x7f08c00cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08a00039a0 0x7f08a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 0x7f08a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08a0004d00 0x7f08a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c006d730 con 0x7f08a0004d00 2024-02-02T13:35:29.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a000bae0 con 0x7f08a00039a0 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a000bca0 con 0x7f08c00cbc90 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08c6200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08a0004d00 0x7f08a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08c6200640 1 -- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 msgr2=0x7f08a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08c6200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08c00cbc90 0x7f08a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08c6200640 1 -- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08a00039a0 msgr2=0x7f08a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.014+0000 7f08c5800640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08a00039a0 0x7f08a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.015+0000 7f08c6200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08a00039a0 0x7f08a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.015+0000 7f08c6200640 1 -- 172.21.15.91:0/1346347184 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a0003520 con 0x7f08a0004d00 2024-02-02T13:35:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.015+0000 7f08c5800640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08a00039a0 0x7f08a0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08c6200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08a0004d00 0x7f08a0005180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009fc0 tx=0x7f08a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ae2d7a25a26168 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802cc60 con 0x7f08a0004d00 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b800f040 con 0x7f08a0004d00 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08bfe00640 1 -- 172.21.15.91:0/1346347184 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8034440 con 0x7f08a0004d00 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08c8200640 1 -- 172.21.15.91:0/1346347184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08a0004d00 msgr2=0x7f08a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.016+0000 7f08c8200640 1 --2- 172.21.15.91:0/1346347184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08a0004d00 0x7f08a0005180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009fc0 tx=0x7f08a00057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.017+0000 7f08c6200640 1 -- 172.21.15.91:0/1346347184 reap_dead start 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.017+0000 7f08c8200640 1 -- 172.21.15.91:0/1346347184 shutdown_connections 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.017+0000 7f08c8200640 1 -- 172.21.15.91:0/1346347184 >> 172.21.15.91:0/1346347184 conn(0x7f08c00b0700 msgr2=0x7f08c00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.017+0000 7f08c8200640 1 -- 172.21.15.91:0/1346347184 shutdown_connections 2024-02-02T13:35:29.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.017+0000 7f08c8200640 1 -- 172.21.15.91:0/1346347184 wait complete. 2024-02-02T13:35:29.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.018+0000 7f08c8200640 1 Processor -- start 2024-02-02T13:35:29.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.019+0000 7f08c8200640 1 -- start start 2024-02-02T13:35:29.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.019+0000 7f08c8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08a0004d00 0x7f08c01ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.019+0000 7f08c8200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.019+0000 7f08c8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 0x7f08c01ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.019+0000 7f08c8200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f2850 con 0x7f08c00ca4f0 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00ca970 con 0x7f08c01eda30 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f1c40 con 0x7f08a0004d00 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47876/0 (socket says 172.21.15.91:47876) 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 -- 172.21.15.91:0/947120048 learned_addr learned my addr 172.21.15.91:0/947120048 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c6200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 0x7f08c01ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 -- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08a0004d00 msgr2=0x7f08c01ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08a0004d00 0x7f08c01ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 -- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 msgr2=0x7f08c01ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 0x7f08c01ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c4e00640 1 -- 172.21.15.91:0/947120048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c01f1f20 con 0x7f08c00ca4f0 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.020+0000 7f08c6200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 0x7f08c01ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08c4e00640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c730 tx=0x7f08a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a80074e0 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8007640 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8004a00 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c01f2200 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.021+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c01ee3d0 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.022+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c01f29c0 con 0x7f08c00ca4f0 2024-02-02T13:35:29.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.023+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f08a8004b60 con 0x7f08c00ca4f0 2024-02-02T13:35:29.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.024+0000 7f08bd600640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 0x7f089806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.024+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f08a80879f0 con 0x7f08c00ca4f0 2024-02-02T13:35:29.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.024+0000 7f08c5800640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 0x7f089806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.025+0000 7f08c5800640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 0x7f089806c8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f08b00016d0 tx=0x7f08b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.028+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f08a8053290 con 0x7f08c00ca4f0 2024-02-02T13:35:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.291+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08c0066cf0 con 0x7f08c00ca4f0 2024-02-02T13:35:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.292+0000 7f08bd600640 1 -- 172.21.15.91:0/947120048 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08a8018020 con 0x7f08c00ca4f0 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:35:29.294 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:35:29.295 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:35:29.296 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:35:29.297 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.298 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.299 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.300 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.301 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.302 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:35:29.303 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.304 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:35:29.305 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:35:29.306 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:35:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 msgr2=0x7f089806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 0x7f089806c8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f08b00016d0 tx=0x7f08b0009040 comp rx=0 tx=0).stop 2024-02-02T13:35:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 msgr2=0x7f08c015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c730 tx=0x7f08a800cbf0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 shutdown_connections 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f089806a430 0x7f089806c8f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08c01eda30 0x7f08c01ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f08c00ca4f0 0x7f08c015b860 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 --2- 172.21.15.91:0/947120048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08a0004d00 0x7f08c01ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 >> 172.21.15.91:0/947120048 conn(0x7f08c00b0700 msgr2=0x7f08c00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.295+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 shutdown_connections 2024-02-02T13:35:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.296+0000 7f08c8200640 1 -- 172.21.15.91:0/947120048 wait complete. 2024-02-02T13:35:29.315 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:29.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.518+0000 7fdbca000640 1 Processor -- start 2024-02-02T13:35:29.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.518+0000 7fdbca000640 1 -- start start 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbca000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdbc4151f90 0x7fdbc415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbca000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 0x7fdbc40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbca000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc4070f80 con 0x7fdbc406f4f0 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbca000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc4152370 con 0x7fdbc4151f90 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbca000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40cb010 con 0x7fdbc40b0e10 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.519+0000 7fdbc3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdbc4151f90 0x7fdbc415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc2a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdbc406f4f0 0x7fdbc40b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46074/0 (socket says 172.21.15.91:46074) 2024-02-02T13:35:29.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc2a00640 1 -- 172.21.15.91:0/4098704945 learned_addr learned my addr 172.21.15.91:0/4098704945 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3e00640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 0x7fdbc40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3400640 1 -- 172.21.15.91:0/4098704945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 msgr2=0x7fdbc40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3400640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 0x7fdbc40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3400640 1 -- 172.21.15.91:0/4098704945 >> v1:172.21.15.91:6789/0 conn(0x7fdbc406f4f0 legacy=0x7fdbc40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc2000640 1 -- 172.21.15.91:0/4098704945 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158889352 0 0) 0x7fdbc4070f80 con 0x7fdbc406f4f0 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3400640 1 -- 172.21.15.91:0/4098704945 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc40cb2f0 con 0x7fdbc4151f90 2024-02-02T13:35:29.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.520+0000 7fdbc3e00640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 0x7fdbc40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.521+0000 7fdbc3400640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc4151f90 0x7fdbc415f5a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40097a0 tx=0x7fdbb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb1a30bb1c9ab26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.521+0000 7fdbc2000640 1 -- 172.21.15.91:0/4098704945 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4031830 con 0x7fdbc4151f90 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.521+0000 7fdbc2000640 1 -- 172.21.15.91:0/4098704945 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4031990 con 0x7fdbc4151f90 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.521+0000 7fdbc2000640 1 -- 172.21.15.91:0/4098704945 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4031d20 con 0x7fdbc4151f90 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 -- 172.21.15.91:0/4098704945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc4151f90 msgr2=0x7fdbc415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc4151f90 0x7fdbc415f5a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40097a0 tx=0x7fdbb402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 -- 172.21.15.91:0/4098704945 shutdown_connections 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc40b0e10 0x7fdbc40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 --2- 172.21.15.91:0/4098704945 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc4151f90 0x7fdbc415f5a0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 -- 172.21.15.91:0/4098704945 >> 172.21.15.91:0/4098704945 conn(0x7fdbc40cec80 msgr2=0x7fdbc40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 -- 172.21.15.91:0/4098704945 shutdown_connections 2024-02-02T13:35:29.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.522+0000 7fdbca000640 1 -- 172.21.15.91:0/4098704945 wait complete. 2024-02-02T13:35:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.524+0000 7fdbca000640 1 Processor -- start 2024-02-02T13:35:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.524+0000 7fdbca000640 1 -- start start 2024-02-02T13:35:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 0x7fdbc40c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbc4151f90 0x7fdbc40b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40b49a0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40c2450 con 0x7fdbc4151f90 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbca000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40c3910 con 0x7fdbc406f4f0 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbc2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbc2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:47894/0 (socket says 172.21.15.91:47894) 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.525+0000 7fdbc2a00640 1 -- 172.21.15.91:0/2268828646 learned_addr learned my addr 172.21.15.91:0/2268828646 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc3400640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 0x7fdbc40c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 -- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 msgr2=0x7fdbc40c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 0x7fdbc40c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 -- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbc4151f90 msgr2=0x7fdbc40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc3e00640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbc4151f90 0x7fdbc40b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbc4151f90 0x7fdbc40b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 -- 172.21.15.91:0/2268828646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4009400 con 0x7fdbc40b0e10 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc3400640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 0x7fdbc40c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbc2a00640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c430 tx=0x7fdbac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0074e0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac007660 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc40c3bf0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.526+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac004a20 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.527+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc40c41b0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.528+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc40c43d0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.531 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.529+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdbac004b80 con 0x7fdbc40b0e10 2024-02-02T13:35:29.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.530+0000 7fdbbb400640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 0x7fdb9c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.530+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbac088330 con 0x7fdbc40b0e10 2024-02-02T13:35:29.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.530+0000 7fdbc3400640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 0x7fdb9c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.532 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.531+0000 7fdbc3400640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 0x7fdb9c06c8f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4036000 tx=0x7fdbb4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.536 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.535+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdbac053d00 con 0x7fdbc40b0e10 2024-02-02T13:35:29.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.728+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbc4066cf0 con 0x7fdbc40b0e10 2024-02-02T13:35:29.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.730+0000 7fdbbb400640 1 -- 172.21.15.91:0/2268828646 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+12400 (secure 0 0 0) 0x7fdbac057320 con 0x7fdbc40b0e10 2024-02-02T13:35:29.732 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:29.732 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":174,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:35:28.772036+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:29.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 msgr2=0x7fdb9c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 0x7fdb9c06c8f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4036000 tx=0x7fdbb4008030 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 msgr2=0x7fdbc41f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c430 tx=0x7fdbac00c8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 shutdown_connections 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdb9c06a430 0x7fdb9c06c8f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbc4151f90 0x7fdbc40b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdbc40b0e10 0x7fdbc41f18e0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 --2- 172.21.15.91:0/2268828646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbc406f4f0 0x7fdbc40c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 >> 172.21.15.91:0/2268828646 conn(0x7fdbc40cec80 msgr2=0x7fdbc40cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 shutdown_connections 2024-02-02T13:35:29.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.733+0000 7fdbca000640 1 -- 172.21.15.91:0/2268828646 wait complete. 2024-02-02T13:35:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:35:29.755 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:35:29.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.968+0000 7f8dc4c00640 1 Processor -- start 2024-02-02T13:35:29.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.969+0000 7f8dc4c00640 1 -- start start 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dc4c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8dc00b0e10 0x7f8dc00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dc4c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00cc7c0 con 0x7f8dc015a010 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dc4c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00ce930 con 0x7f8dc00ce130 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dc4c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00cd640 con 0x7f8dc00b0e10 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dbde00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8dc015a010 0x7f8dc00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46662/0 (socket says 172.21.15.91:46662) 2024-02-02T13:35:29.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.970+0000 7f8dbde00640 1 -- 172.21.15.91:0/2635938892 learned_addr learned my addr 172.21.15.91:0/2635938892 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:29.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395936850 0 0) 0x7f8dc00cc7c0 con 0x7f8dc015a010 2024-02-02T13:35:29.975 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da0005730 con 0x7f8dc015a010 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbe800640 1 --2- 172.21.15.91:0/2635938892 >> v2:172.21.15.113:3300/0 conn(0x7f8dc00b0e10 0x7f8dc00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3346842463 0 0) 0x7f8da0005730 con 0x7f8dc015a010 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 >> v2:172.21.15.113:3300/0 conn(0x7f8dc00b0e10 msgr2=0x7f8dc00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 --2- 172.21.15.91:0/2635938892 >> v2:172.21.15.113:3300/0 conn(0x7f8dc00b0e10 0x7f8dc00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 >> v1:172.21.15.113:6789/0 conn(0x7f8dc00ce130 legacy=0x7f8dc00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc006c890 con 0x7f8dc015a010 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8db0001670 con 0x7f8dc015a010 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 >> v1:172.21.15.91:6789/0 conn(0x7f8dc015a010 legacy=0x7f8dc00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 0x7f8da0005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8da00069d0 0x7f8da0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da00074b0 0x7f8da000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00cd640 con 0x7f8da0005b80 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da000e360 con 0x7f8da00074b0 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.971+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da000e520 con 0x7f8da00069d0 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 0x7f8da0005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbe800640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da00074b0 0x7f8da000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8db5200640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8da00069d0 0x7f8da0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 -- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8da00069d0 msgr2=0x7f8da0006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8da00069d0 0x7f8da0006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 -- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da00074b0 msgr2=0x7f8da000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da00074b0 0x7f8da000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbde00640 1 -- 172.21.15.91:0/2635938892 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0005730 con 0x7f8da0005b80 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8db5200640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8da00069d0 0x7f8da0006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbe800640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da00074b0 0x7f8da000db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.972+0000 7f8dbe800640 1 -- 172.21.15.91:0/2635938892 reap_dead start 2024-02-02T13:35:29.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.973+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 0x7f8da0005f80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8db0001eb0 tx=0x7f8da0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d092e2fb74a59b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0005ad0 con 0x7f8da0005b80 2024-02-02T13:35:29.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0005c50 con 0x7f8da0005b80 2024-02-02T13:35:29.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dbd400640 1 -- 172.21.15.91:0/2635938892 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db002c8b0 con 0x7f8da0005b80 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 msgr2=0x7f8da0005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 0x7f8da0005f80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8db0001eb0 tx=0x7f8da0007a00 comp rx=0 tx=0).stop 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2635938892 shutdown_connections 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2635938892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da0005b80 0x7f8da0005f80 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2635938892 >> 172.21.15.91:0/2635938892 conn(0x7f8dc00cec80 msgr2=0x7f8dc00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2635938892 shutdown_connections 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.974+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2635938892 wait complete. 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.976+0000 7f8dc4c00640 1 Processor -- start 2024-02-02T13:35:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.976+0000 7f8dc4c00640 1 -- start start 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da0005b80 0x7f8dc00ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dc01571e0 0x7f8dc0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0157ba0 con 0x7f8dc00b3100 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00b38a0 con 0x7f8da0005b80 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8dc4c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00b3b80 con 0x7f8dc01571e0 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8db5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8db5200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40912/0 (socket says 172.21.15.91:40912) 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.977+0000 7f8db5200640 1 -- 172.21.15.91:0/2668134542 learned_addr learned my addr 172.21.15.91:0/2668134542 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8dbe800640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dc01571e0 0x7f8dc0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da0005b80 0x7f8dc00ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 -- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dc01571e0 msgr2=0x7f8dc0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dc01571e0 0x7f8dc0157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 -- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da0005b80 msgr2=0x7f8dc00ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da0005b80 0x7f8dc00ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 -- 172.21.15.91:0/2668134542 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00b3e60 con 0x7f8dc00b3100 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db5200640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d3b0 tx=0x7f8da800d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800b070 con 0x7f8dc00b3100 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8007030 con 0x7f8dc00b3100 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00b4140 con 0x7f8dc00b3100 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.978+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8007420 con 0x7f8dc00b3100 2024-02-02T13:35:29.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.979+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00c6bd0 con 0x7f8dc00b3100 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.980+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0066cf0 con 0x7f8dc00b3100 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.981+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8da8007580 con 0x7f8dc00b3100 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.981+0000 7f8db6a00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 0x7f8d9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.982+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8da80875e0 con 0x7f8dc00b3100 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.982+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 0x7f8d9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.983+0000 7f8dbde00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 0x7f8d9806c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034000 tx=0x7f8db0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:29.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:29.986+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8da8052fb0 con 0x7f8dc00b3100 2024-02-02T13:35:30.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.179+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dc00b4c00 con 0x7f8dc00b3100 2024-02-02T13:35:30.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.180+0000 7f8db6a00640 1 -- 172.21.15.91:0/2668134542 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8da8018020 con 0x7f8dc00b3100 2024-02-02T13:35:30.181 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.182+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 msgr2=0x7f8d9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.182+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 0x7f8d9806c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034000 tx=0x7f8db0009030 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.182+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 msgr2=0x7f8dc00c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.182+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d3b0 tx=0x7f8da800d870 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 shutdown_connections 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d9806a430 0x7f8d9806c8f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dc01571e0 0x7f8dc0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8dc00b3100 0x7f8dc00c45d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 --2- 172.21.15.91:0/2668134542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da0005b80 0x7f8dc00ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 >> 172.21.15.91:0/2668134542 conn(0x7f8dc00cec80 msgr2=0x7f8dc0070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 shutdown_connections 2024-02-02T13:35:30.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.183+0000 7f8dc4c00640 1 -- 172.21.15.91:0/2668134542 wait complete. 2024-02-02T13:35:30.204 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:35:30.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.410+0000 7fd1ae000640 1 Processor -- start 2024-02-02T13:35:30.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.410+0000 7fd1ae000640 1 -- start start 2024-02-02T13:35:30.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd1ae000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd1ae000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a80c94a0 con 0x7fd1a80cb580 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd1ae000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a80ad6e0 con 0x7fd1a80ac960 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd1ae000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80ad500 con 0x7fd1a815d3a0 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd19e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd19e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40920/0 (socket says 172.21.15.91:40920) 2024-02-02T13:35:30.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.412+0000 7fd19e800640 1 -- 172.21.15.91:0/615352099 learned_addr learned my addr 172.21.15.91:0/615352099 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.413+0000 7fd19e800640 1 -- 172.21.15.91:0/615352099 >> v1:172.21.15.113:6789/0 conn(0x7fd1a80ac960 legacy=0x7fd1a80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.413+0000 7fd19e800640 1 -- 172.21.15.91:0/615352099 >> v1:172.21.15.91:6789/0 conn(0x7fd1a80cb580 legacy=0x7fd1a80cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.413+0000 7fd19e800640 1 -- 172.21.15.91:0/615352099 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80c96f0 con 0x7fd1a815d3a0 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.413+0000 7fd1a6a00640 1 -- 172.21.15.91:0/615352099 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732775040 0 0) 0x7fd1a80c94a0 con 0x7fd1a80cb580 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.414+0000 7fd19e800640 1 --2- 172.21.15.91:0/615352099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81365935a95e7a0c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.414+0000 7fd1a6a00640 1 -- 172.21.15.91:0/615352099 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190030850 con 0x7fd1a815d3a0 2024-02-02T13:35:30.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.414+0000 7fd1a6a00640 1 -- 172.21.15.91:0/615352099 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1900309b0 con 0x7fd1a815d3a0 2024-02-02T13:35:30.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.414+0000 7fd1a6a00640 1 -- 172.21.15.91:0/615352099 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190030d40 con 0x7fd1a815d3a0 2024-02-02T13:35:30.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 -- 172.21.15.91:0/615352099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a815d3a0 msgr2=0x7fd1a80ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 --2- 172.21.15.91:0/615352099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002b990 comp rx=0 tx=0).stop 2024-02-02T13:35:30.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 -- 172.21.15.91:0/615352099 shutdown_connections 2024-02-02T13:35:30.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 --2- 172.21.15.91:0/615352099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a815d3a0 0x7fd1a80ca1d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 -- 172.21.15.91:0/615352099 >> 172.21.15.91:0/615352099 conn(0x7fd1a80b0700 msgr2=0x7fd1a80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:30.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 -- 172.21.15.91:0/615352099 shutdown_connections 2024-02-02T13:35:30.417 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.415+0000 7fd1ae000640 1 -- 172.21.15.91:0/615352099 wait complete. 2024-02-02T13:35:30.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.417+0000 7fd1ae000640 1 Processor -- start 2024-02-02T13:35:30.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.417+0000 7fd1ae000640 1 -- start start 2024-02-02T13:35:30.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a80ac960 0x7fd1a80c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a815d3a0 0x7fd1a81f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a806b5a0 con 0x7fd1a80cb580 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c5350 con 0x7fd1a80ac960 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1ae000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81f6370 con 0x7fd1a815d3a0 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd19e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd19e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40936/0 (socket says 172.21.15.91:40936) 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd19e800640 1 -- 172.21.15.91:0/3209742999 learned_addr learned my addr 172.21.15.91:0/3209742999 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1a7400640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a80ac960 0x7fd1a80c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd1a7e00640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a815d3a0 0x7fd1a81f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.418+0000 7fd19e800640 1 -- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a815d3a0 msgr2=0x7fd1a81f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19e800640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a815d3a0 0x7fd1a81f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19e800640 1 -- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a80ac960 msgr2=0x7fd1a80c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19e800640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a80ac960 0x7fd1a80c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19e800640 1 -- 172.21.15.91:0/3209742999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900094d0 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19e800640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd1900090b0 tx=0x7fd1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19003e070 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1900398d0 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a81f6650 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190039c80 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.419+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a806bab0 con 0x7fd1a80cb580 2024-02-02T13:35:30.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.421+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a8066cf0 con 0x7fd1a80cb580 2024-02-02T13:35:30.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.421+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd190042070 con 0x7fd1a80cb580 2024-02-02T13:35:30.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.422+0000 7fd19fe00640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 0x7fd18006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.422+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd1900aa360 con 0x7fd1a80cb580 2024-02-02T13:35:30.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.423+0000 7fd1a7400640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 0x7fd18006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.423+0000 7fd1a7400640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 0x7fd18006c8c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd198000f60 tx=0x7fd198009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.427+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd190075d30 con 0x7fd1a80cb580 2024-02-02T13:35:30.621 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.619+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd1a80ca810 con 0x7fd1a80cb580 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.620+0000 7fd19fe00640 1 -- 172.21.15.91:0/3209742999 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v174) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd190079350 con 0x7fd1a80cb580 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:35:30.622 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:35:30.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 msgr2=0x7fd18006c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 0x7fd18006c8c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd198000f60 tx=0x7fd198009040 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 msgr2=0x7fd1a806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd1900090b0 tx=0x7fd1900046c0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 shutdown_connections 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd18006a400 0x7fd18006c8c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a815d3a0 0x7fd1a81f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1a80cb580 0x7fd1a806bfd0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 --2- 172.21.15.91:0/3209742999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a80ac960 0x7fd1a80c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 >> 172.21.15.91:0/3209742999 conn(0x7fd1a80b0700 msgr2=0x7fd1a80ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 shutdown_connections 2024-02-02T13:35:30.625 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.623+0000 7fd1ae000640 1 -- 172.21.15.91:0/3209742999 wait complete. 2024-02-02T13:35:30.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:35:30.644 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:35:30.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.854+0000 7f916f400640 1 Processor -- start 2024-02-02T13:35:30.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.854+0000 7f916f400640 1 -- start start 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f916815a280 0x7f916806a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916f400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91680cace0 con 0x7f916806d180 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916f400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91680cf190 con 0x7f9168159ea0 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916f400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916806cbf0 con 0x7f916815a280 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916d400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f916806d180 0x7f91680cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46674/0 (socket says 172.21.15.91:46674) 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f916d400640 1 -- 172.21.15.91:0/1073467758 learned_addr learned my addr 172.21.15.91:0/1073467758 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:30.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.856+0000 7f9167e00640 1 --2- 172.21.15.91:0/1073467758 >> v2:172.21.15.113:3300/0 conn(0x7f916815a280 0x7f916806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199418509 0 0) 0x7f91680cace0 con 0x7f916806d180 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9148003520 con 0x7f916806d180 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126991305 0 0) 0x7f9148003520 con 0x7f916806d180 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 >> v2:172.21.15.113:3300/0 conn(0x7f916815a280 msgr2=0x7f916806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 --2- 172.21.15.91:0/1073467758 >> v2:172.21.15.113:3300/0 conn(0x7f916815a280 0x7f916806a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 >> v1:172.21.15.113:6789/0 conn(0x7f9168159ea0 legacy=0x7f9168069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916806ced0 con 0x7f916806d180 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9160001670 con 0x7f916806d180 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 >> v1:172.21.15.91:6789/0 conn(0x7f916806d180 legacy=0x7f91680cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91480039c0 0x7f9148003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f9148004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9148005180 0x7f914800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916806cbf0 con 0x7f91480047c0 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914800c080 con 0x7f91480039c0 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.857+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914800e450 con 0x7f9148005180 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f9148004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f916ca00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91480039c0 0x7f9148003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 -- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9148005180 msgr2=0x7f914800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:30.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9148005180 0x7f914800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 -- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91480039c0 msgr2=0x7f9148003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:30.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91480039c0 0x7f9148003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f9167e00640 1 -- 172.21.15.91:0/1073467758 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148003520 con 0x7f91480047c0 2024-02-02T13:35:30.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f916d400640 1 -- 172.21.15.91:0/1073467758 reap_dead start 2024-02-02T13:35:30.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.858+0000 7f916ca00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91480039c0 0x7f9148003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.859+0000 7f9167e00640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f9148004c40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9158009f90 tx=0x7f91480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59e3d43191a5990 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915802cbf0 con 0x7f91480047c0 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915800f040 con 0x7f91480047c0 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f9167400640 1 -- 172.21.15.91:0/1073467758 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158034440 con 0x7f91480047c0 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 -- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 msgr2=0x7f9148004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f9148004c40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9158009f90 tx=0x7f91480057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 -- 172.21.15.91:0/1073467758 shutdown_connections 2024-02-02T13:35:30.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 --2- 172.21.15.91:0/1073467758 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f9148004c40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 -- 172.21.15.91:0/1073467758 >> 172.21.15.91:0/1073467758 conn(0x7f91680cfd40 msgr2=0x7f91680d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:30.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 -- 172.21.15.91:0/1073467758 shutdown_connections 2024-02-02T13:35:30.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.860+0000 7f916f400640 1 -- 172.21.15.91:0/1073467758 wait complete. 2024-02-02T13:35:30.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.862+0000 7f916f400640 1 Processor -- start 2024-02-02T13:35:30.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.862+0000 7f916f400640 1 -- start start 2024-02-02T13:35:30.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91680c6e10 0x7f91680c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91680cd910 0x7f91680cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91680ce2d0 con 0x7f91480047c0 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91680c8a30 con 0x7f91680c6e10 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916f400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91680c9310 con 0x7f91680cd910 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.863+0000 7f916ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40956/0 (socket says 172.21.15.91:40956) 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 -- 172.21.15.91:0/2897217571 learned_addr learned my addr 172.21.15.91:0/2897217571 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 -- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91680cd910 msgr2=0x7f91680cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91680cd910 0x7f91680cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 -- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91680c6e10 msgr2=0x7f91680c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91680c6e10 0x7f91680c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 -- 172.21.15.91:0/2897217571 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680c7390 con 0x7f91480047c0 2024-02-02T13:35:30.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916ca00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f915000d400 tx=0x7f915000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915000b070 con 0x7f91480047c0 2024-02-02T13:35:30.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150003610 con 0x7f91480047c0 2024-02-02T13:35:30.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91680c7670 con 0x7f91480047c0 2024-02-02T13:35:30.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.864+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150003a00 con 0x7f91480047c0 2024-02-02T13:35:30.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.865+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91680c7c30 con 0x7f91480047c0 2024-02-02T13:35:30.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.866+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9168073830 con 0x7f91480047c0 2024-02-02T13:35:30.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.866+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9150003b60 con 0x7f91480047c0 2024-02-02T13:35:30.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.867+0000 7f9164c00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 0x7f914006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:30.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.867+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9150087ce0 con 0x7f91480047c0 2024-02-02T13:35:30.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.868+0000 7f9167e00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 0x7f914006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:30.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.868+0000 7f9167e00640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 0x7f914006c8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9158036000 tx=0x7f915802e070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:30.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:30.872+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f91500536d0 con 0x7f91480047c0 2024-02-02T13:35:31.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.061+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f91680ceb10 con 0x7f91480047c0 2024-02-02T13:35:31.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.148+0000 7f9164c00640 1 -- 172.21.15.91:0/2897217571 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v175) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9150056cf0 con 0x7f91480047c0 2024-02-02T13:35:31.151 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:35:31.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.153+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 msgr2=0x7f914006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.153+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 0x7f914006c8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9158036000 tx=0x7f915802e070 comp rx=0 tx=0).stop 2024-02-02T13:35:31.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 msgr2=0x7f91680c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f915000d400 tx=0x7f915000d8c0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 shutdown_connections 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f914006a430 0x7f914006c8f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91680cd910 0x7f91680cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91680c6e10 0x7f91680c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 --2- 172.21.15.91:0/2897217571 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f91480047c0 0x7f91680c8dd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 >> 172.21.15.91:0/2897217571 conn(0x7f91680cfd40 msgr2=0x7f91680cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:31.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 shutdown_connections 2024-02-02T13:35:31.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.154+0000 7f916f400640 1 -- 172.21.15.91:0/2897217571 wait complete. 2024-02-02T13:35:31.175 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:35:31.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.386+0000 7ff424800640 1 Processor -- start 2024-02-02T13:35:31.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.386+0000 7ff424800640 1 -- start start 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.387+0000 7ff424800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff41c15d5e0 0x7ff41c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.387+0000 7ff424800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0d1280 con 0x7ff41c0d2720 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff424800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c161ed0 con 0x7ff41c0ce070 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff424800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c15d9c0 con 0x7ff41c15d5e0 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff41c15d5e0 0x7ff41c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff41c15d5e0 0x7ff41c0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40968/0 (socket says 172.21.15.91:40968) 2024-02-02T13:35:31.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 -- 172.21.15.91:0/434521158 learned_addr learned my addr 172.21.15.91:0/434521158 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 -- 172.21.15.91:0/434521158 >> v1:172.21.15.113:6789/0 conn(0x7ff41c0ce070 legacy=0x7ff41c1610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 -- 172.21.15.91:0/434521158 >> v1:172.21.15.91:6789/0 conn(0x7ff41c0d2720 legacy=0x7ff41c1617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff421e00640 1 -- 172.21.15.91:0/434521158 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c0d0c70 con 0x7ff41c15d5e0 2024-02-02T13:35:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.388+0000 7ff420a00640 1 -- 172.21.15.91:0/434521158 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896915675 0 0) 0x7ff41c0d1280 con 0x7ff41c0d2720 2024-02-02T13:35:31.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff421e00640 1 --2- 172.21.15.91:0/434521158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c15d5e0 0x7ff41c0cd990 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009870 tx=0x7ff40c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c6eb8822085be7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:31.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff420a00640 1 -- 172.21.15.91:0/434521158 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c030860 con 0x7ff41c15d5e0 2024-02-02T13:35:31.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff420a00640 1 -- 172.21.15.91:0/434521158 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c0309c0 con 0x7ff41c15d5e0 2024-02-02T13:35:31.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff420a00640 1 -- 172.21.15.91:0/434521158 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c030d50 con 0x7ff41c15d5e0 2024-02-02T13:35:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff424800640 1 -- 172.21.15.91:0/434521158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c15d5e0 msgr2=0x7ff41c0cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.390+0000 7ff424800640 1 --2- 172.21.15.91:0/434521158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c15d5e0 0x7ff41c0cd990 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009870 tx=0x7ff40c02b9a0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.391+0000 7ff424800640 1 -- 172.21.15.91:0/434521158 shutdown_connections 2024-02-02T13:35:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.391+0000 7ff424800640 1 --2- 172.21.15.91:0/434521158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c15d5e0 0x7ff41c0cd990 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.391+0000 7ff424800640 1 -- 172.21.15.91:0/434521158 >> 172.21.15.91:0/434521158 conn(0x7ff41c0d30a0 msgr2=0x7ff41c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.391+0000 7ff424800640 1 -- 172.21.15.91:0/434521158 shutdown_connections 2024-02-02T13:35:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.391+0000 7ff424800640 1 -- 172.21.15.91:0/434521158 wait complete. 2024-02-02T13:35:31.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.392+0000 7ff424800640 1 Processor -- start 2024-02-02T13:35:31.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.392+0000 7ff424800640 1 -- start start 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff41c0ce070 0x7ff41c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff41c15d5e0 0x7ff41c1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b48a0 con 0x7ff41c0d2720 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b3c20 con 0x7ff41c0ce070 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.393+0000 7ff424800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b3f00 con 0x7ff41c15d5e0 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40978/0 (socket says 172.21.15.91:40978) 2024-02-02T13:35:31.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 -- 172.21.15.91:0/1698017559 learned_addr learned my addr 172.21.15.91:0/1698017559 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:31.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421e00640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff41c0ce070 0x7ff41c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff422800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff41c15d5e0 0x7ff41c1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 -- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff41c15d5e0 msgr2=0x7ff41c1f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff41c15d5e0 0x7ff41c1f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 -- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff41c0ce070 msgr2=0x7ff41c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff41c0ce070 0x7ff41c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 -- 172.21.15.91:0/1698017559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0094d0 con 0x7ff41c0d2720 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.394+0000 7ff421400640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b3f0 tx=0x7ff40400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.395+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404011830 con 0x7ff41c0d2720 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.395+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404011990 con 0x7ff41c0d2720 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.395+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404011d40 con 0x7ff41c0d2720 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.395+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0b41e0 con 0x7ff41c0d2720 2024-02-02T13:35:31.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.395+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c14afc0 con 0x7ff41c0d2720 2024-02-02T13:35:31.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.396+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c071a30 con 0x7ff41c0d2720 2024-02-02T13:35:31.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.397+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff404004020 con 0x7ff41c0d2720 2024-02-02T13:35:31.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.398+0000 7ff41a000640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 0x7ff3f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:31.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.398+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff4040889f0 con 0x7ff41c0d2720 2024-02-02T13:35:31.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.398+0000 7ff421e00640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 0x7ff3f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:31.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.399+0000 7ff421e00640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 0x7ff3f406c920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff40c036000 tx=0x7ff40c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:31.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.402+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4040541a0 con 0x7ff41c0d2720 2024-02-02T13:35:31.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:31.592+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff41c0b49e0 con 0x7ff41c0d2720 2024-02-02T13:35:31.988 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:35:31.996 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:35:32.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.168+0000 7ff41a000640 1 -- 172.21.15.91:0/1698017559 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v176) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff41c0b49e0 con 0x7ff41c0d2720 2024-02-02T13:35:32.175 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:35:32.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.176+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 msgr2=0x7ff3f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.176+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 0x7ff3f406c920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff40c036000 tx=0x7ff40c008030 comp rx=0 tx=0).stop 2024-02-02T13:35:32.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.176+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 msgr2=0x7ff41c0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.176+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b3f0 tx=0x7ff40400b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 shutdown_connections 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff3f406a460 0x7ff3f406c920 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff41c15d5e0 0x7ff41c1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff41c0d2720 0x7ff41c0b31d0 secure :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b3f0 tx=0x7ff40400b8b0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 --2- 172.21.15.91:0/1698017559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff41c0ce070 0x7ff41c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.177+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 >> 172.21.15.91:0/1698017559 conn(0x7ff41c0d30a0 msgr2=0x7ff41c158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:32.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.179+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 shutdown_connections 2024-02-02T13:35:32.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.179+0000 7ff424800640 1 -- 172.21.15.91:0/1698017559 wait complete. 2024-02-02T13:35:32.201 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:35:32.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.386+0000 7f5ebfe00640 1 Processor -- start 2024-02-02T13:35:32.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.387+0000 7f5ebfe00640 1 -- start start 2024-02-02T13:35:32.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ec00aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebfe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00717b0 con 0x7f5ec00af040 2024-02-02T13:35:32.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebfe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00714d0 con 0x7f5ec00cbbf0 2024-02-02T13:35:32.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebfe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00af4b0 con 0x7f5ec00cd6e0 2024-02-02T13:35:32.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebf400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5ec00af040 0x7f5ec0070400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46682/0 (socket says 172.21.15.91:46682) 2024-02-02T13:35:32.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.388+0000 7f5ebf400640 1 -- 172.21.15.91:0/3463819413 learned_addr learned my addr 172.21.15.91:0/3463819413 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706787228 0 0) 0x7f5ec00717b0 con 0x7f5ec00af040 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea0003520 con 0x7f5ec00af040 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5eb5e00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ec00aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864731104 0 0) 0x7f5ea0003520 con 0x7f5ec00af040 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 msgr2=0x7f5ec00aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ec00aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 >> v1:172.21.15.113:6789/0 conn(0x7f5ec00cbbf0 legacy=0x7f5ec0160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00b1740 con 0x7f5ec00af040 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5eb8001670 con 0x7f5ec00af040 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.389+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 >> v1:172.21.15.91:6789/0 conn(0x7f5ec00af040 legacy=0x7f5ec0070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ea00039a0 0x7f5ea0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ea00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea0004d00 0x7f5ea0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00af4b0 con 0x7f5ea00039a0 2024-02-02T13:35:32.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea000bb60 con 0x7f5ea0004d00 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea000bd20 con 0x7f5ec00cd6e0 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ea00039a0 0x7f5ea0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5eb5e00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ea00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 -- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 msgr2=0x7f5ea00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ec00cd6e0 0x7f5ea00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 -- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea0004d00 msgr2=0x7f5ea0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:32.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea0004d00 0x7f5ea0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.390+0000 7f5ebea00640 1 -- 172.21.15.91:0/3463819413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0003520 con 0x7f5ea00039a0 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.391+0000 7f5ebea00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ea00039a0 0x7f5ea0003dc0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0001e50 tx=0x7f5ea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904aee238d876050 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003b70 con 0x7f5ea00039a0 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0003cd0 con 0x7f5ea00039a0 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebe000640 1 -- 172.21.15.91:0/3463819413 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0005310 con 0x7f5ea00039a0 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 -- 172.21.15.91:0/3463819413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ea00039a0 msgr2=0x7f5ea0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/3463819413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ea00039a0 0x7f5ea0003dc0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0001e50 tx=0x7f5ea00057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebf400640 1 -- 172.21.15.91:0/3463819413 reap_dead start 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 -- 172.21.15.91:0/3463819413 shutdown_connections 2024-02-02T13:35:32.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 -- 172.21.15.91:0/3463819413 >> 172.21.15.91:0/3463819413 conn(0x7f5ec00cfba0 msgr2=0x7f5ec00cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 -- 172.21.15.91:0/3463819413 shutdown_connections 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.392+0000 7f5ebfe00640 1 -- 172.21.15.91:0/3463819413 wait complete. 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.394+0000 7f5ebfe00640 1 Processor -- start 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.394+0000 7f5ebfe00640 1 -- start start 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 0x7f5ec00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 0x7f5ec015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01f6000 con 0x7f5ec00b2740 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00b2f20 con 0x7f5ec01f67e0 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5ebfe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00b3200 con 0x7f5ea00039a0 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.395+0000 7f5eb5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:40994/0 (socket says 172.21.15.91:40994) 2024-02-02T13:35:32.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 -- 172.21.15.91:0/1208721012 learned_addr learned my addr 172.21.15.91:0/1208721012 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5ebf400640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 0x7f5ec015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5ebea00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 0x7f5ec00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 -- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 msgr2=0x7f5ec00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 0x7f5ec00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 -- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 msgr2=0x7f5ec015f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 0x7f5ec015f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 -- 172.21.15.91:0/1208721012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00b34e0 con 0x7f5ec00b2740 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5ebf400640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 0x7f5ec015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5ebea00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 0x7f5ec00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.396+0000 7f5eb5e00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e7e0 tx=0x7f5ea800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.397+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800c6d0 con 0x7f5ec00b2740 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.397+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800c830 con 0x7f5ec00b2740 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.397+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800cba0 con 0x7f5ec00b2740 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.397+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec01f62c0 con 0x7f5ec00b2740 2024-02-02T13:35:32.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.397+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec015c5b0 con 0x7f5ec00b2740 2024-02-02T13:35:32.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.398+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec004dff0 con 0x7f5ec00b2740 2024-02-02T13:35:32.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.399+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5ea800cd00 con 0x7f5ec00b2740 2024-02-02T13:35:32.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.400+0000 7f5eb7400640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 0x7f5e9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.400+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5ea80886a0 con 0x7f5ec00b2740 2024-02-02T13:35:32.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.400+0000 7f5ebea00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 0x7f5e9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.401+0000 7f5ebea00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 0x7f5e9806c8f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0034000 tx=0x7f5eb0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.406+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5ea8053c50 con 0x7f5ec00b2740 2024-02-02T13:35:32.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.635+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5ec006c7d0 con 0x7f5ec00b2740 2024-02-02T13:35:32.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.645+0000 7f5eb7400640 1 -- 172.21.15.91:0/1208721012 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5ea8057270 con 0x7f5ec00b2740 2024-02-02T13:35:32.646 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 20 and data pool 21 2024-02-02T13:35:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.648+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 msgr2=0x7f5e9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.648+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 0x7f5e9806c8f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0034000 tx=0x7f5eb0009030 comp rx=0 tx=0).stop 2024-02-02T13:35:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.648+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 msgr2=0x7f5ec015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.648+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e7e0 tx=0x7f5ea800eca0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 shutdown_connections 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5e9806a430 0x7f5e9806c8f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ec01f67e0 0x7f5ec015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ec00b2740 0x7f5ec015c070 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 --2- 172.21.15.91:0/1208721012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea00039a0 0x7f5ec00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 >> 172.21.15.91:0/1208721012 conn(0x7f5ec00cfba0 msgr2=0x7f5ec00caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 shutdown_connections 2024-02-02T13:35:32.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.649+0000 7f5ebfe00640 1 -- 172.21.15.91:0/1208721012 wait complete. 2024-02-02T13:35:32.681 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:32.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.855+0000 7f5b0ec00640 1 Processor -- start 2024-02-02T13:35:32.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.855+0000 7f5b0ec00640 1 -- start start 2024-02-02T13:35:32.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.856+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0815aec0 0x7f5b080cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.856+0000 7f5b0ec00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b0806f3f0 0x7f5b080ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.856+0000 7f5b0ec00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0806c1c0 con 0x7f5b0815b2a0 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.856+0000 7f5b0ec00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0806b5f0 con 0x7f5b0806f3f0 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.856+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0806cd60 con 0x7f5b0815aec0 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b0806f3f0 0x7f5b080ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5b0806f3f0 0x7f5b080ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41008/0 (socket says 172.21.15.91:41008) 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 -- 172.21.15.91:0/3467286745 learned_addr learned my addr 172.21.15.91:0/3467286745 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0ce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5b0815b2a0 0x7f5b0806c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46698/0 (socket says 172.21.15.91:46698) 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0d800640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0815aec0 0x7f5b080cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 -- 172.21.15.91:0/3467286745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0815aec0 msgr2=0x7f5b080cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0815aec0 0x7f5b080cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 -- 172.21.15.91:0/3467286745 >> v1:172.21.15.91:6789/0 conn(0x7f5b0815b2a0 legacy=0x7f5b0806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b0e200640 1 -- 172.21.15.91:0/3467286745 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0806b890 con 0x7f5b0806f3f0 2024-02-02T13:35:32.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.857+0000 7f5b07e00640 1 -- 172.21.15.91:0/3467286745 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371634128 0 0) 0x7f5b0806c1c0 con 0x7f5b0815b2a0 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.858+0000 7f5b0e200640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0806f3f0 0x7f5b080ca620 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009870 tx=0x7f5b0002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79017a2d3bf45e05 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.858+0000 7f5b07e00640 1 -- 172.21.15.91:0/3467286745 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0002f840 con 0x7f5b0806f3f0 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b07e00640 1 -- 172.21.15.91:0/3467286745 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0002f9a0 con 0x7f5b0806f3f0 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b07e00640 1 -- 172.21.15.91:0/3467286745 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0002fd30 con 0x7f5b0806f3f0 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 -- 172.21.15.91:0/3467286745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0806f3f0 msgr2=0x7f5b080ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0806f3f0 0x7f5b080ca620 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009870 tx=0x7f5b0002ba20 comp rx=0 tx=0).stop 2024-02-02T13:35:32.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 -- 172.21.15.91:0/3467286745 shutdown_connections 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0806f3f0 0x7f5b080ca620 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/3467286745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0815aec0 0x7f5b080cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 -- 172.21.15.91:0/3467286745 >> 172.21.15.91:0/3467286745 conn(0x7f5b080cea90 msgr2=0x7f5b080ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 -- 172.21.15.91:0/3467286745 shutdown_connections 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.859+0000 7f5b0ec00640 1 -- 172.21.15.91:0/3467286745 wait complete. 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.861+0000 7f5b0ec00640 1 Processor -- start 2024-02-02T13:35:32.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.861+0000 7f5b0ec00640 1 -- start start 2024-02-02T13:35:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0806f3f0 0x7f5b08159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0815b2a0 0x7f5b080b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0815dd40 con 0x7f5b0815aec0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0815e1d0 con 0x7f5b0815b2a0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0815d130 con 0x7f5b0806f3f0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41012/0 (socket says 172.21.15.91:41012) 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.862+0000 7f5b0ce00640 1 -- 172.21.15.91:0/2214118244 learned_addr learned my addr 172.21.15.91:0/2214118244 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0d800640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0806f3f0 0x7f5b08159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 -- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0806f3f0 msgr2=0x7f5b08159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0806f3f0 0x7f5b08159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 -- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0815b2a0 msgr2=0x7f5b080b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0e200640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0815b2a0 0x7f5b080b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0815b2a0 0x7f5b080b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 -- 172.21.15.91:0/2214118244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000094d0 con 0x7f5b0815aec0 2024-02-02T13:35:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ce00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c430 tx=0x7f5af000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00074e0 con 0x7f5b0815aec0 2024-02-02T13:35:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0007660 con 0x7f5b0815aec0 2024-02-02T13:35:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0004a20 con 0x7f5b0815aec0 2024-02-02T13:35:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0815d410 con 0x7f5b0815aec0 2024-02-02T13:35:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.863+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0815d9d0 con 0x7f5b0815aec0 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.864+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0815deb0 con 0x7f5b0815aec0 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.865+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5af0004b80 con 0x7f5b0815aec0 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.866+0000 7f5b05600640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 0x7f5ae006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.866+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5af00887f0 con 0x7f5b0815aec0 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.867+0000 7f5b0d800640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 0x7f5ae006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:32.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.867+0000 7f5b0d800640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 0x7f5ae006c8f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5b081f6200 tx=0x7f5af800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:32.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:32.871+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5af0053d40 con 0x7f5b0815aec0 2024-02-02T13:35:33.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.059+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b0804dff0 con 0x7f5b0815aec0 2024-02-02T13:35:33.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.060+0000 7f5b05600640 1 -- 172.21.15.91:0/2214118244 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f5af0057360 con 0x7f5b0815aec0 2024-02-02T13:35:33.062 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:33.062 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":177,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:35:32.636903+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.062+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 msgr2=0x7f5ae006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.062+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 0x7f5ae006c8f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5b081f6200 tx=0x7f5af800b040 comp rx=0 tx=0).stop 2024-02-02T13:35:33.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.062+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 msgr2=0x7f5b080c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.062+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c430 tx=0x7f5af000c8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 shutdown_connections 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5ae006a430 0x7f5ae006c8f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b0815b2a0 0x7f5b080b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5b0815aec0 0x7f5b080c4e90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 --2- 172.21.15.91:0/2214118244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b0806f3f0 0x7f5b08159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 >> 172.21.15.91:0/2214118244 conn(0x7f5b080cea90 msgr2=0x7f5b080ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 shutdown_connections 2024-02-02T13:35:33.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.063+0000 7f5b0ec00640 1 -- 172.21.15.91:0/2214118244 wait complete. 2024-02-02T13:35:33.082 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:33.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.289+0000 7ff8a6400640 1 Processor -- start 2024-02-02T13:35:33.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.289+0000 7ff8a6400640 1 -- start start 2024-02-02T13:35:33.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff8a6400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8a015f5a0 0x7ff8a00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff8a6400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a00b1070 con 0x7ff8a00b0c40 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff8a6400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a006c600 con 0x7ff8a0151f90 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff8a6400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a015fa30 con 0x7ff8a015f5a0 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff897e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8a015f5a0 0x7ff8a00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff897e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8a015f5a0 0x7ff8a00b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41028/0 (socket says 172.21.15.91:41028) 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.291+0000 7ff897e00640 1 -- 172.21.15.91:0/4089651318 learned_addr learned my addr 172.21.15.91:0/4089651318 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:33.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.292+0000 7ff897e00640 1 -- 172.21.15.91:0/4089651318 >> v1:172.21.15.113:6789/0 conn(0x7ff8a0151f90 legacy=0x7ff8a006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.292+0000 7ff897e00640 1 -- 172.21.15.91:0/4089651318 >> v1:172.21.15.91:6789/0 conn(0x7ff8a00b0c40 legacy=0x7ff8a00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.292+0000 7ff897e00640 1 -- 172.21.15.91:0/4089651318 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0070f80 con 0x7ff8a015f5a0 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.293+0000 7ff897e00640 1 --2- 172.21.15.91:0/4089651318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a015f5a0 0x7ff8a00b0700 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff888009870 tx=0x7ff88802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3284988a7463aea7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.293+0000 7ff89ea00640 1 -- 172.21.15.91:0/4089651318 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888030850 con 0x7ff8a015f5a0 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.293+0000 7ff89ea00640 1 -- 172.21.15.91:0/4089651318 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880309b0 con 0x7ff8a015f5a0 2024-02-02T13:35:33.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.293+0000 7ff89ea00640 1 -- 172.21.15.91:0/4089651318 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888030d40 con 0x7ff8a015f5a0 2024-02-02T13:35:33.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 -- 172.21.15.91:0/4089651318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a015f5a0 msgr2=0x7ff8a00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 --2- 172.21.15.91:0/4089651318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a015f5a0 0x7ff8a00b0700 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff888009870 tx=0x7ff88802b990 comp rx=0 tx=0).stop 2024-02-02T13:35:33.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 -- 172.21.15.91:0/4089651318 shutdown_connections 2024-02-02T13:35:33.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 --2- 172.21.15.91:0/4089651318 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a015f5a0 0x7ff8a00b0700 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 -- 172.21.15.91:0/4089651318 >> 172.21.15.91:0/4089651318 conn(0x7ff8a00cec80 msgr2=0x7ff8a00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:33.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 -- 172.21.15.91:0/4089651318 shutdown_connections 2024-02-02T13:35:33.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.294+0000 7ff8a6400640 1 -- 172.21.15.91:0/4089651318 wait complete. 2024-02-02T13:35:33.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.295+0000 7ff8a6400640 1 Processor -- start 2024-02-02T13:35:33.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.296+0000 7ff8a6400640 1 -- start start 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.296+0000 7ff8a6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8a00b0c40 0x7ff8a0150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff8a6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff8a6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8a015f5a0 0x7ff8a0153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff8a6400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c6b20 con 0x7ff8a0151f90 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff8a6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c6280 con 0x7ff8a00b0c40 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff8a6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c6560 con 0x7ff8a015f5a0 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41040/0 (socket says 172.21.15.91:41040) 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 -- 172.21.15.91:0/1102722202 learned_addr learned my addr 172.21.15.91:0/1102722202 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff89fe00640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8a015f5a0 0x7ff8a0153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff89f400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8a00b0c40 0x7ff8a0150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 -- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8a015f5a0 msgr2=0x7ff8a0153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8a015f5a0 0x7ff8a0153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 -- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8a00b0c40 msgr2=0x7ff8a0150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8a00b0c40 0x7ff8a0150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.297+0000 7ff897e00640 1 -- 172.21.15.91:0/1102722202 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880094d0 con 0x7ff8a0151f90 2024-02-02T13:35:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff897e00640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff8880090b0 tx=0x7ff888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88803e040 con 0x7ff8a0151f90 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff888030800 con 0x7ff8a0151f90 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a00c6840 con 0x7ff8a0151f90 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888044630 con 0x7ff8a0151f90 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.298+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a0151170 con 0x7ff8a0151f90 2024-02-02T13:35:33.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.299+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a00c40e0 con 0x7ff8a0151f90 2024-02-02T13:35:33.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.300+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff88802e070 con 0x7ff8a0151f90 2024-02-02T13:35:33.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.302+0000 7ff897400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 0x7ff87806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.302+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8880aaf90 con 0x7ff8a0151f90 2024-02-02T13:35:33.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.302+0000 7ff89f400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 0x7ff87806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.303+0000 7ff89f400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 0x7ff87806c8f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff890006fd0 tx=0x7ff890008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.306+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8880ddca0 con 0x7ff8a0151f90 2024-02-02T13:35:33.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.479+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8a0066cf0 con 0x7ff8a0151f90 2024-02-02T13:35:33.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.480+0000 7ff897400640 1 -- 172.21.15.91:0/1102722202 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16125 (secure 0 0 0) 0x7ff8a00c40e0 con 0x7ff8a0151f90 2024-02-02T13:35:33.482 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:33.482 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":177,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:35:32.636903+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:33.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 msgr2=0x7ff87806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 0x7ff87806c8f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff890006fd0 tx=0x7ff890008040 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 msgr2=0x7ff8a01503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff8880090b0 tx=0x7ff888004730 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 shutdown_connections 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff87806a430 0x7ff87806c8f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8a015f5a0 0x7ff8a0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8a0151f90 0x7ff8a01503c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 --2- 172.21.15.91:0/1102722202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8a00b0c40 0x7ff8a0150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 >> 172.21.15.91:0/1102722202 conn(0x7ff8a00cec80 msgr2=0x7ff8a00c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 shutdown_connections 2024-02-02T13:35:33.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.483+0000 7ff8a6400640 1 -- 172.21.15.91:0/1102722202 wait complete. 2024-02-02T13:35:33.502 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:35:33.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.710+0000 7f8628a00640 1 Processor -- start 2024-02-02T13:35:33.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.710+0000 7f8628a00640 1 -- start start 2024-02-02T13:35:33.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.711+0000 7f8628a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f862415d140 0x7f86240cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.711+0000 7f8628a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86240d1770 0x7f862406a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.711+0000 7f8628a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86240ce070 con 0x7f862406aaa0 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.711+0000 7f8628a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86240cf5d0 con 0x7f862415d140 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.711+0000 7f8628a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862406af20 con 0x7f86240d1770 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f862415d140 0x7f86240cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f862415d140 0x7f86240cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41044/0 (socket says 172.21.15.91:41044) 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 -- 172.21.15.91:0/79694502 learned_addr learned my addr 172.21.15.91:0/79694502 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8623200640 1 --1- 172.21.15.91:0/79694502 >> v1:172.21.15.91:6789/0 conn(0x7f862406aaa0 0x7f86240cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46710/0 (socket says 172.21.15.91:46710) 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f861be00640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86240d1770 0x7f862406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 -- 172.21.15.91:0/79694502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86240d1770 msgr2=0x7f862406a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86240d1770 0x7f862406a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 -- 172.21.15.91:0/79694502 >> v1:172.21.15.91:6789/0 conn(0x7f862406aaa0 legacy=0x7f86240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.712+0000 7f8622800640 1 -- 172.21.15.91:0/79694502 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86240cefc0 con 0x7f862415d140 2024-02-02T13:35:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.713+0000 7f8622800640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f862415d140 0x7f86240cb2a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8614009870 tx=0x7f861402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386715e20dffb67b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8621e00640 1 -- 172.21.15.91:0/79694502 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614031640 con 0x7f862415d140 2024-02-02T13:35:33.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8621e00640 1 -- 172.21.15.91:0/79694502 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86140317a0 con 0x7f862415d140 2024-02-02T13:35:33.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8621e00640 1 -- 172.21.15.91:0/79694502 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614031b30 con 0x7f862415d140 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 -- 172.21.15.91:0/79694502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f862415d140 msgr2=0x7f86240cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f862415d140 0x7f86240cb2a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8614009870 tx=0x7f861402b8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 -- 172.21.15.91:0/79694502 shutdown_connections 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86240d1770 0x7f862406a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 --2- 172.21.15.91:0/79694502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f862415d140 0x7f86240cb2a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 -- 172.21.15.91:0/79694502 >> 172.21.15.91:0/79694502 conn(0x7f86240d0c50 msgr2=0x7f86240d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 -- 172.21.15.91:0/79694502 shutdown_connections 2024-02-02T13:35:33.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.714+0000 7f8628a00640 1 -- 172.21.15.91:0/79694502 wait complete. 2024-02-02T13:35:33.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.716+0000 7f8628a00640 1 Processor -- start 2024-02-02T13:35:33.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.716+0000 7f8628a00640 1 -- start start 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862406aaa0 0x7f862415a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f862415d140 0x7f86240aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86240b0f80 con 0x7f86240d1770 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86240b0300 con 0x7f862415d140 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f8628a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86240b05e0 con 0x7f862406aaa0 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f861be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.717+0000 7f861be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41058/0 (socket says 172.21.15.91:41058) 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 -- 172.21.15.91:0/2753950669 learned_addr learned my addr 172.21.15.91:0/2753950669 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 -- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862406aaa0 msgr2=0x7f862415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f8622800640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862406aaa0 0x7f862415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f8623200640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f862415d140 0x7f86240aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862406aaa0 0x7f862415a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 -- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f862415d140 msgr2=0x7f86240aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f862415d140 0x7f86240aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 -- 172.21.15.91:0/2753950669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86140094d0 con 0x7f86240d1770 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861be00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d5a0 tx=0x7f860c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.718+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c004240 con 0x7f86240d1770 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.719+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c0043a0 con 0x7f86240d1770 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.719+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86240b08c0 con 0x7f86240d1770 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.719+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c003830 con 0x7f86240d1770 2024-02-02T13:35:33.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.719+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86240c8b50 con 0x7f86240d1770 2024-02-02T13:35:33.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.720+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86240b10c0 con 0x7f86240d1770 2024-02-02T13:35:33.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.721+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f860c003990 con 0x7f86240d1770 2024-02-02T13:35:33.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.721+0000 7f861aa00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 0x7f85fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:33.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.722+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f860c0883c0 con 0x7f86240d1770 2024-02-02T13:35:33.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.722+0000 7f8622800640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 0x7f85fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:33.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.722+0000 7f8622800640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 0x7f85fc06c8f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8614036000 tx=0x7f861403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:33.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.726+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f860c053a50 con 0x7f86240d1770 2024-02-02T13:35:33.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:33.955+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f862404dff0 con 0x7f86240d1770 2024-02-02T13:35:34.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.668+0000 7f861aa00640 1 -- 172.21.15.91:0/2753950669 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f860c057070 con 0x7f86240d1770 2024-02-02T13:35:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.671+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 msgr2=0x7f85fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.671+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 0x7f85fc06c8f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8614036000 tx=0x7f861403c040 comp rx=0 tx=0).stop 2024-02-02T13:35:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.671+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 msgr2=0x7f86240ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.671+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d5a0 tx=0x7f860c00da60 comp rx=0 tx=0).stop 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 shutdown_connections 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f85fc06a430 0x7f85fc06c8f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f862415d140 0x7f86240aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86240d1770 0x7f86240ae8f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 --2- 172.21.15.91:0/2753950669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862406aaa0 0x7f862415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 >> 172.21.15.91:0/2753950669 conn(0x7f86240d0c50 msgr2=0x7f86240d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 shutdown_connections 2024-02-02T13:35:34.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.672+0000 7f8628a00640 1 -- 172.21.15.91:0/2753950669 wait complete. 2024-02-02T13:35:34.693 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:35:34.693 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:35:34.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.903+0000 7fbb9d400640 1 Processor -- start 2024-02-02T13:35:34.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.903+0000 7fbb9d400640 1 -- start start 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.904+0000 7fbb9d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 0x7fbb980c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.904+0000 7fbb9d400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbb98070ea0 0x7fbb980b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb9d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 0x7fbb9815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb9d400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980b1060 con 0x7fbb98070ea0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb9d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980cca70 con 0x7fbb980b0bd0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb9d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980cc770 con 0x7fbb9815a1e0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbb98070ea0 0x7fbb980b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbb98070ea0 0x7fbb980b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41068/0 (socket says 172.21.15.91:41068) 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 -- 172.21.15.91:0/2607083344 learned_addr learned my addr 172.21.15.91:0/2607083344 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb96800640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 0x7fbb980c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb97200640 1 --2- 172.21.15.91:0/2607083344 >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 0x7fbb9815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 -- 172.21.15.91:0/2607083344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 msgr2=0x7fbb980c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 0x7fbb980c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 -- 172.21.15.91:0/2607083344 >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 msgr2=0x7fbb9815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:34.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 --2- 172.21.15.91:0/2607083344 >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 0x7fbb9815dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb8fe00640 1 -- 172.21.15.91:0/2607083344 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980ca570 con 0x7fbb98070ea0 2024-02-02T13:35:34.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb97200640 1 --2- 172.21.15.91:0/2607083344 >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 0x7fbb9815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:34.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.905+0000 7fbb96800640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 0x7fbb980c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.906+0000 7fbb8fe00640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb98070ea0 0x7fbb980b0690 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fbb980cfb10 tx=0x7fbb8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a33c0312e505658 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.907+0000 7fbb95e00640 1 -- 172.21.15.91:0/2607083344 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80031870 con 0x7fbb98070ea0 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.907+0000 7fbb95e00640 1 -- 172.21.15.91:0/2607083344 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800319d0 con 0x7fbb98070ea0 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.907+0000 7fbb95e00640 1 -- 172.21.15.91:0/2607083344 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80031d60 con 0x7fbb98070ea0 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.907+0000 7fbb9d400640 1 -- 172.21.15.91:0/2607083344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb98070ea0 msgr2=0x7fbb980b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.907+0000 7fbb9d400640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb98070ea0 0x7fbb980b0690 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fbb980cfb10 tx=0x7fbb8002b910 comp rx=0 tx=0).stop 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 -- 172.21.15.91:0/2607083344 shutdown_connections 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 --2- 172.21.15.91:0/2607083344 >> v2:172.21.15.113:3300/0 conn(0x7fbb980b0bd0 0x7fbb9815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb98070ea0 0x7fbb980b0690 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 --2- 172.21.15.91:0/2607083344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb9815a1e0 0x7fbb980c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 -- 172.21.15.91:0/2607083344 >> 172.21.15.91:0/2607083344 conn(0x7fbb980ceb90 msgr2=0x7fbb980cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:34.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 -- 172.21.15.91:0/2607083344 shutdown_connections 2024-02-02T13:35:34.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.908+0000 7fbb9d400640 1 -- 172.21.15.91:0/2607083344 wait complete. 2024-02-02T13:35:34.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.910+0000 7fbb9d400640 1 Processor -- start 2024-02-02T13:35:34.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.910+0000 7fbb9d400640 1 -- start start 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb98070ea0 0x7fbb98146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb980b0bd0 0x7fbb981555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98156cb0 con 0x7fbb9815a1e0 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981560a0 con 0x7fbb980b0bd0 2024-02-02T13:35:34.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb9d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98156380 con 0x7fbb98070ea0 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb97200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb97200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41084/0 (socket says 172.21.15.91:41084) 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.911+0000 7fbb97200640 1 -- 172.21.15.91:0/91782360 learned_addr learned my addr 172.21.15.91:0/91782360 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb96800640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb98070ea0 0x7fbb98146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb8fe00640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb980b0bd0 0x7fbb981555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 -- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb98070ea0 msgr2=0x7fbb98146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb98070ea0 0x7fbb98146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 -- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb980b0bd0 msgr2=0x7fbb981555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb980b0bd0 0x7fbb981555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 -- 172.21.15.91:0/91782360 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800094d0 con 0x7fbb9815a1e0 2024-02-02T13:35:34.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb97200640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cef0 tx=0x7fbb9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000d840 con 0x7fbb9815a1e0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000d9c0 con 0x7fbb9815a1e0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb98156660 con 0x7fbb9815a1e0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.912+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000dd70 con 0x7fbb9815a1e0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.913+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb98147460 con 0x7fbb9815a1e0 2024-02-02T13:35:34.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.913+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98156e20 con 0x7fbb9815a1e0 2024-02-02T13:35:34.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.914+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb9001e3b0 con 0x7fbb9815a1e0 2024-02-02T13:35:34.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.915+0000 7fbb8ea00640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 0x7fbb7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:34.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.915+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb90089500 con 0x7fbb9815a1e0 2024-02-02T13:35:34.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.915+0000 7fbb96800640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 0x7fbb7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:34.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.916+0000 7fbb96800640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 0x7fbb7006c8f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b7c0 tx=0x7fbb88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:34.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:34.920+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbb90054a90 con 0x7fbb9815a1e0 2024-02-02T13:35:35.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.145+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbb98066cf0 con 0x7fbb9815a1e0 2024-02-02T13:35:35.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.671+0000 7fbb8ea00640 1 -- 172.21.15.91:0/91782360 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb900580b0 con 0x7fbb9815a1e0 2024-02-02T13:35:35.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.674+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 msgr2=0x7fbb7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.674+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 0x7fbb7006c8f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b7c0 tx=0x7fbb88008040 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.674+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 msgr2=0x7fbb980b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.674+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cef0 tx=0x7fbb9000a710 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 shutdown_connections 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbb7006a430 0x7fbb7006c8f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbb9815a1e0 0x7fbb980b17f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb980b0bd0 0x7fbb981555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 --2- 172.21.15.91:0/91782360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb98070ea0 0x7fbb98146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 >> 172.21.15.91:0/91782360 conn(0x7fbb980ceb90 msgr2=0x7fbb9815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 shutdown_connections 2024-02-02T13:35:35.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.675+0000 7fbb9d400640 1 -- 172.21.15.91:0/91782360 wait complete. 2024-02-02T13:35:35.699 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:35:35.699 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:35.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.906+0000 7f8af0a00640 1 Processor -- start 2024-02-02T13:35:35.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.906+0000 7f8af0a00640 1 -- start start 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.907+0000 7f8af0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b0e10 0x7f8ae80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.907+0000 7f8af0a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80cc7c0 con 0x7f8ae80ce130 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.907+0000 7f8af0a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80ce930 con 0x7f8ae815a010 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.907+0000 7f8af0a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80cd640 con 0x7f8ae80b0e10 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aed600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8ae80ce130 0x7f8ae80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46712/0 (socket says 172.21.15.91:46712) 2024-02-02T13:35:35.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aed600640 1 -- 172.21.15.91:0/3707703464 learned_addr learned my addr 172.21.15.91:0/3707703464 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aeea00640 1 --2- 172.21.15.91:0/3707703464 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b0e10 0x7f8ae80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237732141 0 0) 0x7f8ae80cc7c0 con 0x7f8ae80ce130 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac8005730 con 0x7f8ae80ce130 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743464264 0 0) 0x7f8ac8005730 con 0x7f8ae80ce130 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b0e10 msgr2=0x7f8ae80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.908+0000 7f8aecc00640 1 --2- 172.21.15.91:0/3707703464 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b0e10 0x7f8ae80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 >> v1:172.21.15.113:6789/0 conn(0x7f8ae815a010 legacy=0x7f8ae80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae806c890 con 0x7f8ae80ce130 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8ad0001670 con 0x7f8ae80ce130 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 >> v1:172.21.15.91:6789/0 conn(0x7f8ae80ce130 legacy=0x7f8ae80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac8005b80 0x7f8ac8005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ac8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac8007390 0x7f8ac800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80cd640 con 0x7f8ac80069d0 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac800e2c0 con 0x7f8ac8005b80 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac800e480 con 0x7f8ac8007390 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ac8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aee000640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac8005b80 0x7f8ac8005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 -- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac8007390 msgr2=0x7f8ac800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac8007390 0x7f8ac800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 -- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac8005b80 msgr2=0x7f8ac8005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:35.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac8005b80 0x7f8ac8005f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aed600640 1 -- 172.21.15.91:0/3707703464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8005730 con 0x7f8ac80069d0 2024-02-02T13:35:35.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.909+0000 7f8aeea00640 1 -- 172.21.15.91:0/3707703464 reap_dead start 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.910+0000 7f8aed600640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ac8006e50 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c210 tx=0x7f8ac8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5114125a3532b5c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0005ad0 con 0x7f8ac80069d0 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0005c50 con 0x7f8ac80069d0 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8aecc00640 1 -- 172.21.15.91:0/3707703464 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad002c810 con 0x7f8ac80069d0 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 -- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 msgr2=0x7f8ac8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ac8006e50 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c210 tx=0x7f8ac8007a00 comp rx=0 tx=0).stop 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 -- 172.21.15.91:0/3707703464 shutdown_connections 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3707703464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ac8006e50 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 -- 172.21.15.91:0/3707703464 >> 172.21.15.91:0/3707703464 conn(0x7f8ae80cec80 msgr2=0x7f8ae80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 -- 172.21.15.91:0/3707703464 shutdown_connections 2024-02-02T13:35:35.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.911+0000 7f8af0a00640 1 -- 172.21.15.91:0/3707703464 wait complete. 2024-02-02T13:35:35.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.913+0000 7f8af0a00640 1 Processor -- start 2024-02-02T13:35:35.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.913+0000 7f8af0a00640 1 -- start start 2024-02-02T13:35:35.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae80c7990 0x7f8ae80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80b3dc0 0x7f8ae81f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80b4240 con 0x7f8ac80069d0 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80c7500 con 0x7f8ae80b3dc0 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8af0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81f3780 con 0x7f8ae80c7990 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8aee000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8aee000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41104/0 (socket says 172.21.15.91:41104) 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.914+0000 7f8aee000640 1 -- 172.21.15.91:0/3861971470 learned_addr learned my addr 172.21.15.91:0/3861971470 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 -- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae80c7990 msgr2=0x7f8ae80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae80c7990 0x7f8ae80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 -- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80b3dc0 msgr2=0x7f8ae81f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80b3dc0 0x7f8ae81f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 -- 172.21.15.91:0/3861971470 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80c6eb0 con 0x7f8ac80069d0 2024-02-02T13:35:35.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8aee000640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d4e0 tx=0x7f8ae000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0012800 con 0x7f8ac80069d0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0012960 con 0x7f8ac80069d0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.915+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0012d50 con 0x7f8ac80069d0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.916+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae80c7190 con 0x7f8ac80069d0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.916+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae80c4dc0 con 0x7f8ac80069d0 2024-02-02T13:35:35.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.917+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae80c6d10 con 0x7f8ac80069d0 2024-02-02T13:35:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.918+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ae00243a0 con 0x7f8ac80069d0 2024-02-02T13:35:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.919+0000 7f8ae6000640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 0x7f8ac006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.919+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ae0088cd0 con 0x7f8ac80069d0 2024-02-02T13:35:35.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.919+0000 7f8aed600640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 0x7f8ac006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:35.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.920+0000 7f8aed600640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 0x7f8ac006c8f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad0039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:35.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:35.924+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8ae80c6d10 con 0x7f8ac80069d0 2024-02-02T13:35:36.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.151+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ae80c6d10 con 0x7f8ac80069d0 2024-02-02T13:35:36.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.153+0000 7f8ae6000640 1 -- 172.21.15.91:0/3861971470 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7f8ae80c6d10 con 0x7f8ac80069d0 2024-02-02T13:35:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:36.154 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:35:36.155 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:35:36.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 msgr2=0x7f8ac006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 0x7f8ac006c8f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad0039040 comp rx=0 tx=0).stop 2024-02-02T13:35:36.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 msgr2=0x7f8ae80c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d4e0 tx=0x7f8ae000d9a0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 shutdown_connections 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8ac006a430 0x7f8ac006c8f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae80c7990 0x7f8ae80b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80b3dc0 0x7f8ae81f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 --2- 172.21.15.91:0/3861971470 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ac80069d0 0x7f8ae80c46a0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.155+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 >> 172.21.15.91:0/3861971470 conn(0x7f8ae80cec80 msgr2=0x7f8ae815aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.156+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 shutdown_connections 2024-02-02T13:35:36.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.156+0000 7f8af0a00640 1 -- 172.21.15.91:0/3861971470 wait complete. 2024-02-02T13:35:36.174 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:35:36.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.384+0000 7f3277e00640 1 Processor -- start 2024-02-02T13:35:36.386 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.385+0000 7f3277e00640 1 -- start start 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3277e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f327015a010 0x7f32700c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3277e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32700ce130 0x7f32700ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3277e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32700cc7c0 con 0x7f32700cd470 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3277e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32700ce930 con 0x7f327015a010 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3277e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327006c890 con 0x7f32700ce130 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3275400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f327015a010 0x7f32700c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3275400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f327015a010 0x7f32700c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41108/0 (socket says 172.21.15.91:41108) 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3275400640 1 -- 172.21.15.91:0/4013856783 learned_addr learned my addr 172.21.15.91:0/4013856783 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.386+0000 7f3274a00640 1 --2- 172.21.15.91:0/4013856783 >> v2:172.21.15.113:3300/0 conn(0x7f32700ce130 0x7f32700ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.387+0000 7f3275400640 1 -- 172.21.15.91:0/4013856783 >> v2:172.21.15.113:3300/0 conn(0x7f32700ce130 msgr2=0x7f32700ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.387+0000 7f3275400640 1 --2- 172.21.15.91:0/4013856783 >> v2:172.21.15.113:3300/0 conn(0x7f32700ce130 0x7f32700ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.387+0000 7f3275400640 1 -- 172.21.15.91:0/4013856783 >> v1:172.21.15.113:6789/0 conn(0x7f32700cd470 legacy=0x7f32700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.387+0000 7f3275400640 1 -- 172.21.15.91:0/4013856783 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327006cb70 con 0x7f327015a010 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.388+0000 7f3275400640 1 --2- 172.21.15.91:0/4013856783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32700c9670 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f32700b1870 tx=0x7f326002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bacc0ffe6bd6a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.388+0000 7f326fe00640 1 -- 172.21.15.91:0/4013856783 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600317e0 con 0x7f327015a010 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.388+0000 7f326fe00640 1 -- 172.21.15.91:0/4013856783 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3260031940 con 0x7f327015a010 2024-02-02T13:35:36.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.388+0000 7f326fe00640 1 -- 172.21.15.91:0/4013856783 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260031cd0 con 0x7f327015a010 2024-02-02T13:35:36.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 -- 172.21.15.91:0/4013856783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 msgr2=0x7f32700c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 --2- 172.21.15.91:0/4013856783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32700c9670 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f32700b1870 tx=0x7f326002b880 comp rx=0 tx=0).stop 2024-02-02T13:35:36.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 -- 172.21.15.91:0/4013856783 shutdown_connections 2024-02-02T13:35:36.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 --2- 172.21.15.91:0/4013856783 >> v2:172.21.15.113:3300/0 conn(0x7f32700ce130 0x7f32700ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 --2- 172.21.15.91:0/4013856783 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32700c9670 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 -- 172.21.15.91:0/4013856783 >> 172.21.15.91:0/4013856783 conn(0x7f32700cec80 msgr2=0x7f32700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:36.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 -- 172.21.15.91:0/4013856783 shutdown_connections 2024-02-02T13:35:36.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.389+0000 7f3277e00640 1 -- 172.21.15.91:0/4013856783 wait complete. 2024-02-02T13:35:36.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.391+0000 7f3277e00640 1 Processor -- start 2024-02-02T13:35:36.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.391+0000 7f3277e00640 1 -- start start 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 0x7f32701f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 0x7f32701f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327015d5d0 con 0x7f327015a010 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c2630 con 0x7f32700ce130 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3277e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3270150120 con 0x7f327015c770 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3274a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3274a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41112/0 (socket says 172.21.15.91:41112) 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.392+0000 7f3274a00640 1 -- 172.21.15.91:0/922032282 learned_addr learned my addr 172.21.15.91:0/922032282 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3275400640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 0x7f32701f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3275e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 0x7f32701f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 -- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 msgr2=0x7f32701f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 0x7f32701f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 -- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 msgr2=0x7f32701f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 0x7f32701f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 -- 172.21.15.91:0/922032282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600094d0 con 0x7f327015a010 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3275e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 0x7f32701f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3275400640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 0x7f32701f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3274a00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f325800d800 tx=0x7f325800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800b6c0 con 0x7f327015a010 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325800b820 con 0x7f327015a010 2024-02-02T13:35:36.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800bbb0 con 0x7f327015a010 2024-02-02T13:35:36.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.393+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327015cd30 con 0x7f327015a010 2024-02-02T13:35:36.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.394+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327015d2f0 con 0x7f327015a010 2024-02-02T13:35:36.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.395+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700c2390 con 0x7f327015a010 2024-02-02T13:35:36.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.396+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f325800bd10 con 0x7f327015a010 2024-02-02T13:35:36.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.396+0000 7f326d600640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 0x7f324806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.396+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3258088e50 con 0x7f327015a010 2024-02-02T13:35:36.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.397+0000 7f3275400640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 0x7f324806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.397+0000 7f3275400640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 0x7f324806c8f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3260036000 tx=0x7f326003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.401+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f32700c2390 con 0x7f327015a010 2024-02-02T13:35:36.591 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.590+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32700c2390 con 0x7f327015a010 2024-02-02T13:35:36.594 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.593+0000 7f326d600640 1 -- 172.21.15.91:0/922032282 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16112 (secure 0 0 0) 0x7f32700c2390 con 0x7f327015a010 2024-02-02T13:35:36.595 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:36.595 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":178,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:35:33.640553+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 msgr2=0x7f324806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 0x7f324806c8f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3260036000 tx=0x7f326003d040 comp rx=0 tx=0).stop 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 msgr2=0x7f32701f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f325800d800 tx=0x7f325800dcc0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 shutdown_connections 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f324806a430 0x7f324806c8f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f327015c770 0x7f32701f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f327015a010 0x7f32701f5f10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 --2- 172.21.15.91:0/922032282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32700ce130 0x7f32701f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.595+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 >> 172.21.15.91:0/922032282 conn(0x7f32700cec80 msgr2=0x7f327006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:36.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.596+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 shutdown_connections 2024-02-02T13:35:36.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.596+0000 7f3277e00640 1 -- 172.21.15.91:0/922032282 wait complete. 2024-02-02T13:35:36.617 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:35:36.617 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:35:36.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.824+0000 7fcd35a00640 1 Processor -- start 2024-02-02T13:35:36.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.825+0000 7fcd35a00640 1 -- start start 2024-02-02T13:35:36.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.826+0000 7fcd35a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd3006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.826+0000 7fcd35a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd300c9450 0x7fcd300c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.826+0000 7fcd35a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300ce3b0 con 0x7fcd300add80 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.826+0000 7fcd35a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300adbd0 con 0x7fcd300c9450 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.826+0000 7fcd35a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300c59d0 con 0x7fcd3015d3a0 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd35000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd300c9450 0x7fcd300c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd3006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd3006b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:48478/0 (socket says 172.21.15.91:48478) 2024-02-02T13:35:36.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd2fe00640 1 -- 172.21.15.91:0/2963588186 learned_addr learned my addr 172.21.15.91:0/2963588186 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:36.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd35000640 1 -- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 msgr2=0x7fcd3006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:36.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd35000640 1 --2- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd3006b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd35000640 1 -- 172.21.15.91:0/2963588186 >> v1:172.21.15.113:6789/0 conn(0x7fcd300add80 legacy=0x7fcd300706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.827+0000 7fcd35000640 1 -- 172.21.15.91:0/2963588186 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300ce8e0 con 0x7fcd300c9450 2024-02-02T13:35:36.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.829+0000 7fcd35000640 1 --2- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd300c98e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10559b018710e557 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.829+0000 7fcd2f400640 1 -- 172.21.15.91:0/2963588186 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280316f0 con 0x7fcd300c9450 2024-02-02T13:35:36.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.829+0000 7fcd2f400640 1 -- 172.21.15.91:0/2963588186 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28031850 con 0x7fcd300c9450 2024-02-02T13:35:36.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.829+0000 7fcd2f400640 1 -- 172.21.15.91:0/2963588186 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28031be0 con 0x7fcd300c9450 2024-02-02T13:35:36.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 -- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 msgr2=0x7fcd300c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 --2- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd300c98e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802b790 comp rx=0 tx=0).stop 2024-02-02T13:35:36.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 -- 172.21.15.91:0/2963588186 shutdown_connections 2024-02-02T13:35:36.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 --2- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd300c98e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 --2- 172.21.15.91:0/2963588186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd3006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 -- 172.21.15.91:0/2963588186 >> 172.21.15.91:0/2963588186 conn(0x7fcd300cea90 msgr2=0x7fcd300ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:36.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 -- 172.21.15.91:0/2963588186 shutdown_connections 2024-02-02T13:35:36.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.830+0000 7fcd35a00640 1 -- 172.21.15.91:0/2963588186 wait complete. 2024-02-02T13:35:36.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.832+0000 7fcd35a00640 1 Processor -- start 2024-02-02T13:35:36.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.832+0000 7fcd35a00640 1 -- start start 2024-02-02T13:35:36.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.835 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd301e3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd300b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300c5490 con 0x7fcd300add80 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30157550 con 0x7fcd300c9450 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300af860 con 0x7fcd3015d3a0 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41114/0 (socket says 172.21.15.91:41114) 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd2fe00640 1 -- 172.21.15.91:0/3392746792 learned_addr learned my addr 172.21.15.91:0/3392746792 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd27200640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd301e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd35000640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd300b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd2fe00640 1 -- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 msgr2=0x7fcd300b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.833+0000 7fcd2fe00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd300b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2fe00640 1 -- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 msgr2=0x7fcd301e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2fe00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd301e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:36.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2fe00640 1 -- 172.21.15.91:0/3392746792 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd28009400 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd27200640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd301e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2fe00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000efc0 tx=0x7fcd2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20009b60 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2000f040 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200092e0 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300afb40 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.834+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300b0100 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.835+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3004dff0 con 0x7fcd300add80 2024-02-02T13:35:36.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.836+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd20014480 con 0x7fcd300add80 2024-02-02T13:35:36.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.837+0000 7fcd2cc00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 0x7fcd0806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:36.838 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.837+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd200890b0 con 0x7fcd300add80 2024-02-02T13:35:36.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.837+0000 7fcd27200640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 0x7fcd0806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:36.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.838+0000 7fcd27200640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 0x7fcd0806c920 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcd30157190 tx=0x7fcd18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:36.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:36.841+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcd20054600 con 0x7fcd300add80 2024-02-02T13:35:37.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.024+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd3015d780 con 0x7fcd0806a460 2024-02-02T13:35:37.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.036+0000 7fcd2cc00640 1 -- 172.21.15.91:0/3392746792 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283007 (secure 0 0 0) 0x7fcd180013e0 con 0x7fcd0806a460 2024-02-02T13:35:37.041 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:37.043 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:35:37.044 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 msgr2=0x7fcd0806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 0x7fcd0806c920 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcd30157190 tx=0x7fcd18009040 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 msgr2=0x7fcd300c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000efc0 tx=0x7fcd2000c490 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 shutdown_connections 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcd0806a460 0x7fcd0806c920 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd3015d3a0 0x7fcd300b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd300c9450 0x7fcd301e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 --2- 172.21.15.91:0/3392746792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd300add80 0x7fcd300c4f50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 >> 172.21.15.91:0/3392746792 conn(0x7fcd300cea90 msgr2=0x7fcd30159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 shutdown_connections 2024-02-02T13:35:37.045 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.042+0000 7fcd35a00640 1 -- 172.21.15.91:0/3392746792 wait complete. 2024-02-02T13:35:37.062 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-02-02T13:35:36.459119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8952,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553547,"statfs":{"total":1600000000,"available":1590832408,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.313654"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737783+0000","last_change":"2024-02-02T13:35:32.656558+0000","last_active":"2024-02-02T13:35:33.737783+0000","last_peered":"2024-02-02T13:35:33.737783+0000","last_clean":"2024-02-02T13:35:33.737783+0000","last_became_active":"2024-02-02T13:35:32.656311+0000","last_became_peered":"2024-02-02T13:35:32.656311+0000","last_unstale":"2024-02-02T13:35:33.737783+0000","last_undegraded":"2024-02-02T13:35:33.737783+0000","last_fullsized":"2024-02-02T13:35:33.737783+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:44.636889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646372+0000","last_change":"2024-02-02T13:35:32.653078+0000","last_active":"2024-02-02T13:35:33.646372+0000","last_peered":"2024-02-02T13:35:33.646372+0000","last_clean":"2024-02-02T13:35:33.646372+0000","last_became_active":"2024-02-02T13:35:32.652847+0000","last_became_peered":"2024-02-02T13:35:32.652847+0000","last_unstale":"2024-02-02T13:35:33.646372+0000","last_undegraded":"2024-02-02T13:35:33.646372+0000","last_fullsized":"2024-02-02T13:35:33.646372+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:42.753953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740414+0000","last_change":"2024-02-02T13:35:32.671785+0000","last_active":"2024-02-02T13:35:33.740414+0000","last_peered":"2024-02-02T13:35:33.740414+0000","last_clean":"2024-02-02T13:35:33.740414+0000","last_became_active":"2024-02-02T13:35:32.669976+0000","last_became_peered":"2024-02-02T13:35:32.669976+0000","last_unstale":"2024-02-02T13:35:33.740414+0000","last_undegraded":"2024-02-02T13:35:33.740414+0000","last_fullsized":"2024-02-02T13:35:33.740414+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:13.088957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739256+0000","last_change":"2024-02-02T13:35:32.676634+0000","last_active":"2024-02-02T13:35:33.739256+0000","last_peered":"2024-02-02T13:35:33.739256+0000","last_clean":"2024-02-02T13:35:33.739256+0000","last_became_active":"2024-02-02T13:35:32.674092+0000","last_became_peered":"2024-02-02T13:35:32.674092+0000","last_unstale":"2024-02-02T13:35:33.739256+0000","last_undegraded":"2024-02-02T13:35:33.739256+0000","last_fullsized":"2024-02-02T13:35:33.739256+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:36:21.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741108+0000","last_change":"2024-02-02T13:35:32.672181+0000","last_active":"2024-02-02T13:35:33.741108+0000","last_peered":"2024-02-02T13:35:33.741108+0000","last_clean":"2024-02-02T13:35:33.741108+0000","last_became_active":"2024-02-02T13:35:32.671797+0000","last_became_peered":"2024-02-02T13:35:32.671797+0000","last_unstale":"2024-02-02T13:35:33.741108+0000","last_undegraded":"2024-02-02T13:35:33.741108+0000","last_fullsized":"2024-02-02T13:35:33.741108+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:59.421530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737943+0000","last_change":"2024-02-02T13:35:32.666665+0000","last_active":"2024-02-02T13:35:33.737943+0000","last_peered":"2024-02-02T13:35:33.737943+0000","last_clean":"2024-02-02T13:35:33.737943+0000","last_became_active":"2024-02-02T13:35:32.666375+0000","last_became_peered":"2024-02-02T13:35:32.666375+0000","last_unstale":"2024-02-02T13:35:33.737943+0000","last_undegraded":"2024-02-02T13:35:33.737943+0000","last_fullsized":"2024-02-02T13:35:33.737943+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:23:41.531817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650127+0000","last_change":"2024-02-02T13:35:32.673295+0000","last_active":"2024-02-02T13:35:33.650127+0000","last_peered":"2024-02-02T13:35:33.650127+0000","last_clean":"2024-02-02T13:35:33.650127+0000","last_became_active":"2024-02-02T13:35:32.673031+0000","last_became_peered":"2024-02-02T13:35:32.673031+0000","last_unstale":"2024-02-02T13:35:33.650127+0000","last_undegraded":"2024-02-02T13:35:33.650127+0000","last_fullsized":"2024-02-02T13:35:33.650127+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:54.540869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737455+0000","last_change":"2024-02-02T13:35:32.662799+0000","last_active":"2024-02-02T13:35:33.737455+0000","last_peered":"2024-02-02T13:35:33.737455+0000","last_clean":"2024-02-02T13:35:33.737455+0000","last_became_active":"2024-02-02T13:35:32.662582+0000","last_became_peered":"2024-02-02T13:35:32.662582+0000","last_unstale":"2024-02-02T13:35:33.737455+0000","last_undegraded":"2024-02-02T13:35:33.737455+0000","last_fullsized":"2024-02-02T13:35:33.737455+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:46:45.138936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646332+0000","last_change":"2024-02-02T13:35:32.649195+0000","last_active":"2024-02-02T13:35:33.646332+0000","last_peered":"2024-02-02T13:35:33.646332+0000","last_clean":"2024-02-02T13:35:33.646332+0000","last_became_active":"2024-02-02T13:35:32.648951+0000","last_became_peered":"2024-02-02T13:35:32.648951+0000","last_unstale":"2024-02-02T13:35:33.646332+0000","last_undegraded":"2024-02-02T13:35:33.646332+0000","last_fullsized":"2024-02-02T13:35:33.646332+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:26:13.969913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743034+0000","last_change":"2024-02-02T13:35:32.675835+0000","last_active":"2024-02-02T13:35:33.743034+0000","last_peered":"2024-02-02T13:35:33.743034+0000","last_clean":"2024-02-02T13:35:33.743034+0000","last_became_active":"2024-02-02T13:35:32.675606+0000","last_became_peered":"2024-02-02T13:35:32.675606+0000","last_unstale":"2024-02-02T13:35:33.743034+0000","last_undegraded":"2024-02-02T13:35:33.743034+0000","last_fullsized":"2024-02-02T13:35:33.743034+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:45.527941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737442+0000","last_change":"2024-02-02T13:35:32.660393+0000","last_active":"2024-02-02T13:35:33.737442+0000","last_peered":"2024-02-02T13:35:33.737442+0000","last_clean":"2024-02-02T13:35:33.737442+0000","last_became_active":"2024-02-02T13:35:32.659471+0000","last_became_peered":"2024-02-02T13:35:32.659471+0000","last_unstale":"2024-02-02T13:35:33.737442+0000","last_undegraded":"2024-02-02T13:35:33.737442+0000","last_fullsized":"2024-02-02T13:35:33.737442+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:51:59.544691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741491+0000","last_change":"2024-02-02T13:35:32.666027+0000","last_active":"2024-02-02T13:35:33.741491+0000","last_peered":"2024-02-02T13:35:33.741491+0000","last_clean":"2024-02-02T13:35:33.741491+0000","last_became_active":"2024-02-02T13:35:32.665772+0000","last_became_peered":"2024-02-02T13:35:32.665772+0000","last_unstale":"2024-02-02T13:35:33.741491+0000","last_undegraded":"2024-02-02T13:35:33.741491+0000","last_fullsized":"2024-02-02T13:35:33.741491+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:08:34.055868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739366+0000","last_change":"2024-02-02T13:35:32.669229+0000","last_active":"2024-02-02T13:35:33.739366+0000","last_peered":"2024-02-02T13:35:33.739366+0000","last_clean":"2024-02-02T13:35:33.739366+0000","last_became_active":"2024-02-02T13:35:32.667517+0000","last_became_peered":"2024-02-02T13:35:32.667517+0000","last_unstale":"2024-02-02T13:35:33.739366+0000","last_undegraded":"2024-02-02T13:35:33.739366+0000","last_fullsized":"2024-02-02T13:35:33.739366+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:06.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739518+0000","last_change":"2024-02-02T13:35:32.677617+0000","last_active":"2024-02-02T13:35:33.739518+0000","last_peered":"2024-02-02T13:35:33.739518+0000","last_clean":"2024-02-02T13:35:33.739518+0000","last_became_active":"2024-02-02T13:35:32.675768+0000","last_became_peered":"2024-02-02T13:35:32.675768+0000","last_unstale":"2024-02-02T13:35:33.739518+0000","last_undegraded":"2024-02-02T13:35:33.739518+0000","last_fullsized":"2024-02-02T13:35:33.739518+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:13.088957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743155+0000","last_change":"2024-02-02T13:35:32.668729+0000","last_active":"2024-02-02T13:35:33.743155+0000","last_peered":"2024-02-02T13:35:33.743155+0000","last_clean":"2024-02-02T13:35:33.743155+0000","last_became_active":"2024-02-02T13:35:32.668404+0000","last_became_peered":"2024-02-02T13:35:32.668404+0000","last_unstale":"2024-02-02T13:35:33.743155+0000","last_undegraded":"2024-02-02T13:35:33.743155+0000","last_fullsized":"2024-02-02T13:35:33.743155+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:24.955654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651967+0000","last_change":"2024-02-02T13:35:32.661526+0000","last_active":"2024-02-02T13:35:33.651967+0000","last_peered":"2024-02-02T13:35:33.651967+0000","last_clean":"2024-02-02T13:35:33.651967+0000","last_became_active":"2024-02-02T13:35:32.661207+0000","last_became_peered":"2024-02-02T13:35:32.661207+0000","last_unstale":"2024-02-02T13:35:33.651967+0000","last_undegraded":"2024-02-02T13:35:33.651967+0000","last_fullsized":"2024-02-02T13:35:33.651967+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:35.726879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742141+0000","last_change":"2024-02-02T13:35:32.660668+0000","last_active":"2024-02-02T13:35:33.742141+0000","last_peered":"2024-02-02T13:35:33.742141+0000","last_clean":"2024-02-02T13:35:33.742141+0000","last_became_active":"2024-02-02T13:35:32.660158+0000","last_became_peered":"2024-02-02T13:35:32.660158+0000","last_unstale":"2024-02-02T13:35:33.742141+0000","last_undegraded":"2024-02-02T13:35:33.742141+0000","last_fullsized":"2024-02-02T13:35:33.742141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:01.141500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740182+0000","last_change":"2024-02-02T13:35:32.671950+0000","last_active":"2024-02-02T13:35:33.740182+0000","last_peered":"2024-02-02T13:35:33.740182+0000","last_clean":"2024-02-02T13:35:33.740182+0000","last_became_active":"2024-02-02T13:35:32.671113+0000","last_became_peered":"2024-02-02T13:35:32.671113+0000","last_unstale":"2024-02-02T13:35:33.740182+0000","last_undegraded":"2024-02-02T13:35:33.740182+0000","last_fullsized":"2024-02-02T13:35:33.740182+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:05:05.235382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737712+0000","last_change":"2024-02-02T13:35:32.659370+0000","last_active":"2024-02-02T13:35:33.737712+0000","last_peered":"2024-02-02T13:35:33.737712+0000","last_clean":"2024-02-02T13:35:33.737712+0000","last_became_active":"2024-02-02T13:35:32.658767+0000","last_became_peered":"2024-02-02T13:35:32.658767+0000","last_unstale":"2024-02-02T13:35:33.737712+0000","last_undegraded":"2024-02-02T13:35:33.737712+0000","last_fullsized":"2024-02-02T13:35:33.737712+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:39:15.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651939+0000","last_change":"2024-02-02T13:35:32.661505+0000","last_active":"2024-02-02T13:35:33.651939+0000","last_peered":"2024-02-02T13:35:33.651939+0000","last_clean":"2024-02-02T13:35:33.651939+0000","last_became_active":"2024-02-02T13:35:32.660604+0000","last_became_peered":"2024-02-02T13:35:32.660604+0000","last_unstale":"2024-02-02T13:35:33.651939+0000","last_undegraded":"2024-02-02T13:35:33.651939+0000","last_fullsized":"2024-02-02T13:35:33.651939+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:12:50.261878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742050+0000","last_change":"2024-02-02T13:35:32.678267+0000","last_active":"2024-02-02T13:35:33.742050+0000","last_peered":"2024-02-02T13:35:33.742050+0000","last_clean":"2024-02-02T13:35:33.742050+0000","last_became_active":"2024-02-02T13:35:32.678033+0000","last_became_peered":"2024-02-02T13:35:32.678033+0000","last_unstale":"2024-02-02T13:35:33.742050+0000","last_undegraded":"2024-02-02T13:35:33.742050+0000","last_fullsized":"2024-02-02T13:35:33.742050+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:25:06.259445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740397+0000","last_change":"2024-02-02T13:35:32.672458+0000","last_active":"2024-02-02T13:35:33.740397+0000","last_peered":"2024-02-02T13:35:33.740397+0000","last_clean":"2024-02-02T13:35:33.740397+0000","last_became_active":"2024-02-02T13:35:32.671916+0000","last_became_peered":"2024-02-02T13:35:32.671916+0000","last_unstale":"2024-02-02T13:35:33.740397+0000","last_undegraded":"2024-02-02T13:35:33.740397+0000","last_fullsized":"2024-02-02T13:35:33.740397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:22.095504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739103+0000","last_change":"2024-02-02T13:35:32.671447+0000","last_active":"2024-02-02T13:35:33.739103+0000","last_peered":"2024-02-02T13:35:33.739103+0000","last_clean":"2024-02-02T13:35:33.739103+0000","last_became_active":"2024-02-02T13:35:32.670505+0000","last_became_peered":"2024-02-02T13:35:32.670505+0000","last_unstale":"2024-02-02T13:35:33.739103+0000","last_undegraded":"2024-02-02T13:35:33.739103+0000","last_fullsized":"2024-02-02T13:35:33.739103+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:00.094279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740420+0000","last_change":"2024-02-02T13:35:32.675192+0000","last_active":"2024-02-02T13:35:33.740420+0000","last_peered":"2024-02-02T13:35:33.740420+0000","last_clean":"2024-02-02T13:35:33.740420+0000","last_became_active":"2024-02-02T13:35:32.674893+0000","last_became_peered":"2024-02-02T13:35:32.674893+0000","last_unstale":"2024-02-02T13:35:33.740420+0000","last_undegraded":"2024-02-02T13:35:33.740420+0000","last_fullsized":"2024-02-02T13:35:33.740420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:28.365157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.652209+0000","last_change":"2024-02-02T13:35:32.662058+0000","last_active":"2024-02-02T13:35:33.652209+0000","last_peered":"2024-02-02T13:35:33.652209+0000","last_clean":"2024-02-02T13:35:33.652209+0000","last_became_active":"2024-02-02T13:35:32.660893+0000","last_became_peered":"2024-02-02T13:35:32.660893+0000","last_unstale":"2024-02-02T13:35:33.652209+0000","last_undegraded":"2024-02-02T13:35:33.652209+0000","last_fullsized":"2024-02-02T13:35:33.652209+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:09:22.158420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737559+0000","last_change":"2024-02-02T13:35:32.660848+0000","last_active":"2024-02-02T13:35:33.737559+0000","last_peered":"2024-02-02T13:35:33.737559+0000","last_clean":"2024-02-02T13:35:33.737559+0000","last_became_active":"2024-02-02T13:35:32.660179+0000","last_became_peered":"2024-02-02T13:35:32.660179+0000","last_unstale":"2024-02-02T13:35:33.737559+0000","last_undegraded":"2024-02-02T13:35:33.737559+0000","last_fullsized":"2024-02-02T13:35:33.737559+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:56:17.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651734+0000","last_change":"2024-02-02T13:35:32.663063+0000","last_active":"2024-02-02T13:35:33.651734+0000","last_peered":"2024-02-02T13:35:33.651734+0000","last_clean":"2024-02-02T13:35:33.651734+0000","last_became_active":"2024-02-02T13:35:32.662828+0000","last_became_peered":"2024-02-02T13:35:32.662828+0000","last_unstale":"2024-02-02T13:35:33.651734+0000","last_undegraded":"2024-02-02T13:35:33.651734+0000","last_fullsized":"2024-02-02T13:35:33.651734+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:34.989315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650112+0000","last_change":"2024-02-02T13:35:32.672427+0000","last_active":"2024-02-02T13:35:33.650112+0000","last_peered":"2024-02-02T13:35:33.650112+0000","last_clean":"2024-02-02T13:35:33.650112+0000","last_became_active":"2024-02-02T13:35:32.672210+0000","last_became_peered":"2024-02-02T13:35:32.672210+0000","last_unstale":"2024-02-02T13:35:33.650112+0000","last_undegraded":"2024-02-02T13:35:33.650112+0000","last_fullsized":"2024-02-02T13:35:33.650112+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:24.955654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737490+0000","last_change":"2024-02-02T13:35:32.659833+0000","last_active":"2024-02-02T13:35:33.737490+0000","last_peered":"2024-02-02T13:35:33.737490+0000","last_clean":"2024-02-02T13:35:33.737490+0000","last_became_active":"2024-02-02T13:35:32.659623+0000","last_became_peered":"2024-02-02T13:35:32.659623+0000","last_unstale":"2024-02-02T13:35:33.737490+0000","last_undegraded":"2024-02-02T13:35:33.737490+0000","last_fullsized":"2024-02-02T13:35:33.737490+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:55.838868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742077+0000","last_change":"2024-02-02T13:35:32.674732+0000","last_active":"2024-02-02T13:35:33.742077+0000","last_peered":"2024-02-02T13:35:33.742077+0000","last_clean":"2024-02-02T13:35:33.742077+0000","last_became_active":"2024-02-02T13:35:32.672794+0000","last_became_peered":"2024-02-02T13:35:32.672794+0000","last_unstale":"2024-02-02T13:35:33.742077+0000","last_undegraded":"2024-02-02T13:35:33.742077+0000","last_fullsized":"2024-02-02T13:35:33.742077+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:43:01.729452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646357+0000","last_change":"2024-02-02T13:35:32.663491+0000","last_active":"2024-02-02T13:35:33.646357+0000","last_peered":"2024-02-02T13:35:33.646357+0000","last_clean":"2024-02-02T13:35:33.646357+0000","last_became_active":"2024-02-02T13:35:32.663331+0000","last_became_peered":"2024-02-02T13:35:32.663331+0000","last_unstale":"2024-02-02T13:35:33.646357+0000","last_undegraded":"2024-02-02T13:35:33.646357+0000","last_fullsized":"2024-02-02T13:35:33.646357+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:38:01.421691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651824+0000","last_change":"2024-02-02T13:35:32.656809+0000","last_active":"2024-02-02T13:35:33.651824+0000","last_peered":"2024-02-02T13:35:33.651824+0000","last_clean":"2024-02-02T13:35:33.651824+0000","last_became_active":"2024-02-02T13:35:32.656586+0000","last_became_peered":"2024-02-02T13:35:32.656586+0000","last_unstale":"2024-02-02T13:35:33.651824+0000","last_undegraded":"2024-02-02T13:35:33.651824+0000","last_fullsized":"2024-02-02T13:35:33.651824+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:51:06.185014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740685+0000","last_change":"2024-02-02T13:35:32.666330+0000","last_active":"2024-02-02T13:35:33.740685+0000","last_peered":"2024-02-02T13:35:33.740685+0000","last_clean":"2024-02-02T13:35:33.740685+0000","last_became_active":"2024-02-02T13:35:32.665875+0000","last_became_peered":"2024-02-02T13:35:32.665875+0000","last_unstale":"2024-02-02T13:35:33.740685+0000","last_undegraded":"2024-02-02T13:35:33.740685+0000","last_fullsized":"2024-02-02T13:35:33.740685+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:39.061381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"177'4","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677320+0000","last_change":"2024-02-02T13:35:32.184481+0000","last_active":"2024-02-02T13:35:32.677320+0000","last_peered":"2024-02-02T13:35:32.677320+0000","last_clean":"2024-02-02T13:35:32.677320+0000","last_became_active":"2024-02-02T13:35:32.183868+0000","last_became_peered":"2024-02-02T13:35:32.183868+0000","last_unstale":"2024-02-02T13:35:32.677320+0000","last_undegraded":"2024-02-02T13:35:32.677320+0000","last_fullsized":"2024-02-02T13:35:32.677320+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:35:33.209478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646581+0000","last_change":"2024-02-02T13:35:32.656609+0000","last_active":"2024-02-02T13:35:33.646581+0000","last_peered":"2024-02-02T13:35:33.646581+0000","last_clean":"2024-02-02T13:35:33.646581+0000","last_became_active":"2024-02-02T13:35:32.656393+0000","last_became_peered":"2024-02-02T13:35:32.656393+0000","last_unstale":"2024-02-02T13:35:33.646581+0000","last_undegraded":"2024-02-02T13:35:33.646581+0000","last_fullsized":"2024-02-02T13:35:33.646581+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:48:25.655612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.667120+0000","last_change":"2024-02-02T13:35:32.183471+0000","last_active":"2024-02-02T13:35:32.667120+0000","last_peered":"2024-02-02T13:35:32.667120+0000","last_clean":"2024-02-02T13:35:32.667120+0000","last_became_active":"2024-02-02T13:35:32.182955+0000","last_became_peered":"2024-02-02T13:35:32.182955+0000","last_unstale":"2024-02-02T13:35:32.667120+0000","last_undegraded":"2024-02-02T13:35:32.667120+0000","last_fullsized":"2024-02-02T13:35:32.667120+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:21.080419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737677+0000","last_change":"2024-02-02T13:35:32.676683+0000","last_active":"2024-02-02T13:35:33.737677+0000","last_peered":"2024-02-02T13:35:33.737677+0000","last_clean":"2024-02-02T13:35:33.737677+0000","last_became_active":"2024-02-02T13:35:32.676450+0000","last_became_peered":"2024-02-02T13:35:32.676450+0000","last_unstale":"2024-02-02T13:35:33.737677+0000","last_undegraded":"2024-02-02T13:35:33.737677+0000","last_fullsized":"2024-02-02T13:35:33.737677+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:02:59.880675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.672096+0000","last_change":"2024-02-02T13:35:32.179654+0000","last_active":"2024-02-02T13:35:32.672096+0000","last_peered":"2024-02-02T13:35:32.672096+0000","last_clean":"2024-02-02T13:35:32.672096+0000","last_became_active":"2024-02-02T13:35:32.179207+0000","last_became_peered":"2024-02-02T13:35:32.179207+0000","last_unstale":"2024-02-02T13:35:32.672096+0000","last_undegraded":"2024-02-02T13:35:32.672096+0000","last_fullsized":"2024-02-02T13:35:32.672096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:57.846865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743281+0000","last_change":"2024-02-02T13:35:32.672162+0000","last_active":"2024-02-02T13:35:33.743281+0000","last_peered":"2024-02-02T13:35:33.743281+0000","last_clean":"2024-02-02T13:35:33.743281+0000","last_became_active":"2024-02-02T13:35:32.671945+0000","last_became_peered":"2024-02-02T13:35:32.671945+0000","last_unstale":"2024-02-02T13:35:33.743281+0000","last_undegraded":"2024-02-02T13:35:33.743281+0000","last_fullsized":"2024-02-02T13:35:33.743281+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:54.540869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677053+0000","last_change":"2024-02-02T13:35:32.167390+0000","last_active":"2024-02-02T13:35:32.677053+0000","last_peered":"2024-02-02T13:35:32.677053+0000","last_clean":"2024-02-02T13:35:32.677053+0000","last_became_active":"2024-02-02T13:35:32.165528+0000","last_became_peered":"2024-02-02T13:35:32.165528+0000","last_unstale":"2024-02-02T13:35:32.677053+0000","last_undegraded":"2024-02-02T13:35:32.677053+0000","last_fullsized":"2024-02-02T13:35:32.677053+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:58.406445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646819+0000","last_change":"2024-02-02T13:35:32.675839+0000","last_active":"2024-02-02T13:35:33.646819+0000","last_peered":"2024-02-02T13:35:33.646819+0000","last_clean":"2024-02-02T13:35:33.646819+0000","last_became_active":"2024-02-02T13:35:32.675001+0000","last_became_peered":"2024-02-02T13:35:32.675001+0000","last_unstale":"2024-02-02T13:35:33.646819+0000","last_undegraded":"2024-02-02T13:35:33.646819+0000","last_fullsized":"2024-02-02T13:35:33.646819+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:52.932905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671293+0000","last_change":"2024-02-02T13:35:32.182929+0000","last_active":"2024-02-02T13:35:32.671293+0000","last_peered":"2024-02-02T13:35:32.671293+0000","last_clean":"2024-02-02T13:35:32.671293+0000","last_became_active":"2024-02-02T13:35:32.182736+0000","last_became_peered":"2024-02-02T13:35:32.182736+0000","last_unstale":"2024-02-02T13:35:32.671293+0000","last_undegraded":"2024-02-02T13:35:32.671293+0000","last_fullsized":"2024-02-02T13:35:32.671293+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:13:44.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740652+0000","last_change":"2024-02-02T13:35:32.672317+0000","last_active":"2024-02-02T13:35:33.740652+0000","last_peered":"2024-02-02T13:35:33.740652+0000","last_clean":"2024-02-02T13:35:33.740652+0000","last_became_active":"2024-02-02T13:35:32.671670+0000","last_became_peered":"2024-02-02T13:35:32.671670+0000","last_unstale":"2024-02-02T13:35:33.740652+0000","last_undegraded":"2024-02-02T13:35:33.740652+0000","last_fullsized":"2024-02-02T13:35:33.740652+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:53.911546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.675236+0000","last_change":"2024-02-02T13:35:32.182177+0000","last_active":"2024-02-02T13:35:32.675236+0000","last_peered":"2024-02-02T13:35:32.675236+0000","last_clean":"2024-02-02T13:35:32.675236+0000","last_became_active":"2024-02-02T13:35:32.181943+0000","last_became_peered":"2024-02-02T13:35:32.181943+0000","last_unstale":"2024-02-02T13:35:32.675236+0000","last_undegraded":"2024-02-02T13:35:32.675236+0000","last_fullsized":"2024-02-02T13:35:32.675236+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:21:58.025523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646663+0000","last_change":"2024-02-02T13:35:32.656320+0000","last_active":"2024-02-02T13:35:33.646663+0000","last_peered":"2024-02-02T13:35:33.646663+0000","last_clean":"2024-02-02T13:35:33.646663+0000","last_became_active":"2024-02-02T13:35:32.656079+0000","last_became_peered":"2024-02-02T13:35:32.656079+0000","last_unstale":"2024-02-02T13:35:33.646663+0000","last_undegraded":"2024-02-02T13:35:33.646663+0000","last_fullsized":"2024-02-02T13:35:33.646663+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:54.250935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.653829+0000","last_change":"2024-02-02T13:35:32.176176+0000","last_active":"2024-02-02T13:35:32.653829+0000","last_peered":"2024-02-02T13:35:32.653829+0000","last_clean":"2024-02-02T13:35:32.653829+0000","last_became_active":"2024-02-02T13:35:32.175937+0000","last_became_peered":"2024-02-02T13:35:32.175937+0000","last_unstale":"2024-02-02T13:35:32.653829+0000","last_undegraded":"2024-02-02T13:35:32.653829+0000","last_fullsized":"2024-02-02T13:35:32.653829+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:07.131016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741569+0000","last_change":"2024-02-02T13:35:32.664227+0000","last_active":"2024-02-02T13:35:33.741569+0000","last_peered":"2024-02-02T13:35:33.741569+0000","last_clean":"2024-02-02T13:35:33.741569+0000","last_became_active":"2024-02-02T13:35:32.663823+0000","last_became_peered":"2024-02-02T13:35:32.663823+0000","last_unstale":"2024-02-02T13:35:33.741569+0000","last_undegraded":"2024-02-02T13:35:33.741569+0000","last_fullsized":"2024-02-02T13:35:33.741569+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:44.636889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.668117+0000","last_change":"2024-02-02T13:35:32.182783+0000","last_active":"2024-02-02T13:35:32.668117+0000","last_peered":"2024-02-02T13:35:32.668117+0000","last_clean":"2024-02-02T13:35:32.668117+0000","last_became_active":"2024-02-02T13:35:32.180069+0000","last_became_peered":"2024-02-02T13:35:32.180069+0000","last_unstale":"2024-02-02T13:35:32.668117+0000","last_undegraded":"2024-02-02T13:35:32.668117+0000","last_fullsized":"2024-02-02T13:35:32.668117+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:23.793930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739411+0000","last_change":"2024-02-02T13:35:32.658537+0000","last_active":"2024-02-02T13:35:33.739411+0000","last_peered":"2024-02-02T13:35:33.739411+0000","last_clean":"2024-02-02T13:35:33.739411+0000","last_became_active":"2024-02-02T13:35:32.657037+0000","last_became_peered":"2024-02-02T13:35:32.657037+0000","last_unstale":"2024-02-02T13:35:33.739411+0000","last_undegraded":"2024-02-02T13:35:33.739411+0000","last_fullsized":"2024-02-02T13:35:33.739411+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:02:59.880675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.646704+0000","last_change":"2024-02-02T13:35:32.174760+0000","last_active":"2024-02-02T13:35:32.646704+0000","last_peered":"2024-02-02T13:35:32.646704+0000","last_clean":"2024-02-02T13:35:32.646704+0000","last_became_active":"2024-02-02T13:35:32.174521+0000","last_became_peered":"2024-02-02T13:35:32.174521+0000","last_unstale":"2024-02-02T13:35:32.646704+0000","last_undegraded":"2024-02-02T13:35:32.646704+0000","last_fullsized":"2024-02-02T13:35:32.646704+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:59.079193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742291+0000","last_change":"2024-02-02T13:35:32.658661+0000","last_active":"2024-02-02T13:35:33.742291+0000","last_peered":"2024-02-02T13:35:33.742291+0000","last_clean":"2024-02-02T13:35:33.742291+0000","last_became_active":"2024-02-02T13:35:32.657453+0000","last_became_peered":"2024-02-02T13:35:32.657453+0000","last_unstale":"2024-02-02T13:35:33.742291+0000","last_undegraded":"2024-02-02T13:35:33.742291+0000","last_fullsized":"2024-02-02T13:35:33.742291+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:55:33.191478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.649309+0000","last_change":"2024-02-02T13:35:32.185731+0000","last_active":"2024-02-02T13:35:32.649309+0000","last_peered":"2024-02-02T13:35:32.649309+0000","last_clean":"2024-02-02T13:35:32.649309+0000","last_became_active":"2024-02-02T13:35:32.184769+0000","last_became_peered":"2024-02-02T13:35:32.184769+0000","last_unstale":"2024-02-02T13:35:32.649309+0000","last_undegraded":"2024-02-02T13:35:32.649309+0000","last_fullsized":"2024-02-02T13:35:32.649309+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:56:56.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651398+0000","last_change":"2024-02-02T13:35:32.658230+0000","last_active":"2024-02-02T13:35:33.651398+0000","last_peered":"2024-02-02T13:35:33.651398+0000","last_clean":"2024-02-02T13:35:33.651398+0000","last_became_active":"2024-02-02T13:35:32.658011+0000","last_became_peered":"2024-02-02T13:35:32.658011+0000","last_unstale":"2024-02-02T13:35:33.651398+0000","last_undegraded":"2024-02-02T13:35:33.651398+0000","last_fullsized":"2024-02-02T13:35:33.651398+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:46.758785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'6","reported_seq":17,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.673002+0000","last_change":"2024-02-02T13:35:32.177602+0000","last_active":"2024-02-02T13:35:32.673002+0000","last_peered":"2024-02-02T13:35:32.673002+0000","last_clean":"2024-02-02T13:35:32.673002+0000","last_became_active":"2024-02-02T13:35:32.177171+0000","last_became_peered":"2024-02-02T13:35:32.177171+0000","last_unstale":"2024-02-02T13:35:32.673002+0000","last_undegraded":"2024-02-02T13:35:32.673002+0000","last_fullsized":"2024-02-02T13:35:32.673002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:08:25.309223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742202+0000","last_change":"2024-02-02T13:35:32.678320+0000","last_active":"2024-02-02T13:35:33.742202+0000","last_peered":"2024-02-02T13:35:33.742202+0000","last_clean":"2024-02-02T13:35:33.742202+0000","last_became_active":"2024-02-02T13:35:32.677502+0000","last_became_peered":"2024-02-02T13:35:32.677502+0000","last_unstale":"2024-02-02T13:35:33.742202+0000","last_undegraded":"2024-02-02T13:35:33.742202+0000","last_fullsized":"2024-02-02T13:35:33.742202+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:00.094279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.670956+0000","last_change":"2024-02-02T13:35:32.176341+0000","last_active":"2024-02-02T13:35:32.670956+0000","last_peered":"2024-02-02T13:35:32.670956+0000","last_clean":"2024-02-02T13:35:32.670956+0000","last_became_active":"2024-02-02T13:35:32.176034+0000","last_became_peered":"2024-02-02T13:35:32.176034+0000","last_unstale":"2024-02-02T13:35:32.670956+0000","last_undegraded":"2024-02-02T13:35:32.670956+0000","last_fullsized":"2024-02-02T13:35:32.670956+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:42:12.461433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646468+0000","last_change":"2024-02-02T13:35:32.653691+0000","last_active":"2024-02-02T13:35:33.646468+0000","last_peered":"2024-02-02T13:35:33.646468+0000","last_clean":"2024-02-02T13:35:33.646468+0000","last_became_active":"2024-02-02T13:35:32.653471+0000","last_became_peered":"2024-02-02T13:35:32.653471+0000","last_unstale":"2024-02-02T13:35:33.646468+0000","last_undegraded":"2024-02-02T13:35:33.646468+0000","last_fullsized":"2024-02-02T13:35:33.646468+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:00:34.703958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.662777+0000","last_change":"2024-02-02T13:35:32.177542+0000","last_active":"2024-02-02T13:35:32.662777+0000","last_peered":"2024-02-02T13:35:32.662777+0000","last_clean":"2024-02-02T13:35:32.662777+0000","last_became_active":"2024-02-02T13:35:32.177089+0000","last_became_peered":"2024-02-02T13:35:32.177089+0000","last_unstale":"2024-02-02T13:35:32.662777+0000","last_undegraded":"2024-02-02T13:35:32.662777+0000","last_fullsized":"2024-02-02T13:35:32.662777+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:43:13.136015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739606+0000","last_change":"2024-02-02T13:35:32.670012+0000","last_active":"2024-02-02T13:35:33.739606+0000","last_peered":"2024-02-02T13:35:33.739606+0000","last_clean":"2024-02-02T13:35:33.739606+0000","last_became_active":"2024-02-02T13:35:32.668683+0000","last_became_peered":"2024-02-02T13:35:32.668683+0000","last_unstale":"2024-02-02T13:35:33.739606+0000","last_undegraded":"2024-02-02T13:35:33.739606+0000","last_fullsized":"2024-02-02T13:35:33.739606+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:39:08.847788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.642804+0000","last_change":"2024-02-02T13:35:32.177009+0000","last_active":"2024-02-02T13:35:32.642804+0000","last_peered":"2024-02-02T13:35:32.642804+0000","last_clean":"2024-02-02T13:35:32.642804+0000","last_became_active":"2024-02-02T13:35:32.176656+0000","last_became_peered":"2024-02-02T13:35:32.176656+0000","last_unstale":"2024-02-02T13:35:32.642804+0000","last_undegraded":"2024-02-02T13:35:32.642804+0000","last_fullsized":"2024-02-02T13:35:32.642804+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:12.927532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740358+0000","last_change":"2024-02-02T13:35:32.664108+0000","last_active":"2024-02-02T13:35:33.740358+0000","last_peered":"2024-02-02T13:35:33.740358+0000","last_clean":"2024-02-02T13:35:33.740358+0000","last_became_active":"2024-02-02T13:35:32.663461+0000","last_became_peered":"2024-02-02T13:35:32.663461+0000","last_unstale":"2024-02-02T13:35:33.740358+0000","last_undegraded":"2024-02-02T13:35:33.740358+0000","last_fullsized":"2024-02-02T13:35:33.740358+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:22.095504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"177'8","reported_seq":19,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671591+0000","last_change":"2024-02-02T13:35:32.163677+0000","last_active":"2024-02-02T13:35:32.671591+0000","last_peered":"2024-02-02T13:35:32.671591+0000","last_clean":"2024-02-02T13:35:32.671591+0000","last_became_active":"2024-02-02T13:35:32.163008+0000","last_became_peered":"2024-02-02T13:35:32.163008+0000","last_unstale":"2024-02-02T13:35:32.671591+0000","last_undegraded":"2024-02-02T13:35:32.671591+0000","last_fullsized":"2024-02-02T13:35:32.671591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:22:03.837325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743400+0000","last_change":"2024-02-02T13:35:32.656040+0000","last_active":"2024-02-02T13:35:33.743400+0000","last_peered":"2024-02-02T13:35:33.743400+0000","last_clean":"2024-02-02T13:35:33.743400+0000","last_became_active":"2024-02-02T13:35:32.655430+0000","last_became_peered":"2024-02-02T13:35:32.655430+0000","last_unstale":"2024-02-02T13:35:33.743400+0000","last_undegraded":"2024-02-02T13:35:33.743400+0000","last_fullsized":"2024-02-02T13:35:33.743400+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:36:16.308158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671502+0000","last_change":"2024-02-02T13:35:32.175607+0000","last_active":"2024-02-02T13:35:32.671502+0000","last_peered":"2024-02-02T13:35:32.671502+0000","last_clean":"2024-02-02T13:35:32.671502+0000","last_became_active":"2024-02-02T13:35:32.175215+0000","last_became_peered":"2024-02-02T13:35:32.175215+0000","last_unstale":"2024-02-02T13:35:32.671502+0000","last_undegraded":"2024-02-02T13:35:32.671502+0000","last_fullsized":"2024-02-02T13:35:32.671502+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:32:54.398787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651506+0000","last_change":"2024-02-02T13:35:32.665230+0000","last_active":"2024-02-02T13:35:33.651506+0000","last_peered":"2024-02-02T13:35:33.651506+0000","last_clean":"2024-02-02T13:35:33.651506+0000","last_became_active":"2024-02-02T13:35:32.664952+0000","last_became_peered":"2024-02-02T13:35:32.664952+0000","last_unstale":"2024-02-02T13:35:33.651506+0000","last_undegraded":"2024-02-02T13:35:33.651506+0000","last_fullsized":"2024-02-02T13:35:33.651506+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:37:34.460344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.661880+0000","last_change":"2024-02-02T13:35:32.176837+0000","last_active":"2024-02-02T13:35:32.661880+0000","last_peered":"2024-02-02T13:35:32.661880+0000","last_clean":"2024-02-02T13:35:32.661880+0000","last_became_active":"2024-02-02T13:35:32.176119+0000","last_became_peered":"2024-02-02T13:35:32.176119+0000","last_unstale":"2024-02-02T13:35:32.661880+0000","last_undegraded":"2024-02-02T13:35:32.661880+0000","last_fullsized":"2024-02-02T13:35:32.661880+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:12.073872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740609+0000","last_change":"2024-02-02T13:35:32.676995+0000","last_active":"2024-02-02T13:35:33.740609+0000","last_peered":"2024-02-02T13:35:33.740609+0000","last_clean":"2024-02-02T13:35:33.740609+0000","last_became_active":"2024-02-02T13:35:32.675927+0000","last_became_peered":"2024-02-02T13:35:32.675927+0000","last_unstale":"2024-02-02T13:35:33.740609+0000","last_undegraded":"2024-02-02T13:35:33.740609+0000","last_fullsized":"2024-02-02T13:35:33.740609+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:37.834948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.665164+0000","last_change":"2024-02-02T13:35:32.181529+0000","last_active":"2024-02-02T13:35:32.665164+0000","last_peered":"2024-02-02T13:35:32.665164+0000","last_clean":"2024-02-02T13:35:32.665164+0000","last_became_active":"2024-02-02T13:35:32.181311+0000","last_became_peered":"2024-02-02T13:35:32.181311+0000","last_unstale":"2024-02-02T13:35:32.665164+0000","last_undegraded":"2024-02-02T13:35:32.665164+0000","last_fullsized":"2024-02-02T13:35:32.665164+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:24.246451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.738213+0000","last_change":"2024-02-02T13:35:32.666526+0000","last_active":"2024-02-02T13:35:33.738213+0000","last_peered":"2024-02-02T13:35:33.738213+0000","last_clean":"2024-02-02T13:35:33.738213+0000","last_became_active":"2024-02-02T13:35:32.666207+0000","last_became_peered":"2024-02-02T13:35:32.666207+0000","last_unstale":"2024-02-02T13:35:33.738213+0000","last_undegraded":"2024-02-02T13:35:33.738213+0000","last_fullsized":"2024-02-02T13:35:33.738213+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:34:53.279843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.645941+0000","last_change":"2024-02-02T13:35:32.176879+0000","last_active":"2024-02-02T13:35:32.645941+0000","last_peered":"2024-02-02T13:35:32.645941+0000","last_clean":"2024-02-02T13:35:32.645941+0000","last_became_active":"2024-02-02T13:35:32.176328+0000","last_became_peered":"2024-02-02T13:35:32.176328+0000","last_unstale":"2024-02-02T13:35:32.645941+0000","last_undegraded":"2024-02-02T13:35:32.645941+0000","last_fullsized":"2024-02-02T13:35:32.645941+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:52.896461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739890+0000","last_change":"2024-02-02T13:35:32.676016+0000","last_active":"2024-02-02T13:35:33.739890+0000","last_peered":"2024-02-02T13:35:33.739890+0000","last_clean":"2024-02-02T13:35:33.739890+0000","last_became_active":"2024-02-02T13:35:32.673614+0000","last_became_peered":"2024-02-02T13:35:32.673614+0000","last_unstale":"2024-02-02T13:35:33.739890+0000","last_undegraded":"2024-02-02T13:35:33.739890+0000","last_fullsized":"2024-02-02T13:35:33.739890+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:37.834948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.673443+0000","last_change":"2024-02-02T13:35:32.175627+0000","last_active":"2024-02-02T13:35:32.673443+0000","last_peered":"2024-02-02T13:35:32.673443+0000","last_clean":"2024-02-02T13:35:32.673443+0000","last_became_active":"2024-02-02T13:35:32.175309+0000","last_became_peered":"2024-02-02T13:35:32.175309+0000","last_unstale":"2024-02-02T13:35:32.673443+0000","last_undegraded":"2024-02-02T13:35:32.673443+0000","last_fullsized":"2024-02-02T13:35:32.673443+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:51.917820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650359+0000","last_change":"2024-02-02T13:35:32.671398+0000","last_active":"2024-02-02T13:35:33.650359+0000","last_peered":"2024-02-02T13:35:33.650359+0000","last_clean":"2024-02-02T13:35:33.650359+0000","last_became_active":"2024-02-02T13:35:32.671119+0000","last_became_peered":"2024-02-02T13:35:32.671119+0000","last_unstale":"2024-02-02T13:35:33.650359+0000","last_undegraded":"2024-02-02T13:35:33.650359+0000","last_fullsized":"2024-02-02T13:35:33.650359+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:24.760817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.656936+0000","last_change":"2024-02-02T13:35:32.183645+0000","last_active":"2024-02-02T13:35:32.656936+0000","last_peered":"2024-02-02T13:35:32.656936+0000","last_clean":"2024-02-02T13:35:32.656936+0000","last_became_active":"2024-02-02T13:35:32.183279+0000","last_became_peered":"2024-02-02T13:35:32.183279+0000","last_unstale":"2024-02-02T13:35:32.656936+0000","last_undegraded":"2024-02-02T13:35:32.656936+0000","last_fullsized":"2024-02-02T13:35:32.656936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:06.567372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740536+0000","last_change":"2024-02-02T13:35:32.677258+0000","last_active":"2024-02-02T13:35:33.740536+0000","last_peered":"2024-02-02T13:35:33.740536+0000","last_clean":"2024-02-02T13:35:33.740536+0000","last_became_active":"2024-02-02T13:35:32.676086+0000","last_became_peered":"2024-02-02T13:35:32.676086+0000","last_unstale":"2024-02-02T13:35:33.740536+0000","last_undegraded":"2024-02-02T13:35:33.740536+0000","last_fullsized":"2024-02-02T13:35:33.740536+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:00:19.807754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651466+0000","last_change":"2024-02-02T13:35:32.182624+0000","last_active":"2024-02-02T13:35:32.651466+0000","last_peered":"2024-02-02T13:35:32.651466+0000","last_clean":"2024-02-02T13:35:32.651466+0000","last_became_active":"2024-02-02T13:35:32.182141+0000","last_became_peered":"2024-02-02T13:35:32.182141+0000","last_unstale":"2024-02-02T13:35:32.651466+0000","last_undegraded":"2024-02-02T13:35:32.651466+0000","last_fullsized":"2024-02-02T13:35:32.651466+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:03:00.945896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650031+0000","last_change":"2024-02-02T13:35:32.673871+0000","last_active":"2024-02-02T13:35:33.650031+0000","last_peered":"2024-02-02T13:35:33.650031+0000","last_clean":"2024-02-02T13:35:33.650031+0000","last_became_active":"2024-02-02T13:35:32.673492+0000","last_became_peered":"2024-02-02T13:35:32.673492+0000","last_unstale":"2024-02-02T13:35:33.650031+0000","last_undegraded":"2024-02-02T13:35:33.650031+0000","last_fullsized":"2024-02-02T13:35:33.650031+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:45.527941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.662875+0000","last_change":"2024-02-02T13:35:32.174725+0000","last_active":"2024-02-02T13:35:32.662875+0000","last_peered":"2024-02-02T13:35:32.662875+0000","last_clean":"2024-02-02T13:35:32.662875+0000","last_became_active":"2024-02-02T13:35:32.171494+0000","last_became_peered":"2024-02-02T13:35:32.171494+0000","last_unstale":"2024-02-02T13:35:32.662875+0000","last_undegraded":"2024-02-02T13:35:32.662875+0000","last_fullsized":"2024-02-02T13:35:32.662875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:51:14.245561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740014+0000","last_change":"2024-02-02T13:35:32.670509+0000","last_active":"2024-02-02T13:35:33.740014+0000","last_peered":"2024-02-02T13:35:33.740014+0000","last_clean":"2024-02-02T13:35:33.740014+0000","last_became_active":"2024-02-02T13:35:32.669003+0000","last_became_peered":"2024-02-02T13:35:32.669003+0000","last_unstale":"2024-02-02T13:35:33.740014+0000","last_undegraded":"2024-02-02T13:35:33.740014+0000","last_fullsized":"2024-02-02T13:35:33.740014+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:55.838868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677562+0000","last_change":"2024-02-02T13:35:32.185879+0000","last_active":"2024-02-02T13:35:32.677562+0000","last_peered":"2024-02-02T13:35:32.677562+0000","last_clean":"2024-02-02T13:35:32.677562+0000","last_became_active":"2024-02-02T13:35:32.185170+0000","last_became_peered":"2024-02-02T13:35:32.185170+0000","last_unstale":"2024-02-02T13:35:32.677562+0000","last_undegraded":"2024-02-02T13:35:32.677562+0000","last_fullsized":"2024-02-02T13:35:32.677562+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:09:19.060004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740830+0000","last_change":"2024-02-02T13:35:32.676676+0000","last_active":"2024-02-02T13:35:33.740830+0000","last_peered":"2024-02-02T13:35:33.740830+0000","last_clean":"2024-02-02T13:35:33.740830+0000","last_became_active":"2024-02-02T13:35:32.675606+0000","last_became_peered":"2024-02-02T13:35:32.675606+0000","last_unstale":"2024-02-02T13:35:33.740830+0000","last_undegraded":"2024-02-02T13:35:33.740830+0000","last_fullsized":"2024-02-02T13:35:33.740830+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:53.911546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.643459+0000","last_change":"2024-02-02T13:35:32.180293+0000","last_active":"2024-02-02T13:35:32.643459+0000","last_peered":"2024-02-02T13:35:32.643459+0000","last_clean":"2024-02-02T13:35:32.643459+0000","last_became_active":"2024-02-02T13:35:32.180003+0000","last_became_peered":"2024-02-02T13:35:32.180003+0000","last_unstale":"2024-02-02T13:35:32.643459+0000","last_undegraded":"2024-02-02T13:35:32.643459+0000","last_fullsized":"2024-02-02T13:35:32.643459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:36:31.803734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646568+0000","last_change":"2024-02-02T13:35:32.674845+0000","last_active":"2024-02-02T13:35:33.646568+0000","last_peered":"2024-02-02T13:35:33.646568+0000","last_clean":"2024-02-02T13:35:33.646568+0000","last_became_active":"2024-02-02T13:35:32.674553+0000","last_became_peered":"2024-02-02T13:35:32.674553+0000","last_unstale":"2024-02-02T13:35:33.646568+0000","last_undegraded":"2024-02-02T13:35:33.646568+0000","last_fullsized":"2024-02-02T13:35:33.646568+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:18.989067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.644102+0000","last_change":"2024-02-02T13:35:32.158525+0000","last_active":"2024-02-02T13:35:32.644102+0000","last_peered":"2024-02-02T13:35:32.644102+0000","last_clean":"2024-02-02T13:35:32.644102+0000","last_became_active":"2024-02-02T13:35:32.157852+0000","last_became_peered":"2024-02-02T13:35:32.157852+0000","last_unstale":"2024-02-02T13:35:32.644102+0000","last_undegraded":"2024-02-02T13:35:32.644102+0000","last_fullsized":"2024-02-02T13:35:32.644102+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:37:33.445259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737568+0000","last_change":"2024-02-02T13:35:32.674201+0000","last_active":"2024-02-02T13:35:33.737568+0000","last_peered":"2024-02-02T13:35:33.737568+0000","last_clean":"2024-02-02T13:35:33.737568+0000","last_became_active":"2024-02-02T13:35:32.673964+0000","last_became_peered":"2024-02-02T13:35:32.673964+0000","last_unstale":"2024-02-02T13:35:33.737568+0000","last_undegraded":"2024-02-02T13:35:33.737568+0000","last_fullsized":"2024-02-02T13:35:33.737568+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:03.846318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.642963+0000","last_change":"2024-02-02T13:35:32.158967+0000","last_active":"2024-02-02T13:35:32.642963+0000","last_peered":"2024-02-02T13:35:32.642963+0000","last_clean":"2024-02-02T13:35:32.642963+0000","last_became_active":"2024-02-02T13:35:32.157882+0000","last_became_peered":"2024-02-02T13:35:32.157882+0000","last_unstale":"2024-02-02T13:35:32.642963+0000","last_undegraded":"2024-02-02T13:35:32.642963+0000","last_fullsized":"2024-02-02T13:35:32.642963+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:45.743700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651907+0000","last_change":"2024-02-02T13:35:32.673294+0000","last_active":"2024-02-02T13:35:33.651907+0000","last_peered":"2024-02-02T13:35:33.651907+0000","last_clean":"2024-02-02T13:35:33.651907+0000","last_became_active":"2024-02-02T13:35:32.672913+0000","last_became_peered":"2024-02-02T13:35:32.672913+0000","last_unstale":"2024-02-02T13:35:33.651907+0000","last_undegraded":"2024-02-02T13:35:33.651907+0000","last_fullsized":"2024-02-02T13:35:33.651907+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:25.470301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651568+0000","last_change":"2024-02-02T13:35:32.181252+0000","last_active":"2024-02-02T13:35:32.651568+0000","last_peered":"2024-02-02T13:35:32.651568+0000","last_clean":"2024-02-02T13:35:32.651568+0000","last_became_active":"2024-02-02T13:35:32.180517+0000","last_became_peered":"2024-02-02T13:35:32.180517+0000","last_unstale":"2024-02-02T13:35:32.651568+0000","last_undegraded":"2024-02-02T13:35:32.651568+0000","last_fullsized":"2024-02-02T13:35:32.651568+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:00:33.688873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.738803+0000","last_change":"2024-02-02T13:35:32.668535+0000","last_active":"2024-02-02T13:35:33.738803+0000","last_peered":"2024-02-02T13:35:33.738803+0000","last_clean":"2024-02-02T13:35:33.738803+0000","last_became_active":"2024-02-02T13:35:32.667960+0000","last_became_peered":"2024-02-02T13:35:32.667960+0000","last_unstale":"2024-02-02T13:35:33.738803+0000","last_undegraded":"2024-02-02T13:35:33.738803+0000","last_fullsized":"2024-02-02T13:35:33.738803+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:03.846318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651116+0000","last_change":"2024-02-02T13:35:32.186037+0000","last_active":"2024-02-02T13:35:32.651116+0000","last_peered":"2024-02-02T13:35:32.651116+0000","last_clean":"2024-02-02T13:35:32.651116+0000","last_became_active":"2024-02-02T13:35:32.185501+0000","last_became_peered":"2024-02-02T13:35:32.185501+0000","last_unstale":"2024-02-02T13:35:32.651116+0000","last_undegraded":"2024-02-02T13:35:32.651116+0000","last_fullsized":"2024-02-02T13:35:32.651116+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:13.607856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740923+0000","last_change":"2024-02-02T13:35:32.666202+0000","last_active":"2024-02-02T13:35:33.740923+0000","last_peered":"2024-02-02T13:35:33.740923+0000","last_clean":"2024-02-02T13:35:33.740923+0000","last_became_active":"2024-02-02T13:35:32.665407+0000","last_became_peered":"2024-02-02T13:35:32.665407+0000","last_unstale":"2024-02-02T13:35:33.740923+0000","last_undegraded":"2024-02-02T13:35:33.740923+0000","last_fullsized":"2024-02-02T13:35:33.740923+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:51:13.027260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.666410+0000","last_change":"2024-02-02T13:35:32.180218+0000","last_active":"2024-02-02T13:35:32.666410+0000","last_peered":"2024-02-02T13:35:32.666410+0000","last_clean":"2024-02-02T13:35:32.666410+0000","last_became_active":"2024-02-02T13:35:32.176974+0000","last_became_peered":"2024-02-02T13:35:32.176974+0000","last_unstale":"2024-02-02T13:35:32.666410+0000","last_undegraded":"2024-02-02T13:35:32.666410+0000","last_fullsized":"2024-02-02T13:35:32.666410+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:25:32.447201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740244+0000","last_change":"2024-02-02T13:35:32.669497+0000","last_active":"2024-02-02T13:35:33.740244+0000","last_peered":"2024-02-02T13:35:33.740244+0000","last_clean":"2024-02-02T13:35:33.740244+0000","last_became_active":"2024-02-02T13:35:32.668139+0000","last_became_peered":"2024-02-02T13:35:32.668139+0000","last_unstale":"2024-02-02T13:35:33.740244+0000","last_undegraded":"2024-02-02T13:35:33.740244+0000","last_fullsized":"2024-02-02T13:35:33.740244+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:39:15.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.645779+0000","last_change":"2024-02-02T13:35:32.183562+0000","last_active":"2024-02-02T13:35:32.645779+0000","last_peered":"2024-02-02T13:35:32.645779+0000","last_clean":"2024-02-02T13:35:32.645779+0000","last_became_active":"2024-02-02T13:35:32.183302+0000","last_became_peered":"2024-02-02T13:35:32.183302+0000","last_unstale":"2024-02-02T13:35:32.645779+0000","last_undegraded":"2024-02-02T13:35:32.645779+0000","last_fullsized":"2024-02-02T13:35:32.645779+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:36.819863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.649922+0000","last_change":"2024-02-02T13:35:32.674915+0000","last_active":"2024-02-02T13:35:33.649922+0000","last_peered":"2024-02-02T13:35:33.649922+0000","last_clean":"2024-02-02T13:35:33.649922+0000","last_became_active":"2024-02-02T13:35:32.674692+0000","last_became_peered":"2024-02-02T13:35:32.674692+0000","last_unstale":"2024-02-02T13:35:33.649922+0000","last_undegraded":"2024-02-02T13:35:33.649922+0000","last_fullsized":"2024-02-02T13:35:33.649922+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:39.061381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"177'3","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.676255+0000","last_change":"2024-02-02T13:35:32.172620+0000","last_active":"2024-02-02T13:35:32.676255+0000","last_peered":"2024-02-02T13:35:32.676255+0000","last_clean":"2024-02-02T13:35:32.676255+0000","last_became_active":"2024-02-02T13:35:32.170883+0000","last_became_peered":"2024-02-02T13:35:32.170883+0000","last_unstale":"2024-02-02T13:35:32.676255+0000","last_undegraded":"2024-02-02T13:35:32.676255+0000","last_fullsized":"2024-02-02T13:35:32.676255+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:45:26.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":319,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.658611+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:35:32.658611+0000","last_peered":"2024-02-02T13:35:32.658611+0000","last_clean":"2024-02-02T13:35:32.658611+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:35:32.658611+0000","last_undegraded":"2024-02-02T13:35:32.658611+0000","last_fullsized":"2024-02-02T13:35:32.658611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193760,"statfs":{"total":200000000,"available":198410704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1060000000000001,"15min":1.0840000000000001},"min":{"1min":0.79800000000000004,"5min":0.65000000000000002,"15min":0.61499999999999999},"max":{"1min":1.48,"5min":2.266,"15min":2.5310000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.512,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.097,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.73899999999999999,"15min":0.66300000000000003},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.893},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2,"5min":1.1890000000000001,"15min":1.1479999999999999},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.5740000000000001,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.0640000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.258,"15min":1.238},"min":{"1min":0.79000000000000004,"5min":0.73399999999999999,"15min":0.70099999999999996},"max":{"1min":1.6240000000000001,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.236},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.216,"15min":1.1950000000000001},"min":{"1min":0.94099999999999995,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.651,"5min":1.7949999999999999,"15min":3.5819999999999999},"last":0.98699999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.198,"15min":1.1850000000000001},"min":{"1min":0.96899999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5289999999999999,"5min":1.635,"15min":1.819},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.347,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":1.1279999999999999,"5min":0.752,"15min":0.48999999999999999},"max":{"1min":1.548,"5min":2.0259999999999998,"15min":2.4620000000000002},"last":1.4219999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.266,"15min":1.232},"min":{"1min":0.91200000000000003,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.6160000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.323,"15min":1.3129999999999999},"min":{"1min":0.998,"5min":0.86399999999999999,"15min":0.61199999999999999},"max":{"1min":1.6299999999999999,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3779999999999999,"15min":1.3640000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":1.677,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":1.139,"5min":0.84199999999999997,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.921,"15min":3.7949999999999999},"last":1.2150000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5009999999999999,"15min":1.5209999999999999},"min":{"1min":1.2170000000000001,"5min":0.96999999999999997,"15min":0.64200000000000002},"max":{"1min":1.7070000000000001,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4390000000000001,"15min":1.427},"min":{"1min":1.1779999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6739999999999999,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":1.488}]}]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.115,"15min":1.0600000000000001},"min":{"1min":0.877,"5min":0.498,"15min":0.498},"max":{"1min":1.603,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.0669999999999999,"15min":1.0429999999999999},"min":{"1min":0.86899999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.575,"5min":1.768,"15min":1.768},"last":1.054}]},{"osd":1,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.093,"15min":1.05},"min":{"1min":0.91400000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.458,"5min":1.4790000000000001,"15min":1.556},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1659999999999999,"15min":1.115},"min":{"1min":0.94499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5640000000000001,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.196,"15min":1.127},"min":{"1min":0.85399999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.75,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1759999999999999,"15min":1.163},"min":{"1min":0.91100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5169999999999999,"5min":1.849,"15min":1.849},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.254},"min":{"1min":1.0800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5669999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.204,"15min":1.165},"min":{"1min":1.0149999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.458,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.234}]},{"osd":4,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4550000000000001,"15min":1.3069999999999999},"min":{"1min":0.92600000000000005,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.6639999999999999,"5min":17.686,"15min":17.686},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.5409999999999999,"15min":1.452},"min":{"1min":0.95099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.585,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.5089999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.90200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5069999999999999,"5min":1.9810000000000001,"15min":1.986},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.409,"15min":1.385},"min":{"1min":1.0569999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8109999999999999,"5min":3.331,"15min":3.331},"last":1.4339999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.431,"15min":1.4530000000000001},"min":{"1min":1.222,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":1.651,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.415,"15min":1.4339999999999999},"min":{"1min":1.1899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6339999999999999,"5min":2.101,"15min":2.1419999999999999},"last":1.6639999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194337,"statfs":{"total":200000000,"available":199001702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.0620000000000001,"15min":1.036},"min":{"1min":0.85099999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.377,"5min":1.8,"15min":1.8},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0620000000000001,"15min":1.0489999999999999},"min":{"1min":0.80000000000000004,"5min":0.68700000000000006,"15min":0.49199999999999999},"max":{"1min":1.3540000000000001,"5min":1.5580000000000001,"15min":1.7150000000000001},"last":0.96899999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.22},"min":{"1min":1.006,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7370000000000001,"5min":1.841,"15min":4.5899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0649999999999999,"15min":1.117},"min":{"1min":0.89200000000000002,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":4.4500000000000002},"last":1.389}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.284,"15min":1.252},"min":{"1min":1.1200000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.1970000000000001,"15min":1.1899999999999999},"min":{"1min":1.1639999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4350000000000001,"5min":1.7110000000000001,"15min":3.0720000000000001},"last":0.94499999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.242,"15min":1.206},"min":{"1min":0.96999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":1.629,"15min":1.7769999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.27,"15min":1.2150000000000001},"min":{"1min":1.0049999999999999,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":5.4610000000000003},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.194,"15min":1.208},"min":{"1min":0.77800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.647,"5min":1.675,"15min":1.8460000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.319,"15min":1.292},"min":{"1min":1.071,"5min":0.85099999999999998,"15min":0.55100000000000005},"max":{"1min":1.8160000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3460000000000001,"15min":1.3200000000000001},"min":{"1min":0.96899999999999997,"5min":0.745,"15min":0.64700000000000002},"max":{"1min":1.865,"5min":1.865,"15min":2.0009999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.3640000000000001,"15min":1.3520000000000001},"min":{"1min":0.96899999999999997,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.845,"5min":1.845,"15min":1.8640000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4299999999999999,"15min":1.4159999999999999},"min":{"1min":1.1519999999999999,"5min":1.0129999999999999,"15min":0.66500000000000004},"max":{"1min":1.893,"5min":1.893,"15min":1.9670000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4750000000000001,"15min":1.4139999999999999},"min":{"1min":1.097,"5min":0.88300000000000001,"15min":0.61599999999999999},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.4379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":973,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002743,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.752,"5min":0.627,"15min":0.57999999999999996},"max":{"1min":1.4570000000000001,"5min":1.6439999999999999,"15min":4.5949999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.079},"min":{"1min":0.79600000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.411,"5min":1.7609999999999999,"15min":1.839},"last":1.238}]},{"osd":1,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0549999999999999,"15min":1.0229999999999999},"min":{"1min":0.79300000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.4830000000000001,"5min":2.1659999999999999,"15min":2.4809999999999999},"last":1.373},{"interface":"front","average":{"1min":1.238,"5min":1.145,"15min":1.081},"min":{"1min":0.98099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":1.0009999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1259999999999999,"15min":1.1140000000000001},"min":{"1min":0.90200000000000002,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.417,"5min":1.9399999999999999,"15min":3.1200000000000001},"last":1.415},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.212,"15min":1.1299999999999999},"min":{"1min":1.0149999999999999,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.736,"5min":2.077,"15min":2.6000000000000001},"last":1.284}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2569999999999999,"15min":1.2150000000000001},"min":{"1min":1.01,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.452,"5min":2.0470000000000002,"15min":2.9079999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.96199999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.853,"5min":2.1110000000000002,"15min":3.9540000000000002},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2569999999999999,"15min":1.1910000000000001},"min":{"1min":0.76600000000000001,"5min":0.68000000000000005,"15min":0.60899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.331},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3049999999999999},"min":{"1min":0.85599999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5309999999999999,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.585}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4330000000000001,"15min":1.3480000000000001},"min":{"1min":1.1279999999999999,"5min":0.81499999999999995,"15min":0.58899999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.7989999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4750000000000001,"15min":1.427},"min":{"1min":1.2150000000000001,"5min":0.70399999999999996,"15min":0.67800000000000005},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.4379999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4990000000000001,"15min":1.429},"min":{"1min":1.105,"5min":0.76000000000000001,"15min":0.63800000000000001},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5,"15min":1.468},"min":{"1min":1.133,"5min":0.86099999999999999,"15min":0.73599999999999999},"max":{"1min":2.0409999999999999,"5min":2.4900000000000002,"15min":2.698},"last":1.4590000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194337,"statfs":{"total":200000000,"available":199001634,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.73799999999999999},"min":{"1min":0.63200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.234,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.93200000000000005,"15min":0.90400000000000003},"min":{"1min":0.73199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4550000000000001,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.95699999999999996,"15min":0.92400000000000004},"min":{"1min":0.64400000000000002,"5min":0.49099999999999999,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":2.403,"15min":2.403},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.1120000000000001,"15min":1.052},"min":{"1min":0.95799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5549999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.99199999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.0580000000000001,"15min":1.004},"min":{"1min":0.84099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.77400000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.3109999999999999,"5min":1.9159999999999999,"15min":3.4510000000000001},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.1859999999999999},"min":{"1min":0.97599999999999998,"5min":0.71099999999999997,"15min":0.53000000000000003},"max":{"1min":1.4330000000000001,"5min":1.99,"15min":3.585},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2370000000000001},"min":{"1min":1.0189999999999999,"5min":0.70499999999999996,"15min":0.61299999999999999},"max":{"1min":1.7350000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.0620000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.163,"15min":1.206},"min":{"1min":1.0269999999999999,"5min":0.64400000000000002,"15min":0.57099999999999995},"max":{"1min":1.609,"5min":1.8129999999999999,"15min":3.738},"last":1.103},{"interface":"front","average":{"1min":1.282,"5min":1.2490000000000001,"15min":1.2569999999999999},"min":{"1min":1.0509999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7549999999999999,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3779999999999999,"15min":1.3149999999999999},"min":{"1min":1.1839999999999999,"5min":0.81299999999999994,"15min":0.58299999999999996},"max":{"1min":1.823,"5min":1.895,"15min":1.8979999999999999},"last":1.139},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2290000000000001,"15min":1.2809999999999999},"min":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":1.667,"5min":2.0310000000000001,"15min":3.6059999999999999},"last":1.04}]},{"osd":7,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2749999999999999,"15min":1.27},"min":{"1min":1.0349999999999999,"5min":0.69799999999999995,"15min":0.65900000000000003},"max":{"1min":1.851,"5min":1.851,"15min":3.391},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3799999999999999,"15min":1.3440000000000001},"min":{"1min":1.117,"5min":0.80900000000000005,"15min":0.625},"max":{"1min":1.6990000000000001,"5min":1.946,"15min":2.0409999999999999},"last":0.91900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000090,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.68200000000000005,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":1.345,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.097,"5min":0.94899999999999995,"15min":0.85199999999999998},"min":{"1min":0.75600000000000001,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.5720000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.244,"5min":2.153,"15min":2.1680000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.024,"15min":0.93500000000000005},"min":{"1min":0.88300000000000001,"5min":0.47499999999999998,"15min":0.35899999999999999},"max":{"1min":1.5129999999999999,"5min":1.6279999999999999,"15min":1.6499999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91900000000000004,"15min":0.85799999999999998},"min":{"1min":0.78800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.109,"5min":1.9790000000000001,"15min":2.1339999999999999},"last":1.024},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.044,"15min":1.0169999999999999},"min":{"1min":0.84299999999999997,"5min":0.435,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.254,"15min":1.2190000000000001},"min":{"1min":1.073,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4930000000000001,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.262,"15min":1.206},"min":{"1min":1.1659999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.571,"5min":2.02,"15min":2.1360000000000001},"last":1.0509999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.214,"15min":1.149},"min":{"1min":1.028,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5349999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.325,"15min":1.294},"min":{"1min":1.1439999999999999,"5min":0.627,"15min":0.627},"max":{"1min":1.5169999999999999,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3200000000000001,"15min":1.2410000000000001},"min":{"1min":1.101,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.577,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.306,"15min":1.248},"min":{"1min":1.149,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":1.5269999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.339}]},{"osd":7,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.387,"15min":1.286},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.643,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3340000000000001,"15min":1.2709999999999999},"min":{"1min":1.1359999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5449999999999999,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.0609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.81999999999999995,"15min":0.92200000000000004},"min":{"1min":0.63600000000000001,"5min":0.40799999999999997,"15min":0.34200000000000003},"max":{"1min":1.196,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.997,"5min":0.91000000000000003,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":0.996}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.755,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.2250000000000001,"5min":1.476,"15min":1.639},"last":1.028},{"interface":"front","average":{"1min":1.087,"5min":1.0149999999999999,"15min":1.052},"min":{"1min":0.83299999999999996,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.4730000000000001,"5min":1.6890000000000001,"15min":3.0259999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.127,"15min":1.1879999999999999},"min":{"1min":0.96999999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4610000000000001,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0289999999999999,"15min":1.169},"min":{"1min":0.83999999999999997,"5min":0.505,"15min":0.49299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":4.8760000000000003},"last":1.095}]},{"osd":4,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1160000000000001,"15min":1.2350000000000001},"min":{"1min":0.92100000000000004,"5min":0.70099999999999996,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.6220000000000001,"15min":2.9449999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.28,"15min":1.3260000000000001},"min":{"1min":1.1120000000000001,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.615,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.2969999999999999,"15min":1.3859999999999999},"min":{"1min":1.1679999999999999,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.27,"15min":1.367},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.4850000000000001,"5min":1.6970000000000001,"15min":3.0249999999999999},"last":1.212}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.333,"15min":1.462},"min":{"1min":1.1830000000000001,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":4.3819999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.4119999999999999},"min":{"1min":0.995,"5min":0.86399999999999999,"15min":0.81000000000000005},"max":{"1min":1.6419999999999999,"5min":1.77,"15min":2.9910000000000001},"last":1.27}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.385,"15min":1.4770000000000001},"min":{"1min":1.0900000000000001,"5min":0.82099999999999995,"15min":0.77100000000000002},"max":{"1min":1.8380000000000001,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.3919999999999999,"15min":1.5189999999999999},"min":{"1min":1.2310000000000001,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":2.028,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.296}]}]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1551,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193760,"statfs":{"total":200000000,"available":198410777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.84099999999999997,"15min":0.995},"min":{"1min":0.66100000000000003,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":4.8070000000000004},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.91700000000000004,"15min":1.2290000000000001},"min":{"1min":0.72899999999999998,"5min":0.34200000000000003,"15min":0.29199999999999998},"max":{"1min":1.29,"5min":1.6839999999999999,"15min":10.816000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94299999999999995,"15min":1.0660000000000001},"min":{"1min":0.79100000000000004,"5min":0.41099999999999998,"15min":0.32700000000000001},"max":{"1min":1.2450000000000001,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":1.288},{"interface":"front","average":{"1min":1.006,"5min":0.97999999999999998,"15min":1.2509999999999999},"min":{"1min":0.75,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":1.304,"5min":1.7,"15min":10.781000000000001},"last":1.071}]},{"osd":3,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93799999999999994,"15min":1.0980000000000001},"min":{"1min":0.81200000000000006,"5min":0.379,"15min":0.379},"max":{"1min":1.3999999999999999,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":1.109},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0389999999999999,"15min":1.1930000000000001},"min":{"1min":0.88100000000000001,"5min":0.437,"15min":0.36699999999999999},"max":{"1min":1.3600000000000001,"5min":1.6459999999999999,"15min":4.9530000000000003},"last":1.22}]},{"osd":4,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1950000000000001,"15min":1.2749999999999999},"min":{"1min":0.95299999999999996,"5min":0.67000000000000004,"15min":0.66800000000000004},"max":{"1min":1.542,"5min":2.129,"15min":3.669},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.25,"15min":1.339},"min":{"1min":1.0580000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.506,"5min":1.895,"15min":4.8780000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.5780000000000001},"min":{"1min":1.024,"5min":0.74299999999999999,"15min":0.72599999999999998},"max":{"1min":1.407,"5min":2.0150000000000001,"15min":10.651},"last":1.355},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.3480000000000001},"min":{"1min":1.105,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.55,"5min":1.954,"15min":3.7869999999999999},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.407},"min":{"1min":1.1579999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"max":{"1min":1.522,"5min":1.764,"15min":4.9210000000000003},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.321,"15min":1.613},"min":{"1min":1.1160000000000001,"5min":0.79400000000000004,"15min":0.76900000000000002},"max":{"1min":1.95,"5min":2.0510000000000002,"15min":10.708},"last":1.306}]},{"osd":7,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.377,"15min":1.647},"min":{"1min":1.1919999999999999,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":2.0579999999999998,"5min":2.3010000000000002,"15min":10.74},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.321,"15min":1.3879999999999999},"min":{"1min":1.0960000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.625,"5min":1.9159999999999999,"15min":3.863},"last":1.244}]}]}],"pool_statfs":[]}} 2024-02-02T13:35:37.067 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:35:37.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.314+0000 7f9f52600640 1 Processor -- start 2024-02-02T13:35:37.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.314+0000 7f9f52600640 1 -- start start 2024-02-02T13:35:37.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f52600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f4c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f52600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c15fa00 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f52600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c06c600 con 0x7f9f4c151f90 2024-02-02T13:35:37.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f52600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0b10d0 con 0x7f9f4c15f5a0 2024-02-02T13:35:37.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f4be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9f4c0b0c40 0x7f9f4c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46726/0 (socket says 172.21.15.91:46726) 2024-02-02T13:35:37.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f4be00640 1 -- 172.21.15.91:0/2936166912 learned_addr learned my addr 172.21.15.91:0/2936166912 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f43e00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f4c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.316+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303424461 0 0) 0x7f9f44001490 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c003520 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109370949 0 0) 0x7f9f2c003520 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 msgr2=0x7f9f4c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f4c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 >> v1:172.21.15.113:6789/0 conn(0x7f9f4c151f90 legacy=0x7f9f4c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c070f80 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f9f44001b80 con 0x7f9f4c0b0c40 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 >> v1:172.21.15.91:6789/0 conn(0x7f9f4c0b0c40 legacy=0x7f9f4c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f2c0039a0 0x7f9f2c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2c0047c0 0x7f9f2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0b10d0 con 0x7f9f2c0039a0 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00ba60 con 0x7f9f2c0047c0 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00bc20 con 0x7f9f4c15f5a0 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.317+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f2c0039a0 0x7f9f2c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4be00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f43e00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2c0047c0 0x7f9f2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4b400640 1 -- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 msgr2=0x7f9f2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f4c15f5a0 0x7f9f2c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4b400640 1 -- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2c0047c0 msgr2=0x7f9f2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2c0047c0 0x7f9f2c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f4b400640 1 -- 172.21.15.91:0/2936166912 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c003520 con 0x7f9f2c0039a0 2024-02-02T13:35:37.322 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.318+0000 7f9f43e00640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2c0047c0 0x7f9f2c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.319+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f2c0039a0 0x7f9f2c003dc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c001720 tx=0x7f9f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607adf67c3d835a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.319+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0052d0 con 0x7f9f2c0039a0 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.319+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c02e050 con 0x7f9f2c0039a0 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.319+0000 7f9f4aa00640 1 -- 172.21.15.91:0/2936166912 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0035f0 con 0x7f9f2c0039a0 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 -- 172.21.15.91:0/2936166912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f2c0039a0 msgr2=0x7f9f2c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 --2- 172.21.15.91:0/2936166912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f2c0039a0 0x7f9f2c003dc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c001720 tx=0x7f9f2c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f4be00640 1 -- 172.21.15.91:0/2936166912 reap_dead start 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 -- 172.21.15.91:0/2936166912 shutdown_connections 2024-02-02T13:35:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 -- 172.21.15.91:0/2936166912 >> 172.21.15.91:0/2936166912 conn(0x7f9f4c0cec80 msgr2=0x7f9f4c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 -- 172.21.15.91:0/2936166912 shutdown_connections 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.320+0000 7f9f52600640 1 -- 172.21.15.91:0/2936166912 wait complete. 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.321+0000 7f9f52600640 1 Processor -- start 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 -- start start 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 0x7f9f4c150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f4c0c3f60 0x7f9f4c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c153270 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0c3480 con 0x7f9f4c0c3f60 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.322+0000 7f9f52600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0c3760 con 0x7f9f2c0039a0 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41126/0 (socket says 172.21.15.91:41126) 2024-02-02T13:35:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 -- 172.21.15.91:0/2108144462 learned_addr learned my addr 172.21.15.91:0/2108144462 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 0x7f9f4c150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 -- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 msgr2=0x7f9f4c150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 0x7f9f4c150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 -- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f4c0c3f60 msgr2=0x7f9f4c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f4c0c3f60 0x7f9f4c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4be00640 1 -- 172.21.15.91:0/2108144462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0c3a40 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.323+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 0x7f9f4c150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f4be00640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ce9d0 tx=0x7f9f44007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400eaa0 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400ec20 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c150970 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013420 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.324+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c0c6d10 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.325+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c0c32e0 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.326+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9f44013580 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.327+0000 7f9f43400640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 0x7f9f2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.327+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f44088ab0 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.327+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 0x7f9f2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.328+0000 7f9f4b400640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 0x7f9f2406c8f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c033000 tx=0x7f9f3c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.331+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9f4c0c32e0 con 0x7f9f4c0c43c0 2024-02-02T13:35:37.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.500+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f4c0cb010 con 0x7f9f2406a430 2024-02-02T13:35:37.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.513+0000 7f9f43400640 1 -- 172.21.15.91:0/2108144462 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283007 (secure 0 0 0) 0x7f9f3c0054c0 con 0x7f9f2406a430 2024-02-02T13:35:37.518 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:37.520 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:35:37.521 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 msgr2=0x7f9f2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 0x7f9f2406c8f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c033000 tx=0x7f9f3c009030 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 msgr2=0x7f9f4c0cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ce9d0 tx=0x7f9f44007b90 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 shutdown_connections 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9f2406a430 0x7f9f2406c8f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9f4c0c43c0 0x7f9f4c0cba70 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f4c0c3f60 0x7f9f4c0cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 --2- 172.21.15.91:0/2108144462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f2c0039a0 0x7f9f4c150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 >> 172.21.15.91:0/2108144462 conn(0x7f9f4c0cec80 msgr2=0x7f9f4c0ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 shutdown_connections 2024-02-02T13:35:37.522 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.518+0000 7f9f52600640 1 -- 172.21.15.91:0/2108144462 wait complete. 2024-02-02T13:35:37.536 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-02-02T13:35:36.459119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8952,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553547,"statfs":{"total":1600000000,"available":1590832408,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.313654"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737783+0000","last_change":"2024-02-02T13:35:32.656558+0000","last_active":"2024-02-02T13:35:33.737783+0000","last_peered":"2024-02-02T13:35:33.737783+0000","last_clean":"2024-02-02T13:35:33.737783+0000","last_became_active":"2024-02-02T13:35:32.656311+0000","last_became_peered":"2024-02-02T13:35:32.656311+0000","last_unstale":"2024-02-02T13:35:33.737783+0000","last_undegraded":"2024-02-02T13:35:33.737783+0000","last_fullsized":"2024-02-02T13:35:33.737783+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:44.636889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646372+0000","last_change":"2024-02-02T13:35:32.653078+0000","last_active":"2024-02-02T13:35:33.646372+0000","last_peered":"2024-02-02T13:35:33.646372+0000","last_clean":"2024-02-02T13:35:33.646372+0000","last_became_active":"2024-02-02T13:35:32.652847+0000","last_became_peered":"2024-02-02T13:35:32.652847+0000","last_unstale":"2024-02-02T13:35:33.646372+0000","last_undegraded":"2024-02-02T13:35:33.646372+0000","last_fullsized":"2024-02-02T13:35:33.646372+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:44:42.753953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740414+0000","last_change":"2024-02-02T13:35:32.671785+0000","last_active":"2024-02-02T13:35:33.740414+0000","last_peered":"2024-02-02T13:35:33.740414+0000","last_clean":"2024-02-02T13:35:33.740414+0000","last_became_active":"2024-02-02T13:35:32.669976+0000","last_became_peered":"2024-02-02T13:35:32.669976+0000","last_unstale":"2024-02-02T13:35:33.740414+0000","last_undegraded":"2024-02-02T13:35:33.740414+0000","last_fullsized":"2024-02-02T13:35:33.740414+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:13.088957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739256+0000","last_change":"2024-02-02T13:35:32.676634+0000","last_active":"2024-02-02T13:35:33.739256+0000","last_peered":"2024-02-02T13:35:33.739256+0000","last_clean":"2024-02-02T13:35:33.739256+0000","last_became_active":"2024-02-02T13:35:32.674092+0000","last_became_peered":"2024-02-02T13:35:32.674092+0000","last_unstale":"2024-02-02T13:35:33.739256+0000","last_undegraded":"2024-02-02T13:35:33.739256+0000","last_fullsized":"2024-02-02T13:35:33.739256+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:36:21.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741108+0000","last_change":"2024-02-02T13:35:32.672181+0000","last_active":"2024-02-02T13:35:33.741108+0000","last_peered":"2024-02-02T13:35:33.741108+0000","last_clean":"2024-02-02T13:35:33.741108+0000","last_became_active":"2024-02-02T13:35:32.671797+0000","last_became_peered":"2024-02-02T13:35:32.671797+0000","last_unstale":"2024-02-02T13:35:33.741108+0000","last_undegraded":"2024-02-02T13:35:33.741108+0000","last_fullsized":"2024-02-02T13:35:33.741108+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:59.421530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737943+0000","last_change":"2024-02-02T13:35:32.666665+0000","last_active":"2024-02-02T13:35:33.737943+0000","last_peered":"2024-02-02T13:35:33.737943+0000","last_clean":"2024-02-02T13:35:33.737943+0000","last_became_active":"2024-02-02T13:35:32.666375+0000","last_became_peered":"2024-02-02T13:35:32.666375+0000","last_unstale":"2024-02-02T13:35:33.737943+0000","last_undegraded":"2024-02-02T13:35:33.737943+0000","last_fullsized":"2024-02-02T13:35:33.737943+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:23:41.531817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650127+0000","last_change":"2024-02-02T13:35:32.673295+0000","last_active":"2024-02-02T13:35:33.650127+0000","last_peered":"2024-02-02T13:35:33.650127+0000","last_clean":"2024-02-02T13:35:33.650127+0000","last_became_active":"2024-02-02T13:35:32.673031+0000","last_became_peered":"2024-02-02T13:35:32.673031+0000","last_unstale":"2024-02-02T13:35:33.650127+0000","last_undegraded":"2024-02-02T13:35:33.650127+0000","last_fullsized":"2024-02-02T13:35:33.650127+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:54.540869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737455+0000","last_change":"2024-02-02T13:35:32.662799+0000","last_active":"2024-02-02T13:35:33.737455+0000","last_peered":"2024-02-02T13:35:33.737455+0000","last_clean":"2024-02-02T13:35:33.737455+0000","last_became_active":"2024-02-02T13:35:32.662582+0000","last_became_peered":"2024-02-02T13:35:32.662582+0000","last_unstale":"2024-02-02T13:35:33.737455+0000","last_undegraded":"2024-02-02T13:35:33.737455+0000","last_fullsized":"2024-02-02T13:35:33.737455+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:46:45.138936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646332+0000","last_change":"2024-02-02T13:35:32.649195+0000","last_active":"2024-02-02T13:35:33.646332+0000","last_peered":"2024-02-02T13:35:33.646332+0000","last_clean":"2024-02-02T13:35:33.646332+0000","last_became_active":"2024-02-02T13:35:32.648951+0000","last_became_peered":"2024-02-02T13:35:32.648951+0000","last_unstale":"2024-02-02T13:35:33.646332+0000","last_undegraded":"2024-02-02T13:35:33.646332+0000","last_fullsized":"2024-02-02T13:35:33.646332+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:26:13.969913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743034+0000","last_change":"2024-02-02T13:35:32.675835+0000","last_active":"2024-02-02T13:35:33.743034+0000","last_peered":"2024-02-02T13:35:33.743034+0000","last_clean":"2024-02-02T13:35:33.743034+0000","last_became_active":"2024-02-02T13:35:32.675606+0000","last_became_peered":"2024-02-02T13:35:32.675606+0000","last_unstale":"2024-02-02T13:35:33.743034+0000","last_undegraded":"2024-02-02T13:35:33.743034+0000","last_fullsized":"2024-02-02T13:35:33.743034+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:45.527941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737442+0000","last_change":"2024-02-02T13:35:32.660393+0000","last_active":"2024-02-02T13:35:33.737442+0000","last_peered":"2024-02-02T13:35:33.737442+0000","last_clean":"2024-02-02T13:35:33.737442+0000","last_became_active":"2024-02-02T13:35:32.659471+0000","last_became_peered":"2024-02-02T13:35:32.659471+0000","last_unstale":"2024-02-02T13:35:33.737442+0000","last_undegraded":"2024-02-02T13:35:33.737442+0000","last_fullsized":"2024-02-02T13:35:33.737442+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:51:59.544691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741491+0000","last_change":"2024-02-02T13:35:32.666027+0000","last_active":"2024-02-02T13:35:33.741491+0000","last_peered":"2024-02-02T13:35:33.741491+0000","last_clean":"2024-02-02T13:35:33.741491+0000","last_became_active":"2024-02-02T13:35:32.665772+0000","last_became_peered":"2024-02-02T13:35:32.665772+0000","last_unstale":"2024-02-02T13:35:33.741491+0000","last_undegraded":"2024-02-02T13:35:33.741491+0000","last_fullsized":"2024-02-02T13:35:33.741491+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:08:34.055868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739366+0000","last_change":"2024-02-02T13:35:32.669229+0000","last_active":"2024-02-02T13:35:33.739366+0000","last_peered":"2024-02-02T13:35:33.739366+0000","last_clean":"2024-02-02T13:35:33.739366+0000","last_became_active":"2024-02-02T13:35:32.667517+0000","last_became_peered":"2024-02-02T13:35:32.667517+0000","last_unstale":"2024-02-02T13:35:33.739366+0000","last_undegraded":"2024-02-02T13:35:33.739366+0000","last_fullsized":"2024-02-02T13:35:33.739366+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:06.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739518+0000","last_change":"2024-02-02T13:35:32.677617+0000","last_active":"2024-02-02T13:35:33.739518+0000","last_peered":"2024-02-02T13:35:33.739518+0000","last_clean":"2024-02-02T13:35:33.739518+0000","last_became_active":"2024-02-02T13:35:32.675768+0000","last_became_peered":"2024-02-02T13:35:32.675768+0000","last_unstale":"2024-02-02T13:35:33.739518+0000","last_undegraded":"2024-02-02T13:35:33.739518+0000","last_fullsized":"2024-02-02T13:35:33.739518+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:13.088957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743155+0000","last_change":"2024-02-02T13:35:32.668729+0000","last_active":"2024-02-02T13:35:33.743155+0000","last_peered":"2024-02-02T13:35:33.743155+0000","last_clean":"2024-02-02T13:35:33.743155+0000","last_became_active":"2024-02-02T13:35:32.668404+0000","last_became_peered":"2024-02-02T13:35:32.668404+0000","last_unstale":"2024-02-02T13:35:33.743155+0000","last_undegraded":"2024-02-02T13:35:33.743155+0000","last_fullsized":"2024-02-02T13:35:33.743155+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:24.955654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651967+0000","last_change":"2024-02-02T13:35:32.661526+0000","last_active":"2024-02-02T13:35:33.651967+0000","last_peered":"2024-02-02T13:35:33.651967+0000","last_clean":"2024-02-02T13:35:33.651967+0000","last_became_active":"2024-02-02T13:35:32.661207+0000","last_became_peered":"2024-02-02T13:35:32.661207+0000","last_unstale":"2024-02-02T13:35:33.651967+0000","last_undegraded":"2024-02-02T13:35:33.651967+0000","last_fullsized":"2024-02-02T13:35:33.651967+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:35.726879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742141+0000","last_change":"2024-02-02T13:35:32.660668+0000","last_active":"2024-02-02T13:35:33.742141+0000","last_peered":"2024-02-02T13:35:33.742141+0000","last_clean":"2024-02-02T13:35:33.742141+0000","last_became_active":"2024-02-02T13:35:32.660158+0000","last_became_peered":"2024-02-02T13:35:32.660158+0000","last_unstale":"2024-02-02T13:35:33.742141+0000","last_undegraded":"2024-02-02T13:35:33.742141+0000","last_fullsized":"2024-02-02T13:35:33.742141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:01.141500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740182+0000","last_change":"2024-02-02T13:35:32.671950+0000","last_active":"2024-02-02T13:35:33.740182+0000","last_peered":"2024-02-02T13:35:33.740182+0000","last_clean":"2024-02-02T13:35:33.740182+0000","last_became_active":"2024-02-02T13:35:32.671113+0000","last_became_peered":"2024-02-02T13:35:32.671113+0000","last_unstale":"2024-02-02T13:35:33.740182+0000","last_undegraded":"2024-02-02T13:35:33.740182+0000","last_fullsized":"2024-02-02T13:35:33.740182+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:05:05.235382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737712+0000","last_change":"2024-02-02T13:35:32.659370+0000","last_active":"2024-02-02T13:35:33.737712+0000","last_peered":"2024-02-02T13:35:33.737712+0000","last_clean":"2024-02-02T13:35:33.737712+0000","last_became_active":"2024-02-02T13:35:32.658767+0000","last_became_peered":"2024-02-02T13:35:32.658767+0000","last_unstale":"2024-02-02T13:35:33.737712+0000","last_undegraded":"2024-02-02T13:35:33.737712+0000","last_fullsized":"2024-02-02T13:35:33.737712+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:39:15.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651939+0000","last_change":"2024-02-02T13:35:32.661505+0000","last_active":"2024-02-02T13:35:33.651939+0000","last_peered":"2024-02-02T13:35:33.651939+0000","last_clean":"2024-02-02T13:35:33.651939+0000","last_became_active":"2024-02-02T13:35:32.660604+0000","last_became_peered":"2024-02-02T13:35:32.660604+0000","last_unstale":"2024-02-02T13:35:33.651939+0000","last_undegraded":"2024-02-02T13:35:33.651939+0000","last_fullsized":"2024-02-02T13:35:33.651939+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:12:50.261878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742050+0000","last_change":"2024-02-02T13:35:32.678267+0000","last_active":"2024-02-02T13:35:33.742050+0000","last_peered":"2024-02-02T13:35:33.742050+0000","last_clean":"2024-02-02T13:35:33.742050+0000","last_became_active":"2024-02-02T13:35:32.678033+0000","last_became_peered":"2024-02-02T13:35:32.678033+0000","last_unstale":"2024-02-02T13:35:33.742050+0000","last_undegraded":"2024-02-02T13:35:33.742050+0000","last_fullsized":"2024-02-02T13:35:33.742050+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:25:06.259445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740397+0000","last_change":"2024-02-02T13:35:32.672458+0000","last_active":"2024-02-02T13:35:33.740397+0000","last_peered":"2024-02-02T13:35:33.740397+0000","last_clean":"2024-02-02T13:35:33.740397+0000","last_became_active":"2024-02-02T13:35:32.671916+0000","last_became_peered":"2024-02-02T13:35:32.671916+0000","last_unstale":"2024-02-02T13:35:33.740397+0000","last_undegraded":"2024-02-02T13:35:33.740397+0000","last_fullsized":"2024-02-02T13:35:33.740397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:22.095504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739103+0000","last_change":"2024-02-02T13:35:32.671447+0000","last_active":"2024-02-02T13:35:33.739103+0000","last_peered":"2024-02-02T13:35:33.739103+0000","last_clean":"2024-02-02T13:35:33.739103+0000","last_became_active":"2024-02-02T13:35:32.670505+0000","last_became_peered":"2024-02-02T13:35:32.670505+0000","last_unstale":"2024-02-02T13:35:33.739103+0000","last_undegraded":"2024-02-02T13:35:33.739103+0000","last_fullsized":"2024-02-02T13:35:33.739103+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:00.094279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740420+0000","last_change":"2024-02-02T13:35:32.675192+0000","last_active":"2024-02-02T13:35:33.740420+0000","last_peered":"2024-02-02T13:35:33.740420+0000","last_clean":"2024-02-02T13:35:33.740420+0000","last_became_active":"2024-02-02T13:35:32.674893+0000","last_became_peered":"2024-02-02T13:35:32.674893+0000","last_unstale":"2024-02-02T13:35:33.740420+0000","last_undegraded":"2024-02-02T13:35:33.740420+0000","last_fullsized":"2024-02-02T13:35:33.740420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:55:28.365157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.652209+0000","last_change":"2024-02-02T13:35:32.662058+0000","last_active":"2024-02-02T13:35:33.652209+0000","last_peered":"2024-02-02T13:35:33.652209+0000","last_clean":"2024-02-02T13:35:33.652209+0000","last_became_active":"2024-02-02T13:35:32.660893+0000","last_became_peered":"2024-02-02T13:35:32.660893+0000","last_unstale":"2024-02-02T13:35:33.652209+0000","last_undegraded":"2024-02-02T13:35:33.652209+0000","last_fullsized":"2024-02-02T13:35:33.652209+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:09:22.158420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737559+0000","last_change":"2024-02-02T13:35:32.660848+0000","last_active":"2024-02-02T13:35:33.737559+0000","last_peered":"2024-02-02T13:35:33.737559+0000","last_clean":"2024-02-02T13:35:33.737559+0000","last_became_active":"2024-02-02T13:35:32.660179+0000","last_became_peered":"2024-02-02T13:35:32.660179+0000","last_unstale":"2024-02-02T13:35:33.737559+0000","last_undegraded":"2024-02-02T13:35:33.737559+0000","last_fullsized":"2024-02-02T13:35:33.737559+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:56:17.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651734+0000","last_change":"2024-02-02T13:35:32.663063+0000","last_active":"2024-02-02T13:35:33.651734+0000","last_peered":"2024-02-02T13:35:33.651734+0000","last_clean":"2024-02-02T13:35:33.651734+0000","last_became_active":"2024-02-02T13:35:32.662828+0000","last_became_peered":"2024-02-02T13:35:32.662828+0000","last_unstale":"2024-02-02T13:35:33.651734+0000","last_undegraded":"2024-02-02T13:35:33.651734+0000","last_fullsized":"2024-02-02T13:35:33.651734+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:37:34.989315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650112+0000","last_change":"2024-02-02T13:35:32.672427+0000","last_active":"2024-02-02T13:35:33.650112+0000","last_peered":"2024-02-02T13:35:33.650112+0000","last_clean":"2024-02-02T13:35:33.650112+0000","last_became_active":"2024-02-02T13:35:32.672210+0000","last_became_peered":"2024-02-02T13:35:32.672210+0000","last_unstale":"2024-02-02T13:35:33.650112+0000","last_undegraded":"2024-02-02T13:35:33.650112+0000","last_fullsized":"2024-02-02T13:35:33.650112+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:50:24.955654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737490+0000","last_change":"2024-02-02T13:35:32.659833+0000","last_active":"2024-02-02T13:35:33.737490+0000","last_peered":"2024-02-02T13:35:33.737490+0000","last_clean":"2024-02-02T13:35:33.737490+0000","last_became_active":"2024-02-02T13:35:32.659623+0000","last_became_peered":"2024-02-02T13:35:32.659623+0000","last_unstale":"2024-02-02T13:35:33.737490+0000","last_undegraded":"2024-02-02T13:35:33.737490+0000","last_fullsized":"2024-02-02T13:35:33.737490+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:55.838868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742077+0000","last_change":"2024-02-02T13:35:32.674732+0000","last_active":"2024-02-02T13:35:33.742077+0000","last_peered":"2024-02-02T13:35:33.742077+0000","last_clean":"2024-02-02T13:35:33.742077+0000","last_became_active":"2024-02-02T13:35:32.672794+0000","last_became_peered":"2024-02-02T13:35:32.672794+0000","last_unstale":"2024-02-02T13:35:33.742077+0000","last_undegraded":"2024-02-02T13:35:33.742077+0000","last_fullsized":"2024-02-02T13:35:33.742077+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:43:01.729452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646357+0000","last_change":"2024-02-02T13:35:32.663491+0000","last_active":"2024-02-02T13:35:33.646357+0000","last_peered":"2024-02-02T13:35:33.646357+0000","last_clean":"2024-02-02T13:35:33.646357+0000","last_became_active":"2024-02-02T13:35:32.663331+0000","last_became_peered":"2024-02-02T13:35:32.663331+0000","last_unstale":"2024-02-02T13:35:33.646357+0000","last_undegraded":"2024-02-02T13:35:33.646357+0000","last_fullsized":"2024-02-02T13:35:33.646357+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:38:01.421691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651824+0000","last_change":"2024-02-02T13:35:32.656809+0000","last_active":"2024-02-02T13:35:33.651824+0000","last_peered":"2024-02-02T13:35:33.651824+0000","last_clean":"2024-02-02T13:35:33.651824+0000","last_became_active":"2024-02-02T13:35:32.656586+0000","last_became_peered":"2024-02-02T13:35:32.656586+0000","last_unstale":"2024-02-02T13:35:33.651824+0000","last_undegraded":"2024-02-02T13:35:33.651824+0000","last_fullsized":"2024-02-02T13:35:33.651824+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:51:06.185014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740685+0000","last_change":"2024-02-02T13:35:32.666330+0000","last_active":"2024-02-02T13:35:33.740685+0000","last_peered":"2024-02-02T13:35:33.740685+0000","last_clean":"2024-02-02T13:35:33.740685+0000","last_became_active":"2024-02-02T13:35:32.665875+0000","last_became_peered":"2024-02-02T13:35:32.665875+0000","last_unstale":"2024-02-02T13:35:33.740685+0000","last_undegraded":"2024-02-02T13:35:33.740685+0000","last_fullsized":"2024-02-02T13:35:33.740685+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:39.061381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"177'4","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677320+0000","last_change":"2024-02-02T13:35:32.184481+0000","last_active":"2024-02-02T13:35:32.677320+0000","last_peered":"2024-02-02T13:35:32.677320+0000","last_clean":"2024-02-02T13:35:32.677320+0000","last_became_active":"2024-02-02T13:35:32.183868+0000","last_became_peered":"2024-02-02T13:35:32.183868+0000","last_unstale":"2024-02-02T13:35:32.677320+0000","last_undegraded":"2024-02-02T13:35:32.677320+0000","last_fullsized":"2024-02-02T13:35:32.677320+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:35:33.209478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646581+0000","last_change":"2024-02-02T13:35:32.656609+0000","last_active":"2024-02-02T13:35:33.646581+0000","last_peered":"2024-02-02T13:35:33.646581+0000","last_clean":"2024-02-02T13:35:33.646581+0000","last_became_active":"2024-02-02T13:35:32.656393+0000","last_became_peered":"2024-02-02T13:35:32.656393+0000","last_unstale":"2024-02-02T13:35:33.646581+0000","last_undegraded":"2024-02-02T13:35:33.646581+0000","last_fullsized":"2024-02-02T13:35:33.646581+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:48:25.655612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.667120+0000","last_change":"2024-02-02T13:35:32.183471+0000","last_active":"2024-02-02T13:35:32.667120+0000","last_peered":"2024-02-02T13:35:32.667120+0000","last_clean":"2024-02-02T13:35:32.667120+0000","last_became_active":"2024-02-02T13:35:32.182955+0000","last_became_peered":"2024-02-02T13:35:32.182955+0000","last_unstale":"2024-02-02T13:35:32.667120+0000","last_undegraded":"2024-02-02T13:35:32.667120+0000","last_fullsized":"2024-02-02T13:35:32.667120+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:21.080419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737677+0000","last_change":"2024-02-02T13:35:32.676683+0000","last_active":"2024-02-02T13:35:33.737677+0000","last_peered":"2024-02-02T13:35:33.737677+0000","last_clean":"2024-02-02T13:35:33.737677+0000","last_became_active":"2024-02-02T13:35:32.676450+0000","last_became_peered":"2024-02-02T13:35:32.676450+0000","last_unstale":"2024-02-02T13:35:33.737677+0000","last_undegraded":"2024-02-02T13:35:33.737677+0000","last_fullsized":"2024-02-02T13:35:33.737677+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:02:59.880675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.672096+0000","last_change":"2024-02-02T13:35:32.179654+0000","last_active":"2024-02-02T13:35:32.672096+0000","last_peered":"2024-02-02T13:35:32.672096+0000","last_clean":"2024-02-02T13:35:32.672096+0000","last_became_active":"2024-02-02T13:35:32.179207+0000","last_became_peered":"2024-02-02T13:35:32.179207+0000","last_unstale":"2024-02-02T13:35:32.672096+0000","last_undegraded":"2024-02-02T13:35:32.672096+0000","last_fullsized":"2024-02-02T13:35:32.672096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:57.846865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743281+0000","last_change":"2024-02-02T13:35:32.672162+0000","last_active":"2024-02-02T13:35:33.743281+0000","last_peered":"2024-02-02T13:35:33.743281+0000","last_clean":"2024-02-02T13:35:33.743281+0000","last_became_active":"2024-02-02T13:35:32.671945+0000","last_became_peered":"2024-02-02T13:35:32.671945+0000","last_unstale":"2024-02-02T13:35:33.743281+0000","last_undegraded":"2024-02-02T13:35:33.743281+0000","last_fullsized":"2024-02-02T13:35:33.743281+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:57:54.540869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677053+0000","last_change":"2024-02-02T13:35:32.167390+0000","last_active":"2024-02-02T13:35:32.677053+0000","last_peered":"2024-02-02T13:35:32.677053+0000","last_clean":"2024-02-02T13:35:32.677053+0000","last_became_active":"2024-02-02T13:35:32.165528+0000","last_became_peered":"2024-02-02T13:35:32.165528+0000","last_unstale":"2024-02-02T13:35:32.677053+0000","last_undegraded":"2024-02-02T13:35:32.677053+0000","last_fullsized":"2024-02-02T13:35:32.677053+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:09:58.406445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646819+0000","last_change":"2024-02-02T13:35:32.675839+0000","last_active":"2024-02-02T13:35:33.646819+0000","last_peered":"2024-02-02T13:35:33.646819+0000","last_clean":"2024-02-02T13:35:33.646819+0000","last_became_active":"2024-02-02T13:35:32.675001+0000","last_became_peered":"2024-02-02T13:35:32.675001+0000","last_unstale":"2024-02-02T13:35:33.646819+0000","last_undegraded":"2024-02-02T13:35:33.646819+0000","last_fullsized":"2024-02-02T13:35:33.646819+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:52.932905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671293+0000","last_change":"2024-02-02T13:35:32.182929+0000","last_active":"2024-02-02T13:35:32.671293+0000","last_peered":"2024-02-02T13:35:32.671293+0000","last_clean":"2024-02-02T13:35:32.671293+0000","last_became_active":"2024-02-02T13:35:32.182736+0000","last_became_peered":"2024-02-02T13:35:32.182736+0000","last_unstale":"2024-02-02T13:35:32.671293+0000","last_undegraded":"2024-02-02T13:35:32.671293+0000","last_fullsized":"2024-02-02T13:35:32.671293+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:13:44.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740652+0000","last_change":"2024-02-02T13:35:32.672317+0000","last_active":"2024-02-02T13:35:33.740652+0000","last_peered":"2024-02-02T13:35:33.740652+0000","last_clean":"2024-02-02T13:35:33.740652+0000","last_became_active":"2024-02-02T13:35:32.671670+0000","last_became_peered":"2024-02-02T13:35:32.671670+0000","last_unstale":"2024-02-02T13:35:33.740652+0000","last_undegraded":"2024-02-02T13:35:33.740652+0000","last_fullsized":"2024-02-02T13:35:33.740652+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:53.911546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.675236+0000","last_change":"2024-02-02T13:35:32.182177+0000","last_active":"2024-02-02T13:35:32.675236+0000","last_peered":"2024-02-02T13:35:32.675236+0000","last_clean":"2024-02-02T13:35:32.675236+0000","last_became_active":"2024-02-02T13:35:32.181943+0000","last_became_peered":"2024-02-02T13:35:32.181943+0000","last_unstale":"2024-02-02T13:35:32.675236+0000","last_undegraded":"2024-02-02T13:35:32.675236+0000","last_fullsized":"2024-02-02T13:35:32.675236+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:21:58.025523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646663+0000","last_change":"2024-02-02T13:35:32.656320+0000","last_active":"2024-02-02T13:35:33.646663+0000","last_peered":"2024-02-02T13:35:33.646663+0000","last_clean":"2024-02-02T13:35:33.646663+0000","last_became_active":"2024-02-02T13:35:32.656079+0000","last_became_peered":"2024-02-02T13:35:32.656079+0000","last_unstale":"2024-02-02T13:35:33.646663+0000","last_undegraded":"2024-02-02T13:35:33.646663+0000","last_fullsized":"2024-02-02T13:35:33.646663+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:12:54.250935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.653829+0000","last_change":"2024-02-02T13:35:32.176176+0000","last_active":"2024-02-02T13:35:32.653829+0000","last_peered":"2024-02-02T13:35:32.653829+0000","last_clean":"2024-02-02T13:35:32.653829+0000","last_became_active":"2024-02-02T13:35:32.175937+0000","last_became_peered":"2024-02-02T13:35:32.175937+0000","last_unstale":"2024-02-02T13:35:32.653829+0000","last_undegraded":"2024-02-02T13:35:32.653829+0000","last_fullsized":"2024-02-02T13:35:32.653829+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:15:07.131016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.741569+0000","last_change":"2024-02-02T13:35:32.664227+0000","last_active":"2024-02-02T13:35:33.741569+0000","last_peered":"2024-02-02T13:35:33.741569+0000","last_clean":"2024-02-02T13:35:33.741569+0000","last_became_active":"2024-02-02T13:35:32.663823+0000","last_became_peered":"2024-02-02T13:35:32.663823+0000","last_unstale":"2024-02-02T13:35:33.741569+0000","last_undegraded":"2024-02-02T13:35:33.741569+0000","last_fullsized":"2024-02-02T13:35:33.741569+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:44.636889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.668117+0000","last_change":"2024-02-02T13:35:32.182783+0000","last_active":"2024-02-02T13:35:32.668117+0000","last_peered":"2024-02-02T13:35:32.668117+0000","last_clean":"2024-02-02T13:35:32.668117+0000","last_became_active":"2024-02-02T13:35:32.180069+0000","last_became_peered":"2024-02-02T13:35:32.180069+0000","last_unstale":"2024-02-02T13:35:32.668117+0000","last_undegraded":"2024-02-02T13:35:32.668117+0000","last_fullsized":"2024-02-02T13:35:32.668117+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:43:23.793930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739411+0000","last_change":"2024-02-02T13:35:32.658537+0000","last_active":"2024-02-02T13:35:33.739411+0000","last_peered":"2024-02-02T13:35:33.739411+0000","last_clean":"2024-02-02T13:35:33.739411+0000","last_became_active":"2024-02-02T13:35:32.657037+0000","last_became_peered":"2024-02-02T13:35:32.657037+0000","last_unstale":"2024-02-02T13:35:33.739411+0000","last_undegraded":"2024-02-02T13:35:33.739411+0000","last_fullsized":"2024-02-02T13:35:33.739411+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:02:59.880675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.646704+0000","last_change":"2024-02-02T13:35:32.174760+0000","last_active":"2024-02-02T13:35:32.646704+0000","last_peered":"2024-02-02T13:35:32.646704+0000","last_clean":"2024-02-02T13:35:32.646704+0000","last_became_active":"2024-02-02T13:35:32.174521+0000","last_became_peered":"2024-02-02T13:35:32.174521+0000","last_unstale":"2024-02-02T13:35:32.646704+0000","last_undegraded":"2024-02-02T13:35:32.646704+0000","last_fullsized":"2024-02-02T13:35:32.646704+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:59.079193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742291+0000","last_change":"2024-02-02T13:35:32.658661+0000","last_active":"2024-02-02T13:35:33.742291+0000","last_peered":"2024-02-02T13:35:33.742291+0000","last_clean":"2024-02-02T13:35:33.742291+0000","last_became_active":"2024-02-02T13:35:32.657453+0000","last_became_peered":"2024-02-02T13:35:32.657453+0000","last_unstale":"2024-02-02T13:35:33.742291+0000","last_undegraded":"2024-02-02T13:35:33.742291+0000","last_fullsized":"2024-02-02T13:35:33.742291+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:55:33.191478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.649309+0000","last_change":"2024-02-02T13:35:32.185731+0000","last_active":"2024-02-02T13:35:32.649309+0000","last_peered":"2024-02-02T13:35:32.649309+0000","last_clean":"2024-02-02T13:35:32.649309+0000","last_became_active":"2024-02-02T13:35:32.184769+0000","last_became_peered":"2024-02-02T13:35:32.184769+0000","last_unstale":"2024-02-02T13:35:32.649309+0000","last_undegraded":"2024-02-02T13:35:32.649309+0000","last_fullsized":"2024-02-02T13:35:32.649309+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:56:56.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651398+0000","last_change":"2024-02-02T13:35:32.658230+0000","last_active":"2024-02-02T13:35:33.651398+0000","last_peered":"2024-02-02T13:35:33.651398+0000","last_clean":"2024-02-02T13:35:33.651398+0000","last_became_active":"2024-02-02T13:35:32.658011+0000","last_became_peered":"2024-02-02T13:35:32.658011+0000","last_unstale":"2024-02-02T13:35:33.651398+0000","last_undegraded":"2024-02-02T13:35:33.651398+0000","last_fullsized":"2024-02-02T13:35:33.651398+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:46.758785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'6","reported_seq":17,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.673002+0000","last_change":"2024-02-02T13:35:32.177602+0000","last_active":"2024-02-02T13:35:32.673002+0000","last_peered":"2024-02-02T13:35:32.673002+0000","last_clean":"2024-02-02T13:35:32.673002+0000","last_became_active":"2024-02-02T13:35:32.177171+0000","last_became_peered":"2024-02-02T13:35:32.177171+0000","last_unstale":"2024-02-02T13:35:32.673002+0000","last_undegraded":"2024-02-02T13:35:32.673002+0000","last_fullsized":"2024-02-02T13:35:32.673002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:08:25.309223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.742202+0000","last_change":"2024-02-02T13:35:32.678320+0000","last_active":"2024-02-02T13:35:33.742202+0000","last_peered":"2024-02-02T13:35:33.742202+0000","last_clean":"2024-02-02T13:35:33.742202+0000","last_became_active":"2024-02-02T13:35:32.677502+0000","last_became_peered":"2024-02-02T13:35:32.677502+0000","last_unstale":"2024-02-02T13:35:33.742202+0000","last_undegraded":"2024-02-02T13:35:33.742202+0000","last_fullsized":"2024-02-02T13:35:33.742202+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:25:00.094279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.670956+0000","last_change":"2024-02-02T13:35:32.176341+0000","last_active":"2024-02-02T13:35:32.670956+0000","last_peered":"2024-02-02T13:35:32.670956+0000","last_clean":"2024-02-02T13:35:32.670956+0000","last_became_active":"2024-02-02T13:35:32.176034+0000","last_became_peered":"2024-02-02T13:35:32.176034+0000","last_unstale":"2024-02-02T13:35:32.670956+0000","last_undegraded":"2024-02-02T13:35:32.670956+0000","last_fullsized":"2024-02-02T13:35:32.670956+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:42:12.461433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646468+0000","last_change":"2024-02-02T13:35:32.653691+0000","last_active":"2024-02-02T13:35:33.646468+0000","last_peered":"2024-02-02T13:35:33.646468+0000","last_clean":"2024-02-02T13:35:33.646468+0000","last_became_active":"2024-02-02T13:35:32.653471+0000","last_became_peered":"2024-02-02T13:35:32.653471+0000","last_unstale":"2024-02-02T13:35:33.646468+0000","last_undegraded":"2024-02-02T13:35:33.646468+0000","last_fullsized":"2024-02-02T13:35:33.646468+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:00:34.703958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.662777+0000","last_change":"2024-02-02T13:35:32.177542+0000","last_active":"2024-02-02T13:35:32.662777+0000","last_peered":"2024-02-02T13:35:32.662777+0000","last_clean":"2024-02-02T13:35:32.662777+0000","last_became_active":"2024-02-02T13:35:32.177089+0000","last_became_peered":"2024-02-02T13:35:32.177089+0000","last_unstale":"2024-02-02T13:35:32.662777+0000","last_undegraded":"2024-02-02T13:35:32.662777+0000","last_fullsized":"2024-02-02T13:35:32.662777+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:43:13.136015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739606+0000","last_change":"2024-02-02T13:35:32.670012+0000","last_active":"2024-02-02T13:35:33.739606+0000","last_peered":"2024-02-02T13:35:33.739606+0000","last_clean":"2024-02-02T13:35:33.739606+0000","last_became_active":"2024-02-02T13:35:32.668683+0000","last_became_peered":"2024-02-02T13:35:32.668683+0000","last_unstale":"2024-02-02T13:35:33.739606+0000","last_undegraded":"2024-02-02T13:35:33.739606+0000","last_fullsized":"2024-02-02T13:35:33.739606+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:39:08.847788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.642804+0000","last_change":"2024-02-02T13:35:32.177009+0000","last_active":"2024-02-02T13:35:32.642804+0000","last_peered":"2024-02-02T13:35:32.642804+0000","last_clean":"2024-02-02T13:35:32.642804+0000","last_became_active":"2024-02-02T13:35:32.176656+0000","last_became_peered":"2024-02-02T13:35:32.176656+0000","last_unstale":"2024-02-02T13:35:32.642804+0000","last_undegraded":"2024-02-02T13:35:32.642804+0000","last_fullsized":"2024-02-02T13:35:32.642804+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:01:12.927532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740358+0000","last_change":"2024-02-02T13:35:32.664108+0000","last_active":"2024-02-02T13:35:33.740358+0000","last_peered":"2024-02-02T13:35:33.740358+0000","last_clean":"2024-02-02T13:35:33.740358+0000","last_became_active":"2024-02-02T13:35:32.663461+0000","last_became_peered":"2024-02-02T13:35:32.663461+0000","last_unstale":"2024-02-02T13:35:33.740358+0000","last_undegraded":"2024-02-02T13:35:33.740358+0000","last_fullsized":"2024-02-02T13:35:33.740358+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:24:22.095504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"177'8","reported_seq":19,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671591+0000","last_change":"2024-02-02T13:35:32.163677+0000","last_active":"2024-02-02T13:35:32.671591+0000","last_peered":"2024-02-02T13:35:32.671591+0000","last_clean":"2024-02-02T13:35:32.671591+0000","last_became_active":"2024-02-02T13:35:32.163008+0000","last_became_peered":"2024-02-02T13:35:32.163008+0000","last_unstale":"2024-02-02T13:35:32.671591+0000","last_undegraded":"2024-02-02T13:35:32.671591+0000","last_fullsized":"2024-02-02T13:35:32.671591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:22:03.837325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.743400+0000","last_change":"2024-02-02T13:35:32.656040+0000","last_active":"2024-02-02T13:35:33.743400+0000","last_peered":"2024-02-02T13:35:33.743400+0000","last_clean":"2024-02-02T13:35:33.743400+0000","last_became_active":"2024-02-02T13:35:32.655430+0000","last_became_peered":"2024-02-02T13:35:32.655430+0000","last_unstale":"2024-02-02T13:35:33.743400+0000","last_undegraded":"2024-02-02T13:35:33.743400+0000","last_fullsized":"2024-02-02T13:35:33.743400+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:36:16.308158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.671502+0000","last_change":"2024-02-02T13:35:32.175607+0000","last_active":"2024-02-02T13:35:32.671502+0000","last_peered":"2024-02-02T13:35:32.671502+0000","last_clean":"2024-02-02T13:35:32.671502+0000","last_became_active":"2024-02-02T13:35:32.175215+0000","last_became_peered":"2024-02-02T13:35:32.175215+0000","last_unstale":"2024-02-02T13:35:32.671502+0000","last_undegraded":"2024-02-02T13:35:32.671502+0000","last_fullsized":"2024-02-02T13:35:32.671502+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:32:54.398787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651506+0000","last_change":"2024-02-02T13:35:32.665230+0000","last_active":"2024-02-02T13:35:33.651506+0000","last_peered":"2024-02-02T13:35:33.651506+0000","last_clean":"2024-02-02T13:35:33.651506+0000","last_became_active":"2024-02-02T13:35:32.664952+0000","last_became_peered":"2024-02-02T13:35:32.664952+0000","last_unstale":"2024-02-02T13:35:33.651506+0000","last_undegraded":"2024-02-02T13:35:33.651506+0000","last_fullsized":"2024-02-02T13:35:33.651506+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:37:34.460344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.661880+0000","last_change":"2024-02-02T13:35:32.176837+0000","last_active":"2024-02-02T13:35:32.661880+0000","last_peered":"2024-02-02T13:35:32.661880+0000","last_clean":"2024-02-02T13:35:32.661880+0000","last_became_active":"2024-02-02T13:35:32.176119+0000","last_became_peered":"2024-02-02T13:35:32.176119+0000","last_unstale":"2024-02-02T13:35:32.661880+0000","last_undegraded":"2024-02-02T13:35:32.661880+0000","last_fullsized":"2024-02-02T13:35:32.661880+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:36:12.073872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740609+0000","last_change":"2024-02-02T13:35:32.676995+0000","last_active":"2024-02-02T13:35:33.740609+0000","last_peered":"2024-02-02T13:35:33.740609+0000","last_clean":"2024-02-02T13:35:33.740609+0000","last_became_active":"2024-02-02T13:35:32.675927+0000","last_became_peered":"2024-02-02T13:35:32.675927+0000","last_unstale":"2024-02-02T13:35:33.740609+0000","last_undegraded":"2024-02-02T13:35:33.740609+0000","last_fullsized":"2024-02-02T13:35:33.740609+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:37.834948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"177'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.665164+0000","last_change":"2024-02-02T13:35:32.181529+0000","last_active":"2024-02-02T13:35:32.665164+0000","last_peered":"2024-02-02T13:35:32.665164+0000","last_clean":"2024-02-02T13:35:32.665164+0000","last_became_active":"2024-02-02T13:35:32.181311+0000","last_became_peered":"2024-02-02T13:35:32.181311+0000","last_unstale":"2024-02-02T13:35:32.665164+0000","last_undegraded":"2024-02-02T13:35:32.665164+0000","last_fullsized":"2024-02-02T13:35:32.665164+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:24.246451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.738213+0000","last_change":"2024-02-02T13:35:32.666526+0000","last_active":"2024-02-02T13:35:33.738213+0000","last_peered":"2024-02-02T13:35:33.738213+0000","last_clean":"2024-02-02T13:35:33.738213+0000","last_became_active":"2024-02-02T13:35:32.666207+0000","last_became_peered":"2024-02-02T13:35:32.666207+0000","last_unstale":"2024-02-02T13:35:33.738213+0000","last_undegraded":"2024-02-02T13:35:33.738213+0000","last_fullsized":"2024-02-02T13:35:33.738213+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:34:53.279843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.645941+0000","last_change":"2024-02-02T13:35:32.176879+0000","last_active":"2024-02-02T13:35:32.645941+0000","last_peered":"2024-02-02T13:35:32.645941+0000","last_clean":"2024-02-02T13:35:32.645941+0000","last_became_active":"2024-02-02T13:35:32.176328+0000","last_became_peered":"2024-02-02T13:35:32.176328+0000","last_unstale":"2024-02-02T13:35:32.645941+0000","last_undegraded":"2024-02-02T13:35:32.645941+0000","last_fullsized":"2024-02-02T13:35:32.645941+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:52.896461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.739890+0000","last_change":"2024-02-02T13:35:32.676016+0000","last_active":"2024-02-02T13:35:33.739890+0000","last_peered":"2024-02-02T13:35:33.739890+0000","last_clean":"2024-02-02T13:35:33.739890+0000","last_became_active":"2024-02-02T13:35:32.673614+0000","last_became_peered":"2024-02-02T13:35:32.673614+0000","last_unstale":"2024-02-02T13:35:33.739890+0000","last_undegraded":"2024-02-02T13:35:33.739890+0000","last_fullsized":"2024-02-02T13:35:33.739890+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:37.834948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.673443+0000","last_change":"2024-02-02T13:35:32.175627+0000","last_active":"2024-02-02T13:35:32.673443+0000","last_peered":"2024-02-02T13:35:32.673443+0000","last_clean":"2024-02-02T13:35:32.673443+0000","last_became_active":"2024-02-02T13:35:32.175309+0000","last_became_peered":"2024-02-02T13:35:32.175309+0000","last_unstale":"2024-02-02T13:35:32.673443+0000","last_undegraded":"2024-02-02T13:35:32.673443+0000","last_fullsized":"2024-02-02T13:35:32.673443+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:51.917820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650359+0000","last_change":"2024-02-02T13:35:32.671398+0000","last_active":"2024-02-02T13:35:33.650359+0000","last_peered":"2024-02-02T13:35:33.650359+0000","last_clean":"2024-02-02T13:35:33.650359+0000","last_became_active":"2024-02-02T13:35:32.671119+0000","last_became_peered":"2024-02-02T13:35:32.671119+0000","last_unstale":"2024-02-02T13:35:33.650359+0000","last_undegraded":"2024-02-02T13:35:33.650359+0000","last_fullsized":"2024-02-02T13:35:33.650359+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:24.760817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.656936+0000","last_change":"2024-02-02T13:35:32.183645+0000","last_active":"2024-02-02T13:35:32.656936+0000","last_peered":"2024-02-02T13:35:32.656936+0000","last_clean":"2024-02-02T13:35:32.656936+0000","last_became_active":"2024-02-02T13:35:32.183279+0000","last_became_peered":"2024-02-02T13:35:32.183279+0000","last_unstale":"2024-02-02T13:35:32.656936+0000","last_undegraded":"2024-02-02T13:35:32.656936+0000","last_fullsized":"2024-02-02T13:35:32.656936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:06.567372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740536+0000","last_change":"2024-02-02T13:35:32.677258+0000","last_active":"2024-02-02T13:35:33.740536+0000","last_peered":"2024-02-02T13:35:33.740536+0000","last_clean":"2024-02-02T13:35:33.740536+0000","last_became_active":"2024-02-02T13:35:32.676086+0000","last_became_peered":"2024-02-02T13:35:32.676086+0000","last_unstale":"2024-02-02T13:35:33.740536+0000","last_undegraded":"2024-02-02T13:35:33.740536+0000","last_fullsized":"2024-02-02T13:35:33.740536+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:00:19.807754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651466+0000","last_change":"2024-02-02T13:35:32.182624+0000","last_active":"2024-02-02T13:35:32.651466+0000","last_peered":"2024-02-02T13:35:32.651466+0000","last_clean":"2024-02-02T13:35:32.651466+0000","last_became_active":"2024-02-02T13:35:32.182141+0000","last_became_peered":"2024-02-02T13:35:32.182141+0000","last_unstale":"2024-02-02T13:35:32.651466+0000","last_undegraded":"2024-02-02T13:35:32.651466+0000","last_fullsized":"2024-02-02T13:35:32.651466+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:03:00.945896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.650031+0000","last_change":"2024-02-02T13:35:32.673871+0000","last_active":"2024-02-02T13:35:33.650031+0000","last_peered":"2024-02-02T13:35:33.650031+0000","last_clean":"2024-02-02T13:35:33.650031+0000","last_became_active":"2024-02-02T13:35:32.673492+0000","last_became_peered":"2024-02-02T13:35:32.673492+0000","last_unstale":"2024-02-02T13:35:33.650031+0000","last_undegraded":"2024-02-02T13:35:33.650031+0000","last_fullsized":"2024-02-02T13:35:33.650031+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:30:45.527941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.662875+0000","last_change":"2024-02-02T13:35:32.174725+0000","last_active":"2024-02-02T13:35:32.662875+0000","last_peered":"2024-02-02T13:35:32.662875+0000","last_clean":"2024-02-02T13:35:32.662875+0000","last_became_active":"2024-02-02T13:35:32.171494+0000","last_became_peered":"2024-02-02T13:35:32.171494+0000","last_unstale":"2024-02-02T13:35:32.662875+0000","last_undegraded":"2024-02-02T13:35:32.662875+0000","last_fullsized":"2024-02-02T13:35:32.662875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:51:14.245561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740014+0000","last_change":"2024-02-02T13:35:32.670509+0000","last_active":"2024-02-02T13:35:33.740014+0000","last_peered":"2024-02-02T13:35:33.740014+0000","last_clean":"2024-02-02T13:35:33.740014+0000","last_became_active":"2024-02-02T13:35:32.669003+0000","last_became_peered":"2024-02-02T13:35:32.669003+0000","last_unstale":"2024-02-02T13:35:33.740014+0000","last_undegraded":"2024-02-02T13:35:33.740014+0000","last_fullsized":"2024-02-02T13:35:33.740014+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:20:55.838868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"177'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.677562+0000","last_change":"2024-02-02T13:35:32.185879+0000","last_active":"2024-02-02T13:35:32.677562+0000","last_peered":"2024-02-02T13:35:32.677562+0000","last_clean":"2024-02-02T13:35:32.677562+0000","last_became_active":"2024-02-02T13:35:32.185170+0000","last_became_peered":"2024-02-02T13:35:32.185170+0000","last_unstale":"2024-02-02T13:35:32.677562+0000","last_undegraded":"2024-02-02T13:35:32.677562+0000","last_fullsized":"2024-02-02T13:35:32.677562+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:09:19.060004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740830+0000","last_change":"2024-02-02T13:35:32.676676+0000","last_active":"2024-02-02T13:35:33.740830+0000","last_peered":"2024-02-02T13:35:33.740830+0000","last_clean":"2024-02-02T13:35:33.740830+0000","last_became_active":"2024-02-02T13:35:32.675606+0000","last_became_peered":"2024-02-02T13:35:32.675606+0000","last_unstale":"2024-02-02T13:35:33.740830+0000","last_undegraded":"2024-02-02T13:35:33.740830+0000","last_fullsized":"2024-02-02T13:35:33.740830+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:13:53.911546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.643459+0000","last_change":"2024-02-02T13:35:32.180293+0000","last_active":"2024-02-02T13:35:32.643459+0000","last_peered":"2024-02-02T13:35:32.643459+0000","last_clean":"2024-02-02T13:35:32.643459+0000","last_became_active":"2024-02-02T13:35:32.180003+0000","last_became_peered":"2024-02-02T13:35:32.180003+0000","last_unstale":"2024-02-02T13:35:32.643459+0000","last_undegraded":"2024-02-02T13:35:32.643459+0000","last_fullsized":"2024-02-02T13:35:32.643459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:36:31.803734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.646568+0000","last_change":"2024-02-02T13:35:32.674845+0000","last_active":"2024-02-02T13:35:33.646568+0000","last_peered":"2024-02-02T13:35:33.646568+0000","last_clean":"2024-02-02T13:35:33.646568+0000","last_became_active":"2024-02-02T13:35:32.674553+0000","last_became_peered":"2024-02-02T13:35:32.674553+0000","last_unstale":"2024-02-02T13:35:33.646568+0000","last_undegraded":"2024-02-02T13:35:33.646568+0000","last_fullsized":"2024-02-02T13:35:33.646568+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:16:18.989067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.644102+0000","last_change":"2024-02-02T13:35:32.158525+0000","last_active":"2024-02-02T13:35:32.644102+0000","last_peered":"2024-02-02T13:35:32.644102+0000","last_clean":"2024-02-02T13:35:32.644102+0000","last_became_active":"2024-02-02T13:35:32.157852+0000","last_became_peered":"2024-02-02T13:35:32.157852+0000","last_unstale":"2024-02-02T13:35:32.644102+0000","last_undegraded":"2024-02-02T13:35:32.644102+0000","last_fullsized":"2024-02-02T13:35:32.644102+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:37:33.445259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.737568+0000","last_change":"2024-02-02T13:35:32.674201+0000","last_active":"2024-02-02T13:35:33.737568+0000","last_peered":"2024-02-02T13:35:33.737568+0000","last_clean":"2024-02-02T13:35:33.737568+0000","last_became_active":"2024-02-02T13:35:32.673964+0000","last_became_peered":"2024-02-02T13:35:32.673964+0000","last_unstale":"2024-02-02T13:35:33.737568+0000","last_undegraded":"2024-02-02T13:35:33.737568+0000","last_fullsized":"2024-02-02T13:35:33.737568+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:03.846318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.642963+0000","last_change":"2024-02-02T13:35:32.158967+0000","last_active":"2024-02-02T13:35:32.642963+0000","last_peered":"2024-02-02T13:35:32.642963+0000","last_clean":"2024-02-02T13:35:32.642963+0000","last_became_active":"2024-02-02T13:35:32.157882+0000","last_became_peered":"2024-02-02T13:35:32.157882+0000","last_unstale":"2024-02-02T13:35:32.642963+0000","last_undegraded":"2024-02-02T13:35:32.642963+0000","last_fullsized":"2024-02-02T13:35:32.642963+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:45.743700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.651907+0000","last_change":"2024-02-02T13:35:32.673294+0000","last_active":"2024-02-02T13:35:33.651907+0000","last_peered":"2024-02-02T13:35:33.651907+0000","last_clean":"2024-02-02T13:35:33.651907+0000","last_became_active":"2024-02-02T13:35:32.672913+0000","last_became_peered":"2024-02-02T13:35:32.672913+0000","last_unstale":"2024-02-02T13:35:33.651907+0000","last_undegraded":"2024-02-02T13:35:33.651907+0000","last_fullsized":"2024-02-02T13:35:33.651907+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:35:25.470301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651568+0000","last_change":"2024-02-02T13:35:32.181252+0000","last_active":"2024-02-02T13:35:32.651568+0000","last_peered":"2024-02-02T13:35:32.651568+0000","last_clean":"2024-02-02T13:35:32.651568+0000","last_became_active":"2024-02-02T13:35:32.180517+0000","last_became_peered":"2024-02-02T13:35:32.180517+0000","last_unstale":"2024-02-02T13:35:32.651568+0000","last_undegraded":"2024-02-02T13:35:32.651568+0000","last_fullsized":"2024-02-02T13:35:32.651568+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:00:33.688873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.738803+0000","last_change":"2024-02-02T13:35:32.668535+0000","last_active":"2024-02-02T13:35:33.738803+0000","last_peered":"2024-02-02T13:35:33.738803+0000","last_clean":"2024-02-02T13:35:33.738803+0000","last_became_active":"2024-02-02T13:35:32.667960+0000","last_became_peered":"2024-02-02T13:35:32.667960+0000","last_unstale":"2024-02-02T13:35:33.738803+0000","last_undegraded":"2024-02-02T13:35:33.738803+0000","last_fullsized":"2024-02-02T13:35:33.738803+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:04:03.846318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.651116+0000","last_change":"2024-02-02T13:35:32.186037+0000","last_active":"2024-02-02T13:35:32.651116+0000","last_peered":"2024-02-02T13:35:32.651116+0000","last_clean":"2024-02-02T13:35:32.651116+0000","last_became_active":"2024-02-02T13:35:32.185501+0000","last_became_peered":"2024-02-02T13:35:32.185501+0000","last_unstale":"2024-02-02T13:35:32.651116+0000","last_undegraded":"2024-02-02T13:35:32.651116+0000","last_fullsized":"2024-02-02T13:35:32.651116+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:53:13.607856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740923+0000","last_change":"2024-02-02T13:35:32.666202+0000","last_active":"2024-02-02T13:35:33.740923+0000","last_peered":"2024-02-02T13:35:33.740923+0000","last_clean":"2024-02-02T13:35:33.740923+0000","last_became_active":"2024-02-02T13:35:32.665407+0000","last_became_peered":"2024-02-02T13:35:32.665407+0000","last_unstale":"2024-02-02T13:35:33.740923+0000","last_undegraded":"2024-02-02T13:35:33.740923+0000","last_fullsized":"2024-02-02T13:35:33.740923+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:51:13.027260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.666410+0000","last_change":"2024-02-02T13:35:32.180218+0000","last_active":"2024-02-02T13:35:32.666410+0000","last_peered":"2024-02-02T13:35:32.666410+0000","last_clean":"2024-02-02T13:35:32.666410+0000","last_became_active":"2024-02-02T13:35:32.176974+0000","last_became_peered":"2024-02-02T13:35:32.176974+0000","last_unstale":"2024-02-02T13:35:32.666410+0000","last_undegraded":"2024-02-02T13:35:32.666410+0000","last_fullsized":"2024-02-02T13:35:32.666410+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:25:32.447201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.740244+0000","last_change":"2024-02-02T13:35:32.669497+0000","last_active":"2024-02-02T13:35:33.740244+0000","last_peered":"2024-02-02T13:35:33.740244+0000","last_clean":"2024-02-02T13:35:33.740244+0000","last_became_active":"2024-02-02T13:35:32.668139+0000","last_became_peered":"2024-02-02T13:35:32.668139+0000","last_unstale":"2024-02-02T13:35:33.740244+0000","last_undegraded":"2024-02-02T13:35:33.740244+0000","last_fullsized":"2024-02-02T13:35:33.740244+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:39:15.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.645779+0000","last_change":"2024-02-02T13:35:32.183562+0000","last_active":"2024-02-02T13:35:32.645779+0000","last_peered":"2024-02-02T13:35:32.645779+0000","last_clean":"2024-02-02T13:35:32.645779+0000","last_became_active":"2024-02-02T13:35:32.183302+0000","last_became_peered":"2024-02-02T13:35:32.183302+0000","last_unstale":"2024-02-02T13:35:32.645779+0000","last_undegraded":"2024-02-02T13:35:32.645779+0000","last_fullsized":"2024-02-02T13:35:32.645779+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:36.819863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-02-02T13:35:33.649922+0000","last_change":"2024-02-02T13:35:32.674915+0000","last_active":"2024-02-02T13:35:33.649922+0000","last_peered":"2024-02-02T13:35:33.649922+0000","last_clean":"2024-02-02T13:35:33.649922+0000","last_became_active":"2024-02-02T13:35:32.674692+0000","last_became_peered":"2024-02-02T13:35:32.674692+0000","last_unstale":"2024-02-02T13:35:33.649922+0000","last_undegraded":"2024-02-02T13:35:33.649922+0000","last_fullsized":"2024-02-02T13:35:33.649922+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:32.144237+0000","last_clean_scrub_stamp":"2024-02-02T13:35:32.144237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:54:39.061381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"177'3","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.676255+0000","last_change":"2024-02-02T13:35:32.172620+0000","last_active":"2024-02-02T13:35:32.676255+0000","last_peered":"2024-02-02T13:35:32.676255+0000","last_clean":"2024-02-02T13:35:32.676255+0000","last_became_active":"2024-02-02T13:35:32.170883+0000","last_became_peered":"2024-02-02T13:35:32.170883+0000","last_unstale":"2024-02-02T13:35:32.676255+0000","last_undegraded":"2024-02-02T13:35:32.676255+0000","last_fullsized":"2024-02-02T13:35:32.676255+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:35:31.129152+0000","last_clean_scrub_stamp":"2024-02-02T13:35:31.129152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:45:26.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":319,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-02-02T13:35:32.658611+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:35:32.658611+0000","last_peered":"2024-02-02T13:35:32.658611+0000","last_clean":"2024-02-02T13:35:32.658611+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:35:32.658611+0000","last_undegraded":"2024-02-02T13:35:32.658611+0000","last_fullsized":"2024-02-02T13:35:32.658611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193760,"statfs":{"total":200000000,"available":198410704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1060000000000001,"15min":1.0840000000000001},"min":{"1min":0.79800000000000004,"5min":0.65000000000000002,"15min":0.61499999999999999},"max":{"1min":1.48,"5min":2.266,"15min":2.5310000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.512,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.097,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.73899999999999999,"15min":0.66300000000000003},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.893},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2,"5min":1.1890000000000001,"15min":1.1479999999999999},"min":{"1min":0.87,"5min":0.63600000000000001,"15min":0.56299999999999994},"max":{"1min":1.5740000000000001,"5min":1.8839999999999999,"15min":2.5840000000000001},"last":1.0640000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.258,"15min":1.238},"min":{"1min":0.79000000000000004,"5min":0.73399999999999999,"15min":0.70099999999999996},"max":{"1min":1.6240000000000001,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.236},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.216,"15min":1.1950000000000001},"min":{"1min":0.94099999999999995,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.651,"5min":1.7949999999999999,"15min":3.5819999999999999},"last":0.98699999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.198,"15min":1.1850000000000001},"min":{"1min":0.96899999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5289999999999999,"5min":1.635,"15min":1.819},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.347,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":1.1279999999999999,"5min":0.752,"15min":0.48999999999999999},"max":{"1min":1.548,"5min":2.0259999999999998,"15min":2.4620000000000002},"last":1.4219999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.266,"15min":1.232},"min":{"1min":0.91200000000000003,"5min":0.68999999999999995,"15min":0.57599999999999996},"max":{"1min":1.6160000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.323,"15min":1.3129999999999999},"min":{"1min":0.998,"5min":0.86399999999999999,"15min":0.61199999999999999},"max":{"1min":1.6299999999999999,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3779999999999999,"15min":1.3640000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":1.677,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":1.139,"5min":0.84199999999999997,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.921,"15min":3.7949999999999999},"last":1.2150000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:34:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5009999999999999,"15min":1.5209999999999999},"min":{"1min":1.2170000000000001,"5min":0.96999999999999997,"15min":0.64200000000000002},"max":{"1min":1.7070000000000001,"5min":2.2210000000000001,"15min":3.6840000000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4390000000000001,"15min":1.427},"min":{"1min":1.1779999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6739999999999999,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":1.488}]}]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.115,"15min":1.0600000000000001},"min":{"1min":0.877,"5min":0.498,"15min":0.498},"max":{"1min":1.603,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.0669999999999999,"15min":1.0429999999999999},"min":{"1min":0.86899999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.575,"5min":1.768,"15min":1.768},"last":1.054}]},{"osd":1,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.093,"15min":1.05},"min":{"1min":0.91400000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.458,"5min":1.4790000000000001,"15min":1.556},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1659999999999999,"15min":1.115},"min":{"1min":0.94499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5640000000000001,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.196,"15min":1.127},"min":{"1min":0.85399999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.75,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1759999999999999,"15min":1.163},"min":{"1min":0.91100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5169999999999999,"5min":1.849,"15min":1.849},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.254},"min":{"1min":1.0800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5669999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.204,"15min":1.165},"min":{"1min":1.0149999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.458,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.234}]},{"osd":4,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4550000000000001,"15min":1.3069999999999999},"min":{"1min":0.92600000000000005,"5min":0.58999999999999997,"15min":0.55100000000000005},"max":{"1min":1.6639999999999999,"5min":17.686,"15min":17.686},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.5409999999999999,"15min":1.452},"min":{"1min":0.95099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.585,"5min":17.388999999999999,"15min":17.388999999999999},"last":1.5089999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.90200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5069999999999999,"5min":1.9810000000000001,"15min":1.986},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.409,"15min":1.385},"min":{"1min":1.0569999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8109999999999999,"5min":3.331,"15min":3.331},"last":1.4339999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.431,"15min":1.4530000000000001},"min":{"1min":1.222,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":1.651,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.415,"15min":1.4339999999999999},"min":{"1min":1.1899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6339999999999999,"5min":2.101,"15min":2.1419999999999999},"last":1.6639999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194337,"statfs":{"total":200000000,"available":199001702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.0620000000000001,"15min":1.036},"min":{"1min":0.85099999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.377,"5min":1.8,"15min":1.8},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0620000000000001,"15min":1.0489999999999999},"min":{"1min":0.80000000000000004,"5min":0.68700000000000006,"15min":0.49199999999999999},"max":{"1min":1.3540000000000001,"5min":1.5580000000000001,"15min":1.7150000000000001},"last":0.96899999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.22},"min":{"1min":1.006,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7370000000000001,"5min":1.841,"15min":4.5899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0649999999999999,"15min":1.117},"min":{"1min":0.89200000000000002,"5min":0.68200000000000005,"15min":0.65500000000000003},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":4.4500000000000002},"last":1.389}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.284,"15min":1.252},"min":{"1min":1.1200000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.1970000000000001,"15min":1.1899999999999999},"min":{"1min":1.1639999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4350000000000001,"5min":1.7110000000000001,"15min":3.0720000000000001},"last":0.94499999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.242,"15min":1.206},"min":{"1min":0.96999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":1.629,"15min":1.7769999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.27,"15min":1.2150000000000001},"min":{"1min":1.0049999999999999,"5min":0.68200000000000005,"15min":0.61599999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":5.4610000000000003},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.194,"15min":1.208},"min":{"1min":0.77800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.647,"5min":1.675,"15min":1.8460000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.319,"15min":1.292},"min":{"1min":1.071,"5min":0.85099999999999998,"15min":0.55100000000000005},"max":{"1min":1.8160000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3460000000000001,"15min":1.3200000000000001},"min":{"1min":0.96899999999999997,"5min":0.745,"15min":0.64700000000000002},"max":{"1min":1.865,"5min":1.865,"15min":2.0009999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.3640000000000001,"15min":1.3520000000000001},"min":{"1min":0.96899999999999997,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.845,"5min":1.845,"15min":1.8640000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4299999999999999,"15min":1.4159999999999999},"min":{"1min":1.1519999999999999,"5min":1.0129999999999999,"15min":0.66500000000000004},"max":{"1min":1.893,"5min":1.893,"15min":1.9670000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4750000000000001,"15min":1.4139999999999999},"min":{"1min":1.097,"5min":0.88300000000000001,"15min":0.61599999999999999},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.4379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":973,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002743,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.752,"5min":0.627,"15min":0.57999999999999996},"max":{"1min":1.4570000000000001,"5min":1.6439999999999999,"15min":4.5949999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.079},"min":{"1min":0.79600000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.411,"5min":1.7609999999999999,"15min":1.839},"last":1.238}]},{"osd":1,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0549999999999999,"15min":1.0229999999999999},"min":{"1min":0.79300000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.4830000000000001,"5min":2.1659999999999999,"15min":2.4809999999999999},"last":1.373},{"interface":"front","average":{"1min":1.238,"5min":1.145,"15min":1.081},"min":{"1min":0.98099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":1.0009999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1259999999999999,"15min":1.1140000000000001},"min":{"1min":0.90200000000000002,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.417,"5min":1.9399999999999999,"15min":3.1200000000000001},"last":1.415},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.212,"15min":1.1299999999999999},"min":{"1min":1.0149999999999999,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.736,"5min":2.077,"15min":2.6000000000000001},"last":1.284}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2569999999999999,"15min":1.2150000000000001},"min":{"1min":1.01,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.452,"5min":2.0470000000000002,"15min":2.9079999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.96199999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.853,"5min":2.1110000000000002,"15min":3.9540000000000002},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2569999999999999,"15min":1.1910000000000001},"min":{"1min":0.76600000000000001,"5min":0.68000000000000005,"15min":0.60899999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.331},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3049999999999999},"min":{"1min":0.85599999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5309999999999999,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.585}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4330000000000001,"15min":1.3480000000000001},"min":{"1min":1.1279999999999999,"5min":0.81499999999999995,"15min":0.58899999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.7989999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4750000000000001,"15min":1.427},"min":{"1min":1.2150000000000001,"5min":0.70399999999999996,"15min":0.67800000000000005},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.4379999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4990000000000001,"15min":1.429},"min":{"1min":1.105,"5min":0.76000000000000001,"15min":0.63800000000000001},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5,"15min":1.468},"min":{"1min":1.133,"5min":0.86099999999999999,"15min":0.73599999999999999},"max":{"1min":2.0409999999999999,"5min":2.4900000000000002,"15min":2.698},"last":1.4590000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194337,"statfs":{"total":200000000,"available":199001634,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.73799999999999999},"min":{"1min":0.63200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.234,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.93200000000000005,"15min":0.90400000000000003},"min":{"1min":0.73199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4550000000000001,"5min":1.8400000000000001,"15min":2.6040000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.95699999999999996,"15min":0.92400000000000004},"min":{"1min":0.64400000000000002,"5min":0.49099999999999999,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":2.403,"15min":2.403},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.1120000000000001,"15min":1.052},"min":{"1min":0.95799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5549999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.99199999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.0580000000000001,"15min":1.004},"min":{"1min":0.84099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.77400000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.3109999999999999,"5min":1.9159999999999999,"15min":3.4510000000000001},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.1859999999999999},"min":{"1min":0.97599999999999998,"5min":0.71099999999999997,"15min":0.53000000000000003},"max":{"1min":1.4330000000000001,"5min":1.99,"15min":3.585},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2370000000000001},"min":{"1min":1.0189999999999999,"5min":0.70499999999999996,"15min":0.61299999999999999},"max":{"1min":1.7350000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.0620000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.163,"15min":1.206},"min":{"1min":1.0269999999999999,"5min":0.64400000000000002,"15min":0.57099999999999995},"max":{"1min":1.609,"5min":1.8129999999999999,"15min":3.738},"last":1.103},{"interface":"front","average":{"1min":1.282,"5min":1.2490000000000001,"15min":1.2569999999999999},"min":{"1min":1.0509999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7549999999999999,"5min":1.8899999999999999,"15min":3.4950000000000001},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3779999999999999,"15min":1.3149999999999999},"min":{"1min":1.1839999999999999,"5min":0.81299999999999994,"15min":0.58299999999999996},"max":{"1min":1.823,"5min":1.895,"15min":1.8979999999999999},"last":1.139},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2290000000000001,"15min":1.2809999999999999},"min":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.63300000000000001},"max":{"1min":1.667,"5min":2.0310000000000001,"15min":3.6059999999999999},"last":1.04}]},{"osd":7,"last update":"Fri Feb 2 13:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2749999999999999,"15min":1.27},"min":{"1min":1.0349999999999999,"5min":0.69799999999999995,"15min":0.65900000000000003},"max":{"1min":1.851,"5min":1.851,"15min":3.391},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3799999999999999,"15min":1.3440000000000001},"min":{"1min":1.117,"5min":0.80900000000000005,"15min":0.625},"max":{"1min":1.6990000000000001,"5min":1.946,"15min":2.0409999999999999},"last":0.91900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000090,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.68200000000000005,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":1.345,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.097,"5min":0.94899999999999995,"15min":0.85199999999999998},"min":{"1min":0.75600000000000001,"5min":0.41599999999999998,"15min":0.30299999999999999},"max":{"1min":1.5720000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.244,"5min":2.153,"15min":2.1680000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.024,"15min":0.93500000000000005},"min":{"1min":0.88300000000000001,"5min":0.47499999999999998,"15min":0.35899999999999999},"max":{"1min":1.5129999999999999,"5min":1.6279999999999999,"15min":1.6499999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91900000000000004,"15min":0.85799999999999998},"min":{"1min":0.78800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.109,"5min":1.9790000000000001,"15min":2.1339999999999999},"last":1.024},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.044,"15min":1.0169999999999999},"min":{"1min":0.84299999999999997,"5min":0.435,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.254,"15min":1.2190000000000001},"min":{"1min":1.073,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4930000000000001,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.262,"15min":1.206},"min":{"1min":1.1659999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.571,"5min":2.02,"15min":2.1360000000000001},"last":1.0509999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.214,"15min":1.149},"min":{"1min":1.028,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5349999999999999,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.325,"15min":1.294},"min":{"1min":1.1439999999999999,"5min":0.627,"15min":0.627},"max":{"1min":1.5169999999999999,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3200000000000001,"15min":1.2410000000000001},"min":{"1min":1.101,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.577,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.306,"15min":1.248},"min":{"1min":1.149,"5min":0.73299999999999998,"15min":0.70499999999999996},"max":{"1min":1.5269999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.339}]},{"osd":7,"last update":"Fri Feb 2 13:34:40 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.387,"15min":1.286},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.643,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3340000000000001,"15min":1.2709999999999999},"min":{"1min":1.1359999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5449999999999999,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.0609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":974,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194338,"statfs":{"total":200000000,"available":199002161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.81999999999999995,"15min":0.92200000000000004},"min":{"1min":0.63600000000000001,"5min":0.40799999999999997,"15min":0.34200000000000003},"max":{"1min":1.196,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.997,"5min":0.91000000000000003,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":0.996}]},{"osd":2,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.755,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.2250000000000001,"5min":1.476,"15min":1.639},"last":1.028},{"interface":"front","average":{"1min":1.087,"5min":1.0149999999999999,"15min":1.052},"min":{"1min":0.83299999999999996,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.4730000000000001,"5min":1.6890000000000001,"15min":3.0259999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.127,"15min":1.1879999999999999},"min":{"1min":0.96999999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4610000000000001,"5min":2.2029999999999998,"15min":3.1539999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0289999999999999,"15min":1.169},"min":{"1min":0.83999999999999997,"5min":0.505,"15min":0.49299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":4.8760000000000003},"last":1.095}]},{"osd":4,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1160000000000001,"15min":1.2350000000000001},"min":{"1min":0.92100000000000004,"5min":0.70099999999999996,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.6220000000000001,"15min":2.9449999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.28,"15min":1.3260000000000001},"min":{"1min":1.1120000000000001,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.615,"5min":2.5129999999999999,"15min":3.2970000000000002},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.2969999999999999,"15min":1.3859999999999999},"min":{"1min":1.1679999999999999,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.27,"15min":1.367},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.4850000000000001,"5min":1.6970000000000001,"15min":3.0249999999999999},"last":1.212}]},{"osd":6,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.333,"15min":1.462},"min":{"1min":1.1830000000000001,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":4.3819999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.4119999999999999},"min":{"1min":0.995,"5min":0.86399999999999999,"15min":0.81000000000000005},"max":{"1min":1.6419999999999999,"5min":1.77,"15min":2.9910000000000001},"last":1.27}]},{"osd":7,"last update":"Fri Feb 2 13:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.385,"15min":1.4770000000000001},"min":{"1min":1.0900000000000001,"5min":0.82099999999999995,"15min":0.77100000000000002},"max":{"1min":1.8380000000000001,"5min":2.5750000000000002,"15min":3.4689999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.3919999999999999,"15min":1.5189999999999999},"min":{"1min":1.2310000000000001,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":2.028,"5min":2.2040000000000002,"15min":4.3040000000000003},"last":1.296}]}]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1551,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193760,"statfs":{"total":200000000,"available":198410777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.84099999999999997,"15min":0.995},"min":{"1min":0.66100000000000003,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":4.8070000000000004},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.91700000000000004,"15min":1.2290000000000001},"min":{"1min":0.72899999999999998,"5min":0.34200000000000003,"15min":0.29199999999999998},"max":{"1min":1.29,"5min":1.6839999999999999,"15min":10.816000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94299999999999995,"15min":1.0660000000000001},"min":{"1min":0.79100000000000004,"5min":0.41099999999999998,"15min":0.32700000000000001},"max":{"1min":1.2450000000000001,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":1.288},{"interface":"front","average":{"1min":1.006,"5min":0.97999999999999998,"15min":1.2509999999999999},"min":{"1min":0.75,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":1.304,"5min":1.7,"15min":10.781000000000001},"last":1.071}]},{"osd":3,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93799999999999994,"15min":1.0980000000000001},"min":{"1min":0.81200000000000006,"5min":0.379,"15min":0.379},"max":{"1min":1.3999999999999999,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":1.109},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0389999999999999,"15min":1.1930000000000001},"min":{"1min":0.88100000000000001,"5min":0.437,"15min":0.36699999999999999},"max":{"1min":1.3600000000000001,"5min":1.6459999999999999,"15min":4.9530000000000003},"last":1.22}]},{"osd":4,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1950000000000001,"15min":1.2749999999999999},"min":{"1min":0.95299999999999996,"5min":0.67000000000000004,"15min":0.66800000000000004},"max":{"1min":1.542,"5min":2.129,"15min":3.669},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.25,"15min":1.339},"min":{"1min":1.0580000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.506,"5min":1.895,"15min":4.8780000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.5780000000000001},"min":{"1min":1.024,"5min":0.74299999999999999,"15min":0.72599999999999998},"max":{"1min":1.407,"5min":2.0150000000000001,"15min":10.651},"last":1.355},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.3480000000000001},"min":{"1min":1.105,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.55,"5min":1.954,"15min":3.7869999999999999},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.407},"min":{"1min":1.1579999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"max":{"1min":1.522,"5min":1.764,"15min":4.9210000000000003},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.321,"15min":1.613},"min":{"1min":1.1160000000000001,"5min":0.79400000000000004,"15min":0.76900000000000002},"max":{"1min":1.95,"5min":2.0510000000000002,"15min":10.708},"last":1.306}]},{"osd":7,"last update":"Fri Feb 2 13:34:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.377,"15min":1.647},"min":{"1min":1.1919999999999999,"5min":0.75800000000000001,"15min":0.70999999999999996},"max":{"1min":2.0579999999999998,"5min":2.3010000000000002,"15min":10.74},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.321,"15min":1.3879999999999999},"min":{"1min":1.0960000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.625,"5min":1.9159999999999999,"15min":3.863},"last":1.244}]}]}],"pool_statfs":[]}} 2024-02-02T13:35:37.541 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:35:37.541 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:35:37.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.792+0000 7fa00fa00640 1 Processor -- start 2024-02-02T13:35:37.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.792+0000 7fa00fa00640 1 -- start start 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.793+0000 7fa00fa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa008151f90 0x7fa00815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.793+0000 7fa00fa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa00806f4f0 0x7fa0080b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7fa00fa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0080b1070 con 0x7fa0080b0c40 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7fa00fa00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa008070f80 con 0x7fa00806f4f0 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7fa00fa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa008152370 con 0x7fa008151f90 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa00806f4f0 0x7fa0080b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa00806f4f0 0x7fa0080b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41128/0 (socket says 172.21.15.91:41128) 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 -- 172.21.15.91:0/2247120842 learned_addr learned my addr 172.21.15.91:0/2247120842 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7fa00d000640 1 --2- 172.21.15.91:0/2247120842 >> v2:172.21.15.113:3300/0 conn(0x7fa008151f90 0x7fa00815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 -- 172.21.15.91:0/2247120842 >> v2:172.21.15.113:3300/0 conn(0x7fa008151f90 msgr2=0x7fa00815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 --2- 172.21.15.91:0/2247120842 >> v2:172.21.15.113:3300/0 conn(0x7fa008151f90 0x7fa00815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 -- 172.21.15.91:0/2247120842 >> v1:172.21.15.113:6789/0 conn(0x7fa0080b0c40 legacy=0x7fa0080b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:37.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.794+0000 7f9fffc00640 1 -- 172.21.15.91:0/2247120842 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080cb010 con 0x7fa00806f4f0 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.795+0000 7f9fffc00640 1 --2- 172.21.15.91:0/2247120842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0080b0700 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cc770 tx=0x7f9ff002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2946c601271f08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa007e00640 1 -- 172.21.15.91:0/2247120842 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0031870 con 0x7fa00806f4f0 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa007e00640 1 -- 172.21.15.91:0/2247120842 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff00319d0 con 0x7fa00806f4f0 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa007e00640 1 -- 172.21.15.91:0/2247120842 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0031d60 con 0x7fa00806f4f0 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 -- 172.21.15.91:0/2247120842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 msgr2=0x7fa0080b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 --2- 172.21.15.91:0/2247120842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0080b0700 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cc770 tx=0x7f9ff002b910 comp rx=0 tx=0).stop 2024-02-02T13:35:37.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 -- 172.21.15.91:0/2247120842 shutdown_connections 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 --2- 172.21.15.91:0/2247120842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0080b0700 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 --2- 172.21.15.91:0/2247120842 >> v2:172.21.15.113:3300/0 conn(0x7fa008151f90 0x7fa00815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.796+0000 7fa00fa00640 1 -- 172.21.15.91:0/2247120842 >> 172.21.15.91:0/2247120842 conn(0x7fa0080cec80 msgr2=0x7fa0080cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.797+0000 7fa00fa00640 1 -- 172.21.15.91:0/2247120842 shutdown_connections 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.797+0000 7fa00fa00640 1 -- 172.21.15.91:0/2247120842 wait complete. 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.798+0000 7fa00fa00640 1 Processor -- start 2024-02-02T13:35:37.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.798+0000 7fa00fa00640 1 -- start start 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0080b0c40 0x7fa00814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa008151f90 0x7fa0080c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081e62f0 con 0x7fa00806f4f0 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080cd9f0 con 0x7fa0080b0c40 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00fa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081e70e0 con 0x7fa008151f90 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41134/0 (socket says 172.21.15.91:41134) 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.799+0000 7fa00d000640 1 -- 172.21.15.91:0/685849832 learned_addr learned my addr 172.21.15.91:0/685849832 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 -- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa008151f90 msgr2=0x7fa0080c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa008151f90 0x7fa0080c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.802 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 -- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0080b0c40 msgr2=0x7fa00814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0080b0c40 0x7fa00814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 -- 172.21.15.91:0/685849832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00094d0 con 0x7fa00806f4f0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00d000640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800c820 tx=0x7f9ff800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8004220 con 0x7fa00806f4f0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff80043a0 con 0x7fa00806f4f0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8007a70 con 0x7fa00806f4f0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.800+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0081e64d0 con 0x7fa00806f4f0 2024-02-02T13:35:37.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.801+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081e6a90 con 0x7fa00806f4f0 2024-02-02T13:35:37.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.802+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa008066cf0 con 0x7fa00806f4f0 2024-02-02T13:35:37.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.803+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9ff8007bd0 con 0x7fa00806f4f0 2024-02-02T13:35:37.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.803+0000 7fa005600640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 0x7f9fe006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:37.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.803+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ff8088750 con 0x7fa00806f4f0 2024-02-02T13:35:37.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.804+0000 7f9fffc00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 0x7f9fe006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:37.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.804+0000 7f9fffc00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 0x7f9fe006c8f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0036000 tx=0x7f9ff0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:37.809 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:37.808+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9ff8053d30 con 0x7fa00806f4f0 2024-02-02T13:35:38.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.064+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa00806c7b0 con 0x7fa00806f4f0 2024-02-02T13:35:38.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.070+0000 7fa005600640 1 -- 172.21.15.91:0/685849832 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9ff8057350 con 0x7fa00806f4f0 2024-02-02T13:35:38.071 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:35:38.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.072+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 msgr2=0x7f9fe006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.072+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 0x7f9fe006c8f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0036000 tx=0x7f9ff0030000 comp rx=0 tx=0).stop 2024-02-02T13:35:38.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.072+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 msgr2=0x7fa0081e5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.072+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800c820 tx=0x7f9ff800cce0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 shutdown_connections 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9fe006a430 0x7f9fe006c8f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa008151f90 0x7fa0080c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0080b0c40 0x7fa00814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 --2- 172.21.15.91:0/685849832 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa00806f4f0 0x7fa0081e5db0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 >> 172.21.15.91:0/685849832 conn(0x7fa0080cec80 msgr2=0x7fa0080cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 shutdown_connections 2024-02-02T13:35:38.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.073+0000 7fa00fa00640 1 -- 172.21.15.91:0/685849832 wait complete. 2024-02-02T13:35:38.092 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:38.114 INFO:tasks.ceph.mds.c.smithi091.stderr:2024-02-02T13:35:38.113+0000 7f4fad600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:35:38.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.293+0000 7f0fd7400640 1 Processor -- start 2024-02-02T13:35:38.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.293+0000 7f0fd7400640 1 -- start start 2024-02-02T13:35:38.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd7400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0fd015d650 0x7f0fd00cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 0x7f0fd00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd00d12c0 con 0x7f0fd01610b0 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd7400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd00cea30 con 0x7f0fd015d650 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd7400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd00cded0 con 0x7f0fd00ce070 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd4a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0fd015d650 0x7f0fd00cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd4a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0fd015d650 0x7f0fd00cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41148/0 (socket says 172.21.15.91:41148) 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.294+0000 7f0fd4a00640 1 -- 172.21.15.91:0/3621795819 learned_addr learned my addr 172.21.15.91:0/3621795819 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fd4a00640 1 -- 172.21.15.91:0/3621795819 >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 msgr2=0x7f0fd00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fcfe00640 1 --2- 172.21.15.91:0/3621795819 >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 0x7f0fd00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fd4a00640 1 --2- 172.21.15.91:0/3621795819 >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 0x7f0fd00ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fd4a00640 1 -- 172.21.15.91:0/3621795819 >> v1:172.21.15.113:6789/0 conn(0x7f0fd01610b0 legacy=0x7f0fd0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fd4a00640 1 -- 172.21.15.91:0/3621795819 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd0071410 con 0x7f0fd015d650 2024-02-02T13:35:38.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.295+0000 7f0fcfe00640 1 --2- 172.21.15.91:0/3621795819 >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 0x7f0fd00ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.296+0000 7f0fd4a00640 1 --2- 172.21.15.91:0/3621795819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd015d650 0x7f0fd00cd990 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0161010 tx=0x7f0fc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de51f480b3d224d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.296+0000 7f0fcf400640 1 -- 172.21.15.91:0/3621795819 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0031870 con 0x7f0fd015d650 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.296+0000 7f0fcf400640 1 -- 172.21.15.91:0/3621795819 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc00319d0 con 0x7f0fd015d650 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.296+0000 7f0fcf400640 1 -- 172.21.15.91:0/3621795819 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0031d60 con 0x7f0fd015d650 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 -- 172.21.15.91:0/3621795819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd015d650 msgr2=0x7f0fd00cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 --2- 172.21.15.91:0/3621795819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd015d650 0x7f0fd00cd990 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0161010 tx=0x7f0fc002b910 comp rx=0 tx=0).stop 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 -- 172.21.15.91:0/3621795819 shutdown_connections 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 --2- 172.21.15.91:0/3621795819 >> v2:172.21.15.113:3300/0 conn(0x7f0fd00ce070 0x7f0fd00ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 --2- 172.21.15.91:0/3621795819 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd015d650 0x7f0fd00cd990 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 -- 172.21.15.91:0/3621795819 >> 172.21.15.91:0/3621795819 conn(0x7f0fd00d30a0 msgr2=0x7f0fd00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:38.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 -- 172.21.15.91:0/3621795819 shutdown_connections 2024-02-02T13:35:38.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.297+0000 7f0fd7400640 1 -- 172.21.15.91:0/3621795819 wait complete. 2024-02-02T13:35:38.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.299+0000 7f0fd7400640 1 Processor -- start 2024-02-02T13:35:38.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.299+0000 7f0fd7400640 1 -- start start 2024-02-02T13:35:38.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0fd00ce070 0x7f0fd015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd015d650 0x7f0fd00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015a8c0 con 0x7f0fd01610b0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00b23f0 con 0x7f0fd015d650 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd7400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00ca8d0 con 0x7f0fd00ce070 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd5400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41152/0 (socket says 172.21.15.91:41152) 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.300+0000 7f0fd5400640 1 -- 172.21.15.91:0/968261701 learned_addr learned my addr 172.21.15.91:0/968261701 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd4a00640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0fd00ce070 0x7f0fd015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 -- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0fd00ce070 msgr2=0x7f0fd015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0fd00ce070 0x7f0fd015a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 -- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd015d650 msgr2=0x7f0fd00b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd015d650 0x7f0fd00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 -- 172.21.15.91:0/968261701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00094d0 con 0x7f0fd01610b0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd5400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800c7b0 tx=0x7f0fc800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8003d80 con 0x7f0fd01610b0 2024-02-02T13:35:38.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800ce60 con 0x7f0fd01610b0 2024-02-02T13:35:38.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80052b0 con 0x7f0fd01610b0 2024-02-02T13:35:38.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd00ca310 con 0x7f0fd01610b0 2024-02-02T13:35:38.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.301+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd00b2080 con 0x7f0fd01610b0 2024-02-02T13:35:38.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.302+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0071a30 con 0x7f0fd01610b0 2024-02-02T13:35:38.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.303+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0fc8006b10 con 0x7f0fd01610b0 2024-02-02T13:35:38.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.304+0000 7f0fccc00640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 0x7f0fa806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:38.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.304+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc8087ba0 con 0x7f0fd01610b0 2024-02-02T13:35:38.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.304+0000 7f0fd4a00640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 0x7f0fa806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:38.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.305+0000 7f0fd4a00640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 0x7f0fa806c8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0036000 tx=0x7f0fc003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:38.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.308+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0fc80532d0 con 0x7f0fd01610b0 2024-02-02T13:35:38.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.520+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fd0066cf0 con 0x7f0fd01610b0 2024-02-02T13:35:38.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.521+0000 7f0fccc00640 1 -- 172.21.15.91:0/968261701 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7f0fc8017070 con 0x7f0fd01610b0 2024-02-02T13:35:38.524 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:38.524 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:35:38.524 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:35:38.525 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 msgr2=0x7f0fa806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 0x7f0fa806c8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0036000 tx=0x7f0fc003c040 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 msgr2=0x7f0fd00b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800c7b0 tx=0x7f0fc800cc70 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 shutdown_connections 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0fa806a430 0x7f0fa806c8f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0fd01610b0 0x7f0fd00b2880 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd015d650 0x7f0fd00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 --2- 172.21.15.91:0/968261701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0fd00ce070 0x7f0fd015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 >> 172.21.15.91:0/968261701 conn(0x7f0fd00d30a0 msgr2=0x7f0fd00c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 shutdown_connections 2024-02-02T13:35:38.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:38.524+0000 7f0fd7400640 1 -- 172.21.15.91:0/968261701 wait complete. 2024-02-02T13:35:38.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:35:32.637261+0000', 'modified': '2024-02-02T13:35:35.664997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8439': {'gid': 8439, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/2606135586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 2606135586}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 2606135586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:35:38.946 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:35:38.946 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T13:35:39.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.159+0000 7f33e9400640 1 Processor -- start 2024-02-02T13:35:39.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.159+0000 7f33e9400640 1 -- start start 2024-02-02T13:35:39.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f33e4159de0 0x7f33e40cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33e415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e9400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40cbf20 con 0x7f33e415e170 2024-02-02T13:35:39.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e9400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e4070ca0 con 0x7f33e4159de0 2024-02-02T13:35:39.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e9400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40ca0c0 con 0x7f33e40cbac0 2024-02-02T13:35:39.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e3200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f33e415e170 0x7f33e40cd850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:46728/0 (socket says 172.21.15.91:46728) 2024-02-02T13:35:39.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.161+0000 7f33e3200640 1 -- 172.21.15.91:0/4158553269 learned_addr learned my addr 172.21.15.91:0/4158553269 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e2800640 1 --2- 172.21.15.91:0/4158553269 >> v2:172.21.15.113:3300/0 conn(0x7f33e4159de0 0x7f33e40cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33d9c00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33e415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053165949 0 0) 0x7f33e40cbf20 con 0x7f33e415e170 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c4003520 con 0x7f33e415e170 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261325903 0 0) 0x7f33c4003520 con 0x7f33e415e170 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 msgr2=0x7f33e415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33e415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 >> v2:172.21.15.113:3300/0 conn(0x7f33e4159de0 msgr2=0x7f33e40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 --2- 172.21.15.91:0/4158553269 >> v2:172.21.15.113:3300/0 conn(0x7f33e4159de0 0x7f33e40cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40ca380 con 0x7f33e415e170 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f33dc0019f0 con 0x7f33e415e170 2024-02-02T13:35:39.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 >> v1:172.21.15.91:6789/0 conn(0x7f33e415e170 legacy=0x7f33e40cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33c4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c4003f40 0x7f33c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4070ca0 con 0x7f33c4003f40 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40ca0c0 con 0x7f33c4004d00 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.162+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400b980 con 0x7f33e40cbac0 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c4003f40 0x7f33c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33e3200640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33e2800640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33c4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 -- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 msgr2=0x7f33c4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33e40cbac0 0x7f33c4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 -- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c4004d00 msgr2=0x7f33c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.163+0000 7f33d9c00640 1 -- 172.21.15.91:0/4158553269 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4003520 con 0x7f33c4003f40 2024-02-02T13:35:39.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.164+0000 7f33d9c00640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c4003f40 0x7f33c40047c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009af0 tx=0x7f33c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7383d6975aed4d07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.164+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc02c6b0 con 0x7f33c4003f40 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.164+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc02c810 con 0x7f33c4003f40 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.164+0000 7f33e1e00640 1 -- 172.21.15.91:0/4158553269 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc02cba0 con 0x7f33c4003f40 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 -- 172.21.15.91:0/4158553269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c4003f40 msgr2=0x7f33c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 --2- 172.21.15.91:0/4158553269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33c4003f40 0x7f33c40047c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009af0 tx=0x7f33c40057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e3200640 1 -- 172.21.15.91:0/4158553269 reap_dead start 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 -- 172.21.15.91:0/4158553269 shutdown_connections 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 -- 172.21.15.91:0/4158553269 >> 172.21.15.91:0/4158553269 conn(0x7f33e40b0700 msgr2=0x7f33e40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 -- 172.21.15.91:0/4158553269 shutdown_connections 2024-02-02T13:35:39.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.165+0000 7f33e9400640 1 -- 172.21.15.91:0/4158553269 wait complete. 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.167+0000 7f33e9400640 1 Processor -- start 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.167+0000 7f33e9400640 1 -- start start 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c4003f40 0x7f33e40af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33e4155c70 0x7f33e40afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40c6ac0 con 0x7f33e4150570 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40b22b0 con 0x7f33e4155c70 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e9400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40b2590 con 0x7f33c4003f40 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41170/0 (socket says 172.21.15.91:41170) 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.168+0000 7f33e3200640 1 -- 172.21.15.91:0/1429219942 learned_addr learned my addr 172.21.15.91:0/1429219942 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e2800640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c4003f40 0x7f33e40af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33d9c00640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33e4155c70 0x7f33e40afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 -- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c4003f40 msgr2=0x7f33e40af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c4003f40 0x7f33e40af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 -- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33e4155c70 msgr2=0x7f33e40afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33e4155c70 0x7f33e40afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 -- 172.21.15.91:0/1429219942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40b2870 con 0x7f33e4150570 2024-02-02T13:35:39.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e3200640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0013d0 tx=0x7f33dc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33db400640 1 -- 172.21.15.91:0/1429219942 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00ec10 con 0x7f33e4150570 2024-02-02T13:35:39.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33db400640 1 -- 172.21.15.91:0/1429219942 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc00ed90 con 0x7f33e4150570 2024-02-02T13:35:39.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33db400640 1 -- 172.21.15.91:0/1429219942 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc013660 con 0x7f33e4150570 2024-02-02T13:35:39.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e40b2b50 con 0x7f33e4150570 2024-02-02T13:35:39.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.169+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e40c6480 con 0x7f33e4150570 2024-02-02T13:35:39.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.172+0000 7f33db400640 1 -- 172.21.15.91:0/1429219942 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f33dc0137c0 con 0x7f33e4150570 2024-02-02T13:35:39.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.172+0000 7f33db400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 0x7f33bc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.172+0000 7f33db400640 1 -- 172.21.15.91:0/1429219942 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33dc088dc0 con 0x7f33e4150570 2024-02-02T13:35:39.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.173+0000 7f33e2800640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 0x7f33bc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.173+0000 7f33e2800640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 0x7f33bc06c920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009a90 tx=0x7f33d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.173+0000 7f33ec1e0740 1 Processor -- start 2024-02-02T13:35:39.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.173+0000 7f33ec1e0740 1 -- start start 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b4ed0 0x55a2701e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a270190650 con 0x55a2701b8ad0 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a270125000 con 0x55a2701b4ed0 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.174+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a270122790 con 0x55a2701e4f50 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41184/0 (socket says 172.21.15.91:41184) 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 -- 172.21.15.91:0/3113139654 learned_addr learned my addr 172.21.15.91:0/3113139654 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 -- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 msgr2=0x55a2701e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 -- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b4ed0 msgr2=0x55a2701e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b4ed0 0x55a2701e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 -- 172.21.15.91:0/3113139654 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2700e5710 con 0x55a2701b8ad0 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33e3200640 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f33dc05fb10 tx=0x7f33dc057cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a704067bd67dcab4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33d2a00640 1 -- 172.21.15.91:0/3113139654 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc05be90 con 0x55a2701b8ad0 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33d2a00640 1 -- 172.21.15.91:0/3113139654 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc060040 con 0x55a2701b8ad0 2024-02-02T13:35:39.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.175+0000 7f33d2a00640 1 -- 172.21.15.91:0/3113139654 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc054770 con 0x55a2701b8ad0 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 -- 172.21.15.91:0/3113139654 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 msgr2=0x55a2701e2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f33dc05fb10 tx=0x7f33dc057cd0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 -- 172.21.15.91:0/3113139654 shutdown_connections 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701e2700 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b4ed0 0x55a2701e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/3113139654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 -- 172.21.15.91:0/3113139654 >> 172.21.15.91:0/3113139654 conn(0x55a2701b71e0 msgr2=0x55a2701ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 -- 172.21.15.91:0/3113139654 shutdown_connections 2024-02-02T13:35:39.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.176+0000 7f33ec1e0740 1 -- 172.21.15.91:0/3113139654 wait complete. 2024-02-02T13:35:39.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.177+0000 7f33ec1e0740 1 -- start start 2024-02-02T13:35:39.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.177+0000 7f33ec1e0740 1 Processor -- start 2024-02-02T13:35:39.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b8ad0 0x55a2701da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a2701dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27011e080 con 0x55a2701e4f50 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2700bf850 con 0x55a2701b8ad0 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2700bf4f0 con 0x55a2701ba4d0 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33ec1e0740 10 client.0 _ll_register_callbacks cb 0x55a2701179c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.178+0000 7f33d9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41188/0 (socket says 172.21.15.91:41188) 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 -- 172.21.15.91:0/2216177994 learned_addr learned my addr 172.21.15.91:0/2216177994 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 -- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 msgr2=0x55a2701dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a2701dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 -- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b8ad0 msgr2=0x55a2701da570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b8ad0 0x55a2701da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 -- 172.21.15.91:0/2216177994 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e40c6480 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33d9c00640 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f33cc037040 tx=0x7f33cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca5d2db62d814a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33cb400640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc02c6b0 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33cc02cb80 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.179+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2701485d0 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33cc02cd40 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 10 client.8586 fetch_fsmap learned FSMap version 80 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55a27011c700 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2701485d0 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f33cc003610 con 0x55a2701e4f50 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 10 client.8586 fetch_fsmap finished waiting for FSMap version 80 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 10 client.8586 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 20 client.8586 populate_metadata read hostname 'smithi091' 2024-02-02T13:35:39.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a270212a10 0x55a270212df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 4 client.8586 mds_command: new command op to 8439 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:35:39.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 --> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2701108d0 con 0x55a270212a10 2024-02-02T13:35:39.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.180+0000 7f33e2800640 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a270212a10 0x55a270212df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.181+0000 7f33cb400640 10 client.8586 ms_handle_connect on v2:172.21.15.91:6835/2606135586 2024-02-02T13:35:39.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.181+0000 7f33e2800640 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a270212a10 0x55a270212df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc6405e2e406e04 server_cookie=9b1c0a672f706311 in_seq=0 out_seq=0 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33cb400640 1 -- 172.21.15.91:0/2216177994 <== mds.0 v2:172.21.15.91:6835/2606135586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a2701108d0 con 0x55a270212a10 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33cb400640 10 client.8586 handle_command_reply: tid=0 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 client.8586 shutdown 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 msgr2=0x55a2701b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f33cc037040 tx=0x7f33cc004730 comp rx=0 tx=0).stop 2024-02-02T13:35:39.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 shutdown_connections 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a270212a10 0x55a270212df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a2701dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701e4f50 0x55a2701b9f90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33cb400640 0 client.8586 ms_handle_reset on v2:172.21.15.91:6835/2606135586 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/2216177994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701b8ad0 0x55a2701da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 >> 172.21.15.91:0/2216177994 conn(0x55a2701b4ed0 msgr2=0x55a270203270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 shutdown_connections 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.182+0000 7f33ec1e0740 1 -- 172.21.15.91:0/2216177994 wait complete. 2024-02-02T13:35:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.183+0000 7f33ec1e0740 20 client.8586 trim_cache size 0 max 16384 2024-02-02T13:35:39.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.199+0000 7f33ec1e0740 1 Processor -- start 2024-02-02T13:35:39.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.199+0000 7f33ec1e0740 1 -- start start 2024-02-02T13:35:39.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.199+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.201 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701ba4d0 0x55a2701e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26fe85f20 con 0x55a2701b8ad0 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2701dd8d0 con 0x55a2701ba4d0 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2701afd90 con 0x55a2701e4f50 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41204/0 (socket says 172.21.15.91:41204) 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 -- 172.21.15.91:0/1951365050 learned_addr learned my addr 172.21.15.91:0/1951365050 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 -- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 msgr2=0x55a2701e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 -- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701ba4d0 msgr2=0x55a2701e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701ba4d0 0x55a2701e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 -- 172.21.15.91:0/1951365050 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a270152280 con 0x55a2701b8ad0 2024-02-02T13:35:39.202 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.200+0000 7f33e2800640 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x55a2701db5d0 tx=0x7f33d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be00b75fca0df1d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.203 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33d3400640 1 -- 172.21.15.91:0/1951365050 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40270f0 con 0x55a2701b8ad0 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33d3400640 1 -- 172.21.15.91:0/1951365050 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4027250 con 0x55a2701b8ad0 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33d3400640 1 -- 172.21.15.91:0/1951365050 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4016b70 con 0x55a2701b8ad0 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1951365050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 msgr2=0x55a2701df300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x55a2701db5d0 tx=0x7f33d4004730 comp rx=0 tx=0).stop 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1951365050 shutdown_connections 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701e4f50 0x55a2701e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701ba4d0 0x55a2701e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1951365050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701b8ad0 0x55a2701df300 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1951365050 >> 172.21.15.91:0/1951365050 conn(0x55a2701b4ed0 msgr2=0x55a2701e6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1951365050 shutdown_connections 2024-02-02T13:35:39.204 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.201+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1951365050 wait complete. 2024-02-02T13:35:39.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.202+0000 7f33ec1e0740 1 -- start start 2024-02-02T13:35:39.205 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.202+0000 7f33ec1e0740 1 Processor -- start 2024-02-02T13:35:39.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a27025c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701e4f50 0x55a270258b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2701de860 con 0x55a2701ddea0 2024-02-02T13:35:39.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2701deca0 con 0x55a2701e4f50 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a270259340 con 0x55a2701ba4d0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.203+0000 7f33ec1e0740 10 client.0 _ll_register_callbacks cb 0x7f33c40045d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41216/0 (socket says 172.21.15.91:41216) 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 -- 172.21.15.91:0/1448598070 learned_addr learned my addr 172.21.15.91:0/1448598070 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 -- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 msgr2=0x55a27025c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e2800640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a27025c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33d9c00640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701e4f50 0x55a270258b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a27025c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 -- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701e4f50 msgr2=0x55a270258b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701e4f50 0x55a270258b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 -- 172.21.15.91:0/1448598070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d40096e0 con 0x55a2701ddea0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33e3200640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f33dc057fa0 tx=0x7f33dc0871b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63c79aaffc06423 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33cb400640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:35:39.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.204+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0543a0 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33dc05be90 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33dc060040 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33dc057550 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 10 client.8592 fetch_fsmap learned FSMap version 80 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55a2700bea60 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc060040 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f33dc05cbd0 con 0x55a2701ddea0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 10 client.8592 fetch_fsmap finished waiting for FSMap version 80 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 10 client.8592 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 20 client.8592 populate_metadata read hostname 'smithi091' 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a2701e0480 0x55a2701e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 4 client.8592 mds_command: new command op to 8439 tid=0[{"prefix": "status"}] 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 --> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a270117100 con 0x55a2701e0480 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.205+0000 7f33e2800640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a2701e0480 0x55a2701e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33e2800640 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a2701e0480 0x55a2701e0860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dfd2e1f3a06591 server_cookie=9975540ae5ee5522 in_seq=0 out_seq=0 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33cb400640 10 client.8592 ms_handle_connect on v2:172.21.15.91:6835/2606135586 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33cb400640 1 -- 172.21.15.91:0/1448598070 <== mds.0 v2:172.21.15.91:6835/2606135586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a270117100 con 0x55a2701e0480 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33cb400640 10 client.8592 handle_command_reply: tid=0 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33ec1e0740 1 client.8592 shutdown 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 msgr2=0x55a2701de320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.206+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f33dc057fa0 tx=0x7f33dc0871b0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 shutdown_connections 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:6835/2606135586,v1:172.21.15.91:6837/2606135586] conn(0x55a2701e0480 0x55a2701e0860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a2701ddea0 0x55a2701de320 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2701e4f50 0x55a270258b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 --2- 172.21.15.91:0/1448598070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a2701ba4d0 0x55a27025c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33cb400640 0 client.8592 ms_handle_reset on v2:172.21.15.91:6835/2606135586 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 >> 172.21.15.91:0/1448598070 conn(0x55a2701b4ed0 msgr2=0x55a270218e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 shutdown_connections 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 1 -- 172.21.15.91:0/1448598070 wait complete. 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33ec1e0740 20 client.8592 trim_cache size 0 max 16384 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 8439, 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.5653156600000004, 2024-02-02T13:35:39.212 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 80, 2024-02-02T13:35:39.213 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 178, 2024-02-02T13:35:39.213 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 178, 2024-02-02T13:35:39.213 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 11.479095166 2024-02-02T13:35:39.213 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:35:39.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 msgr2=0x7f33bc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 0x7f33bc06c920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009a90 tx=0x7f33d4008040 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 msgr2=0x7f33e40c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.207+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0013d0 tx=0x7f33dc00e530 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 shutdown_connections 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f33bc06a460 0x7f33bc06c920 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4150570 0x7f33e40c5d60 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33e4155c70 0x7f33e40afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 --2- 172.21.15.91:0/1429219942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33c4003f40 0x7f33e40af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 >> 172.21.15.91:0/1429219942 conn(0x7f33e40b0700 msgr2=0x7f33e40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 shutdown_connections 2024-02-02T13:35:39.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.208+0000 7f33e9400640 1 -- 172.21.15.91:0/1429219942 wait complete. 2024-02-02T13:35:39.226 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:35:39.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.425+0000 7fb268200640 1 Processor -- start 2024-02-02T13:35:39.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.426+0000 7fb268200640 1 -- start start 2024-02-02T13:35:39.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb268200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb260151f90 0x7fb26015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb268200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb26006f4f0 0x7fb2600b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb268200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb260070f80 con 0x7fb2600b0c40 2024-02-02T13:35:39.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb268200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2600b10d0 con 0x7fb26006f4f0 2024-02-02T13:35:39.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb268200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb260152370 con 0x7fb260151f90 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb264e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb26006f4f0 0x7fb2600b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb264e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb26006f4f0 0x7fb2600b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41222/0 (socket says 172.21.15.91:41222) 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.427+0000 7fb264e00640 1 -- 172.21.15.91:0/2551258792 learned_addr learned my addr 172.21.15.91:0/2551258792 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.428+0000 7fb265800640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb260151f90 0x7fb26015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.428+0000 7fb264e00640 1 -- 172.21.15.91:0/2551258792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb260151f90 msgr2=0x7fb26015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.428+0000 7fb264e00640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb260151f90 0x7fb26015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.428+0000 7fb264e00640 1 -- 172.21.15.91:0/2551258792 >> v1:172.21.15.113:6789/0 conn(0x7fb2600b0c40 legacy=0x7fb2600b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:39.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.428+0000 7fb264e00640 1 -- 172.21.15.91:0/2551258792 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600cb010 con 0x7fb26006f4f0 2024-02-02T13:35:39.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.429+0000 7fb264e00640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb26006f4f0 0x7fb2600b0700 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb248009870 tx=0x7fb24802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56690bbe3b613fcd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.429+0000 7fb25fe00640 1 -- 172.21.15.91:0/2551258792 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248031640 con 0x7fb26006f4f0 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.429+0000 7fb25fe00640 1 -- 172.21.15.91:0/2551258792 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2480317a0 con 0x7fb26006f4f0 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.429+0000 7fb25fe00640 1 -- 172.21.15.91:0/2551258792 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248031b30 con 0x7fb26006f4f0 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 -- 172.21.15.91:0/2551258792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb26006f4f0 msgr2=0x7fb2600b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb26006f4f0 0x7fb2600b0700 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb248009870 tx=0x7fb24802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 -- 172.21.15.91:0/2551258792 shutdown_connections 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb26006f4f0 0x7fb2600b0700 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 --2- 172.21.15.91:0/2551258792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb260151f90 0x7fb26015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 -- 172.21.15.91:0/2551258792 >> 172.21.15.91:0/2551258792 conn(0x7fb2600cec80 msgr2=0x7fb2600cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 -- 172.21.15.91:0/2551258792 shutdown_connections 2024-02-02T13:35:39.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.430+0000 7fb268200640 1 -- 172.21.15.91:0/2551258792 wait complete. 2024-02-02T13:35:39.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.433+0000 7fb268200640 1 Processor -- start 2024-02-02T13:35:39.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.433+0000 7fb268200640 1 -- start start 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26006f4f0 0x7fb2601effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 0x7fb2601e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26015df20 con 0x7fb260151f90 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600c6b70 con 0x7fb2600b0c40 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb268200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26015d960 con 0x7fb26006f4f0 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb266200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb266200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41234/0 (socket says 172.21.15.91:41234) 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.434+0000 7fb266200640 1 -- 172.21.15.91:0/425183365 learned_addr learned my addr 172.21.15.91:0/425183365 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb264e00640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 0x7fb2601e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb265800640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26006f4f0 0x7fb2601effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 -- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26006f4f0 msgr2=0x7fb2601effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26006f4f0 0x7fb2601effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 -- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 msgr2=0x7fb2601e4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 0x7fb2601e4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 -- 172.21.15.91:0/425183365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480094d0 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb264e00640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 0x7fb2601e4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb266200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b3e0 tx=0x7fb25800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258011830 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258011990 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26015dc40 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.435+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258011d40 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.436+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2601f04f0 con 0x7fb260151f90 2024-02-02T13:35:39.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.437+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2600c69d0 con 0x7fb260151f90 2024-02-02T13:35:39.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.438+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb25801a400 con 0x7fb260151f90 2024-02-02T13:35:39.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.439+0000 7fb25d600640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 0x7fb23806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:39.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.439+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb258088710 con 0x7fb260151f90 2024-02-02T13:35:39.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.439+0000 7fb265800640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 0x7fb23806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:39.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.440+0000 7fb265800640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 0x7fb23806c8f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb250008ef0 tx=0x7fb250008440 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:39.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.443+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb2600c69d0 con 0x7fb260151f90 2024-02-02T13:35:39.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.673+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2600c7050 con 0x7fb260151f90 2024-02-02T13:35:39.674 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.673+0000 7fb25d600640 1 -- 172.21.15.91:0/425183365 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb2600c7050 con 0x7fb260151f90 2024-02-02T13:35:39.675 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 msgr2=0x7fb23806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 0x7fb23806c8f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb250008ef0 tx=0x7fb250008440 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 msgr2=0x7fb2601e4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b3e0 tx=0x7fb25800b8a0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 shutdown_connections 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb23806a430 0x7fb23806c8f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb260151f90 0x7fb2601e4e60 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2600b0c40 0x7fb2601e4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 --2- 172.21.15.91:0/425183365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26006f4f0 0x7fb2601effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 >> 172.21.15.91:0/425183365 conn(0x7fb2600cec80 msgr2=0x7fb260159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 shutdown_connections 2024-02-02T13:35:39.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:39.676+0000 7fb268200640 1 -- 172.21.15.91:0/425183365 wait complete. 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:35:39.696 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:35:39.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:39.697 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.749 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 41986sec preferred_lft 41986sec 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:35:39.750 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:35:39.751 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:39.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:39.752 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:35:39.832 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:35:39.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:39.836 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:35:39.888 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:35:39.889 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:35:39.889 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:35:39.889 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:35:39.889 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:35:39.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:39.889 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:35:39.889 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:35:39.889 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:35:39.889 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:35:39.890 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:35:39.890 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:35:40.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:40.072 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:35:40.123 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:35:40.123 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:35:40.124 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:35:40.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:40.124 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:35:40.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:35:40.221 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:35:40.274 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:35:40.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:35:40.274 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:35:40.325 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:35:40.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:40.325 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:40.460 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:40.460 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:35:40.491 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:40.491 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:35:40.491 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:35:40.492 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:35:40.492 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:35:40.494 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:35:40.494 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:35:40.494 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:35:40.494 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:35:40.494 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:35:40.498 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:35:40.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:40.499 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:35:40.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:40.584 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:35:40.652 INFO:teuthology.orchestra.run.smithi091.stdout:client8601 2024-02-02T13:35:40.653 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:35:40.653 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:40.709 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:35:40.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.912+0000 7f0199a00640 1 Processor -- start 2024-02-02T13:35:40.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.913+0000 7f0199a00640 1 -- start start 2024-02-02T13:35:40.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f019415d590 0x7f01940b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f01940b1ba0 0x7f01940b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01940b1360 0x7f0194069e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940b1980 con 0x7f01940b1ba0 2024-02-02T13:35:40.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940d07b0 con 0x7f019415d590 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0199a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940cff10 con 0x7f01940b1360 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.914+0000 7f0192000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f01940b1ba0 0x7f01940b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f01940b1ba0 0x7f01940b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41444/0 (socket says 172.21.15.91:41444) 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 -- 172.21.15.91:0/2867575936 learned_addr learned my addr 172.21.15.91:0/2867575936 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 -- 172.21.15.91:0/2867575936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01940b1360 msgr2=0x7f0194069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 --2- 172.21.15.91:0/2867575936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01940b1360 0x7f0194069e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 -- 172.21.15.91:0/2867575936 >> v2:172.21.15.113:3300/0 conn(0x7f019415d590 msgr2=0x7f01940b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 --2- 172.21.15.91:0/2867575936 >> v2:172.21.15.113:3300/0 conn(0x7f019415d590 0x7f01940b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.915+0000 7f0192000640 1 -- 172.21.15.91:0/2867575936 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940d01f0 con 0x7f01940b1ba0 2024-02-02T13:35:40.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.916+0000 7f0192000640 1 --2- 172.21.15.91:0/2867575936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940b0e20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f019408b090 tx=0x7f017c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b5dfe1fa23b22c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:40.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.916+0000 7f0191600640 1 -- 172.21.15.91:0/2867575936 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c02c950 con 0x7f01940b1ba0 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.916+0000 7f0191600640 1 -- 172.21.15.91:0/2867575936 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c02cab0 con 0x7f01940b1ba0 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.916+0000 7f0191600640 1 -- 172.21.15.91:0/2867575936 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c02ce40 con 0x7f01940b1ba0 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 -- 172.21.15.91:0/2867575936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 msgr2=0x7f01940b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 --2- 172.21.15.91:0/2867575936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940b0e20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f019408b090 tx=0x7f017c02bdb0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 -- 172.21.15.91:0/2867575936 shutdown_connections 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 --2- 172.21.15.91:0/2867575936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01940b1360 0x7f0194069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 --2- 172.21.15.91:0/2867575936 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940b0e20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 --2- 172.21.15.91:0/2867575936 >> v2:172.21.15.113:3300/0 conn(0x7f019415d590 0x7f01940b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 -- 172.21.15.91:0/2867575936 >> 172.21.15.91:0/2867575936 conn(0x7f01940d0fa0 msgr2=0x7f01940d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 -- 172.21.15.91:0/2867575936 shutdown_connections 2024-02-02T13:35:40.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.917+0000 7f0199a00640 1 -- 172.21.15.91:0/2867575936 wait complete. 2024-02-02T13:35:40.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.919+0000 7f0199a00640 1 Processor -- start 2024-02-02T13:35:40.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.919+0000 7f0199a00640 1 -- start start 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 0x7f019415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f019415d590 0x7f019415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941f7430 con 0x7f01940b1ba0 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941f7150 con 0x7f01940b1360 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0199a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019415db10 con 0x7f019415d590 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0192000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0192000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41446/0 (socket says 172.21.15.91:41446) 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0192000640 1 -- 172.21.15.91:0/1564109925 learned_addr learned my addr 172.21.15.91:0/1564109925 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:40.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0192a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 0x7f019415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0193400640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f019415d590 0x7f019415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.920+0000 7f0192000640 1 -- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f019415d590 msgr2=0x7f019415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192000640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f019415d590 0x7f019415a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192000640 1 -- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 msgr2=0x7f019415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192000640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 0x7f019415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192000640 1 -- 172.21.15.91:0/1564109925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c009b30 con 0x7f01940b1ba0 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 0x7f019415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0192000640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f017c000f80 tx=0x7f017c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c03e450 con 0x7f01940b1ba0 2024-02-02T13:35:40.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c03e5b0 con 0x7f01940b1ba0 2024-02-02T13:35:40.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c03e960 con 0x7f01940b1ba0 2024-02-02T13:35:40.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019415ddf0 con 0x7f01940b1ba0 2024-02-02T13:35:40.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.921+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01940c6840 con 0x7f01940b1ba0 2024-02-02T13:35:40.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.923+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019415d970 con 0x7f01940b1ba0 2024-02-02T13:35:40.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.924+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f017c03eac0 con 0x7f01940b1ba0 2024-02-02T13:35:40.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.924+0000 7f018aa00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 0x7f016c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:40.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.925+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f017c0ab980 con 0x7f01940b1ba0 2024-02-02T13:35:40.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.925+0000 7f0192a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 0x7f016c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:40.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.925+0000 7f0192a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 0x7f016c06c8f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0184006fd0 tx=0x7f0184008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:40.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:40.929+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f019415d970 con 0x7f01940b1ba0 2024-02-02T13:35:41.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.156+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f019415d970 con 0x7f01940b1ba0 2024-02-02T13:35:41.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.157+0000 7f018aa00640 1 -- 172.21.15.91:0/1564109925 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7f017c043060 con 0x7f01940b1ba0 2024-02-02T13:35:41.232 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 msgr2=0x7f016c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 0x7f016c06c8f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0184006fd0 tx=0x7f0184008040 comp rx=0 tx=0).stop 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 msgr2=0x7f01940c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f017c000f80 tx=0x7f017c004a40 comp rx=0 tx=0).stop 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 shutdown_connections 2024-02-02T13:35:41.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f016c06a430 0x7f016c06c8f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f019415d590 0x7f019415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01940b1ba0 0x7f01940c6300 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 --2- 172.21.15.91:0/1564109925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940b1360 0x7f019415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.159+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 >> 172.21.15.91:0/1564109925 conn(0x7f01940d0fa0 msgr2=0x7f019415c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.160+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 shutdown_connections 2024-02-02T13:35:41.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:41.160+0000 7f0199a00640 1 -- 172.21.15.91:0/1564109925 wait complete. 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:35:41.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:35:41.235 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:35:41.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.235 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:35:41.293 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42025sec preferred_lft 42025sec 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.294 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:35:41.295 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:35:41.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.296 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:35:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:35:41.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.389 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:35:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:35:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:35:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:35:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:35:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:35:41.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:35:41.441 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:35:41.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.620 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:35:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:35:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:35:41.670 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:35:41.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.670 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:35:41.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:35:41.763 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:35:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:35:41.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:35:41.816 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:35:41.867 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:35:41.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:41.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:35:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:35:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:35:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:35:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:35:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:35:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:35:42.036 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:35:42.036 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:35:42.036 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:35:42.036 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:35:42.040 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:35:42.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:42.040 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:35:42.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:35:42.122 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:35:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:client8610 2024-02-02T13:35:42.176 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:35:42.177 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:35:42.226 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:35:42.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.442+0000 7fa5f1400640 1 Processor -- start 2024-02-02T13:35:42.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.442+0000 7fa5f1400640 1 -- start start 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.443+0000 7fa5f1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5ec0b0e10 0x7fa5ec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.443+0000 7fa5f1400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec070f80 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.443+0000 7fa5f1400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec06c600 con 0x7fa5ec151f90 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.443+0000 7fa5f1400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec15fa30 con 0x7fa5ec0b0e10 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.443+0000 7fa5e1c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa5ec15f5a0 0x7fa5ec0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39404/0 (socket says 172.21.15.91:39404) 2024-02-02T13:35:42.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e1c00640 1 -- 172.21.15.91:0/2810796684 learned_addr learned my addr 172.21.15.91:0/2810796684 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334396778 0 0) 0x7fa5ec070f80 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc003520 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5eb200640 1 --2- 172.21.15.91:0/2810796684 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec0b0e10 0x7fa5ec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242218125 0 0) 0x7fa5cc003520 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec0b0e10 msgr2=0x7fa5ec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 --2- 172.21.15.91:0/2810796684 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec0b0e10 0x7fa5ec0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 >> v1:172.21.15.113:6789/0 conn(0x7fa5ec151f90 legacy=0x7fa5ec06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec152370 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa5d4001670 con 0x7fa5ec15f5a0 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.444+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 >> v1:172.21.15.91:6789/0 conn(0x7fa5ec15f5a0 legacy=0x7fa5ec0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 0x7fa5cc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5cc0047c0 0x7fa5cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc005180 0x7fa5cc00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec15fa30 con 0x7fa5cc003970 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0104c0 con 0x7fa5cc005180 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc010680 con 0x7fa5cc0047c0 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 0x7fa5cc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 -- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5cc0047c0 msgr2=0x7fa5cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:42.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e1c00640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5cc0047c0 0x7fa5cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5cc0047c0 0x7fa5cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 -- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc005180 msgr2=0x7fa5cc00fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc005180 0x7fa5cc00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5ea800640 1 -- 172.21.15.91:0/2810796684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc003520 con 0x7fa5cc003970 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5eb200640 1 -- 172.21.15.91:0/2810796684 reap_dead start 2024-02-02T13:35:42.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.445+0000 7fa5e1c00640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5cc0047c0 0x7fa5cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.446+0000 7fa5ea800640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 0x7fa5cc003d70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc003d80 tx=0x7fa5cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f541272d3bff918 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc032600 con 0x7fa5cc003970 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc02e050 con 0x7fa5cc003970 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5e9e00640 1 -- 172.21.15.91:0/2810796684 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0329e0 con 0x7fa5cc003970 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5f1400640 1 -- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 msgr2=0x7fa5cc003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5f1400640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 0x7fa5cc003d70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc003d80 tx=0x7fa5cc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5f1400640 1 -- 172.21.15.91:0/2810796684 shutdown_connections 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5f1400640 1 --2- 172.21.15.91:0/2810796684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc003970 0x7fa5cc003d70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.447+0000 7fa5f1400640 1 -- 172.21.15.91:0/2810796684 >> 172.21.15.91:0/2810796684 conn(0x7fa5ec0cec80 msgr2=0x7fa5ec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:42.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.448+0000 7fa5f1400640 1 -- 172.21.15.91:0/2810796684 shutdown_connections 2024-02-02T13:35:42.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.448+0000 7fa5f1400640 1 -- 172.21.15.91:0/2810796684 wait complete. 2024-02-02T13:35:42.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.449+0000 7fa5f1400640 1 Processor -- start 2024-02-02T13:35:42.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.449+0000 7fa5f1400640 1 -- start start 2024-02-02T13:35:42.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc003970 0x7fa5ec0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec14c1f0 0x7fa5ec0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec150ea0 con 0x7fa5ec1504e0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0af650 con 0x7fa5cc003970 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.450+0000 7fa5f1400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c3030 con 0x7fa5ec14c1f0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41470/0 (socket says 172.21.15.91:41470) 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 -- 172.21.15.91:0/1084305437 learned_addr learned my addr 172.21.15.91:0/1084305437 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5ea800640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc003970 0x7fa5ec0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 -- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec14c1f0 msgr2=0x7fa5ec0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec14c1f0 0x7fa5ec0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 -- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc003970 msgr2=0x7fa5ec0c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc003970 0x7fa5ec0c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 -- 172.21.15.91:0/1084305437 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec14c670 con 0x7fa5ec1504e0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.451+0000 7fa5eb200640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0cfcf0 tx=0x7fa5e4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:42.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.452+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400ec50 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.452+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400edb0 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.452+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4014660 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.452+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec0c7610 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.452+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec0c7bd0 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.453+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec066cf0 con 0x7fa5ec1504e0 2024-02-02T13:35:42.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.454+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa5e40147c0 con 0x7fa5ec1504e0 2024-02-02T13:35:42.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.455+0000 7fa5e3400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 0x7fa5c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:42.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.455+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5e4088e20 con 0x7fa5ec1504e0 2024-02-02T13:35:42.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.455+0000 7fa5ea800640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 0x7fa5c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:42.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.456+0000 7fa5ea800640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 0x7fa5c406c920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc034000 tx=0x7fa5dc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:42.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.461+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa5e4054370 con 0x7fa5ec1504e0 2024-02-02T13:35:42.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.694+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5ec0cd300 con 0x7fa5ec1504e0 2024-02-02T13:35:42.696 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.695+0000 7fa5e3400640 1 -- 172.21.15.91:0/1084305437 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7fa5e4057990 con 0x7fa5ec1504e0 2024-02-02T13:35:42.696 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:35:42.696 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:35:42.697 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:35:42.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.697+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 msgr2=0x7fa5c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.697+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 0x7fa5c406c920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc034000 tx=0x7fa5dc009030 comp rx=0 tx=0).stop 2024-02-02T13:35:42.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.697+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 msgr2=0x7fa5ec150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:42.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.697+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0cfcf0 tx=0x7fa5e4007f40 comp rx=0 tx=0).stop 2024-02-02T13:35:42.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 shutdown_connections 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa5c406a460 0x7fa5c406c920 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec14c1f0 0x7fa5ec0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5ec1504e0 0x7fa5ec150960 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 --2- 172.21.15.91:0/1084305437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5cc003970 0x7fa5ec0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 >> 172.21.15.91:0/1084305437 conn(0x7fa5ec0cec80 msgr2=0x7fa5ec0cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 shutdown_connections 2024-02-02T13:35:42.700 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:42.698+0000 7fa5f1400640 1 -- 172.21.15.91:0/1084305437 wait complete. 2024-02-02T13:35:42.716 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-02-02T13:35:42.946 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":8439,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.287231582,"mdsmap_epoch":80,"osdmap_epoch":178,"osdmap_epoch_barrier":178,"uptime":15.20102189} 2024-02-02T13:35:42.946 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 8439, "mdsmap_epoch": 80, "osdmap_epoch": 178, "osdmap_epoch_barrier": 178, "rank_uptime": 10.287231582, "state": "up:active", "uptime": 15.20102189, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:35:42.947 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-02-02T13:35:43.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.160+0000 7f57bac00640 1 Processor -- start 2024-02-02T13:35:43.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.161+0000 7f57bac00640 1 -- start start 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f57b40b62f0 0x7f57b406f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57b40b28b0 0x7f57b415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 0x7f57b40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b415b470 con 0x7f57b40b28b0 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b40b2d40 con 0x7f57b40b62f0 2024-02-02T13:35:43.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57bac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40b2fe0 con 0x7f57b415c170 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57b3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57b40b28b0 0x7f57b415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57b3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57b40b28b0 0x7f57b415bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41486/0 (socket says 172.21.15.91:41486) 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57b3400640 1 -- 172.21.15.91:0/4152892735 learned_addr learned my addr 172.21.15.91:0/4152892735 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.162+0000 7f57b3e00640 1 --2- 172.21.15.91:0/4152892735 >> v2:172.21.15.113:3300/0 conn(0x7f57b40b62f0 0x7f57b406f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b8c00640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 0x7f57b40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b3400640 1 -- 172.21.15.91:0/4152892735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 msgr2=0x7f57b40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b3400640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 0x7f57b40cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b3400640 1 -- 172.21.15.91:0/4152892735 >> v2:172.21.15.113:3300/0 conn(0x7f57b40b62f0 msgr2=0x7f57b406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b3400640 1 --2- 172.21.15.91:0/4152892735 >> v2:172.21.15.113:3300/0 conn(0x7f57b40b62f0 0x7f57b406f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b3400640 1 -- 172.21.15.91:0/4152892735 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40b32c0 con 0x7f57b40b28b0 2024-02-02T13:35:43.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.163+0000 7f57b8c00640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 0x7f57b40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:43.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.164+0000 7f57b3400640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40b28b0 0x7f57b415bc30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f57b40cd810 tx=0x7f579c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aed75d2abf12065 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:43.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.164+0000 7f57b2a00640 1 -- 172.21.15.91:0/4152892735 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c031860 con 0x7f57b40b28b0 2024-02-02T13:35:43.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.164+0000 7f57b2a00640 1 -- 172.21.15.91:0/4152892735 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c0319c0 con 0x7f57b40b28b0 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.164+0000 7f57b2a00640 1 -- 172.21.15.91:0/4152892735 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c031d50 con 0x7f57b40b28b0 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 -- 172.21.15.91:0/4152892735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40b28b0 msgr2=0x7f57b415bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40b28b0 0x7f57b415bc30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f57b40cd810 tx=0x7f579c02b900 comp rx=0 tx=0).stop 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 -- 172.21.15.91:0/4152892735 shutdown_connections 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b415c170 0x7f57b40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 --2- 172.21.15.91:0/4152892735 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b40b28b0 0x7f57b415bc30 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 --2- 172.21.15.91:0/4152892735 >> v2:172.21.15.113:3300/0 conn(0x7f57b40b62f0 0x7f57b406f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 -- 172.21.15.91:0/4152892735 >> 172.21.15.91:0/4152892735 conn(0x7f57b406d840 msgr2=0x7f57b406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 -- 172.21.15.91:0/4152892735 shutdown_connections 2024-02-02T13:35:43.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.165+0000 7f57bac00640 1 -- 172.21.15.91:0/4152892735 wait complete. 2024-02-02T13:35:43.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.167+0000 7f57bac00640 1 Processor -- start 2024-02-02T13:35:43.168 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.167+0000 7f57bac00640 1 -- start start 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 0x7f57b40d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40b62f0 0x7f57b4157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40d1020 con 0x7f57b415c170 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40d0750 con 0x7f57b40b28b0 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57bac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40ca8a0 con 0x7f57b40b62f0 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41498/0 (socket says 172.21.15.91:41498) 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 -- 172.21.15.91:0/1679011141 learned_addr learned my addr 172.21.15.91:0/1679011141 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b3400640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40b62f0 0x7f57b4157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b3e00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 0x7f57b40d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 -- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40b62f0 msgr2=0x7f57b4157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:43.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40b62f0 0x7f57b4157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 -- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 msgr2=0x7f57b40d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 0x7f57b40d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b8c00640 1 -- 172.21.15.91:0/1679011141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0094d0 con 0x7f57b415c170 2024-02-02T13:35:43.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.168+0000 7f57b3e00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 0x7f57b40d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57b8c00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00cd80 tx=0x7f57ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00f680 con 0x7f57b415c170 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac00f800 con 0x7f57b415c170 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00fbb0 con 0x7f57b415c170 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b40cab80 con 0x7f57b415c170 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.169+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b40d0c40 con 0x7f57b415c170 2024-02-02T13:35:43.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.170+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4067400 con 0x7f57b415c170 2024-02-02T13:35:43.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.171+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f57ac00b040 con 0x7f57b415c170 2024-02-02T13:35:43.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.172+0000 7f57abe00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 0x7f578c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:35:43.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.172+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57ac089320 con 0x7f57b415c170 2024-02-02T13:35:43.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.172+0000 7f57b3e00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 0x7f578c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:35:43.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.173+0000 7f57b3e00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 0x7f578c06c8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:35:43.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.176+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f57ac054870 con 0x7f57b415c170 2024-02-02T13:35:43.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.368+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f57b40b22c0 con 0x7f57b415c170 2024-02-02T13:35:43.526 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.524+0000 7f57abe00640 1 -- 172.21.15.91:0/1679011141 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v179) v1 ==== 181+0+0 (secure 0 0 0) 0x7f57ac01d0c0 con 0x7f57b415c170 2024-02-02T13:35:43.526 INFO:teuthology.orchestra.run.smithi091.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 msgr2=0x7f578c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 0x7f578c06c8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4008040 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 msgr2=0x7f57b4069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00cd80 tx=0x7f57ac00a650 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 shutdown_connections 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f578c06a430 0x7f578c06c8f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57b40b62f0 0x7f57b4157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b415c170 0x7f57b4069c50 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 --2- 172.21.15.91:0/1679011141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57b40b28b0 0x7f57b40d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 >> 172.21.15.91:0/1679011141 conn(0x7f57b406d840 msgr2=0x7f57b406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 shutdown_connections 2024-02-02T13:35:43.530 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:35:43.528+0000 7f57bac00640 1 -- 172.21.15.91:0/1679011141 wait complete. 2024-02-02T13:35:43.550 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:35:43.774 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:35:43.774 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:35:43.774 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-02-02T13:35:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:{"mon_tick_interval":"5"} 2024-02-02T13:35:43.986 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-02-02T13:35:43.986 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-02-02T13:35:43.986 DEBUG:teuthology.orchestra.run.smithi091:> adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T13:35:43.986 DEBUG:teuthology.orchestra.run.smithi091:> import time 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> import datetime 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> import os 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> # Write some buffered data through before going full, all should be well 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> print("writing some data through which we expect to succeed") 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> bytes = 0 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> # buffered IO, the error should always only appear from fsync and not 2024-02-02T13:35:43.987 DEBUG:teuthology.orchestra.run.smithi091:> # from write 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> full = False 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> for n in range(0, int(32 * 1.1)): 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> print("wrote bytes via buffered write, moving on to fsync") 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> except OSError as e: 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> if False: 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> raise 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> full = True 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> break 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> try: 2024-02-02T13:35:43.988 DEBUG:teuthology.orchestra.run.smithi091:> os.fsync(f) 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> print("fsync'"'"'ed successfully") 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> except OSError as e: 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> full = True 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> break 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> else: 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> if n > 32 * 0.9: 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> # Be cautious in the last region where we expect to hit 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> time.sleep(15.0) 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> if not full: 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.989 DEBUG:teuthology.orchestra.run.smithi091:> # close() should not raise an error because we already caught it in 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> # since then because all IOs got cancelled on the full flag. 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> print("calling close") 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> os.close(f) 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> print("close() did not raise error") 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-02-02T13:35:43.990 DEBUG:teuthology.orchestra.run.smithi091:> ' 2024-02-02T13:36:02.122 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:36:02.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:36:32.352 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:36:32.361 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:36:48.219 INFO:teuthology.orchestra.run.smithi091.stdout:writing some data through which we expect to succeed 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 1.00 MB 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 2.00 MB 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 3.00 MB 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 4.00 MB 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 5.00 MB 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 6.00 MB 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 7.00 MB 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 8.00 MB 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.221 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 9.00 MB 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 10.00 MB 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 11.00 MB 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 12.00 MB 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 13.00 MB 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 14.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 15.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 16.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 17.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 18.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 19.00 MB 2024-02-02T13:36:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 20.00 MB 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 21.00 MB 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 22.00 MB 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 23.00 MB 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 24.00 MB 2024-02-02T13:36:48.224 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 25.00 MB 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 26.00 MB 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 27.00 MB 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 28.00 MB 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 29.00 MB 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 30.00 MB 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 31.00 MB 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 32.00 MB 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:wrote bytes via buffered write, moving on to fsync 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:fsync'ed successfully 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:Not full yet after 33.00 MB 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:Reached fullness after 33.00 MB 2024-02-02T13:36:48.226 INFO:teuthology.orchestra.run.smithi091.stdout:calling close 2024-02-02T13:36:48.227 INFO:teuthology.orchestra.run.smithi091.stdout:close() did not raise error 2024-02-02T13:36:48.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:36:48.420 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:36:48.542 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:36:48.660 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:36:48.660 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:36:48.660 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:48.717 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:36:48.718 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:48.768 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:36:48.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.981+0000 7f0d5ae00640 1 Processor -- start 2024-02-02T13:36:48.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.981+0000 7f0d5ae00640 1 -- start start 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d5ae00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d54160a20 0x7f0d540b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d5ae00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d540b5010 0x7f0d540b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d5ae00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540b7730 con 0x7f0d540b4ba0 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d5ae00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d540b85e0 con 0x7f0d54160a20 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d5ae00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d540b9140 con 0x7f0d540b5010 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d53e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d54160a20 0x7f0d540b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d4b200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0d540b4ba0 0x7f0d540d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:60594/0 (socket says 172.21.15.91:60594) 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d4b200640 1 -- 172.21.15.91:0/1567966778 learned_addr learned my addr 172.21.15.91:0/1567966778 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d58e00640 1 --2- 172.21.15.91:0/1567966778 >> v2:172.21.15.113:3300/0 conn(0x7f0d540b5010 0x7f0d540b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d53e00640 1 -- 172.21.15.91:0/1567966778 >> v2:172.21.15.113:3300/0 conn(0x7f0d540b5010 msgr2=0x7f0d540b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d53e00640 1 --2- 172.21.15.91:0/1567966778 >> v2:172.21.15.113:3300/0 conn(0x7f0d540b5010 0x7f0d540b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d53e00640 1 -- 172.21.15.91:0/1567966778 >> v1:172.21.15.91:6789/0 conn(0x7f0d540b4ba0 legacy=0x7f0d540d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:48.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.984+0000 7f0d53e00640 1 -- 172.21.15.91:0/1567966778 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540b7960 con 0x7f0d54160a20 2024-02-02T13:36:48.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.983+0000 7f0d53400640 1 -- 172.21.15.91:0/1567966778 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173302113 0 0) 0x7f0d540b7730 con 0x7f0d540b4ba0 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d53e00640 1 --2- 172.21.15.91:0/1567966778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540b8c00 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0d540cd920 tx=0x7f0d4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc816d825876d17c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d53400640 1 -- 172.21.15.91:0/1567966778 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44031830 con 0x7f0d54160a20 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d53400640 1 -- 172.21.15.91:0/1567966778 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44031990 con 0x7f0d54160a20 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d53400640 1 -- 172.21.15.91:0/1567966778 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44031d20 con 0x7f0d54160a20 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d5ae00640 1 -- 172.21.15.91:0/1567966778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 msgr2=0x7f0d540b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/1567966778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540b8c00 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0d540cd920 tx=0x7f0d4402b8d0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d5ae00640 1 -- 172.21.15.91:0/1567966778 shutdown_connections 2024-02-02T13:36:48.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/1567966778 >> v2:172.21.15.113:3300/0 conn(0x7f0d540b5010 0x7f0d540b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.985+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/1567966778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540b8c00 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.986+0000 7f0d5ae00640 1 -- 172.21.15.91:0/1567966778 >> 172.21.15.91:0/1567966778 conn(0x7f0d54070480 msgr2=0x7f0d54070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.986+0000 7f0d5ae00640 1 -- 172.21.15.91:0/1567966778 shutdown_connections 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.986+0000 7f0d5ae00640 1 -- 172.21.15.91:0/1567966778 wait complete. 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.987+0000 7f0d5ae00640 1 Processor -- start 2024-02-02T13:36:48.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.987+0000 7f0d5ae00640 1 -- start start 2024-02-02T13:36:48.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d540b4ba0 0x7f0d541f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d540b5010 0x7f0d5415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541f3b00 con 0x7f0d54160a20 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540aa2f0 con 0x7f0d540b4ba0 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.988+0000 7f0d5ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540aa5d0 con 0x7f0d540b5010 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39190/0 (socket says 172.21.15.91:39190) 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 -- 172.21.15.91:0/413521350 learned_addr learned my addr 172.21.15.91:0/413521350 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 -- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d540b5010 msgr2=0x7f0d5415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d540b5010 0x7f0d5415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d53e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d540b4ba0 0x7f0d541f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 -- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d540b4ba0 msgr2=0x7f0d541f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d540b4ba0 0x7f0d541f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009400 con 0x7f0d54160a20 2024-02-02T13:36:48.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.989+0000 7f0d58e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c007e40 tx=0x7f0d4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:48.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.990+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c014620 con 0x7f0d54160a20 2024-02-02T13:36:48.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.990+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c00ee90 con 0x7f0d54160a20 2024-02-02T13:36:48.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.990+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0149e0 con 0x7f0d54160a20 2024-02-02T13:36:48.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.990+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d540aa8b0 con 0x7f0d54160a20 2024-02-02T13:36:48.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.990+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d540aae70 con 0x7f0d54160a20 2024-02-02T13:36:48.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.991+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d540ab090 con 0x7f0d54160a20 2024-02-02T13:36:48.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.992+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d4c014b40 con 0x7f0d54160a20 2024-02-02T13:36:48.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.993+0000 7f0d50c00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 0x7f0d2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:48.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.993+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0d4c0892f0 con 0x7f0d54160a20 2024-02-02T13:36:48.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.993+0000 7f0d53e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 0x7f0d2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:48.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.993+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f0d2c071fc0 con 0x7f0d54160a20 2024-02-02T13:36:48.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.994+0000 7f0d53e00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 0x7f0d2c06c8f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d44036000 tx=0x7f0d4403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:48.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:48.997+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d4c0548b0 con 0x7f0d54160a20 2024-02-02T13:36:49.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.190+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d54066cf0 con 0x7f0d54160a20 2024-02-02T13:36:49.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.192+0000 7f0d50c00640 1 -- 172.21.15.91:0/413521350 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16188 (secure 0 0 0) 0x7f0d4c057ed0 con 0x7f0d54160a20 2024-02-02T13:36:49.194 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:49.194 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":182,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:36:28.579012+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.28","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:36:49.197 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 msgr2=0x7f0d2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 0x7f0d2c06c8f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d44036000 tx=0x7f0d4403c040 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 msgr2=0x7f0d540abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c007e40 tx=0x7f0d4c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 shutdown_connections 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0d2c06a430 0x7f0d2c06c8f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d540b5010 0x7f0d5415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d54160a20 0x7f0d540abb70 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 --2- 172.21.15.91:0/413521350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d540b4ba0 0x7f0d541f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.195+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 >> 172.21.15.91:0/413521350 conn(0x7f0d54070480 msgr2=0x7f0d540c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.196+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 shutdown_connections 2024-02-02T13:36:49.198 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.196+0000 7f0d5ae00640 1 -- 172.21.15.91:0/413521350 wait complete. 2024-02-02T13:36:49.216 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:35:31.112699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:35:31.642725+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:36:49.216 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:36:49.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.421+0000 7fb211200640 1 Processor -- start 2024-02-02T13:36:49.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.421+0000 7fb211200640 1 -- start start 2024-02-02T13:36:49.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb211200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb20c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb211200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb20c0b0e10 0x7fb20c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb211200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c070f80 con 0x7fb20c06f4f0 2024-02-02T13:36:49.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb211200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c152370 con 0x7fb20c0b0e10 2024-02-02T13:36:49.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb211200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0cb010 con 0x7fb20c151f90 2024-02-02T13:36:49.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb209400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb20c06f4f0 0x7fb20c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:60598/0 (socket says 172.21.15.91:60598) 2024-02-02T13:36:49.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb209400640 1 -- 172.21.15.91:0/355106461 learned_addr learned my addr 172.21.15.91:0/355106461 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb209e00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb20c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb20a800640 1 --2- 172.21.15.91:0/355106461 >> v2:172.21.15.113:3300/0 conn(0x7fb20c0b0e10 0x7fb20c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020591099 0 0) 0x7fb20c070f80 con 0x7fb20c06f4f0 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.423+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1ec003520 con 0x7fb20c06f4f0 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801854658 0 0) 0x7fb1ec003520 con 0x7fb20c06f4f0 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 msgr2=0x7fb20c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb20c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 >> v2:172.21.15.113:3300/0 conn(0x7fb20c0b0e10 msgr2=0x7fb20c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 --2- 172.21.15.91:0/355106461 >> v2:172.21.15.113:3300/0 conn(0x7fb20c0b0e10 0x7fb20c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0cb2f0 con 0x7fb20c06f4f0 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fb1f4001670 con 0x7fb20c06f4f0 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 >> v1:172.21.15.91:6789/0 conn(0x7fb20c06f4f0 legacy=0x7fb20c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec003970 0x7fb1ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb1ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1ec004d00 0x7fb1ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c152370 con 0x7fb1ec004d00 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0cb010 con 0x7fb1ec003970 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec00b980 con 0x7fb20c151f90 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.424+0000 7fb20a800640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1ec004d00 0x7fb1ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb209e00640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec003970 0x7fb1ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb209400640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb1ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb20a800640 1 -- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 msgr2=0x7fb1ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb20a800640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c151f90 0x7fb1ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb20a800640 1 -- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec003970 msgr2=0x7fb1ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb20a800640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec003970 0x7fb1ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.425+0000 7fb20a800640 1 -- 172.21.15.91:0/355106461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec003520 con 0x7fb1ec004d00 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.426+0000 7fb20a800640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1ec004d00 0x7fb1ec005180 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb204007330 tx=0x7fb1ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ddb105bbaca8b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.426+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20402cc80 con 0x7fb1ec004d00 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20402cde0 con 0x7fb1ec004d00 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb208a00640 1 -- 172.21.15.91:0/355106461 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204034660 con 0x7fb1ec004d00 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 -- 172.21.15.91:0/355106461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1ec004d00 msgr2=0x7fb1ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 --2- 172.21.15.91:0/355106461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1ec004d00 0x7fb1ec005180 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb204007330 tx=0x7fb1ec0057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb20a800640 1 -- 172.21.15.91:0/355106461 reap_dead start 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 -- 172.21.15.91:0/355106461 shutdown_connections 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 -- 172.21.15.91:0/355106461 >> 172.21.15.91:0/355106461 conn(0x7fb20c0cec80 msgr2=0x7fb20c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 -- 172.21.15.91:0/355106461 shutdown_connections 2024-02-02T13:36:49.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.427+0000 7fb211200640 1 -- 172.21.15.91:0/355106461 wait complete. 2024-02-02T13:36:49.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.429+0000 7fb211200640 1 Processor -- start 2024-02-02T13:36:49.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.429+0000 7fb211200640 1 -- start start 2024-02-02T13:36:49.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec004d00 0x7fb20c151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c152ea0 0x7fb20c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c155320 con 0x7fb20c153320 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c154ac0 con 0x7fb1ec004d00 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.430+0000 7fb211200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c154d60 con 0x7fb20c152ea0 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39212/0 (socket says 172.21.15.91:39212) 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 -- 172.21.15.91:0/642029954 learned_addr learned my addr 172.21.15.91:0/642029954 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb209400640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c152ea0 0x7fb20c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb209e00640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec004d00 0x7fb20c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 -- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c152ea0 msgr2=0x7fb20c0c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c152ea0 0x7fb20c0c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 -- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec004d00 msgr2=0x7fb20c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec004d00 0x7fb20c151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c155040 con 0x7fb20c153320 2024-02-02T13:36:49.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.431+0000 7fb20a800640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb204002230 tx=0x7fb204004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.432+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20403e600 con 0x7fb20c153320 2024-02-02T13:36:49.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.432+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20403e760 con 0x7fb20c153320 2024-02-02T13:36:49.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.432+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20403eb10 con 0x7fb20c153320 2024-02-02T13:36:49.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.432+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c0cc8f0 con 0x7fb20c153320 2024-02-02T13:36:49.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.432+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c1f3910 con 0x7fb20c153320 2024-02-02T13:36:49.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.433+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c1f3aa0 con 0x7fb20c153320 2024-02-02T13:36:49.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.434+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb20403ec70 con 0x7fb20c153320 2024-02-02T13:36:49.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.435+0000 7fb202000640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 0x7fb1e406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.435+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb2040aae10 con 0x7fb20c153320 2024-02-02T13:36:49.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.435+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fb1e40720c0 con 0x7fb20c153320 2024-02-02T13:36:49.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.435+0000 7fb209e00640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 0x7fb1e406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.435+0000 7fb209e00640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 0x7fb1e406c9f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098d0 tx=0x7fb1fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.439+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb204039020 con 0x7fb20c153320 2024-02-02T13:36:49.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.628+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb20c0cce10 con 0x7fb20c153320 2024-02-02T13:36:49.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.629+0000 7fb202000640 1 -- 172.21.15.91:0/642029954 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb20c0cce10 con 0x7fb20c153320 2024-02-02T13:36:49.631 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:36:49.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 msgr2=0x7fb1e406c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 0x7fb1e406c9f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098d0 tx=0x7fb1fc008040 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 msgr2=0x7fb20c0c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb204002230 tx=0x7fb204004ab0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 shutdown_connections 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb1e406a530 0x7fb1e406c9f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb20c152ea0 0x7fb20c0c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb20c153320 0x7fb20c0c43b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 --2- 172.21.15.91:0/642029954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ec004d00 0x7fb20c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 >> 172.21.15.91:0/642029954 conn(0x7fb20c0cec80 msgr2=0x7fb20c0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 shutdown_connections 2024-02-02T13:36:49.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.632+0000 7fb211200640 1 -- 172.21.15.91:0/642029954 wait complete. 2024-02-02T13:36:49.651 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:36:49.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.848+0000 7f5dd6400640 1 Processor -- start 2024-02-02T13:36:49.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.848+0000 7f5dd6400640 1 -- start start 2024-02-02T13:36:49.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.849+0000 7f5dd6400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5dd015d490 0x7f5dd006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.849+0000 7f5dd6400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5dd00ac8d0 0x7f5dd00acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.849+0000 7f5dd6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00b0700 0x7f5dd00b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.849+0000 7f5dd6400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00ad2a0 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.849+0000 7f5dd6400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00b10b0 con 0x7f5dd015d490 2024-02-02T13:36:49.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dd6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd015d870 con 0x7f5dd00b0700 2024-02-02T13:36:49.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5dd00ac8d0 0x7f5dd00acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5dd00ac8d0 0x7f5dd00acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39224/0 (socket says 172.21.15.91:39224) 2024-02-02T13:36:49.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 -- 172.21.15.91:0/3304200211 learned_addr learned my addr 172.21.15.91:0/3304200211 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 -- 172.21.15.91:0/3304200211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00b0700 msgr2=0x7f5dd00b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3304200211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00b0700 0x7f5dd00b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dcf400640 1 --2- 172.21.15.91:0/3304200211 >> v2:172.21.15.113:3300/0 conn(0x7f5dd015d490 0x7f5dd006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 -- 172.21.15.91:0/3304200211 >> v2:172.21.15.113:3300/0 conn(0x7f5dd015d490 msgr2=0x7f5dd006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3304200211 >> v2:172.21.15.113:3300/0 conn(0x7f5dd015d490 0x7f5dd006c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.850+0000 7f5dc6800640 1 -- 172.21.15.91:0/3304200211 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd006bce0 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.851+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3304200211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00ac8d0 0x7f5dd00acd60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00ae6b0 tx=0x7f5db802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271b1026a7170a0a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dcea00640 1 -- 172.21.15.91:0/3304200211 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8031640 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dcea00640 1 -- 172.21.15.91:0/3304200211 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db80317a0 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dcea00640 1 -- 172.21.15.91:0/3304200211 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8031b30 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 -- 172.21.15.91:0/3304200211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00ac8d0 msgr2=0x7f5dd00acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3304200211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00ac8d0 0x7f5dd00acd60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00ae6b0 tx=0x7f5db802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 -- 172.21.15.91:0/3304200211 shutdown_connections 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3304200211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00b0700 0x7f5dd00b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3304200211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00ac8d0 0x7f5dd00acd60 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.852+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3304200211 >> v2:172.21.15.113:3300/0 conn(0x7f5dd015d490 0x7f5dd006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.853+0000 7f5dd6400640 1 -- 172.21.15.91:0/3304200211 >> 172.21.15.91:0/3304200211 conn(0x7f5dd00cec80 msgr2=0x7f5dd00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.853+0000 7f5dd6400640 1 -- 172.21.15.91:0/3304200211 shutdown_connections 2024-02-02T13:36:49.855 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.853+0000 7f5dd6400640 1 -- 172.21.15.91:0/3304200211 wait complete. 2024-02-02T13:36:49.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.855+0000 7f5dd6400640 1 Processor -- start 2024-02-02T13:36:49.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.855+0000 7f5dd6400640 1 -- start start 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00ac8d0 0x7f5dd01f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dd015d490 0x7f5dd014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd015c8f0 con 0x7f5dd00b0700 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00c2cf0 con 0x7f5dd015d490 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dd6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0151770 con 0x7f5dd00ac8d0 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.856+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:39240/0 (socket says 172.21.15.91:39240) 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 -- 172.21.15.91:0/3785446320 learned_addr learned my addr 172.21.15.91:0/3785446320 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:49.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dcf400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00ac8d0 0x7f5dd01f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dcfe00640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dd015d490 0x7f5dd014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 -- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00ac8d0 msgr2=0x7f5dd01f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00ac8d0 0x7f5dd01f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 -- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dd015d490 msgr2=0x7f5dd014f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dd015d490 0x7f5dd014f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80094d0 con 0x7f5dd00b0700 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.857+0000 7f5dc6800640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.858+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db803ac60 con 0x7f5dd00b0700 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.858+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db803ade0 con 0x7f5dd00b0700 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.858+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8044630 con 0x7f5dd00b0700 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.858+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd00c2f40 con 0x7f5dd00b0700 2024-02-02T13:36:49.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.858+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd00c3500 con 0x7f5dd00b0700 2024-02-02T13:36:49.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.859+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd00c36f0 con 0x7f5dd00b0700 2024-02-02T13:36:49.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.860+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5db802f050 con 0x7f5dd00b0700 2024-02-02T13:36:49.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.860+0000 7f5dc7e00640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 0x7f5da806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:49.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.861+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5db80aa9b0 con 0x7f5dd00b0700 2024-02-02T13:36:49.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.861+0000 7f5dcf400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 0x7f5da806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:49.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.861+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f5da8071fc0 con 0x7f5dd00b0700 2024-02-02T13:36:49.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.861+0000 7f5dcf400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 0x7f5da806c8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000a970 tx=0x7f5dc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:49.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:49.865+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5db80760d0 con 0x7f5dd00b0700 2024-02-02T13:36:50.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.055+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5dd0066cf0 con 0x7f5dd00b0700 2024-02-02T13:36:50.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.055+0000 7f5dc7e00640 1 -- 172.21.15.91:0/3785446320 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5db8035070 con 0x7f5dd00b0700 2024-02-02T13:36:50.057 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 msgr2=0x7f5da806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 0x7f5da806c8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000a970 tx=0x7f5dc0008040 comp rx=0 tx=0).stop 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 msgr2=0x7f5dd014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db80046c0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 shutdown_connections 2024-02-02T13:36:50.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f5da806a430 0x7f5da806c8f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dd015d490 0x7f5dd014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5dd00b0700 0x7f5dd014eae0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 --2- 172.21.15.91:0/3785446320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dd00ac8d0 0x7f5dd01f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 >> 172.21.15.91:0/3785446320 conn(0x7f5dd00cec80 msgr2=0x7f5dd00c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 shutdown_connections 2024-02-02T13:36:50.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.058+0000 7f5dd6400640 1 -- 172.21.15.91:0/3785446320 wait complete. 2024-02-02T13:36:50.076 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:36:50.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.291+0000 7fa6c1600640 1 Processor -- start 2024-02-02T13:36:50.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.291+0000 7fa6c1600640 1 -- start start 2024-02-02T13:36:50.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.292+0000 7fa6c1600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 0x7fa6bc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.292+0000 7fa6c1600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0cc7c0 con 0x7fa6bc15a010 2024-02-02T13:36:50.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.292+0000 7fa6c1600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0ce930 con 0x7fa6bc0ce130 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.292+0000 7fa6c1600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0cd640 con 0x7fa6bc0b0e10 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6baa00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa6bc15a010 0x7fa6bc0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49272/0 (socket says 172.21.15.91:49272) 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6baa00640 1 -- 172.21.15.91:0/1055333193 learned_addr learned my addr 172.21.15.91:0/1055333193 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427025372 0 0) 0x7fa6bc0cc7c0 con 0x7fa6bc15a010 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa69c003520 con 0x7fa6bc15a010 2024-02-02T13:36:50.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6bb400640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 0x7fa6bc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231893959 0 0) 0x7fa69c003520 con 0x7fa6bc15a010 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 msgr2=0x7fa6bc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 0x7fa6bc0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 >> v1:172.21.15.113:6789/0 conn(0x7fa6bc0ce130 legacy=0x7fa6bc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc06c890 con 0x7fa6bc15a010 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa6ac001670 con 0x7fa6bc15a010 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 >> v1:172.21.15.91:6789/0 conn(0x7fa6bc15a010 legacy=0x7fa6bc0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.293+0000 7fa6ba000640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa69c0039a0 0x7fa69c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6ba000640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0047c0 0x7fa69c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6ba000640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 0x7fa69c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0cd640 con 0x7fa69c0039a0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c00dd50 con 0x7fa69c0047c0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c00df10 con 0x7fa6bc0b0e10 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa69c0039a0 0x7fa69c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 -- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 msgr2=0x7fa69c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0b0e10 0x7fa69c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 -- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0047c0 msgr2=0x7fa69c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6b3e00640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0047c0 0x7fa69c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0047c0 0x7fa69c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.294+0000 7fa6baa00640 1 -- 172.21.15.91:0/1055333193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c003520 con 0x7fa69c0039a0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.295+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa69c0039a0 0x7fa69c003dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c0e0 tx=0x7fa69c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7ba4eccd26120c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac005ad0 con 0x7fa69c0039a0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac005c30 con 0x7fa69c0039a0 2024-02-02T13:36:50.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6ba000640 1 -- 172.21.15.91:0/1055333193 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac02c810 con 0x7fa69c0039a0 2024-02-02T13:36:50.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 -- 172.21.15.91:0/1055333193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa69c0039a0 msgr2=0x7fa69c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1055333193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa69c0039a0 0x7fa69c003dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c0e0 tx=0x7fa69c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6bb400640 1 -- 172.21.15.91:0/1055333193 reap_dead start 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 -- 172.21.15.91:0/1055333193 shutdown_connections 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 -- 172.21.15.91:0/1055333193 >> 172.21.15.91:0/1055333193 conn(0x7fa6bc0cec80 msgr2=0x7fa6bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 -- 172.21.15.91:0/1055333193 shutdown_connections 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.296+0000 7fa6c1600640 1 -- 172.21.15.91:0/1055333193 wait complete. 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.298+0000 7fa6c1600640 1 Processor -- start 2024-02-02T13:36:50.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.298+0000 7fa6c1600640 1 -- start start 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 0x7fa6bc154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0aeb70 0x7fa6bc0af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0ae750 con 0x7fa6bc1f2710 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0af540 con 0x7fa69c0039a0 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6c1600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc15c830 con 0x7fa6bc0aeb70 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.299+0000 7fa6b3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60572/0 (socket says 172.21.15.91:60572) 2024-02-02T13:36:50.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 -- 172.21.15.91:0/1688450969 learned_addr learned my addr 172.21.15.91:0/1688450969 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 0x7fa6bc154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 -- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0aeb70 msgr2=0x7fa6bc0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0aeb70 0x7fa6bc0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 -- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 msgr2=0x7fa6bc154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 0x7fa6bc154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc15cb10 con 0x7fa6bc1f2710 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 0x7fa6bc154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b3e00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d560 tx=0x7fa6a400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4005480 con 0x7fa6bc1f2710 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a4003610 con 0x7fa6bc1f2710 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4003980 con 0x7fa6bc1f2710 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.300+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc15cdf0 con 0x7fa6bc1f2710 2024-02-02T13:36:50.303 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.301+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc15c500 con 0x7fa6bc1f2710 2024-02-02T13:36:50.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.302+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc066cf0 con 0x7fa6bc1f2710 2024-02-02T13:36:50.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.303+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa6a4003ae0 con 0x7fa6bc1f2710 2024-02-02T13:36:50.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.304+0000 7fa6b2a00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 0x7fa69406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.304+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa6a40888e0 con 0x7fa6bc1f2710 2024-02-02T13:36:50.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.304+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fa694071fc0 con 0x7fa6bc1f2710 2024-02-02T13:36:50.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.304+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 0x7fa69406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.305+0000 7fa6baa00640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 0x7fa69406c8f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac034000 tx=0x7fa6ac009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.308+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa6a4088030 con 0x7fa6bc1f2710 2024-02-02T13:36:50.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.480+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6bc0b4c00 con 0x7fa6bc1f2710 2024-02-02T13:36:50.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.481+0000 7fa6b2a00640 1 -- 172.21.15.91:0/1688450969 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6a4088030 con 0x7fa6bc1f2710 2024-02-02T13:36:50.482 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 msgr2=0x7fa69406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 0x7fa69406c8f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac034000 tx=0x7fa6ac009030 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 msgr2=0x7fa6bc1f2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d560 tx=0x7fa6a400da20 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 shutdown_connections 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa69406a430 0x7fa69406c8f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6bc0aeb70 0x7fa6bc0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa6bc1f2710 0x7fa6bc1f2b90 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 --2- 172.21.15.91:0/1688450969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa69c0039a0 0x7fa6bc154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 >> 172.21.15.91:0/1688450969 conn(0x7fa6bc0cec80 msgr2=0x7fa6bc1590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.483+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 shutdown_connections 2024-02-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.484+0000 7fa6c1600640 1 -- 172.21.15.91:0/1688450969 wait complete. 2024-02-02T13:36:50.503 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:36:50.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.704+0000 7eff67200640 1 Processor -- start 2024-02-02T13:36:50.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.705+0000 7eff67200640 1 -- start start 2024-02-02T13:36:50.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff67200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff600ce4b0 0x7eff601680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff67200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600704a0 con 0x7eff6015d3e0 2024-02-02T13:36:50.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff67200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600d1f60 con 0x7eff600d1a60 2024-02-02T13:36:50.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff67200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff600d2800 con 0x7eff600ce4b0 2024-02-02T13:36:50.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff5fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7eff6015d3e0 0x7eff600b88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49278/0 (socket says 172.21.15.91:49278) 2024-02-02T13:36:50.708 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff5fe00640 1 -- 172.21.15.91:0/3769184254 learned_addr learned my addr 172.21.15.91:0/3769184254 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:50.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.706+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710465313 0 0) 0x7eff600704a0 con 0x7eff6015d3e0 2024-02-02T13:36:50.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff40003520 con 0x7eff6015d3e0 2024-02-02T13:36:50.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> v2:172.21.15.113:3300/0 conn(0x7eff600ce4b0 0x7eff601680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992250824 0 0) 0x7eff40003520 con 0x7eff6015d3e0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 >> v2:172.21.15.113:3300/0 conn(0x7eff600ce4b0 msgr2=0x7eff601680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 --2- 172.21.15.91:0/3769184254 >> v2:172.21.15.113:3300/0 conn(0x7eff600ce4b0 0x7eff601680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 >> v1:172.21.15.113:6789/0 conn(0x7eff600d1a60 legacy=0x7eff600cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600d2240 con 0x7eff6015d3e0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7eff50005090 con 0x7eff6015d3e0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> v2:172.21.15.113:3300/0 conn(0x7eff600ce4b0 0x7eff601680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 >> v1:172.21.15.91:6789/0 conn(0x7eff6015d3e0 legacy=0x7eff600b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff40003970 0x7eff40003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff400047c0 0x7eff40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600d2800 con 0x7eff400052a0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4000c1a0 con 0x7eff400047c0 2024-02-02T13:36:50.711 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4000c360 con 0x7eff40003970 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.707+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff5f400640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff400047c0 0x7eff40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff5fe00640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff40003970 0x7eff40003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff65200640 1 -- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff40003970 msgr2=0x7eff40003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff40003970 0x7eff40003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff65200640 1 -- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff400047c0 msgr2=0x7eff40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff400047c0 0x7eff40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.708+0000 7eff65200640 1 -- 172.21.15.91:0/3769184254 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40003520 con 0x7eff400052a0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.709+0000 7eff65200640 1 -- 172.21.15.91:0/3769184254 reap_dead start 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.709+0000 7eff65200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff4000b980 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7eff6008c2d0 tx=0x7eff400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475dbbbd182b4b6f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.709+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58034630 con 0x7eff400052a0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.709+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff580347b0 con 0x7eff400052a0 2024-02-02T13:36:50.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.709+0000 7eff5ea00640 1 -- 172.21.15.91:0/3769184254 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58034b60 con 0x7eff400052a0 2024-02-02T13:36:50.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 -- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 msgr2=0x7eff4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff4000b980 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7eff6008c2d0 tx=0x7eff400057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 -- 172.21.15.91:0/3769184254 shutdown_connections 2024-02-02T13:36:50.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 --2- 172.21.15.91:0/3769184254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff4000b980 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 -- 172.21.15.91:0/3769184254 >> 172.21.15.91:0/3769184254 conn(0x7eff600cfa50 msgr2=0x7eff600cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 -- 172.21.15.91:0/3769184254 shutdown_connections 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.710+0000 7eff67200640 1 -- 172.21.15.91:0/3769184254 wait complete. 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.711+0000 7eff67200640 1 Processor -- start 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.712+0000 7eff67200640 1 -- start start 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.712+0000 7eff67200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.712+0000 7eff67200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff600d0e40 0x7eff600d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff67200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6015e1d0 0x7eff6015e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff67200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600d1800 con 0x7eff400052a0 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff67200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6015ef00 con 0x7eff600d0e40 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff67200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6015f1e0 con 0x7eff6015e1d0 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60586/0 (socket says 172.21.15.91:60586) 2024-02-02T13:36:50.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 -- 172.21.15.91:0/3460183237 learned_addr learned my addr 172.21.15.91:0/3460183237 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 -- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6015e1d0 msgr2=0x7eff6015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6015e1d0 0x7eff6015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 -- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff600d0e40 msgr2=0x7eff600d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff600d0e40 0x7eff600d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6015f4c0 con 0x7eff400052a0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.713+0000 7eff5fe00640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7eff5000d7a0 tx=0x7eff5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.714+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50007270 con 0x7eff400052a0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.714+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000b6b0 con 0x7eff400052a0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.714+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6015f7a0 con 0x7eff400052a0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.714+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ba20 con 0x7eff400052a0 2024-02-02T13:36:50.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.714+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff60150a80 con 0x7eff400052a0 2024-02-02T13:36:50.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.715+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6015f9e0 con 0x7eff400052a0 2024-02-02T13:36:50.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.716+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7eff5000bb80 con 0x7eff400052a0 2024-02-02T13:36:50.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.717+0000 7eff57e00640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 0x7eff3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:50.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.717+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7eff50088a90 con 0x7eff400052a0 2024-02-02T13:36:50.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.717+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7eff38071fc0 con 0x7eff400052a0 2024-02-02T13:36:50.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.717+0000 7eff5f400640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 0x7eff3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:50.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.718+0000 7eff5f400640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 0x7eff3806c8f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7eff600cc900 tx=0x7eff48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:50.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.721+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7eff6015f9e0 con 0x7eff400052a0 2024-02-02T13:36:50.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.922+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff6015f9e0 con 0x7eff400052a0 2024-02-02T13:36:50.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.923+0000 7eff57e00640 1 -- 172.21.15.91:0/3460183237 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v182) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff6015f9e0 con 0x7eff400052a0 2024-02-02T13:36:50.924 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:36:50.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 msgr2=0x7eff3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 0x7eff3806c8f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7eff600cc900 tx=0x7eff48009040 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 msgr2=0x7eff60150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7eff5000d7a0 tx=0x7eff5000dc60 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 shutdown_connections 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7eff3806a430 0x7eff3806c8f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6015e1d0 0x7eff6015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff600d0e40 0x7eff600d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 --2- 172.21.15.91:0/3460183237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7eff400052a0 0x7eff60150540 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 >> 172.21.15.91:0/3460183237 conn(0x7eff600cfa50 msgr2=0x7eff600ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 shutdown_connections 2024-02-02T13:36:50.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:50.925+0000 7eff67200640 1 -- 172.21.15.91:0/3460183237 wait complete. 2024-02-02T13:36:50.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:36:50.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:36:50.946 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:36:51.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:36:51.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:36:51.043 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:36:51.091 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:36:51.091 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:51.149 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:36:51.149 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:51.201 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:36:51.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.419+0000 7fd771a00640 1 Processor -- start 2024-02-02T13:36:51.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.420+0000 7fd771a00640 1 -- start start 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd771a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd76c15a010 0x7fd76c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd771a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd76c0ce130 0x7fd76c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd771a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c0cc7c0 con 0x7fd76c0cd470 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd771a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c0ce930 con 0x7fd76c15a010 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd771a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c06c890 con 0x7fd76c0ce130 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd76aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd76c15a010 0x7fd76c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd76aa00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd76c15a010 0x7fd76c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60602/0 (socket says 172.21.15.91:60602) 2024-02-02T13:36:51.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.421+0000 7fd76aa00640 1 -- 172.21.15.91:0/2422753779 learned_addr learned my addr 172.21.15.91:0/2422753779 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:51.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.422+0000 7fd76aa00640 1 -- 172.21.15.91:0/2422753779 >> v2:172.21.15.113:3300/0 conn(0x7fd76c0ce130 msgr2=0x7fd76c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:51.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.422+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2422753779 >> v2:172.21.15.113:3300/0 conn(0x7fd76c0ce130 0x7fd76c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.422+0000 7fd76aa00640 1 -- 172.21.15.91:0/2422753779 >> v1:172.21.15.113:6789/0 conn(0x7fd76c0cd470 legacy=0x7fd76c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:51.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.422+0000 7fd76aa00640 1 -- 172.21.15.91:0/2422753779 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c06cb70 con 0x7fd76c15a010 2024-02-02T13:36:51.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.423+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2422753779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c15a010 0x7fd76c0c9670 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0b1870 tx=0x7fd75c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5efceab2e15f5d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.423+0000 7fd76a000640 1 -- 172.21.15.91:0/2422753779 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0317e0 con 0x7fd76c15a010 2024-02-02T13:36:51.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.423+0000 7fd76a000640 1 -- 172.21.15.91:0/2422753779 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c031940 con 0x7fd76c15a010 2024-02-02T13:36:51.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.423+0000 7fd76a000640 1 -- 172.21.15.91:0/2422753779 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c031cd0 con 0x7fd76c15a010 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 -- 172.21.15.91:0/2422753779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c15a010 msgr2=0x7fd76c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 --2- 172.21.15.91:0/2422753779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c15a010 0x7fd76c0c9670 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0b1870 tx=0x7fd75c02b880 comp rx=0 tx=0).stop 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 -- 172.21.15.91:0/2422753779 shutdown_connections 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 --2- 172.21.15.91:0/2422753779 >> v2:172.21.15.113:3300/0 conn(0x7fd76c0ce130 0x7fd76c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 --2- 172.21.15.91:0/2422753779 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c15a010 0x7fd76c0c9670 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 -- 172.21.15.91:0/2422753779 >> 172.21.15.91:0/2422753779 conn(0x7fd76c0cec80 msgr2=0x7fd76c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 -- 172.21.15.91:0/2422753779 shutdown_connections 2024-02-02T13:36:51.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.424+0000 7fd771a00640 1 -- 172.21.15.91:0/2422753779 wait complete. 2024-02-02T13:36:51.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.426+0000 7fd771a00640 1 Processor -- start 2024-02-02T13:36:51.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.426+0000 7fd771a00640 1 -- start start 2024-02-02T13:36:51.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 0x7fd76c14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd76c15a010 0x7fd76c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0ad680 con 0x7fd76c0aec20 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c15c840 con 0x7fd76c15a010 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd771a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0aca00 con 0x7fd76c0ce130 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd76b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd76b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60604/0 (socket says 172.21.15.91:60604) 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.427+0000 7fd76b400640 1 -- 172.21.15.91:0/2572170358 learned_addr learned my addr 172.21.15.91:0/2572170358 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 0x7fd76c14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd761e00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd76c15a010 0x7fd76c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 -- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 msgr2=0x7fd76c14f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 0x7fd76c14f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 -- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd76c15a010 msgr2=0x7fd76c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd76c15a010 0x7fd76c0ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0094d0 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 0x7fd76c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd76b400640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c780 tx=0x7fd76400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764003b70 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764003cd0 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0acce0 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640052b0 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.428+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c0ad2a0 con 0x7fd76c0aec20 2024-02-02T13:36:51.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.429+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c0ad7c0 con 0x7fd76c0aec20 2024-02-02T13:36:51.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.430+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd764005410 con 0x7fd76c0aec20 2024-02-02T13:36:51.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.431+0000 7fd763400640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 0x7fd74406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.431+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd764087d80 con 0x7fd76c0aec20 2024-02-02T13:36:51.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.431+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fd744071fc0 con 0x7fd76c0aec20 2024-02-02T13:36:51.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.431+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 0x7fd74406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.432+0000 7fd76aa00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 0x7fd74406c8f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd75c036000 tx=0x7fd75c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.436+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd76c0ad7c0 con 0x7fd76c0aec20 2024-02-02T13:36:51.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.628+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd76c0ad7c0 con 0x7fd76c0aec20 2024-02-02T13:36:51.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.630+0000 7fd763400640 1 -- 172.21.15.91:0/2572170358 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16188 (secure 0 0 0) 0x7fd76c0ad7c0 con 0x7fd76c0aec20 2024-02-02T13:36:51.632 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:51.632 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":182,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:36:28.579012+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.28","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:36:51.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.632+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 msgr2=0x7fd74406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.632+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 0x7fd74406c8f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd75c036000 tx=0x7fd75c03d040 comp rx=0 tx=0).stop 2024-02-02T13:36:51.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.632+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 msgr2=0x7fd76c0c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.632+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c780 tx=0x7fd76400cc40 comp rx=0 tx=0).stop 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 shutdown_connections 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd74406a430 0x7fd74406c8f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd76c0aec20 0x7fd76c0c3fa0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd76c15a010 0x7fd76c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 --2- 172.21.15.91:0/2572170358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd76c0ce130 0x7fd76c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 >> 172.21.15.91:0/2572170358 conn(0x7fd76c0cec80 msgr2=0x7fd76c15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 shutdown_connections 2024-02-02T13:36:51.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.633+0000 7fd771a00640 1 -- 172.21.15.91:0/2572170358 wait complete. 2024-02-02T13:36:51.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:35:31.112699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:35:31.642725+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:36:51.653 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:36:51.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.868+0000 7fcfc0a00640 1 Processor -- start 2024-02-02T13:36:51.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.868+0000 7fcfc0a00640 1 -- start start 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.869+0000 7fcfc0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 0x7fcfb80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.869+0000 7fcfc0a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcfb80b0c40 0x7fcfb80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.869+0000 7fcfc0a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80b1070 con 0x7fcfb8151f90 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.869+0000 7fcfc0a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb815fa30 con 0x7fcfb80b0c40 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.869+0000 7fcfc0a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb8070f80 con 0x7fcfb815f5a0 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbe000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcfb8151f90 0x7fcfb806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49284/0 (socket says 172.21.15.91:49284) 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbe000640 1 -- 172.21.15.91:0/4115129848 learned_addr learned my addr 172.21.15.91:0/4115129848 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:51.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbea00640 1 --2- 172.21.15.91:0/4115129848 >> v2:172.21.15.91:3300/0 conn(0x7fcfb80b0c40 0x7fcfb80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbcc00640 1 -- 172.21.15.91:0/4115129848 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013612758 0 0) 0x7fcfb80b1070 con 0x7fcfb8151f90 2024-02-02T13:36:51.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbcc00640 1 -- 172.21.15.91:0/4115129848 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf98003520 con 0x7fcfb8151f90 2024-02-02T13:36:51.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbd600640 1 --2- 172.21.15.91:0/4115129848 >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 0x7fcfb80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbea00640 1 -- 172.21.15.91:0/4115129848 >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 msgr2=0x7fcfb80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:51.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbea00640 1 --2- 172.21.15.91:0/4115129848 >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 0x7fcfb80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbea00640 1 -- 172.21.15.91:0/4115129848 >> v1:172.21.15.91:6789/0 conn(0x7fcfb8151f90 legacy=0x7fcfb806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbea00640 1 -- 172.21.15.91:0/4115129848 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8152370 con 0x7fcfb80b0c40 2024-02-02T13:36:51.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.870+0000 7fcfbd600640 1 --2- 172.21.15.91:0/4115129848 >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 0x7fcfb80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfbea00640 1 --2- 172.21.15.91:0/4115129848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb80b3af0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8071a00 tx=0x7fcfb002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd51802c0e38da1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfbcc00640 1 -- 172.21.15.91:0/4115129848 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb002cc80 con 0x7fcfb80b0c40 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfbcc00640 1 -- 172.21.15.91:0/4115129848 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb002cde0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfbcc00640 1 -- 172.21.15.91:0/4115129848 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0034660 con 0x7fcfb80b0c40 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfc0a00640 1 -- 172.21.15.91:0/4115129848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 msgr2=0x7fcfb80b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.872+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/4115129848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb80b3af0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8071a00 tx=0x7fcfb002bed0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 -- 172.21.15.91:0/4115129848 shutdown_connections 2024-02-02T13:36:51.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/4115129848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb80b3af0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/4115129848 >> v2:172.21.15.113:3300/0 conn(0x7fcfb815f5a0 0x7fcfb80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 -- 172.21.15.91:0/4115129848 >> 172.21.15.91:0/4115129848 conn(0x7fcfb80cec80 msgr2=0x7fcfb80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:51.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 -- 172.21.15.91:0/4115129848 shutdown_connections 2024-02-02T13:36:51.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.873+0000 7fcfc0a00640 1 -- 172.21.15.91:0/4115129848 wait complete. 2024-02-02T13:36:51.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.874+0000 7fcfc0a00640 1 Processor -- start 2024-02-02T13:36:51.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.875+0000 7fcfc0a00640 1 -- start start 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.875+0000 7fcfc0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.875+0000 7fcfc0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfb8151f90 0x7fcfb8155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfc0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb815f5a0 0x7fcfb80c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfc0a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb8157260 con 0x7fcfb80b0c40 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfc0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81565e0 con 0x7fcfb815f5a0 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfc0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81568c0 con 0x7fcfb8151f90 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60632/0 (socket says 172.21.15.91:60632) 2024-02-02T13:36:51.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 -- 172.21.15.91:0/346233357 learned_addr learned my addr 172.21.15.91:0/346233357 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbea00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb815f5a0 0x7fcfb80c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 -- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfb8151f90 msgr2=0x7fcfb8155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfb8151f90 0x7fcfb8155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 -- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb815f5a0 msgr2=0x7fcfb80c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb815f5a0 0x7fcfb80c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009a70 con 0x7fcfb80b0c40 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.876+0000 7fcfbe000640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c410 tx=0x7fcfa800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.877+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8005ce0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.877+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8005e40 con 0x7fcfb80b0c40 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.877+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80072f0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.877+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb8156ba0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.877+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb80c5ef0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.878+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb80ae550 con 0x7fcfb80b0c40 2024-02-02T13:36:51.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.879+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcfa8007510 con 0x7fcfb80b0c40 2024-02-02T13:36:51.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.880+0000 7fcfb6000640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 0x7fcf9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:51.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.880+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcfa8087fb0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.880+0000 7fcfbd600640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 0x7fcf9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:51.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.881+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fcf90071fc0 con 0x7fcfb80b0c40 2024-02-02T13:36:51.883 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.881+0000 7fcfbd600640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 0x7fcf9006c8f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006fd0 tx=0x7fcfa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:51.886 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:51.885+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcfa805be60 con 0x7fcfb80b0c40 2024-02-02T13:36:52.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.075+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfb8066cf0 con 0x7fcfb80b0c40 2024-02-02T13:36:52.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.076+0000 7fcfb6000640 1 -- 172.21.15.91:0/346233357 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfa8017030 con 0x7fcfb80b0c40 2024-02-02T13:36:52.077 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:36:52.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 msgr2=0x7fcf9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 0x7fcf9006c8f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006fd0 tx=0x7fcfa0008040 comp rx=0 tx=0).stop 2024-02-02T13:36:52.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 msgr2=0x7fcfb8144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c410 tx=0x7fcfa800c8d0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 shutdown_connections 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcf9006a430 0x7fcf9006c8f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfb8151f90 0x7fcfb8155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb815f5a0 0x7fcfb80c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 --2- 172.21.15.91:0/346233357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfb80b0c40 0x7fcfb8144f50 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.078+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 >> 172.21.15.91:0/346233357 conn(0x7fcfb80cec80 msgr2=0x7fcfb80ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.079+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 shutdown_connections 2024-02-02T13:36:52.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.079+0000 7fcfc0a00640 1 -- 172.21.15.91:0/346233357 wait complete. 2024-02-02T13:36:52.099 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:36:52.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.313+0000 7fbd92e00640 1 Processor -- start 2024-02-02T13:36:52.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.313+0000 7fbd92e00640 1 -- start start 2024-02-02T13:36:52.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd92e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 0x7fbd8c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd92e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbd8c0b1d70 0x7fbd8c0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd92e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0ce210 con 0x7fbd8c160830 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd92e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8c0ce4f0 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd92e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0ce7d0 con 0x7fbd8c070980 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd90e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbd8c0b1d70 0x7fbd8c0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd8be00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbd8c160830 0x7fbd8c070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49300/0 (socket says 172.21.15.91:49300) 2024-02-02T13:36:52.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.315+0000 7fbd8be00640 1 -- 172.21.15.91:0/3116001504 learned_addr learned my addr 172.21.15.91:0/3116001504 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd8b400640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 0x7fbd8c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd8aa00640 1 -- 172.21.15.91:0/3116001504 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213887105 0 0) 0x7fbd8c0ce210 con 0x7fbd8c160830 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd8aa00640 1 -- 172.21.15.91:0/3116001504 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c003520 con 0x7fbd8c160830 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd90e00640 1 -- 172.21.15.91:0/3116001504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 msgr2=0x7fbd8c0b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd90e00640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 0x7fbd8c0b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd90e00640 1 -- 172.21.15.91:0/3116001504 >> v1:172.21.15.91:6789/0 conn(0x7fbd8c160830 legacy=0x7fbd8c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd90e00640 1 -- 172.21.15.91:0/3116001504 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0d17b0 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.316+0000 7fbd8b400640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 0x7fbd8c0b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:52.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.317+0000 7fbd90e00640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c0b1d70 0x7fbd8c0711f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbd840097a0 tx=0x7fbd8402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50abb43daedeeb9f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.317+0000 7fbd8aa00640 1 -- 172.21.15.91:0/3116001504 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8402e070 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.319 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.317+0000 7fbd8aa00640 1 -- 172.21.15.91:0/3116001504 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd840315a0 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.317+0000 7fbd8aa00640 1 -- 172.21.15.91:0/3116001504 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84031930 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 -- 172.21.15.91:0/3116001504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c0b1d70 msgr2=0x7fbd8c0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c0b1d70 0x7fbd8c0711f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbd840097a0 tx=0x7fbd8402b840 comp rx=0 tx=0).stop 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 -- 172.21.15.91:0/3116001504 shutdown_connections 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c0b1d70 0x7fbd8c0711f0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3116001504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c070980 0x7fbd8c0b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 -- 172.21.15.91:0/3116001504 >> 172.21.15.91:0/3116001504 conn(0x7fbd8c0d3030 msgr2=0x7fbd8c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 -- 172.21.15.91:0/3116001504 shutdown_connections 2024-02-02T13:36:52.320 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.318+0000 7fbd92e00640 1 -- 172.21.15.91:0/3116001504 wait complete. 2024-02-02T13:36:52.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.320+0000 7fbd92e00640 1 Processor -- start 2024-02-02T13:36:52.321 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.320+0000 7fbd92e00640 1 -- start start 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c0b1d70 0x7fbd8c0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c160830 0x7fbd8c0b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1f2450 con 0x7fbd8c070980 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1f3210 con 0x7fbd8c160830 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd92e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1f3620 con 0x7fbd8c0b1d70 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60646/0 (socket says 172.21.15.91:60646) 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 -- 172.21.15.91:0/3890496502 learned_addr learned my addr 172.21.15.91:0/3890496502 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 -- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c0b1d70 msgr2=0x7fbd8c0b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c0b1d70 0x7fbd8c0b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.323 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 -- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c160830 msgr2=0x7fbd8c0b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c160830 0x7fbd8c0b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.321+0000 7fbd8be00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84009400 con 0x7fbd8c070980 2024-02-02T13:36:52.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd8be00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00cbf0 tx=0x7fbd7c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.324 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00ad40 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c012400 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00aea0 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c1f2640 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.322+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c1f2bc0 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.323+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c066cf0 con 0x7fbd8c070980 2024-02-02T13:36:52.325 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.324+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbd7c007070 con 0x7fbd8c070980 2024-02-02T13:36:52.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.325+0000 7fbd83e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 0x7fbd6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.325+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbd7c088a10 con 0x7fbd8c070980 2024-02-02T13:36:52.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.325+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fbd64071fc0 con 0x7fbd8c070980 2024-02-02T13:36:52.326 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.325+0000 7fbd8b400640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 0x7fbd6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.326+0000 7fbd8b400640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 0x7fbd6406c8f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006fd0 tx=0x7fbd74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.330+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbd7c017020 con 0x7fbd8c070980 2024-02-02T13:36:52.519 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.518+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd8c0d3f40 con 0x7fbd8c070980 2024-02-02T13:36:52.520 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.519+0000 7fbd83e00640 1 -- 172.21.15.91:0/3890496502 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd7c053f40 con 0x7fbd8c070980 2024-02-02T13:36:52.520 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 msgr2=0x7fbd6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 0x7fbd6406c8f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006fd0 tx=0x7fbd74008040 comp rx=0 tx=0).stop 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 msgr2=0x7fbd8c0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00cbf0 tx=0x7fbd7c00a430 comp rx=0 tx=0).stop 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 shutdown_connections 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbd6406a430 0x7fbd6406c8f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c0b1d70 0x7fbd8c0b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.523 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c160830 0x7fbd8c0b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 --2- 172.21.15.91:0/3890496502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd8c070980 0x7fbd8c0c8eb0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.521+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 >> 172.21.15.91:0/3890496502 conn(0x7fbd8c0d3030 msgr2=0x7fbd8c15b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:52.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.522+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 shutdown_connections 2024-02-02T13:36:52.524 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.522+0000 7fbd92e00640 1 -- 172.21.15.91:0/3890496502 wait complete. 2024-02-02T13:36:52.541 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:36:52.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.745+0000 7f2f65600640 1 Processor -- start 2024-02-02T13:36:52.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.746+0000 7f2f65600640 1 -- start start 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f65600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f600addd0 0x7f2f600cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f65600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f600ae1b0 con 0x7f2f6006b8a0 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f65600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f600c9430 con 0x7f2f600c1c70 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f65600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6006b450 con 0x7f2f600addd0 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f5fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f600addd0 0x7f2f600cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f5fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2f600addd0 0x7f2f600cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60654/0 (socket says 172.21.15.91:60654) 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f64c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2f6006b8a0 0x7f2f6006bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49304/0 (socket says 172.21.15.91:49304) 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.747+0000 7f2f5fe00640 1 -- 172.21.15.91:0/1600708859 learned_addr learned my addr 172.21.15.91:0/1600708859 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.748+0000 7f2f5fe00640 1 -- 172.21.15.91:0/1600708859 >> v1:172.21.15.113:6789/0 conn(0x7f2f600c1c70 legacy=0x7f2f600ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.748+0000 7f2f5fe00640 1 -- 172.21.15.91:0/1600708859 >> v1:172.21.15.91:6789/0 conn(0x7f2f6006b8a0 legacy=0x7f2f6006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.748+0000 7f2f5fe00640 1 -- 172.21.15.91:0/1600708859 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600cade0 con 0x7f2f600addd0 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.749+0000 7f2f5fe00640 1 --2- 172.21.15.91:0/1600708859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f600cc660 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009870 tx=0x7f2f5002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa48bf419326a8a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.749+0000 7f2f5f400640 1 -- 172.21.15.91:0/1600708859 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50030850 con 0x7f2f600addd0 2024-02-02T13:36:52.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.749+0000 7f2f5f400640 1 -- 172.21.15.91:0/1600708859 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f500309b0 con 0x7f2f600addd0 2024-02-02T13:36:52.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.749+0000 7f2f5f400640 1 -- 172.21.15.91:0/1600708859 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50030d40 con 0x7f2f600addd0 2024-02-02T13:36:52.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 -- 172.21.15.91:0/1600708859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 msgr2=0x7f2f600cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 --2- 172.21.15.91:0/1600708859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f600cc660 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009870 tx=0x7f2f5002b990 comp rx=0 tx=0).stop 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 -- 172.21.15.91:0/1600708859 shutdown_connections 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 --2- 172.21.15.91:0/1600708859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f600cc660 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 -- 172.21.15.91:0/1600708859 >> 172.21.15.91:0/1600708859 conn(0x7f2f600cea90 msgr2=0x7f2f600ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 -- 172.21.15.91:0/1600708859 shutdown_connections 2024-02-02T13:36:52.752 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.750+0000 7f2f65600640 1 -- 172.21.15.91:0/1600708859 wait complete. 2024-02-02T13:36:52.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.752+0000 7f2f65600640 1 Processor -- start 2024-02-02T13:36:52.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.752+0000 7f2f65600640 1 -- start start 2024-02-02T13:36:52.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 0x7f2f600c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 0x7f2f601fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c2c40 con 0x7f2f600addd0 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c2660 con 0x7f2f6006b8a0 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f65600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c5030 con 0x7f2f600c1c70 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60658/0 (socket says 172.21.15.91:60658) 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 -- 172.21.15.91:0/3364386413 learned_addr learned my addr 172.21.15.91:0/3364386413 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f5fe00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 0x7f2f600c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f64c00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 0x7f2f601fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 -- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 msgr2=0x7f2f601fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 0x7f2f601fa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.756 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 -- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 msgr2=0x7f2f600c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 0x7f2f600c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.753+0000 7f2f57200640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500094d0 con 0x7f2f600addd0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f5fe00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 0x7f2f600c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f64c00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 0x7f2f601fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f57200640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800b3f0 tx=0x7f2f4800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48011830 con 0x7f2f600addd0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48011990 con 0x7f2f600addd0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48011d40 con 0x7f2f600addd0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f601fad10 con 0x7f2f600addd0 2024-02-02T13:36:52.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.754+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f601564e0 con 0x7f2f600addd0 2024-02-02T13:36:52.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.755+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6004dff0 con 0x7f2f600addd0 2024-02-02T13:36:52.758 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.756+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f4801a400 con 0x7f2f600addd0 2024-02-02T13:36:52.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.757+0000 7f2f5cc00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 0x7f2f3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:52.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.757+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2f48088810 con 0x7f2f600addd0 2024-02-02T13:36:52.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.757+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f2f38071fc0 con 0x7f2f600addd0 2024-02-02T13:36:52.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.757+0000 7f2f5fe00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 0x7f2f3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:52.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.758+0000 7f2f5fe00640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 0x7f2f3806c8f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f50036000 tx=0x7f2f5003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:52.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.762+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f48015060 con 0x7f2f600addd0 2024-02-02T13:36:52.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.952+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f600b4a10 con 0x7f2f600addd0 2024-02-02T13:36:52.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.952+0000 7f2f5cc00640 1 -- 172.21.15.91:0/3364386413 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f48053dd0 con 0x7f2f600addd0 2024-02-02T13:36:52.954 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:36:52.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 msgr2=0x7f2f3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 0x7f2f3806c8f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f50036000 tx=0x7f2f5003b040 comp rx=0 tx=0).stop 2024-02-02T13:36:52.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 msgr2=0x7f2f60155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:52.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800b3f0 tx=0x7f2f4800b8b0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 shutdown_connections 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f3806a430 0x7f2f3806c8f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f600c1c70 0x7f2f601fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f600addd0 0x7f2f60155fa0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 --2- 172.21.15.91:0/3364386413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6006b8a0 0x7f2f600c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 >> 172.21.15.91:0/3364386413 conn(0x7f2f600cea90 msgr2=0x7f2f600ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 shutdown_connections 2024-02-02T13:36:52.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:52.955+0000 7f2f65600640 1 -- 172.21.15.91:0/3364386413 wait complete. 2024-02-02T13:36:52.974 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:36:53.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.176+0000 7fc72f200640 1 Processor -- start 2024-02-02T13:36:53.177 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.176+0000 7fc72f200640 1 -- start start 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7300c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73006faa0 con 0x7fc7300ce4d0 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7300ca420 con 0x7fc7300c9a80 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015e800 con 0x7fc73015dfb0 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc7300ce4d0 0x7fc7300707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49312/0 (socket says 172.21.15.91:49312) 2024-02-02T13:36:53.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.177+0000 7fc72de00640 1 -- 172.21.15.91:0/4128969532 learned_addr learned my addr 172.21.15.91:0/4128969532 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378941696 0 0) 0x7fc73006faa0 con 0x7fc7300ce4d0 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72e800640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7300c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc710003520 con 0x7fc7300ce4d0 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 726088947 0 0) 0x7fc710003520 con 0x7fc7300ce4d0 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 msgr2=0x7fc7300c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7300c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 >> v1:172.21.15.113:6789/0 conn(0x7fc7300c9a80 legacy=0x7fc73015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730070320 con 0x7fc7300ce4d0 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc720001670 con 0x7fc7300ce4d0 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 >> v1:172.21.15.91:6789/0 conn(0x7fc7300ce4d0 legacy=0x7fc7300707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.178+0000 7fc72d400640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710003f20 0x7fc7100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72d400640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc710004d00 0x7fc710005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015e800 con 0x7fc710004d00 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000ba60 con 0x7fc710003f20 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000bc20 con 0x7fc73015dfb0 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc710004d00 0x7fc710005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc725200640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710003f20 0x7fc7100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 -- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 msgr2=0x7fc7100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72de00640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc73015dfb0 0x7fc7100039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 -- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710003f20 msgr2=0x7fc7100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710003f20 0x7fc7100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc72e800640 1 -- 172.21.15.91:0/4128969532 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710003520 con 0x7fc710004d00 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.179+0000 7fc725200640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710003f20 0x7fc7100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.180+0000 7fc72e800640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc710004d00 0x7fc710005180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc728009fc0 tx=0x7fc7100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72ed9c24cfcf759 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72802cc20 con 0x7fc710004d00 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800f040 con 0x7fc710004d00 2024-02-02T13:36:53.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72d400640 1 -- 172.21.15.91:0/4128969532 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728034440 con 0x7fc710004d00 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72f200640 1 -- 172.21.15.91:0/4128969532 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc710004d00 msgr2=0x7fc710005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72f200640 1 --2- 172.21.15.91:0/4128969532 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc710004d00 0x7fc710005180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc728009fc0 tx=0x7fc7100057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72e800640 1 -- 172.21.15.91:0/4128969532 reap_dead start 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72f200640 1 -- 172.21.15.91:0/4128969532 shutdown_connections 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72f200640 1 -- 172.21.15.91:0/4128969532 >> 172.21.15.91:0/4128969532 conn(0x7fc7300cb4a0 msgr2=0x7fc7300cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.181+0000 7fc72f200640 1 -- 172.21.15.91:0/4128969532 shutdown_connections 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.182+0000 7fc72f200640 1 -- 172.21.15.91:0/4128969532 wait complete. 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.184+0000 7fc72f200640 1 Processor -- start 2024-02-02T13:36:53.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.184+0000 7fc72f200640 1 -- start start 2024-02-02T13:36:53.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710004d00 0x7fc7300b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 0x7fc7300c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730070ec0 con 0x7fc7300b1fc0 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300ad710 con 0x7fc710004d00 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300b2980 con 0x7fc7301f3160 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60676/0 (socket says 172.21.15.91:60676) 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.185+0000 7fc72e800640 1 -- 172.21.15.91:0/2156679906 learned_addr learned my addr 172.21.15.91:0/2156679906 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc725200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 0x7fc7300c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 -- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 msgr2=0x7fc7300c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 0x7fc7300c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 -- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710004d00 msgr2=0x7fc7300b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72de00640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710004d00 0x7fc7300b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710004d00 0x7fc7300b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300c4520 con 0x7fc7300b1fc0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc725200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 0x7fc7300c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc72e800640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fc728002230 tx=0x7fc728009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728043070 con 0x7fc7300b1fc0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.186+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728034ad0 con 0x7fc7300b1fc0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.187+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728034e80 con 0x7fc7300b1fc0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.187+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300aca90 con 0x7fc7300b1fc0 2024-02-02T13:36:53.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.187+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300ad050 con 0x7fc7300b1fc0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.188+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7300ad850 con 0x7fc7300b1fc0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.189+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc72803e440 con 0x7fc7300b1fc0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.189+0000 7fc726a00640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 0x7fc70806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.190+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc7280aa540 con 0x7fc7300b1fc0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.190+0000 7fc72de00640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 0x7fc70806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.190+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fc708071ff0 con 0x7fc7300b1fc0 2024-02-02T13:36:53.192 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.190+0000 7fc72de00640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 0x7fc70806c920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc720000f60 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.194+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc7300ad850 con 0x7fc7300b1fc0 2024-02-02T13:36:53.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.394+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7300ad850 con 0x7fc7300b1fc0 2024-02-02T13:36:53.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.395+0000 7fc726a00640 1 -- 172.21.15.91:0/2156679906 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v182) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc7300ad850 con 0x7fc7300b1fc0 2024-02-02T13:36:53.397 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:36:53.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.397+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 msgr2=0x7fc70806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 0x7fc70806c920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc720000f60 tx=0x7fc720009040 comp rx=0 tx=0).stop 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 msgr2=0x7fc7300b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fc728002230 tx=0x7fc728009750 comp rx=0 tx=0).stop 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 shutdown_connections 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc70806a460 0x7fc70806c920 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7301f3160 0x7fc7300c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc7300b1fc0 0x7fc7300b2440 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 --2- 172.21.15.91:0/2156679906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc710004d00 0x7fc7300b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 >> 172.21.15.91:0/2156679906 conn(0x7fc7300cb4a0 msgr2=0x7fc73006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:53.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 shutdown_connections 2024-02-02T13:36:53.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.398+0000 7fc72f200640 1 -- 172.21.15.91:0/2156679906 wait complete. 2024-02-02T13:36:53.417 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:36:53.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:36:53.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:36:53.530 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:36:53.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:36:53.531 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:36:53.582 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:36:53.582 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:53.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:36:53.637 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:36:53.638 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:36:53.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:36:53.694 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:36:53.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.912+0000 7f65de800640 1 Processor -- start 2024-02-02T13:36:53.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.912+0000 7f65de800640 1 -- start start 2024-02-02T13:36:53.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d815d650 0x7f65d80cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f65d80ce070 0x7f65d80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 0x7f65d8161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d80cea30 con 0x7f65d80ce070 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d80cded0 con 0x7f65d81610b0 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65de800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80d12c0 con 0x7f65d815d650 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65ce800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f65d80ce070 0x7f65d80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65ce800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f65d80ce070 0x7f65d80ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60692/0 (socket says 172.21.15.91:60692) 2024-02-02T13:36:53.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.914+0000 7f65ce800640 1 -- 172.21.15.91:0/3139964267 learned_addr learned my addr 172.21.15.91:0/3139964267 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65d7400640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d815d650 0x7f65d80cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65ce800640 1 -- 172.21.15.91:0/3139964267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d815d650 msgr2=0x7f65d80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65d7e00640 1 --2- 172.21.15.91:0/3139964267 >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 0x7f65d8161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65ce800640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d815d650 0x7f65d80cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65ce800640 1 -- 172.21.15.91:0/3139964267 >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 msgr2=0x7f65d8161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65ce800640 1 --2- 172.21.15.91:0/3139964267 >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 0x7f65d8161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65ce800640 1 -- 172.21.15.91:0/3139964267 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8071410 con 0x7f65d80ce070 2024-02-02T13:36:53.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.915+0000 7f65d7e00640 1 --2- 172.21.15.91:0/3139964267 >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 0x7f65d8161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.916+0000 7f65ce800640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d80ce070 0x7f65d80ce4f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b8d40 tx=0x7f65c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98d91fe74c8d5f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.916+0000 7f65d6a00640 1 -- 172.21.15.91:0/3139964267 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031870 con 0x7f65d80ce070 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.916+0000 7f65d6a00640 1 -- 172.21.15.91:0/3139964267 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00319d0 con 0x7f65d80ce070 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.916+0000 7f65d6a00640 1 -- 172.21.15.91:0/3139964267 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031d60 con 0x7f65d80ce070 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 -- 172.21.15.91:0/3139964267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d80ce070 msgr2=0x7f65d80ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d80ce070 0x7f65d80ce4f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b8d40 tx=0x7f65c002b910 comp rx=0 tx=0).stop 2024-02-02T13:36:53.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 -- 172.21.15.91:0/3139964267 shutdown_connections 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 --2- 172.21.15.91:0/3139964267 >> v2:172.21.15.113:3300/0 conn(0x7f65d81610b0 0x7f65d8161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d80ce070 0x7f65d80ce4f0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 --2- 172.21.15.91:0/3139964267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d815d650 0x7f65d80cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 -- 172.21.15.91:0/3139964267 >> 172.21.15.91:0/3139964267 conn(0x7f65d80d30a0 msgr2=0x7f65d80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 -- 172.21.15.91:0/3139964267 shutdown_connections 2024-02-02T13:36:53.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.917+0000 7f65de800640 1 -- 172.21.15.91:0/3139964267 wait complete. 2024-02-02T13:36:53.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.919+0000 7f65de800640 1 Processor -- start 2024-02-02T13:36:53.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.919+0000 7f65de800640 1 -- start start 2024-02-02T13:36:53.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.919+0000 7f65de800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d80ce070 0x7f65d8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65de800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65de800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65d81610b0 0x7f65d81f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65de800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d815f9e0 con 0x7f65d815d650 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65de800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8158b40 con 0x7f65d81610b0 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65de800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80ca2f0 con 0x7f65d80ce070 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60700/0 (socket says 172.21.15.91:60700) 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 -- 172.21.15.91:0/2916688848 learned_addr learned my addr 172.21.15.91:0/2916688848 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65d7400640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d80ce070 0x7f65d8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65d7e00640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65d81610b0 0x7f65d81f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 -- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d80ce070 msgr2=0x7f65d8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d80ce070 0x7f65d8158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 -- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65d81610b0 msgr2=0x7f65d81f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65d81610b0 0x7f65d81f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:53.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.920+0000 7f65ce800640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c00094d0 con 0x7f65d815d650 2024-02-02T13:36:53.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65ce800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f65c002b950 tx=0x7f65c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031860 con 0x7f65d815d650 2024-02-02T13:36:53.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00443f0 con 0x7f65d815d650 2024-02-02T13:36:53.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d815f700 con 0x7f65d815d650 2024-02-02T13:36:53.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00447a0 con 0x7f65d815d650 2024-02-02T13:36:53.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.921+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d80ca890 con 0x7f65d815d650 2024-02-02T13:36:53.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.922+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8071a30 con 0x7f65d815d650 2024-02-02T13:36:53.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.923+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f65c004d450 con 0x7f65d815d650 2024-02-02T13:36:53.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.924+0000 7f65cfe00640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 0x7f65b006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:53.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.924+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f65c002d040 con 0x7f65d815d650 2024-02-02T13:36:53.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.924+0000 7f65d7400640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 0x7f65b006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:53.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.924+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f65b0071ff0 con 0x7f65d815d650 2024-02-02T13:36:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.925+0000 7f65d7400640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 0x7f65b006c920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:53.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:53.928+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f65c0035020 con 0x7f65d815d650 2024-02-02T13:36:54.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.157+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65d80ca150 con 0x7f65d815d650 2024-02-02T13:36:54.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.158+0000 7f65cfe00640 1 -- 172.21.15.91:0/2916688848 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7f65d80ca150 con 0x7f65d815d650 2024-02-02T13:36:54.160 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:54.160 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:36:54.160 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 msgr2=0x7f65b006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 0x7f65b006c920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8008040 comp rx=0 tx=0).stop 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 msgr2=0x7f65d81f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f65c002b950 tx=0x7f65c00046c0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 shutdown_connections 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f65b006a460 0x7f65b006c920 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65d81610b0 0x7f65d81f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65d815d650 0x7f65d81f64d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 --2- 172.21.15.91:0/2916688848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65d80ce070 0x7f65d8158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.161+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 >> 172.21.15.91:0/2916688848 conn(0x7f65d80d30a0 msgr2=0x7f65d815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:54.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.162+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 shutdown_connections 2024-02-02T13:36:54.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.162+0000 7f65de800640 1 -- 172.21.15.91:0/2916688848 wait complete. 2024-02-02T13:36:54.182 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:36:54.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.390+0000 7fc976200640 1 Processor -- start 2024-02-02T13:36:54.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.391+0000 7fc976200640 1 -- start start 2024-02-02T13:36:54.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc976200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9701598a0 0x7fc9700ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc976200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97015dec0 con 0x7fc9700cd1f0 2024-02-02T13:36:54.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc976200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9700c9d30 con 0x7fc9700b16d0 2024-02-02T13:36:54.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc976200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97015e1a0 con 0x7fc9701598a0 2024-02-02T13:36:54.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc966800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc9700cd1f0 0x7fc9700b0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49318/0 (socket says 172.21.15.91:49318) 2024-02-02T13:36:54.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.392+0000 7fc966800640 1 -- 172.21.15.91:0/1247529099 learned_addr learned my addr 172.21.15.91:0/1247529099 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:54.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96f400640 1 --2- 172.21.15.91:0/1247529099 >> v2:172.21.15.113:3300/0 conn(0x7fc9701598a0 0x7fc9700ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836989140 0 0) 0x7fc97015dec0 con 0x7fc9700cd1f0 2024-02-02T13:36:54.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc950003520 con 0x7fc9700cd1f0 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285596053 0 0) 0x7fc950003520 con 0x7fc9700cd1f0 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 >> v2:172.21.15.113:3300/0 conn(0x7fc9701598a0 msgr2=0x7fc9700ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 --2- 172.21.15.91:0/1247529099 >> v2:172.21.15.113:3300/0 conn(0x7fc9701598a0 0x7fc9700ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 >> v1:172.21.15.113:6789/0 conn(0x7fc9700b16d0 legacy=0x7fc9700cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700ca010 con 0x7fc9700cd1f0 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc958001690 con 0x7fc9700cd1f0 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.393+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 >> v1:172.21.15.91:6789/0 conn(0x7fc9700cd1f0 legacy=0x7fc9700b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950003970 0x7fc950003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 0x7fc950004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc950005180 0x7fc95000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97015e1a0 con 0x7fc9500047c0 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000e330 con 0x7fc950005180 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000e4f0 con 0x7fc950003970 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 0x7fc950004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96f400640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950003970 0x7fc950003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96fe00640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc950005180 0x7fc95000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 -- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950003970 msgr2=0x7fc950003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950003970 0x7fc950003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 -- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc950005180 msgr2=0x7fc95000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc950005180 0x7fc95000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc966800640 1 -- 172.21.15.91:0/1247529099 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950003520 con 0x7fc9500047c0 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96fe00640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc950005180 0x7fc95000db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.394+0000 7fc96fe00640 1 -- 172.21.15.91:0/1247529099 reap_dead start 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.395+0000 7fc966800640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 0x7fc950004c40 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc9700c65c0 tx=0x7fc9500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf2556de9f0da3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958005ad0 con 0x7fc9500047c0 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958005c30 con 0x7fc9500047c0 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc96ea00640 1 -- 172.21.15.91:0/1247529099 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95802cab0 con 0x7fc9500047c0 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 -- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 msgr2=0x7fc950004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 0x7fc950004c40 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc9700c65c0 tx=0x7fc9500057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 -- 172.21.15.91:0/1247529099 shutdown_connections 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 --2- 172.21.15.91:0/1247529099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9500047c0 0x7fc950004c40 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 -- 172.21.15.91:0/1247529099 >> 172.21.15.91:0/1247529099 conn(0x7fc9700cbfc0 msgr2=0x7fc9700cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:54.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.396+0000 7fc976200640 1 -- 172.21.15.91:0/1247529099 shutdown_connections 2024-02-02T13:36:54.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.397+0000 7fc976200640 1 -- 172.21.15.91:0/1247529099 wait complete. 2024-02-02T13:36:54.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.398+0000 7fc976200640 1 Processor -- start 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.398+0000 7fc976200640 1 -- start start 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9500047c0 0x7fc970157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97015cf50 0x7fc97015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97015d920 con 0x7fc9700c80b0 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9701578f0 con 0x7fc9500047c0 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.399+0000 7fc976200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc970157bd0 con 0x7fc97015cf50 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60714/0 (socket says 172.21.15.91:60714) 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 -- 172.21.15.91:0/518663010 learned_addr learned my addr 172.21.15.91:0/518663010 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc96f400640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9500047c0 0x7fc970157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 -- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97015cf50 msgr2=0x7fc97015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97015cf50 0x7fc97015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 -- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9500047c0 msgr2=0x7fc970157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9500047c0 0x7fc970157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970157eb0 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.400+0000 7fc966800640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc95802c430 tx=0x7fc95802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.401+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95803c040 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.401+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95802cca0 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.401+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc970158190 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.401+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9580072b0 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.401+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9700c8510 con 0x7fc9700c80b0 2024-02-02T13:36:54.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.402+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9700c86a0 con 0x7fc9700c80b0 2024-02-02T13:36:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.403+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc958032070 con 0x7fc9700c80b0 2024-02-02T13:36:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.404+0000 7fc967e00640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 0x7fc94806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.404+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc9580a9ec0 con 0x7fc9700c80b0 2024-02-02T13:36:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.404+0000 7fc96f400640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 0x7fc94806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.404+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fc948071fc0 con 0x7fc9700c80b0 2024-02-02T13:36:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.405+0000 7fc96f400640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 0x7fc94806c8f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc960009990 tx=0x7fc960008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.408+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc9580753f0 con 0x7fc9700c80b0 2024-02-02T13:36:54.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.599+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc970066980 con 0x7fc9700c80b0 2024-02-02T13:36:54.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.601+0000 7fc967e00640 1 -- 172.21.15.91:0/518663010 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16188 (secure 0 0 0) 0x7fc958078a10 con 0x7fc9700c80b0 2024-02-02T13:36:54.603 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:54.603 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":182,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:36:28.579012+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.28","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:36:54.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 msgr2=0x7fc94806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 0x7fc94806c8f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc960009990 tx=0x7fc960008040 comp rx=0 tx=0).stop 2024-02-02T13:36:54.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 msgr2=0x7fc9700cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc95802c430 tx=0x7fc95802e070 comp rx=0 tx=0).stop 2024-02-02T13:36:54.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 shutdown_connections 2024-02-02T13:36:54.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc94806a430 0x7fc94806c8f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc97015cf50 0x7fc97015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc9700c80b0 0x7fc9700cfdb0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 --2- 172.21.15.91:0/518663010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9500047c0 0x7fc970157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 >> 172.21.15.91:0/518663010 conn(0x7fc9700cbfc0 msgr2=0x7fc970073fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 shutdown_connections 2024-02-02T13:36:54.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.604+0000 7fc976200640 1 -- 172.21.15.91:0/518663010 wait complete. 2024-02-02T13:36:54.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:35:31.112699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:35:31.642725+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:36:54.625 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.837+0000 7ff21fe00640 1 Processor -- start 2024-02-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.837+0000 7ff21fe00640 1 -- start start 2024-02-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2200b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2200ce1d0 0x7ff2200d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2200b7d40 con 0x7ff2200b6990 2024-02-02T13:36:54.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200b81d0 con 0x7ff2200ce1d0 2024-02-02T13:36:54.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b7130 con 0x7ff22015cee0 2024-02-02T13:36:54.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21e000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff2200b6990 0x7ff2200cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49334/0 (socket says 172.21.15.91:49334) 2024-02-02T13:36:54.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.839+0000 7ff21e000640 1 -- 172.21.15.91:0/4183769637 learned_addr learned my addr 172.21.15.91:0/4183769637 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21f400640 1 --2- 172.21.15.91:0/4183769637 >> v2:172.21.15.113:3300/0 conn(0x7ff2200ce1d0 0x7ff2200d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21ea00640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2200b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228631624 0 0) 0x7ff2200b7d40 con 0x7ff2200b6990 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff200003520 con 0x7ff2200b6990 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2716521020 0 0) 0x7ff200003520 con 0x7ff2200b6990 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 msgr2=0x7ff2200b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2200b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 >> v2:172.21.15.113:3300/0 conn(0x7ff2200ce1d0 msgr2=0x7ff2200d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 --2- 172.21.15.91:0/4183769637 >> v2:172.21.15.113:3300/0 conn(0x7ff2200ce1d0 0x7ff2200d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200b7410 con 0x7ff2200b6990 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff2080019f0 con 0x7ff2200b6990 2024-02-02T13:36:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 >> v1:172.21.15.91:6789/0 conn(0x7ff2200b6990 legacy=0x7ff2200cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200003970 0x7ff200003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff200004d00 0x7ff200005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b81d0 con 0x7ff200004d00 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b7130 con 0x7ff200003970 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.840+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20000b980 con 0x7ff22015cee0 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff200004d00 0x7ff200005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21e000640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 -- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 msgr2=0x7ff2000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21ea00640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200003970 0x7ff200003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 -- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200003970 msgr2=0x7ff200003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200003970 0x7ff200003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21f400640 1 -- 172.21.15.91:0/4183769637 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200003520 con 0x7ff200004d00 2024-02-02T13:36:54.844 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21e000640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff22015cee0 0x7ff2000047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:54.845 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.841+0000 7ff21ea00640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200003970 0x7ff200003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.842+0000 7ff21f400640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff200004d00 0x7ff200005180 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a90 tx=0x7ff2000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262530fe0ed2cdd9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.842+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21802c6b0 con 0x7ff200004d00 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.842+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21802c810 con 0x7ff200004d00 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21d600640 1 -- 172.21.15.91:0/4183769637 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21802cba0 con 0x7ff200004d00 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 -- 172.21.15.91:0/4183769637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff200004d00 msgr2=0x7ff200005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 --2- 172.21.15.91:0/4183769637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff200004d00 0x7ff200005180 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a90 tx=0x7ff2000057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21f400640 1 -- 172.21.15.91:0/4183769637 reap_dead start 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 -- 172.21.15.91:0/4183769637 shutdown_connections 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 -- 172.21.15.91:0/4183769637 >> 172.21.15.91:0/4183769637 conn(0x7ff220070690 msgr2=0x7ff220070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 -- 172.21.15.91:0/4183769637 shutdown_connections 2024-02-02T13:36:54.846 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.843+0000 7ff21fe00640 1 -- 172.21.15.91:0/4183769637 wait complete. 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.845+0000 7ff21fe00640 1 Processor -- start 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.845+0000 7ff21fe00640 1 -- start start 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff200004d00 0x7ff2200b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2200b45c0 0x7ff2200b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220158a60 con 0x7ff2201580a0 2024-02-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b35d0 con 0x7ff2200b45c0 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b38b0 con 0x7ff200004d00 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60738/0 (socket says 172.21.15.91:60738) 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21f400640 1 -- 172.21.15.91:0/999647320 learned_addr learned my addr 172.21.15.91:0/999647320 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.846+0000 7ff21ea00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff200004d00 0x7ff2200b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21e000640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2200b45c0 0x7ff2200b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 -- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff200004d00 msgr2=0x7ff2200b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff200004d00 0x7ff2200b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 -- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2200b45c0 msgr2=0x7ff2200b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2200b45c0 0x7ff2200b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200b3b90 con 0x7ff2201580a0 2024-02-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21f400640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a90 tx=0x7ff218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21802cd90 con 0x7ff2201580a0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff218003610 con 0x7ff2201580a0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180039c0 con 0x7ff2201580a0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.847+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2200b3e70 con 0x7ff2201580a0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.848+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2200b3220 con 0x7ff2201580a0 2024-02-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.849+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22004dff0 con 0x7ff2201580a0 2024-02-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.849+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff218003b20 con 0x7ff2201580a0 2024-02-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.850+0000 7ff216a00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 0x7ff1f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.850+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff218000e40 con 0x7ff2201580a0 2024-02-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.851+0000 7ff21ea00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 0x7ff1f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.851+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7ff1f8071fc0 con 0x7ff2201580a0 2024-02-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.851+0000 7ff21ea00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 0x7ff1f806c8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff210009aa0 tx=0x7ff210008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:54.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:54.855+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff2180753f0 con 0x7ff2201580a0 2024-02-02T13:36:55.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.046+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff22015c6e0 con 0x7ff2201580a0 2024-02-02T13:36:55.048 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.047+0000 7ff216a00640 1 -- 172.21.15.91:0/999647320 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff218078a10 con 0x7ff2201580a0 2024-02-02T13:36:55.048 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:36:55.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.049+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 msgr2=0x7ff1f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.049+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 0x7ff1f806c8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff210009aa0 tx=0x7ff210008040 comp rx=0 tx=0).stop 2024-02-02T13:36:55.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.049+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 msgr2=0x7ff220158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.051 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a90 tx=0x7ff218004730 comp rx=0 tx=0).stop 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 shutdown_connections 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1f806a430 0x7ff1f806c8f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff2201580a0 0x7ff220158520 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2200b45c0 0x7ff2200b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 --2- 172.21.15.91:0/999647320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff200004d00 0x7ff2200b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 >> 172.21.15.91:0/999647320 conn(0x7ff220070690 msgr2=0x7ff22015dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 shutdown_connections 2024-02-02T13:36:55.052 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.050+0000 7ff21fe00640 1 -- 172.21.15.91:0/999647320 wait complete. 2024-02-02T13:36:55.069 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:36:55.280 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.279+0000 7f6e56400640 1 Processor -- start 2024-02-02T13:36:55.281 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.279+0000 7f6e56400640 1 -- start start 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e56400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e50151f90 0x7f6e5015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e56400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e56400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e50070f80 con 0x7f6e500b0c40 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e56400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e500b10d0 con 0x7f6e50151f90 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e56400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e50152370 con 0x7f6e5006f4f0 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e50151f90 0x7f6e5015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6e50151f90 0x7f6e5015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60748/0 (socket says 172.21.15.91:60748) 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 -- 172.21.15.91:0/445245482 learned_addr learned my addr 172.21.15.91:0/445245482 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e47e00640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 -- 172.21.15.91:0/445245482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 msgr2=0x7f6e500b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 -- 172.21.15.91:0/445245482 >> v1:172.21.15.113:6789/0 conn(0x7f6e500b0c40 legacy=0x7f6e500b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:55.283 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.281+0000 7f6e4f400640 1 -- 172.21.15.91:0/445245482 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500cb010 con 0x7f6e50151f90 2024-02-02T13:36:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.283+0000 7f6e4f400640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e50151f90 0x7f6e5015f5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009870 tx=0x7f6e4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394653c58b3305a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.283+0000 7f6e4ea00640 1 -- 172.21.15.91:0/445245482 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40031640 con 0x7f6e50151f90 2024-02-02T13:36:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.283+0000 7f6e4ea00640 1 -- 172.21.15.91:0/445245482 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e400317a0 con 0x7f6e50151f90 2024-02-02T13:36:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.283+0000 7f6e4ea00640 1 -- 172.21.15.91:0/445245482 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40031b30 con 0x7f6e50151f90 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 -- 172.21.15.91:0/445245482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e50151f90 msgr2=0x7f6e5015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e50151f90 0x7f6e5015f5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009870 tx=0x7f6e4002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 -- 172.21.15.91:0/445245482 shutdown_connections 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 --2- 172.21.15.91:0/445245482 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e50151f90 0x7f6e5015f5a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.285 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 -- 172.21.15.91:0/445245482 >> 172.21.15.91:0/445245482 conn(0x7f6e500cec80 msgr2=0x7f6e500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:55.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 -- 172.21.15.91:0/445245482 shutdown_connections 2024-02-02T13:36:55.286 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.284+0000 7f6e56400640 1 -- 172.21.15.91:0/445245482 wait complete. 2024-02-02T13:36:55.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.286+0000 7f6e56400640 1 Processor -- start 2024-02-02T13:36:55.287 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.286+0000 7f6e56400640 1 -- start start 2024-02-02T13:36:55.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 0x7f6e501f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500c4aa0 con 0x7f6e500b0c40 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500c6240 con 0x7f6e50151f90 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.287+0000 7f6e56400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500b4310 con 0x7f6e5006f4f0 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60762/0 (socket says 172.21.15.91:60762) 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 -- 172.21.15.91:0/2848507461 learned_addr learned my addr 172.21.15.91:0/2848507461 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e4f400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e4fe00640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 0x7f6e501f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 -- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 msgr2=0x7f6e500b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 -- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 msgr2=0x7f6e501f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.290 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 0x7f6e501f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400094d0 con 0x7f6e500b0c40 2024-02-02T13:36:55.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e4fe00640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 0x7f6e501f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:55.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e4f400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:55.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.288+0000 7f6e47e00640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800d5f0 tx=0x7f6e3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.289+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380125f0 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.289+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38012750 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.289+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e501f6fa0 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.289+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38012b00 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.289+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e500c44e0 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.290+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e50066cf0 con 0x7f6e500b0c40 2024-02-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.291+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6e38012c60 con 0x7f6e500b0c40 2024-02-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.292+0000 7f6e47400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 0x7f6e2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.292+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6e38088dd0 con 0x7f6e500b0c40 2024-02-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.292+0000 7f6e4f400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 0x7f6e2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.292+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f6e28071fc0 con 0x7f6e500b0c40 2024-02-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.293+0000 7f6e4f400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 0x7f6e2806c8f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036000 tx=0x7f6e4003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.296+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6e38054390 con 0x7f6e500b0c40 2024-02-02T13:36:55.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.483+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e5006c7b0 con 0x7f6e500b0c40 2024-02-02T13:36:55.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.484+0000 7f6e47400640 1 -- 172.21.15.91:0/2848507461 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e380579b0 con 0x7f6e500b0c40 2024-02-02T13:36:55.486 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 msgr2=0x7f6e2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 0x7f6e2806c8f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036000 tx=0x7f6e4003c040 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 msgr2=0x7f6e500c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800d5f0 tx=0x7f6e3800dab0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 shutdown_connections 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6e2806a430 0x7f6e2806c8f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e50151f90 0x7f6e501f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6e500b0c40 0x7f6e500c81d0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 --2- 172.21.15.91:0/2848507461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e5006f4f0 0x7f6e500b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 >> 172.21.15.91:0/2848507461 conn(0x7f6e500cec80 msgr2=0x7f6e500afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 shutdown_connections 2024-02-02T13:36:55.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.487+0000 7f6e56400640 1 -- 172.21.15.91:0/2848507461 wait complete. 2024-02-02T13:36:55.506 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:36:55.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.716+0000 7f83da800640 1 Processor -- start 2024-02-02T13:36:55.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.716+0000 7f83da800640 1 -- start start 2024-02-02T13:36:55.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.717+0000 7f83da800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d4151f90 0x7f83d415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.717+0000 7f83da800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d40b0e10 0x7f83d40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.717+0000 7f83da800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d4070f80 con 0x7f83d406f4f0 2024-02-02T13:36:55.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.717+0000 7f83da800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d4152370 con 0x7f83d4151f90 2024-02-02T13:36:55.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.717+0000 7f83da800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cb010 con 0x7f83d40b0e10 2024-02-02T13:36:55.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.718+0000 7f83d3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d4151f90 0x7f83d415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.718+0000 7f83d3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d40b0e10 0x7f83d40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.718+0000 7f83d3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d4151f90 0x7f83d415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:48138/0 (socket says 172.21.15.91:48138) 2024-02-02T13:36:55.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.718+0000 7f83d3400640 1 -- 172.21.15.91:0/3510260537 learned_addr learned my addr 172.21.15.91:0/3510260537 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:55.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.719+0000 7f83d3400640 1 -- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d40b0e10 msgr2=0x7f83d40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:55.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.719+0000 7f83d3400640 1 --2- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d40b0e10 0x7f83d40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.719+0000 7f83d3400640 1 -- 172.21.15.91:0/3510260537 >> v1:172.21.15.113:6789/0 conn(0x7f83d406f4f0 legacy=0x7f83d40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.719+0000 7f83d3400640 1 -- 172.21.15.91:0/3510260537 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d40cb2f0 con 0x7f83d4151f90 2024-02-02T13:36:55.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.721+0000 7f83d3400640 1 --2- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d4151f90 0x7f83d415f5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a59bb0ac8e1e93 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.721+0000 7f83d2a00640 1 -- 172.21.15.91:0/3510260537 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4030840 con 0x7f83d4151f90 2024-02-02T13:36:55.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.721+0000 7f83d2a00640 1 -- 172.21.15.91:0/3510260537 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40309a0 con 0x7f83d4151f90 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.721+0000 7f83d2a00640 1 -- 172.21.15.91:0/3510260537 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4030d30 con 0x7f83d4151f90 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 -- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d4151f90 msgr2=0x7f83d415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 --2- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d4151f90 0x7f83d415f5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c402b980 comp rx=0 tx=0).stop 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 -- 172.21.15.91:0/3510260537 shutdown_connections 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 --2- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d40b0e10 0x7f83d40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 --2- 172.21.15.91:0/3510260537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d4151f90 0x7f83d415f5a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 -- 172.21.15.91:0/3510260537 >> 172.21.15.91:0/3510260537 conn(0x7f83d40cec80 msgr2=0x7f83d40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:55.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 -- 172.21.15.91:0/3510260537 shutdown_connections 2024-02-02T13:36:55.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.722+0000 7f83da800640 1 -- 172.21.15.91:0/3510260537 wait complete. 2024-02-02T13:36:55.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.724+0000 7f83da800640 1 Processor -- start 2024-02-02T13:36:55.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.724+0000 7f83da800640 1 -- start start 2024-02-02T13:36:55.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b0e10 0x7f83d415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d4151f90 0x7f83d415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d41f32e0 con 0x7f83d406f4f0 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d41f35c0 con 0x7f83d40b0e10 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83da800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d41f38a0 con 0x7f83d4151f90 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83d3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83d3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60778/0 (socket says 172.21.15.91:60778) 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83d3400640 1 -- 172.21.15.91:0/4206298138 learned_addr learned my addr 172.21.15.91:0/4206298138 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83ca800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b0e10 0x7f83d415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.725+0000 7f83d3e00640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d4151f90 0x7f83d415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 -- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d4151f90 msgr2=0x7f83d415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d4151f90 0x7f83d415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 -- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b0e10 msgr2=0x7f83d415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b0e10 0x7f83d415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40094d0 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83d3400640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4039c90 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4039df0 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4043640 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d414fe90 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.726+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d40b2e30 con 0x7f83d406f4f0 2024-02-02T13:36:55.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.728+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d4066cf0 con 0x7f83d406f4f0 2024-02-02T13:36:55.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.729+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f83c40437a0 con 0x7f83d406f4f0 2024-02-02T13:36:55.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.729+0000 7f83cbe00640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 0x7f83ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:55.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.730+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f83c403e070 con 0x7f83d406f4f0 2024-02-02T13:36:55.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.730+0000 7f83ca800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 0x7f83ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:55.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.730+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f83ac071fc0 con 0x7f83d406f4f0 2024-02-02T13:36:55.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.730+0000 7f83ca800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 0x7f83ac06c8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83bc007fd0 tx=0x7f83bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:55.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.734+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f83c4034060 con 0x7f83d406f4f0 2024-02-02T13:36:55.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.923+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83d406c7b0 con 0x7f83d406f4f0 2024-02-02T13:36:55.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.924+0000 7f83cbe00640 1 -- 172.21.15.91:0/4206298138 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83c4034070 con 0x7f83d406f4f0 2024-02-02T13:36:55.926 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:36:55.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.926+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 msgr2=0x7f83ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.926+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 0x7f83ac06c8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83bc007fd0 tx=0x7f83bc009040 comp rx=0 tx=0).stop 2024-02-02T13:36:55.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 msgr2=0x7f83d40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c4004730 comp rx=0 tx=0).stop 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 shutdown_connections 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83ac06a430 0x7f83ac06c8f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d4151f90 0x7f83d415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b0e10 0x7f83d415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 --2- 172.21.15.91:0/4206298138 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83d406f4f0 0x7f83d40b28f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 >> 172.21.15.91:0/4206298138 conn(0x7f83d40cec80 msgr2=0x7f83d415eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 shutdown_connections 2024-02-02T13:36:55.929 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:55.927+0000 7f83da800640 1 -- 172.21.15.91:0/4206298138 wait complete. 2024-02-02T13:36:55.948 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:36:56.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.158+0000 7fc234600640 1 Processor -- start 2024-02-02T13:36:56.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.158+0000 7fc234600640 1 -- start start 2024-02-02T13:36:56.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.159+0000 7fc234600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 0x7fc22c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.159+0000 7fc234600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc22c0b0c40 0x7fc22c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.159+0000 7fc234600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c15fa00 con 0x7fc22c151f90 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.159+0000 7fc234600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c0b10d0 con 0x7fc22c0b0c40 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.159+0000 7fc234600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c070f80 con 0x7fc22c15f5a0 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc231c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc22c151f90 0x7fc22c06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49346/0 (socket says 172.21.15.91:49346) 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc231c00640 1 -- 172.21.15.91:0/2097467128 learned_addr learned my addr 172.21.15.91:0/2097467128 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc232600640 1 --2- 172.21.15.91:0/2097467128 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0b0c40 0x7fc22c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827960152 0 0) 0x7fc21c001490 con 0x7fc22c151f90 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc231200640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 0x7fc22c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c003520 con 0x7fc22c151f90 2024-02-02T13:36:56.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784061188 0 0) 0x7fc20c003520 con 0x7fc22c151f90 2024-02-02T13:36:56.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 msgr2=0x7fc22c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 0x7fc22c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0b0c40 msgr2=0x7fc22c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.160+0000 7fc22be00640 1 --2- 172.21.15.91:0/2097467128 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0b0c40 0x7fc22c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c152370 con 0x7fc22c151f90 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc21c001b80 con 0x7fc22c151f90 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 >> v1:172.21.15.91:6789/0 conn(0x7fc22c151f90 legacy=0x7fc22c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc20c003970 0x7fc20c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc20c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 0x7fc20c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b10d0 con 0x7fc20c0047c0 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c070f80 con 0x7fc20c003970 2024-02-02T13:36:56.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00b980 con 0x7fc22c15f5a0 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc20c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 -- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 msgr2=0x7fc20c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15f5a0 0x7fc20c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 -- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc20c003970 msgr2=0x7fc20c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc20c003970 0x7fc20c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.161+0000 7fc231200640 1 -- 172.21.15.91:0/2097467128 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c003520 con 0x7fc20c0047c0 2024-02-02T13:36:56.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.162+0000 7fc231200640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc20c004c40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc214009ac0 tx=0x7fc20c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e3e6587cc3ae77 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21402c710 con 0x7fc20c0047c0 2024-02-02T13:36:56.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21402c870 con 0x7fc20c0047c0 2024-02-02T13:36:56.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc22be00640 1 -- 172.21.15.91:0/2097467128 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21402cc00 con 0x7fc20c0047c0 2024-02-02T13:36:56.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 -- 172.21.15.91:0/2097467128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 msgr2=0x7fc20c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 --2- 172.21.15.91:0/2097467128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc20c004c40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc214009ac0 tx=0x7fc20c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc232600640 1 -- 172.21.15.91:0/2097467128 reap_dead start 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 -- 172.21.15.91:0/2097467128 shutdown_connections 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 -- 172.21.15.91:0/2097467128 >> 172.21.15.91:0/2097467128 conn(0x7fc22c0cec80 msgr2=0x7fc22c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 -- 172.21.15.91:0/2097467128 shutdown_connections 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.163+0000 7fc234600640 1 -- 172.21.15.91:0/2097467128 wait complete. 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.165+0000 7fc234600640 1 Processor -- start 2024-02-02T13:36:56.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.165+0000 7fc234600640 1 -- start start 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c1f4030 0x7fc22c1f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c1f38f0 0x7fc22c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b48b0 con 0x7fc20c0047c0 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0cc8f0 con 0x7fc22c1f4030 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc234600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0ccb90 con 0x7fc22c1f38f0 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc231c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc231c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60790/0 (socket says 172.21.15.91:60790) 2024-02-02T13:36:56.169 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.166+0000 7fc231c00640 1 -- 172.21.15.91:0/1100153869 learned_addr learned my addr 172.21.15.91:0/1100153869 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 -- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c1f38f0 msgr2=0x7fc22c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c1f38f0 0x7fc22c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 -- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c1f4030 msgr2=0x7fc22c1f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231200640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c1f4030 0x7fc22c1f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c1f4030 0x7fc22c1f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c0cce70 con 0x7fc20c0047c0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc231c00640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c9f0 tx=0x7fc21c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00a760 con 0x7fc20c0047c0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c00a8c0 con 0x7fc20c0047c0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00ac70 con 0x7fc20c0047c0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.167+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0cd150 con 0x7fc20c0047c0 2024-02-02T13:36:56.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.168+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0b2050 con 0x7fc20c0047c0 2024-02-02T13:36:56.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.169+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0cd340 con 0x7fc20c0047c0 2024-02-02T13:36:56.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.170+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc21c0070d0 con 0x7fc20c0047c0 2024-02-02T13:36:56.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.171+0000 7fc229600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 0x7fc20406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.171+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc21c0880e0 con 0x7fc20c0047c0 2024-02-02T13:36:56.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.171+0000 7fc231200640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 0x7fc20406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.171+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fc204071ff0 con 0x7fc20c0047c0 2024-02-02T13:36:56.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.172+0000 7fc231200640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 0x7fc20406c920 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc214036000 tx=0x7fc214030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.175+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc21c018070 con 0x7fc20c0047c0 2024-02-02T13:36:56.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.401+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc22c15fa00 con 0x7fc20c0047c0 2024-02-02T13:36:56.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.402+0000 7fc229600640 1 -- 172.21.15.91:0/1100153869 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7fc22c15fa00 con 0x7fc20c0047c0 2024-02-02T13:36:56.404 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:56.404 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:36:56.404 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.404+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 msgr2=0x7fc20406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.404+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 0x7fc20406c920 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc214036000 tx=0x7fc214030000 comp rx=0 tx=0).stop 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.404+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 msgr2=0x7fc22c0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.404+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c9f0 tx=0x7fc21c00ceb0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 shutdown_connections 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc20406a460 0x7fc20406c920 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c1f38f0 0x7fc22c0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c1f4030 0x7fc22c1f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 --2- 172.21.15.91:0/1100153869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc20c0047c0 0x7fc22c0b1b10 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 >> 172.21.15.91:0/1100153869 conn(0x7fc22c0cec80 msgr2=0x7fc22c0c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 shutdown_connections 2024-02-02T13:36:56.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.405+0000 7fc234600640 1 -- 172.21.15.91:0/1100153869 wait complete. 2024-02-02T13:36:56.424 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:36:56.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.621+0000 7fad14c00640 1 Processor -- start 2024-02-02T13:36:56.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.621+0000 7fad14c00640 1 -- start start 2024-02-02T13:36:56.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad14c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad101597a0 0x7fad1006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad14c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1006b5f0 con 0x7fad100c9950 2024-02-02T13:36:56.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad14c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1006c450 con 0x7fad100cd690 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad14c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad10159b80 con 0x7fad101597a0 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad0e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fad100c9950 0x7fad1006f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49350/0 (socket says 172.21.15.91:49350) 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.622+0000 7fad0e800640 1 -- 172.21.15.91:0/988400954 learned_addr learned my addr 172.21.15.91:0/988400954 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0de00640 1 --2- 172.21.15.91:0/988400954 >> v2:172.21.15.113:3300/0 conn(0x7fad101597a0 0x7fad1006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961621876 0 0) 0x7fad1006b5f0 con 0x7fad100c9950 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf0003520 con 0x7fad100c9950 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300525488 0 0) 0x7facf0003520 con 0x7fad100c9950 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 >> v2:172.21.15.113:3300/0 conn(0x7fad101597a0 msgr2=0x7fad1006c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 --2- 172.21.15.91:0/988400954 >> v2:172.21.15.113:3300/0 conn(0x7fad101597a0 0x7fad1006c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 >> v1:172.21.15.113:6789/0 conn(0x7fad100cd690 legacy=0x7fad100c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1006b840 con 0x7fad100c9950 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fad080019f0 con 0x7fad100c9950 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 >> v1:172.21.15.91:6789/0 conn(0x7fad100c9950 legacy=0x7fad1006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 0x7facf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10159b80 con 0x7facf00039c0 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf000c080 con 0x7facf0005180 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.623+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf000c240 con 0x7facf00047c0 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 0x7facf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0e800640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0d400640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 -- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00047c0 msgr2=0x7facf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 -- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0005180 msgr2=0x7facf000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0de00640 1 -- 172.21.15.91:0/988400954 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0003520 con 0x7facf00039c0 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0d400640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0e800640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.624+0000 7fad0e800640 1 -- 172.21.15.91:0/988400954 reap_dead start 2024-02-02T13:36:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.625+0000 7fad0de00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 0x7facf0003de0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fad00009ac0 tx=0x7facf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964cfe74d879d289 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0002c6b0 con 0x7facf00039c0 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0002c810 con 0x7facf00039c0 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad0ca00640 1 -- 172.21.15.91:0/988400954 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0002cba0 con 0x7facf00039c0 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 -- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 msgr2=0x7facf0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 0x7facf0003de0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fad00009ac0 tx=0x7facf00057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 -- 172.21.15.91:0/988400954 shutdown_connections 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 --2- 172.21.15.91:0/988400954 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facf00039c0 0x7facf0003de0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 -- 172.21.15.91:0/988400954 >> 172.21.15.91:0/988400954 conn(0x7fad100cec80 msgr2=0x7fad100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 -- 172.21.15.91:0/988400954 shutdown_connections 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.626+0000 7fad14c00640 1 -- 172.21.15.91:0/988400954 wait complete. 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.628+0000 7fad14c00640 1 Processor -- start 2024-02-02T13:36:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.628+0000 7fad14c00640 1 -- start start 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00039c0 0x7fad1014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1014fc30 0x7fad1015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10158d30 con 0x7fad100c5cc0 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1014db20 con 0x7fad1014fc30 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad14c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1014ddf0 con 0x7facf00039c0 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad0e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad0e800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60804/0 (socket says 172.21.15.91:60804) 2024-02-02T13:36:56.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.629+0000 7fad0e800640 1 -- 172.21.15.91:0/3327403291 learned_addr learned my addr 172.21.15.91:0/3327403291 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0de00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00039c0 0x7fad1014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 -- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00039c0 msgr2=0x7fad1014ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00039c0 0x7fad1014ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0d400640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1014fc30 0x7fad1015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 -- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1014fc30 msgr2=0x7fad1015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1014fc30 0x7fad1015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1014e0d0 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad0e800640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fad08000f80 tx=0x7fad0800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800b860 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800b9e0 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800bd90 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.630+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1014e3b0 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.631+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1014f370 con 0x7fad100c5cc0 2024-02-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.632+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1014e8d0 con 0x7fad100c5cc0 2024-02-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.633+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fad08007110 con 0x7fad100c5cc0 2024-02-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.633+0000 7fad06000640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 0x7face806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.634+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fad08088c70 con 0x7fad100c5cc0 2024-02-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.634+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7face8071fc0 con 0x7fad100c5cc0 2024-02-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.634+0000 7fad0de00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 0x7face806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:56.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.635+0000 7fad0de00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 0x7face806c8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad00036000 tx=0x7fad00030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:56.639 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.638+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fad1014e8d0 con 0x7fad100c5cc0 2024-02-02T13:36:56.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.830+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad1014e8d0 con 0x7fad100c5cc0 2024-02-02T13:36:56.834 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.832+0000 7fad06000640 1 -- 172.21.15.91:0/3327403291 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16188 (secure 0 0 0) 0x7fad1014e8d0 con 0x7fad100c5cc0 2024-02-02T13:36:56.834 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:56.834 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":182,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:36:28.579012+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.28","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:36:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 msgr2=0x7face806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 0x7face806c8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad00036000 tx=0x7fad00030000 comp rx=0 tx=0).stop 2024-02-02T13:36:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 msgr2=0x7fad100c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fad08000f80 tx=0x7fad0800df70 comp rx=0 tx=0).stop 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 shutdown_connections 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7face806a430 0x7face806c8f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad100c5cc0 0x7fad100c6150 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1014fc30 0x7fad1015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 --2- 172.21.15.91:0/3327403291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facf00039c0 0x7fad1014ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 >> 172.21.15.91:0/3327403291 conn(0x7fad100cec80 msgr2=0x7fad100cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 shutdown_connections 2024-02-02T13:36:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:56.835+0000 7fad14c00640 1 -- 172.21.15.91:0/3327403291 wait complete. 2024-02-02T13:36:56.856 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:36:56.856 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:36:57.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.067+0000 7fcba2a00640 1 Processor -- start 2024-02-02T13:36:57.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.067+0000 7fcba2a00640 1 -- start start 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb9c06c820 0x7fcb9c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c0ad2e0 con 0x7fcb9c15d2c0 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0ad5c0 con 0x7fcb9c06cd50 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c06b4f0 con 0x7fcb9c06c820 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60818/0 (socket says 172.21.15.91:60818) 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 -- 172.21.15.91:0/1478930568 learned_addr learned my addr 172.21.15.91:0/1478930568 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 -- 172.21.15.91:0/1478930568 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c06c820 msgr2=0x7fcb9c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 --2- 172.21.15.91:0/1478930568 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c06c820 0x7fcb9c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 -- 172.21.15.91:0/1478930568 >> v1:172.21.15.113:6789/0 conn(0x7fcb9c15d2c0 legacy=0x7fcb9c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:57.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.069+0000 7fcba2000640 1 -- 172.21.15.91:0/1478930568 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c06b7d0 con 0x7fcb9c06cd50 2024-02-02T13:36:57.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.071+0000 7fcba2000640 1 --2- 172.21.15.91:0/1478930568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c067780 tx=0x7fcb9402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db05d645cc593363 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.071+0000 7fcb9be00640 1 -- 172.21.15.91:0/1478930568 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9402ca80 con 0x7fcb9c06cd50 2024-02-02T13:36:57.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.071+0000 7fcb9be00640 1 -- 172.21.15.91:0/1478930568 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9402cbe0 con 0x7fcb9c06cd50 2024-02-02T13:36:57.073 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.071+0000 7fcb9be00640 1 -- 172.21.15.91:0/1478930568 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940343f0 con 0x7fcb9c06cd50 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 -- 172.21.15.91:0/1478930568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06cd50 msgr2=0x7fcb9c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1478930568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c067780 tx=0x7fcb9402bee0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 -- 172.21.15.91:0/1478930568 shutdown_connections 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1478930568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06cd50 0x7fcb9c0ccba0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1478930568 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c06c820 0x7fcb9c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 -- 172.21.15.91:0/1478930568 >> 172.21.15.91:0/1478930568 conn(0x7fcb9c0cea90 msgr2=0x7fcb9c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 -- 172.21.15.91:0/1478930568 shutdown_connections 2024-02-02T13:36:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.072+0000 7fcba2a00640 1 -- 172.21.15.91:0/1478930568 wait complete. 2024-02-02T13:36:57.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.073+0000 7fcba2a00640 1 Processor -- start 2024-02-02T13:36:57.075 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.074+0000 7fcba2a00640 1 -- start start 2024-02-02T13:36:57.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.074+0000 7fcba2a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.074+0000 7fcba2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c06cd50 0x7fcb9c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c15d2c0 0x7fcb9c0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.077 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba2a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0b27b0 con 0x7fcb9c06c820 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0b2230 con 0x7fcb9c15d2c0 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba2a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0b24d0 con 0x7fcb9c06cd50 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60830/0 (socket says 172.21.15.91:60830) 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 -- 172.21.15.91:0/1774922839 learned_addr learned my addr 172.21.15.91:0/1774922839 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 -- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c06cd50 msgr2=0x7fcb9c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c06cd50 0x7fcb9c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba2000640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c15d2c0 0x7fcb9c0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 -- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c15d2c0 msgr2=0x7fcb9c0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c15d2c0 0x7fcb9c0b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.075+0000 7fcba1600640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94009b30 con 0x7fcb9c06c820 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcba1600640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b3e0 tx=0x7fcb8400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011830 con 0x7fcb9c06c820 2024-02-02T13:36:57.078 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb840119b0 con 0x7fcb9c06c820 2024-02-02T13:36:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c15d820 con 0x7fcb9c06c820 2024-02-02T13:36:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011d60 con 0x7fcb9c06c820 2024-02-02T13:36:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.076+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c0c5fe0 con 0x7fcb9c06c820 2024-02-02T13:36:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.077+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c0c5b40 con 0x7fcb9c06c820 2024-02-02T13:36:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.078+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcb8401a400 con 0x7fcb9c06c820 2024-02-02T13:36:57.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.079+0000 7fcb99600640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 0x7fcb7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.080 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.079+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcb84088c10 con 0x7fcb9c06c820 2024-02-02T13:36:57.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.079+0000 7fcba0c00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 0x7fcb7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.079+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fcb74071fc0 con 0x7fcb9c06c820 2024-02-02T13:36:57.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.080+0000 7fcba0c00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 0x7fcb7406c8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b1250 tx=0x7fcb8c006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.083+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcb84015020 con 0x7fcb9c06c820 2024-02-02T13:36:57.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.310+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb9c0c6200 con 0x7fcb9c06c820 2024-02-02T13:36:57.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.311+0000 7fcb99600640 1 -- 172.21.15.91:0/1774922839 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7fcb84054140 con 0x7fcb9c06c820 2024-02-02T13:36:57.313 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:57.313 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:36:57.314 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:36:57.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 msgr2=0x7fcb7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 0x7fcb7406c8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b1250 tx=0x7fcb8c006c90 comp rx=0 tx=0).stop 2024-02-02T13:36:57.316 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 msgr2=0x7fcb9c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b3e0 tx=0x7fcb8400b8a0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 shutdown_connections 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb7406a430 0x7fcb7406c8f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c06cd50 0x7fcb9c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c15d2c0 0x7fcb9c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 --2- 172.21.15.91:0/1774922839 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb9c06c820 0x7fcb9c0b0d10 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.314+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 >> 172.21.15.91:0/1774922839 conn(0x7fcb9c0cea90 msgr2=0x7fcb9c15a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.315+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 shutdown_connections 2024-02-02T13:36:57.317 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.315+0000 7fcba2a00640 1 -- 172.21.15.91:0/1774922839 wait complete. 2024-02-02T13:36:57.335 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:36:57.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.540+0000 7fd679a00640 1 Processor -- start 2024-02-02T13:36:57.542 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.540+0000 7fd679a00640 1 -- start start 2024-02-02T13:36:57.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.541+0000 7fd679a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd6741597a0 0x7fd67406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.541+0000 7fd679a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 0x7fd6740cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd679a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406b5f0 con 0x7fd6740cd690 2024-02-02T13:36:57.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd679a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67406c450 con 0x7fd6741597a0 2024-02-02T13:36:57.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd679a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd674159b80 con 0x7fd6740c9950 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd6741597a0 0x7fd67406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd6741597a0 0x7fd67406c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60834/0 (socket says 172.21.15.91:60834) 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 -- 172.21.15.91:0/4006584580 learned_addr learned my addr 172.21.15.91:0/4006584580 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 -- 172.21.15.91:0/4006584580 >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 msgr2=0x7fd6740cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd673400640 1 --2- 172.21.15.91:0/4006584580 >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 0x7fd6740cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 --2- 172.21.15.91:0/4006584580 >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 0x7fd6740cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd671600640 1 -- 172.21.15.91:0/4006584580 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151372715 0 0) 0x7fd67406b5f0 con 0x7fd6740cd690 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 -- 172.21.15.91:0/4006584580 >> v1:172.21.15.91:6789/0 conn(0x7fd6740cd690 legacy=0x7fd6740c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd672a00640 1 -- 172.21.15.91:0/4006584580 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67406b840 con 0x7fd6741597a0 2024-02-02T13:36:57.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.542+0000 7fd673400640 1 --2- 172.21.15.91:0/4006584580 >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 0x7fd6740cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:57.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd672a00640 1 --2- 172.21.15.91:0/4006584580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6741597a0 0x7fd67406c8e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b11b0 tx=0x7fd66402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a559610689db97 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd671600640 1 -- 172.21.15.91:0/4006584580 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664030860 con 0x7fd6741597a0 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd671600640 1 -- 172.21.15.91:0/4006584580 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6640309c0 con 0x7fd6741597a0 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd671600640 1 -- 172.21.15.91:0/4006584580 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664030d50 con 0x7fd6741597a0 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd679a00640 1 -- 172.21.15.91:0/4006584580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6741597a0 msgr2=0x7fd67406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.544+0000 7fd679a00640 1 --2- 172.21.15.91:0/4006584580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6741597a0 0x7fd67406c8e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b11b0 tx=0x7fd66402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 -- 172.21.15.91:0/4006584580 shutdown_connections 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 --2- 172.21.15.91:0/4006584580 >> v2:172.21.15.113:3300/0 conn(0x7fd6740c9950 0x7fd6740cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 --2- 172.21.15.91:0/4006584580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6741597a0 0x7fd67406c8e0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 -- 172.21.15.91:0/4006584580 >> 172.21.15.91:0/4006584580 conn(0x7fd6740cec80 msgr2=0x7fd6740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:57.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 -- 172.21.15.91:0/4006584580 shutdown_connections 2024-02-02T13:36:57.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.545+0000 7fd679a00640 1 -- 172.21.15.91:0/4006584580 wait complete. 2024-02-02T13:36:57.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.546+0000 7fd679a00640 1 Processor -- start 2024-02-02T13:36:57.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 -- start start 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6740c9950 0x7fd6740c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6741597a0 0x7fd6740c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c74a0 con 0x7fd6740cd690 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c7780 con 0x7fd6741597a0 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.547+0000 7fd679a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c7a60 con 0x7fd6740c9950 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60838/0 (socket says 172.21.15.91:60838) 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 -- 172.21.15.91:0/297366463 learned_addr learned my addr 172.21.15.91:0/297366463 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd673400640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6741597a0 0x7fd6740c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6740c9950 0x7fd6740c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 -- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6740c9950 msgr2=0x7fd6740c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6740c9950 0x7fd6740c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 -- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6741597a0 msgr2=0x7fd6740c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6741597a0 0x7fd6740c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640094d0 con 0x7fd6740cd690 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.548+0000 7fd672000640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c4b0 tx=0x7fd65c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.549+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c005480 con 0x7fd6740cd690 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.549+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c0035f0 con 0x7fd6740cd690 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.549+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0039e0 con 0x7fd6740cd690 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.549+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6740c7d40 con 0x7fd6740cd690 2024-02-02T13:36:57.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.549+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740b01c0 con 0x7fd6740cd690 2024-02-02T13:36:57.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.550+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6740c8260 con 0x7fd6740cd690 2024-02-02T13:36:57.552 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.551+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd65c003b40 con 0x7fd6740cd690 2024-02-02T13:36:57.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.552+0000 7fd66aa00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 0x7fd64c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:57.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.552+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd65c087f70 con 0x7fd6740cd690 2024-02-02T13:36:57.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.552+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fd64c071fc0 con 0x7fd6740cd690 2024-02-02T13:36:57.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.552+0000 7fd672a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 0x7fd64c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:57.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.553+0000 7fd672a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 0x7fd64c06c8f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd664036000 tx=0x7fd66403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:57.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.556+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd65c017020 con 0x7fd6740cd690 2024-02-02T13:36:57.789 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.788+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd674066cf0 con 0x7fd6740cd690 2024-02-02T13:36:57.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.788+0000 7fd66aa00640 1 -- 172.21.15.91:0/297366463 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4716 (secure 0 0 0) 0x7fd6740c8260 con 0x7fd6740cd690 2024-02-02T13:36:57.790 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:57.790 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:35:32.637261+0000","modified":"2024-02-02T13:35:35.664997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8439":{"gid":8439,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/2606135586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2606135586},{"type":"v1","addr":"172.21.15.91:6837","nonce":2606135586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-02-02T13:36:57.791 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 80 2024-02-02T13:36:57.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 msgr2=0x7fd64c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 0x7fd64c06c8f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd664036000 tx=0x7fd66403b040 comp rx=0 tx=0).stop 2024-02-02T13:36:57.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 msgr2=0x7fd6740afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:57.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c4b0 tx=0x7fd65c00c970 comp rx=0 tx=0).stop 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 shutdown_connections 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd64c06a430 0x7fd64c06c8f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6741597a0 0x7fd6740c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6740cd690 0x7fd6740afc80 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 --2- 172.21.15.91:0/297366463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6740c9950 0x7fd6740c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 >> 172.21.15.91:0/297366463 conn(0x7fd6740cec80 msgr2=0x7fd674070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:57.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 shutdown_connections 2024-02-02T13:36:57.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:57.791+0000 7fd679a00640 1 -- 172.21.15.91:0/297366463 wait complete. 2024-02-02T13:36:57.810 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:36:58.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.012+0000 7fae3a000640 1 Processor -- start 2024-02-02T13:36:58.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.012+0000 7fae3a000640 1 -- start start 2024-02-02T13:36:58.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae3a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae3415d490 0x7fae3406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae3a000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3415d870 con 0x7fae340b0e10 2024-02-02T13:36:58.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae3a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3406bce0 con 0x7fae340ac8d0 2024-02-02T13:36:58.015 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae3a000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae34161220 con 0x7fae3415d490 2024-02-02T13:36:58.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae33e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fae340b0e10 0x7fae340c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49370/0 (socket says 172.21.15.91:49370) 2024-02-02T13:36:58.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.014+0000 7fae33e00640 1 -- 172.21.15.91:0/2580303875 learned_addr learned my addr 172.21.15.91:0/2580303875 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:58.017 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae33400640 1 --2- 172.21.15.91:0/2580303875 >> v2:172.21.15.113:3300/0 conn(0x7fae3415d490 0x7fae3406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084196804 0 0) 0x7fae3415d870 con 0x7fae340b0e10 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae14003520 con 0x7fae340b0e10 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536399127 0 0) 0x7fae14003520 con 0x7fae340b0e10 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 >> v2:172.21.15.113:3300/0 conn(0x7fae3415d490 msgr2=0x7fae3406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 --2- 172.21.15.91:0/2580303875 >> v2:172.21.15.113:3300/0 conn(0x7fae3415d490 0x7fae3406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 >> v1:172.21.15.113:6789/0 conn(0x7fae340ac8d0 legacy=0x7fae340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340ad490 con 0x7fae340b0e10 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fae2c0019f0 con 0x7fae340b0e10 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 >> v1:172.21.15.91:6789/0 conn(0x7fae340b0e10 legacy=0x7fae340c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14005180 0x7fae1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae34161220 con 0x7fae14003970 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1400c020 con 0x7fae140047c0 2024-02-02T13:36:58.018 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.015+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1400c1e0 con 0x7fae14005180 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae32a00640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 -- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14005180 msgr2=0x7fae1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14005180 0x7fae1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 -- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae140047c0 msgr2=0x7fae14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33e00640 1 -- 172.21.15.91:0/2580303875 reap_dead start 2024-02-02T13:36:58.019 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.016+0000 7fae33400640 1 -- 172.21.15.91:0/2580303875 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14003520 con 0x7fae14003970 2024-02-02T13:36:58.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.018+0000 7fae33400640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 0x7fae14003d90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fae24009ac0 tx=0x7fae140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9893ede9cd83ebb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.018+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2402c6d0 con 0x7fae14003970 2024-02-02T13:36:58.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.018+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2402c830 con 0x7fae14003970 2024-02-02T13:36:58.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.018+0000 7fae32000640 1 -- 172.21.15.91:0/2580303875 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2402cbc0 con 0x7fae14003970 2024-02-02T13:36:58.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 -- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 msgr2=0x7fae14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 0x7fae14003d90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fae24009ac0 tx=0x7fae140057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 -- 172.21.15.91:0/2580303875 shutdown_connections 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 --2- 172.21.15.91:0/2580303875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 -- 172.21.15.91:0/2580303875 >> 172.21.15.91:0/2580303875 conn(0x7fae340cec80 msgr2=0x7fae340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 -- 172.21.15.91:0/2580303875 shutdown_connections 2024-02-02T13:36:58.021 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.019+0000 7fae3a000640 1 -- 172.21.15.91:0/2580303875 wait complete. 2024-02-02T13:36:58.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.021+0000 7fae3a000640 1 Processor -- start 2024-02-02T13:36:58.022 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.021+0000 7fae3a000640 1 -- start start 2024-02-02T13:36:58.024 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14003970 0x7fae340c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae340b2740 0x7fae3415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340c89a0 con 0x7fae340c5cc0 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340b2e20 con 0x7fae340b2740 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae3a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae340c42b0 con 0x7fae14003970 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60854/0 (socket says 172.21.15.91:60854) 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 -- 172.21.15.91:0/1596940419 learned_addr learned my addr 172.21.15.91:0/1596940419 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 -- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14003970 msgr2=0x7fae340c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14003970 0x7fae340c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 -- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae340b2740 msgr2=0x7fae3415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae32a00640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae340b2740 0x7fae3415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae340b2740 0x7fae3415c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.022+0000 7fae33e00640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340b2bc0 con 0x7fae340c5cc0 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae33e00640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fae2c001560 tx=0x7fae2c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00ce70 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00a610 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00a9c0 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae340c3980 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.023+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae340c3f40 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.024+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae34066cf0 con 0x7fae340c5cc0 2024-02-02T13:36:58.026 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.025+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fae2c00ab20 con 0x7fae340c5cc0 2024-02-02T13:36:58.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.026+0000 7fae2b400640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 0x7fae0c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.026+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fae2c0881d0 con 0x7fae340c5cc0 2024-02-02T13:36:58.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.026+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fae0c071ff0 con 0x7fae340c5cc0 2024-02-02T13:36:58.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.026+0000 7fae33400640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 0x7fae0c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.027+0000 7fae33400640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 0x7fae0c06c920 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae24036000 tx=0x7fae2402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.031+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fae2c053790 con 0x7fae340c5cc0 2024-02-02T13:36:58.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.220+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae340c43f0 con 0x7fae340c5cc0 2024-02-02T13:36:58.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.222+0000 7fae2b400640 1 -- 172.21.15.91:0/1596940419 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16188 (secure 0 0 0) 0x7fae2c019070 con 0x7fae340c5cc0 2024-02-02T13:36:58.224 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:36:58.224 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":182,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:36:28.579012+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:35:31.112699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-02-02T13:35:31.642725+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.28","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.225+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 msgr2=0x7fae0c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.225+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 0x7fae0c06c920 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae24036000 tx=0x7fae2402e000 comp rx=0 tx=0).stop 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.225+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 msgr2=0x7fae340c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.225+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fae2c001560 tx=0x7fae2c00cc20 comp rx=0 tx=0).stop 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 shutdown_connections 2024-02-02T13:36:58.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae0c06a460 0x7fae0c06c920 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae340c5cc0 0x7fae340c6140 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae340b2740 0x7fae3415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 --2- 172.21.15.91:0/1596940419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae14003970 0x7fae340c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 >> 172.21.15.91:0/1596940419 conn(0x7fae340cec80 msgr2=0x7fae340b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 shutdown_connections 2024-02-02T13:36:58.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.226+0000 7fae3a000640 1 -- 172.21.15.91:0/1596940419 wait complete. 2024-02-02T13:36:58.245 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:36:58.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.442+0000 7f5048200640 1 Processor -- start 2024-02-02T13:36:58.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.443+0000 7f5048200640 1 -- start start 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5048200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5040159de0 0x7f50400cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5048200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 0x7f504015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5048200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400cbf20 con 0x7f504015e170 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5048200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5040070ca0 con 0x7f5040159de0 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5048200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ca0c0 con 0x7f50400cbac0 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5046200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f504015e170 0x7f50400cd850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:49386/0 (socket says 172.21.15.91:49386) 2024-02-02T13:36:58.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.444+0000 7f5046200640 1 -- 172.21.15.91:0/2019974818 learned_addr learned my addr 172.21.15.91:0/2019974818 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f5045800640 1 --2- 172.21.15.91:0/2019974818 >> v2:172.21.15.113:3300/0 conn(0x7f5040159de0 0x7f50400cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f5044e00640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 0x7f504015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234888677 0 0) 0x7f50400cbf20 con 0x7f504015e170 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5020003520 con 0x7f504015e170 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474408043 0 0) 0x7f5020003520 con 0x7f504015e170 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 msgr2=0x7f504015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 0x7f504015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 >> v2:172.21.15.113:3300/0 conn(0x7f5040159de0 msgr2=0x7f50400cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 --2- 172.21.15.91:0/2019974818 >> v2:172.21.15.113:3300/0 conn(0x7f5040159de0 0x7f50400cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400ca380 con 0x7f504015e170 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f5038001670 con 0x7f504015e170 2024-02-02T13:36:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 >> v1:172.21.15.91:6789/0 conn(0x7f504015e170 legacy=0x7f50400cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.445+0000 7f503fe00640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5020003970 0x7f5020003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f503fe00640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50200047c0 0x7f5020004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f503fe00640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 0x7f5020005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040070ca0 con 0x7f5020003970 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ca0c0 con 0x7f50200047c0 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502000b980 con 0x7f50400cbac0 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5020003970 0x7f5020003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 -- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 msgr2=0x7f5020005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50400cbac0 0x7f5020005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 -- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50200047c0 msgr2=0x7f5020004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50200047c0 0x7f5020004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.446+0000 7f5045800640 1 -- 172.21.15.91:0/2019974818 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020003520 con 0x7f5020003970 2024-02-02T13:36:58.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.447+0000 7f5045800640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5020003970 0x7f5020003d90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fc0 tx=0x7f50200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535c630b5f64dfee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503002cc40 con 0x7f5020003970 2024-02-02T13:36:58.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503000f040 con 0x7f5020003970 2024-02-02T13:36:58.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f503fe00640 1 -- 172.21.15.91:0/2019974818 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030034440 con 0x7f5020003970 2024-02-02T13:36:58.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f5048200640 1 -- 172.21.15.91:0/2019974818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5020003970 msgr2=0x7f5020003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f5048200640 1 --2- 172.21.15.91:0/2019974818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5020003970 0x7f5020003d90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fc0 tx=0x7f50200057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f5046200640 1 -- 172.21.15.91:0/2019974818 reap_dead start 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f5048200640 1 -- 172.21.15.91:0/2019974818 shutdown_connections 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.448+0000 7f5048200640 1 -- 172.21.15.91:0/2019974818 >> 172.21.15.91:0/2019974818 conn(0x7f50400b0700 msgr2=0x7f50400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.449+0000 7f5048200640 1 -- 172.21.15.91:0/2019974818 shutdown_connections 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.449+0000 7f5048200640 1 -- 172.21.15.91:0/2019974818 wait complete. 2024-02-02T13:36:58.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.450+0000 7f5048200640 1 Processor -- start 2024-02-02T13:36:58.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 -- start start 2024-02-02T13:36:58.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5020003970 0x7f50401530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 0x7f50400b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c28e0 con 0x7f50400c7fb0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040153600 con 0x7f504015b3b0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.451+0000 7f5048200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504015b850 con 0x7f5020003970 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60872/0 (socket says 172.21.15.91:60872) 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 -- 172.21.15.91:0/3585294731 learned_addr learned my addr 172.21.15.91:0/3585294731 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5046200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 0x7f50400b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5045800640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5020003970 0x7f50401530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 -- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5020003970 msgr2=0x7f50401530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5020003970 0x7f50401530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 -- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 msgr2=0x7f50400b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 0x7f50400b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5044e00640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504015bb30 con 0x7f50400c7fb0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.452+0000 7f5046200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 0x7f50400b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f5044e00640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f502800e7a0 tx=0x7f502800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800eeb0 con 0x7f50400c7fb0 2024-02-02T13:36:58.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800c940 con 0x7f50400c7fb0 2024-02-02T13:36:58.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800cd10 con 0x7f50400c7fb0 2024-02-02T13:36:58.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504015be10 con 0x7f50400c7fb0 2024-02-02T13:36:58.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.453+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504015c3d0 con 0x7f50400c7fb0 2024-02-02T13:36:58.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.454+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040066cf0 con 0x7f50400c7fb0 2024-02-02T13:36:58.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.455+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5028013470 con 0x7f50400c7fb0 2024-02-02T13:36:58.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.456+0000 7f503d600640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 0x7f501806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.456+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f50280889a0 con 0x7f50400c7fb0 2024-02-02T13:36:58.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.456+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f5018071ff0 con 0x7f50400c7fb0 2024-02-02T13:36:58.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.456+0000 7f5045800640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 0x7f501806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.457+0000 7f5045800640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 0x7f501806c920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5030036000 tx=0x7f503002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.461+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5028017060 con 0x7f50400c7fb0 2024-02-02T13:36:58.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.690+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f50400c9a60 con 0x7f50400c7fb0 2024-02-02T13:36:58.693 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:36:58.692+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:36:58.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.694+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(183..183 src has 1..183) v4 ==== 374+0+0 (secure 0 0 0) 0x7f502800ce70 con 0x7f50400c7fb0 2024-02-02T13:36:58.695 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.694+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f50180728a0 con 0x7f50400c7fb0 2024-02-02T13:36:58.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.711+0000 7f503d600640 1 -- 172.21.15.91:0/3585294731 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5028021020 con 0x7f50400c7fb0 2024-02-02T13:36:58.713 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 msgr2=0x7f501806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 0x7f501806c920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5030036000 tx=0x7f503002e000 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 msgr2=0x7f504015ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f502800e7a0 tx=0x7f502800ec60 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 shutdown_connections 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f501806a460 0x7f501806c920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f504015b3b0 0x7f50400b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50400c7fb0 0x7f504015ae70 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 --2- 172.21.15.91:0/3585294731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5020003970 0x7f50401530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 >> 172.21.15.91:0/3585294731 conn(0x7f50400b0700 msgr2=0x7f50400cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 shutdown_connections 2024-02-02T13:36:58.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.715+0000 7f5048200640 1 -- 172.21.15.91:0/3585294731 wait complete. 2024-02-02T13:36:58.738 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:36:58.793 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:36:58.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.914+0000 7fcd19c00640 1 Processor -- start 2024-02-02T13:36:58.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.914+0000 7fcd19c00640 1 -- start start 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd19c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd1406a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd19c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd140b1300 0x7fcd140ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd19c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406ae40 con 0x7fcd140cf8c0 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd19c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd140b1790 con 0x7fcd140b1300 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd19c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1415d6e0 con 0x7fcd140cf4e0 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd13400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd140b1300 0x7fcd140ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd13400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd140b1300 0x7fcd140ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60906/0 (socket says 172.21.15.91:60906) 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.915+0000 7fcd13400640 1 -- 172.21.15.91:0/398142159 learned_addr learned my addr 172.21.15.91:0/398142159 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd12a00640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd1406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd13400640 1 -- 172.21.15.91:0/398142159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 msgr2=0x7fcd1406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd13400640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd1406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd13400640 1 -- 172.21.15.91:0/398142159 >> v1:172.21.15.113:6789/0 conn(0x7fcd140cf8c0 legacy=0x7fcd140adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:58.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd13400640 1 -- 172.21.15.91:0/398142159 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140b19a0 con 0x7fcd140b1300 2024-02-02T13:36:58.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.916+0000 7fcd12a00640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd1406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.917+0000 7fcd13400640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ae270 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009870 tx=0x7fcd0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c4812522e4a34b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.917+0000 7fcd11600640 1 -- 172.21.15.91:0/398142159 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c031640 con 0x7fcd140b1300 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.917+0000 7fcd11600640 1 -- 172.21.15.91:0/398142159 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c0317a0 con 0x7fcd140b1300 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.917+0000 7fcd11600640 1 -- 172.21.15.91:0/398142159 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c031b30 con 0x7fcd140b1300 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 -- 172.21.15.91:0/398142159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 msgr2=0x7fcd140ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ae270 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009870 tx=0x7fcd0c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 -- 172.21.15.91:0/398142159 shutdown_connections 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ae270 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 --2- 172.21.15.91:0/398142159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd1406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 -- 172.21.15.91:0/398142159 >> 172.21.15.91:0/398142159 conn(0x7fcd140d0ea0 msgr2=0x7fcd140d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:58.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 -- 172.21.15.91:0/398142159 shutdown_connections 2024-02-02T13:36:58.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.918+0000 7fcd19c00640 1 -- 172.21.15.91:0/398142159 wait complete. 2024-02-02T13:36:58.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.920+0000 7fcd19c00640 1 Processor -- start 2024-02-02T13:36:58.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.920+0000 7fcd19c00640 1 -- start start 2024-02-02T13:36:58.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd140c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd140cf8c0 0x7fcd141f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c4b30 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c6f10 con 0x7fcd140cf8c0 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.921+0000 7fcd19c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c71f0 con 0x7fcd140cf4e0 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60918/0 (socket says 172.21.15.91:60918) 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 -- 172.21.15.91:0/1234837094 learned_addr learned my addr 172.21.15.91:0/1234837094 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 -- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 msgr2=0x7fcd140c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd140c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 -- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd140cf8c0 msgr2=0x7fcd141f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd140cf8c0 0x7fcd141f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0094d0 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.922+0000 7fcd12a00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c7f0 tx=0x7fcd0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.923+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007a30 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.923+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd04007b90 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.923+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400ce40 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.923+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd140c74d0 con 0x7fcd140b1300 2024-02-02T13:36:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.923+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd140c4e10 con 0x7fcd140b1300 2024-02-02T13:36:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.924+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd140d07d0 con 0x7fcd140b1300 2024-02-02T13:36:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.925+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd040113f0 con 0x7fcd140b1300 2024-02-02T13:36:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.926+0000 7fcd0aa00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 0x7fccec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.926+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fcd04088c20 con 0x7fcd140b1300 2024-02-02T13:36:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.926+0000 7fcd12000640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 0x7fccec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.926+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7fccec0720d0 con 0x7fcd140b1300 2024-02-02T13:36:58.928 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.927+0000 7fcd12000640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 0x7fccec06c8f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcd140b2f50 tx=0x7fccfc009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:58.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:58.930+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcd0401f060 con 0x7fcd140b1300 2024-02-02T13:36:59.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.139+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcd14066cf0 con 0x7fcd140b1300 2024-02-02T13:36:59.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.536+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(184..184 src has 1..184) v4 ==== 679+0+0 (secure 0 0 0) 0x7fcd040584e0 con 0x7fcd140b1300 2024-02-02T13:36:59.539 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.536+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7fccec072c60 con 0x7fcd140b1300 2024-02-02T13:36:59.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.715+0000 7fcd0aa00640 1 -- 172.21.15.91:0/1234837094 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd0401f0a0 con 0x7fcd140b1300 2024-02-02T13:36:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.719+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 msgr2=0x7fccec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.719+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 0x7fccec06c8f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcd140b2f50 tx=0x7fccfc009230 comp rx=0 tx=0).stop 2024-02-02T13:36:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.719+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 msgr2=0x7fcd140ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.719+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c7f0 tx=0x7fcd0400cbf0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 shutdown_connections 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fccec06a430 0x7fccec06c8f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd140cf4e0 0x7fcd140c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd140cf8c0 0x7fcd141f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 --2- 172.21.15.91:0/1234837094 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd140b1300 0x7fcd140ce010 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 >> 172.21.15.91:0/1234837094 conn(0x7fcd140d0ea0 msgr2=0x7fcd140b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 shutdown_connections 2024-02-02T13:36:59.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.720+0000 7fcd19c00640 1 -- 172.21.15.91:0/1234837094 wait complete. 2024-02-02T13:36:59.742 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:36:59.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.939+0000 7f0ea9200640 1 Processor -- start 2024-02-02T13:36:59.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.939+0000 7f0ea9200640 1 -- start start 2024-02-02T13:36:59.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.940+0000 7f0ea9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 0x7f0ea40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.940+0000 7f0ea9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ea40b0c40 0x7f0ea40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.940+0000 7f0ea9200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea415fa00 con 0x7f0ea4151f90 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.940+0000 7f0ea9200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40b10d0 con 0x7f0ea40b0c40 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.940+0000 7f0ea9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4070f80 con 0x7f0ea415f5a0 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f0ea4151f90 0x7f0ea406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41586/0 (socket says 172.21.15.91:41586) 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1e00640 1 -- 172.21.15.91:0/3265271514 learned_addr learned my addr 172.21.15.91:0/3265271514 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:36:59.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338167405 0 0) 0x7f0e94001110 con 0x7f0ea4151f90 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e84003520 con 0x7f0ea4151f90 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0e9be00640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 0x7f0ea40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea2800640 1 --2- 172.21.15.91:0/3265271514 >> v2:172.21.15.113:3300/0 conn(0x7f0ea40b0c40 0x7f0ea40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993611680 0 0) 0x7f0e84003520 con 0x7f0ea4151f90 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 msgr2=0x7f0ea40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:59.949 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 0x7f0ea40b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 >> v2:172.21.15.113:3300/0 conn(0x7f0ea40b0c40 msgr2=0x7f0ea40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 --2- 172.21.15.91:0/3265271514 >> v2:172.21.15.113:3300/0 conn(0x7f0ea40b0c40 0x7f0ea40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4152370 con 0x7f0ea4151f90 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f0e94001800 con 0x7f0ea4151f90 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 >> v1:172.21.15.91:6789/0 conn(0x7f0ea4151f90 legacy=0x7f0ea406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.941+0000 7f0ea1400640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e84003970 0x7f0e84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1400640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 0x7f0e840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1400640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e84004d00 0x7f0e84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40b10d0 con 0x7f0e84003970 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4070f80 con 0x7f0e84004d00 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8400ba60 con 0x7f0ea415f5a0 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e84003970 0x7f0e84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 -- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 msgr2=0x7f0e840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea415f5a0 0x7f0e840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 -- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e84004d00 msgr2=0x7f0e84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e84004d00 0x7f0e84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.942+0000 7f0ea1e00640 1 -- 172.21.15.91:0/3265271514 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84003520 con 0x7f0e84003970 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.943+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e84003970 0x7f0e84003d90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c210 tx=0x7f0e840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3ddfa43d5549b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:59.952 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.943+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94005ad0 con 0x7f0e84003970 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.943+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e94005c30 con 0x7f0e84003970 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.943+0000 7f0ea1400640 1 -- 172.21.15.91:0/3265271514 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9402c8c0 con 0x7f0e84003970 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 -- 172.21.15.91:0/3265271514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e84003970 msgr2=0x7f0e84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 --2- 172.21.15.91:0/3265271514 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e84003970 0x7f0e84003d90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c210 tx=0x7f0e840057f0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea2800640 1 -- 172.21.15.91:0/3265271514 reap_dead start 2024-02-02T13:36:59.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 -- 172.21.15.91:0/3265271514 shutdown_connections 2024-02-02T13:36:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 -- 172.21.15.91:0/3265271514 >> 172.21.15.91:0/3265271514 conn(0x7f0ea40cec80 msgr2=0x7f0ea40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:36:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 -- 172.21.15.91:0/3265271514 shutdown_connections 2024-02-02T13:36:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.944+0000 7f0ea9200640 1 -- 172.21.15.91:0/3265271514 wait complete. 2024-02-02T13:36:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.946+0000 7f0ea9200640 1 Processor -- start 2024-02-02T13:36:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.946+0000 7f0ea9200640 1 -- start start 2024-02-02T13:36:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e84003970 0x7f0ea415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4159980 0x7f0ea4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea414db40 con 0x7f0ea4148ad0 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea415e400 con 0x7f0ea4159980 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0ea9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea414d210 con 0x7f0e84003970 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52970/0 (socket says 172.21.15.91:52970) 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 -- 172.21.15.91:0/693572972 learned_addr learned my addr 172.21.15.91:0/693572972 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:36:59.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 -- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e84003970 msgr2=0x7f0ea415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e84003970 0x7f0ea415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 -- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4159980 msgr2=0x7f0ea4159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4159980 0x7f0ea4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.947+0000 7f0e9be00640 1 -- 172.21.15.91:0/693572972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea414d4f0 con 0x7f0ea4148ad0 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0e9be00640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00d5f0 tx=0x7f0e8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:59.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0125f0 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c012750 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea414d7d0 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c012b00 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.948+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea40c5cc0 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.949+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4155990 con 0x7f0ea4148ad0 2024-02-02T13:36:59.961 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.950+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0e8c012c60 con 0x7f0ea4148ad0 2024-02-02T13:36:59.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.951+0000 7f0e9a000640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 0x7f0e7c06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:36:59.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.951+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0e8c088dc0 con 0x7f0ea4148ad0 2024-02-02T13:36:59.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.951+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 0x7f0e7c06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:36:59.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.951+0000 7f0ea1e00640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 0x7f0e7c06c8b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0e94034000 tx=0x7f0e9402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:36:59.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:36:59.955+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0e8c0543d0 con 0x7f0ea4148ad0 2024-02-02T13:37:00.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.146+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ea4066cf0 con 0x7f0ea4148ad0 2024-02-02T13:37:00.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.738+0000 7f0e9a000640 1 -- 172.21.15.91:0/693572972 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0e8c0579f0 con 0x7f0ea4148ad0 2024-02-02T13:37:00.742 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:37:00.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 msgr2=0x7f0e7c06c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:00.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 0x7f0e7c06c8b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0e94034000 tx=0x7f0e9402e000 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 msgr2=0x7f0ea414f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00d5f0 tx=0x7f0e8c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 shutdown_connections 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f0e7c06a3f0 0x7f0e7c06c8b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4159980 0x7f0ea4159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ea4148ad0 0x7f0ea414f570 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 --2- 172.21.15.91:0/693572972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e84003970 0x7f0ea415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.746+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 >> 172.21.15.91:0/693572972 conn(0x7f0ea40cec80 msgr2=0x7f0ea40cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:00.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.747+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 shutdown_connections 2024-02-02T13:37:00.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.747+0000 7f0ea9200640 1 -- 172.21.15.91:0/693572972 wait complete. 2024-02-02T13:37:00.770 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:37:00.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.976+0000 7feb79400640 1 Processor -- start 2024-02-02T13:37:00.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.976+0000 7feb79400640 1 -- start start 2024-02-02T13:37:00.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb79400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb740ce130 0x7feb740c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb79400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740b0700 0x7feb740b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb79400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740cc7c0 con 0x7feb7415a010 2024-02-02T13:37:00.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb79400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb740b10d0 con 0x7feb740ce130 2024-02-02T13:37:00.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb79400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb740ce930 con 0x7feb740b0700 2024-02-02T13:37:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb71e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb740ce130 0x7feb740c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb71e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feb740ce130 0x7feb740c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52986/0 (socket says 172.21.15.91:52986) 2024-02-02T13:37:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb71e00640 1 -- 172.21.15.91:0/1451796913 learned_addr learned my addr 172.21.15.91:0/1451796913 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.978+0000 7feb73200640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740b0700 0x7feb740b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.979+0000 7feb71e00640 1 -- 172.21.15.91:0/1451796913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740b0700 msgr2=0x7feb740b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:00.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.979+0000 7feb71e00640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740b0700 0x7feb740b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.979+0000 7feb71e00640 1 -- 172.21.15.91:0/1451796913 >> v1:172.21.15.113:6789/0 conn(0x7feb7415a010 legacy=0x7feb740ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:00.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.979+0000 7feb71e00640 1 -- 172.21.15.91:0/1451796913 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740cd640 con 0x7feb740ce130 2024-02-02T13:37:00.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.980+0000 7feb71e00640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb740ce130 0x7feb740c9670 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009870 tx=0x7feb5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9361180d49a77d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.980+0000 7feb71400640 1 -- 172.21.15.91:0/1451796913 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c031640 con 0x7feb740ce130 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb71400640 1 -- 172.21.15.91:0/1451796913 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c0317a0 con 0x7feb740ce130 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb71400640 1 -- 172.21.15.91:0/1451796913 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c031b30 con 0x7feb740ce130 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 -- 172.21.15.91:0/1451796913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb740ce130 msgr2=0x7feb740c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb740ce130 0x7feb740c9670 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009870 tx=0x7feb5c02b8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 -- 172.21.15.91:0/1451796913 shutdown_connections 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740b0700 0x7feb740b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 --2- 172.21.15.91:0/1451796913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb740ce130 0x7feb740c9670 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 -- 172.21.15.91:0/1451796913 >> 172.21.15.91:0/1451796913 conn(0x7feb740cec80 msgr2=0x7feb740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 -- 172.21.15.91:0/1451796913 shutdown_connections 2024-02-02T13:37:00.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.981+0000 7feb79400640 1 -- 172.21.15.91:0/1451796913 wait complete. 2024-02-02T13:37:00.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.983+0000 7feb79400640 1 Processor -- start 2024-02-02T13:37:00.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.983+0000 7feb79400640 1 -- start start 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 0x7feb740c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740ce130 0x7feb74156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74157bc0 con 0x7feb7415a010 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb740c2d40 con 0x7feb740b0700 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb79400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb740c5ab0 con 0x7feb740ce130 2024-02-02T13:37:00.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb73200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb73200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:52998/0 (socket says 172.21.15.91:52998) 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.984+0000 7feb73200640 1 -- 172.21.15.91:0/1975432345 learned_addr learned my addr 172.21.15.91:0/1975432345 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb72800640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 0x7feb740c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb71e00640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740ce130 0x7feb74156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 -- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740ce130 msgr2=0x7feb74156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740ce130 0x7feb74156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 -- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 msgr2=0x7feb740c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 0x7feb740c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 -- 172.21.15.91:0/1975432345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0094d0 con 0x7feb7415a010 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb72800640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 0x7feb740c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb73200640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7feb6c007d90 tx=0x7feb6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.985+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c014420 con 0x7feb7415a010 2024-02-02T13:37:00.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.986+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c0145a0 con 0x7feb7415a010 2024-02-02T13:37:00.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.986+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb740c35e0 con 0x7feb7415a010 2024-02-02T13:37:00.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.986+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c014950 con 0x7feb7415a010 2024-02-02T13:37:00.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.986+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb740c2f90 con 0x7feb7415a010 2024-02-02T13:37:00.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.987+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74066cf0 con 0x7feb7415a010 2024-02-02T13:37:00.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.988+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feb6c014ab0 con 0x7feb7415a010 2024-02-02T13:37:00.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.988+0000 7feb6aa00640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 0x7feb4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:00.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.989+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5386+0+0 (secure 0 0 0) 0x7feb6c0890c0 con 0x7feb7415a010 2024-02-02T13:37:00.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.989+0000 7feb72800640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 0x7feb4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:00.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.989+0000 7feb72800640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 0x7feb4c06c8f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb6400a830 tx=0x7feb64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:00.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:00.993+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feb6c0547e0 con 0x7feb7415a010 2024-02-02T13:37:01.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.186+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb74155320 con 0x7feb7415a010 2024-02-02T13:37:01.741 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.740+0000 7feb6aa00640 1 -- 172.21.15.91:0/1975432345 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) v1 ==== 169+0+0 (secure 0 0 0) 0x7feb6c057e00 con 0x7feb7415a010 2024-02-02T13:37:01.742 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:37:01.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.745+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 msgr2=0x7feb4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.745+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 0x7feb4c06c8f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb6400a830 tx=0x7feb64008040 comp rx=0 tx=0).stop 2024-02-02T13:37:01.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.745+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 msgr2=0x7feb740aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.745+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7feb6c007d90 tx=0x7feb6c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 shutdown_connections 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7feb4c06a430 0x7feb4c06c8f0 secure :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb6400a830 tx=0x7feb64008040 comp rx=0 tx=0).stop 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb740ce130 0x7feb74156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feb7415a010 0x7feb740aecc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 --2- 172.21.15.91:0/1975432345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb740b0700 0x7feb740c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.746+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 >> 172.21.15.91:0/1975432345 conn(0x7feb740cec80 msgr2=0x7feb74070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.747+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 shutdown_connections 2024-02-02T13:37:01.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.747+0000 7feb79400640 1 -- 172.21.15.91:0/1975432345 wait complete. 2024-02-02T13:37:01.773 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:37:01.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.976+0000 7fc8d8200640 1 Processor -- start 2024-02-02T13:37:01.978 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.977+0000 7fc8d8200640 1 -- start start 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.978+0000 7fc8d8200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d015e640 0x7fc8d015f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.978+0000 7fc8d8200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d015fc70 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.978+0000 7fc8d8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d0070450 con 0x7fc8d006cbb0 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.978+0000 7fc8d8200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00b50c0 con 0x7fc8d015e640 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.978+0000 7fc8d6200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc8d00cc8d0 0x7fc8d00cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41594/0 (socket says 172.21.15.91:41594) 2024-02-02T13:37:01.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8d6200640 1 -- 172.21.15.91:0/3947000724 learned_addr learned my addr 172.21.15.91:0/3947000724 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994712056 0 0) 0x7fc8d015fc70 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b0003520 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8d5800640 1 --2- 172.21.15.91:0/3947000724 >> v2:172.21.15.113:3300/0 conn(0x7fc8d015e640 0x7fc8d015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 995189099 0 0) 0x7fc8b0003520 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 >> v2:172.21.15.113:3300/0 conn(0x7fc8d015e640 msgr2=0x7fc8d015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:01.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 --2- 172.21.15.91:0/3947000724 >> v2:172.21.15.113:3300/0 conn(0x7fc8d015e640 0x7fc8d015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 >> v1:172.21.15.113:6789/0 conn(0x7fc8d006cbb0 legacy=0x7fc8d00cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00718c0 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc8c8001690 con 0x7fc8d00cc8d0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 >> v1:172.21.15.91:6789/0 conn(0x7fc8d00cc8d0 legacy=0x7fc8d00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.979+0000 7fc8cfe00640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00039c0 0x7fc8b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8cfe00640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00047c0 0x7fc8b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8cfe00640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 0x7fc8b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00b50c0 con 0x7fc8b0005250 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b000c1a0 con 0x7fc8b00047c0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b000c360 con 0x7fc8b00039c0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 0x7fc8b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d4e00640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00047c0 0x7fc8b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d5800640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00039c0 0x7fc8b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 -- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00039c0 msgr2=0x7fc8b0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00039c0 0x7fc8b0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 -- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00047c0 msgr2=0x7fc8b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:01.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00047c0 0x7fc8b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.980+0000 7fc8d6200640 1 -- 172.21.15.91:0/3947000724 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b0003520 con 0x7fc8b0005250 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.981+0000 7fc8d6200640 1 -- 172.21.15.91:0/3947000724 reap_dead start 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.981+0000 7fc8d6200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 0x7fc8b000b980 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c160 tx=0x7fc8b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4902431342fb6dfa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8005ad0 con 0x7fc8b0005250 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8005c50 con 0x7fc8b0005250 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8cfe00640 1 -- 172.21.15.91:0/3947000724 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c802c7d0 con 0x7fc8b0005250 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 -- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 msgr2=0x7fc8b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 0x7fc8b000b980 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c160 tx=0x7fc8b00057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 -- 172.21.15.91:0/3947000724 shutdown_connections 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 --2- 172.21.15.91:0/3947000724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8b0005250 0x7fc8b000b980 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 -- 172.21.15.91:0/3947000724 >> 172.21.15.91:0/3947000724 conn(0x7fc8d0069e30 msgr2=0x7fc8d006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:01.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.982+0000 7fc8d8200640 1 -- 172.21.15.91:0/3947000724 shutdown_connections 2024-02-02T13:37:01.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.983+0000 7fc8d8200640 1 -- 172.21.15.91:0/3947000724 wait complete. 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.984+0000 7fc8d8200640 1 Processor -- start 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.984+0000 7fc8d8200640 1 -- start start 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0005250 0x7fc8d00c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d01f0f80 0x7fc8d00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00c4c30 con 0x7fc8d01efe90 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d01f0860 con 0x7fc8d01f0f80 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d8200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00c4e40 con 0x7fc8b0005250 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.985+0000 7fc8d6200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53014/0 (socket says 172.21.15.91:53014) 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 -- 172.21.15.91:0/2638006473 learned_addr learned my addr 172.21.15.91:0/2638006473 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d4e00640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d01f0f80 0x7fc8d00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d5800640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0005250 0x7fc8d00c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 -- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0005250 msgr2=0x7fc8d00c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0005250 0x7fc8d00c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 -- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d01f0f80 msgr2=0x7fc8d00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d01f0f80 0x7fc8d00cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:01.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 -- 172.21.15.91:0/2638006473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00c5120 con 0x7fc8d01efe90 2024-02-02T13:37:01.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8d6200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c280 tx=0x7fc8c8001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:01.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.986+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c80049e0 con 0x7fc8d01efe90 2024-02-02T13:37:01.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.987+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8004b40 con 0x7fc8d01efe90 2024-02-02T13:37:01.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.987+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c802de60 con 0x7fc8d01efe90 2024-02-02T13:37:01.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.987+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d00c5400 con 0x7fc8d01efe90 2024-02-02T13:37:01.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.987+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d01f1400 con 0x7fc8d01efe90 2024-02-02T13:37:01.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.988+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d00c5920 con 0x7fc8d01efe90 2024-02-02T13:37:01.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.989+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc8c8032070 con 0x7fc8d01efe90 2024-02-02T13:37:01.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.990+0000 7fc8cd600640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 0x7fc8a806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:01.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.990+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8c803c070 con 0x7fc8d01efe90 2024-02-02T13:37:01.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.990+0000 7fc8d5800640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 0x7fc8a806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:01.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.991+0000 7fc8d5800640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 0x7fc8a806c920 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:01.995+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8d00c5920 con 0x7fc8d01efe90 2024-02-02T13:37:02.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.188+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc8d00c5920 con 0x7fc8d01efe90 2024-02-02T13:37:02.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.194+0000 7fc8cd600640 1 -- 172.21.15.91:0/2638006473 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc8d00c5920 con 0x7fc8d01efe90 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 msgr2=0x7fc8a806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 0x7fc8a806c920 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0008040 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 msgr2=0x7fc8d01f0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c280 tx=0x7fc8c8001950 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 shutdown_connections 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc8a806a460 0x7fc8a806c920 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8d01efe90 0x7fc8d01f0320 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d01f0f80 0x7fc8d00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 --2- 172.21.15.91:0/2638006473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0005250 0x7fc8d00c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 >> 172.21.15.91:0/2638006473 conn(0x7fc8d0069e30 msgr2=0x7fc8d00b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:02.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 shutdown_connections 2024-02-02T13:37:02.200 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.197+0000 7fc8d8200640 1 -- 172.21.15.91:0/2638006473 wait complete. 2024-02-02T13:37:02.217 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-02-02T13:37:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.428+0000 7fcb41a00640 1 Processor -- start 2024-02-02T13:37:02.430 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.428+0000 7fcb41a00640 1 -- start start 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb41a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb3c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb41a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0b3f60 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb41a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0d0010 con 0x7fcb3c15e9a0 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb41a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0ccfe0 con 0x7fcb3c0b5a10 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb31e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcb3c0cbb80 0x7fcb3c0cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41610/0 (socket says 172.21.15.91:41610) 2024-02-02T13:37:02.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb31e00640 1 -- 172.21.15.91:0/1361414982 learned_addr learned my addr 172.21.15.91:0/1361414982 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb3c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795824175 0 0) 0x7fcb3c0b3f60 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.430+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb1c003520 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763303042 0 0) 0x7fcb1c003520 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 msgr2=0x7fcb3c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb3c0cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 >> v1:172.21.15.113:6789/0 conn(0x7fcb3c15e9a0 legacy=0x7fcb3c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0cfd30 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fcb24001670 con 0x7fcb3c0cbb80 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 >> v1:172.21.15.91:6789/0 conn(0x7fcb3c0cbb80 legacy=0x7fcb3c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb1c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb1c003fd0 0x7fcb1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb1c004d00 0x7fcb1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0ccfe0 con 0x7fcb1c003fd0 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c00bb40 con 0x7fcb1c004d00 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c00bd00 con 0x7fcb3c0b5a10 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.431+0000 7fcb31e00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb1c003fd0 0x7fcb1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb3aa00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb1c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb1c004d00 0x7fcb1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb31e00640 1 -- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 msgr2=0x7fcb1c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb31e00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb3c0b5a10 0x7fcb1c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb31e00640 1 -- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb1c004d00 msgr2=0x7fcb1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb31e00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb1c004d00 0x7fcb1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb31e00640 1 -- 172.21.15.91:0/1361414982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c003520 con 0x7fcb1c003fd0 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.432+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb1c004d00 0x7fcb1c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:02.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.433+0000 7fcb31e00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb1c003fd0 0x7fcb1c0047c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c240 tx=0x7fcb1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d51721ae93fae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.433+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24005ad0 con 0x7fcb1c003fd0 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.433+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24005c30 con 0x7fcb1c003fd0 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.433+0000 7fcb3a000640 1 -- 172.21.15.91:0/1361414982 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2402caf0 con 0x7fcb1c003fd0 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 -- 172.21.15.91:0/1361414982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb1c003fd0 msgr2=0x7fcb1c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1361414982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb1c003fd0 0x7fcb1c0047c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c240 tx=0x7fcb1c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb3b400640 1 -- 172.21.15.91:0/1361414982 reap_dead start 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 -- 172.21.15.91:0/1361414982 shutdown_connections 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 -- 172.21.15.91:0/1361414982 >> 172.21.15.91:0/1361414982 conn(0x7fcb3c0d0f70 msgr2=0x7fcb3c0d1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 -- 172.21.15.91:0/1361414982 shutdown_connections 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.434+0000 7fcb41a00640 1 -- 172.21.15.91:0/1361414982 wait complete. 2024-02-02T13:37:02.437 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.436+0000 7fcb41a00640 1 Processor -- start 2024-02-02T13:37:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.436+0000 7fcb41a00640 1 -- start start 2024-02-02T13:37:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb1c003fd0 0x7fcb3c0c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb3c15dee0 0x7fcb3c0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0b5240 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c151d50 con 0x7fcb3c15dee0 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb41a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0c59b0 con 0x7fcb1c003fd0 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53034/0 (socket says 172.21.15.91:53034) 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 -- 172.21.15.91:0/1152728102 learned_addr learned my addr 172.21.15.91:0/1152728102 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb31e00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb3c15dee0 0x7fcb3c0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3aa00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb1c003fd0 0x7fcb3c0c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 -- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb1c003fd0 msgr2=0x7fcb3c0c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb1c003fd0 0x7fcb3c0c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 -- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb3c15dee0 msgr2=0x7fcb3c0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb3c15dee0 0x7fcb3c0b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.437+0000 7fcb3b400640 1 -- 172.21.15.91:0/1152728102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0c5c80 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb3b400640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0b5ea0 tx=0x7fcb34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007cc0 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34007e40 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34014640 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c0c5f60 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.438+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c0c6520 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.439+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c0b2860 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.440+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcb3401d3b0 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.441+0000 7fcb33400640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 0x7fcb1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.441+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb34088520 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.442+0000 7fcb3aa00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 0x7fcb1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:02.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.442+0000 7fcb3aa00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 0x7fcb1406c8f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001640 tx=0x7fcb2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:02.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.446+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcb34053d70 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.690 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:37:02.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:37:02.707 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.705+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fcb3c066cf0 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.752+0000 7fcb33400640 1 -- 172.21.15.91:0/1152728102 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v586) v1 ==== 131+0+0 (secure 0 0 0) 0x7fcb34057390 con 0x7fcb3c0b4d90 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 msgr2=0x7fcb1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 0x7fcb1406c8f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001640 tx=0x7fcb2c009040 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 msgr2=0x7fcb3c0c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0b5ea0 tx=0x7fcb34009da0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 shutdown_connections 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcb1406a430 0x7fcb1406c8f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb3c0b4d90 0x7fcb3c0c9090 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb3c15dee0 0x7fcb3c0b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 --2- 172.21.15.91:0/1152728102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb1c003fd0 0x7fcb3c0c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 >> 172.21.15.91:0/1152728102 conn(0x7fcb3c0d0f70 msgr2=0x7fcb3c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.755+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 shutdown_connections 2024-02-02T13:37:02.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:02.756+0000 7fcb41a00640 1 -- 172.21.15.91:0/1152728102 wait complete. 2024-02-02T13:37:02.774 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-02-02T13:37:02.774 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-02-02T13:37:02.774 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-02-02T13:37:03.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.003+0000 7f95e0a00640 1 Processor -- start 2024-02-02T13:37:03.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.003+0000 7f95e0a00640 1 -- start start 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 0x7f95d815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95d806f4f0 0x7f95d80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d80b0c40 0x7f95d80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d80b10b0 con 0x7f95d806f4f0 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d8070f80 con 0x7f95d8151f90 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.004+0000 7f95e0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d8152370 con 0x7f95d80b0c40 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95d806f4f0 0x7f95d80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95d806f4f0 0x7f95d80b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53042/0 (socket says 172.21.15.91:53042) 2024-02-02T13:37:03.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 -- 172.21.15.91:0/55952393 learned_addr learned my addr 172.21.15.91:0/55952393 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:03.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dea00640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d80b0c40 0x7f95d80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95de000640 1 --2- 172.21.15.91:0/55952393 >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 0x7f95d815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 -- 172.21.15.91:0/55952393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d80b0c40 msgr2=0x7f95d80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:03.007 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d80b0c40 0x7f95d80b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 -- 172.21.15.91:0/55952393 >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 msgr2=0x7f95d815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:03.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 --2- 172.21.15.91:0/55952393 >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 0x7f95d815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95dd600640 1 -- 172.21.15.91:0/55952393 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80cb010 con 0x7f95d806f4f0 2024-02-02T13:37:03.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.005+0000 7f95de000640 1 --2- 172.21.15.91:0/55952393 >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 0x7f95d815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:03.008 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.006+0000 7f95dd600640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80b0700 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f95d80cc770 tx=0x7f95c002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5392740599b5b669 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95dcc00640 1 -- 172.21.15.91:0/55952393 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0031860 con 0x7f95d806f4f0 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95dcc00640 1 -- 172.21.15.91:0/55952393 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c00319c0 con 0x7f95d806f4f0 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95dcc00640 1 -- 172.21.15.91:0/55952393 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0031d50 con 0x7f95d806f4f0 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 -- 172.21.15.91:0/55952393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 msgr2=0x7f95d80b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80b0700 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f95d80cc770 tx=0x7f95c002b900 comp rx=0 tx=0).stop 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 -- 172.21.15.91:0/55952393 shutdown_connections 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d80b0c40 0x7f95d80b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 --2- 172.21.15.91:0/55952393 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80b0700 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 --2- 172.21.15.91:0/55952393 >> v2:172.21.15.113:3300/0 conn(0x7f95d8151f90 0x7f95d815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 -- 172.21.15.91:0/55952393 >> 172.21.15.91:0/55952393 conn(0x7f95d80cec80 msgr2=0x7f95d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:03.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 -- 172.21.15.91:0/55952393 shutdown_connections 2024-02-02T13:37:03.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.007+0000 7f95e0a00640 1 -- 172.21.15.91:0/55952393 wait complete. 2024-02-02T13:37:03.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.009+0000 7f95e0a00640 1 Processor -- start 2024-02-02T13:37:03.010 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.009+0000 7f95e0a00640 1 -- start start 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 0x7f95d80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d8151f90 0x7f95d80cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80ce390 con 0x7f95d806f4f0 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80ccb50 con 0x7f95d80b0c40 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95e0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80cce30 con 0x7f95d8151f90 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53058/0 (socket says 172.21.15.91:53058) 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 -- 172.21.15.91:0/300433444 learned_addr learned my addr 172.21.15.91:0/300433444 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95dd600640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 0x7f95d80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.012 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95dea00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d8151f90 0x7f95d80cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 -- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d8151f90 msgr2=0x7f95d80cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d8151f90 0x7f95d80cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 -- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 msgr2=0x7f95d80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.010+0000 7f95de000640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 0x7f95d80cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95de000640 1 -- 172.21.15.91:0/300433444 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00094d0 con 0x7f95d806f4f0 2024-02-02T13:37:03.013 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95dd600640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 0x7f95d80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95de000640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c7f0 tx=0x7f95c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8007a30 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8007b90 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d80cd110 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800ce40 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.011+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d814eaa0 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.012+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d80ccc90 con 0x7f95d806f4f0 2024-02-02T13:37:03.014 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.013+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f95c80113f0 con 0x7f95d806f4f0 2024-02-02T13:37:03.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.014+0000 7f95d6000640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 0x7f95b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.014+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95c8088840 con 0x7f95d806f4f0 2024-02-02T13:37:03.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.014+0000 7f95dd600640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 0x7f95b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.016 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.015+0000 7f95dd600640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 0x7f95b006c8f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f95c0036000 tx=0x7f95c003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:03.020 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.019+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f95d80ccc90 con 0x7f95d806f4f0 2024-02-02T13:37:03.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.274+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f95d80cd630 con 0x7f95d806f4f0 2024-02-02T13:37:03.759 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.757+0000 7f95d6000640 1 -- 172.21.15.91:0/300433444 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v587) v1 ==== 138+0+0 (secure 0 0 0) 0x7f95d80cd630 con 0x7f95d806f4f0 2024-02-02T13:37:03.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.760+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 msgr2=0x7f95b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.762 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.760+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 0x7f95b006c8f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f95c0036000 tx=0x7f95c003c040 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 msgr2=0x7f95d80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c7f0 tx=0x7f95c800cbf0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 shutdown_connections 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f95b006a430 0x7f95b006c8f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95d8151f90 0x7f95d80cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95d80b0c40 0x7f95d80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 --2- 172.21.15.91:0/300433444 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d806f4f0 0x7f95d80ac8d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 >> 172.21.15.91:0/300433444 conn(0x7f95d80cec80 msgr2=0x7f95d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 shutdown_connections 2024-02-02T13:37:03.763 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.761+0000 7f95e0a00640 1 -- 172.21.15.91:0/300433444 wait complete. 2024-02-02T13:37:03.781 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-02T13:37:03.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.992+0000 7fd5c1800640 1 Processor -- start 2024-02-02T13:37:03.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.992+0000 7fd5c1800640 1 -- start start 2024-02-02T13:37:03.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5c1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc15f5a0 0x7fd5bc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5c1800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:03.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5c1800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc15fa00 con 0x7fd5bc151f90 2024-02-02T13:37:03.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5c1800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc0b10d0 con 0x7fd5bc0b0c40 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5c1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc070f80 con 0x7fd5bc15f5a0 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53066/0 (socket says 172.21.15.91:53066) 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 -- 172.21.15.91:0/149067613 learned_addr learned my addr 172.21.15.91:0/149067613 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 -- 172.21.15.91:0/149067613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc15f5a0 msgr2=0x7fd5bc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 --2- 172.21.15.91:0/149067613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc15f5a0 0x7fd5bc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.994+0000 7fd5bb400640 1 -- 172.21.15.91:0/149067613 >> v1:172.21.15.113:6789/0 conn(0x7fd5bc151f90 legacy=0x7fd5bc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:03.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.995+0000 7fd5bb400640 1 -- 172.21.15.91:0/149067613 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc152370 con 0x7fd5bc0b0c40 2024-02-02T13:37:03.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5bb400640 1 --2- 172.21.15.91:0/149067613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009e50 tx=0x7fd5b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40df4084fe8dfe96 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:03.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5ba000640 1 -- 172.21.15.91:0/149067613 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b402ca70 con 0x7fd5bc0b0c40 2024-02-02T13:37:03.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5ba000640 1 -- 172.21.15.91:0/149067613 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b402cbd0 con 0x7fd5bc0b0c40 2024-02-02T13:37:03.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5ba000640 1 -- 172.21.15.91:0/149067613 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40343f0 con 0x7fd5bc0b0c40 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5c1800640 1 -- 172.21.15.91:0/149067613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc0b0c40 msgr2=0x7fd5bc0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.996+0000 7fd5c1800640 1 --2- 172.21.15.91:0/149067613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009e50 tx=0x7fd5b402bed0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 -- 172.21.15.91:0/149067613 shutdown_connections 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 --2- 172.21.15.91:0/149067613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc0b0c40 0x7fd5bc0b3af0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 --2- 172.21.15.91:0/149067613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc15f5a0 0x7fd5bc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 -- 172.21.15.91:0/149067613 >> 172.21.15.91:0/149067613 conn(0x7fd5bc0cec80 msgr2=0x7fd5bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 -- 172.21.15.91:0/149067613 shutdown_connections 2024-02-02T13:37:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.997+0000 7fd5c1800640 1 -- 172.21.15.91:0/149067613 wait complete. 2024-02-02T13:37:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.998+0000 7fd5c1800640 1 Processor -- start 2024-02-02T13:37:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 -- start start 2024-02-02T13:37:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc0b0c40 0x7fd5bc0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc151f90 0x7fd5bc1f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c6c10 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15c620 con 0x7fd5bc151f90 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:03.999+0000 7fd5c1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c8820 con 0x7fd5bc0b0c40 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53082/0 (socket says 172.21.15.91:53082) 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 -- 172.21.15.91:0/464023101 learned_addr learned my addr 172.21.15.91:0/464023101 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 -- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc0b0c40 msgr2=0x7fd5bc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc0b0c40 0x7fd5bc0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 -- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc151f90 msgr2=0x7fd5bc1f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5b3e00640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc151f90 0x7fd5bc1f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc151f90 0x7fd5bc1f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 -- 172.21.15.91:0/464023101 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4009b30 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.002 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.000+0000 7fd5bb400640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009e50 tx=0x7fd5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:04.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.001+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4009510 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.001+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b403e450 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.001+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b403e800 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.001+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc15ba10 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.003 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.001+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc15bfd0 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.002+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc066cf0 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.004 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.003+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd5b403e960 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.004+0000 7fd5b2a00640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 0x7fd59406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.004+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd5b40ab0a0 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.005 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.004+0000 7fd5baa00640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 0x7fd59406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.006 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.005+0000 7fd5baa00640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 0x7fd59406c920 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006fd0 tx=0x7fd5ac006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:04.009 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.008+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd5b4076860 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.199 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.197+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd5bc15fa00 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.205+0000 7fd5b2a00640 1 -- 172.21.15.91:0/464023101 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd5bc15fa00 con 0x7fd5bc15f5a0 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 msgr2=0x7fd59406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 0x7fd59406c920 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006fd0 tx=0x7fd5ac006bc0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 msgr2=0x7fd5bc15cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009e50 tx=0x7fd5b4004a40 comp rx=0 tx=0).stop 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 shutdown_connections 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd59406a460 0x7fd59406c920 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5bc15f5a0 0x7fd5bc15cab0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc151f90 0x7fd5bc1f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 --2- 172.21.15.91:0/464023101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc0b0c40 0x7fd5bc0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 >> 172.21.15.91:0/464023101 conn(0x7fd5bc0cec80 msgr2=0x7fd5bc0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:04.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 shutdown_connections 2024-02-02T13:37:04.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.208+0000 7fd5c1800640 1 -- 172.21.15.91:0/464023101 wait complete. 2024-02-02T13:37:04.227 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:37:04.228 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:37:04.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:37:04.453 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:37:04.453 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:04.453 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:04.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:37:04.510 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:04.510 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:04.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:37:04.569 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:04.569 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:04.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:37:04.624 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:04.624 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:04.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:37:04.679 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-02T13:37:04.679 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:04.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.891+0000 7f5200e00640 1 Processor -- start 2024-02-02T13:37:04.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.891+0000 7f5200e00640 1 -- start start 2024-02-02T13:37:04.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.892+0000 7f5200e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f51fc0cd690 0x7f51fc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.892+0000 7f5200e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51fc0c9240 0x7f51fc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.892+0000 7f5200e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc06b5f0 con 0x7f51fc1597a0 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.892+0000 7f5200e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06cff0 con 0x7f51fc0cd690 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.892+0000 7f5200e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06c450 con 0x7f51fc0c9240 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f51fc0cd690 0x7f51fc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f51fc0cd690 0x7f51fc0cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53084/0 (socket says 172.21.15.91:53084) 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 -- 172.21.15.91:0/312723570 learned_addr learned my addr 172.21.15.91:0/312723570 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51fa800640 1 --2- 172.21.15.91:0/312723570 >> v2:172.21.15.113:3300/0 conn(0x7f51fc0c9240 0x7f51fc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 -- 172.21.15.91:0/312723570 >> v2:172.21.15.113:3300/0 conn(0x7f51fc0c9240 msgr2=0x7f51fc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 --2- 172.21.15.91:0/312723570 >> v2:172.21.15.113:3300/0 conn(0x7f51fc0c9240 0x7f51fc0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 -- 172.21.15.91:0/312723570 >> v1:172.21.15.113:6789/0 conn(0x7f51fc1597a0 legacy=0x7f51fc06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.893+0000 7f51f9400640 1 -- 172.21.15.91:0/312723570 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0c9bb0 con 0x7f51fc0cd690 2024-02-02T13:37:04.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.894+0000 7f51f9400640 1 --2- 172.21.15.91:0/312723570 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0cd690 0x7f51fc0cdb20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0b11b0 tx=0x7f51e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fcd61890a83d17 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:04.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f51f8a00640 1 -- 172.21.15.91:0/312723570 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4031870 con 0x7f51fc0cd690 2024-02-02T13:37:04.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f51f8a00640 1 -- 172.21.15.91:0/312723570 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e40319d0 con 0x7f51fc0cd690 2024-02-02T13:37:04.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f51f8a00640 1 -- 172.21.15.91:0/312723570 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4031d60 con 0x7f51fc0cd690 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 -- 172.21.15.91:0/312723570 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0cd690 msgr2=0x7f51fc0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 --2- 172.21.15.91:0/312723570 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0cd690 0x7f51fc0cdb20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0b11b0 tx=0x7f51e402b910 comp rx=0 tx=0).stop 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 -- 172.21.15.91:0/312723570 shutdown_connections 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 --2- 172.21.15.91:0/312723570 >> v2:172.21.15.113:3300/0 conn(0x7f51fc0c9240 0x7f51fc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 --2- 172.21.15.91:0/312723570 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0cd690 0x7f51fc0cdb20 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.895+0000 7f5200e00640 1 -- 172.21.15.91:0/312723570 >> 172.21.15.91:0/312723570 conn(0x7f51fc0cec80 msgr2=0x7f51fc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.896+0000 7f5200e00640 1 -- 172.21.15.91:0/312723570 shutdown_connections 2024-02-02T13:37:04.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.896+0000 7f5200e00640 1 -- 172.21.15.91:0/312723570 wait complete. 2024-02-02T13:37:04.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.897+0000 7f5200e00640 1 Processor -- start 2024-02-02T13:37:04.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.897+0000 7f5200e00640 1 -- start start 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc0cd690 0x7f51fc14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51fc1597a0 0x7f51fc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc150340 con 0x7f51fc0c9240 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc158fb0 con 0x7f51fc0cd690 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.898+0000 7f5200e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc14f730 con 0x7f51fc1597a0 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53090/0 (socket says 172.21.15.91:53090) 2024-02-02T13:37:04.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 -- 172.21.15.91:0/2244038462 learned_addr learned my addr 172.21.15.91:0/2244038462 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 -- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51fc1597a0 msgr2=0x7f51fc15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51fc1597a0 0x7f51fc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9400640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc0cd690 0x7f51fc14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 -- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc0cd690 msgr2=0x7f51fc14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc0cd690 0x7f51fc14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 -- 172.21.15.91:0/2244038462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40094d0 con 0x7f51fc0c9240 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.899+0000 7f51f9e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b3f0 tx=0x7f51ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.900+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec011830 con 0x7f51fc0c9240 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.900+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec011990 con 0x7f51fc0c9240 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.900+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec011d40 con 0x7f51fc0c9240 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.900+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc14fa10 con 0x7f51fc0c9240 2024-02-02T13:37:04.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.900+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc14ffd0 con 0x7f51fc0c9240 2024-02-02T13:37:04.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.901+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc066cf0 con 0x7f51fc0c9240 2024-02-02T13:37:04.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.902+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f51ec01a400 con 0x7f51fc0c9240 2024-02-02T13:37:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.903+0000 7f51f2000640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 0x7f51d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.903+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51ec088580 con 0x7f51fc0c9240 2024-02-02T13:37:04.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.903+0000 7f51f9400640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 0x7f51d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:04.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.904+0000 7f51f9400640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 0x7f51d406c8f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f51e4036000 tx=0x7f51e403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:04.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:04.908+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f51ec053e10 con 0x7f51fc0c9240 2024-02-02T13:37:05.136 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.135+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51fc0cf2b0 con 0x7f51fc0c9240 2024-02-02T13:37:05.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.136+0000 7f51f2000640 1 -- 172.21.15.91:0/2244038462 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f51ec015de0 con 0x7f51fc0c9240 2024-02-02T13:37:05.138 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:05.138 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/225491764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":225491764},{"type":"v1","addr":"172.21.15.113:6834","nonce":225491764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":6124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4279640759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4279640759},{"type":"v1","addr":"172.21.15.113:6835","nonce":4279640759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3935339391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3935339391},{"type":"v1","addr":"172.21.15.91:6836","nonce":3935339391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8769,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3239986579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3239986579},{"type":"v1","addr":"172.21.15.91:6837","nonce":3239986579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2024-02-02T13:37:05.138 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 82 2024-02-02T13:37:05.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 msgr2=0x7f51d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 0x7f51d406c8f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f51e4036000 tx=0x7f51e403c040 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 msgr2=0x7f51fc158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b3f0 tx=0x7f51ec00b8b0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 shutdown_connections 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f51d406a430 0x7f51d406c8f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51fc1597a0 0x7f51fc15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc0cd690 0x7f51fc14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 --2- 172.21.15.91:0/2244038462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f51fc0c9240 0x7f51fc158a70 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 >> 172.21.15.91:0/2244038462 conn(0x7f51fc0cec80 msgr2=0x7f51fc15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 shutdown_connections 2024-02-02T13:37:05.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.139+0000 7f5200e00640 1 -- 172.21.15.91:0/2244038462 wait complete. 2024-02-02T13:37:05.160 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-02T13:37:05.160 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-02T13:37:05.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:37:05.160 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:37:05.160 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-02T13:37:05.160 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-02T13:37:05.161 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:37:05.161 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:37:05.161 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-02T13:37:05.161 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-02T13:37:05.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:37:05.161 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:37:05.161 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-02T13:37:05.161 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-02T13:37:05.162 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:37:05.162 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:37:05.179 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:37:05.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-02T13:37:05.180 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:37:05.180 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-02T13:37:05.186 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:37:05.187 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-02T13:37:05.192 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:37:05.192 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-02T13:37:05.233 INFO:tasks.ceph.mds.a:Started 2024-02-02T13:37:05.234 DEBUG:teuthology.parallel:result is None 2024-02-02T13:37:05.239 INFO:tasks.ceph.mds.d:Started 2024-02-02T13:37:05.239 DEBUG:teuthology.parallel:result is None 2024-02-02T13:37:05.240 INFO:tasks.ceph.mds.c:Started 2024-02-02T13:37:05.240 DEBUG:teuthology.parallel:result is None 2024-02-02T13:37:05.245 INFO:tasks.ceph.mds.b:Started 2024-02-02T13:37:05.246 DEBUG:teuthology.parallel:result is None 2024-02-02T13:37:05.246 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-02T13:37:05.389 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:37:05.397 INFO:tasks.ceph.mds.c.smithi091.stdout:starting mds.c at 2024-02-02T13:37:05.398 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-02-02T13:37:05.398 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-02-02T13:37:05.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.436+0000 7ff5a8a00640 1 Processor -- start 2024-02-02T13:37:05.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.436+0000 7ff5a8a00640 1 -- start start 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.437+0000 7ff5a8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5a00cb580 0x7ff5a00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.437+0000 7ff5a8a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5a015d3a0 0x7ff5a00cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.437+0000 7ff5a8a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a00c94a0 con 0x7ff5a00ac960 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.437+0000 7ff5a8a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a00ad500 con 0x7ff5a015d3a0 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.437+0000 7ff5a8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a00cbea0 con 0x7ff5a00cb580 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5a015d3a0 0x7ff5a00cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5a015d3a0 0x7ff5a00cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53126/0 (socket says 172.21.15.91:53126) 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a6a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff5a00ac960 0x7ff5a00acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41636/0 (socket says 172.21.15.91:41636) 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 -- 172.21.15.91:0/3499190950 learned_addr learned my addr 172.21.15.91:0/3499190950 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a6000640 1 --2- 172.21.15.91:0/3499190950 >> v2:172.21.15.113:3300/0 conn(0x7ff5a00cb580 0x7ff5a00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 -- 172.21.15.91:0/3499190950 >> v2:172.21.15.113:3300/0 conn(0x7ff5a00cb580 msgr2=0x7ff5a00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 --2- 172.21.15.91:0/3499190950 >> v2:172.21.15.113:3300/0 conn(0x7ff5a00cb580 0x7ff5a00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 -- 172.21.15.91:0/3499190950 >> v1:172.21.15.91:6789/0 conn(0x7ff5a00ac960 legacy=0x7ff5a00acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a5600640 1 -- 172.21.15.91:0/3499190950 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00ad6e0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.438+0000 7ff5a4c00640 1 -- 172.21.15.91:0/3499190950 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364831024 0 0) 0x7ff5a00c94a0 con 0x7ff5a00ac960 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.439+0000 7ff5a5600640 1 --2- 172.21.15.91:0/3499190950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00cb960 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00cc320 tx=0x7ff58802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efb831f08ddc03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a4c00640 1 -- 172.21.15.91:0/3499190950 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588030860 con 0x7ff5a015d3a0 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a4c00640 1 -- 172.21.15.91:0/3499190950 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5880309c0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a4c00640 1 -- 172.21.15.91:0/3499190950 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588030d50 con 0x7ff5a015d3a0 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3499190950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 msgr2=0x7ff5a00cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3499190950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00cb960 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00cc320 tx=0x7ff58802b9a0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3499190950 shutdown_connections 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3499190950 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00cb960 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3499190950 >> v2:172.21.15.113:3300/0 conn(0x7ff5a00cb580 0x7ff5a00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3499190950 >> 172.21.15.91:0/3499190950 conn(0x7ff5a00b0700 msgr2=0x7ff5a00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:05.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3499190950 shutdown_connections 2024-02-02T13:37:05.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.440+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3499190950 wait complete. 2024-02-02T13:37:05.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.442+0000 7ff5a8a00640 1 Processor -- start 2024-02-02T13:37:05.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.442+0000 7ff5a8a00640 1 -- start start 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5a00ac960 0x7ff5a00c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5a00cb580 0x7ff5a014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00c3b10 con 0x7ff5a015d3a0 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00c3df0 con 0x7ff5a00ac960 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00c40d0 con 0x7ff5a00cb580 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.445 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53140/0 (socket says 172.21.15.91:53140) 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a6a00640 1 -- 172.21.15.91:0/3999934338 learned_addr learned my addr 172.21.15.91:0/3999934338 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.443+0000 7ff5a6000640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5a00ac960 0x7ff5a00c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a5600640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5a00cb580 0x7ff5a014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 -- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5a00cb580 msgr2=0x7ff5a014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5a00cb580 0x7ff5a014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 -- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5a00ac960 msgr2=0x7ff5a00c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5a00ac960 0x7ff5a00c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 -- 172.21.15.91:0/3999934338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880094d0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a6a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c7e0 tx=0x7ff59800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800a640 con 0x7ff5a015d3a0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800a7c0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800ab70 con 0x7ff5a015d3a0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a00c43b0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.444+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a015f690 con 0x7ff5a015d3a0 2024-02-02T13:37:05.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.446+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a00c45a0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.447+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff59800acd0 con 0x7ff5a015d3a0 2024-02-02T13:37:05.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.448+0000 7ff59e000640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 0x7ff57806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:05.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.448+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff598087c10 con 0x7ff5a015d3a0 2024-02-02T13:37:05.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.449+0000 7ff5a6000640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 0x7ff57806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:05.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.449+0000 7ff5a6000640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 0x7ff57806c8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff590009b40 tx=0x7ff590009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:05.454 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.453+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff598053510 con 0x7ff5a015d3a0 2024-02-02T13:37:05.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:05.647+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff5a00c3c50 con 0x7ff5a015d3a0 2024-02-02T13:37:06.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.437+0000 7ff59e000640 1 -- 172.21.15.91:0/3999934338 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v191) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff5a00c3c50 con 0x7ff5a015d3a0 2024-02-02T13:37:06.439 INFO:teuthology.orchestra.run.smithi091.stderr: removed all blocklist entries 2024-02-02T13:37:06.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 msgr2=0x7ff57806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 0x7ff57806c8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff590009b40 tx=0x7ff590009190 comp rx=0 tx=0).stop 2024-02-02T13:37:06.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 msgr2=0x7ff5a00b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c7e0 tx=0x7ff59800cca0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 shutdown_connections 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff57806a430 0x7ff57806c8f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5a00cb580 0x7ff5a014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5a015d3a0 0x7ff5a00b3b30 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 --2- 172.21.15.91:0/3999934338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5a00ac960 0x7ff5a00c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 >> 172.21.15.91:0/3999934338 conn(0x7ff5a00b0700 msgr2=0x7ff5a006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 shutdown_connections 2024-02-02T13:37:06.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.441+0000 7ff5a8a00640 1 -- 172.21.15.91:0/3999934338 wait complete. 2024-02-02T13:37:06.465 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-02T13:37:06.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.674+0000 7fcc3a200640 1 Processor -- start 2024-02-02T13:37:06.675 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.674+0000 7fcc3a200640 1 -- start start 2024-02-02T13:37:06.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.675+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.675+0000 7fcc3a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc3415b0f0 0x7fcc3406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.675+0000 7fcc3a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3406d910 con 0x7fcc340cb580 2024-02-02T13:37:06.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.675+0000 7fcc3a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc340cd610 con 0x7fcc3415b0f0 2024-02-02T13:37:06.677 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.675+0000 7fcc3a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340cc850 con 0x7fcc3415ad10 2024-02-02T13:37:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.676+0000 7fcc33400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.676+0000 7fcc33400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:42836/0 (socket says 172.21.15.91:42836) 2024-02-02T13:37:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.676+0000 7fcc33400640 1 -- 172.21.15.91:0/653510683 learned_addr learned my addr 172.21.15.91:0/653510683 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.676+0000 7fcc33e00640 1 --1- 172.21.15.91:0/653510683 >> v1:172.21.15.113:6789/0 conn(0x7fcc340cb580 0x7fcc3406f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.91:49956/0 (socket says 172.21.15.91:49956) 2024-02-02T13:37:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.676+0000 7fcc2a800640 1 --2- 172.21.15.91:0/653510683 >> v2:172.21.15.113:3300/0 conn(0x7fcc3415b0f0 0x7fcc3406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc32a00640 1 -- 172.21.15.91:0/653510683 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616658632 0 0) 0x7fcc3406d910 con 0x7fcc340cb580 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc32a00640 1 -- 172.21.15.91:0/653510683 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc14003520 con 0x7fcc340cb580 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc2a800640 1 -- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 msgr2=0x7fcc3406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc2a800640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc2a800640 1 -- 172.21.15.91:0/653510683 >> v1:172.21.15.113:6789/0 conn(0x7fcc340cb580 legacy=0x7fcc3406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc2a800640 1 -- 172.21.15.91:0/653510683 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340ccb30 con 0x7fcc3415b0f0 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.677+0000 7fcc33400640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:37:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.678+0000 7fcc2a800640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415b0f0 0x7fcc3406d020 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0097a0 tx=0x7fcc1c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fd16de2bdef8e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc32a00640 1 -- 172.21.15.91:0/653510683 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c02d040 con 0x7fcc3415b0f0 2024-02-02T13:37:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc32a00640 1 -- 172.21.15.91:0/653510683 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c031810 con 0x7fcc3415b0f0 2024-02-02T13:37:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc32a00640 1 -- 172.21.15.91:0/653510683 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c031ba0 con 0x7fcc3415b0f0 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 -- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415b0f0 msgr2=0x7fcc3406d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415b0f0 0x7fcc3406d020 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0097a0 tx=0x7fcc1c02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 -- 172.21.15.91:0/653510683 shutdown_connections 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415b0f0 0x7fcc3406d020 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 --2- 172.21.15.91:0/653510683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415ad10 0x7fcc3406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 -- 172.21.15.91:0/653510683 >> 172.21.15.91:0/653510683 conn(0x7fcc340b0700 msgr2=0x7fcc340b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 -- 172.21.15.91:0/653510683 shutdown_connections 2024-02-02T13:37:06.681 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.679+0000 7fcc3a200640 1 -- 172.21.15.91:0/653510683 wait complete. 2024-02-02T13:37:06.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.681+0000 7fcc3a200640 1 Processor -- start 2024-02-02T13:37:06.682 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.681+0000 7fcc3a200640 1 -- start start 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415ad10 0x7fcc340ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415b0f0 0x7fcc340ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340c6050 con 0x7fcc340cb580 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340adcc0 con 0x7fcc3415ad10 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc3a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340adfa0 con 0x7fcc3415b0f0 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc33400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc33400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53156/0 (socket says 172.21.15.91:53156) 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.682+0000 7fcc33400640 1 -- 172.21.15.91:0/876355033 learned_addr learned my addr 172.21.15.91:0/876355033 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 -- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415b0f0 msgr2=0x7fcc340ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415b0f0 0x7fcc340ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 -- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415ad10 msgr2=0x7fcc340ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415ad10 0x7fcc340ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.685 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 -- 172.21.15.91:0/876355033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240094d0 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc33400640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400e9d0 tx=0x7fcc2400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400ca60 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400cbc0 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24013470 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.683+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c009400 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.684+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340ae280 con 0x7fcc340cb580 2024-02-02T13:37:06.686 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.685+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3406d540 con 0x7fcc340cb580 2024-02-02T13:37:06.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.685+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcc240135d0 con 0x7fcc340cb580 2024-02-02T13:37:06.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.686+0000 7fcc2be00640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 0x7fcc0c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:06.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.686+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc24088dd0 con 0x7fcc340cb580 2024-02-02T13:37:06.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.687+0000 7fcc2a800640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 0x7fcc0c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:06.688 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.687+0000 7fcc2a800640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 0x7fcc0c06c8f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c036000 tx=0x7fcc1c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:06.692 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.692+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcc3406d540 con 0x7fcc340cb580 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.953+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcc3406d540 con 0x7fcc340cb580 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.954+0000 7fcc2be00640 1 -- 172.21.15.91:0/876355033 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcc3406d540 con 0x7fcc340cb580 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_dump": [ 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.a", 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlEqPsGRAAGeroztNJqCzya/CLXrrCaQ==", 2024-02-02T13:37:06.956 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.b", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl8G+LOxAAirqWqEQuNmnrRClTeYZq4g==", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.957 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.c", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxlSTLNKhAAcxSSdycDTmdi0u6OccOw2Q==", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mds.d", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlT/2gDxAAMFY9iZUSbTFZIVhRZSU8Ag==", 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.958 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.0", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD37LxlVM+RMxAAQW4lK9TEVAracw62d8/Q4Q==", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.959 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.1", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl/5oHBxAAD6bJQp8paXB0CzanoaI7rQ==", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.2", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47Lxl3dNaFRAApeXXBrDk8niWL5lViwQcaQ==", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.960 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.3", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD47LxliuxHIxAAT4GJEs9cxqtRQU2Fi4Beqw==", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.4", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD87LxlmZxpLhAAanK5TaXnT3W9RGxeoa8xzA==", 2024-02-02T13:37:06.961 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.5", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97Lxl+Uv/ARAATJCFIe4x2yEECVwkw7ueAg==", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.962 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.6", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlsB49ERAA5PWNWKu4LzlznhGK/fOM4Q==", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "osd.7", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQD97LxlFrhNIBAAu4GHLQbcSvPgXXGpvkpUNw==", 2024-02-02T13:37:06.963 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow profile osd", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile osd", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.0", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlsqOAHBAARz4ORJ2HSbljdxFJSMmKBA==", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow r", 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.964 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.1", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlSQDZMRAAUdTa/3pSUyCLof+zHO5NGA==", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.2", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlQ7JRJxAA37jM3I8N5kLX6jimCDemmA==", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:37:06.965 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.3", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDz7LxlY8GmOhAAYE3Gf6Bx5Pfo1hcOPbbjCA==", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow r", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow rw", 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow rwx" 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.966 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "client.admin", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDx7LxlD/0EDhAAruyTHdc+I5F9OxkYYXCkeA==", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": "allow *", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow *", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.x", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7LxllNJRDhAAfE8Omjp5VxQANk2GZ8haaQ==", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.967 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "entity": "mgr.y", 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "AQDy7Lxl3cYaARAAxOLHRYuRGVGSaKvwohHXdA==", 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "caps": { 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "mds": "allow *", 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "mon": "allow profile mgr", 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": "allow *" 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:06.968 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:06.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 msgr2=0x7fcc0c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 0x7fcc0c06c8f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c036000 tx=0x7fcc1c030000 comp rx=0 tx=0).stop 2024-02-02T13:37:06.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 msgr2=0x7fcc341501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:06.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400e9d0 tx=0x7fcc2400ee90 comp rx=0 tx=0).stop 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 shutdown_connections 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcc0c06a430 0x7fcc0c06c8f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3415b0f0 0x7fcc340ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3415ad10 0x7fcc340ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 --2- 172.21.15.91:0/876355033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcc340cb580 0x7fcc341501b0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 >> 172.21.15.91:0/876355033 conn(0x7fcc340b0700 msgr2=0x7fcc340c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 shutdown_connections 2024-02-02T13:37:06.970 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:06.957+0000 7fcc3a200640 1 -- 172.21.15.91:0/876355033 wait complete. 2024-02-02T13:37:06.977 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 Processor -- start 2024-02-02T13:37:07.175 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 -- start start 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82dc15f5a0 0x7f82dc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0b1070 con 0x7f82dc0b0c40 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc06c600 con 0x7f82dc151f90 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.174+0000 7f82e1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc15fa30 con 0x7f82dc15f5a0 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82db400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f82dc0b0c40 0x7f82dc0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41640/0 (socket says 172.21.15.91:41640) 2024-02-02T13:37:07.176 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82db400640 1 -- 172.21.15.91:0/2072571352 learned_addr learned my addr 172.21.15.91:0/2072571352 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:07.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82da000640 1 --2- 172.21.15.91:0/2072571352 >> v2:172.21.15.113:3300/0 conn(0x7f82dc15f5a0 0x7f82dc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876423002 0 0) 0x7f82dc0b1070 con 0x7f82dc0b0c40 2024-02-02T13:37:07.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82bc003520 con 0x7f82dc0b0c40 2024-02-02T13:37:07.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518366394 0 0) 0x7f82bc003520 con 0x7f82dc0b0c40 2024-02-02T13:37:07.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 >> v2:172.21.15.113:3300/0 conn(0x7f82dc15f5a0 msgr2=0x7f82dc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 --2- 172.21.15.91:0/2072571352 >> v2:172.21.15.113:3300/0 conn(0x7f82dc15f5a0 0x7f82dc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.175+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 >> v1:172.21.15.113:6789/0 conn(0x7f82dc151f90 legacy=0x7f82dc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc070f80 con 0x7f82dc0b0c40 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f82d40019f0 con 0x7f82dc0b0c40 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 >> v1:172.21.15.91:6789/0 conn(0x7f82dc0b0c40 legacy=0x7f82dc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82bc003970 0x7f82bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15fa30 con 0x7f82bc005180 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00c0d0 con 0x7f82bc0047c0 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00c290 con 0x7f82bc003970 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82da000640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82daa00640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82bc003970 0x7f82bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 -- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82bc003970 msgr2=0x7f82bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82bc003970 0x7f82bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 -- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82bc0047c0 msgr2=0x7f82bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:07.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82db400640 1 -- 172.21.15.91:0/2072571352 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc003520 con 0x7f82bc005180 2024-02-02T13:37:07.180 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.176+0000 7f82daa00640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82bc003970 0x7f82bc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.177+0000 7f82db400640 1 -- 172.21.15.91:0/2072571352 reap_dead start 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82db400640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82bc00b980 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f82d400a830 tx=0x7f82bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d26d5e10cd1e72 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d40072d0 con 0x7f82bc005180 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d402f070 con 0x7f82bc005180 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82d9600640 1 -- 172.21.15.91:0/2072571352 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4007650 con 0x7f82bc005180 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82e1800640 1 -- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 msgr2=0x7f82bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82e1800640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82bc00b980 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f82d400a830 tx=0x7f82bc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82e1800640 1 -- 172.21.15.91:0/2072571352 shutdown_connections 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82e1800640 1 --2- 172.21.15.91:0/2072571352 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82bc00b980 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.178+0000 7f82e1800640 1 -- 172.21.15.91:0/2072571352 >> 172.21.15.91:0/2072571352 conn(0x7f82dc0cec80 msgr2=0x7f82dc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:07.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.179+0000 7f82e1800640 1 -- 172.21.15.91:0/2072571352 shutdown_connections 2024-02-02T13:37:07.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.179+0000 7f82e1800640 1 -- 172.21.15.91:0/2072571352 wait complete. 2024-02-02T13:37:07.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.180+0000 7f82e1800640 1 Processor -- start 2024-02-02T13:37:07.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.180+0000 7f82e1800640 1 -- start start 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 0x7f82dc0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c2a10 0x7f82dc0c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0af2d0 con 0x7f82bc005180 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc140a60 con 0x7f82dc0c2a10 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.181+0000 7f82e1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc140d40 con 0x7f82dc0ae950 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53176/0 (socket says 172.21.15.91:53176) 2024-02-02T13:37:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 -- 172.21.15.91:0/4151815312 learned_addr learned my addr 172.21.15.91:0/4151815312 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82db400640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c2a10 0x7f82dc0c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82da000640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 0x7f82dc0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 -- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 msgr2=0x7f82dc0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 0x7f82dc0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 -- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c2a10 msgr2=0x7f82dc0c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c2a10 0x7f82dc0c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 -- 172.21.15.91:0/4151815312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc141020 con 0x7f82bc005180 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82da000640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 0x7f82dc0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.182+0000 7f82daa00640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000f60 tx=0x7f82c400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.183+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4005480 con 0x7f82bc005180 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.183+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4003610 con 0x7f82bc005180 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.183+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40039c0 con 0x7f82bc005180 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.183+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc141300 con 0x7f82bc005180 2024-02-02T13:37:07.185 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.183+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0c33d0 con 0x7f82bc005180 2024-02-02T13:37:07.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.184+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc066cf0 con 0x7f82bc005180 2024-02-02T13:37:07.186 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.185+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f82c4003b20 con 0x7f82bc005180 2024-02-02T13:37:07.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.186+0000 7f82d2a00640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 0x7f82b406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.186+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82c4088590 con 0x7f82bc005180 2024-02-02T13:37:07.187 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.186+0000 7f82da000640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 0x7f82b406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.187+0000 7f82da000640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 0x7f82b406c920 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82dc1419c0 tx=0x7f82cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.191+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f82c4053ee0 con 0x7f82bc005180 2024-02-02T13:37:07.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.366+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82dc06c7b0 con 0x7f82bc005180 2024-02-02T13:37:07.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.367+0000 7f82d2a00640 1 -- 172.21.15.91:0/4151815312 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f82c4001230 con 0x7f82bc005180 2024-02-02T13:37:07.368 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:07.368 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":191,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:06.426794+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:07.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 msgr2=0x7f82b406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 0x7f82b406c920 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82dc1419c0 tx=0x7f82cc008040 comp rx=0 tx=0).stop 2024-02-02T13:37:07.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 msgr2=0x7f82dc1f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000f60 tx=0x7f82c400d9f0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 shutdown_connections 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f82b406a460 0x7f82b406c920 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82dc0ae950 0x7f82dc0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c2a10 0x7f82dc0c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 --2- 172.21.15.91:0/4151815312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f82bc005180 0x7f82dc1f0290 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 >> 172.21.15.91:0/4151815312 conn(0x7f82dc0cec80 msgr2=0x7f82dc15a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 shutdown_connections 2024-02-02T13:37:07.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.369+0000 7f82e1800640 1 -- 172.21.15.91:0/4151815312 wait complete. 2024-02-02T13:37:07.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:37:07.389 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:37:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.595+0000 7f623b000640 1 Processor -- start 2024-02-02T13:37:07.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.595+0000 7f623b000640 1 -- start start 2024-02-02T13:37:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f623415a010 0x7f62340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62340ce130 0x7f62340ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62340cd470 0x7f62340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340cc7c0 con 0x7f62340cd470 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340ce930 con 0x7f62340ce130 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.596+0000 7f623b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623406c890 con 0x7f623415a010 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62340cd470 0x7f62340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62340cd470 0x7f62340b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53188/0 (socket says 172.21.15.91:53188) 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 -- 172.21.15.91:0/1774826726 learned_addr learned my addr 172.21.15.91:0/1774826726 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6233e00640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f623415a010 0x7f62340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f622b200640 1 --2- 172.21.15.91:0/1774826726 >> v2:172.21.15.113:3300/0 conn(0x7f62340ce130 0x7f62340ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 -- 172.21.15.91:0/1774826726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f623415a010 msgr2=0x7f62340c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f623415a010 0x7f62340c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 -- 172.21.15.91:0/1774826726 >> v2:172.21.15.113:3300/0 conn(0x7f62340ce130 msgr2=0x7f62340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 --2- 172.21.15.91:0/1774826726 >> v2:172.21.15.113:3300/0 conn(0x7f62340ce130 0x7f62340ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.597+0000 7f6239000640 1 -- 172.21.15.91:0/1774826726 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623406cb70 con 0x7f62340cd470 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.598+0000 7f6239000640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340cd470 0x7f62340b0700 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f623406d7d0 tx=0x7f622c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b368b0dc6edc035 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f6233400640 1 -- 172.21.15.91:0/1774826726 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c02ca70 con 0x7f62340cd470 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f6233400640 1 -- 172.21.15.91:0/1774826726 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c02cbd0 con 0x7f62340cd470 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f6233400640 1 -- 172.21.15.91:0/1774826726 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0343f0 con 0x7f62340cd470 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 -- 172.21.15.91:0/1774826726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340cd470 msgr2=0x7f62340b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340cd470 0x7f62340b0700 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f623406d7d0 tx=0x7f622c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 -- 172.21.15.91:0/1774826726 shutdown_connections 2024-02-02T13:37:07.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340cd470 0x7f62340b0700 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 --2- 172.21.15.91:0/1774826726 >> v2:172.21.15.113:3300/0 conn(0x7f62340ce130 0x7f62340ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 --2- 172.21.15.91:0/1774826726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f623415a010 0x7f62340c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 -- 172.21.15.91:0/1774826726 >> 172.21.15.91:0/1774826726 conn(0x7f62340cec80 msgr2=0x7f62340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:07.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 -- 172.21.15.91:0/1774826726 shutdown_connections 2024-02-02T13:37:07.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.599+0000 7f623b000640 1 -- 172.21.15.91:0/1774826726 wait complete. 2024-02-02T13:37:07.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.601+0000 7f623b000640 1 Processor -- start 2024-02-02T13:37:07.601 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.601+0000 7f623b000640 1 -- start start 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f623415a010 0x7f6234154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62340b4650 0x7f6234155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234155680 con 0x7f62340ce130 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234155aa0 con 0x7f623415a010 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f623b000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6234154a00 con 0x7f62340b4650 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f6233e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f6233e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53192/0 (socket says 172.21.15.91:53192) 2024-02-02T13:37:07.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.602+0000 7f6233e00640 1 -- 172.21.15.91:0/3496909359 learned_addr learned my addr 172.21.15.91:0/3496909359 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:07.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 -- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62340b4650 msgr2=0x7f6234155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62340b4650 0x7f6234155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 -- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f623415a010 msgr2=0x7f6234154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.604 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f623415a010 0x7f6234154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 -- 172.21.15.91:0/3496909359 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009b30 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6233e00640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f622400d5f0 tx=0x7f622400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240125f0 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224012750 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6234154ce0 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.603+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224012b00 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.604+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62341552a0 con 0x7f62340ce130 2024-02-02T13:37:07.605 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.605+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234066cf0 con 0x7f62340ce130 2024-02-02T13:37:07.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.606+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6224012c60 con 0x7f62340ce130 2024-02-02T13:37:07.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.607+0000 7f6230c00640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 0x7f620c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:07.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.607+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62240889f0 con 0x7f62340ce130 2024-02-02T13:37:07.607 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.607+0000 7f622b200640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 0x7f620c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:07.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.608+0000 7f622b200640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 0x7f620c06c920 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f623415d6e0 tx=0x7f621c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.612+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6224054310 con 0x7f62340ce130 2024-02-02T13:37:07.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.800+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62340b4c00 con 0x7f62340ce130 2024-02-02T13:37:07.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.801+0000 7f6230c00640 1 -- 172.21.15.91:0/3496909359 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6224057930 con 0x7f62340ce130 2024-02-02T13:37:07.801 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 msgr2=0x7f620c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 0x7f620c06c920 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f623415d6e0 tx=0x7f621c008040 comp rx=0 tx=0).stop 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 msgr2=0x7f62341530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f622400d5f0 tx=0x7f622400dab0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 shutdown_connections 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f620c06a460 0x7f620c06c920 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62340b4650 0x7f6234155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f623415a010 0x7f6234154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 --2- 172.21.15.91:0/3496909359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62340ce130 0x7f62341530d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:07.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.803+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 >> 172.21.15.91:0/3496909359 conn(0x7f62340cec80 msgr2=0x7f62340acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:07.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.804+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 shutdown_connections 2024-02-02T13:37:07.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:07.804+0000 7f623b000640 1 -- 172.21.15.91:0/3496909359 wait complete. 2024-02-02T13:37:07.823 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T13:37:08.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.027+0000 7fbe15800640 1 Processor -- start 2024-02-02T13:37:08.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.027+0000 7fbe15800640 1 -- start start 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.028+0000 7fbe15800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbe100cbc90 0x7fbe1006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.028+0000 7fbe15800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1006d910 con 0x7fbe1015b0f0 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.028+0000 7fbe15800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100cc850 con 0x7fbe1015ad10 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.028+0000 7fbe15800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1006d730 con 0x7fbe100cbc90 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe05e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbe1015b0f0 0x7fbe100cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41648/0 (socket says 172.21.15.91:41648) 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe05e00640 1 -- 172.21.15.91:0/1099537492 learned_addr learned my addr 172.21.15.91:0/1099537492 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0f400640 1 --2- 172.21.15.91:0/1099537492 >> v2:172.21.15.91:3300/0 conn(0x7fbe100cbc90 0x7fbe1006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0e000640 1 -- 172.21.15.91:0/1099537492 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245858033 0 0) 0x7fbe1006d910 con 0x7fbe1015b0f0 2024-02-02T13:37:08.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0e000640 1 -- 172.21.15.91:0/1099537492 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf0003520 con 0x7fbe1015b0f0 2024-02-02T13:37:08.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0f400640 1 -- 172.21.15.91:0/1099537492 >> v1:172.21.15.113:6789/0 conn(0x7fbe1015ad10 legacy=0x7fbe1006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0f400640 1 -- 172.21.15.91:0/1099537492 >> v1:172.21.15.91:6789/0 conn(0x7fbe1015b0f0 legacy=0x7fbe100cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.029+0000 7fbe0f400640 1 -- 172.21.15.91:0/1099537492 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe100ccb30 con 0x7fbe100cbc90 2024-02-02T13:37:08.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.030+0000 7fbe0f400640 1 --2- 172.21.15.91:0/1099537492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe100cbc90 0x7fbe1006b7a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009dc0 tx=0x7fbe0802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a416150ebbc9f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe0e000640 1 -- 172.21.15.91:0/1099537492 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0802cc40 con 0x7fbe100cbc90 2024-02-02T13:37:08.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe0e000640 1 -- 172.21.15.91:0/1099537492 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0802cda0 con 0x7fbe100cbc90 2024-02-02T13:37:08.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe0e000640 1 -- 172.21.15.91:0/1099537492 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08034660 con 0x7fbe100cbc90 2024-02-02T13:37:08.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe15800640 1 -- 172.21.15.91:0/1099537492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe100cbc90 msgr2=0x7fbe1006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe15800640 1 --2- 172.21.15.91:0/1099537492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe100cbc90 0x7fbe1006b7a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009dc0 tx=0x7fbe0802be90 comp rx=0 tx=0).stop 2024-02-02T13:37:08.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe15800640 1 -- 172.21.15.91:0/1099537492 shutdown_connections 2024-02-02T13:37:08.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe15800640 1 --2- 172.21.15.91:0/1099537492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe100cbc90 0x7fbe1006b7a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.031+0000 7fbe15800640 1 -- 172.21.15.91:0/1099537492 >> 172.21.15.91:0/1099537492 conn(0x7fbe100b0700 msgr2=0x7fbe100b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:08.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.032+0000 7fbe15800640 1 -- 172.21.15.91:0/1099537492 shutdown_connections 2024-02-02T13:37:08.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.032+0000 7fbe15800640 1 -- 172.21.15.91:0/1099537492 wait complete. 2024-02-02T13:37:08.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.033+0000 7fbe15800640 1 Processor -- start 2024-02-02T13:37:08.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.034+0000 7fbe15800640 1 -- start start 2024-02-02T13:37:08.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.034+0000 7fbe15800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 0x7fbe1014b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.034+0000 7fbe15800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe15800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 0x7fbe100ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe15800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1014ce60 con 0x7fbe1015ad10 2024-02-02T13:37:08.036 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe15800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1014bd10 con 0x7fbe1015b0f0 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe15800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1014bf90 con 0x7fbe100cbc90 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53210/0 (socket says 172.21.15.91:53210) 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 -- 172.21.15.91:0/2330684591 learned_addr learned my addr 172.21.15.91:0/2330684591 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe0ea00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 0x7fbe1014b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe0f400640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 0x7fbe100ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 -- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 msgr2=0x7fbe1014b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 0x7fbe1014b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 -- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 msgr2=0x7fbe100ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 0x7fbe100ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe05e00640 1 -- 172.21.15.91:0/2330684591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08009a70 con 0x7fbe1015ad10 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe0ea00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 0x7fbe1014b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.035+0000 7fbe0f400640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 0x7fbe100ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe05e00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c440 tx=0x7fbdf800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005ce0 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf8005e40 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80072b0 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1014c1e0 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.036+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1014c7a0 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.037+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe10066cf0 con 0x7fbe1015ad10 2024-02-02T13:37:08.038 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.038+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbdf8007410 con 0x7fbe1015ad10 2024-02-02T13:37:08.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.039+0000 7fbe07400640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 0x7fbde806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.039+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbdf8087b80 con 0x7fbe1015ad10 2024-02-02T13:37:08.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.039+0000 7fbe0ea00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 0x7fbde806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.040 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.040+0000 7fbe0ea00640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 0x7fbde806c8f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbe1015e330 tx=0x7fbe00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.043 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.043+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbdf80534e0 con 0x7fbe1015ad10 2024-02-02T13:37:08.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.236+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbe100c7980 con 0x7fbe1015ad10 2024-02-02T13:37:08.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.237+0000 7fbe07400640 1 -- 172.21.15.91:0/2330684591 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v191) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbdf8017090 con 0x7fbe1015ad10 2024-02-02T13:37:08.237 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout:[ 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout: "poolnum": 1, 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout: "poolname": ".mgr" 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:08.238 INFO:teuthology.orchestra.run.smithi091.stdout:] 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.239+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 msgr2=0x7fbde806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 0x7fbde806c8f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbe1015e330 tx=0x7fbe00009040 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 msgr2=0x7fbe100c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c440 tx=0x7fbdf800c900 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 shutdown_connections 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbde806a430 0x7fbde806c8f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1015b0f0 0x7fbe100ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbe1015ad10 0x7fbe100c7440 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 --2- 172.21.15.91:0/2330684591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe100cbc90 0x7fbe1014b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 >> 172.21.15.91:0/2330684591 conn(0x7fbe100b0700 msgr2=0x7fbe1006fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 shutdown_connections 2024-02-02T13:37:08.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.240+0000 7fbe15800640 1 -- 172.21.15.91:0/2330684591 wait complete. 2024-02-02T13:37:08.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T13:37:08.258 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T13:37:08.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.469+0000 7fbd19800640 1 Processor -- start 2024-02-02T13:37:08.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.469+0000 7fbd19800640 1 -- start start 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd19800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbd140ce130 0x7fbd140c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd19800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 0x7fbd140b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd19800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd140cc7c0 con 0x7fbd1415a010 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd19800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140b10b0 con 0x7fbd140ce130 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd19800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140ce930 con 0x7fbd140b0700 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.470+0000 7fbd09e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbd140ce130 0x7fbd140c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd12a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbd1415a010 0x7fbd140ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:41652/0 (socket says 172.21.15.91:41652) 2024-02-02T13:37:08.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd12a00640 1 -- 172.21.15.91:0/3825576437 learned_addr learned my addr 172.21.15.91:0/3825576437 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd13400640 1 --2- 172.21.15.91:0/3825576437 >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 0x7fbd140b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd09e00640 1 -- 172.21.15.91:0/3825576437 >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 msgr2=0x7fbd140b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd09e00640 1 --2- 172.21.15.91:0/3825576437 >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 0x7fbd140b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd09e00640 1 -- 172.21.15.91:0/3825576437 >> v1:172.21.15.91:6789/0 conn(0x7fbd1415a010 legacy=0x7fbd140ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd12000640 1 -- 172.21.15.91:0/3825576437 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678831326 0 0) 0x7fbd140cc7c0 con 0x7fbd1415a010 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd09e00640 1 -- 172.21.15.91:0/3825576437 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140cd640 con 0x7fbd140ce130 2024-02-02T13:37:08.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.471+0000 7fbd13400640 1 --2- 172.21.15.91:0/3825576437 >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 0x7fbd140b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:08.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.472+0000 7fbd09e00640 1 --2- 172.21.15.91:0/3825576437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140ce130 0x7fbd140c9670 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbd140cec10 tx=0x7fbcfc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f2f7cb030a3179 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd12000640 1 -- 172.21.15.91:0/3825576437 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc031830 con 0x7fbd140ce130 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd12000640 1 -- 172.21.15.91:0/3825576437 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc031990 con 0x7fbd140ce130 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd12000640 1 -- 172.21.15.91:0/3825576437 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc031d20 con 0x7fbd140ce130 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 -- 172.21.15.91:0/3825576437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140ce130 msgr2=0x7fbd140c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 --2- 172.21.15.91:0/3825576437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140ce130 0x7fbd140c9670 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbd140cec10 tx=0x7fbcfc02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 -- 172.21.15.91:0/3825576437 shutdown_connections 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 --2- 172.21.15.91:0/3825576437 >> v2:172.21.15.113:3300/0 conn(0x7fbd140b0700 0x7fbd140b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 --2- 172.21.15.91:0/3825576437 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140ce130 0x7fbd140c9670 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 -- 172.21.15.91:0/3825576437 >> 172.21.15.91:0/3825576437 conn(0x7fbd140cec80 msgr2=0x7fbd140cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 -- 172.21.15.91:0/3825576437 shutdown_connections 2024-02-02T13:37:08.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.473+0000 7fbd19800640 1 -- 172.21.15.91:0/3825576437 wait complete. 2024-02-02T13:37:08.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.475+0000 7fbd19800640 1 Processor -- start 2024-02-02T13:37:08.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.475+0000 7fbd19800640 1 -- start start 2024-02-02T13:37:08.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd140ce130 0x7fbd140c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd1415a010 0x7fbd140c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c29c0 con 0x7fbd140b0700 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c2390 con 0x7fbd140ce130 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd19800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c5620 con 0x7fbd1415a010 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd12a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd12a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53236/0 (socket says 172.21.15.91:53236) 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.476+0000 7fbd12a00640 1 -- 172.21.15.91:0/1990786932 learned_addr learned my addr 172.21.15.91:0/1990786932 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 -- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd1415a010 msgr2=0x7fbd140c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd1415a010 0x7fbd140c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 -- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd140ce130 msgr2=0x7fbd140c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd09e00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd140ce130 0x7fbd140c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd140ce130 0x7fbd140c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 -- 172.21.15.91:0/1990786932 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc009400 con 0x7fbd140b0700 2024-02-02T13:37:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd12a00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c430 tx=0x7fbd0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040074e0 con 0x7fbd140b0700 2024-02-02T13:37:08.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd04007660 con 0x7fbd140b0700 2024-02-02T13:37:08.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.477+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd140c5900 con 0x7fbd140b0700 2024-02-02T13:37:08.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.478+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004a20 con 0x7fbd140b0700 2024-02-02T13:37:08.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.478+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd141f43e0 con 0x7fbd140b0700 2024-02-02T13:37:08.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.479+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd140c2c30 con 0x7fbd140b0700 2024-02-02T13:37:08.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.480+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbd04004b80 con 0x7fbd140b0700 2024-02-02T13:37:08.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.481+0000 7fbd0b400640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 0x7fbcec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:08.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.481+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd040883f0 con 0x7fbd140b0700 2024-02-02T13:37:08.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.481+0000 7fbd09e00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 0x7fbcec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:08.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.482+0000 7fbd09e00640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 0x7fbcec06c8f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc036000 tx=0x7fbcfc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:08.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.486+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbd04053d20 con 0x7fbd140b0700 2024-02-02T13:37:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:08.675+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbd14066cf0 con 0x7fbd140b0700 2024-02-02T13:37:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.240+0000 7fbd0b400640 1 -- 172.21.15.91:0/1990786932 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v192) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd04057340 con 0x7fbd140b0700 2024-02-02T13:37:09.241 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' created 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.244+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 msgr2=0x7fbcec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.244+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 0x7fbcec06c8f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc036000 tx=0x7fbcfc008030 comp rx=0 tx=0).stop 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.244+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 msgr2=0x7fbd14157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.244+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c430 tx=0x7fbd0400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 shutdown_connections 2024-02-02T13:37:09.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbcec06a430 0x7fbcec06c8f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd1415a010 0x7fbd140c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd140ce130 0x7fbd140c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 --2- 172.21.15.91:0/1990786932 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd140b0700 0x7fbd14157110 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 >> 172.21.15.91:0/1990786932 conn(0x7fbd140cec80 msgr2=0x7fbd1415cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.245+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 shutdown_connections 2024-02-02T13:37:09.246 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.246+0000 7fbd19800640 1 -- 172.21.15.91:0/1990786932 wait complete. 2024-02-02T13:37:09.268 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T13:37:09.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.480+0000 7ff1ef200640 1 Processor -- start 2024-02-02T13:37:09.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.481+0000 7ff1ef200640 1 -- start start 2024-02-02T13:37:09.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.482+0000 7ff1ef200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff1e80b0700 0x7ff1e80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.482+0000 7ff1ef200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.482+0000 7ff1ef200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80ccdd0 con 0x7ff1e8161660 2024-02-02T13:37:09.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.482+0000 7ff1ef200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e80b10d0 con 0x7ff1e80b0700 2024-02-02T13:37:09.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.482+0000 7ff1ef200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80cc7c0 con 0x7ff1e80cd260 2024-02-02T13:37:09.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.483+0000 7ff1ed200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.483+0000 7ff1ed200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:42916/0 (socket says 172.21.15.91:42916) 2024-02-02T13:37:09.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.483+0000 7ff1ed200640 1 -- 172.21.15.91:0/1419962243 learned_addr learned my addr 172.21.15.91:0/1419962243 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:09.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.483+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1419962243 >> v2:172.21.15.113:3300/0 conn(0x7ff1e80b0700 0x7ff1e80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.484+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 msgr2=0x7ff1e80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:09.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.484+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.484+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1419962243 >> v1:172.21.15.113:6789/0 conn(0x7ff1e8161660 legacy=0x7ff1e806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.484+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1419962243 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80c9570 con 0x7ff1e80b0700 2024-02-02T13:37:09.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.484+0000 7ff1ed200640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:37:09.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.486+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b0b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097a0 tx=0x7ff1d002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69fb2875a84f876b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:09.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.486+0000 7ff1e7400640 1 -- 172.21.15.91:0/1419962243 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d002e070 con 0x7ff1e80b0700 2024-02-02T13:37:09.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.486+0000 7ff1e7400640 1 -- 172.21.15.91:0/1419962243 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00315a0 con 0x7ff1e80b0700 2024-02-02T13:37:09.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.486+0000 7ff1e7400640 1 -- 172.21.15.91:0/1419962243 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0031930 con 0x7ff1e80b0700 2024-02-02T13:37:09.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 -- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 msgr2=0x7ff1e80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b0b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097a0 tx=0x7ff1d002b840 comp rx=0 tx=0).stop 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 -- 172.21.15.91:0/1419962243 shutdown_connections 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e80cd260 0x7ff1e80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1419962243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b0b90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 -- 172.21.15.91:0/1419962243 >> 172.21.15.91:0/1419962243 conn(0x7ff1e80cec80 msgr2=0x7ff1e80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 -- 172.21.15.91:0/1419962243 shutdown_connections 2024-02-02T13:37:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.487+0000 7ff1ef200640 1 -- 172.21.15.91:0/1419962243 wait complete. 2024-02-02T13:37:09.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.489+0000 7ff1ef200640 1 Processor -- start 2024-02-02T13:37:09.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.489+0000 7ff1ef200640 1 -- start start 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e8161660 0x7ff1e8070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80b4860 con 0x7ff1e80cd260 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c6c00 con 0x7ff1e80b0700 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ef200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c6080 con 0x7ff1e8161660 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:53246/0 (socket says 172.21.15.91:53246) 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1225663071 learned_addr learned my addr 172.21.15.91:0/1225663071 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1e7e00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.490+0000 7ff1ed200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e8161660 0x7ff1e8070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e8161660 msgr2=0x7ff1e8070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e8161660 0x7ff1e8070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 msgr2=0x7ff1e80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 -- 172.21.15.91:0/1225663071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0009b30 con 0x7ff1e80cd260 2024-02-02T13:37:09.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1dfe00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0034040 tx=0x7ff1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0032630 con 0x7ff1e80cd260 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0032790 con 0x7ff1e80cd260 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0032b40 con 0x7ff1e80cd260 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0009400 con 0x7ff1e80cd260 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.491+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e80c6640 con 0x7ff1e80cd260 2024-02-02T13:37:09.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.492+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e80cca40 con 0x7ff1e80cd260 2024-02-02T13:37:09.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.493+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff1d0032ca0 con 0x7ff1e80cd260 2024-02-02T13:37:09.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.494+0000 7ff1e4c00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 0x7ff1c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:09.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.494+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff1d00aa610 con 0x7ff1e80cd260 2024-02-02T13:37:09.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.494+0000 7ff1e7e00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 0x7ff1c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:09.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.495+0000 7ff1e7e00640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 0x7ff1c006c8f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8001780 tx=0x7ff1d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:09.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.500+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff1e80cca40 con 0x7ff1e80cd260 2024-02-02T13:37:09.694 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:09.692+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff1e80c2720 con 0x7ff1e80cd260 2024-02-02T13:37:10.255 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.253+0000 7ff1e4c00640 1 -- 172.21.15.91:0/1225663071 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v193) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff1e80c2720 con 0x7ff1e80cd260 2024-02-02T13:37:10.260 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' created 2024-02-02T13:37:10.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.264+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 msgr2=0x7ff1c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.264+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 0x7ff1c006c8f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8001780 tx=0x7ff1d8009040 comp rx=0 tx=0).stop 2024-02-02T13:37:10.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.264+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 msgr2=0x7ff1e81f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.264+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0034040 tx=0x7ff1d00046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.265+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 shutdown_connections 2024-02-02T13:37:10.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.265+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff1c006a430 0x7ff1c006c8f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.265+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e8161660 0x7ff1e8070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.265+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1e80cd260 0x7ff1e81f67b0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.265+0000 7ff1ef200640 1 --2- 172.21.15.91:0/1225663071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e80b0700 0x7ff1e80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.266+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 >> 172.21.15.91:0/1225663071 conn(0x7ff1e80cec80 msgr2=0x7ff1e80c1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:10.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.266+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 shutdown_connections 2024-02-02T13:37:10.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.266+0000 7ff1ef200640 1 -- 172.21.15.91:0/1225663071 wait complete. 2024-02-02T13:37:10.290 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T13:37:10.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.473+0000 7fcb05600640 1 Processor -- start 2024-02-02T13:37:10.475 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.473+0000 7fcb05600640 1 -- start start 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.474+0000 7fcb05600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb0015f5a0 0x7fcb000b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.474+0000 7fcb05600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb000b1070 con 0x7fcb00151f90 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.474+0000 7fcb05600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0006c600 con 0x7fcb000b0c40 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.474+0000 7fcb05600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0015fa30 con 0x7fcb0015f5a0 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcb00151f90 0x7fcb0006f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51478/0 (socket says 172.21.15.91:51478) 2024-02-02T13:37:10.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafea00640 1 -- 172.21.15.91:0/4063164014 learned_addr learned my addr 172.21.15.91:0/4063164014 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:10.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafe000640 1 --2- 172.21.15.91:0/4063164014 >> v2:172.21.15.91:3300/0 conn(0x7fcb0015f5a0 0x7fcb000b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafd600640 1 -- 172.21.15.91:0/4063164014 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208856872 0 0) 0x7fcb000b1070 con 0x7fcb00151f90 2024-02-02T13:37:10.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafd600640 1 -- 172.21.15.91:0/4063164014 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae0003520 con 0x7fcb00151f90 2024-02-02T13:37:10.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafe000640 1 -- 172.21.15.91:0/4063164014 >> v1:172.21.15.113:6789/0 conn(0x7fcb000b0c40 legacy=0x7fcb000b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:10.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafe000640 1 -- 172.21.15.91:0/4063164014 >> v1:172.21.15.91:6789/0 conn(0x7fcb00151f90 legacy=0x7fcb0006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.475+0000 7fcafe000640 1 -- 172.21.15.91:0/4063164014 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00070f80 con 0x7fcb0015f5a0 2024-02-02T13:37:10.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.476+0000 7fcafe000640 1 --2- 172.21.15.91:0/4063164014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000b0700 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097a0 tx=0x7fcae802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a1e06cdee1637 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcafd600640 1 -- 172.21.15.91:0/4063164014 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8031830 con 0x7fcb0015f5a0 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcafd600640 1 -- 172.21.15.91:0/4063164014 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae8031990 con 0x7fcb0015f5a0 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcafd600640 1 -- 172.21.15.91:0/4063164014 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8031d20 con 0x7fcb0015f5a0 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 -- 172.21.15.91:0/4063164014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 msgr2=0x7fcb000b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 --2- 172.21.15.91:0/4063164014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000b0700 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097a0 tx=0x7fcae802b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 -- 172.21.15.91:0/4063164014 shutdown_connections 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 --2- 172.21.15.91:0/4063164014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000b0700 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 -- 172.21.15.91:0/4063164014 >> 172.21.15.91:0/4063164014 conn(0x7fcb000cec80 msgr2=0x7fcb000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 -- 172.21.15.91:0/4063164014 shutdown_connections 2024-02-02T13:37:10.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.477+0000 7fcb05600640 1 -- 172.21.15.91:0/4063164014 wait complete. 2024-02-02T13:37:10.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.479+0000 7fcb05600640 1 Processor -- start 2024-02-02T13:37:10.480 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.479+0000 7fcb05600640 1 -- start start 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb000b0c40 0x7fcb0015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb00151f90 0x7fcb000c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00150dc0 con 0x7fcb0015f5a0 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b18a0 con 0x7fcb00151f90 2024-02-02T13:37:10.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcb05600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b1b80 con 0x7fcb000b0c40 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcaff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcaff400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41442/0 (socket says 172.21.15.91:41442) 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.480+0000 7fcaff400640 1 -- 172.21.15.91:0/3522941008 learned_addr learned my addr 172.21.15.91:0/3522941008 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcafe000640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb00151f90 0x7fcb000c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcafea00640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb000b0c40 0x7fcb0015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 -- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb000b0c40 msgr2=0x7fcb0015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb000b0c40 0x7fcb0015a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 -- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb00151f90 msgr2=0x7fcb000c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb00151f90 0x7fcb000c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 -- 172.21.15.91:0/3522941008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8009400 con 0x7fcb0015f5a0 2024-02-02T13:37:10.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaff400640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b9e0 tx=0x7fcaf800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8003b70 con 0x7fcb0015f5a0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8003cd0 con 0x7fcb0015f5a0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800cae0 con 0x7fcb0015f5a0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.481+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb000b1e60 con 0x7fcb0015f5a0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.482+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb000b2420 con 0x7fcb0015f5a0 2024-02-02T13:37:10.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.483+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00155990 con 0x7fcb0015f5a0 2024-02-02T13:37:10.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.484+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcaf800cc40 con 0x7fcb0015f5a0 2024-02-02T13:37:10.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.485+0000 7fcaf6a00640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 0x7fcad806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.485+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcaf8088640 con 0x7fcb0015f5a0 2024-02-02T13:37:10.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.486+0000 7fcafea00640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 0x7fcad806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.486+0000 7fcafea00640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 0x7fcad806c8f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00011e0 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.491+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcaf8053bf0 con 0x7fcb0015f5a0 2024-02-02T13:37:10.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.716+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcb00066cf0 con 0x7fcb0015f5a0 2024-02-02T13:37:10.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.728+0000 7fcaf6a00640 1 -- 172.21.15.91:0/3522941008 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v84) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcaf8057210 con 0x7fcb0015f5a0 2024-02-02T13:37:10.730 INFO:teuthology.orchestra.run.smithi091.stderr:new fs with metadata pool 22 and data pool 23 2024-02-02T13:37:10.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 msgr2=0x7fcad806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 0x7fcad806c8f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00011e0 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 msgr2=0x7fcb000c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b9e0 tx=0x7fcaf800bea0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 shutdown_connections 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcad806a430 0x7fcad806c8f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb0015f5a0 0x7fcb000c2a00 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb00151f90 0x7fcb000c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 --2- 172.21.15.91:0/3522941008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb000b0c40 0x7fcb0015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.733+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 >> 172.21.15.91:0/3522941008 conn(0x7fcb000cec80 msgr2=0x7fcb000cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:10.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.735+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 shutdown_connections 2024-02-02T13:37:10.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.736+0000 7fcb05600640 1 -- 172.21.15.91:0/3522941008 wait complete. 2024-02-02T13:37:10.761 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:10.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.928+0000 7fde69800640 1 Processor -- start 2024-02-02T13:37:10.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.928+0000 7fde69800640 1 -- start start 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.929+0000 7fde69800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 0x7fde640cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.929+0000 7fde69800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde6406f3f0 0x7fde640ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.929+0000 7fde69800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6406c1c0 con 0x7fde6415aec0 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.929+0000 7fde69800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6406b5f0 con 0x7fde6406f3f0 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.929+0000 7fde69800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406cd60 con 0x7fde6415b2a0 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde63e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fde6415aec0 0x7fde6406c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51492/0 (socket says 172.21.15.91:51492) 2024-02-02T13:37:10.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde63e00640 1 -- 172.21.15.91:0/4238289277 learned_addr learned my addr 172.21.15.91:0/4238289277 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde68e00640 1 --2- 172.21.15.91:0/4238289277 >> v2:172.21.15.113:3300/0 conn(0x7fde6406f3f0 0x7fde640ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde63400640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 0x7fde640cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051668707 0 0) 0x7fde6406c1c0 con 0x7fde6415aec0 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde44003520 con 0x7fde6415aec0 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.930+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261414326 0 0) 0x7fde44003520 con 0x7fde6415aec0 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 msgr2=0x7fde640cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 0x7fde640cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 >> v2:172.21.15.113:3300/0 conn(0x7fde6406f3f0 msgr2=0x7fde640ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:10.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 --2- 172.21.15.91:0/4238289277 >> v2:172.21.15.113:3300/0 conn(0x7fde6406f3f0 0x7fde640ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6406b890 con 0x7fde6415aec0 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fde54001670 con 0x7fde6415aec0 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 >> v1:172.21.15.91:6789/0 conn(0x7fde6415aec0 legacy=0x7fde6406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde44003970 0x7fde44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 0x7fde44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406b5f0 con 0x7fde440047c0 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406cd60 con 0x7fde44003970 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4400b9b0 con 0x7fde6415b2a0 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 -- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 msgr2=0x7fde44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:10.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde6415b2a0 0x7fde44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 -- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde44003970 msgr2=0x7fde44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde44003970 0x7fde44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.931+0000 7fde63400640 1 -- 172.21.15.91:0/4238289277 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44003520 con 0x7fde440047c0 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.932+0000 7fde63400640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde44004c40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009ac0 tx=0x7fde440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af35e9bf3c350f1f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c02c710 con 0x7fde440047c0 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c02c870 con 0x7fde440047c0 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde62a00640 1 -- 172.21.15.91:0/4238289277 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c02cc00 con 0x7fde440047c0 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 -- 172.21.15.91:0/4238289277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 msgr2=0x7fde44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 --2- 172.21.15.91:0/4238289277 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde44004c40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009ac0 tx=0x7fde440057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde68e00640 1 -- 172.21.15.91:0/4238289277 reap_dead start 2024-02-02T13:37:10.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 -- 172.21.15.91:0/4238289277 shutdown_connections 2024-02-02T13:37:10.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 -- 172.21.15.91:0/4238289277 >> 172.21.15.91:0/4238289277 conn(0x7fde640cea90 msgr2=0x7fde640ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:10.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 -- 172.21.15.91:0/4238289277 shutdown_connections 2024-02-02T13:37:10.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.933+0000 7fde69800640 1 -- 172.21.15.91:0/4238289277 wait complete. 2024-02-02T13:37:10.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.935+0000 7fde69800640 1 Processor -- start 2024-02-02T13:37:10.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.935+0000 7fde69800640 1 -- start start 2024-02-02T13:37:10.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 0x7fde64158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde64070ba0 0x7fde64071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64150580 con 0x7fde440047c0 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64071560 con 0x7fde64070ba0 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde69800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640cd110 con 0x7fde640cc6f0 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde63e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde63e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41456/0 (socket says 172.21.15.91:41456) 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.936+0000 7fde63e00640 1 -- 172.21.15.91:0/3084910652 learned_addr learned my addr 172.21.15.91:0/3084910652 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63400640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 0x7fde64158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 -- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 msgr2=0x7fde64158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde68e00640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde64070ba0 0x7fde64071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 0x7fde64158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 -- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde64070ba0 msgr2=0x7fde64071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde64070ba0 0x7fde64071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 -- 172.21.15.91:0/3084910652 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640cd3f0 con 0x7fde440047c0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63400640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 0x7fde64158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde63e00640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c7a0 tx=0x7fde5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540074e0 con 0x7fde440047c0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54007660 con 0x7fde440047c0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54004a20 con 0x7fde440047c0 2024-02-02T13:37:10.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde640cd6d0 con 0x7fde440047c0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.937+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641544f0 con 0x7fde440047c0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.938+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6404dff0 con 0x7fde440047c0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.939+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde54004b80 con 0x7fde440047c0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.940+0000 7fde5be00640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 0x7fde3c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.940+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde54087dc0 con 0x7fde440047c0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.940+0000 7fde63400640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 0x7fde3c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:10.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.941+0000 7fde63400640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 0x7fde3c06c920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:10.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:10.945+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde54053310 con 0x7fde440047c0 2024-02-02T13:37:11.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.136+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde640cab30 con 0x7fde440047c0 2024-02-02T13:37:11.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.139+0000 7fde5be00640 1 -- 172.21.15.91:0/3084910652 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16117 (secure 0 0 0) 0x7fde54018090 con 0x7fde440047c0 2024-02-02T13:37:11.141 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:11.141 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":194,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:10.718728+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:11.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 msgr2=0x7fde3c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 0x7fde3c06c920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c030000 comp rx=0 tx=0).stop 2024-02-02T13:37:11.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 msgr2=0x7fde64150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c7a0 tx=0x7fde5400cc60 comp rx=0 tx=0).stop 2024-02-02T13:37:11.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 shutdown_connections 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde3c06a460 0x7fde3c06c920 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde640cc6f0 0x7fde64158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde64070ba0 0x7fde64071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 --2- 172.21.15.91:0/3084910652 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde440047c0 0x7fde64150040 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.142+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 >> 172.21.15.91:0/3084910652 conn(0x7fde640cea90 msgr2=0x7fde6406d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.143+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 shutdown_connections 2024-02-02T13:37:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.143+0000 7fde69800640 1 -- 172.21.15.91:0/3084910652 wait complete. 2024-02-02T13:37:11.163 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:11.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.367+0000 7f1be1400640 1 Processor -- start 2024-02-02T13:37:11.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.367+0000 7f1be1400640 1 -- start start 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1bdc15aec0 0x7f1bdc0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bdc15b2a0 0x7f1bdc06c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc06cb90 0x7f1bdc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc06c1c0 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc06b5f0 con 0x7f1bdc15aec0 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.368+0000 7f1be1400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc06b890 con 0x7f1bdc06cb90 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bdc15b2a0 0x7f1bdc06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1bdc15b2a0 0x7f1bdc06c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41472/0 (socket says 172.21.15.91:41472) 2024-02-02T13:37:11.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 -- 172.21.15.91:0/3897245567 learned_addr learned my addr 172.21.15.91:0/3897245567 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 -- 172.21.15.91:0/3897245567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc06cb90 msgr2=0x7f1bdc0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 --2- 172.21.15.91:0/3897245567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc06cb90 0x7f1bdc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 -- 172.21.15.91:0/3897245567 >> v2:172.21.15.113:3300/0 conn(0x7f1bdc15aec0 msgr2=0x7f1bdc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bdbe00640 1 --2- 172.21.15.91:0/3897245567 >> v2:172.21.15.113:3300/0 conn(0x7f1bdc15aec0 0x7f1bdc0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 --2- 172.21.15.91:0/3897245567 >> v2:172.21.15.113:3300/0 conn(0x7f1bdc15aec0 0x7f1bdc0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.369+0000 7f1bd3200640 1 -- 172.21.15.91:0/3897245567 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc06bb70 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.370+0000 7f1bd3200640 1 --2- 172.21.15.91:0/3897245567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc06c650 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc067800 tx=0x7f1bc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620b5e42a886f389 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.370+0000 7f1bdb400640 1 -- 172.21.15.91:0/3897245567 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4030860 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1bdb400640 1 -- 172.21.15.91:0/3897245567 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc40309c0 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1bdb400640 1 -- 172.21.15.91:0/3897245567 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4030d50 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 -- 172.21.15.91:0/3897245567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc15b2a0 msgr2=0x7f1bdc06c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 --2- 172.21.15.91:0/3897245567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc06c650 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc067800 tx=0x7f1bc402b9a0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 -- 172.21.15.91:0/3897245567 shutdown_connections 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 --2- 172.21.15.91:0/3897245567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc06cb90 0x7f1bdc0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 --2- 172.21.15.91:0/3897245567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc06c650 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 --2- 172.21.15.91:0/3897245567 >> v2:172.21.15.113:3300/0 conn(0x7f1bdc15aec0 0x7f1bdc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 -- 172.21.15.91:0/3897245567 >> 172.21.15.91:0/3897245567 conn(0x7f1bdc0cea90 msgr2=0x7f1bdc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:11.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 -- 172.21.15.91:0/3897245567 shutdown_connections 2024-02-02T13:37:11.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.371+0000 7f1be1400640 1 -- 172.21.15.91:0/3897245567 wait complete. 2024-02-02T13:37:11.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.373+0000 7f1be1400640 1 Processor -- start 2024-02-02T13:37:11.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.373+0000 7f1be1400640 1 -- start start 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc15aec0 0x7f1bdc154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc152560 con 0x7f1bdc06cb90 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1548a0 con 0x7f1bdc15b2a0 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1be1400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc154b80 con 0x7f1bdc15aec0 2024-02-02T13:37:11.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41476/0 (socket says 172.21.15.91:41476) 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 -- 172.21.15.91:0/2655395321 learned_addr learned my addr 172.21.15.91:0/2655395321 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 -- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc15aec0 msgr2=0x7f1bdc154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc15aec0 0x7f1bdc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 -- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bdc15b2a0 msgr2=0x7f1bdc1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.374+0000 7f1bdbe00640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1bdbe00640 1 -- 172.21.15.91:0/2655395321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc40094d0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1bdbe00640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00c7f0 tx=0x7f1bcc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004220 con 0x7f1bdc06cb90 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0043a0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc007860 con 0x7f1bdc06cb90 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc154e60 con 0x7f1bdc06cb90 2024-02-02T13:37:11.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.375+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc1554c0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.376+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bdc0b14f0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.377+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1bcc007bd0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.378+0000 7f1bd8c00640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 0x7f1bb406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.378+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bcc088cf0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.379+0000 7f1bd3200640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 0x7f1bb406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.379+0000 7f1bd3200640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 0x7f1bb406c8f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4036000 tx=0x7f1bc403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.382+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1bcc054350 con 0x7f1bdc06cb90 2024-02-02T13:37:11.575 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.574+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bdc1556b0 con 0x7f1bdc06cb90 2024-02-02T13:37:11.578 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.576+0000 7f1bd8c00640 1 -- 172.21.15.91:0/2655395321 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16117 (secure 0 0 0) 0x7f1bcc057970 con 0x7f1bdc06cb90 2024-02-02T13:37:11.578 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:11.578 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":194,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:10.718728+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:11.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 msgr2=0x7f1bb406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 0x7f1bb406c8f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4036000 tx=0x7f1bc403b040 comp rx=0 tx=0).stop 2024-02-02T13:37:11.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 msgr2=0x7f1bdc153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00c7f0 tx=0x7f1bcc00ccb0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 shutdown_connections 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1bb406a430 0x7f1bb406c8f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bdc15aec0 0x7f1bdc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bdc15b2a0 0x7f1bdc1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 --2- 172.21.15.91:0/2655395321 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1bdc06cb90 0x7f1bdc153b30 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.579+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 >> 172.21.15.91:0/2655395321 conn(0x7f1bdc0cea90 msgr2=0x7f1bdc0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.580+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 shutdown_connections 2024-02-02T13:37:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.580+0000 7f1be1400640 1 -- 172.21.15.91:0/2655395321 wait complete. 2024-02-02T13:37:11.600 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T13:37:11.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.806+0000 7f1ef8a00640 1 Processor -- start 2024-02-02T13:37:11.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.806+0000 7f1ef8a00640 1 -- start start 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.807+0000 7f1ef8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ef00b0da0 0x7f1ef00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.807+0000 7f1ef8a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00b1e60 con 0x7f1ef00b1a00 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.807+0000 7f1ef8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef015f1b0 con 0x7f1ef015b2f0 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.807+0000 7f1ef8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef015fdc0 con 0x7f1ef00b0da0 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef5600640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1ef00b1a00 0x7f1ef00b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51502/0 (socket says 172.21.15.91:51502) 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef5600640 1 -- 172.21.15.91:0/1897976804 learned_addr learned my addr 172.21.15.91:0/1897976804 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093750488 0 0) 0x7f1ef00b1e60 con 0x7f1ef00b1a00 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed0003520 con 0x7f1ef00b1a00 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef6a00640 1 --2- 172.21.15.91:0/1897976804 >> v2:172.21.15.113:3300/0 conn(0x7f1ef00b0da0 0x7f1ef00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.808+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200113845 0 0) 0x7f1ed0003520 con 0x7f1ef00b1a00 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 >> v2:172.21.15.113:3300/0 conn(0x7f1ef00b0da0 msgr2=0x7f1ef00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 --2- 172.21.15.91:0/1897976804 >> v2:172.21.15.113:3300/0 conn(0x7f1ef00b0da0 0x7f1ef00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 >> v1:172.21.15.113:6789/0 conn(0x7f1ef015b2f0 legacy=0x7f1ef00711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.811 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef015f490 con 0x7f1ef00b1a00 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1ed8001670 con 0x7f1ef00b1a00 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 >> v1:172.21.15.91:6789/0 conn(0x7f1ef00b1a00 legacy=0x7f1ef00b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed0003970 0x7f1ed0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ed0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ed00052a0 0x7f1ed000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef015fdc0 con 0x7f1ed00047c0 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed000c150 con 0x7f1ed0003970 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed000c310 con 0x7f1ed00052a0 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ed0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 -- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ed00052a0 msgr2=0x7f1ed000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ed00052a0 0x7f1ed000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 -- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed0003970 msgr2=0x7f1ed0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed0003970 0x7f1ed0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef5600640 1 -- 172.21.15.91:0/1897976804 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0003520 con 0x7f1ed00047c0 2024-02-02T13:37:11.813 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.809+0000 7f1ef6a00640 1 -- 172.21.15.91:0/1897976804 reap_dead start 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.810+0000 7f1ef5600640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ed0004c40 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001eb0 tx=0x7f1ed00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7157368e645e6ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.810+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005ad0 con 0x7f1ed00047c0 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8005c50 con 0x7f1ed00047c0 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef4c00640 1 -- 172.21.15.91:0/1897976804 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed802c890 con 0x7f1ed00047c0 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 -- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 msgr2=0x7f1ed0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ed0004c40 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001eb0 tx=0x7f1ed00057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 -- 172.21.15.91:0/1897976804 shutdown_connections 2024-02-02T13:37:11.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/1897976804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ed0004c40 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 -- 172.21.15.91:0/1897976804 >> 172.21.15.91:0/1897976804 conn(0x7f1ef00ceb90 msgr2=0x7f1ef00cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:11.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 -- 172.21.15.91:0/1897976804 shutdown_connections 2024-02-02T13:37:11.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.811+0000 7f1ef8a00640 1 -- 172.21.15.91:0/1897976804 wait complete. 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.813+0000 7f1ef8a00640 1 Processor -- start 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.813+0000 7f1ef8a00640 1 -- start start 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ef0157290 0x7f1ef00c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef00c2f70 0x7f1ef00c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c6420 con 0x7f1ed00047c0 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c6700 con 0x7f1ef00c2f70 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c69e0 con 0x7f1ef0157290 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef6000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41494/0 (socket says 172.21.15.91:41494) 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.814+0000 7f1ef6000640 1 -- 172.21.15.91:0/4281886513 learned_addr learned my addr 172.21.15.91:0/4281886513 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:11.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 -- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ef0157290 msgr2=0x7f1ef00c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ef0157290 0x7f1ef00c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 -- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef00c2f70 msgr2=0x7f1ef00c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef00c2f70 0x7f1ef00c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 -- 172.21.15.91:0/4281886513 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00c6cc0 con 0x7f1ed00047c0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef6000640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c3c0 tx=0x7f1ee000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0011790 con 0x7f1ed00047c0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee00118f0 con 0x7f1ed00047c0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0011ce0 con 0x7f1ed00047c0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.815+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00c6fa0 con 0x7f1ed00047c0 2024-02-02T13:37:11.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.816+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef0157710 con 0x7f1ed00047c0 2024-02-02T13:37:11.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.816+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef00c7190 con 0x7f1ed00047c0 2024-02-02T13:37:11.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.817+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1ee001c460 con 0x7f1ed00047c0 2024-02-02T13:37:11.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.818+0000 7f1eee000640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 0x7f1ec806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:11.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.818+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ee0088880 con 0x7f1ed00047c0 2024-02-02T13:37:11.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.819+0000 7f1ef5600640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 0x7f1ec806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:11.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.819+0000 7f1ef5600640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 0x7f1ec806c8f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8034000 tx=0x7f1ed8039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:11.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:11.823+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1ee0053e60 con 0x7f1ed00047c0 2024-02-02T13:37:12.042 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.041+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1ef0066cf0 con 0x7f1ed00047c0 2024-02-02T13:37:12.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.742+0000 7f1eee000640 1 -- 172.21.15.91:0/4281886513 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ee0057480 con 0x7f1ed00047c0 2024-02-02T13:37:12.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 msgr2=0x7f1ec806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:12.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 0x7f1ec806c8f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8034000 tx=0x7f1ed8039040 comp rx=0 tx=0).stop 2024-02-02T13:37:12.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 msgr2=0x7f1ef00c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:12.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c3c0 tx=0x7f1ee000c7c0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 shutdown_connections 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1ec806a430 0x7f1ec806c8f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ef0157290 0x7f1ef00c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef00c2f70 0x7f1ef00c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 --2- 172.21.15.91:0/4281886513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ed00047c0 0x7f1ef00c5bd0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 >> 172.21.15.91:0/4281886513 conn(0x7f1ef00ceb90 msgr2=0x7f1ef015a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 shutdown_connections 2024-02-02T13:37:12.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.746+0000 7f1ef8a00640 1 -- 172.21.15.91:0/4281886513 wait complete. 2024-02-02T13:37:12.769 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-02T13:37:12.769 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-02T13:37:12.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.981+0000 7f39fdc00640 1 Processor -- start 2024-02-02T13:37:12.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.981+0000 7f39fdc00640 1 -- start start 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.982+0000 7f39fdc00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39f8159c50 0x7f39f80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.982+0000 7f39fdc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39f806cce0 0x7f39f815db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.982+0000 7f39fdc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80ca740 con 0x7f39f806fc00 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.982+0000 7f39fdc00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f815d710 con 0x7f39f8159c50 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.982+0000 7f39fdc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f80cc340 con 0x7f39f806cce0 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39f8159c50 0x7f39f80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f39f8159c50 0x7f39f80cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41502/0 (socket says 172.21.15.91:41502) 2024-02-02T13:37:12.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 -- 172.21.15.91:0/2100814991 learned_addr learned my addr 172.21.15.91:0/2100814991 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:12.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 -- 172.21.15.91:0/2100814991 >> v2:172.21.15.113:3300/0 conn(0x7f39f806cce0 msgr2=0x7f39f815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:12.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 --2- 172.21.15.91:0/2100814991 >> v2:172.21.15.113:3300/0 conn(0x7f39f806cce0 0x7f39f815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 -- 172.21.15.91:0/2100814991 >> v1:172.21.15.113:6789/0 conn(0x7f39f806fc00 legacy=0x7f39f806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:12.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.983+0000 7f39f7e00640 1 -- 172.21.15.91:0/2100814991 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f815a030 con 0x7f39f8159c50 2024-02-02T13:37:12.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.984+0000 7f39f7e00640 1 --2- 172.21.15.91:0/2100814991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f8159c50 0x7f39f80cbe00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f39f806f220 tx=0x7f39e802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae94bbb29334125 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:12.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39f7400640 1 -- 172.21.15.91:0/2100814991 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8031890 con 0x7f39f8159c50 2024-02-02T13:37:12.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39f7400640 1 -- 172.21.15.91:0/2100814991 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e80319f0 con 0x7f39f8159c50 2024-02-02T13:37:12.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39f7400640 1 -- 172.21.15.91:0/2100814991 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8031d80 con 0x7f39f8159c50 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 -- 172.21.15.91:0/2100814991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f8159c50 msgr2=0x7f39f80cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 --2- 172.21.15.91:0/2100814991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f8159c50 0x7f39f80cbe00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f39f806f220 tx=0x7f39e802b930 comp rx=0 tx=0).stop 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 -- 172.21.15.91:0/2100814991 shutdown_connections 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 --2- 172.21.15.91:0/2100814991 >> v2:172.21.15.113:3300/0 conn(0x7f39f806cce0 0x7f39f815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 --2- 172.21.15.91:0/2100814991 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f8159c50 0x7f39f80cbe00 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.985+0000 7f39fdc00640 1 -- 172.21.15.91:0/2100814991 >> 172.21.15.91:0/2100814991 conn(0x7f39f80b0560 msgr2=0x7f39f80b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:12.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.986+0000 7f39fdc00640 1 -- 172.21.15.91:0/2100814991 shutdown_connections 2024-02-02T13:37:12.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.986+0000 7f39fdc00640 1 -- 172.21.15.91:0/2100814991 wait complete. 2024-02-02T13:37:12.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.988+0000 7f39fdc00640 1 Processor -- start 2024-02-02T13:37:12.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.988+0000 7f39fdc00640 1 -- start start 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39f806cce0 0x7f39f814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 0x7f39f8149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80c8e60 con 0x7f39f806fc00 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80c8040 con 0x7f39f8159c50 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39fdc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8149740 con 0x7f39f806cce0 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39ef200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39ef200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41506/0 (socket says 172.21.15.91:41506) 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.989+0000 7f39ef200640 1 -- 172.21.15.91:0/1268427567 learned_addr learned my addr 172.21.15.91:0/1268427567 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39f7e00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39f806cce0 0x7f39f814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39fd200640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 0x7f39f8149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:12.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 -- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39f806cce0 msgr2=0x7f39f814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39f806cce0 0x7f39f814c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 -- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 msgr2=0x7f39f8149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 0x7f39f8149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 -- 172.21.15.91:0/1268427567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80094d0 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39fd200640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 0x7f39f8149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39ef200640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c410 tx=0x7f39e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e00035f0 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0003750 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0003b40 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.990+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f80c82e0 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.991+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f80c88a0 con 0x7f39f806fc00 2024-02-02T13:37:12.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.992+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8149ae0 con 0x7f39f806fc00 2024-02-02T13:37:12.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.993+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f39e0003ca0 con 0x7f39f806fc00 2024-02-02T13:37:12.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.994+0000 7f39f4c00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 0x7f39d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:12.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.994+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39e0087b50 con 0x7f39f806fc00 2024-02-02T13:37:12.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.994+0000 7f39f7e00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 0x7f39d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:12.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.995+0000 7f39f7e00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 0x7f39d006c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f39e8036000 tx=0x7f39e803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:13.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:12.999+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f39f8149ae0 con 0x7f39f806fc00 2024-02-02T13:37:13.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.226+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f39f8149ae0 con 0x7f39f806fc00 2024-02-02T13:37:13.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.745+0000 7f39f4c00640 1 -- 172.21.15.91:0/1268427567 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f39f8149ae0 con 0x7f39f806fc00 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.748+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 msgr2=0x7f39d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.748+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 0x7f39d006c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f39e8036000 tx=0x7f39e803c040 comp rx=0 tx=0).stop 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.748+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 msgr2=0x7f39f80c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.748+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c410 tx=0x7f39e000c8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 shutdown_connections 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f39d006a430 0x7f39d006c8f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39f8159c50 0x7f39f8149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39f806fc00 0x7f39f80c7b00 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 --2- 172.21.15.91:0/1268427567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39f806cce0 0x7f39f814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 >> 172.21.15.91:0/1268427567 conn(0x7f39f80b0560 msgr2=0x7f39f80ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 shutdown_connections 2024-02-02T13:37:13.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.749+0000 7f39fdc00640 1 -- 172.21.15.91:0/1268427567 wait complete. 2024-02-02T13:37:13.771 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-02T13:37:13.771 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:13.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.979+0000 7fbff3600640 1 Processor -- start 2024-02-02T13:37:13.981 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.979+0000 7fbff3600640 1 -- start start 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff3600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff3600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec070f80 con 0x7fbfec151f90 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff3600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec06c600 con 0x7fbfec15f5a0 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff3600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15fa30 con 0x7fbfec0b0e10 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff0c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fbfec151f90 0x7fbfec06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51510/0 (socket says 172.21.15.91:51510) 2024-02-02T13:37:13.982 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff0c00640 1 -- 172.21.15.91:0/2825338087 learned_addr learned my addr 172.21.15.91:0/2825338087 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:13.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.981+0000 7fbff1600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417562017 0 0) 0x7fbfec070f80 con 0x7fbfec151f90 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc003520 con 0x7fbfec151f90 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4039237556 0 0) 0x7fbfcc003520 con 0x7fbfec151f90 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 msgr2=0x7fbfec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfec0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 >> v1:172.21.15.113:6789/0 conn(0x7fbfec15f5a0 legacy=0x7fbfec0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec152370 con 0x7fbfec151f90 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fbfdc001670 con 0x7fbfec151f90 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 >> v1:172.21.15.91:6789/0 conn(0x7fbfec151f90 legacy=0x7fbfec06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfcc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfcc003f20 0x7fbfcc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfcc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15fa30 con 0x7fbfcc004d00 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc00ba60 con 0x7fbfcc003f20 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc00bc20 con 0x7fbfec0b0e10 2024-02-02T13:37:13.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.982+0000 7fbff1600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfcc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff0c00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfcc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfcc003f20 0x7fbfcc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff1600640 1 -- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 msgr2=0x7fbfcc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff1600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec0b0e10 0x7fbfcc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff1600640 1 -- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfcc003f20 msgr2=0x7fbfcc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff1600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfcc003f20 0x7fbfcc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbff1600640 1 -- 172.21.15.91:0/2825338087 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc003520 con 0x7fbfcc004d00 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.983+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfcc003f20 0x7fbfcc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.984+0000 7fbff1600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfcc005180 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0ca540 tx=0x7fbfcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176b83fec9a667a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.984+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe402cb90 con 0x7fbfcc004d00 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.984+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe400f040 con 0x7fbfcc004d00 2024-02-02T13:37:13.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.984+0000 7fbfebe00640 1 -- 172.21.15.91:0/2825338087 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4034440 con 0x7fbfcc004d00 2024-02-02T13:37:13.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 -- 172.21.15.91:0/2825338087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 msgr2=0x7fbfcc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 --2- 172.21.15.91:0/2825338087 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfcc005180 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0ca540 tx=0x7fbfcc0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff1600640 1 -- 172.21.15.91:0/2825338087 reap_dead start 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 -- 172.21.15.91:0/2825338087 shutdown_connections 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 -- 172.21.15.91:0/2825338087 >> 172.21.15.91:0/2825338087 conn(0x7fbfec0cec80 msgr2=0x7fbfec0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 -- 172.21.15.91:0/2825338087 shutdown_connections 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.985+0000 7fbff3600640 1 -- 172.21.15.91:0/2825338087 wait complete. 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.987+0000 7fbff3600640 1 Processor -- start 2024-02-02T13:37:13.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.987+0000 7fbff3600640 1 -- start start 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 0x7fbfec15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec15e210 0x7fbfec150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1f6210 con 0x7fbfcc004d00 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1550a0 con 0x7fbfec0c9fc0 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff3600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0c2720 con 0x7fbfec15e210 2024-02-02T13:37:13.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41522/0 (socket says 172.21.15.91:41522) 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 -- 172.21.15.91:0/2680696459 learned_addr learned my addr 172.21.15.91:0/2680696459 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 0x7fbfec15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 -- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec15e210 msgr2=0x7fbfec150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec15e210 0x7fbfec150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 -- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 msgr2=0x7fbfec15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 0x7fbfec15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbff0c00640 1 -- 172.21.15.91:0/2680696459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec0c2970 con 0x7fbfcc004d00 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.988+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 0x7fbfec15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbff0c00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00e870 tx=0x7fbfdc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00c870 con 0x7fbfcc004d00 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc00c9d0 con 0x7fbfcc004d00 2024-02-02T13:37:13.992 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00cd80 con 0x7fbfcc004d00 2024-02-02T13:37:13.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec0c2c50 con 0x7fbfcc004d00 2024-02-02T13:37:13.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.989+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec0c3210 con 0x7fbfcc004d00 2024-02-02T13:37:13.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.991+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec066cf0 con 0x7fbfcc004d00 2024-02-02T13:37:13.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.991+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbfdc013420 con 0x7fbfcc004d00 2024-02-02T13:37:13.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.992+0000 7fbfe9600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 0x7fbfc406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.993+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfdc088740 con 0x7fbfcc004d00 2024-02-02T13:37:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.993+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 0x7fbfc406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.993+0000 7fbfe3e00640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 0x7fbfc406c920 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0ca910 tx=0x7fbfd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:13.998 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:13.997+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbfdc053d50 con 0x7fbfcc004d00 2024-02-02T13:37:14.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.230+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfec0714d0 con 0x7fbfcc004d00 2024-02-02T13:37:14.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.231+0000 7fbfe9600640 1 -- 172.21.15.91:0/2680696459 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fbfdc057370 con 0x7fbfcc004d00 2024-02-02T13:37:14.232 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:14.232 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:14.233 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.233+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 msgr2=0x7fbfc406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.233+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 0x7fbfc406c920 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0ca910 tx=0x7fbfd4009040 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.233+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 msgr2=0x7fbfec1f2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.233+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00e870 tx=0x7fbfdc00ed30 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.233+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 shutdown_connections 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fbfc406a460 0x7fbfc406c920 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfec15e210 0x7fbfec150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfec0c9fc0 0x7fbfec15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 --2- 172.21.15.91:0/2680696459 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfcc004d00 0x7fbfec1f2b90 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.236 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 >> 172.21.15.91:0/2680696459 conn(0x7fbfec0cec80 msgr2=0x7fbfec0cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:14.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 shutdown_connections 2024-02-02T13:37:14.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.234+0000 7fbff3600640 1 -- 172.21.15.91:0/2680696459 wait complete. 2024-02-02T13:37:14.253 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.445+0000 7f1997e00640 1 Processor -- start 2024-02-02T13:37:14.446 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.445+0000 7f1997e00640 1 -- start start 2024-02-02T13:37:14.448 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.446+0000 7f1997e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19980cad30 0x7f19980cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.446+0000 7f1997e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199806c1c0 con 0x7f199815b2a0 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.446+0000 7f1997e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199806b5f0 con 0x7f199815aec0 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.446+0000 7f1997e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199806cd60 con 0x7f19980cad30 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.446+0000 7f1997400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19980cad30 0x7f19980cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f198de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f199815b2a0 0x7f19980ca620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51524/0 (socket says 172.21.15.91:51524) 2024-02-02T13:37:14.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f198de00640 1 -- 172.21.15.91:0/2976837581 learned_addr learned my addr 172.21.15.91:0/2976837581 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f1996000640 1 -- 172.21.15.91:0/2976837581 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098725676 0 0) 0x7f199806c1c0 con 0x7f199815b2a0 2024-02-02T13:37:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f1997400640 1 -- 172.21.15.91:0/2976837581 >> v1:172.21.15.113:6789/0 conn(0x7f199815aec0 legacy=0x7f199806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f1997400640 1 -- 172.21.15.91:0/2976837581 >> v1:172.21.15.91:6789/0 conn(0x7f199815b2a0 legacy=0x7f19980ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.447+0000 7f1997400640 1 -- 172.21.15.91:0/2976837581 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199806b890 con 0x7f19980cad30 2024-02-02T13:37:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.448+0000 7f1997400640 1 --2- 172.21.15.91:0/2976837581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19980cf5b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f199002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bc9c3cda79ba0c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.451 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1996000640 1 -- 172.21.15.91:0/2976837581 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199002cd00 con 0x7f19980cad30 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1996000640 1 -- 172.21.15.91:0/2976837581 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199002ce60 con 0x7f19980cad30 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1996000640 1 -- 172.21.15.91:0/2976837581 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990034660 con 0x7f19980cad30 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 -- 172.21.15.91:0/2976837581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 msgr2=0x7f19980cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 --2- 172.21.15.91:0/2976837581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19980cf5b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f199002bf50 comp rx=0 tx=0).stop 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 -- 172.21.15.91:0/2976837581 shutdown_connections 2024-02-02T13:37:14.452 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 --2- 172.21.15.91:0/2976837581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19980cf5b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 -- 172.21.15.91:0/2976837581 >> 172.21.15.91:0/2976837581 conn(0x7f19980cea90 msgr2=0x7f19980ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 -- 172.21.15.91:0/2976837581 shutdown_connections 2024-02-02T13:37:14.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.449+0000 7f1997e00640 1 -- 172.21.15.91:0/2976837581 wait complete. 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.451+0000 7f1997e00640 1 Processor -- start 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.451+0000 7f1997e00640 1 -- start start 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f199815aec0 0x7f19981f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f199815b2a0 0x7f19981f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c3220 con 0x7f19980cad30 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c25a0 con 0x7f199815aec0 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1997e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c2880 con 0x7f199815b2a0 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1996a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1996a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41546/0 (socket says 172.21.15.91:41546) 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.452+0000 7f1996a00640 1 -- 172.21.15.91:0/3274319559 learned_addr learned my addr 172.21.15.91:0/3274319559 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 -- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f199815b2a0 msgr2=0x7f19981f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f199815b2a0 0x7f19981f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 -- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f199815aec0 msgr2=0x7f19981f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.455 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f199815aec0 0x7f19981f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 -- 172.21.15.91:0/3274319559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990009a70 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1996a00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f198800c430 tx=0x7f198800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988011790 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19880118f0 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19980c2b60 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988011ce0 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.453+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19980c3640 con 0x7f19980cad30 2024-02-02T13:37:14.456 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.454+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19980c3360 con 0x7f19980cad30 2024-02-02T13:37:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.455+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f198801c460 con 0x7f19980cad30 2024-02-02T13:37:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.456+0000 7f198f400640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 0x7f197006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.456+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1988088880 con 0x7f19980cad30 2024-02-02T13:37:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.456+0000 7f198de00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 0x7f197006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.457+0000 7f198de00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 0x7f197006c8f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1980007fd0 tx=0x7f1980007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.460+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f19980c3360 con 0x7f19980cad30 2024-02-02T13:37:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.638+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19980c3360 con 0x7f19980cad30 2024-02-02T13:37:14.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.641+0000 7f198f400640 1 -- 172.21.15.91:0/3274319559 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16104 (secure 0 0 0) 0x7f1988053e60 con 0x7f19980cad30 2024-02-02T13:37:14.643 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:14.643 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":195,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:11.723921+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:14.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 msgr2=0x7f197006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 0x7f197006c8f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1980007fd0 tx=0x7f1980007960 comp rx=0 tx=0).stop 2024-02-02T13:37:14.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 msgr2=0x7f19981588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f198800c430 tx=0x7f198800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 shutdown_connections 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f197006a430 0x7f197006c8f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f199815b2a0 0x7f19981f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f199815aec0 0x7f19981f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 --2- 172.21.15.91:0/3274319559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19980cad30 0x7f19981588a0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 >> 172.21.15.91:0/3274319559 conn(0x7f19980cea90 msgr2=0x7f199806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 shutdown_connections 2024-02-02T13:37:14.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.644+0000 7f1997e00640 1 -- 172.21.15.91:0/3274319559 wait complete. 2024-02-02T13:37:14.663 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T13:37:14.663 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:37:14.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.867+0000 7ff333a00640 1 Processor -- start 2024-02-02T13:37:14.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.867+0000 7ff333a00640 1 -- start start 2024-02-02T13:37:14.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff333a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff32c0b2260 0x7ff32c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff333a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 0x7ff32c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff333a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c0cd930 con 0x7ff32c0d0070 2024-02-02T13:37:14.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff333a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0b8f90 con 0x7ff32c0b2260 2024-02-02T13:37:14.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff333a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0d1680 con 0x7ff32c0b6260 2024-02-02T13:37:14.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff331000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff32c0d0070 0x7ff32c0d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51534/0 (socket says 172.21.15.91:51534) 2024-02-02T13:37:14.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.868+0000 7ff331000640 1 -- 172.21.15.91:0/697213402 learned_addr learned my addr 172.21.15.91:0/697213402 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:14.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494197117 0 0) 0x7ff31c001490 con 0x7ff32c0d0070 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff30c003520 con 0x7ff32c0d0070 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff331a00640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 0x7ff32c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32be00640 1 --2- 172.21.15.91:0/697213402 >> v2:172.21.15.113:3300/0 conn(0x7ff32c0b2260 0x7ff32c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117639568 0 0) 0x7ff30c003520 con 0x7ff32c0d0070 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 msgr2=0x7ff32c0b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 0x7ff32c0b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 >> v2:172.21.15.113:3300/0 conn(0x7ff32c0b2260 msgr2=0x7ff32c0b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 --2- 172.21.15.91:0/697213402 >> v2:172.21.15.113:3300/0 conn(0x7ff32c0b2260 0x7ff32c0b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0d1960 con 0x7ff32c0d0070 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff31c001b80 con 0x7ff32c0d0070 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 >> v1:172.21.15.91:6789/0 conn(0x7ff32c0d0070 legacy=0x7ff32c0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff30c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 0x7ff30c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c004d00 0x7ff30c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0b8f90 con 0x7ff30c003970 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0d1680 con 0x7ff30c004d00 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.869+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c00ba30 con 0x7ff32c0b6260 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff30c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 -- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 msgr2=0x7ff30c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c0b6260 0x7ff30c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 -- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c004d00 msgr2=0x7ff30c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c004d00 0x7ff30c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.870+0000 7ff331000640 1 -- 172.21.15.91:0/697213402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c003520 con 0x7ff30c003970 2024-02-02T13:37:14.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.871+0000 7ff331000640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff30c003d90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a860 tx=0x7ff30c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6edb84f060b8342 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.871+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c034630 con 0x7ff30c003970 2024-02-02T13:37:14.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.871+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c02f070 con 0x7ff30c003970 2024-02-02T13:37:14.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.871+0000 7ff32b400640 1 -- 172.21.15.91:0/697213402 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c034990 con 0x7ff30c003970 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 -- 172.21.15.91:0/697213402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 msgr2=0x7ff30c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 --2- 172.21.15.91:0/697213402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff30c003d90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a860 tx=0x7ff30c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff331a00640 1 -- 172.21.15.91:0/697213402 reap_dead start 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 -- 172.21.15.91:0/697213402 shutdown_connections 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 -- 172.21.15.91:0/697213402 >> 172.21.15.91:0/697213402 conn(0x7ff32c070660 msgr2=0x7ff32c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 -- 172.21.15.91:0/697213402 shutdown_connections 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.872+0000 7ff333a00640 1 -- 172.21.15.91:0/697213402 wait complete. 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.874+0000 7ff333a00640 1 Processor -- start 2024-02-02T13:37:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.874+0000 7ff333a00640 1 -- start start 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c157190 0x7ff32c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff32c0cbbf0 0x7ff32c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0ca200 con 0x7ff30c003970 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0cc4d0 con 0x7ff32c0cbbf0 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff333a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0cc7b0 con 0x7ff32c157190 2024-02-02T13:37:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff331000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff331000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41564/0 (socket says 172.21.15.91:41564) 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.875+0000 7ff331000640 1 -- 172.21.15.91:0/3175867640 learned_addr learned my addr 172.21.15.91:0/3175867640 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 -- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c157190 msgr2=0x7ff32c0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c157190 0x7ff32c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 -- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff32c0cbbf0 msgr2=0x7ff32c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff32c0cbbf0 0x7ff32c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 -- 172.21.15.91:0/3175867640 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0ca450 con 0x7ff30c003970 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff331000640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a860 tx=0x7ff31c001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c043070 con 0x7ff30c003970 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c004b60 con 0x7ff30c003970 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c005050 con 0x7ff30c003970 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.876+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c0ca730 con 0x7ff30c003970 2024-02-02T13:37:14.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.877+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c0cb190 con 0x7ff30c003970 2024-02-02T13:37:14.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.878+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0ca8c0 con 0x7ff30c003970 2024-02-02T13:37:14.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.878+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff31c031030 con 0x7ff30c003970 2024-02-02T13:37:14.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.879+0000 7ff328c00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 0x7ff30406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:14.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.879+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff31c0aa280 con 0x7ff30c003970 2024-02-02T13:37:14.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.879+0000 7ff32be00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 0x7ff30406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:14.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.880+0000 7ff32be00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 0x7ff30406c920 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff314006e80 tx=0x7ff314008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:14.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:14.884+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff31c075880 con 0x7ff30c003970 2024-02-02T13:37:15.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.069+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff32c0d1f20 con 0x7ff30406a460 2024-02-02T13:37:15.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.093+0000 7ff328c00640 1 -- 172.21.15.91:0/3175867640 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283030 (secure 0 0 0) 0x7ff314009d60 con 0x7ff30406a460 2024-02-02T13:37:15.098 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:15.100 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:37:15.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 msgr2=0x7ff30406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 0x7ff30406c920 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff314006e80 tx=0x7ff314008040 comp rx=0 tx=0).stop 2024-02-02T13:37:15.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 msgr2=0x7ff32c156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a860 tx=0x7ff31c001f10 comp rx=0 tx=0).stop 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 shutdown_connections 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff30406a460 0x7ff30406c920 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff32c157190 0x7ff32c0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff32c0cbbf0 0x7ff32c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 --2- 172.21.15.91:0/3175867640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff30c003970 0x7ff32c156920 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 >> 172.21.15.91:0/3175867640 conn(0x7ff32c070660 msgr2=0x7ff32c160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:15.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 shutdown_connections 2024-02-02T13:37:15.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.098+0000 7ff333a00640 1 -- 172.21.15.91:0/3175867640 wait complete. 2024-02-02T13:37:15.118 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-02-02T13:37:14.494499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552841,"statfs":{"total":1600000000,"available":1590109261,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.261026"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758980+0000","last_change":"2024-02-02T13:37:10.757647+0000","last_active":"2024-02-02T13:37:10.758980+0000","last_peered":"2024-02-02T13:37:10.758980+0000","last_clean":"2024-02-02T13:37:10.758980+0000","last_became_active":"2024-02-02T13:37:10.757412+0000","last_became_peered":"2024-02-02T13:37:10.757412+0000","last_unstale":"2024-02-02T13:37:10.758980+0000","last_undegraded":"2024-02-02T13:37:10.758980+0000","last_fullsized":"2024-02-02T13:37:10.758980+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:49.483601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761642+0000","last_change":"2024-02-02T13:37:10.761288+0000","last_active":"2024-02-02T13:37:10.761642+0000","last_peered":"2024-02-02T13:37:10.761642+0000","last_clean":"2024-02-02T13:37:10.761642+0000","last_became_active":"2024-02-02T13:37:10.761000+0000","last_became_peered":"2024-02-02T13:37:10.761000+0000","last_unstale":"2024-02-02T13:37:10.761642+0000","last_undegraded":"2024-02-02T13:37:10.761642+0000","last_fullsized":"2024-02-02T13:37:10.761642+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:04.144171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763219+0000","last_change":"2024-02-02T13:37:10.759825+0000","last_active":"2024-02-02T13:37:10.763219+0000","last_peered":"2024-02-02T13:37:10.763219+0000","last_clean":"2024-02-02T13:37:10.763219+0000","last_became_active":"2024-02-02T13:37:10.759566+0000","last_became_peered":"2024-02-02T13:37:10.759566+0000","last_unstale":"2024-02-02T13:37:10.763219+0000","last_undegraded":"2024-02-02T13:37:10.763219+0000","last_fullsized":"2024-02-02T13:37:10.763219+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:42.207188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737151+0000","last_change":"2024-02-02T13:37:10.736637+0000","last_active":"2024-02-02T13:37:10.737151+0000","last_peered":"2024-02-02T13:37:10.737151+0000","last_clean":"2024-02-02T13:37:10.737151+0000","last_became_active":"2024-02-02T13:37:10.736317+0000","last_became_peered":"2024-02-02T13:37:10.736317+0000","last_unstale":"2024-02-02T13:37:10.737151+0000","last_undegraded":"2024-02-02T13:37:10.737151+0000","last_fullsized":"2024-02-02T13:37:10.737151+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:02.995879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752828+0000","last_change":"2024-02-02T13:37:10.751108+0000","last_active":"2024-02-02T13:37:10.752828+0000","last_peered":"2024-02-02T13:37:10.752828+0000","last_clean":"2024-02-02T13:37:10.752828+0000","last_became_active":"2024-02-02T13:37:10.750717+0000","last_became_peered":"2024-02-02T13:37:10.750717+0000","last_unstale":"2024-02-02T13:37:10.752828+0000","last_undegraded":"2024-02-02T13:37:10.752828+0000","last_fullsized":"2024-02-02T13:37:10.752828+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:17:41.702140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743123+0000","last_change":"2024-02-02T13:37:10.742550+0000","last_active":"2024-02-02T13:37:10.743123+0000","last_peered":"2024-02-02T13:37:10.743123+0000","last_clean":"2024-02-02T13:37:10.743123+0000","last_became_active":"2024-02-02T13:37:10.742321+0000","last_became_peered":"2024-02-02T13:37:10.742321+0000","last_unstale":"2024-02-02T13:37:10.743123+0000","last_undegraded":"2024-02-02T13:37:10.743123+0000","last_fullsized":"2024-02-02T13:37:10.743123+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:44.853948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741354+0000","last_change":"2024-02-02T13:37:10.741216+0000","last_active":"2024-02-02T13:37:10.741354+0000","last_peered":"2024-02-02T13:37:10.741354+0000","last_clean":"2024-02-02T13:37:10.741354+0000","last_became_active":"2024-02-02T13:37:10.741007+0000","last_became_peered":"2024-02-02T13:37:10.741007+0000","last_unstale":"2024-02-02T13:37:10.741354+0000","last_undegraded":"2024-02-02T13:37:10.741354+0000","last_fullsized":"2024-02-02T13:37:10.741354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:01:55.569563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.756625+0000","last_change":"2024-02-02T13:37:10.755805+0000","last_active":"2024-02-02T13:37:10.756625+0000","last_peered":"2024-02-02T13:37:10.756625+0000","last_clean":"2024-02-02T13:37:10.756625+0000","last_became_active":"2024-02-02T13:37:10.755547+0000","last_became_peered":"2024-02-02T13:37:10.755547+0000","last_unstale":"2024-02-02T13:37:10.756625+0000","last_undegraded":"2024-02-02T13:37:10.756625+0000","last_fullsized":"2024-02-02T13:37:10.756625+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:37.771425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.757655+0000","last_change":"2024-02-02T13:37:10.757526+0000","last_active":"2024-02-02T13:37:10.757655+0000","last_peered":"2024-02-02T13:37:10.757655+0000","last_clean":"2024-02-02T13:37:10.757655+0000","last_became_active":"2024-02-02T13:37:10.757090+0000","last_became_peered":"2024-02-02T13:37:10.757090+0000","last_unstale":"2024-02-02T13:37:10.757655+0000","last_undegraded":"2024-02-02T13:37:10.757655+0000","last_fullsized":"2024-02-02T13:37:10.757655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:09:52.578652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752792+0000","last_change":"2024-02-02T13:37:10.750964+0000","last_active":"2024-02-02T13:37:10.752792+0000","last_peered":"2024-02-02T13:37:10.752792+0000","last_clean":"2024-02-02T13:37:10.752792+0000","last_became_active":"2024-02-02T13:37:10.750365+0000","last_became_peered":"2024-02-02T13:37:10.750365+0000","last_unstale":"2024-02-02T13:37:10.752792+0000","last_undegraded":"2024-02-02T13:37:10.752792+0000","last_fullsized":"2024-02-02T13:37:10.752792+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:23.943367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755900+0000","last_change":"2024-02-02T13:37:10.755826+0000","last_active":"2024-02-02T13:37:10.755900+0000","last_peered":"2024-02-02T13:37:10.755900+0000","last_clean":"2024-02-02T13:37:10.755900+0000","last_became_active":"2024-02-02T13:37:10.755564+0000","last_became_peered":"2024-02-02T13:37:10.755564+0000","last_unstale":"2024-02-02T13:37:10.755900+0000","last_undegraded":"2024-02-02T13:37:10.755900+0000","last_fullsized":"2024-02-02T13:37:10.755900+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:29.390021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758450+0000","last_change":"2024-02-02T13:37:10.758289+0000","last_active":"2024-02-02T13:37:10.758450+0000","last_peered":"2024-02-02T13:37:10.758450+0000","last_clean":"2024-02-02T13:37:10.758450+0000","last_became_active":"2024-02-02T13:37:10.758057+0000","last_became_peered":"2024-02-02T13:37:10.758057+0000","last_unstale":"2024-02-02T13:37:10.758450+0000","last_undegraded":"2024-02-02T13:37:10.758450+0000","last_fullsized":"2024-02-02T13:37:10.758450+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:12:20.100581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763053+0000","last_change":"2024-02-02T13:37:10.758831+0000","last_active":"2024-02-02T13:37:10.763053+0000","last_peered":"2024-02-02T13:37:10.763053+0000","last_clean":"2024-02-02T13:37:10.763053+0000","last_became_active":"2024-02-02T13:37:10.752456+0000","last_became_peered":"2024-02-02T13:37:10.752456+0000","last_unstale":"2024-02-02T13:37:10.763053+0000","last_undegraded":"2024-02-02T13:37:10.763053+0000","last_fullsized":"2024-02-02T13:37:10.763053+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:23.309621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763090+0000","last_change":"2024-02-02T13:37:10.759032+0000","last_active":"2024-02-02T13:37:10.763090+0000","last_peered":"2024-02-02T13:37:10.763090+0000","last_clean":"2024-02-02T13:37:10.763090+0000","last_became_active":"2024-02-02T13:37:10.758452+0000","last_became_peered":"2024-02-02T13:37:10.758452+0000","last_unstale":"2024-02-02T13:37:10.763090+0000","last_undegraded":"2024-02-02T13:37:10.763090+0000","last_fullsized":"2024-02-02T13:37:10.763090+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:17.955367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.744475+0000","last_change":"2024-02-02T13:37:10.741111+0000","last_active":"2024-02-02T13:37:10.744475+0000","last_peered":"2024-02-02T13:37:10.744475+0000","last_clean":"2024-02-02T13:37:10.744475+0000","last_became_active":"2024-02-02T13:37:10.739871+0000","last_became_peered":"2024-02-02T13:37:10.739871+0000","last_unstale":"2024-02-02T13:37:10.744475+0000","last_undegraded":"2024-02-02T13:37:10.744475+0000","last_fullsized":"2024-02-02T13:37:10.744475+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:25.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.739991+0000","last_change":"2024-02-02T13:37:10.739867+0000","last_active":"2024-02-02T13:37:10.739991+0000","last_peered":"2024-02-02T13:37:10.739991+0000","last_clean":"2024-02-02T13:37:10.739991+0000","last_became_active":"2024-02-02T13:37:10.739141+0000","last_became_peered":"2024-02-02T13:37:10.739141+0000","last_unstale":"2024-02-02T13:37:10.739991+0000","last_undegraded":"2024-02-02T13:37:10.739991+0000","last_fullsized":"2024-02-02T13:37:10.739991+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:49.483601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752453+0000","last_change":"2024-02-02T13:37:10.752390+0000","last_active":"2024-02-02T13:37:10.752453+0000","last_peered":"2024-02-02T13:37:10.752453+0000","last_clean":"2024-02-02T13:37:10.752453+0000","last_became_active":"2024-02-02T13:37:10.752163+0000","last_became_peered":"2024-02-02T13:37:10.752163+0000","last_unstale":"2024-02-02T13:37:10.752453+0000","last_undegraded":"2024-02-02T13:37:10.752453+0000","last_fullsized":"2024-02-02T13:37:10.752453+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:41:20.266573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742285+0000","last_change":"2024-02-02T13:37:10.741671+0000","last_active":"2024-02-02T13:37:10.742285+0000","last_peered":"2024-02-02T13:37:10.742285+0000","last_clean":"2024-02-02T13:37:10.742285+0000","last_became_active":"2024-02-02T13:37:10.741449+0000","last_became_peered":"2024-02-02T13:37:10.741449+0000","last_unstale":"2024-02-02T13:37:10.742285+0000","last_undegraded":"2024-02-02T13:37:10.742285+0000","last_fullsized":"2024-02-02T13:37:10.742285+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:22:50.781898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.745913+0000","last_change":"2024-02-02T13:37:10.745858+0000","last_active":"2024-02-02T13:37:10.745913+0000","last_peered":"2024-02-02T13:37:10.745913+0000","last_clean":"2024-02-02T13:37:10.745913+0000","last_became_active":"2024-02-02T13:37:10.745738+0000","last_became_peered":"2024-02-02T13:37:10.745738+0000","last_unstale":"2024-02-02T13:37:10.745913+0000","last_undegraded":"2024-02-02T13:37:10.745913+0000","last_fullsized":"2024-02-02T13:37:10.745913+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:02.958869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763418+0000","last_change":"2024-02-02T13:37:10.762747+0000","last_active":"2024-02-02T13:37:10.763418+0000","last_peered":"2024-02-02T13:37:10.763418+0000","last_clean":"2024-02-02T13:37:10.763418+0000","last_became_active":"2024-02-02T13:37:10.762328+0000","last_became_peered":"2024-02-02T13:37:10.762328+0000","last_unstale":"2024-02-02T13:37:10.763418+0000","last_undegraded":"2024-02-02T13:37:10.763418+0000","last_fullsized":"2024-02-02T13:37:10.763418+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:02.995879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.756131+0000","last_change":"2024-02-02T13:37:10.756072+0000","last_active":"2024-02-02T13:37:10.756131+0000","last_peered":"2024-02-02T13:37:10.756131+0000","last_clean":"2024-02-02T13:37:10.756131+0000","last_became_active":"2024-02-02T13:37:10.755911+0000","last_became_peered":"2024-02-02T13:37:10.755911+0000","last_unstale":"2024-02-02T13:37:10.756131+0000","last_undegraded":"2024-02-02T13:37:10.756131+0000","last_fullsized":"2024-02-02T13:37:10.756131+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:25.836890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.754656+0000","last_change":"2024-02-02T13:37:10.754205+0000","last_active":"2024-02-02T13:37:10.754656+0000","last_peered":"2024-02-02T13:37:10.754656+0000","last_clean":"2024-02-02T13:37:10.754656+0000","last_became_active":"2024-02-02T13:37:10.753357+0000","last_became_peered":"2024-02-02T13:37:10.753357+0000","last_unstale":"2024-02-02T13:37:10.754656+0000","last_undegraded":"2024-02-02T13:37:10.754656+0000","last_fullsized":"2024-02-02T13:37:10.754656+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:58.647237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753359+0000","last_change":"2024-02-02T13:37:10.753290+0000","last_active":"2024-02-02T13:37:10.753359+0000","last_peered":"2024-02-02T13:37:10.753359+0000","last_clean":"2024-02-02T13:37:10.753359+0000","last_became_active":"2024-02-02T13:37:10.753119+0000","last_became_peered":"2024-02-02T13:37:10.753119+0000","last_unstale":"2024-02-02T13:37:10.753359+0000","last_undegraded":"2024-02-02T13:37:10.753359+0000","last_fullsized":"2024-02-02T13:37:10.753359+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:15.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763005+0000","last_change":"2024-02-02T13:37:10.762920+0000","last_active":"2024-02-02T13:37:10.763005+0000","last_peered":"2024-02-02T13:37:10.763005+0000","last_clean":"2024-02-02T13:37:10.763005+0000","last_became_active":"2024-02-02T13:37:10.762663+0000","last_became_peered":"2024-02-02T13:37:10.762663+0000","last_unstale":"2024-02-02T13:37:10.763005+0000","last_undegraded":"2024-02-02T13:37:10.763005+0000","last_fullsized":"2024-02-02T13:37:10.763005+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:43.040365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761694+0000","last_change":"2024-02-02T13:37:10.757723+0000","last_active":"2024-02-02T13:37:10.761694+0000","last_peered":"2024-02-02T13:37:10.761694+0000","last_clean":"2024-02-02T13:37:10.761694+0000","last_became_active":"2024-02-02T13:37:10.757487+0000","last_became_peered":"2024-02-02T13:37:10.757487+0000","last_unstale":"2024-02-02T13:37:10.761694+0000","last_undegraded":"2024-02-02T13:37:10.761694+0000","last_fullsized":"2024-02-02T13:37:10.761694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:39:47.177676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.736320+0000","last_change":"2024-02-02T13:37:10.735510+0000","last_active":"2024-02-02T13:37:10.736320+0000","last_peered":"2024-02-02T13:37:10.736320+0000","last_clean":"2024-02-02T13:37:10.736320+0000","last_became_active":"2024-02-02T13:37:10.734952+0000","last_became_peered":"2024-02-02T13:37:10.734952+0000","last_unstale":"2024-02-02T13:37:10.736320+0000","last_undegraded":"2024-02-02T13:37:10.736320+0000","last_fullsized":"2024-02-02T13:37:10.736320+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:34.763594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755430+0000","last_change":"2024-02-02T13:37:10.752353+0000","last_active":"2024-02-02T13:37:10.755430+0000","last_peered":"2024-02-02T13:37:10.755430+0000","last_clean":"2024-02-02T13:37:10.755430+0000","last_became_active":"2024-02-02T13:37:10.751213+0000","last_became_peered":"2024-02-02T13:37:10.751213+0000","last_unstale":"2024-02-02T13:37:10.755430+0000","last_undegraded":"2024-02-02T13:37:10.755430+0000","last_fullsized":"2024-02-02T13:37:10.755430+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:45:45.696881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741133+0000","last_change":"2024-02-02T13:37:10.740415+0000","last_active":"2024-02-02T13:37:10.741133+0000","last_peered":"2024-02-02T13:37:10.741133+0000","last_clean":"2024-02-02T13:37:10.741133+0000","last_became_active":"2024-02-02T13:37:10.740197+0000","last_became_peered":"2024-02-02T13:37:10.740197+0000","last_unstale":"2024-02-02T13:37:10.741133+0000","last_undegraded":"2024-02-02T13:37:10.741133+0000","last_fullsized":"2024-02-02T13:37:10.741133+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:35.274145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763031+0000","last_change":"2024-02-02T13:37:10.758711+0000","last_active":"2024-02-02T13:37:10.763031+0000","last_peered":"2024-02-02T13:37:10.763031+0000","last_clean":"2024-02-02T13:37:10.763031+0000","last_became_active":"2024-02-02T13:37:10.755528+0000","last_became_peered":"2024-02-02T13:37:10.755528+0000","last_unstale":"2024-02-02T13:37:10.763031+0000","last_undegraded":"2024-02-02T13:37:10.763031+0000","last_fullsized":"2024-02-02T13:37:10.763031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:31.472624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743271+0000","last_change":"2024-02-02T13:37:10.738241+0000","last_active":"2024-02-02T13:37:10.743271+0000","last_peered":"2024-02-02T13:37:10.743271+0000","last_clean":"2024-02-02T13:37:10.743271+0000","last_became_active":"2024-02-02T13:37:10.737661+0000","last_became_peered":"2024-02-02T13:37:10.737661+0000","last_unstale":"2024-02-02T13:37:10.743271+0000","last_undegraded":"2024-02-02T13:37:10.743271+0000","last_fullsized":"2024-02-02T13:37:10.743271+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:58.647237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749291+0000","last_change":"2024-02-02T13:37:10.748594+0000","last_active":"2024-02-02T13:37:10.749291+0000","last_peered":"2024-02-02T13:37:10.749291+0000","last_clean":"2024-02-02T13:37:10.749291+0000","last_became_active":"2024-02-02T13:37:10.746337+0000","last_became_peered":"2024-02-02T13:37:10.746337+0000","last_unstale":"2024-02-02T13:37:10.749291+0000","last_undegraded":"2024-02-02T13:37:10.749291+0000","last_fullsized":"2024-02-02T13:37:10.749291+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:44.815783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742314+0000","last_change":"2024-02-02T13:37:10.742230+0000","last_active":"2024-02-02T13:37:10.742314+0000","last_peered":"2024-02-02T13:37:10.742314+0000","last_clean":"2024-02-02T13:37:10.742314+0000","last_became_active":"2024-02-02T13:37:10.741881+0000","last_became_peered":"2024-02-02T13:37:10.741881+0000","last_unstale":"2024-02-02T13:37:10.742314+0000","last_undegraded":"2024-02-02T13:37:10.742314+0000","last_fullsized":"2024-02-02T13:37:10.742314+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:56:55.759396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735180+0000","last_change":"2024-02-02T13:37:10.734889+0000","last_active":"2024-02-02T13:37:10.735180+0000","last_peered":"2024-02-02T13:37:10.735180+0000","last_clean":"2024-02-02T13:37:10.735180+0000","last_became_active":"2024-02-02T13:37:10.734691+0000","last_became_peered":"2024-02-02T13:37:10.734691+0000","last_unstale":"2024-02-02T13:37:10.735180+0000","last_undegraded":"2024-02-02T13:37:10.735180+0000","last_fullsized":"2024-02-02T13:37:10.735180+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:25.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"194'4","reported_seq":14,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763490+0000","last_change":"2024-02-02T13:37:10.264228+0000","last_active":"2024-02-02T13:37:10.763490+0000","last_peered":"2024-02-02T13:37:10.763490+0000","last_clean":"2024-02-02T13:37:10.763490+0000","last_became_active":"2024-02-02T13:37:10.263859+0000","last_became_peered":"2024-02-02T13:37:10.263859+0000","last_unstale":"2024-02-02T13:37:10.763490+0000","last_undegraded":"2024-02-02T13:37:10.763490+0000","last_fullsized":"2024-02-02T13:37:10.763490+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:33.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.751298+0000","last_change":"2024-02-02T13:37:10.749933+0000","last_active":"2024-02-02T13:37:10.751298+0000","last_peered":"2024-02-02T13:37:10.751298+0000","last_clean":"2024-02-02T13:37:10.751298+0000","last_became_active":"2024-02-02T13:37:10.745311+0000","last_became_peered":"2024-02-02T13:37:10.745311+0000","last_unstale":"2024-02-02T13:37:10.751298+0000","last_undegraded":"2024-02-02T13:37:10.751298+0000","last_fullsized":"2024-02-02T13:37:10.751298+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:43.040365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758431+0000","last_change":"2024-02-02T13:37:10.270975+0000","last_active":"2024-02-02T13:37:10.758431+0000","last_peered":"2024-02-02T13:37:10.758431+0000","last_clean":"2024-02-02T13:37:10.758431+0000","last_became_active":"2024-02-02T13:37:10.270698+0000","last_became_peered":"2024-02-02T13:37:10.270698+0000","last_unstale":"2024-02-02T13:37:10.758431+0000","last_undegraded":"2024-02-02T13:37:10.758431+0000","last_fullsized":"2024-02-02T13:37:10.758431+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:00:23.242278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.747889+0000","last_change":"2024-02-02T13:37:10.747825+0000","last_active":"2024-02-02T13:37:10.747889+0000","last_peered":"2024-02-02T13:37:10.747889+0000","last_clean":"2024-02-02T13:37:10.747889+0000","last_became_active":"2024-02-02T13:37:10.747606+0000","last_became_peered":"2024-02-02T13:37:10.747606+0000","last_unstale":"2024-02-02T13:37:10.747889+0000","last_undegraded":"2024-02-02T13:37:10.747889+0000","last_fullsized":"2024-02-02T13:37:10.747889+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.681025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755362+0000","last_change":"2024-02-02T13:37:10.261869+0000","last_active":"2024-02-02T13:37:10.755362+0000","last_peered":"2024-02-02T13:37:10.755362+0000","last_clean":"2024-02-02T13:37:10.755362+0000","last_became_active":"2024-02-02T13:37:10.261441+0000","last_became_peered":"2024-02-02T13:37:10.261441+0000","last_unstale":"2024-02-02T13:37:10.755362+0000","last_undegraded":"2024-02-02T13:37:10.755362+0000","last_fullsized":"2024-02-02T13:37:10.755362+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:33:08.796969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758375+0000","last_change":"2024-02-02T13:37:10.756423+0000","last_active":"2024-02-02T13:37:10.758375+0000","last_peered":"2024-02-02T13:37:10.758375+0000","last_clean":"2024-02-02T13:37:10.758375+0000","last_became_active":"2024-02-02T13:37:10.756206+0000","last_became_peered":"2024-02-02T13:37:10.756206+0000","last_unstale":"2024-02-02T13:37:10.758375+0000","last_undegraded":"2024-02-02T13:37:10.758375+0000","last_fullsized":"2024-02-02T13:37:10.758375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:28:46.985468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750186+0000","last_change":"2024-02-02T13:37:10.261110+0000","last_active":"2024-02-02T13:37:10.750186+0000","last_peered":"2024-02-02T13:37:10.750186+0000","last_clean":"2024-02-02T13:37:10.750186+0000","last_became_active":"2024-02-02T13:37:10.260657+0000","last_became_peered":"2024-02-02T13:37:10.260657+0000","last_unstale":"2024-02-02T13:37:10.750186+0000","last_undegraded":"2024-02-02T13:37:10.750186+0000","last_fullsized":"2024-02-02T13:37:10.750186+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:47:34.212325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734757+0000","last_change":"2024-02-02T13:37:10.734178+0000","last_active":"2024-02-02T13:37:10.734757+0000","last_peered":"2024-02-02T13:37:10.734757+0000","last_clean":"2024-02-02T13:37:10.734757+0000","last_became_active":"2024-02-02T13:37:10.733878+0000","last_became_peered":"2024-02-02T13:37:10.733878+0000","last_unstale":"2024-02-02T13:37:10.734757+0000","last_undegraded":"2024-02-02T13:37:10.734757+0000","last_fullsized":"2024-02-02T13:37:10.734757+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:43.754974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758005+0000","last_change":"2024-02-02T13:37:10.263667+0000","last_active":"2024-02-02T13:37:10.758005+0000","last_peered":"2024-02-02T13:37:10.758005+0000","last_clean":"2024-02-02T13:37:10.758005+0000","last_became_active":"2024-02-02T13:37:10.262372+0000","last_became_peered":"2024-02-02T13:37:10.262372+0000","last_unstale":"2024-02-02T13:37:10.758005+0000","last_undegraded":"2024-02-02T13:37:10.758005+0000","last_fullsized":"2024-02-02T13:37:10.758005+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:07:21.527509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735997+0000","last_change":"2024-02-02T13:37:10.735832+0000","last_active":"2024-02-02T13:37:10.735997+0000","last_peered":"2024-02-02T13:37:10.735997+0000","last_clean":"2024-02-02T13:37:10.735997+0000","last_became_active":"2024-02-02T13:37:10.735433+0000","last_became_peered":"2024-02-02T13:37:10.735433+0000","last_unstale":"2024-02-02T13:37:10.735997+0000","last_undegraded":"2024-02-02T13:37:10.735997+0000","last_fullsized":"2024-02-02T13:37:10.735997+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:31.472624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742922+0000","last_change":"2024-02-02T13:37:10.266904+0000","last_active":"2024-02-02T13:37:10.742922+0000","last_peered":"2024-02-02T13:37:10.742922+0000","last_clean":"2024-02-02T13:37:10.742922+0000","last_became_active":"2024-02-02T13:37:10.265737+0000","last_became_peered":"2024-02-02T13:37:10.265737+0000","last_unstale":"2024-02-02T13:37:10.742922+0000","last_undegraded":"2024-02-02T13:37:10.742922+0000","last_fullsized":"2024-02-02T13:37:10.742922+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:56:15.283974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.740045+0000","last_change":"2024-02-02T13:37:10.739985+0000","last_active":"2024-02-02T13:37:10.740045+0000","last_peered":"2024-02-02T13:37:10.740045+0000","last_clean":"2024-02-02T13:37:10.740045+0000","last_became_active":"2024-02-02T13:37:10.739475+0000","last_became_peered":"2024-02-02T13:37:10.739475+0000","last_unstale":"2024-02-02T13:37:10.740045+0000","last_undegraded":"2024-02-02T13:37:10.740045+0000","last_fullsized":"2024-02-02T13:37:10.740045+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:42.207188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742581+0000","last_change":"2024-02-02T13:37:10.267684+0000","last_active":"2024-02-02T13:37:10.742581+0000","last_peered":"2024-02-02T13:37:10.742581+0000","last_clean":"2024-02-02T13:37:10.742581+0000","last_became_active":"2024-02-02T13:37:10.267126+0000","last_became_peered":"2024-02-02T13:37:10.267126+0000","last_unstale":"2024-02-02T13:37:10.742581+0000","last_undegraded":"2024-02-02T13:37:10.742581+0000","last_fullsized":"2024-02-02T13:37:10.742581+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:22.294877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741601+0000","last_change":"2024-02-02T13:37:10.741539+0000","last_active":"2024-02-02T13:37:10.741601+0000","last_peered":"2024-02-02T13:37:10.741601+0000","last_clean":"2024-02-02T13:37:10.741601+0000","last_became_active":"2024-02-02T13:37:10.741344+0000","last_became_peered":"2024-02-02T13:37:10.741344+0000","last_unstale":"2024-02-02T13:37:10.741601+0000","last_undegraded":"2024-02-02T13:37:10.741601+0000","last_fullsized":"2024-02-02T13:37:10.741601+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:41:54.323783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.729912+0000","last_change":"2024-02-02T13:37:10.266071+0000","last_active":"2024-02-02T13:37:10.729912+0000","last_peered":"2024-02-02T13:37:10.729912+0000","last_clean":"2024-02-02T13:37:10.729912+0000","last_became_active":"2024-02-02T13:37:10.265721+0000","last_became_peered":"2024-02-02T13:37:10.265721+0000","last_unstale":"2024-02-02T13:37:10.729912+0000","last_undegraded":"2024-02-02T13:37:10.729912+0000","last_fullsized":"2024-02-02T13:37:10.729912+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:49.530021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734007+0000","last_change":"2024-02-02T13:37:10.733708+0000","last_active":"2024-02-02T13:37:10.734007+0000","last_peered":"2024-02-02T13:37:10.734007+0000","last_clean":"2024-02-02T13:37:10.734007+0000","last_became_active":"2024-02-02T13:37:10.733476+0000","last_became_peered":"2024-02-02T13:37:10.733476+0000","last_unstale":"2024-02-02T13:37:10.734007+0000","last_undegraded":"2024-02-02T13:37:10.734007+0000","last_fullsized":"2024-02-02T13:37:10.734007+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:59:32.706979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.738698+0000","last_change":"2024-02-02T13:37:10.258610+0000","last_active":"2024-02-02T13:37:10.738698+0000","last_peered":"2024-02-02T13:37:10.738698+0000","last_clean":"2024-02-02T13:37:10.738698+0000","last_became_active":"2024-02-02T13:37:10.257988+0000","last_became_peered":"2024-02-02T13:37:10.257988+0000","last_unstale":"2024-02-02T13:37:10.738698+0000","last_undegraded":"2024-02-02T13:37:10.738698+0000","last_fullsized":"2024-02-02T13:37:10.738698+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:23.009072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734025+0000","last_change":"2024-02-02T13:37:10.733739+0000","last_active":"2024-02-02T13:37:10.734025+0000","last_peered":"2024-02-02T13:37:10.734025+0000","last_clean":"2024-02-02T13:37:10.734025+0000","last_became_active":"2024-02-02T13:37:10.733493+0000","last_became_peered":"2024-02-02T13:37:10.733493+0000","last_unstale":"2024-02-02T13:37:10.734025+0000","last_undegraded":"2024-02-02T13:37:10.734025+0000","last_fullsized":"2024-02-02T13:37:10.734025+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:29.728065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"194'3","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749715+0000","last_change":"2024-02-02T13:37:10.262108+0000","last_active":"2024-02-02T13:37:10.749715+0000","last_peered":"2024-02-02T13:37:10.749715+0000","last_clean":"2024-02-02T13:37:10.749715+0000","last_became_active":"2024-02-02T13:37:10.261689+0000","last_became_peered":"2024-02-02T13:37:10.261689+0000","last_unstale":"2024-02-02T13:37:10.749715+0000","last_undegraded":"2024-02-02T13:37:10.749715+0000","last_fullsized":"2024-02-02T13:37:10.749715+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:20:00.812570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758584+0000","last_change":"2024-02-02T13:37:10.755414+0000","last_active":"2024-02-02T13:37:10.758584+0000","last_peered":"2024-02-02T13:37:10.758584+0000","last_clean":"2024-02-02T13:37:10.758584+0000","last_became_active":"2024-02-02T13:37:10.755133+0000","last_became_peered":"2024-02-02T13:37:10.755133+0000","last_unstale":"2024-02-02T13:37:10.758584+0000","last_undegraded":"2024-02-02T13:37:10.758584+0000","last_fullsized":"2024-02-02T13:37:10.758584+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:21:48.940322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"194'6","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761691+0000","last_change":"2024-02-02T13:37:10.264832+0000","last_active":"2024-02-02T13:37:10.761691+0000","last_peered":"2024-02-02T13:37:10.761691+0000","last_clean":"2024-02-02T13:37:10.761691+0000","last_became_active":"2024-02-02T13:37:10.264585+0000","last_became_peered":"2024-02-02T13:37:10.264585+0000","last_unstale":"2024-02-02T13:37:10.761691+0000","last_undegraded":"2024-02-02T13:37:10.761691+0000","last_fullsized":"2024-02-02T13:37:10.761691+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:01.981135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755552+0000","last_change":"2024-02-02T13:37:10.755413+0000","last_active":"2024-02-02T13:37:10.755552+0000","last_peered":"2024-02-02T13:37:10.755552+0000","last_clean":"2024-02-02T13:37:10.755552+0000","last_became_active":"2024-02-02T13:37:10.755196+0000","last_became_peered":"2024-02-02T13:37:10.755196+0000","last_unstale":"2024-02-02T13:37:10.755552+0000","last_undegraded":"2024-02-02T13:37:10.755552+0000","last_fullsized":"2024-02-02T13:37:10.755552+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:59:43.507576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743996+0000","last_change":"2024-02-02T13:37:10.260677+0000","last_active":"2024-02-02T13:37:10.743996+0000","last_peered":"2024-02-02T13:37:10.743996+0000","last_clean":"2024-02-02T13:37:10.743996+0000","last_became_active":"2024-02-02T13:37:10.260231+0000","last_became_peered":"2024-02-02T13:37:10.260231+0000","last_unstale":"2024-02-02T13:37:10.743996+0000","last_undegraded":"2024-02-02T13:37:10.743996+0000","last_fullsized":"2024-02-02T13:37:10.743996+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:54:29.014585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761158+0000","last_change":"2024-02-02T13:37:10.761115+0000","last_active":"2024-02-02T13:37:10.761158+0000","last_peered":"2024-02-02T13:37:10.761158+0000","last_clean":"2024-02-02T13:37:10.761158+0000","last_became_active":"2024-02-02T13:37:10.760969+0000","last_became_peered":"2024-02-02T13:37:10.760969+0000","last_unstale":"2024-02-02T13:37:10.761158+0000","last_undegraded":"2024-02-02T13:37:10.761158+0000","last_fullsized":"2024-02-02T13:37:10.761158+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:35.973959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.728351+0000","last_change":"2024-02-02T13:37:10.267512+0000","last_active":"2024-02-02T13:37:10.728351+0000","last_peered":"2024-02-02T13:37:10.728351+0000","last_clean":"2024-02-02T13:37:10.728351+0000","last_became_active":"2024-02-02T13:37:10.267270+0000","last_became_peered":"2024-02-02T13:37:10.267270+0000","last_unstale":"2024-02-02T13:37:10.728351+0000","last_undegraded":"2024-02-02T13:37:10.728351+0000","last_fullsized":"2024-02-02T13:37:10.728351+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:17:40.687396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753007+0000","last_change":"2024-02-02T13:37:10.752945+0000","last_active":"2024-02-02T13:37:10.753007+0000","last_peered":"2024-02-02T13:37:10.753007+0000","last_clean":"2024-02-02T13:37:10.753007+0000","last_became_active":"2024-02-02T13:37:10.752716+0000","last_became_peered":"2024-02-02T13:37:10.752716+0000","last_unstale":"2024-02-02T13:37:10.753007+0000","last_undegraded":"2024-02-02T13:37:10.753007+0000","last_fullsized":"2024-02-02T13:37:10.753007+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:17.504456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732032+0000","last_change":"2024-02-02T13:37:10.263254+0000","last_active":"2024-02-02T13:37:10.732032+0000","last_peered":"2024-02-02T13:37:10.732032+0000","last_clean":"2024-02-02T13:37:10.732032+0000","last_became_active":"2024-02-02T13:37:10.262564+0000","last_became_peered":"2024-02-02T13:37:10.262564+0000","last_unstale":"2024-02-02T13:37:10.732032+0000","last_undegraded":"2024-02-02T13:37:10.732032+0000","last_fullsized":"2024-02-02T13:37:10.732032+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:01:03.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755466+0000","last_change":"2024-02-02T13:37:10.752814+0000","last_active":"2024-02-02T13:37:10.755466+0000","last_peered":"2024-02-02T13:37:10.755466+0000","last_clean":"2024-02-02T13:37:10.755466+0000","last_became_active":"2024-02-02T13:37:10.752052+0000","last_became_peered":"2024-02-02T13:37:10.752052+0000","last_unstale":"2024-02-02T13:37:10.755466+0000","last_undegraded":"2024-02-02T13:37:10.755466+0000","last_fullsized":"2024-02-02T13:37:10.755466+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:50.153639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"194'8","reported_seq":18,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.764998+0000","last_change":"2024-02-02T13:37:10.263616+0000","last_active":"2024-02-02T13:37:10.764998+0000","last_peered":"2024-02-02T13:37:10.764998+0000","last_clean":"2024-02-02T13:37:10.764998+0000","last_became_active":"2024-02-02T13:37:10.263370+0000","last_became_peered":"2024-02-02T13:37:10.263370+0000","last_unstale":"2024-02-02T13:37:10.764998+0000","last_undegraded":"2024-02-02T13:37:10.764998+0000","last_fullsized":"2024-02-02T13:37:10.764998+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:49.138894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743915+0000","last_change":"2024-02-02T13:37:10.743838+0000","last_active":"2024-02-02T13:37:10.743915+0000","last_peered":"2024-02-02T13:37:10.743915+0000","last_clean":"2024-02-02T13:37:10.743915+0000","last_became_active":"2024-02-02T13:37:10.743355+0000","last_became_peered":"2024-02-02T13:37:10.743355+0000","last_unstale":"2024-02-02T13:37:10.743915+0000","last_undegraded":"2024-02-02T13:37:10.743915+0000","last_fullsized":"2024-02-02T13:37:10.743915+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:50.153639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763169+0000","last_change":"2024-02-02T13:37:10.269272+0000","last_active":"2024-02-02T13:37:10.763169+0000","last_peered":"2024-02-02T13:37:10.763169+0000","last_clean":"2024-02-02T13:37:10.763169+0000","last_became_active":"2024-02-02T13:37:10.268980+0000","last_became_peered":"2024-02-02T13:37:10.268980+0000","last_unstale":"2024-02-02T13:37:10.763169+0000","last_undegraded":"2024-02-02T13:37:10.763169+0000","last_fullsized":"2024-02-02T13:37:10.763169+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:41.693441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.740720+0000","last_change":"2024-02-02T13:37:10.740641+0000","last_active":"2024-02-02T13:37:10.740720+0000","last_peered":"2024-02-02T13:37:10.740720+0000","last_clean":"2024-02-02T13:37:10.740720+0000","last_became_active":"2024-02-02T13:37:10.740386+0000","last_became_peered":"2024-02-02T13:37:10.740386+0000","last_unstale":"2024-02-02T13:37:10.740720+0000","last_undegraded":"2024-02-02T13:37:10.740720+0000","last_fullsized":"2024-02-02T13:37:10.740720+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:59:04.790269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763374+0000","last_change":"2024-02-02T13:37:10.259458+0000","last_active":"2024-02-02T13:37:10.763374+0000","last_peered":"2024-02-02T13:37:10.763374+0000","last_clean":"2024-02-02T13:37:10.763374+0000","last_became_active":"2024-02-02T13:37:10.258977+0000","last_became_peered":"2024-02-02T13:37:10.258977+0000","last_unstale":"2024-02-02T13:37:10.763374+0000","last_undegraded":"2024-02-02T13:37:10.763374+0000","last_fullsized":"2024-02-02T13:37:10.763374+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:49:05.058927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735979+0000","last_change":"2024-02-02T13:37:10.735667+0000","last_active":"2024-02-02T13:37:10.735979+0000","last_peered":"2024-02-02T13:37:10.735979+0000","last_clean":"2024-02-02T13:37:10.735979+0000","last_became_active":"2024-02-02T13:37:10.735161+0000","last_became_peered":"2024-02-02T13:37:10.735161+0000","last_unstale":"2024-02-02T13:37:10.735979+0000","last_undegraded":"2024-02-02T13:37:10.735979+0000","last_fullsized":"2024-02-02T13:37:10.735979+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:17.955367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.751746+0000","last_change":"2024-02-02T13:37:10.270996+0000","last_active":"2024-02-02T13:37:10.751746+0000","last_peered":"2024-02-02T13:37:10.751746+0000","last_clean":"2024-02-02T13:37:10.751746+0000","last_became_active":"2024-02-02T13:37:10.269739+0000","last_became_peered":"2024-02-02T13:37:10.269739+0000","last_unstale":"2024-02-02T13:37:10.751746+0000","last_undegraded":"2024-02-02T13:37:10.751746+0000","last_fullsized":"2024-02-02T13:37:10.751746+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:19:27.013508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.747722+0000","last_change":"2024-02-02T13:37:10.747652+0000","last_active":"2024-02-02T13:37:10.747722+0000","last_peered":"2024-02-02T13:37:10.747722+0000","last_clean":"2024-02-02T13:37:10.747722+0000","last_became_active":"2024-02-02T13:37:10.747313+0000","last_became_peered":"2024-02-02T13:37:10.747313+0000","last_unstale":"2024-02-02T13:37:10.747722+0000","last_undegraded":"2024-02-02T13:37:10.747722+0000","last_fullsized":"2024-02-02T13:37:10.747722+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:53:53.923501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.730883+0000","last_change":"2024-02-02T13:37:10.263774+0000","last_active":"2024-02-02T13:37:10.730883+0000","last_peered":"2024-02-02T13:37:10.730883+0000","last_clean":"2024-02-02T13:37:10.730883+0000","last_became_active":"2024-02-02T13:37:10.263472+0000","last_became_peered":"2024-02-02T13:37:10.263472+0000","last_unstale":"2024-02-02T13:37:10.730883+0000","last_undegraded":"2024-02-02T13:37:10.730883+0000","last_fullsized":"2024-02-02T13:37:10.730883+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:10:56.736246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763071+0000","last_change":"2024-02-02T13:37:10.758876+0000","last_active":"2024-02-02T13:37:10.763071+0000","last_peered":"2024-02-02T13:37:10.763071+0000","last_clean":"2024-02-02T13:37:10.763071+0000","last_became_active":"2024-02-02T13:37:10.758129+0000","last_became_peered":"2024-02-02T13:37:10.758129+0000","last_unstale":"2024-02-02T13:37:10.763071+0000","last_undegraded":"2024-02-02T13:37:10.763071+0000","last_fullsized":"2024-02-02T13:37:10.763071+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:17.504456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755452+0000","last_change":"2024-02-02T13:37:10.266740+0000","last_active":"2024-02-02T13:37:10.755452+0000","last_peered":"2024-02-02T13:37:10.755452+0000","last_clean":"2024-02-02T13:37:10.755452+0000","last_became_active":"2024-02-02T13:37:10.263522+0000","last_became_peered":"2024-02-02T13:37:10.263522+0000","last_unstale":"2024-02-02T13:37:10.755452+0000","last_undegraded":"2024-02-02T13:37:10.755452+0000","last_fullsized":"2024-02-02T13:37:10.755452+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:55.383915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750548+0000","last_change":"2024-02-02T13:37:10.750435+0000","last_active":"2024-02-02T13:37:10.750548+0000","last_peered":"2024-02-02T13:37:10.750548+0000","last_clean":"2024-02-02T13:37:10.750548+0000","last_became_active":"2024-02-02T13:37:10.750123+0000","last_became_peered":"2024-02-02T13:37:10.750123+0000","last_unstale":"2024-02-02T13:37:10.750548+0000","last_undegraded":"2024-02-02T13:37:10.750548+0000","last_fullsized":"2024-02-02T13:37:10.750548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:19:53.625421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.733512+0000","last_change":"2024-02-02T13:37:10.268767+0000","last_active":"2024-02-02T13:37:10.733512+0000","last_peered":"2024-02-02T13:37:10.733512+0000","last_clean":"2024-02-02T13:37:10.733512+0000","last_became_active":"2024-02-02T13:37:10.266597+0000","last_became_peered":"2024-02-02T13:37:10.266597+0000","last_unstale":"2024-02-02T13:37:10.733512+0000","last_undegraded":"2024-02-02T13:37:10.733512+0000","last_fullsized":"2024-02-02T13:37:10.733512+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:59:03.775525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.760534+0000","last_change":"2024-02-02T13:37:10.760442+0000","last_active":"2024-02-02T13:37:10.760534+0000","last_peered":"2024-02-02T13:37:10.760534+0000","last_clean":"2024-02-02T13:37:10.760534+0000","last_became_active":"2024-02-02T13:37:10.759219+0000","last_became_peered":"2024-02-02T13:37:10.759219+0000","last_unstale":"2024-02-02T13:37:10.760534+0000","last_undegraded":"2024-02-02T13:37:10.760534+0000","last_fullsized":"2024-02-02T13:37:10.760534+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:35.973959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732546+0000","last_change":"2024-02-02T13:37:10.257847+0000","last_active":"2024-02-02T13:37:10.732546+0000","last_peered":"2024-02-02T13:37:10.732546+0000","last_clean":"2024-02-02T13:37:10.732546+0000","last_became_active":"2024-02-02T13:37:10.253858+0000","last_became_peered":"2024-02-02T13:37:10.253858+0000","last_unstale":"2024-02-02T13:37:10.732546+0000","last_undegraded":"2024-02-02T13:37:10.732546+0000","last_fullsized":"2024-02-02T13:37:10.732546+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:00.482921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753771+0000","last_change":"2024-02-02T13:37:10.753646+0000","last_active":"2024-02-02T13:37:10.753771+0000","last_peered":"2024-02-02T13:37:10.753771+0000","last_clean":"2024-02-02T13:37:10.753771+0000","last_became_active":"2024-02-02T13:37:10.753450+0000","last_became_peered":"2024-02-02T13:37:10.753450+0000","last_unstale":"2024-02-02T13:37:10.753771+0000","last_undegraded":"2024-02-02T13:37:10.753771+0000","last_fullsized":"2024-02-02T13:37:10.753771+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:07.886376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734473+0000","last_change":"2024-02-02T13:37:10.271570+0000","last_active":"2024-02-02T13:37:10.734473+0000","last_peered":"2024-02-02T13:37:10.734473+0000","last_clean":"2024-02-02T13:37:10.734473+0000","last_became_active":"2024-02-02T13:37:10.270785+0000","last_became_peered":"2024-02-02T13:37:10.270785+0000","last_unstale":"2024-02-02T13:37:10.734473+0000","last_undegraded":"2024-02-02T13:37:10.734473+0000","last_fullsized":"2024-02-02T13:37:10.734473+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:28.713320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734951+0000","last_change":"2024-02-02T13:37:10.734752+0000","last_active":"2024-02-02T13:37:10.734951+0000","last_peered":"2024-02-02T13:37:10.734951+0000","last_clean":"2024-02-02T13:37:10.734951+0000","last_became_active":"2024-02-02T13:37:10.734494+0000","last_became_peered":"2024-02-02T13:37:10.734494+0000","last_unstale":"2024-02-02T13:37:10.734951+0000","last_undegraded":"2024-02-02T13:37:10.734951+0000","last_fullsized":"2024-02-02T13:37:10.734951+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:36:58.507132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750411+0000","last_change":"2024-02-02T13:37:10.270731+0000","last_active":"2024-02-02T13:37:10.750411+0000","last_peered":"2024-02-02T13:37:10.750411+0000","last_clean":"2024-02-02T13:37:10.750411+0000","last_became_active":"2024-02-02T13:37:10.270243+0000","last_became_peered":"2024-02-02T13:37:10.270243+0000","last_unstale":"2024-02-02T13:37:10.750411+0000","last_undegraded":"2024-02-02T13:37:10.750411+0000","last_fullsized":"2024-02-02T13:37:10.750411+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:09:51.563908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.754667+0000","last_change":"2024-02-02T13:37:10.754618+0000","last_active":"2024-02-02T13:37:10.754667+0000","last_peered":"2024-02-02T13:37:10.754667+0000","last_clean":"2024-02-02T13:37:10.754667+0000","last_became_active":"2024-02-02T13:37:10.754468+0000","last_became_peered":"2024-02-02T13:37:10.754468+0000","last_unstale":"2024-02-02T13:37:10.754667+0000","last_undegraded":"2024-02-02T13:37:10.754667+0000","last_fullsized":"2024-02-02T13:37:10.754667+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:03.196854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750723+0000","last_change":"2024-02-02T13:37:10.262995+0000","last_active":"2024-02-02T13:37:10.750723+0000","last_peered":"2024-02-02T13:37:10.750723+0000","last_clean":"2024-02-02T13:37:10.750723+0000","last_became_active":"2024-02-02T13:37:10.261862+0000","last_became_peered":"2024-02-02T13:37:10.261862+0000","last_unstale":"2024-02-02T13:37:10.750723+0000","last_undegraded":"2024-02-02T13:37:10.750723+0000","last_fullsized":"2024-02-02T13:37:10.750723+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:35.588116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743143+0000","last_change":"2024-02-02T13:37:10.742778+0000","last_active":"2024-02-02T13:37:10.743143+0000","last_peered":"2024-02-02T13:37:10.743143+0000","last_clean":"2024-02-02T13:37:10.743143+0000","last_became_active":"2024-02-02T13:37:10.742480+0000","last_became_peered":"2024-02-02T13:37:10.742480+0000","last_unstale":"2024-02-02T13:37:10.743143+0000","last_undegraded":"2024-02-02T13:37:10.743143+0000","last_fullsized":"2024-02-02T13:37:10.743143+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:16:47.842854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742863+0000","last_change":"2024-02-02T13:37:10.267857+0000","last_active":"2024-02-02T13:37:10.742863+0000","last_peered":"2024-02-02T13:37:10.742863+0000","last_clean":"2024-02-02T13:37:10.742863+0000","last_became_active":"2024-02-02T13:37:10.267461+0000","last_became_peered":"2024-02-02T13:37:10.267461+0000","last_unstale":"2024-02-02T13:37:10.742863+0000","last_undegraded":"2024-02-02T13:37:10.742863+0000","last_fullsized":"2024-02-02T13:37:10.742863+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:16.489712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737430+0000","last_change":"2024-02-02T13:37:10.737367+0000","last_active":"2024-02-02T13:37:10.737430+0000","last_peered":"2024-02-02T13:37:10.737430+0000","last_clean":"2024-02-02T13:37:10.737430+0000","last_became_active":"2024-02-02T13:37:10.736896+0000","last_became_peered":"2024-02-02T13:37:10.736896+0000","last_unstale":"2024-02-02T13:37:10.737430+0000","last_undegraded":"2024-02-02T13:37:10.737430+0000","last_fullsized":"2024-02-02T13:37:10.737430+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:28:35.262486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.727533+0000","last_change":"2024-02-02T13:37:10.262945+0000","last_active":"2024-02-02T13:37:10.727533+0000","last_peered":"2024-02-02T13:37:10.727533+0000","last_clean":"2024-02-02T13:37:10.727533+0000","last_became_active":"2024-02-02T13:37:10.262589+0000","last_became_peered":"2024-02-02T13:37:10.262589+0000","last_unstale":"2024-02-02T13:37:10.727533+0000","last_undegraded":"2024-02-02T13:37:10.727533+0000","last_fullsized":"2024-02-02T13:37:10.727533+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:03.129427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742300+0000","last_change":"2024-02-02T13:37:10.742099+0000","last_active":"2024-02-02T13:37:10.742300+0000","last_peered":"2024-02-02T13:37:10.742300+0000","last_clean":"2024-02-02T13:37:10.742300+0000","last_became_active":"2024-02-02T13:37:10.741616+0000","last_became_peered":"2024-02-02T13:37:10.741616+0000","last_unstale":"2024-02-02T13:37:10.742300+0000","last_undegraded":"2024-02-02T13:37:10.742300+0000","last_fullsized":"2024-02-02T13:37:10.742300+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:50.544766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743929+0000","last_change":"2024-02-02T13:37:10.261714+0000","last_active":"2024-02-02T13:37:10.743929+0000","last_peered":"2024-02-02T13:37:10.743929+0000","last_clean":"2024-02-02T13:37:10.743929+0000","last_became_active":"2024-02-02T13:37:10.260970+0000","last_became_peered":"2024-02-02T13:37:10.260970+0000","last_unstale":"2024-02-02T13:37:10.743929+0000","last_undegraded":"2024-02-02T13:37:10.743929+0000","last_fullsized":"2024-02-02T13:37:10.743929+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:28:45.970724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749322+0000","last_change":"2024-02-02T13:37:10.749148+0000","last_active":"2024-02-02T13:37:10.749322+0000","last_peered":"2024-02-02T13:37:10.749322+0000","last_clean":"2024-02-02T13:37:10.749322+0000","last_became_active":"2024-02-02T13:37:10.748346+0000","last_became_peered":"2024-02-02T13:37:10.748346+0000","last_unstale":"2024-02-02T13:37:10.749322+0000","last_undegraded":"2024-02-02T13:37:10.749322+0000","last_fullsized":"2024-02-02T13:37:10.749322+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:03:34.307896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732132+0000","last_change":"2024-02-02T13:37:10.262937+0000","last_active":"2024-02-02T13:37:10.732132+0000","last_peered":"2024-02-02T13:37:10.732132+0000","last_clean":"2024-02-02T13:37:10.732132+0000","last_became_active":"2024-02-02T13:37:10.261912+0000","last_became_peered":"2024-02-02T13:37:10.261912+0000","last_unstale":"2024-02-02T13:37:10.732132+0000","last_undegraded":"2024-02-02T13:37:10.732132+0000","last_fullsized":"2024-02-02T13:37:10.732132+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:19:10.553594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737096+0000","last_change":"2024-02-02T13:37:10.736695+0000","last_active":"2024-02-02T13:37:10.737096+0000","last_peered":"2024-02-02T13:37:10.737096+0000","last_clean":"2024-02-02T13:37:10.737096+0000","last_became_active":"2024-02-02T13:37:10.736489+0000","last_became_peered":"2024-02-02T13:37:10.736489+0000","last_unstale":"2024-02-02T13:37:10.737096+0000","last_undegraded":"2024-02-02T13:37:10.737096+0000","last_fullsized":"2024-02-02T13:37:10.737096+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:56:58.205780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.739774+0000","last_change":"2024-02-02T13:37:10.265283+0000","last_active":"2024-02-02T13:37:10.739774+0000","last_peered":"2024-02-02T13:37:10.739774+0000","last_clean":"2024-02-02T13:37:10.739774+0000","last_became_active":"2024-02-02T13:37:10.265014+0000","last_became_peered":"2024-02-02T13:37:10.265014+0000","last_unstale":"2024-02-02T13:37:10.739774+0000","last_undegraded":"2024-02-02T13:37:10.739774+0000","last_fullsized":"2024-02-02T13:37:10.739774+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:45:44.682137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763106+0000","last_change":"2024-02-02T13:37:10.759246+0000","last_active":"2024-02-02T13:37:10.763106+0000","last_peered":"2024-02-02T13:37:10.763106+0000","last_clean":"2024-02-02T13:37:10.763106+0000","last_became_active":"2024-02-02T13:37:10.754087+0000","last_became_peered":"2024-02-02T13:37:10.754087+0000","last_unstale":"2024-02-02T13:37:10.763106+0000","last_undegraded":"2024-02-02T13:37:10.763106+0000","last_fullsized":"2024-02-02T13:37:10.763106+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:35.274145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.736810+0000","last_change":"2024-02-02T13:37:10.271912+0000","last_active":"2024-02-02T13:37:10.736810+0000","last_peered":"2024-02-02T13:37:10.736810+0000","last_clean":"2024-02-02T13:37:10.736810+0000","last_became_active":"2024-02-02T13:37:10.271636+0000","last_became_peered":"2024-02-02T13:37:10.271636+0000","last_unstale":"2024-02-02T13:37:10.736810+0000","last_undegraded":"2024-02-02T13:37:10.736810+0000","last_fullsized":"2024-02-02T13:37:10.736810+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:41.192444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735895+0000","last_change":"2024-02-02T13:37:10.735806+0000","last_active":"2024-02-02T13:37:10.735895+0000","last_peered":"2024-02-02T13:37:10.735895+0000","last_clean":"2024-02-02T13:37:10.735895+0000","last_became_active":"2024-02-02T13:37:10.734954+0000","last_became_peered":"2024-02-02T13:37:10.734954+0000","last_unstale":"2024-02-02T13:37:10.735895+0000","last_undegraded":"2024-02-02T13:37:10.735895+0000","last_fullsized":"2024-02-02T13:37:10.735895+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:03.624254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.744175+0000","last_change":"2024-02-02T13:37:10.263276+0000","last_active":"2024-02-02T13:37:10.744175+0000","last_peered":"2024-02-02T13:37:10.744175+0000","last_clean":"2024-02-02T13:37:10.744175+0000","last_became_active":"2024-02-02T13:37:10.261375+0000","last_became_peered":"2024-02-02T13:37:10.261375+0000","last_unstale":"2024-02-02T13:37:10.744175+0000","last_undegraded":"2024-02-02T13:37:10.744175+0000","last_fullsized":"2024-02-02T13:37:10.744175+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:21:39.188714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":336,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.254141+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:10.254141+0000","last_peered":"2024-02-02T13:37:10.254141+0000","last_clean":"2024-02-02T13:37:10.254141+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:10.254141+0000","last_undegraded":"2024-02-02T13:37:10.254141+0000","last_fullsized":"2024-02-02T13:37:10.254141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1644,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193667,"statfs":{"total":200000000,"available":198315832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.64600000000000002,"15min":0.61499999999999999},"max":{"1min":1.4570000000000001,"5min":2.266,"15min":2.5310000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.81699999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.524,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1240000000000001,"15min":1.1040000000000001},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.66300000000000003},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.893},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.56299999999999994},"max":{"1min":1.385,"5min":1.6399999999999999,"15min":2.5840000000000001},"last":0.628}]},{"osd":2,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2210000000000001,"15min":1.234},"min":{"1min":0.71799999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"max":{"1min":1.623,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2030000000000001},"min":{"1min":0.91900000000000004,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.5700000000000001,"5min":1.6639999999999999,"15min":3.5819999999999999},"last":0.502}]},{"osd":3,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2050000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.66200000000000003},"max":{"1min":1.5740000000000001,"5min":1.635,"15min":1.819},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.323,"5min":1.333,"15min":1.3280000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.48999999999999999},"max":{"1min":1.6910000000000001,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.258,"15min":1.242},"min":{"1min":0.79500000000000004,"5min":0.72899999999999998,"15min":0.57599999999999996},"max":{"1min":1.7110000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.278,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.753,"15min":0.61199999999999999},"max":{"1min":1.655,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.349,"15min":1.361},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.54000000000000004},"max":{"1min":1.758,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.90700000000000003,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":3.7949999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.506,"15min":1.5109999999999999},"min":{"1min":1.194,"5min":0.95899999999999996,"15min":0.64200000000000002},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":3.6840000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4139999999999999,"15min":1.4159999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.63700000000000001},"max":{"1min":1.8140000000000001,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":0.69699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1064,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194247,"statfs":{"total":200000000,"available":198909503,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.127,"15min":1.077},"min":{"1min":0.64000000000000001,"5min":0.63200000000000001,"15min":0.498},"max":{"1min":1.496,"5min":1.603,"15min":2.3780000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.077,"15min":1.056},"min":{"1min":0.81599999999999995,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.5589999999999999,"5min":1.776,"15min":1.776},"last":1.171}]},{"osd":1,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1140000000000001,"15min":1.0649999999999999},"min":{"1min":0.78900000000000003,"5min":0.68799999999999994,"15min":0.54000000000000004},"max":{"1min":1.3839999999999999,"5min":1.4790000000000001,"15min":1.556},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.175,"15min":1.135},"min":{"1min":0.80100000000000005,"5min":0.66000000000000003,"15min":0.55900000000000005},"max":{"1min":1.5720000000000001,"5min":1.677,"15min":2.9300000000000002},"last":1.401}]},{"osd":2,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.226,"15min":1.1539999999999999},"min":{"1min":1.012,"5min":0.77300000000000002,"15min":0.63700000000000001},"max":{"1min":1.627,"5min":1.75,"15min":2.6309999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2090000000000001,"15min":1.181},"min":{"1min":0.93400000000000005,"5min":0.80400000000000005,"15min":0.63},"max":{"1min":1.603,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.76000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2609999999999999,"15min":1.256},"min":{"1min":1.0149999999999999,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.577,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.083},{"interface":"front","average":{"1min":1.266,"5min":1.2430000000000001,"15min":1.1890000000000001},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.61299999999999999},"max":{"1min":1.665,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.232}]},{"osd":4,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.345,"15min":1.3320000000000001},"min":{"1min":0.80500000000000005,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.6919999999999999,"5min":1.8420000000000001,"15min":17.686},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.419,"15min":1.466},"min":{"1min":0.96899999999999997,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.899,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2869999999999999,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.56399999999999995,"15min":0.45900000000000002},"max":{"1min":1.782,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4179999999999999,"15min":1.3979999999999999},"min":{"1min":0.95299999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.95,"15min":3.331},"last":1.512}]},{"osd":7,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.454,"15min":1.462},"min":{"1min":1.0629999999999999,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":2.0179999999999998,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.446,"15min":1.444},"min":{"1min":1.044,"5min":0.80200000000000005,"15min":0.65300000000000002},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.647}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1059,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194253,"statfs":{"total":200000000,"available":198915281,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.085,"15min":1.0449999999999999},"min":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.61099999999999999},"max":{"1min":1.4990000000000001,"5min":1.8,"15min":1.8},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.135,"5min":1.0760000000000001,"15min":1.0609999999999999},"min":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.49199999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1870000000000001,"15min":1.2010000000000001},"min":{"1min":0.83899999999999997,"5min":0.67500000000000004,"15min":0.61899999999999999},"max":{"1min":1.498,"5min":1.841,"15min":4.5899999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.1100000000000001},"min":{"1min":0.84599999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.5249999999999999,"5min":1.6379999999999999,"15min":4.4500000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3069999999999999,"15min":1.266},"min":{"1min":1.0309999999999999,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6160000000000001,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.95399999999999996,"5min":0.76000000000000001,"15min":0.65300000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.0860000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.26,"15min":1.214},"min":{"1min":1.032,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.478,"5min":1.629,"15min":1.7769999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2689999999999999,"15min":1.232},"min":{"1min":1.073,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.5960000000000001,"5min":1.8009999999999999,"15min":5.4610000000000003},"last":1.5820000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.78400000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6140000000000001,"5min":1.675,"15min":1.8460000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3260000000000001,"15min":1.298},"min":{"1min":1.0169999999999999,"5min":0.61799999999999999,"15min":0.55100000000000005},"max":{"1min":1.5549999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.335}]},{"osd":6,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.341,"15min":1.325},"min":{"1min":1.016,"5min":0.745,"15min":0.64700000000000002},"max":{"1min":1.6659999999999999,"5min":1.865,"15min":2.0009999999999999},"last":1.653},{"interface":"front","average":{"1min":1.375,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.0660000000000001,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.8109999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4350000000000001,"15min":1.417},"min":{"1min":0.92200000000000004,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.613,"5min":1.893,"15min":1.9670000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.478,"15min":1.429},"min":{"1min":1.1180000000000001,"5min":0.88300000000000001,"15min":0.61599999999999999},"max":{"1min":1.722,"5min":2.024,"15min":2.024},"last":1.706}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1058,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194254,"statfs":{"total":200000000,"available":198916300,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.57999999999999996},"max":{"1min":1.431,"5min":1.5309999999999999,"15min":4.5949999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.83499999999999996,"5min":0.70399999999999996,"15min":0.59199999999999997},"max":{"1min":1.4470000000000001,"5min":1.7609999999999999,"15min":1.839},"last":1.153}]},{"osd":1,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.0800000000000001,"15min":1.0469999999999999},"min":{"1min":0.78400000000000003,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.5229999999999999,"5min":1.54,"15min":2.4809999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1299999999999999,"15min":1.0920000000000001},"min":{"1min":0.78300000000000003,"5min":0.73799999999999999,"15min":0.55800000000000005},"max":{"1min":1.5,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":1.0089999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.1579999999999999,"15min":1.133},"min":{"1min":0.97799999999999998,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.556,"5min":1.8580000000000001,"15min":3.1200000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.1459999999999999},"min":{"1min":0.93200000000000005,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.508,"5min":1.736,"15min":2.6000000000000001},"last":0.94499999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.2709999999999999,"15min":1.2350000000000001},"min":{"1min":0.97599999999999998,"5min":0.82099999999999995,"15min":0.68999999999999995},"max":{"1min":1.5569999999999999,"5min":1.661,"15min":2.9079999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3160000000000001,"15min":1.2889999999999999},"min":{"1min":0.92400000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.331}]},{"osd":5,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.1990000000000001,"15min":1.1950000000000001},"min":{"1min":0.77100000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3640000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.89700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5109999999999999,"5min":1.944,"15min":2.4529999999999998},"last":1.375}]},{"osd":6,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4059999999999999,"15min":1.3620000000000001},"min":{"1min":1.075,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7250000000000001,"5min":2.403,"15min":2.7989999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4390000000000001,"15min":1.4279999999999999},"min":{"1min":0.877,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.5640000000000001,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.466,"15min":1.4379999999999999},"min":{"1min":1.1659999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5740000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.48},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4790000000000001,"15min":1.468},"min":{"1min":1.131,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.903,"5min":2.0680000000000001,"15min":2.698},"last":1.534}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1059,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194253,"statfs":{"total":200000000,"available":198915281,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79600000000000004,"15min":0.755},"min":{"1min":0.63600000000000001,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.4199999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.71199999999999997,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.323,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.71999999999999997,"5min":0.58899999999999997,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5609999999999999,"15min":2.403},"last":1.105},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1419999999999999,"15min":1.0700000000000001},"min":{"1min":0.90700000000000003,"5min":0.60299999999999998,"15min":0.47999999999999998},"max":{"1min":1.4099999999999999,"5min":1.5589999999999999,"15min":2.2709999999999999},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1020000000000001,"15min":1.026},"min":{"1min":0.83199999999999996,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":1.659,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.91700000000000004},"min":{"1min":0.72999999999999998,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.45,"15min":3.4510000000000001},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.1910000000000001},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.53000000000000003},"max":{"1min":1.4830000000000001,"5min":1.5940000000000001,"15min":3.585},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.248},"min":{"1min":1.0189999999999999,"5min":0.72499999999999998,"15min":0.61299999999999999},"max":{"1min":1.5800000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2190000000000001,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.73799999999999999,"15min":0.57099999999999995},"max":{"1min":1.546,"5min":1.609,"15min":3.738},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.26,"15min":1.26},"min":{"1min":0.97499999999999998,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.6719999999999999,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":1.302}]},{"osd":6,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3999999999999999,"15min":1.3320000000000001},"min":{"1min":1.159,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.5589999999999999,"5min":1.823,"15min":1.8979999999999999},"last":1.45},{"interface":"front","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.95399999999999996,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.6200000000000001,"5min":1.667,"15min":3.6059999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3129999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.577,"5min":1.851,"15min":3.391},"last":1.417},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.397,"15min":1.3560000000000001},"min":{"1min":1.1180000000000001,"5min":0.83699999999999997,"15min":0.625},"max":{"1min":1.7669999999999999,"5min":1.8360000000000001,"15min":2.0409999999999999},"last":1.5129999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908416,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.81999999999999995},"min":{"1min":0.64600000000000002,"5min":0.435,"15min":0.27900000000000003},"max":{"1min":1.2529999999999999,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0189999999999999,"15min":0.89400000000000002},"min":{"1min":0.76700000000000002,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.4750000000000001,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.93600000000000005,"15min":0.92800000000000005},"min":{"1min":0.81699999999999995,"5min":0.60999999999999999,"15min":0.40999999999999998},"max":{"1min":1.3460000000000001,"5min":1.387,"15min":2.1680000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.0760000000000001,"15min":0.96899999999999997},"min":{"1min":0.84799999999999998,"5min":0.58099999999999996,"15min":0.35899999999999999},"max":{"1min":1.4470000000000001,"5min":1.6279999999999999,"15min":1.6499999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.97399999999999998,"15min":0.89000000000000001},"min":{"1min":0.74399999999999999,"5min":0.46000000000000002,"15min":0.40200000000000002},"max":{"1min":1.4259999999999999,"5min":1.619,"15min":2.1339999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0920000000000001,"15min":1.0409999999999999},"min":{"1min":0.89500000000000002,"5min":0.56399999999999995,"15min":0.432},"max":{"1min":1.653,"5min":1.653,"15min":2.6589999999999998},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.2929999999999999,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.77000000000000002,"15min":0.56799999999999995},"max":{"1min":1.877,"5min":1.877,"15min":2.8759999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2869999999999999,"15min":1.224},"min":{"1min":1.1000000000000001,"5min":0.86699999999999999,"15min":0.503},"max":{"1min":1.5449999999999999,"5min":2.02,"15min":2.1360000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.244,"15min":1.1759999999999999},"min":{"1min":0.94799999999999995,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":2.024,"5min":2.028,"15min":2.8239999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3360000000000001,"15min":1.3049999999999999},"min":{"1min":1.1259999999999999,"5min":0.85599999999999998,"15min":0.627},"max":{"1min":1.7949999999999999,"5min":2.21,"15min":3.0070000000000001},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.3540000000000001,"15min":1.27},"min":{"1min":1.0800000000000001,"5min":0.83899999999999997,"15min":0.53800000000000003},"max":{"1min":1.647,"5min":1.919,"15min":2.8620000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3360000000000001,"15min":1.266},"min":{"1min":1.127,"5min":0.80700000000000005,"15min":0.70499999999999996},"max":{"1min":1.7949999999999999,"5min":2.6920000000000002,"15min":2.9750000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4159999999999999,"15min":1.319},"min":{"1min":1.1910000000000001,"5min":0.96799999999999997,"15min":0.65100000000000002},"max":{"1min":1.663,"5min":1.976,"15min":2.7749999999999999},"last":1.335},{"interface":"front","average":{"1min":1.397,"5min":1.351,"15min":1.2949999999999999},"min":{"1min":1.0760000000000001,"5min":0.85899999999999999,"15min":0.52500000000000002},"max":{"1min":1.9330000000000001,"5min":2.0179999999999998,"15min":3.0830000000000002},"last":1.3009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1064,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194247,"statfs":{"total":200000000,"available":198909491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.81999999999999995,"15min":0.88400000000000001},"min":{"1min":0.65200000000000002,"5min":0.433,"15min":0.34200000000000003},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.96399999999999997,"15min":1.095},"min":{"1min":0.70199999999999996,"5min":0.308,"15min":0.308},"max":{"1min":1.395,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":1.145}]},{"osd":2,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.76400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.639},"last":1.091},{"interface":"front","average":{"1min":1.044,"5min":0.99399999999999999,"15min":1.03},"min":{"1min":0.76600000000000001,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":3.0259999999999998},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1279999999999999,"15min":1.1579999999999999},"min":{"1min":0.84599999999999997,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":1.704,"15min":3.1539999999999999},"last":1.327},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.052,"15min":1.1220000000000001},"min":{"1min":0.749,"5min":0.505,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.778,"15min":4.8760000000000003},"last":1.659}]},{"osd":4,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1359999999999999,"15min":1.2010000000000001},"min":{"1min":0.92800000000000005,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.3340000000000001,"5min":1.6220000000000001,"15min":2.9449999999999998},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2689999999999999,"15min":1.3029999999999999},"min":{"1min":1.03,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.585,"5min":1.9490000000000001,"15min":3.2970000000000002},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.319,"15min":1.359},"min":{"1min":1.0449999999999999,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.577,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":1.1639999999999999,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.536,"5min":1.6970000000000001,"15min":3.0249999999999999},"last":1.9570000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3779999999999999,"15min":1.4239999999999999},"min":{"1min":1.095,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.581,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.879},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3149999999999999,"15min":1.379},"min":{"1min":1.05,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5820000000000001,"5min":1.77,"15min":2.9910000000000001},"last":2.181}]},{"osd":7,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3919999999999999,"15min":1.4430000000000001},"min":{"1min":1.155,"5min":0.89200000000000002,"15min":0.77100000000000002},"max":{"1min":1.8500000000000001,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4390000000000001,"15min":1.484},"min":{"1min":1.147,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":1.9750000000000001,"5min":2.0790000000000002,"15min":4.3040000000000003},"last":2.0169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193671,"statfs":{"total":200000000,"available":198319157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86299999999999999,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":1.2909999999999999,"5min":1.5840000000000001,"15min":4.8070000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.91400000000000003,"15min":1.1180000000000001},"min":{"1min":0.59499999999999997,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.4259999999999999,"5min":1.6459999999999999,"15min":10.816000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.96099999999999997,"15min":1.0249999999999999},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.4850000000000001,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.004,"5min":0.995,"15min":1.153},"min":{"1min":0.64500000000000002,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":1.355,"5min":1.7,"15min":10.781000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.96699999999999997,"15min":1.0409999999999999},"min":{"1min":0.52100000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.607,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":0.503},{"interface":"front","average":{"1min":1.093,"5min":1.0609999999999999,"15min":1.1439999999999999},"min":{"1min":0.58099999999999996,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.446,"5min":1.4610000000000001,"15min":4.9530000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1970000000000001,"15min":1.242},"min":{"1min":0.83999999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5189999999999999,"5min":2.129,"15min":3.669},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.284,"5min":1.256,"15min":1.3120000000000001},"min":{"1min":1.077,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5129999999999999,"5min":1.893,"15min":4.8780000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.272,"15min":1.472},"min":{"1min":0.81499999999999995,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":1.5740000000000001,"5min":1.9970000000000001,"15min":10.651},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.274,"5min":1.2490000000000001,"15min":1.3109999999999999},"min":{"1min":0.89700000000000002,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.516,"5min":1.954,"15min":3.7869999999999999},"last":1.0169999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.339,"15min":1.3779999999999999},"min":{"1min":0.89100000000000001,"5min":0.72699999999999998,"15min":0.66500000000000004},"max":{"1min":1.577,"5min":1.871,"15min":4.9210000000000003},"last":1.095},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.5049999999999999},"min":{"1min":0.873,"5min":0.747,"15min":0.747},"max":{"1min":1.651,"5min":1.95,"15min":10.708},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.351,"15min":1.5429999999999999},"min":{"1min":1.0249999999999999,"5min":0.77700000000000002,"15min":0.70999999999999996},"max":{"1min":1.792,"5min":2.085,"15min":10.74},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.3149999999999999,"15min":1.357},"min":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6639999999999999,"5min":1.8700000000000001,"15min":3.863},"last":1.0800000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:37:15.123 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:37:15.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.370+0000 7f83e7000640 1 Processor -- start 2024-02-02T13:37:15.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.370+0000 7f83e7000640 1 -- start start 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83e7000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83e00b0e10 0x7f83e00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83e7000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e00cc7c0 con 0x7f83e015a010 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83e7000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e00ce930 con 0x7f83e00ce130 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83e7000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e00cd640 con 0x7f83e00b0e10 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83dfe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f83e015a010 0x7f83e00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51540/0 (socket says 172.21.15.91:51540) 2024-02-02T13:37:15.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.371+0000 7f83dfe00640 1 -- 172.21.15.91:0/2989141565 learned_addr learned my addr 172.21.15.91:0/2989141565 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069925441 0 0) 0x7f83e00cc7c0 con 0x7f83e015a010 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003520 con 0x7f83e015a010 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83e5000640 1 --2- 172.21.15.91:0/2989141565 >> v2:172.21.15.113:3300/0 conn(0x7f83e00b0e10 0x7f83e00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842251198 0 0) 0x7f83c0003520 con 0x7f83e015a010 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 >> v2:172.21.15.113:3300/0 conn(0x7f83e00b0e10 msgr2=0x7f83e00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 --2- 172.21.15.91:0/2989141565 >> v2:172.21.15.113:3300/0 conn(0x7f83e00b0e10 0x7f83e00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 >> v1:172.21.15.113:6789/0 conn(0x7f83e00ce130 legacy=0x7f83e00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e006c890 con 0x7f83e015a010 2024-02-02T13:37:15.375 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f83d0001670 con 0x7f83e015a010 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 >> v1:172.21.15.91:6789/0 conn(0x7f83e015a010 legacy=0x7f83e00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 0x7f83c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c00047c0 0x7f83c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c0005330 0x7f83c000b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00cd640 con 0x7f83c00039c0 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c280 con 0x7f83c00047c0 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c440 con 0x7f83c0005330 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.372+0000 7f83e5000640 1 --2- 172.21.15.91:0/2989141565 >> v2:172.21.15.113:3300/0 conn(0x7f83e00b0e10 0x7f83e00c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 0x7f83c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 -- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c0005330 msgr2=0x7f83c000b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c0005330 0x7f83c000b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 -- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c00047c0 msgr2=0x7f83c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c00047c0 0x7f83c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.376 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83dfe00640 1 -- 172.21.15.91:0/2989141565 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0003520 con 0x7f83c00039c0 2024-02-02T13:37:15.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.373+0000 7f83e5000640 1 -- 172.21.15.91:0/2989141565 reap_dead start 2024-02-02T13:37:15.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.374+0000 7f83dfe00640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 0x7f83c0003dc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c190 tx=0x7f83c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1d6e822e1031cc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.374+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0005ad0 con 0x7f83c00039c0 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.374+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d0005c30 con 0x7f83c00039c0 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.374+0000 7f83df400640 1 -- 172.21.15.91:0/2989141565 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d002c810 con 0x7f83c00039c0 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 -- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 msgr2=0x7f83c0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 0x7f83c0003dc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c190 tx=0x7f83c00057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 -- 172.21.15.91:0/2989141565 shutdown_connections 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 --2- 172.21.15.91:0/2989141565 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83c00039c0 0x7f83c0003dc0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 -- 172.21.15.91:0/2989141565 >> 172.21.15.91:0/2989141565 conn(0x7f83e00cec80 msgr2=0x7f83e00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 -- 172.21.15.91:0/2989141565 shutdown_connections 2024-02-02T13:37:15.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.375+0000 7f83e7000640 1 -- 172.21.15.91:0/2989141565 wait complete. 2024-02-02T13:37:15.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.377+0000 7f83e7000640 1 Processor -- start 2024-02-02T13:37:15.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.377+0000 7f83e7000640 1 -- start start 2024-02-02T13:37:15.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c00039c0 0x7f83e01593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e00b1b50 0x7f83e00c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00c36f0 con 0x7f83e00b1fd0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00c33a0 con 0x7f83e00b1b50 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e7000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e015c290 con 0x7f83c00039c0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e5000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41578/0 (socket says 172.21.15.91:41578) 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.378+0000 7f83e5000640 1 -- 172.21.15.91:0/1711931483 learned_addr learned my addr 172.21.15.91:0/1711931483 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 -- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c00039c0 msgr2=0x7f83e01593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c00039c0 0x7f83e01593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 -- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e00b1b50 msgr2=0x7f83e00c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e00b1b50 0x7f83e00c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 -- 172.21.15.91:0/1711931483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00c68a0 con 0x7f83e00b1fd0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e5000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f83d800cb20 tx=0x7f83d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8016470 con 0x7f83e00b1fd0 2024-02-02T13:37:15.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80165d0 con 0x7f83e00b1fd0 2024-02-02T13:37:15.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8016980 con 0x7f83e00b1fd0 2024-02-02T13:37:15.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e00c6b80 con 0x7f83e00b1fd0 2024-02-02T13:37:15.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.379+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e0070af0 con 0x7f83e00b1fd0 2024-02-02T13:37:15.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.380+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e00c3830 con 0x7f83e00b1fd0 2024-02-02T13:37:15.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.381+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f83d8013070 con 0x7f83e00b1fd0 2024-02-02T13:37:15.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.382+0000 7f83dcc00640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 0x7f83b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.382+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83d8088d30 con 0x7f83e00b1fd0 2024-02-02T13:37:15.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.382+0000 7f83dfe00640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 0x7f83b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.383+0000 7f83dfe00640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 0x7f83b806c8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f83d0034000 tx=0x7f83d0039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.387+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f83e00c3830 con 0x7f83e00b1fd0 2024-02-02T13:37:15.569 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.568+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83e006cb70 con 0x7f83b806a430 2024-02-02T13:37:15.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.587+0000 7f83dcc00640 1 -- 172.21.15.91:0/1711931483 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283030 (secure 0 0 0) 0x7f83d00058f0 con 0x7f83b806a430 2024-02-02T13:37:15.592 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:15.593 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 msgr2=0x7f83b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 0x7f83b806c8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f83d0034000 tx=0x7f83d0039040 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 msgr2=0x7f83e00c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f83d800cb20 tx=0x7f83d800a340 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 shutdown_connections 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f83b806a430 0x7f83b806c8f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83e00b1fd0 0x7f83e00c6200 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e00b1b50 0x7f83e00c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 --2- 172.21.15.91:0/1711931483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83c00039c0 0x7f83e01593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 >> 172.21.15.91:0/1711931483 conn(0x7f83e00cec80 msgr2=0x7f83e015a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 shutdown_connections 2024-02-02T13:37:15.595 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.592+0000 7f83e7000640 1 -- 172.21.15.91:0/1711931483 wait complete. 2024-02-02T13:37:15.610 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-02-02T13:37:14.494499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552841,"statfs":{"total":1600000000,"available":1590109261,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.261026"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758980+0000","last_change":"2024-02-02T13:37:10.757647+0000","last_active":"2024-02-02T13:37:10.758980+0000","last_peered":"2024-02-02T13:37:10.758980+0000","last_clean":"2024-02-02T13:37:10.758980+0000","last_became_active":"2024-02-02T13:37:10.757412+0000","last_became_peered":"2024-02-02T13:37:10.757412+0000","last_unstale":"2024-02-02T13:37:10.758980+0000","last_undegraded":"2024-02-02T13:37:10.758980+0000","last_fullsized":"2024-02-02T13:37:10.758980+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:49.483601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761642+0000","last_change":"2024-02-02T13:37:10.761288+0000","last_active":"2024-02-02T13:37:10.761642+0000","last_peered":"2024-02-02T13:37:10.761642+0000","last_clean":"2024-02-02T13:37:10.761642+0000","last_became_active":"2024-02-02T13:37:10.761000+0000","last_became_peered":"2024-02-02T13:37:10.761000+0000","last_unstale":"2024-02-02T13:37:10.761642+0000","last_undegraded":"2024-02-02T13:37:10.761642+0000","last_fullsized":"2024-02-02T13:37:10.761642+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:04.144171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763219+0000","last_change":"2024-02-02T13:37:10.759825+0000","last_active":"2024-02-02T13:37:10.763219+0000","last_peered":"2024-02-02T13:37:10.763219+0000","last_clean":"2024-02-02T13:37:10.763219+0000","last_became_active":"2024-02-02T13:37:10.759566+0000","last_became_peered":"2024-02-02T13:37:10.759566+0000","last_unstale":"2024-02-02T13:37:10.763219+0000","last_undegraded":"2024-02-02T13:37:10.763219+0000","last_fullsized":"2024-02-02T13:37:10.763219+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:42.207188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737151+0000","last_change":"2024-02-02T13:37:10.736637+0000","last_active":"2024-02-02T13:37:10.737151+0000","last_peered":"2024-02-02T13:37:10.737151+0000","last_clean":"2024-02-02T13:37:10.737151+0000","last_became_active":"2024-02-02T13:37:10.736317+0000","last_became_peered":"2024-02-02T13:37:10.736317+0000","last_unstale":"2024-02-02T13:37:10.737151+0000","last_undegraded":"2024-02-02T13:37:10.737151+0000","last_fullsized":"2024-02-02T13:37:10.737151+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:02.995879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752828+0000","last_change":"2024-02-02T13:37:10.751108+0000","last_active":"2024-02-02T13:37:10.752828+0000","last_peered":"2024-02-02T13:37:10.752828+0000","last_clean":"2024-02-02T13:37:10.752828+0000","last_became_active":"2024-02-02T13:37:10.750717+0000","last_became_peered":"2024-02-02T13:37:10.750717+0000","last_unstale":"2024-02-02T13:37:10.752828+0000","last_undegraded":"2024-02-02T13:37:10.752828+0000","last_fullsized":"2024-02-02T13:37:10.752828+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:17:41.702140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743123+0000","last_change":"2024-02-02T13:37:10.742550+0000","last_active":"2024-02-02T13:37:10.743123+0000","last_peered":"2024-02-02T13:37:10.743123+0000","last_clean":"2024-02-02T13:37:10.743123+0000","last_became_active":"2024-02-02T13:37:10.742321+0000","last_became_peered":"2024-02-02T13:37:10.742321+0000","last_unstale":"2024-02-02T13:37:10.743123+0000","last_undegraded":"2024-02-02T13:37:10.743123+0000","last_fullsized":"2024-02-02T13:37:10.743123+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:48:44.853948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741354+0000","last_change":"2024-02-02T13:37:10.741216+0000","last_active":"2024-02-02T13:37:10.741354+0000","last_peered":"2024-02-02T13:37:10.741354+0000","last_clean":"2024-02-02T13:37:10.741354+0000","last_became_active":"2024-02-02T13:37:10.741007+0000","last_became_peered":"2024-02-02T13:37:10.741007+0000","last_unstale":"2024-02-02T13:37:10.741354+0000","last_undegraded":"2024-02-02T13:37:10.741354+0000","last_fullsized":"2024-02-02T13:37:10.741354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:01:55.569563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.756625+0000","last_change":"2024-02-02T13:37:10.755805+0000","last_active":"2024-02-02T13:37:10.756625+0000","last_peered":"2024-02-02T13:37:10.756625+0000","last_clean":"2024-02-02T13:37:10.756625+0000","last_became_active":"2024-02-02T13:37:10.755547+0000","last_became_peered":"2024-02-02T13:37:10.755547+0000","last_unstale":"2024-02-02T13:37:10.756625+0000","last_undegraded":"2024-02-02T13:37:10.756625+0000","last_fullsized":"2024-02-02T13:37:10.756625+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:19:37.771425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.757655+0000","last_change":"2024-02-02T13:37:10.757526+0000","last_active":"2024-02-02T13:37:10.757655+0000","last_peered":"2024-02-02T13:37:10.757655+0000","last_clean":"2024-02-02T13:37:10.757655+0000","last_became_active":"2024-02-02T13:37:10.757090+0000","last_became_peered":"2024-02-02T13:37:10.757090+0000","last_unstale":"2024-02-02T13:37:10.757655+0000","last_undegraded":"2024-02-02T13:37:10.757655+0000","last_fullsized":"2024-02-02T13:37:10.757655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:09:52.578652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752792+0000","last_change":"2024-02-02T13:37:10.750964+0000","last_active":"2024-02-02T13:37:10.752792+0000","last_peered":"2024-02-02T13:37:10.752792+0000","last_clean":"2024-02-02T13:37:10.752792+0000","last_became_active":"2024-02-02T13:37:10.750365+0000","last_became_peered":"2024-02-02T13:37:10.750365+0000","last_unstale":"2024-02-02T13:37:10.752792+0000","last_undegraded":"2024-02-02T13:37:10.752792+0000","last_fullsized":"2024-02-02T13:37:10.752792+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:23.943367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755900+0000","last_change":"2024-02-02T13:37:10.755826+0000","last_active":"2024-02-02T13:37:10.755900+0000","last_peered":"2024-02-02T13:37:10.755900+0000","last_clean":"2024-02-02T13:37:10.755900+0000","last_became_active":"2024-02-02T13:37:10.755564+0000","last_became_peered":"2024-02-02T13:37:10.755564+0000","last_unstale":"2024-02-02T13:37:10.755900+0000","last_undegraded":"2024-02-02T13:37:10.755900+0000","last_fullsized":"2024-02-02T13:37:10.755900+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:29.390021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758450+0000","last_change":"2024-02-02T13:37:10.758289+0000","last_active":"2024-02-02T13:37:10.758450+0000","last_peered":"2024-02-02T13:37:10.758450+0000","last_clean":"2024-02-02T13:37:10.758450+0000","last_became_active":"2024-02-02T13:37:10.758057+0000","last_became_peered":"2024-02-02T13:37:10.758057+0000","last_unstale":"2024-02-02T13:37:10.758450+0000","last_undegraded":"2024-02-02T13:37:10.758450+0000","last_fullsized":"2024-02-02T13:37:10.758450+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:12:20.100581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763053+0000","last_change":"2024-02-02T13:37:10.758831+0000","last_active":"2024-02-02T13:37:10.763053+0000","last_peered":"2024-02-02T13:37:10.763053+0000","last_clean":"2024-02-02T13:37:10.763053+0000","last_became_active":"2024-02-02T13:37:10.752456+0000","last_became_peered":"2024-02-02T13:37:10.752456+0000","last_unstale":"2024-02-02T13:37:10.763053+0000","last_undegraded":"2024-02-02T13:37:10.763053+0000","last_fullsized":"2024-02-02T13:37:10.763053+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:23.309621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763090+0000","last_change":"2024-02-02T13:37:10.759032+0000","last_active":"2024-02-02T13:37:10.763090+0000","last_peered":"2024-02-02T13:37:10.763090+0000","last_clean":"2024-02-02T13:37:10.763090+0000","last_became_active":"2024-02-02T13:37:10.758452+0000","last_became_peered":"2024-02-02T13:37:10.758452+0000","last_unstale":"2024-02-02T13:37:10.763090+0000","last_undegraded":"2024-02-02T13:37:10.763090+0000","last_fullsized":"2024-02-02T13:37:10.763090+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:17.955367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.744475+0000","last_change":"2024-02-02T13:37:10.741111+0000","last_active":"2024-02-02T13:37:10.744475+0000","last_peered":"2024-02-02T13:37:10.744475+0000","last_clean":"2024-02-02T13:37:10.744475+0000","last_became_active":"2024-02-02T13:37:10.739871+0000","last_became_peered":"2024-02-02T13:37:10.739871+0000","last_unstale":"2024-02-02T13:37:10.744475+0000","last_undegraded":"2024-02-02T13:37:10.744475+0000","last_fullsized":"2024-02-02T13:37:10.744475+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:25.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.739991+0000","last_change":"2024-02-02T13:37:10.739867+0000","last_active":"2024-02-02T13:37:10.739991+0000","last_peered":"2024-02-02T13:37:10.739991+0000","last_clean":"2024-02-02T13:37:10.739991+0000","last_became_active":"2024-02-02T13:37:10.739141+0000","last_became_peered":"2024-02-02T13:37:10.739141+0000","last_unstale":"2024-02-02T13:37:10.739991+0000","last_undegraded":"2024-02-02T13:37:10.739991+0000","last_fullsized":"2024-02-02T13:37:10.739991+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:18:49.483601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.752453+0000","last_change":"2024-02-02T13:37:10.752390+0000","last_active":"2024-02-02T13:37:10.752453+0000","last_peered":"2024-02-02T13:37:10.752453+0000","last_clean":"2024-02-02T13:37:10.752453+0000","last_became_active":"2024-02-02T13:37:10.752163+0000","last_became_peered":"2024-02-02T13:37:10.752163+0000","last_unstale":"2024-02-02T13:37:10.752453+0000","last_undegraded":"2024-02-02T13:37:10.752453+0000","last_fullsized":"2024-02-02T13:37:10.752453+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:41:20.266573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742285+0000","last_change":"2024-02-02T13:37:10.741671+0000","last_active":"2024-02-02T13:37:10.742285+0000","last_peered":"2024-02-02T13:37:10.742285+0000","last_clean":"2024-02-02T13:37:10.742285+0000","last_became_active":"2024-02-02T13:37:10.741449+0000","last_became_peered":"2024-02-02T13:37:10.741449+0000","last_unstale":"2024-02-02T13:37:10.742285+0000","last_undegraded":"2024-02-02T13:37:10.742285+0000","last_fullsized":"2024-02-02T13:37:10.742285+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:22:50.781898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.745913+0000","last_change":"2024-02-02T13:37:10.745858+0000","last_active":"2024-02-02T13:37:10.745913+0000","last_peered":"2024-02-02T13:37:10.745913+0000","last_clean":"2024-02-02T13:37:10.745913+0000","last_became_active":"2024-02-02T13:37:10.745738+0000","last_became_peered":"2024-02-02T13:37:10.745738+0000","last_unstale":"2024-02-02T13:37:10.745913+0000","last_undegraded":"2024-02-02T13:37:10.745913+0000","last_fullsized":"2024-02-02T13:37:10.745913+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:04:02.958869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763418+0000","last_change":"2024-02-02T13:37:10.762747+0000","last_active":"2024-02-02T13:37:10.763418+0000","last_peered":"2024-02-02T13:37:10.763418+0000","last_clean":"2024-02-02T13:37:10.763418+0000","last_became_active":"2024-02-02T13:37:10.762328+0000","last_became_peered":"2024-02-02T13:37:10.762328+0000","last_unstale":"2024-02-02T13:37:10.763418+0000","last_undegraded":"2024-02-02T13:37:10.763418+0000","last_fullsized":"2024-02-02T13:37:10.763418+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:02.995879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.756131+0000","last_change":"2024-02-02T13:37:10.756072+0000","last_active":"2024-02-02T13:37:10.756131+0000","last_peered":"2024-02-02T13:37:10.756131+0000","last_clean":"2024-02-02T13:37:10.756131+0000","last_became_active":"2024-02-02T13:37:10.755911+0000","last_became_peered":"2024-02-02T13:37:10.755911+0000","last_unstale":"2024-02-02T13:37:10.756131+0000","last_undegraded":"2024-02-02T13:37:10.756131+0000","last_fullsized":"2024-02-02T13:37:10.756131+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:37:25.836890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.754656+0000","last_change":"2024-02-02T13:37:10.754205+0000","last_active":"2024-02-02T13:37:10.754656+0000","last_peered":"2024-02-02T13:37:10.754656+0000","last_clean":"2024-02-02T13:37:10.754656+0000","last_became_active":"2024-02-02T13:37:10.753357+0000","last_became_peered":"2024-02-02T13:37:10.753357+0000","last_unstale":"2024-02-02T13:37:10.754656+0000","last_undegraded":"2024-02-02T13:37:10.754656+0000","last_fullsized":"2024-02-02T13:37:10.754656+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:58.647237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753359+0000","last_change":"2024-02-02T13:37:10.753290+0000","last_active":"2024-02-02T13:37:10.753359+0000","last_peered":"2024-02-02T13:37:10.753359+0000","last_clean":"2024-02-02T13:37:10.753359+0000","last_became_active":"2024-02-02T13:37:10.753119+0000","last_became_peered":"2024-02-02T13:37:10.753119+0000","last_unstale":"2024-02-02T13:37:10.753359+0000","last_undegraded":"2024-02-02T13:37:10.753359+0000","last_fullsized":"2024-02-02T13:37:10.753359+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:25:15.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763005+0000","last_change":"2024-02-02T13:37:10.762920+0000","last_active":"2024-02-02T13:37:10.763005+0000","last_peered":"2024-02-02T13:37:10.763005+0000","last_clean":"2024-02-02T13:37:10.763005+0000","last_became_active":"2024-02-02T13:37:10.762663+0000","last_became_peered":"2024-02-02T13:37:10.762663+0000","last_unstale":"2024-02-02T13:37:10.763005+0000","last_undegraded":"2024-02-02T13:37:10.763005+0000","last_fullsized":"2024-02-02T13:37:10.763005+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:43.040365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761694+0000","last_change":"2024-02-02T13:37:10.757723+0000","last_active":"2024-02-02T13:37:10.761694+0000","last_peered":"2024-02-02T13:37:10.761694+0000","last_clean":"2024-02-02T13:37:10.761694+0000","last_became_active":"2024-02-02T13:37:10.757487+0000","last_became_peered":"2024-02-02T13:37:10.757487+0000","last_unstale":"2024-02-02T13:37:10.761694+0000","last_undegraded":"2024-02-02T13:37:10.761694+0000","last_fullsized":"2024-02-02T13:37:10.761694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:39:47.177676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.736320+0000","last_change":"2024-02-02T13:37:10.735510+0000","last_active":"2024-02-02T13:37:10.736320+0000","last_peered":"2024-02-02T13:37:10.736320+0000","last_clean":"2024-02-02T13:37:10.736320+0000","last_became_active":"2024-02-02T13:37:10.734952+0000","last_became_peered":"2024-02-02T13:37:10.734952+0000","last_unstale":"2024-02-02T13:37:10.736320+0000","last_undegraded":"2024-02-02T13:37:10.736320+0000","last_fullsized":"2024-02-02T13:37:10.736320+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:34.763594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755430+0000","last_change":"2024-02-02T13:37:10.752353+0000","last_active":"2024-02-02T13:37:10.755430+0000","last_peered":"2024-02-02T13:37:10.755430+0000","last_clean":"2024-02-02T13:37:10.755430+0000","last_became_active":"2024-02-02T13:37:10.751213+0000","last_became_peered":"2024-02-02T13:37:10.751213+0000","last_unstale":"2024-02-02T13:37:10.755430+0000","last_undegraded":"2024-02-02T13:37:10.755430+0000","last_fullsized":"2024-02-02T13:37:10.755430+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:45:45.696881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741133+0000","last_change":"2024-02-02T13:37:10.740415+0000","last_active":"2024-02-02T13:37:10.741133+0000","last_peered":"2024-02-02T13:37:10.741133+0000","last_clean":"2024-02-02T13:37:10.741133+0000","last_became_active":"2024-02-02T13:37:10.740197+0000","last_became_peered":"2024-02-02T13:37:10.740197+0000","last_unstale":"2024-02-02T13:37:10.741133+0000","last_undegraded":"2024-02-02T13:37:10.741133+0000","last_fullsized":"2024-02-02T13:37:10.741133+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:35.274145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763031+0000","last_change":"2024-02-02T13:37:10.758711+0000","last_active":"2024-02-02T13:37:10.763031+0000","last_peered":"2024-02-02T13:37:10.763031+0000","last_clean":"2024-02-02T13:37:10.763031+0000","last_became_active":"2024-02-02T13:37:10.755528+0000","last_became_peered":"2024-02-02T13:37:10.755528+0000","last_unstale":"2024-02-02T13:37:10.763031+0000","last_undegraded":"2024-02-02T13:37:10.763031+0000","last_fullsized":"2024-02-02T13:37:10.763031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:31.472624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743271+0000","last_change":"2024-02-02T13:37:10.738241+0000","last_active":"2024-02-02T13:37:10.743271+0000","last_peered":"2024-02-02T13:37:10.743271+0000","last_clean":"2024-02-02T13:37:10.743271+0000","last_became_active":"2024-02-02T13:37:10.737661+0000","last_became_peered":"2024-02-02T13:37:10.737661+0000","last_unstale":"2024-02-02T13:37:10.743271+0000","last_undegraded":"2024-02-02T13:37:10.743271+0000","last_fullsized":"2024-02-02T13:37:10.743271+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:58.647237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749291+0000","last_change":"2024-02-02T13:37:10.748594+0000","last_active":"2024-02-02T13:37:10.749291+0000","last_peered":"2024-02-02T13:37:10.749291+0000","last_clean":"2024-02-02T13:37:10.749291+0000","last_became_active":"2024-02-02T13:37:10.746337+0000","last_became_peered":"2024-02-02T13:37:10.746337+0000","last_unstale":"2024-02-02T13:37:10.749291+0000","last_undegraded":"2024-02-02T13:37:10.749291+0000","last_fullsized":"2024-02-02T13:37:10.749291+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:44.815783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742314+0000","last_change":"2024-02-02T13:37:10.742230+0000","last_active":"2024-02-02T13:37:10.742314+0000","last_peered":"2024-02-02T13:37:10.742314+0000","last_clean":"2024-02-02T13:37:10.742314+0000","last_became_active":"2024-02-02T13:37:10.741881+0000","last_became_peered":"2024-02-02T13:37:10.741881+0000","last_unstale":"2024-02-02T13:37:10.742314+0000","last_undegraded":"2024-02-02T13:37:10.742314+0000","last_fullsized":"2024-02-02T13:37:10.742314+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:56:55.759396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735180+0000","last_change":"2024-02-02T13:37:10.734889+0000","last_active":"2024-02-02T13:37:10.735180+0000","last_peered":"2024-02-02T13:37:10.735180+0000","last_clean":"2024-02-02T13:37:10.735180+0000","last_became_active":"2024-02-02T13:37:10.734691+0000","last_became_peered":"2024-02-02T13:37:10.734691+0000","last_unstale":"2024-02-02T13:37:10.735180+0000","last_undegraded":"2024-02-02T13:37:10.735180+0000","last_fullsized":"2024-02-02T13:37:10.735180+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:34:25.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"194'4","reported_seq":14,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763490+0000","last_change":"2024-02-02T13:37:10.264228+0000","last_active":"2024-02-02T13:37:10.763490+0000","last_peered":"2024-02-02T13:37:10.763490+0000","last_clean":"2024-02-02T13:37:10.763490+0000","last_became_active":"2024-02-02T13:37:10.263859+0000","last_became_peered":"2024-02-02T13:37:10.263859+0000","last_unstale":"2024-02-02T13:37:10.763490+0000","last_undegraded":"2024-02-02T13:37:10.763490+0000","last_fullsized":"2024-02-02T13:37:10.763490+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:43:33.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.751298+0000","last_change":"2024-02-02T13:37:10.749933+0000","last_active":"2024-02-02T13:37:10.751298+0000","last_peered":"2024-02-02T13:37:10.751298+0000","last_clean":"2024-02-02T13:37:10.751298+0000","last_became_active":"2024-02-02T13:37:10.745311+0000","last_became_peered":"2024-02-02T13:37:10.745311+0000","last_unstale":"2024-02-02T13:37:10.751298+0000","last_undegraded":"2024-02-02T13:37:10.751298+0000","last_fullsized":"2024-02-02T13:37:10.751298+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:59:43.040365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758431+0000","last_change":"2024-02-02T13:37:10.270975+0000","last_active":"2024-02-02T13:37:10.758431+0000","last_peered":"2024-02-02T13:37:10.758431+0000","last_clean":"2024-02-02T13:37:10.758431+0000","last_became_active":"2024-02-02T13:37:10.270698+0000","last_became_peered":"2024-02-02T13:37:10.270698+0000","last_unstale":"2024-02-02T13:37:10.758431+0000","last_undegraded":"2024-02-02T13:37:10.758431+0000","last_fullsized":"2024-02-02T13:37:10.758431+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:00:23.242278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.747889+0000","last_change":"2024-02-02T13:37:10.747825+0000","last_active":"2024-02-02T13:37:10.747889+0000","last_peered":"2024-02-02T13:37:10.747889+0000","last_clean":"2024-02-02T13:37:10.747889+0000","last_became_active":"2024-02-02T13:37:10.747606+0000","last_became_peered":"2024-02-02T13:37:10.747606+0000","last_unstale":"2024-02-02T13:37:10.747889+0000","last_undegraded":"2024-02-02T13:37:10.747889+0000","last_fullsized":"2024-02-02T13:37:10.747889+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.681025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755362+0000","last_change":"2024-02-02T13:37:10.261869+0000","last_active":"2024-02-02T13:37:10.755362+0000","last_peered":"2024-02-02T13:37:10.755362+0000","last_clean":"2024-02-02T13:37:10.755362+0000","last_became_active":"2024-02-02T13:37:10.261441+0000","last_became_peered":"2024-02-02T13:37:10.261441+0000","last_unstale":"2024-02-02T13:37:10.755362+0000","last_undegraded":"2024-02-02T13:37:10.755362+0000","last_fullsized":"2024-02-02T13:37:10.755362+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:33:08.796969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758375+0000","last_change":"2024-02-02T13:37:10.756423+0000","last_active":"2024-02-02T13:37:10.758375+0000","last_peered":"2024-02-02T13:37:10.758375+0000","last_clean":"2024-02-02T13:37:10.758375+0000","last_became_active":"2024-02-02T13:37:10.756206+0000","last_became_peered":"2024-02-02T13:37:10.756206+0000","last_unstale":"2024-02-02T13:37:10.758375+0000","last_undegraded":"2024-02-02T13:37:10.758375+0000","last_fullsized":"2024-02-02T13:37:10.758375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:28:46.985468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750186+0000","last_change":"2024-02-02T13:37:10.261110+0000","last_active":"2024-02-02T13:37:10.750186+0000","last_peered":"2024-02-02T13:37:10.750186+0000","last_clean":"2024-02-02T13:37:10.750186+0000","last_became_active":"2024-02-02T13:37:10.260657+0000","last_became_peered":"2024-02-02T13:37:10.260657+0000","last_unstale":"2024-02-02T13:37:10.750186+0000","last_undegraded":"2024-02-02T13:37:10.750186+0000","last_fullsized":"2024-02-02T13:37:10.750186+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:47:34.212325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734757+0000","last_change":"2024-02-02T13:37:10.734178+0000","last_active":"2024-02-02T13:37:10.734757+0000","last_peered":"2024-02-02T13:37:10.734757+0000","last_clean":"2024-02-02T13:37:10.734757+0000","last_became_active":"2024-02-02T13:37:10.733878+0000","last_became_peered":"2024-02-02T13:37:10.733878+0000","last_unstale":"2024-02-02T13:37:10.734757+0000","last_undegraded":"2024-02-02T13:37:10.734757+0000","last_fullsized":"2024-02-02T13:37:10.734757+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:44:43.754974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758005+0000","last_change":"2024-02-02T13:37:10.263667+0000","last_active":"2024-02-02T13:37:10.758005+0000","last_peered":"2024-02-02T13:37:10.758005+0000","last_clean":"2024-02-02T13:37:10.758005+0000","last_became_active":"2024-02-02T13:37:10.262372+0000","last_became_peered":"2024-02-02T13:37:10.262372+0000","last_unstale":"2024-02-02T13:37:10.758005+0000","last_undegraded":"2024-02-02T13:37:10.758005+0000","last_fullsized":"2024-02-02T13:37:10.758005+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:07:21.527509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735997+0000","last_change":"2024-02-02T13:37:10.735832+0000","last_active":"2024-02-02T13:37:10.735997+0000","last_peered":"2024-02-02T13:37:10.735997+0000","last_clean":"2024-02-02T13:37:10.735997+0000","last_became_active":"2024-02-02T13:37:10.735433+0000","last_became_peered":"2024-02-02T13:37:10.735433+0000","last_unstale":"2024-02-02T13:37:10.735997+0000","last_undegraded":"2024-02-02T13:37:10.735997+0000","last_fullsized":"2024-02-02T13:37:10.735997+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:31.472624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742922+0000","last_change":"2024-02-02T13:37:10.266904+0000","last_active":"2024-02-02T13:37:10.742922+0000","last_peered":"2024-02-02T13:37:10.742922+0000","last_clean":"2024-02-02T13:37:10.742922+0000","last_became_active":"2024-02-02T13:37:10.265737+0000","last_became_peered":"2024-02-02T13:37:10.265737+0000","last_unstale":"2024-02-02T13:37:10.742922+0000","last_undegraded":"2024-02-02T13:37:10.742922+0000","last_fullsized":"2024-02-02T13:37:10.742922+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:56:15.283974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.740045+0000","last_change":"2024-02-02T13:37:10.739985+0000","last_active":"2024-02-02T13:37:10.740045+0000","last_peered":"2024-02-02T13:37:10.740045+0000","last_clean":"2024-02-02T13:37:10.740045+0000","last_became_active":"2024-02-02T13:37:10.739475+0000","last_became_peered":"2024-02-02T13:37:10.739475+0000","last_unstale":"2024-02-02T13:37:10.740045+0000","last_undegraded":"2024-02-02T13:37:10.740045+0000","last_fullsized":"2024-02-02T13:37:10.740045+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:42.207188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742581+0000","last_change":"2024-02-02T13:37:10.267684+0000","last_active":"2024-02-02T13:37:10.742581+0000","last_peered":"2024-02-02T13:37:10.742581+0000","last_clean":"2024-02-02T13:37:10.742581+0000","last_became_active":"2024-02-02T13:37:10.267126+0000","last_became_peered":"2024-02-02T13:37:10.267126+0000","last_unstale":"2024-02-02T13:37:10.742581+0000","last_undegraded":"2024-02-02T13:37:10.742581+0000","last_fullsized":"2024-02-02T13:37:10.742581+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:22.294877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.741601+0000","last_change":"2024-02-02T13:37:10.741539+0000","last_active":"2024-02-02T13:37:10.741601+0000","last_peered":"2024-02-02T13:37:10.741601+0000","last_clean":"2024-02-02T13:37:10.741601+0000","last_became_active":"2024-02-02T13:37:10.741344+0000","last_became_peered":"2024-02-02T13:37:10.741344+0000","last_unstale":"2024-02-02T13:37:10.741601+0000","last_undegraded":"2024-02-02T13:37:10.741601+0000","last_fullsized":"2024-02-02T13:37:10.741601+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:41:54.323783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.729912+0000","last_change":"2024-02-02T13:37:10.266071+0000","last_active":"2024-02-02T13:37:10.729912+0000","last_peered":"2024-02-02T13:37:10.729912+0000","last_clean":"2024-02-02T13:37:10.729912+0000","last_became_active":"2024-02-02T13:37:10.265721+0000","last_became_peered":"2024-02-02T13:37:10.265721+0000","last_unstale":"2024-02-02T13:37:10.729912+0000","last_undegraded":"2024-02-02T13:37:10.729912+0000","last_fullsized":"2024-02-02T13:37:10.729912+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:49.530021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734007+0000","last_change":"2024-02-02T13:37:10.733708+0000","last_active":"2024-02-02T13:37:10.734007+0000","last_peered":"2024-02-02T13:37:10.734007+0000","last_clean":"2024-02-02T13:37:10.734007+0000","last_became_active":"2024-02-02T13:37:10.733476+0000","last_became_peered":"2024-02-02T13:37:10.733476+0000","last_unstale":"2024-02-02T13:37:10.734007+0000","last_undegraded":"2024-02-02T13:37:10.734007+0000","last_fullsized":"2024-02-02T13:37:10.734007+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:59:32.706979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.738698+0000","last_change":"2024-02-02T13:37:10.258610+0000","last_active":"2024-02-02T13:37:10.738698+0000","last_peered":"2024-02-02T13:37:10.738698+0000","last_clean":"2024-02-02T13:37:10.738698+0000","last_became_active":"2024-02-02T13:37:10.257988+0000","last_became_peered":"2024-02-02T13:37:10.257988+0000","last_unstale":"2024-02-02T13:37:10.738698+0000","last_undegraded":"2024-02-02T13:37:10.738698+0000","last_fullsized":"2024-02-02T13:37:10.738698+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:23.009072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734025+0000","last_change":"2024-02-02T13:37:10.733739+0000","last_active":"2024-02-02T13:37:10.734025+0000","last_peered":"2024-02-02T13:37:10.734025+0000","last_clean":"2024-02-02T13:37:10.734025+0000","last_became_active":"2024-02-02T13:37:10.733493+0000","last_became_peered":"2024-02-02T13:37:10.733493+0000","last_unstale":"2024-02-02T13:37:10.734025+0000","last_undegraded":"2024-02-02T13:37:10.734025+0000","last_fullsized":"2024-02-02T13:37:10.734025+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:29.728065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"194'3","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749715+0000","last_change":"2024-02-02T13:37:10.262108+0000","last_active":"2024-02-02T13:37:10.749715+0000","last_peered":"2024-02-02T13:37:10.749715+0000","last_clean":"2024-02-02T13:37:10.749715+0000","last_became_active":"2024-02-02T13:37:10.261689+0000","last_became_peered":"2024-02-02T13:37:10.261689+0000","last_unstale":"2024-02-02T13:37:10.749715+0000","last_undegraded":"2024-02-02T13:37:10.749715+0000","last_fullsized":"2024-02-02T13:37:10.749715+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:20:00.812570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.758584+0000","last_change":"2024-02-02T13:37:10.755414+0000","last_active":"2024-02-02T13:37:10.758584+0000","last_peered":"2024-02-02T13:37:10.758584+0000","last_clean":"2024-02-02T13:37:10.758584+0000","last_became_active":"2024-02-02T13:37:10.755133+0000","last_became_peered":"2024-02-02T13:37:10.755133+0000","last_unstale":"2024-02-02T13:37:10.758584+0000","last_undegraded":"2024-02-02T13:37:10.758584+0000","last_fullsized":"2024-02-02T13:37:10.758584+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:21:48.940322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"194'6","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761691+0000","last_change":"2024-02-02T13:37:10.264832+0000","last_active":"2024-02-02T13:37:10.761691+0000","last_peered":"2024-02-02T13:37:10.761691+0000","last_clean":"2024-02-02T13:37:10.761691+0000","last_became_active":"2024-02-02T13:37:10.264585+0000","last_became_peered":"2024-02-02T13:37:10.264585+0000","last_unstale":"2024-02-02T13:37:10.761691+0000","last_undegraded":"2024-02-02T13:37:10.761691+0000","last_fullsized":"2024-02-02T13:37:10.761691+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:28:01.981135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755552+0000","last_change":"2024-02-02T13:37:10.755413+0000","last_active":"2024-02-02T13:37:10.755552+0000","last_peered":"2024-02-02T13:37:10.755552+0000","last_clean":"2024-02-02T13:37:10.755552+0000","last_became_active":"2024-02-02T13:37:10.755196+0000","last_became_peered":"2024-02-02T13:37:10.755196+0000","last_unstale":"2024-02-02T13:37:10.755552+0000","last_undegraded":"2024-02-02T13:37:10.755552+0000","last_fullsized":"2024-02-02T13:37:10.755552+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:59:43.507576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743996+0000","last_change":"2024-02-02T13:37:10.260677+0000","last_active":"2024-02-02T13:37:10.743996+0000","last_peered":"2024-02-02T13:37:10.743996+0000","last_clean":"2024-02-02T13:37:10.743996+0000","last_became_active":"2024-02-02T13:37:10.260231+0000","last_became_peered":"2024-02-02T13:37:10.260231+0000","last_unstale":"2024-02-02T13:37:10.743996+0000","last_undegraded":"2024-02-02T13:37:10.743996+0000","last_fullsized":"2024-02-02T13:37:10.743996+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:54:29.014585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.761158+0000","last_change":"2024-02-02T13:37:10.761115+0000","last_active":"2024-02-02T13:37:10.761158+0000","last_peered":"2024-02-02T13:37:10.761158+0000","last_clean":"2024-02-02T13:37:10.761158+0000","last_became_active":"2024-02-02T13:37:10.760969+0000","last_became_peered":"2024-02-02T13:37:10.760969+0000","last_unstale":"2024-02-02T13:37:10.761158+0000","last_undegraded":"2024-02-02T13:37:10.761158+0000","last_fullsized":"2024-02-02T13:37:10.761158+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:35.973959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.728351+0000","last_change":"2024-02-02T13:37:10.267512+0000","last_active":"2024-02-02T13:37:10.728351+0000","last_peered":"2024-02-02T13:37:10.728351+0000","last_clean":"2024-02-02T13:37:10.728351+0000","last_became_active":"2024-02-02T13:37:10.267270+0000","last_became_peered":"2024-02-02T13:37:10.267270+0000","last_unstale":"2024-02-02T13:37:10.728351+0000","last_undegraded":"2024-02-02T13:37:10.728351+0000","last_fullsized":"2024-02-02T13:37:10.728351+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:17:40.687396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753007+0000","last_change":"2024-02-02T13:37:10.752945+0000","last_active":"2024-02-02T13:37:10.753007+0000","last_peered":"2024-02-02T13:37:10.753007+0000","last_clean":"2024-02-02T13:37:10.753007+0000","last_became_active":"2024-02-02T13:37:10.752716+0000","last_became_peered":"2024-02-02T13:37:10.752716+0000","last_unstale":"2024-02-02T13:37:10.753007+0000","last_undegraded":"2024-02-02T13:37:10.753007+0000","last_fullsized":"2024-02-02T13:37:10.753007+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:17.504456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732032+0000","last_change":"2024-02-02T13:37:10.263254+0000","last_active":"2024-02-02T13:37:10.732032+0000","last_peered":"2024-02-02T13:37:10.732032+0000","last_clean":"2024-02-02T13:37:10.732032+0000","last_became_active":"2024-02-02T13:37:10.262564+0000","last_became_peered":"2024-02-02T13:37:10.262564+0000","last_unstale":"2024-02-02T13:37:10.732032+0000","last_undegraded":"2024-02-02T13:37:10.732032+0000","last_fullsized":"2024-02-02T13:37:10.732032+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:01:03.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755466+0000","last_change":"2024-02-02T13:37:10.752814+0000","last_active":"2024-02-02T13:37:10.755466+0000","last_peered":"2024-02-02T13:37:10.755466+0000","last_clean":"2024-02-02T13:37:10.755466+0000","last_became_active":"2024-02-02T13:37:10.752052+0000","last_became_peered":"2024-02-02T13:37:10.752052+0000","last_unstale":"2024-02-02T13:37:10.755466+0000","last_undegraded":"2024-02-02T13:37:10.755466+0000","last_fullsized":"2024-02-02T13:37:10.755466+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:50.153639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"194'8","reported_seq":18,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.764998+0000","last_change":"2024-02-02T13:37:10.263616+0000","last_active":"2024-02-02T13:37:10.764998+0000","last_peered":"2024-02-02T13:37:10.764998+0000","last_clean":"2024-02-02T13:37:10.764998+0000","last_became_active":"2024-02-02T13:37:10.263370+0000","last_became_peered":"2024-02-02T13:37:10.263370+0000","last_unstale":"2024-02-02T13:37:10.764998+0000","last_undegraded":"2024-02-02T13:37:10.764998+0000","last_fullsized":"2024-02-02T13:37:10.764998+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:49.138894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743915+0000","last_change":"2024-02-02T13:37:10.743838+0000","last_active":"2024-02-02T13:37:10.743915+0000","last_peered":"2024-02-02T13:37:10.743915+0000","last_clean":"2024-02-02T13:37:10.743915+0000","last_became_active":"2024-02-02T13:37:10.743355+0000","last_became_peered":"2024-02-02T13:37:10.743355+0000","last_unstale":"2024-02-02T13:37:10.743915+0000","last_undegraded":"2024-02-02T13:37:10.743915+0000","last_fullsized":"2024-02-02T13:37:10.743915+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:25:50.153639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763169+0000","last_change":"2024-02-02T13:37:10.269272+0000","last_active":"2024-02-02T13:37:10.763169+0000","last_peered":"2024-02-02T13:37:10.763169+0000","last_clean":"2024-02-02T13:37:10.763169+0000","last_became_active":"2024-02-02T13:37:10.268980+0000","last_became_peered":"2024-02-02T13:37:10.268980+0000","last_unstale":"2024-02-02T13:37:10.763169+0000","last_undegraded":"2024-02-02T13:37:10.763169+0000","last_fullsized":"2024-02-02T13:37:10.763169+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:34:41.693441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.740720+0000","last_change":"2024-02-02T13:37:10.740641+0000","last_active":"2024-02-02T13:37:10.740720+0000","last_peered":"2024-02-02T13:37:10.740720+0000","last_clean":"2024-02-02T13:37:10.740720+0000","last_became_active":"2024-02-02T13:37:10.740386+0000","last_became_peered":"2024-02-02T13:37:10.740386+0000","last_unstale":"2024-02-02T13:37:10.740720+0000","last_undegraded":"2024-02-02T13:37:10.740720+0000","last_fullsized":"2024-02-02T13:37:10.740720+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:59:04.790269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763374+0000","last_change":"2024-02-02T13:37:10.259458+0000","last_active":"2024-02-02T13:37:10.763374+0000","last_peered":"2024-02-02T13:37:10.763374+0000","last_clean":"2024-02-02T13:37:10.763374+0000","last_became_active":"2024-02-02T13:37:10.258977+0000","last_became_peered":"2024-02-02T13:37:10.258977+0000","last_unstale":"2024-02-02T13:37:10.763374+0000","last_undegraded":"2024-02-02T13:37:10.763374+0000","last_fullsized":"2024-02-02T13:37:10.763374+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:49:05.058927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735979+0000","last_change":"2024-02-02T13:37:10.735667+0000","last_active":"2024-02-02T13:37:10.735979+0000","last_peered":"2024-02-02T13:37:10.735979+0000","last_clean":"2024-02-02T13:37:10.735979+0000","last_became_active":"2024-02-02T13:37:10.735161+0000","last_became_peered":"2024-02-02T13:37:10.735161+0000","last_unstale":"2024-02-02T13:37:10.735979+0000","last_undegraded":"2024-02-02T13:37:10.735979+0000","last_fullsized":"2024-02-02T13:37:10.735979+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:51:17.955367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"194'1","reported_seq":11,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.751746+0000","last_change":"2024-02-02T13:37:10.270996+0000","last_active":"2024-02-02T13:37:10.751746+0000","last_peered":"2024-02-02T13:37:10.751746+0000","last_clean":"2024-02-02T13:37:10.751746+0000","last_became_active":"2024-02-02T13:37:10.269739+0000","last_became_peered":"2024-02-02T13:37:10.269739+0000","last_unstale":"2024-02-02T13:37:10.751746+0000","last_undegraded":"2024-02-02T13:37:10.751746+0000","last_fullsized":"2024-02-02T13:37:10.751746+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:19:27.013508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.747722+0000","last_change":"2024-02-02T13:37:10.747652+0000","last_active":"2024-02-02T13:37:10.747722+0000","last_peered":"2024-02-02T13:37:10.747722+0000","last_clean":"2024-02-02T13:37:10.747722+0000","last_became_active":"2024-02-02T13:37:10.747313+0000","last_became_peered":"2024-02-02T13:37:10.747313+0000","last_unstale":"2024-02-02T13:37:10.747722+0000","last_undegraded":"2024-02-02T13:37:10.747722+0000","last_fullsized":"2024-02-02T13:37:10.747722+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:53:53.923501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.730883+0000","last_change":"2024-02-02T13:37:10.263774+0000","last_active":"2024-02-02T13:37:10.730883+0000","last_peered":"2024-02-02T13:37:10.730883+0000","last_clean":"2024-02-02T13:37:10.730883+0000","last_became_active":"2024-02-02T13:37:10.263472+0000","last_became_peered":"2024-02-02T13:37:10.263472+0000","last_unstale":"2024-02-02T13:37:10.730883+0000","last_undegraded":"2024-02-02T13:37:10.730883+0000","last_fullsized":"2024-02-02T13:37:10.730883+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:10:56.736246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763071+0000","last_change":"2024-02-02T13:37:10.758876+0000","last_active":"2024-02-02T13:37:10.763071+0000","last_peered":"2024-02-02T13:37:10.763071+0000","last_clean":"2024-02-02T13:37:10.763071+0000","last_became_active":"2024-02-02T13:37:10.758129+0000","last_became_peered":"2024-02-02T13:37:10.758129+0000","last_unstale":"2024-02-02T13:37:10.763071+0000","last_undegraded":"2024-02-02T13:37:10.763071+0000","last_fullsized":"2024-02-02T13:37:10.763071+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:17.504456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.755452+0000","last_change":"2024-02-02T13:37:10.266740+0000","last_active":"2024-02-02T13:37:10.755452+0000","last_peered":"2024-02-02T13:37:10.755452+0000","last_clean":"2024-02-02T13:37:10.755452+0000","last_became_active":"2024-02-02T13:37:10.263522+0000","last_became_peered":"2024-02-02T13:37:10.263522+0000","last_unstale":"2024-02-02T13:37:10.755452+0000","last_undegraded":"2024-02-02T13:37:10.755452+0000","last_fullsized":"2024-02-02T13:37:10.755452+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:55.383915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750548+0000","last_change":"2024-02-02T13:37:10.750435+0000","last_active":"2024-02-02T13:37:10.750548+0000","last_peered":"2024-02-02T13:37:10.750548+0000","last_clean":"2024-02-02T13:37:10.750548+0000","last_became_active":"2024-02-02T13:37:10.750123+0000","last_became_peered":"2024-02-02T13:37:10.750123+0000","last_unstale":"2024-02-02T13:37:10.750548+0000","last_undegraded":"2024-02-02T13:37:10.750548+0000","last_fullsized":"2024-02-02T13:37:10.750548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:19:53.625421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.733512+0000","last_change":"2024-02-02T13:37:10.268767+0000","last_active":"2024-02-02T13:37:10.733512+0000","last_peered":"2024-02-02T13:37:10.733512+0000","last_clean":"2024-02-02T13:37:10.733512+0000","last_became_active":"2024-02-02T13:37:10.266597+0000","last_became_peered":"2024-02-02T13:37:10.266597+0000","last_unstale":"2024-02-02T13:37:10.733512+0000","last_undegraded":"2024-02-02T13:37:10.733512+0000","last_fullsized":"2024-02-02T13:37:10.733512+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:59:03.775525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.760534+0000","last_change":"2024-02-02T13:37:10.760442+0000","last_active":"2024-02-02T13:37:10.760534+0000","last_peered":"2024-02-02T13:37:10.760534+0000","last_clean":"2024-02-02T13:37:10.760534+0000","last_became_active":"2024-02-02T13:37:10.759219+0000","last_became_peered":"2024-02-02T13:37:10.759219+0000","last_unstale":"2024-02-02T13:37:10.760534+0000","last_undegraded":"2024-02-02T13:37:10.760534+0000","last_fullsized":"2024-02-02T13:37:10.760534+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:52:35.973959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732546+0000","last_change":"2024-02-02T13:37:10.257847+0000","last_active":"2024-02-02T13:37:10.732546+0000","last_peered":"2024-02-02T13:37:10.732546+0000","last_clean":"2024-02-02T13:37:10.732546+0000","last_became_active":"2024-02-02T13:37:10.253858+0000","last_became_peered":"2024-02-02T13:37:10.253858+0000","last_unstale":"2024-02-02T13:37:10.732546+0000","last_undegraded":"2024-02-02T13:37:10.732546+0000","last_fullsized":"2024-02-02T13:37:10.732546+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:41:00.482921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.753771+0000","last_change":"2024-02-02T13:37:10.753646+0000","last_active":"2024-02-02T13:37:10.753771+0000","last_peered":"2024-02-02T13:37:10.753771+0000","last_clean":"2024-02-02T13:37:10.753771+0000","last_became_active":"2024-02-02T13:37:10.753450+0000","last_became_peered":"2024-02-02T13:37:10.753450+0000","last_unstale":"2024-02-02T13:37:10.753771+0000","last_undegraded":"2024-02-02T13:37:10.753771+0000","last_fullsized":"2024-02-02T13:37:10.753771+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:07.886376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734473+0000","last_change":"2024-02-02T13:37:10.271570+0000","last_active":"2024-02-02T13:37:10.734473+0000","last_peered":"2024-02-02T13:37:10.734473+0000","last_clean":"2024-02-02T13:37:10.734473+0000","last_became_active":"2024-02-02T13:37:10.270785+0000","last_became_peered":"2024-02-02T13:37:10.270785+0000","last_unstale":"2024-02-02T13:37:10.734473+0000","last_undegraded":"2024-02-02T13:37:10.734473+0000","last_fullsized":"2024-02-02T13:37:10.734473+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:21:28.713320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.734951+0000","last_change":"2024-02-02T13:37:10.734752+0000","last_active":"2024-02-02T13:37:10.734951+0000","last_peered":"2024-02-02T13:37:10.734951+0000","last_clean":"2024-02-02T13:37:10.734951+0000","last_became_active":"2024-02-02T13:37:10.734494+0000","last_became_peered":"2024-02-02T13:37:10.734494+0000","last_unstale":"2024-02-02T13:37:10.734951+0000","last_undegraded":"2024-02-02T13:37:10.734951+0000","last_fullsized":"2024-02-02T13:37:10.734951+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:36:58.507132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"194'2","reported_seq":12,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750411+0000","last_change":"2024-02-02T13:37:10.270731+0000","last_active":"2024-02-02T13:37:10.750411+0000","last_peered":"2024-02-02T13:37:10.750411+0000","last_clean":"2024-02-02T13:37:10.750411+0000","last_became_active":"2024-02-02T13:37:10.270243+0000","last_became_peered":"2024-02-02T13:37:10.270243+0000","last_unstale":"2024-02-02T13:37:10.750411+0000","last_undegraded":"2024-02-02T13:37:10.750411+0000","last_fullsized":"2024-02-02T13:37:10.750411+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:09:51.563908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.754667+0000","last_change":"2024-02-02T13:37:10.754618+0000","last_active":"2024-02-02T13:37:10.754667+0000","last_peered":"2024-02-02T13:37:10.754667+0000","last_clean":"2024-02-02T13:37:10.754667+0000","last_became_active":"2024-02-02T13:37:10.754468+0000","last_became_peered":"2024-02-02T13:37:10.754468+0000","last_unstale":"2024-02-02T13:37:10.754667+0000","last_undegraded":"2024-02-02T13:37:10.754667+0000","last_fullsized":"2024-02-02T13:37:10.754667+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:53:03.196854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.750723+0000","last_change":"2024-02-02T13:37:10.262995+0000","last_active":"2024-02-02T13:37:10.750723+0000","last_peered":"2024-02-02T13:37:10.750723+0000","last_clean":"2024-02-02T13:37:10.750723+0000","last_became_active":"2024-02-02T13:37:10.261862+0000","last_became_peered":"2024-02-02T13:37:10.261862+0000","last_unstale":"2024-02-02T13:37:10.750723+0000","last_undegraded":"2024-02-02T13:37:10.750723+0000","last_fullsized":"2024-02-02T13:37:10.750723+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:35.588116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743143+0000","last_change":"2024-02-02T13:37:10.742778+0000","last_active":"2024-02-02T13:37:10.743143+0000","last_peered":"2024-02-02T13:37:10.743143+0000","last_clean":"2024-02-02T13:37:10.743143+0000","last_became_active":"2024-02-02T13:37:10.742480+0000","last_became_peered":"2024-02-02T13:37:10.742480+0000","last_unstale":"2024-02-02T13:37:10.743143+0000","last_undegraded":"2024-02-02T13:37:10.743143+0000","last_fullsized":"2024-02-02T13:37:10.743143+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:16:47.842854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742863+0000","last_change":"2024-02-02T13:37:10.267857+0000","last_active":"2024-02-02T13:37:10.742863+0000","last_peered":"2024-02-02T13:37:10.742863+0000","last_clean":"2024-02-02T13:37:10.742863+0000","last_became_active":"2024-02-02T13:37:10.267461+0000","last_became_peered":"2024-02-02T13:37:10.267461+0000","last_unstale":"2024-02-02T13:37:10.742863+0000","last_undegraded":"2024-02-02T13:37:10.742863+0000","last_fullsized":"2024-02-02T13:37:10.742863+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:00:16.489712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737430+0000","last_change":"2024-02-02T13:37:10.737367+0000","last_active":"2024-02-02T13:37:10.737430+0000","last_peered":"2024-02-02T13:37:10.737430+0000","last_clean":"2024-02-02T13:37:10.737430+0000","last_became_active":"2024-02-02T13:37:10.736896+0000","last_became_peered":"2024-02-02T13:37:10.736896+0000","last_unstale":"2024-02-02T13:37:10.737430+0000","last_undegraded":"2024-02-02T13:37:10.737430+0000","last_fullsized":"2024-02-02T13:37:10.737430+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:28:35.262486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.727533+0000","last_change":"2024-02-02T13:37:10.262945+0000","last_active":"2024-02-02T13:37:10.727533+0000","last_peered":"2024-02-02T13:37:10.727533+0000","last_clean":"2024-02-02T13:37:10.727533+0000","last_became_active":"2024-02-02T13:37:10.262589+0000","last_became_peered":"2024-02-02T13:37:10.262589+0000","last_unstale":"2024-02-02T13:37:10.727533+0000","last_undegraded":"2024-02-02T13:37:10.727533+0000","last_fullsized":"2024-02-02T13:37:10.727533+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:03.129427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.742300+0000","last_change":"2024-02-02T13:37:10.742099+0000","last_active":"2024-02-02T13:37:10.742300+0000","last_peered":"2024-02-02T13:37:10.742300+0000","last_clean":"2024-02-02T13:37:10.742300+0000","last_became_active":"2024-02-02T13:37:10.741616+0000","last_became_peered":"2024-02-02T13:37:10.741616+0000","last_unstale":"2024-02-02T13:37:10.742300+0000","last_undegraded":"2024-02-02T13:37:10.742300+0000","last_fullsized":"2024-02-02T13:37:10.742300+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:13:50.544766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.743929+0000","last_change":"2024-02-02T13:37:10.261714+0000","last_active":"2024-02-02T13:37:10.743929+0000","last_peered":"2024-02-02T13:37:10.743929+0000","last_clean":"2024-02-02T13:37:10.743929+0000","last_became_active":"2024-02-02T13:37:10.260970+0000","last_became_peered":"2024-02-02T13:37:10.260970+0000","last_unstale":"2024-02-02T13:37:10.743929+0000","last_undegraded":"2024-02-02T13:37:10.743929+0000","last_fullsized":"2024-02-02T13:37:10.743929+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:28:45.970724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.749322+0000","last_change":"2024-02-02T13:37:10.749148+0000","last_active":"2024-02-02T13:37:10.749322+0000","last_peered":"2024-02-02T13:37:10.749322+0000","last_clean":"2024-02-02T13:37:10.749322+0000","last_became_active":"2024-02-02T13:37:10.748346+0000","last_became_peered":"2024-02-02T13:37:10.748346+0000","last_unstale":"2024-02-02T13:37:10.749322+0000","last_undegraded":"2024-02-02T13:37:10.749322+0000","last_fullsized":"2024-02-02T13:37:10.749322+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:03:34.307896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.732132+0000","last_change":"2024-02-02T13:37:10.262937+0000","last_active":"2024-02-02T13:37:10.732132+0000","last_peered":"2024-02-02T13:37:10.732132+0000","last_clean":"2024-02-02T13:37:10.732132+0000","last_became_active":"2024-02-02T13:37:10.261912+0000","last_became_peered":"2024-02-02T13:37:10.261912+0000","last_unstale":"2024-02-02T13:37:10.732132+0000","last_undegraded":"2024-02-02T13:37:10.732132+0000","last_fullsized":"2024-02-02T13:37:10.732132+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:19:10.553594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.737096+0000","last_change":"2024-02-02T13:37:10.736695+0000","last_active":"2024-02-02T13:37:10.737096+0000","last_peered":"2024-02-02T13:37:10.737096+0000","last_clean":"2024-02-02T13:37:10.737096+0000","last_became_active":"2024-02-02T13:37:10.736489+0000","last_became_peered":"2024-02-02T13:37:10.736489+0000","last_unstale":"2024-02-02T13:37:10.737096+0000","last_undegraded":"2024-02-02T13:37:10.737096+0000","last_fullsized":"2024-02-02T13:37:10.737096+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:56:58.205780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.739774+0000","last_change":"2024-02-02T13:37:10.265283+0000","last_active":"2024-02-02T13:37:10.739774+0000","last_peered":"2024-02-02T13:37:10.739774+0000","last_clean":"2024-02-02T13:37:10.739774+0000","last_became_active":"2024-02-02T13:37:10.265014+0000","last_became_peered":"2024-02-02T13:37:10.265014+0000","last_unstale":"2024-02-02T13:37:10.739774+0000","last_undegraded":"2024-02-02T13:37:10.739774+0000","last_fullsized":"2024-02-02T13:37:10.739774+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T13:45:44.682137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.763106+0000","last_change":"2024-02-02T13:37:10.759246+0000","last_active":"2024-02-02T13:37:10.763106+0000","last_peered":"2024-02-02T13:37:10.763106+0000","last_clean":"2024-02-02T13:37:10.763106+0000","last_became_active":"2024-02-02T13:37:10.754087+0000","last_became_peered":"2024-02-02T13:37:10.754087+0000","last_unstale":"2024-02-02T13:37:10.763106+0000","last_undegraded":"2024-02-02T13:37:10.763106+0000","last_fullsized":"2024-02-02T13:37:10.763106+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:48:35.274145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.736810+0000","last_change":"2024-02-02T13:37:10.271912+0000","last_active":"2024-02-02T13:37:10.736810+0000","last_peered":"2024-02-02T13:37:10.736810+0000","last_clean":"2024-02-02T13:37:10.736810+0000","last_became_active":"2024-02-02T13:37:10.271636+0000","last_became_peered":"2024-02-02T13:37:10.271636+0000","last_unstale":"2024-02-02T13:37:10.736810+0000","last_undegraded":"2024-02-02T13:37:10.736810+0000","last_fullsized":"2024-02-02T13:37:10.736810+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:47:41.192444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.735895+0000","last_change":"2024-02-02T13:37:10.735806+0000","last_active":"2024-02-02T13:37:10.735895+0000","last_peered":"2024-02-02T13:37:10.735895+0000","last_clean":"2024-02-02T13:37:10.735895+0000","last_became_active":"2024-02-02T13:37:10.734954+0000","last_became_peered":"2024-02-02T13:37:10.734954+0000","last_unstale":"2024-02-02T13:37:10.735895+0000","last_undegraded":"2024-02-02T13:37:10.735895+0000","last_fullsized":"2024-02-02T13:37:10.735895+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:10.230114+0000","last_clean_scrub_stamp":"2024-02-02T13:37:10.230114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:03.624254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.744175+0000","last_change":"2024-02-02T13:37:10.263276+0000","last_active":"2024-02-02T13:37:10.744175+0000","last_peered":"2024-02-02T13:37:10.744175+0000","last_clean":"2024-02-02T13:37:10.744175+0000","last_became_active":"2024-02-02T13:37:10.261375+0000","last_became_peered":"2024-02-02T13:37:10.261375+0000","last_unstale":"2024-02-02T13:37:10.744175+0000","last_undegraded":"2024-02-02T13:37:10.744175+0000","last_fullsized":"2024-02-02T13:37:10.744175+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:37:09.215369+0000","last_clean_scrub_stamp":"2024-02-02T13:37:09.215369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:21:39.188714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":336,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-02-02T13:37:10.254141+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:10.254141+0000","last_peered":"2024-02-02T13:37:10.254141+0000","last_clean":"2024-02-02T13:37:10.254141+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:10.254141+0000","last_undegraded":"2024-02-02T13:37:10.254141+0000","last_fullsized":"2024-02-02T13:37:10.254141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1644,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193667,"statfs":{"total":200000000,"available":198315832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.64600000000000002,"15min":0.61499999999999999},"max":{"1min":1.4570000000000001,"5min":2.266,"15min":2.5310000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.81699999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.524,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1240000000000001,"15min":1.1040000000000001},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.66300000000000003},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.893},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.56299999999999994},"max":{"1min":1.385,"5min":1.6399999999999999,"15min":2.5840000000000001},"last":0.628}]},{"osd":2,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2210000000000001,"15min":1.234},"min":{"1min":0.71799999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"max":{"1min":1.623,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2030000000000001},"min":{"1min":0.91900000000000004,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.5700000000000001,"5min":1.6639999999999999,"15min":3.5819999999999999},"last":0.502}]},{"osd":3,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2050000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.66200000000000003},"max":{"1min":1.5740000000000001,"5min":1.635,"15min":1.819},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.323,"5min":1.333,"15min":1.3280000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.48999999999999999},"max":{"1min":1.6910000000000001,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.258,"15min":1.242},"min":{"1min":0.79500000000000004,"5min":0.72899999999999998,"15min":0.57599999999999996},"max":{"1min":1.7110000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.278,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.753,"15min":0.61199999999999999},"max":{"1min":1.655,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.349,"15min":1.361},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.54000000000000004},"max":{"1min":1.758,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.90700000000000003,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":3.7949999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.506,"15min":1.5109999999999999},"min":{"1min":1.194,"5min":0.95899999999999996,"15min":0.64200000000000002},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":3.6840000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4139999999999999,"15min":1.4159999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.63700000000000001},"max":{"1min":1.8140000000000001,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":0.69699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1064,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194247,"statfs":{"total":200000000,"available":198909503,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.127,"15min":1.077},"min":{"1min":0.64000000000000001,"5min":0.63200000000000001,"15min":0.498},"max":{"1min":1.496,"5min":1.603,"15min":2.3780000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.077,"15min":1.056},"min":{"1min":0.81599999999999995,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.5589999999999999,"5min":1.776,"15min":1.776},"last":1.171}]},{"osd":1,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1140000000000001,"15min":1.0649999999999999},"min":{"1min":0.78900000000000003,"5min":0.68799999999999994,"15min":0.54000000000000004},"max":{"1min":1.3839999999999999,"5min":1.4790000000000001,"15min":1.556},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.175,"15min":1.135},"min":{"1min":0.80100000000000005,"5min":0.66000000000000003,"15min":0.55900000000000005},"max":{"1min":1.5720000000000001,"5min":1.677,"15min":2.9300000000000002},"last":1.401}]},{"osd":2,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.226,"15min":1.1539999999999999},"min":{"1min":1.012,"5min":0.77300000000000002,"15min":0.63700000000000001},"max":{"1min":1.627,"5min":1.75,"15min":2.6309999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2090000000000001,"15min":1.181},"min":{"1min":0.93400000000000005,"5min":0.80400000000000005,"15min":0.63},"max":{"1min":1.603,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.76000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2609999999999999,"15min":1.256},"min":{"1min":1.0149999999999999,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.577,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.083},{"interface":"front","average":{"1min":1.266,"5min":1.2430000000000001,"15min":1.1890000000000001},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.61299999999999999},"max":{"1min":1.665,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.232}]},{"osd":4,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.345,"15min":1.3320000000000001},"min":{"1min":0.80500000000000005,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.6919999999999999,"5min":1.8420000000000001,"15min":17.686},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.419,"15min":1.466},"min":{"1min":0.96899999999999997,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.899,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2869999999999999,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.56399999999999995,"15min":0.45900000000000002},"max":{"1min":1.782,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4179999999999999,"15min":1.3979999999999999},"min":{"1min":0.95299999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.95,"15min":3.331},"last":1.512}]},{"osd":7,"last update":"Fri Feb 2 13:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.454,"15min":1.462},"min":{"1min":1.0629999999999999,"5min":0.64700000000000002,"15min":0.61199999999999999},"max":{"1min":2.0179999999999998,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.446,"15min":1.444},"min":{"1min":1.044,"5min":0.80200000000000005,"15min":0.65300000000000002},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.647}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1059,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194253,"statfs":{"total":200000000,"available":198915281,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.085,"15min":1.0449999999999999},"min":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.61099999999999999},"max":{"1min":1.4990000000000001,"5min":1.8,"15min":1.8},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.135,"5min":1.0760000000000001,"15min":1.0609999999999999},"min":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.49199999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1870000000000001,"15min":1.2010000000000001},"min":{"1min":0.83899999999999997,"5min":0.67500000000000004,"15min":0.61899999999999999},"max":{"1min":1.498,"5min":1.841,"15min":4.5899999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.1100000000000001},"min":{"1min":0.84599999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.5249999999999999,"5min":1.6379999999999999,"15min":4.4500000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3069999999999999,"15min":1.266},"min":{"1min":1.0309999999999999,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6160000000000001,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.95399999999999996,"5min":0.76000000000000001,"15min":0.65300000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.0860000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.26,"15min":1.214},"min":{"1min":1.032,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.478,"5min":1.629,"15min":1.7769999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2689999999999999,"15min":1.232},"min":{"1min":1.073,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.5960000000000001,"5min":1.8009999999999999,"15min":5.4610000000000003},"last":1.5820000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.78400000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6140000000000001,"5min":1.675,"15min":1.8460000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3260000000000001,"15min":1.298},"min":{"1min":1.0169999999999999,"5min":0.61799999999999999,"15min":0.55100000000000005},"max":{"1min":1.5549999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.335}]},{"osd":6,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.341,"15min":1.325},"min":{"1min":1.016,"5min":0.745,"15min":0.64700000000000002},"max":{"1min":1.6659999999999999,"5min":1.865,"15min":2.0009999999999999},"last":1.653},{"interface":"front","average":{"1min":1.375,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.0660000000000001,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.8109999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4350000000000001,"15min":1.417},"min":{"1min":0.92200000000000004,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.613,"5min":1.893,"15min":1.9670000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.478,"15min":1.429},"min":{"1min":1.1180000000000001,"5min":0.88300000000000001,"15min":0.61599999999999999},"max":{"1min":1.722,"5min":2.024,"15min":2.024},"last":1.706}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1058,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194254,"statfs":{"total":200000000,"available":198916300,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.57999999999999996},"max":{"1min":1.431,"5min":1.5309999999999999,"15min":4.5949999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.83499999999999996,"5min":0.70399999999999996,"15min":0.59199999999999997},"max":{"1min":1.4470000000000001,"5min":1.7609999999999999,"15min":1.839},"last":1.153}]},{"osd":1,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.0800000000000001,"15min":1.0469999999999999},"min":{"1min":0.78400000000000003,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.5229999999999999,"5min":1.54,"15min":2.4809999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1299999999999999,"15min":1.0920000000000001},"min":{"1min":0.78300000000000003,"5min":0.73799999999999999,"15min":0.55800000000000005},"max":{"1min":1.5,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":1.0089999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.1579999999999999,"15min":1.133},"min":{"1min":0.97799999999999998,"5min":0.68700000000000006,"15min":0.68500000000000005},"max":{"1min":1.556,"5min":1.8580000000000001,"15min":3.1200000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.1459999999999999},"min":{"1min":0.93200000000000005,"5min":0.67300000000000004,"15min":0.58799999999999997},"max":{"1min":1.508,"5min":1.736,"15min":2.6000000000000001},"last":0.94499999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.2709999999999999,"15min":1.2350000000000001},"min":{"1min":0.97599999999999998,"5min":0.82099999999999995,"15min":0.68999999999999995},"max":{"1min":1.5569999999999999,"5min":1.661,"15min":2.9079999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3160000000000001,"15min":1.2889999999999999},"min":{"1min":0.92400000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.331}]},{"osd":5,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.1990000000000001,"15min":1.1950000000000001},"min":{"1min":0.77100000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3640000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.89700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5109999999999999,"5min":1.944,"15min":2.4529999999999998},"last":1.375}]},{"osd":6,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4059999999999999,"15min":1.3620000000000001},"min":{"1min":1.075,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7250000000000001,"5min":2.403,"15min":2.7989999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4390000000000001,"15min":1.4279999999999999},"min":{"1min":0.877,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.5640000000000001,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:47 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.466,"15min":1.4379999999999999},"min":{"1min":1.1659999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5740000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.48},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4790000000000001,"15min":1.468},"min":{"1min":1.131,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.903,"5min":2.0680000000000001,"15min":2.698},"last":1.534}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1059,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194253,"statfs":{"total":200000000,"available":198915281,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79600000000000004,"15min":0.755},"min":{"1min":0.63600000000000001,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.4199999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.71199999999999997,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.323,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.71999999999999997,"5min":0.58899999999999997,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5609999999999999,"15min":2.403},"last":1.105},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1419999999999999,"15min":1.0700000000000001},"min":{"1min":0.90700000000000003,"5min":0.60299999999999998,"15min":0.47999999999999998},"max":{"1min":1.4099999999999999,"5min":1.5589999999999999,"15min":2.2709999999999999},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1020000000000001,"15min":1.026},"min":{"1min":0.83199999999999996,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":1.659,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.91700000000000004},"min":{"1min":0.72999999999999998,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.45,"15min":3.4510000000000001},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.1910000000000001},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.53000000000000003},"max":{"1min":1.4830000000000001,"5min":1.5940000000000001,"15min":3.585},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.248},"min":{"1min":1.0189999999999999,"5min":0.72499999999999998,"15min":0.61299999999999999},"max":{"1min":1.5800000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2190000000000001,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.73799999999999999,"15min":0.57099999999999995},"max":{"1min":1.546,"5min":1.609,"15min":3.738},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.26,"15min":1.26},"min":{"1min":0.97499999999999998,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.6719999999999999,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":1.302}]},{"osd":6,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3999999999999999,"15min":1.3320000000000001},"min":{"1min":1.159,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.5589999999999999,"5min":1.823,"15min":1.8979999999999999},"last":1.45},{"interface":"front","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.95399999999999996,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.6200000000000001,"5min":1.667,"15min":3.6059999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3129999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.577,"5min":1.851,"15min":3.391},"last":1.417},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.397,"15min":1.3560000000000001},"min":{"1min":1.1180000000000001,"5min":0.83699999999999997,"15min":0.625},"max":{"1min":1.7669999999999999,"5min":1.8360000000000001,"15min":2.0409999999999999},"last":1.5129999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908416,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.81999999999999995},"min":{"1min":0.64600000000000002,"5min":0.435,"15min":0.27900000000000003},"max":{"1min":1.2529999999999999,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0189999999999999,"15min":0.89400000000000002},"min":{"1min":0.76700000000000002,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.4750000000000001,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.93600000000000005,"15min":0.92800000000000005},"min":{"1min":0.81699999999999995,"5min":0.60999999999999999,"15min":0.40999999999999998},"max":{"1min":1.3460000000000001,"5min":1.387,"15min":2.1680000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.0760000000000001,"15min":0.96899999999999997},"min":{"1min":0.84799999999999998,"5min":0.58099999999999996,"15min":0.35899999999999999},"max":{"1min":1.4470000000000001,"5min":1.6279999999999999,"15min":1.6499999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.97399999999999998,"15min":0.89000000000000001},"min":{"1min":0.74399999999999999,"5min":0.46000000000000002,"15min":0.40200000000000002},"max":{"1min":1.4259999999999999,"5min":1.619,"15min":2.1339999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0920000000000001,"15min":1.0409999999999999},"min":{"1min":0.89500000000000002,"5min":0.56399999999999995,"15min":0.432},"max":{"1min":1.653,"5min":1.653,"15min":2.6589999999999998},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.2929999999999999,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.77000000000000002,"15min":0.56799999999999995},"max":{"1min":1.877,"5min":1.877,"15min":2.8759999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2869999999999999,"15min":1.224},"min":{"1min":1.1000000000000001,"5min":0.86699999999999999,"15min":0.503},"max":{"1min":1.5449999999999999,"5min":2.02,"15min":2.1360000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.244,"15min":1.1759999999999999},"min":{"1min":0.94799999999999995,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":2.024,"5min":2.028,"15min":2.8239999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3360000000000001,"15min":1.3049999999999999},"min":{"1min":1.1259999999999999,"5min":0.85599999999999998,"15min":0.627},"max":{"1min":1.7949999999999999,"5min":2.21,"15min":3.0070000000000001},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.3540000000000001,"15min":1.27},"min":{"1min":1.0800000000000001,"5min":0.83899999999999997,"15min":0.53800000000000003},"max":{"1min":1.647,"5min":1.919,"15min":2.8620000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3360000000000001,"15min":1.266},"min":{"1min":1.127,"5min":0.80700000000000005,"15min":0.70499999999999996},"max":{"1min":1.7949999999999999,"5min":2.6920000000000002,"15min":2.9750000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4159999999999999,"15min":1.319},"min":{"1min":1.1910000000000001,"5min":0.96799999999999997,"15min":0.65100000000000002},"max":{"1min":1.663,"5min":1.976,"15min":2.7749999999999999},"last":1.335},{"interface":"front","average":{"1min":1.397,"5min":1.351,"15min":1.2949999999999999},"min":{"1min":1.0760000000000001,"5min":0.85899999999999999,"15min":0.52500000000000002},"max":{"1min":1.9330000000000001,"5min":2.0179999999999998,"15min":3.0830000000000002},"last":1.3009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1064,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194247,"statfs":{"total":200000000,"available":198909491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.81999999999999995,"15min":0.88400000000000001},"min":{"1min":0.65200000000000002,"5min":0.433,"15min":0.34200000000000003},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.96399999999999997,"15min":1.095},"min":{"1min":0.70199999999999996,"5min":0.308,"15min":0.308},"max":{"1min":1.395,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":1.145}]},{"osd":2,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.76400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.639},"last":1.091},{"interface":"front","average":{"1min":1.044,"5min":0.99399999999999999,"15min":1.03},"min":{"1min":0.76600000000000001,"5min":0.48199999999999998,"15min":0.39900000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":3.0259999999999998},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1279999999999999,"15min":1.1579999999999999},"min":{"1min":0.84599999999999997,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":1.704,"15min":3.1539999999999999},"last":1.327},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.052,"15min":1.1220000000000001},"min":{"1min":0.749,"5min":0.505,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.778,"15min":4.8760000000000003},"last":1.659}]},{"osd":4,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1359999999999999,"15min":1.2010000000000001},"min":{"1min":0.92800000000000005,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.3340000000000001,"5min":1.6220000000000001,"15min":2.9449999999999998},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2689999999999999,"15min":1.3029999999999999},"min":{"1min":1.03,"5min":0.79900000000000004,"15min":0.68899999999999995},"max":{"1min":1.585,"5min":1.9490000000000001,"15min":3.2970000000000002},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.319,"15min":1.359},"min":{"1min":1.0449999999999999,"5min":0.82199999999999995,"15min":0.753},"max":{"1min":1.577,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":1.1639999999999999,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.536,"5min":1.6970000000000001,"15min":3.0249999999999999},"last":1.9570000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3779999999999999,"15min":1.4239999999999999},"min":{"1min":1.095,"5min":0.85699999999999998,"15min":0.73599999999999999},"max":{"1min":1.581,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.879},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3149999999999999,"15min":1.379},"min":{"1min":1.05,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5820000000000001,"5min":1.77,"15min":2.9910000000000001},"last":2.181}]},{"osd":7,"last update":"Fri Feb 2 13:36:44 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3919999999999999,"15min":1.4430000000000001},"min":{"1min":1.155,"5min":0.89200000000000002,"15min":0.77100000000000002},"max":{"1min":1.8500000000000001,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4390000000000001,"15min":1.484},"min":{"1min":1.147,"5min":0.79800000000000004,"15min":0.78000000000000003},"max":{"1min":1.9750000000000001,"5min":2.0790000000000002,"15min":4.3040000000000003},"last":2.0169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193671,"statfs":{"total":200000000,"available":198319157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86299999999999999,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":1.2909999999999999,"5min":1.5840000000000001,"15min":4.8070000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.91400000000000003,"15min":1.1180000000000001},"min":{"1min":0.59499999999999997,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.4259999999999999,"5min":1.6459999999999999,"15min":10.816000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.96099999999999997,"15min":1.0249999999999999},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.4850000000000001,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.004,"5min":0.995,"15min":1.153},"min":{"1min":0.64500000000000002,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":1.355,"5min":1.7,"15min":10.781000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.96699999999999997,"15min":1.0409999999999999},"min":{"1min":0.52100000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.607,"5min":1.6339999999999999,"15min":5.8410000000000002},"last":0.503},{"interface":"front","average":{"1min":1.093,"5min":1.0609999999999999,"15min":1.1439999999999999},"min":{"1min":0.58099999999999996,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.446,"5min":1.4610000000000001,"15min":4.9530000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1970000000000001,"15min":1.242},"min":{"1min":0.83999999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5189999999999999,"5min":2.129,"15min":3.669},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.284,"5min":1.256,"15min":1.3120000000000001},"min":{"1min":1.077,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5129999999999999,"5min":1.893,"15min":4.8780000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.272,"15min":1.472},"min":{"1min":0.81499999999999995,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":1.5740000000000001,"5min":1.9970000000000001,"15min":10.651},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.274,"5min":1.2490000000000001,"15min":1.3109999999999999},"min":{"1min":0.89700000000000002,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":1.516,"5min":1.954,"15min":3.7869999999999999},"last":1.0169999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.339,"15min":1.3779999999999999},"min":{"1min":0.89100000000000001,"5min":0.72699999999999998,"15min":0.66500000000000004},"max":{"1min":1.577,"5min":1.871,"15min":4.9210000000000003},"last":1.095},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.5049999999999999},"min":{"1min":0.873,"5min":0.747,"15min":0.747},"max":{"1min":1.651,"5min":1.95,"15min":10.708},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.351,"15min":1.5429999999999999},"min":{"1min":1.0249999999999999,"5min":0.77700000000000002,"15min":0.70999999999999996},"max":{"1min":1.792,"5min":2.085,"15min":10.74},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.3149999999999999,"15min":1.357},"min":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6639999999999999,"5min":1.8700000000000001,"15min":3.863},"last":1.0800000000000001}]}]}],"pool_statfs":[]}} 2024-02-02T13:37:15.615 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T13:37:15.615 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-02T13:37:15.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.863+0000 7f3354000640 1 Processor -- start 2024-02-02T13:37:15.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.864+0000 7f3354000640 1 -- start start 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3354000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f334c0ce130 0x7f334c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3354000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0cc7c0 con 0x7f334c0b0700 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3354000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0ce930 con 0x7f334c15a010 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3354000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0cd640 con 0x7f334c0ce130 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3352000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f334c0b0700 0x7f334c06ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51542/0 (socket says 172.21.15.91:51542) 2024-02-02T13:37:15.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.865+0000 7f3352000640 1 -- 172.21.15.91:0/415491613 learned_addr learned my addr 172.21.15.91:0/415491613 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175577216 0 0) 0x7f334c0cc7c0 con 0x7f334c0b0700 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c005730 con 0x7f334c0b0700 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f3350c00640 1 --2- 172.21.15.91:0/415491613 >> v2:172.21.15.113:3300/0 conn(0x7f334c0ce130 0x7f334c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36704271 0 0) 0x7f332c005730 con 0x7f334c0b0700 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 >> v2:172.21.15.113:3300/0 conn(0x7f334c0ce130 msgr2=0x7f334c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 --2- 172.21.15.91:0/415491613 >> v2:172.21.15.113:3300/0 conn(0x7f334c0ce130 0x7f334c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 >> v1:172.21.15.113:6789/0 conn(0x7f334c15a010 legacy=0x7f334c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c06c890 con 0x7f334c0b0700 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f33440019f0 con 0x7f334c0b0700 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 >> v1:172.21.15.91:6789/0 conn(0x7f334c0b0700 legacy=0x7f334c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f332c005bd0 0x7f332c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 0x7f332c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0073e0 0x7f332c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0cd640 con 0x7f332c0069d0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00e330 con 0x7f332c0073e0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.866+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00e4f0 con 0x7f332c005bd0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 0x7f332c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3351600640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f332c005bd0 0x7f332c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 -- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f332c005bd0 msgr2=0x7f332c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3352000640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0073e0 0x7f332c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f332c005bd0 0x7f332c005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 -- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0073e0 msgr2=0x7f332c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0073e0 0x7f332c00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3352000640 1 -- 172.21.15.91:0/415491613 reap_dead start 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.867+0000 7f3350c00640 1 -- 172.21.15.91:0/415491613 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c005730 con 0x7f332c0069d0 2024-02-02T13:37:15.871 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.868+0000 7f3350c00640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 0x7f332c006e50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f333400ba30 tx=0x7f332c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cb16af43542c84 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.868+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334032640 con 0x7f332c0069d0 2024-02-02T13:37:15.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.868+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33340327a0 con 0x7f332c0069d0 2024-02-02T13:37:15.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.868+0000 7f334be00640 1 -- 172.21.15.91:0/415491613 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334032b30 con 0x7f332c0069d0 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 -- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 msgr2=0x7f332c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 0x7f332c006e50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f333400ba30 tx=0x7f332c007a00 comp rx=0 tx=0).stop 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 -- 172.21.15.91:0/415491613 shutdown_connections 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 --2- 172.21.15.91:0/415491613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f332c0069d0 0x7f332c006e50 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 -- 172.21.15.91:0/415491613 >> 172.21.15.91:0/415491613 conn(0x7f334c0cec80 msgr2=0x7f334c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 -- 172.21.15.91:0/415491613 shutdown_connections 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.869+0000 7f3354000640 1 -- 172.21.15.91:0/415491613 wait complete. 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.871+0000 7f3354000640 1 Processor -- start 2024-02-02T13:37:15.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.871+0000 7f3354000640 1 -- start start 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0069d0 0x7f334c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 0x7f334c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c3e00 con 0x7f334c0c3440 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c70f0 con 0x7f332c0069d0 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3354000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c73c0 con 0x7f334c0c5cc0 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3352000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3352000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41602/0 (socket says 172.21.15.91:41602) 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.872+0000 7f3352000640 1 -- 172.21.15.91:0/3289353423 learned_addr learned my addr 172.21.15.91:0/3289353423 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3350c00640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 0x7f334c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3351600640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0069d0 0x7f334c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 -- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 msgr2=0x7f334c0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 0x7f334c0c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 -- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0069d0 msgr2=0x7f334c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0069d0 0x7f334c156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 -- 172.21.15.91:0/3289353423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0c76a0 con 0x7f334c0c3440 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3350c00640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 0x7f334c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3352000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f334c0c2320 tx=0x7f334400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400b7a0 con 0x7f334c0c3440 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334400b920 con 0x7f334c0c3440 2024-02-02T13:37:15.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400bcd0 con 0x7f334c0c3440 2024-02-02T13:37:15.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c0c7980 con 0x7f334c0c3440 2024-02-02T13:37:15.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.873+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c15a3f0 con 0x7f334c0c3440 2024-02-02T13:37:15.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.875+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c15a840 con 0x7f334c0c3440 2024-02-02T13:37:15.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.875+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3344007070 con 0x7f334c0c3440 2024-02-02T13:37:15.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.876+0000 7f3349600640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 0x7f332406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:15.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.876+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3344088d10 con 0x7f334c0c3440 2024-02-02T13:37:15.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.877+0000 7f3351600640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 0x7f332406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:15.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.877+0000 7f3351600640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 0x7f332406c8f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f333c001590 tx=0x7f333c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:15.880+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3344054140 con 0x7f334c0c3440 2024-02-02T13:37:16.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.116+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f334c066cf0 con 0x7f334c0c3440 2024-02-02T13:37:16.126 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.125+0000 7f3349600640 1 -- 172.21.15.91:0/3289353423 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3344056070 con 0x7f334c0c3440 2024-02-02T13:37:16.126 INFO:teuthology.orchestra.run.smithi091.stderr:updated caps for client.0 2024-02-02T13:37:16.128 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 msgr2=0x7f332406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 0x7f332406c8f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f333c001590 tx=0x7f333c009040 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 msgr2=0x7f334c0c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f334c0c2320 tx=0x7f334400df60 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 shutdown_connections 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f332406a430 0x7f332406c8f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334c0c5cc0 0x7f334c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f334c0c3440 0x7f334c0c38c0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 --2- 172.21.15.91:0/3289353423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0069d0 0x7f334c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 >> 172.21.15.91:0/3289353423 conn(0x7f334c0cec80 msgr2=0x7f334c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 shutdown_connections 2024-02-02T13:37:16.129 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.127+0000 7f3354000640 1 -- 172.21.15.91:0/3289353423 wait complete. 2024-02-02T13:37:16.146 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:16.221 INFO:tasks.ceph.mds.a.smithi091.stderr:2024-02-02T13:37:16.220+0000 7f3321600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:37:16.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.340+0000 7faea7800640 1 Processor -- start 2024-02-02T13:37:16.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.340+0000 7faea7800640 1 -- start start 2024-02-02T13:37:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea7800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faea0151f90 0x7faea015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 0x7faea00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea7800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea0070f80 con 0x7faea00b0c40 2024-02-02T13:37:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea7800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00b10d0 con 0x7faea0151f90 2024-02-02T13:37:16.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea0152370 con 0x7faea006f4f0 2024-02-02T13:37:16.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea5800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faea00b0c40 0x7faea00b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51552/0 (socket says 172.21.15.91:51552) 2024-02-02T13:37:16.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea5800640 1 -- 172.21.15.91:0/661809473 learned_addr learned my addr 172.21.15.91:0/661809473 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:16.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7fae9fe00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 0x7faea00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.342+0000 7faea4e00640 1 --2- 172.21.15.91:0/661809473 >> v2:172.21.15.113:3300/0 conn(0x7faea0151f90 0x7faea015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515587788 0 0) 0x7faea0070f80 con 0x7faea00b0c40 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80003520 con 0x7faea00b0c40 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1074312868 0 0) 0x7fae80003520 con 0x7faea00b0c40 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 msgr2=0x7faea00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 0x7faea00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 >> v2:172.21.15.113:3300/0 conn(0x7faea0151f90 msgr2=0x7faea015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 --2- 172.21.15.91:0/661809473 >> v2:172.21.15.113:3300/0 conn(0x7faea0151f90 0x7faea015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00cb010 con 0x7faea00b0c40 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fae98001670 con 0x7faea00b0c40 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 >> v1:172.21.15.91:6789/0 conn(0x7faea00b0c40 legacy=0x7faea00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae80003970 0x7fae80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7fae80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 0x7fae80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b10d0 con 0x7fae800047c0 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0152370 con 0x7fae80003970 2024-02-02T13:37:16.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.343+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8000b980 con 0x7faea006f4f0 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7fae80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7faea4e00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae80003970 0x7fae80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 -- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 msgr2=0x7fae80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea006f4f0 0x7fae80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 -- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae80003970 msgr2=0x7fae80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae80003970 0x7fae80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.344+0000 7fae9fe00640 1 -- 172.21.15.91:0/661809473 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80003520 con 0x7fae800047c0 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.345+0000 7fae9fe00640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7fae80004c40 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fae88009ac0 tx=0x7fae800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf76ecbb339cfe7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.345+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8802c710 con 0x7fae800047c0 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.345+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8802c870 con 0x7fae800047c0 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.345+0000 7fae9f400640 1 -- 172.21.15.91:0/661809473 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8802cc00 con 0x7fae800047c0 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 -- 172.21.15.91:0/661809473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 msgr2=0x7fae80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 --2- 172.21.15.91:0/661809473 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7fae80004c40 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fae88009ac0 tx=0x7fae800057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea5800640 1 -- 172.21.15.91:0/661809473 reap_dead start 2024-02-02T13:37:16.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 -- 172.21.15.91:0/661809473 shutdown_connections 2024-02-02T13:37:16.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 -- 172.21.15.91:0/661809473 >> 172.21.15.91:0/661809473 conn(0x7faea00cec80 msgr2=0x7faea00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:16.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 -- 172.21.15.91:0/661809473 shutdown_connections 2024-02-02T13:37:16.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.346+0000 7faea7800640 1 -- 172.21.15.91:0/661809473 wait complete. 2024-02-02T13:37:16.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.348+0000 7faea7800640 1 Processor -- start 2024-02-02T13:37:16.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.348+0000 7faea7800640 1 -- start start 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faea01588e0 0x7faea015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea00b2870 0x7faea00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b18a0 con 0x7fae800047c0 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00c8ae0 con 0x7faea01588e0 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea7800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea0155b20 con 0x7faea00b2870 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea4e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41626/0 (socket says 172.21.15.91:41626) 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea4e00640 1 -- 172.21.15.91:0/1072486914 learned_addr learned my addr 172.21.15.91:0/1072486914 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.349+0000 7faea4e00640 1 -- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea00b2870 msgr2=0x7faea00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea4e00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea00b2870 0x7faea00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea4e00640 1 -- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faea01588e0 msgr2=0x7faea015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7fae9fe00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faea01588e0 0x7faea015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea4e00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faea01588e0 0x7faea015f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea4e00640 1 -- 172.21.15.91:0/1072486914 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00b1ab0 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea4e00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fae90007f30 tx=0x7fae90011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000e030 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90016470 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90016820 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea00b1d90 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.350+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea00b2350 con 0x7fae800047c0 2024-02-02T13:37:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.351+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0066cf0 con 0x7fae800047c0 2024-02-02T13:37:16.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.353+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fae90016980 con 0x7fae800047c0 2024-02-02T13:37:16.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.354+0000 7fae9cc00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 0x7fae7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:16.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.354+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae9000f070 con 0x7fae800047c0 2024-02-02T13:37:16.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.354+0000 7fae9fe00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 0x7fae7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:16.356 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.354+0000 7fae9fe00640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 0x7fae7806c8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:16.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.358+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fae90054f60 con 0x7fae800047c0 2024-02-02T13:37:16.583 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.581+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea006c7b0 con 0x7fae800047c0 2024-02-02T13:37:16.584 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.583+0000 7fae9cc00640 1 -- 172.21.15.91:0/1072486914 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fae90058580 con 0x7fae800047c0 2024-02-02T13:37:16.584 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:16.584 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:16.585 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 msgr2=0x7fae7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 0x7fae7806c8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88030000 comp rx=0 tx=0).stop 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 msgr2=0x7faea015ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fae90007f30 tx=0x7fae90011660 comp rx=0 tx=0).stop 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 shutdown_connections 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fae7806a430 0x7fae7806c8f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.587 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faea00b2870 0x7faea00b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faea01588e0 0x7faea015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 --2- 172.21.15.91:0/1072486914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae800047c0 0x7faea015ecb0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:16.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 >> 172.21.15.91:0/1072486914 conn(0x7faea00cec80 msgr2=0x7faea0159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:16.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 shutdown_connections 2024-02-02T13:37:16.588 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:16.585+0000 7faea7800640 1 -- 172.21.15.91:0/1072486914 wait complete. 2024-02-02T13:37:17.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T13:37:10.719224+0000', 'modified': '2024-02-02T13:37:13.738863+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8835}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8835': {'gid': 8835, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6835/115193290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6834', 'nonce': 115193290}, {'type': 'v1', 'addr': '172.21.15.91:6835', 'nonce': 115193290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T13:37:17.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T13:37:17.007 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-02T13:37:17.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.226+0000 7fd814200640 1 Processor -- start 2024-02-02T13:37:17.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.227+0000 7fd814200640 1 -- start start 2024-02-02T13:37:17.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd814200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 0x7fd80c0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd814200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd80c0cde90 0x7fd80c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd814200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c0d0480 con 0x7fd80c15d480 2024-02-02T13:37:17.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd814200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c0cfec0 con 0x7fd80c0cde90 2024-02-02T13:37:17.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd814200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c0d01a0 con 0x7fd80c06ae70 2024-02-02T13:37:17.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd811800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd80c15d480 0x7fd80c0cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51568/0 (socket says 172.21.15.91:51568) 2024-02-02T13:37:17.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd811800640 1 -- 172.21.15.91:0/1935755536 learned_addr learned my addr 172.21.15.91:0/1935755536 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.228+0000 7fd812200640 1 --2- 172.21.15.91:0/1935755536 >> v2:172.21.15.91:3300/0 conn(0x7fd80c0cde90 0x7fd80c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd80be00640 1 -- 172.21.15.91:0/1935755536 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737548168 0 0) 0x7fd80c0d0480 con 0x7fd80c15d480 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd80be00640 1 -- 172.21.15.91:0/1935755536 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ec003520 con 0x7fd80c15d480 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd810e00640 1 --2- 172.21.15.91:0/1935755536 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 0x7fd80c0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd812200640 1 -- 172.21.15.91:0/1935755536 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 msgr2=0x7fd80c0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd812200640 1 --2- 172.21.15.91:0/1935755536 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 0x7fd80c0cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd812200640 1 -- 172.21.15.91:0/1935755536 >> v1:172.21.15.91:6789/0 conn(0x7fd80c15d480 legacy=0x7fd80c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd812200640 1 -- 172.21.15.91:0/1935755536 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c0cb580 con 0x7fd80c0cde90 2024-02-02T13:37:17.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.229+0000 7fd810e00640 1 --2- 172.21.15.91:0/1935755536 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 0x7fd80c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:17.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.230+0000 7fd812200640 1 --2- 172.21.15.91:0/1935755536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c0cde90 0x7fd80c0cc2c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd80c070e80 tx=0x7fd80402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb8bbc6297ea51c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.230+0000 7fd80be00640 1 -- 172.21.15.91:0/1935755536 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80402cc70 con 0x7fd80c0cde90 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.230+0000 7fd80be00640 1 -- 172.21.15.91:0/1935755536 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80402cdd0 con 0x7fd80c0cde90 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.230+0000 7fd80be00640 1 -- 172.21.15.91:0/1935755536 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804034660 con 0x7fd80c0cde90 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 -- 172.21.15.91:0/1935755536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c0cde90 msgr2=0x7fd80c0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 --2- 172.21.15.91:0/1935755536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c0cde90 0x7fd80c0cc2c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd80c070e80 tx=0x7fd80402bec0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 -- 172.21.15.91:0/1935755536 shutdown_connections 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 --2- 172.21.15.91:0/1935755536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c0cde90 0x7fd80c0cc2c0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 --2- 172.21.15.91:0/1935755536 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06ae70 0x7fd80c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 -- 172.21.15.91:0/1935755536 >> 172.21.15.91:0/1935755536 conn(0x7fd80c09a160 msgr2=0x7fd80c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 -- 172.21.15.91:0/1935755536 shutdown_connections 2024-02-02T13:37:17.233 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.231+0000 7fd814200640 1 -- 172.21.15.91:0/1935755536 wait complete. 2024-02-02T13:37:17.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.233+0000 7fd814200640 1 Processor -- start 2024-02-02T13:37:17.234 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.233+0000 7fd814200640 1 -- start start 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c0cde90 0x7fd80c1e4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 0x7fd80c0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0b3c20 con 0x7fd80c06ae70 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c1e5a00 con 0x7fd80c15d480 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd814200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c15bdf0 con 0x7fd80c0cde90 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41646/0 (socket says 172.21.15.91:41646) 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.234+0000 7fd811800640 1 -- 172.21.15.91:0/490752257 learned_addr learned my addr 172.21.15.91:0/490752257 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd812200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 0x7fd80c0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd810e00640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c0cde90 0x7fd80c1e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 -- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c0cde90 msgr2=0x7fd80c1e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.237 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c0cde90 0x7fd80c1e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 -- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 msgr2=0x7fd80c0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 0x7fd80c0c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 -- 172.21.15.91:0/490752257 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804009a70 con 0x7fd80c06ae70 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd812200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 0x7fd80c0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd811800640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c410 tx=0x7fd7fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd809600640 1 -- 172.21.15.91:0/490752257 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc005ce0 con 0x7fd80c06ae70 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd809600640 1 -- 172.21.15.91:0/490752257 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc005e40 con 0x7fd80c06ae70 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c1e5160 con 0x7fd80c06ae70 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd809600640 1 -- 172.21.15.91:0/490752257 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0072f0 con 0x7fd80c06ae70 2024-02-02T13:37:17.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.235+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c1e5720 con 0x7fd80c06ae70 2024-02-02T13:37:17.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.238+0000 7fd809600640 1 -- 172.21.15.91:0/490752257 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7fc007510 con 0x7fd80c06ae70 2024-02-02T13:37:17.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.239+0000 7fd809600640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 0x7fd7e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.239+0000 7fd817026740 1 Processor -- start 2024-02-02T13:37:17.240 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.239+0000 7fd809600640 1 -- 172.21.15.91:0/490752257 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7fc088020 con 0x7fd80c06ae70 2024-02-02T13:37:17.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.239+0000 7fd817026740 1 -- start start 2024-02-02T13:37:17.241 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.239+0000 7fd810e00640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 0x7fd7e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd810e00640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 0x7fd7e406c8f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0cab60 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3eded0 0x55a8ca41b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca41b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca3c9650 con 0x55a8ca41df50 2024-02-02T13:37:17.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca35e000 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca35b790 con 0x55a8ca3eded0 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41654/0 (socket says 172.21.15.91:41654) 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.240+0000 7fd811800640 1 -- 172.21.15.91:0/1842561640 learned_addr learned my addr 172.21.15.91:0/1842561640 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 -- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3eded0 msgr2=0x55a8ca41b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3eded0 0x55a8ca41b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 -- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 msgr2=0x55a8ca41b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca41b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 -- 172.21.15.91:0/1842561640 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ca31e710 con 0x55a8ca41df50 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd811800640 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0078e0 tx=0x7fd7fc056d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fca4ab874b6c790 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd802000640 1 -- 172.21.15.91:0/1842561640 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc05f2d0 con 0x55a8ca41df50 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd802000640 1 -- 172.21.15.91:0/1842561640 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc055690 con 0x55a8ca41df50 2024-02-02T13:37:17.243 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.241+0000 7fd802000640 1 -- 172.21.15.91:0/1842561640 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc05d070 con 0x55a8ca41df50 2024-02-02T13:37:17.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- 172.21.15.91:0/1842561640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 msgr2=0x55a8ca41e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.244 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0078e0 tx=0x7fd7fc056d80 comp rx=0 tx=0).stop 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- 172.21.15.91:0/1842561640 shutdown_connections 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3eded0 0x55a8ca41b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca41b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 --2- 172.21.15.91:0/1842561640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca41df50 0x55a8ca41e330 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- 172.21.15.91:0/1842561640 >> 172.21.15.91:0/1842561640 conn(0x55a8ca3f01e0 msgr2=0x55a8ca438d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- 172.21.15.91:0/1842561640 shutdown_connections 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- 172.21.15.91:0/1842561640 wait complete. 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.242+0000 7fd817026740 1 -- start start 2024-02-02T13:37:17.245 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.243+0000 7fd817026740 1 Processor -- start 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 0x55a8ca4362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca41df50 0x55a8ca436980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca357080 con 0x55a8ca421020 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca2f8850 con 0x55a8ca41df50 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca2f84f0 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd817026740 10 client.0 _ll_register_callbacks cb 0x55a8ca3509c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41666/0 (socket says 172.21.15.91:41666) 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 -- 172.21.15.91:0/2459294369 learned_addr learned my addr 172.21.15.91:0/2459294369 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd810e00640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca41df50 0x55a8ca436980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd811800640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 0x55a8ca4362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 -- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 msgr2=0x55a8ca4362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 0x55a8ca4362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 -- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca41df50 msgr2=0x55a8ca436980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca41df50 0x55a8ca436980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd812200640 1 -- 172.21.15.91:0/2459294369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c1e5720 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.244+0000 7fd811800640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 0x55a8ca4362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd812200640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd80402c4d0 tx=0x7fd804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1115cdb11ec9b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd7faa00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804034a50 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd804034bb0 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8ca3815d0 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd804034d70 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd817026740 10 client.8976 fetch_fsmap learned FSMap version 88 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55a8ca355700 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.245+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8ca3815d0 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 88) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd804047410 con 0x55a8ca421020 2024-02-02T13:37:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 10 client.8976 fetch_fsmap finished waiting for FSMap version 88 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 10 client.8976 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 20 client.8976 populate_metadata read hostname 'smithi091' 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca4388b0 0x55a8ca438c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 4 client.8976 mds_command: new command op to 8835 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 --> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8ca3498d0 con 0x55a8ca4388b0 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd811800640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca4388b0 0x55a8ca438c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd811800640 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca4388b0 0x55a8ca438c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a24b430d70d876 server_cookie=553249c55c4ac194 in_seq=0 out_seq=0 2024-02-02T13:37:17.250 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.246+0000 7fd7faa00640 10 client.8976 ms_handle_connect on v2:172.21.15.91:6834/115193290 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd7faa00640 1 -- 172.21.15.91:0/2459294369 <== mds.0 v2:172.21.15.91:6834/115193290 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a8ca3498d0 con 0x55a8ca4388b0 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd7faa00640 10 client.8976 handle_command_reply: tid=0 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 client.8976 shutdown 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 msgr2=0x55a8ca4214a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd80402c4d0 tx=0x7fd804004a40 comp rx=0 tx=0).stop 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 shutdown_connections 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca4388b0 0x55a8ca438c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca4214a0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca41df50 0x55a8ca436980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.251 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 --2- 172.21.15.91:0/2459294369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca3f1ad0 0x55a8ca4362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd7faa00640 0 client.8976 ms_handle_reset on v2:172.21.15.91:6834/115193290 2024-02-02T13:37:17.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.247+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 >> 172.21.15.91:0/2459294369 conn(0x55a8ca3eded0 msgr2=0x55a8ca3f2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.248+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 shutdown_connections 2024-02-02T13:37:17.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.248+0000 7fd817026740 1 -- 172.21.15.91:0/2459294369 wait complete. 2024-02-02T13:37:17.252 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.248+0000 7fd817026740 20 client.8976 trim_cache size 0 max 16384 2024-02-02T13:37:17.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.264+0000 7fd817026740 1 Processor -- start 2024-02-02T13:37:17.265 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.264+0000 7fd817026740 1 -- start start 2024-02-02T13:37:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca421c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca0bef20 con 0x55a8ca421020 2024-02-02T13:37:17.266 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca422170 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca420ae0 con 0x55a8ca41df50 2024-02-02T13:37:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41678/0 (socket says 172.21.15.91:41678) 2024-02-02T13:37:17.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 -- 172.21.15.91:0/34001891 learned_addr learned my addr 172.21.15.91:0/34001891 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 -- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 msgr2=0x55a8ca421c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca421c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 -- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 msgr2=0x55a8ca3f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.265+0000 7fd812200640 1 -- 172.21.15.91:0/34001891 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ca38b280 con 0x55a8ca421020 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd812200640 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd804034750 tx=0x7fd80403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e845d9eee516fe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.268 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd802a00640 1 -- 172.21.15.91:0/34001891 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804047880 con 0x55a8ca421020 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd802a00640 1 -- 172.21.15.91:0/34001891 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd804047410 con 0x55a8ca421020 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd802a00640 1 -- 172.21.15.91:0/34001891 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804047d80 con 0x55a8ca421020 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd817026740 1 -- 172.21.15.91:0/34001891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 msgr2=0x55a8ca43c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.266+0000 7fd817026740 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd804034750 tx=0x7fd80403b000 comp rx=0 tx=0).stop 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 -- 172.21.15.91:0/34001891 shutdown_connections 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca421c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca421020 0x55a8ca43c270 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 --2- 172.21.15.91:0/34001891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 -- 172.21.15.91:0/34001891 >> 172.21.15.91:0/34001891 conn(0x55a8ca3eded0 msgr2=0x55a8ca44b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 -- 172.21.15.91:0/34001891 shutdown_connections 2024-02-02T13:37:17.269 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 -- 172.21.15.91:0/34001891 wait complete. 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 -- start start 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.267+0000 7fd817026740 1 Processor -- start 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.268+0000 7fd817026740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.268+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca439ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.268+0000 7fd817026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca421020 0x55a8ca43a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.273 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.268+0000 7fd817026740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca44b8d0 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca3f3570 con 0x55a8ca421020 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd817026740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ca43ad80 con 0x55a8ca41df50 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd817026740 10 client.0 _ll_register_callbacks cb 0x7fd804037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41684/0 (socket says 172.21.15.91:41684) 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 -- 172.21.15.91:0/2632430329 learned_addr learned my addr 172.21.15.91:0/2632430329 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 -- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 msgr2=0x55a8ca439ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca439ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 -- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca421020 msgr2=0x55a8ca43a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca421020 0x55a8ca43a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 -- 172.21.15.91:0/2632430329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8ca355700 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd811800640 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc056ff0 tx=0x7fd7fc086380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2623589f1cc2e1f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.269+0000 7fd7faa00640 10 client.0 ms_handle_connect on v2:172.21.15.91:3300/0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc053a00 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mon.0 v2:172.21.15.91:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7fc053b60 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.274 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7fc055690 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7fc05d070 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 10 client.8982 fetch_fsmap learned FSMap version 88 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x7fd80c0b5380 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mon.0 v2:172.21.15.91:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc055690 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mon.0 v2:172.21.15.91:3300/0 5 ==== fsmap(e 88) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd7fc05ee40 con 0x55a8ca3f1ad0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 10 client.8982 fetch_fsmap finished waiting for FSMap version 88 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 10 client.8982 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 20 client.8982 populate_metadata read hostname 'smithi091' 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca43c270 0x55a8ca43c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 4 client.8982 mds_command: new command op to 8835 tid=0[{"prefix": "status"}] 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.270+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 --> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8ca350100 con 0x55a8ca43c270 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd810e00640 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca43c270 0x55a8ca43c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd810e00640 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca43c270 0x55a8ca43c650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72da712a7b0f33d7 server_cookie=f2f0c6276f04d33 in_seq=0 out_seq=0 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd7faa00640 10 client.8982 ms_handle_connect on v2:172.21.15.91:6834/115193290 2024-02-02T13:37:17.275 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd7faa00640 1 -- 172.21.15.91:0/2632430329 <== mds.0 v2:172.21.15.91:6834/115193290 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a8ca350100 con 0x55a8ca43c270 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd7faa00640 10 client.8982 handle_command_reply: tid=0 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd817026740 1 client.8982 shutdown 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 msgr2=0x55a8ca3f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc056ff0 tx=0x7fd7fc086380 comp rx=0 tx=0).stop 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.271+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 shutdown_connections 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:6834/115193290,v1:172.21.15.91:6835/115193290] conn(0x55a8ca43c270 0x55a8ca43c650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a8ca41df50 0x55a8ca439ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8ca421020 0x55a8ca43a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 --2- 172.21.15.91:0/2632430329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55a8ca3f1ad0 0x55a8ca3f3990 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd7faa00640 0 client.8982 ms_handle_reset on v2:172.21.15.91:6834/115193290 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 >> 172.21.15.91:0/2632430329 conn(0x55a8ca3eded0 msgr2=0x55a8ca3f1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 shutdown_connections 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 1 -- 172.21.15.91:0/2632430329 wait complete. 2024-02-02T13:37:17.276 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd817026740 20 client.8982 trim_cache size 0 max 16384 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "whoami": 0, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 8835, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "want_state": "up:active", 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "state": "up:active", 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs", 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "rank_uptime": 6.5484295130000003, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "mdsmap_epoch": 88, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch": 195, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "osdmap_epoch_barrier": 195, 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout: "uptime": 11.878135390000001 2024-02-02T13:37:17.277 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:17.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 msgr2=0x7fd7e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 0x7fd7e406c8f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0cab60 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-02-02T13:37:17.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 msgr2=0x7fd80c15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c410 tx=0x7fd7fc00c8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.278 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 shutdown_connections 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd7e406a430 0x7fd7e406c8f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c0cde90 0x7fd80c1e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c15d480 0x7fd80c0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.272+0000 7fd814200640 1 --2- 172.21.15.91:0/490752257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd80c06ae70 0x7fd80c15b8b0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.273+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 >> 172.21.15.91:0/490752257 conn(0x7fd80c09a160 msgr2=0x7fd80c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.273+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 shutdown_connections 2024-02-02T13:37:17.279 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.273+0000 7fd814200640 1 -- 172.21.15.91:0/490752257 wait complete. 2024-02-02T13:37:17.290 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:37:17.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.494+0000 7f103b600640 1 Processor -- start 2024-02-02T13:37:17.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.494+0000 7f103b600640 1 -- start start 2024-02-02T13:37:17.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f103415a010 0x7f10340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340ce130 0x7f10340ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 0x7f10340b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10340cc7c0 con 0x7f103415a010 2024-02-02T13:37:17.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10340ce930 con 0x7f10340cd470 2024-02-02T13:37:17.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f103b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103406c890 con 0x7f10340ce130 2024-02-02T13:37:17.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f103415a010 0x7f10340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f103415a010 0x7f10340c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41694/0 (socket says 172.21.15.91:41694) 2024-02-02T13:37:17.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 -- 172.21.15.91:0/2950498006 learned_addr learned my addr 172.21.15.91:0/2950498006 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 -- 172.21.15.91:0/2950498006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340ce130 msgr2=0x7f10340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1039600640 1 --2- 172.21.15.91:0/2950498006 >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 0x7f10340b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1033e00640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340ce130 0x7f10340ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340ce130 0x7f10340ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 -- 172.21.15.91:0/2950498006 >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 msgr2=0x7f10340b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.496+0000 7f1038c00640 1 --2- 172.21.15.91:0/2950498006 >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 0x7f10340b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.497+0000 7f1038c00640 1 -- 172.21.15.91:0/2950498006 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103406cb70 con 0x7f103415a010 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.497+0000 7f1039600640 1 --2- 172.21.15.91:0/2950498006 >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 0x7f10340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:17.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.498+0000 7f1038c00640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f103415a010 0x7f10340c9670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f10340ce5c0 tx=0x7f102402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf6c700112d45a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.498+0000 7f1033400640 1 -- 172.21.15.91:0/2950498006 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102402f8e0 con 0x7f103415a010 2024-02-02T13:37:17.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.498+0000 7f1033400640 1 -- 172.21.15.91:0/2950498006 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102402fa40 con 0x7f103415a010 2024-02-02T13:37:17.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.498+0000 7f1033400640 1 -- 172.21.15.91:0/2950498006 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102402fdd0 con 0x7f103415a010 2024-02-02T13:37:17.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.498+0000 7f103b600640 1 -- 172.21.15.91:0/2950498006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f103415a010 msgr2=0x7f10340c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f103415a010 0x7f10340c9670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f10340ce5c0 tx=0x7f102402db20 comp rx=0 tx=0).stop 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 -- 172.21.15.91:0/2950498006 shutdown_connections 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 --2- 172.21.15.91:0/2950498006 >> v2:172.21.15.113:3300/0 conn(0x7f10340cd470 0x7f10340b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340ce130 0x7f10340ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 --2- 172.21.15.91:0/2950498006 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f103415a010 0x7f10340c9670 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 -- 172.21.15.91:0/2950498006 >> 172.21.15.91:0/2950498006 conn(0x7f10340cec80 msgr2=0x7f10340cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 -- 172.21.15.91:0/2950498006 shutdown_connections 2024-02-02T13:37:17.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.499+0000 7f103b600640 1 -- 172.21.15.91:0/2950498006 wait complete. 2024-02-02T13:37:17.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.500+0000 7f103b600640 1 Processor -- start 2024-02-02T13:37:17.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 -- start start 2024-02-02T13:37:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103415a010 0x7f103415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340c81f0 0x7f10340c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340c8bc0 con 0x7f10340ce130 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341f2b00 con 0x7f103415a010 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.501+0000 7f103b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340c7a50 con 0x7f10340c81f0 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41700/0 (socket says 172.21.15.91:41700) 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 -- 172.21.15.91:0/2464827056 learned_addr learned my addr 172.21.15.91:0/2464827056 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 -- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340c81f0 msgr2=0x7f10340c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340c81f0 0x7f10340c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1033e00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103415a010 0x7f103415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 -- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103415a010 msgr2=0x7f103415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103415a010 0x7f103415d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.504 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 -- 172.21.15.91:0/2464827056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240094d0 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.502+0000 7f1038c00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f102402db60 tx=0x7f10240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.503+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024040420 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.503+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024040580 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.503+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024040930 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.503+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341f2260 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.503+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341f2820 con 0x7f10340ce130 2024-02-02T13:37:17.505 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.504+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034066cf0 con 0x7f10340ce130 2024-02-02T13:37:17.506 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.505+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1024040a90 con 0x7f10340ce130 2024-02-02T13:37:17.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.506+0000 7f1030c00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 0x7f100c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:17.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.506+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1024033070 con 0x7f10340ce130 2024-02-02T13:37:17.507 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.506+0000 7f1033e00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 0x7f100c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:17.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.507+0000 7f1033e00640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 0x7f100c06c8f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f10340c8d30 tx=0x7f101c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:17.511 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.510+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f10240789d0 con 0x7f10340ce130 2024-02-02T13:37:17.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.721+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f103406ce20 con 0x7f10340ce130 2024-02-02T13:37:17.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.722+0000 7f1030c00640 1 -- 172.21.15.91:0/2464827056 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) v1 ==== 53+0+73 (secure 0 0 0) 0x7f102407bff0 con 0x7f10340ce130 2024-02-02T13:37:17.723 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 msgr2=0x7f100c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 0x7f100c06c8f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f10340c8d30 tx=0x7f101c008040 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 msgr2=0x7f10341f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f102402db60 tx=0x7f10240046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 shutdown_connections 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f100c06a430 0x7f100c06c8f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10340c81f0 0x7f10340c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103415a010 0x7f103415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 --2- 172.21.15.91:0/2464827056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10340ce130 0x7f10341f1bd0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 >> 172.21.15.91:0/2464827056 conn(0x7f10340cec80 msgr2=0x7f103415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 shutdown_connections 2024-02-02T13:37:17.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:17.724+0000 7f103b600640 1 -- 172.21.15.91:0/2464827056 wait complete. 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:37:17.744 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:37:17.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:17.744 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2024-02-02T13:37:17.793 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link proto kernel_ll 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::4ca9:5ff:fe8b:e9/64 scope link proto kernel_ll 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:8a:65:f1:b2:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:37:17.794 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::248a:65ff:fef1:b2e4/64 scope link proto kernel_ll 2024-02-02T13:37:17.795 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.795 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:37:17.795 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 9e:75:8e:c9:17:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:37:17.795 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::9c75:8eff:fec9:17bc/64 scope link proto kernel_ll 2024-02-02T13:37:17.795 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:17.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:17.796 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:37:17.924 INFO:teuthology.orchestra.run.smithi091.stdout:1 2024-02-02T13:37:17.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:17.928 DEBUG:teuthology.orchestra.run.smithi091:> route 2024-02-02T13:37:17.980 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2024-02-02T13:37:17.980 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:37:17.980 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:37:17.980 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:37:17.980 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:37:17.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:37:17.981 DEBUG:teuthology.orchestra.run.smithi091:> ') 2024-02-02T13:37:18.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:18.164 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:37:18.212 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:37:18.212 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:37:18.212 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:37:18.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:18.213 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2024-02-02T13:37:18.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:37:18.303 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T13:37:18.354 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T13:37:18.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:37:18.354 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:37:18.403 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-02T13:37:18.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:18.404 DEBUG:teuthology.orchestra.run.smithi091:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:18.534 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:18.534 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:37:18.568 INFO:teuthology.orchestra.run.smithi091.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:18.568 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:37:18.568 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:37:18.568 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: trying mount with new device syntax: 0@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:37:18.568 INFO:teuthology.orchestra.run.smithi091.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:37:18.570 INFO:teuthology.orchestra.run.smithi091.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-02T13:37:18.570 INFO:teuthology.orchestra.run.smithi091.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:37:18.570 INFO:teuthology.orchestra.run.smithi091.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:37:18.570 INFO:teuthology.orchestra.run.smithi091.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:37:18.570 INFO:teuthology.orchestra.run.smithi091.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:37:18.574 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:37:18.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:18.574 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T13:37:18.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:18.657 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:37:18.728 INFO:teuthology.orchestra.run.smithi091.stdout:client6325 2024-02-02T13:37:18.728 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:18.728 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:18.781 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-02T13:37:18.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.982+0000 7f8b5ea00640 1 Processor -- start 2024-02-02T13:37:18.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.982+0000 7f8b5ea00640 1 -- start start 2024-02-02T13:37:18.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 0x7f8b5806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ea00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5815d870 con 0x7f8b580b0e10 2024-02-02T13:37:18.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5806bce0 con 0x7f8b580ac8d0 2024-02-02T13:37:18.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58161220 con 0x7f8b5815d490 2024-02-02T13:37:18.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ca00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8b580b0e10 0x7f8b580c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:51584/0 (socket says 172.21.15.91:51584) 2024-02-02T13:37:18.986 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b5ca00640 1 -- 172.21.15.91:0/2224972402 learned_addr learned my addr 172.21.15.91:0/2224972402 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:18.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 0x7f8b5806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.987 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226014841 0 0) 0x7f8b5815d870 con 0x7f8b580b0e10 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.984+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b38003520 con 0x7f8b580b0e10 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272765354 0 0) 0x7f8b38003520 con 0x7f8b580b0e10 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 msgr2=0x7f8b5806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 0x7f8b5806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 >> v1:172.21.15.113:6789/0 conn(0x7f8b580ac8d0 legacy=0x7f8b580b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580ad490 con 0x7f8b580b0e10 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f8b500019f0 con 0x7f8b580b0e10 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 >> v1:172.21.15.91:6789/0 conn(0x7f8b580b0e10 legacy=0x7f8b580c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b380039a0 0x7f8b38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 0x7f8b380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b38004d00 0x7f8b38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58161220 con 0x7f8b380039a0 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800bb10 con 0x7f8b38004d00 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800bca0 con 0x7f8b5815d490 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.985+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b380039a0 0x7f8b38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b5ca00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b38004d00 0x7f8b38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b57e00640 1 -- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 msgr2=0x7f8b380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b5815d490 0x7f8b380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b57e00640 1 -- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b38004d00 msgr2=0x7f8b38005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b38004d00 0x7f8b38005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b57e00640 1 -- 172.21.15.91:0/2224972402 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38003520 con 0x7f8b380039a0 2024-02-02T13:37:18.989 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.986+0000 7f8b5ca00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b38004d00 0x7f8b38005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:18.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.987+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b380039a0 0x7f8b38003dc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ba60 tx=0x7f8b380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee85e60f77ab23c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:18.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.987+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48032680 con 0x7f8b380039a0 2024-02-02T13:37:18.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.987+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b480327e0 con 0x7f8b380039a0 2024-02-02T13:37:18.990 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.987+0000 7f8b57400640 1 -- 172.21.15.91:0/2224972402 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48032b70 con 0x7f8b380039a0 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2224972402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b380039a0 msgr2=0x7f8b38003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2224972402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b380039a0 0x7f8b38003dc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ba60 tx=0x7f8b380057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ca00640 1 -- 172.21.15.91:0/2224972402 reap_dead start 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2224972402 shutdown_connections 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2224972402 >> 172.21.15.91:0/2224972402 conn(0x7f8b580cec80 msgr2=0x7f8b580cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2224972402 shutdown_connections 2024-02-02T13:37:18.991 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.988+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2224972402 wait complete. 2024-02-02T13:37:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.989+0000 7f8b5ea00640 1 Processor -- start 2024-02-02T13:37:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.990+0000 7f8b5ea00640 1 -- start start 2024-02-02T13:37:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.990+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b380039a0 0x7f8b581489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b580c6d10 0x7f8b580c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580c8610 con 0x7f8b581f26a0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58149960 con 0x7f8b580c6d10 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580c76d0 con 0x7f8b380039a0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:41710/0 (socket says 172.21.15.91:41710) 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 -- 172.21.15.91:0/2431410566 learned_addr learned my addr 172.21.15.91:0/2431410566 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b5ca00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b580c6d10 0x7f8b580c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b380039a0 0x7f8b581489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 -- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b380039a0 msgr2=0x7f8b581489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b380039a0 0x7f8b581489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 -- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b580c6d10 msgr2=0x7f8b580c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b580c6d10 0x7f8b580c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.991+0000 7f8b4f200640 1 -- 172.21.15.91:0/2431410566 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580c7990 con 0x7f8b581f26a0 2024-02-02T13:37:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b4f200640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c500 tx=0x7f8b4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400052d0 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40005430 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b580c7c70 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40012870 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.992+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b580c8230 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.993+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b58066cf0 con 0x7f8b581f26a0 2024-02-02T13:37:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.994+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b400129d0 con 0x7f8b581f26a0 2024-02-02T13:37:18.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.995+0000 7f8b54c00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 0x7f8b3006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:18.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.995+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b40088530 con 0x7f8b581f26a0 2024-02-02T13:37:18.996 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.995+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 0x7f8b3006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:18.997 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.996+0000 7f8b57e00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 0x7f8b3006c920 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ba30 tx=0x7f8b48030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:19.000 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:18.999+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b40053b10 con 0x7f8b581f26a0 2024-02-02T13:37:19.226 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.225+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b5806b660 con 0x7f8b581f26a0 2024-02-02T13:37:19.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.226+0000 7f8b54c00640 1 -- 172.21.15.91:0/2431410566 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b40057130 con 0x7f8b581f26a0 2024-02-02T13:37:19.227 INFO:teuthology.orchestra.run.smithi091.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 msgr2=0x7f8b3006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 0x7f8b3006c920 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ba30 tx=0x7f8b48030000 comp rx=0 tx=0).stop 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 msgr2=0x7f8b58148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c500 tx=0x7f8b4000c9c0 comp rx=0 tx=0).stop 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 shutdown_connections 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8b3006a460 0x7f8b3006c920 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b580c6d10 0x7f8b580c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b581f26a0 0x7f8b58148f30 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:19.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 --2- 172.21.15.91:0/2431410566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b380039a0 0x7f8b581489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:19.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 >> 172.21.15.91:0/2431410566 conn(0x7f8b580cec80 msgr2=0x7f8b581613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:19.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 shutdown_connections 2024-02-02T13:37:19.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:19.228+0000 7f8b5ea00640 1 -- 172.21.15.91:0/2431410566 wait complete. 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-02T13:37:19.247 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T13:37:19.248 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:37:19.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.248 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-02-02T13:37:19.306 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T13:37:19.306 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T13:37:19.306 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T13:37:19.306 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.306 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 41927sec preferred_lft 41927sec 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link proto kernel_ll 2024-02-02T13:37:19.307 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f082:93ff:fe23:3e4c/64 scope link proto kernel_ll 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:f4:5b:3a:f8:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84f4:5bff:fe3a:f8d3/64 scope link proto kernel_ll 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ea:04:d2:5d:6c:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::e804:d2ff:fe5d:6c60/64 scope link proto kernel_ll 2024-02-02T13:37:19.308 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-02-02T13:37:19.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.309 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T13:37:19.395 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-02-02T13:37:19.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.399 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-02-02T13:37:19.451 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-02-02T13:37:19.451 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T13:37:19.452 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-02T13:37:19.452 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-02T13:37:19.452 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T13:37:19.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.452 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-02-02T13:37:19.452 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-02-02T13:37:19.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-02T13:37:19.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-02T13:37:19.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-02T13:37:19.453 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-02-02T13:37:19.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.633 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:37:19.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:37:19.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:37:19.684 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:37:19.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-02-02T13:37:19.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:37:19.779 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-02T13:37:19.832 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-02T13:37:19.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T13:37:19.833 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:37:19.884 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-02T13:37:19.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:19.885 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:20.022 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:20.022 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:37:20.056 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-02T13:37:20.056 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-02T13:37:20.056 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.91:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-02-02T13:37:20.056 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@cc006970-f900-4781-8947-81ed22cd959b.cephfs=/ 2024-02-02T13:37:20.056 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.91:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-02-02T13:37:20.060 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-02T13:37:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-02-02T13:37:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-02T13:37:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-02-02T13:37:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-02T13:37:20.064 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T13:37:20.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:20.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-02T13:37:20.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:20.147 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T13:37:20.196 INFO:teuthology.orchestra.run.smithi113.stdout:client5717 2024-02-02T13:37:20.196 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:20.197 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:20.245 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:20.459 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.458+0000 7f2357e00640 1 Processor -- start 2024-02-02T13:37:20.460 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.458+0000 7f2357e00640 1 -- start start 2024-02-02T13:37:20.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.459+0000 7f2357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23500c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.461 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2357e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23500cc7c0 con 0x7f23500ce130 2024-02-02T13:37:20.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2357e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23500ce930 con 0x7f23500b0700 2024-02-02T13:37:20.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500cd640 con 0x7f235015a010 2024-02-02T13:37:20.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2354a00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f23500ce130 0x7f23500ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54884/0 (socket says 172.21.15.91:54884) 2024-02-02T13:37:20.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2354a00640 1 -- 172.21.15.91:0/441818008 learned_addr learned my addr 172.21.15.91:0/441818008 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:20.462 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f2355400640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23500c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.460+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608883539 0 0) 0x7f23500cc7c0 con 0x7f23500ce130 2024-02-02T13:37:20.465 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2330005730 con 0x7f23500ce130 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599414491 0 0) 0x7f2330005730 con 0x7f23500ce130 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 msgr2=0x7f23500c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23500c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 >> v1:172.21.15.113:6789/0 conn(0x7f23500b0700 legacy=0x7f235006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235006c890 con 0x7f23500ce130 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2338001670 con 0x7f23500ce130 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 >> v1:172.21.15.91:6789/0 conn(0x7f23500ce130 legacy=0x7f23500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330005bb0 0x7f2330005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2330006f10 0x7f2330007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.466 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500cd640 con 0x7f2330006f10 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233000dc70 con 0x7f2330005bb0 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233000de30 con 0x7f235015a010 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.461+0000 7f2355e00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2330006f10 0x7f2330007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355400640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330005bb0 0x7f2330005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2354a00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355e00640 1 -- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 msgr2=0x7f23300069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355e00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f235015a010 0x7f23300069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355e00640 1 -- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330005bb0 msgr2=0x7f2330005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355e00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330005bb0 0x7f2330005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.462+0000 7f2355e00640 1 -- 172.21.15.91:0/441818008 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2330005730 con 0x7f2330006f10 2024-02-02T13:37:20.467 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.463+0000 7f2355e00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2330006f10 0x7f2330007390 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2348001e20 tx=0x7f2330007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4505e107f9abf7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.463+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348003b70 con 0x7f2330006f10 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.463+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348003cd0 con 0x7f2330006f10 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.463+0000 7f234fe00640 1 -- 172.21.15.91:0/441818008 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005310 con 0x7f2330006f10 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 -- 172.21.15.91:0/441818008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2330006f10 msgr2=0x7f2330007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 --2- 172.21.15.91:0/441818008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2330006f10 0x7f2330007390 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2348001e20 tx=0x7f2330007a00 comp rx=0 tx=0).stop 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2355e00640 1 -- 172.21.15.91:0/441818008 reap_dead start 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 -- 172.21.15.91:0/441818008 shutdown_connections 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 -- 172.21.15.91:0/441818008 >> 172.21.15.91:0/441818008 conn(0x7f23500cec80 msgr2=0x7f23500cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 -- 172.21.15.91:0/441818008 shutdown_connections 2024-02-02T13:37:20.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.464+0000 7f2357e00640 1 -- 172.21.15.91:0/441818008 wait complete. 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.466+0000 7f2357e00640 1 Processor -- start 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.466+0000 7f2357e00640 1 -- start start 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330006f10 0x7f2350158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23500c8950 0x7f2350158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500c78b0 con 0x7f23500c72f0 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500c7b90 con 0x7f2330006f10 2024-02-02T13:37:20.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2357e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500c7e70 con 0x7f23500c8950 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2355e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2355e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57242/0 (socket says 172.21.15.91:57242) 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.467+0000 7f2355e00640 1 -- 172.21.15.91:0/219830090 learned_addr learned my addr 172.21.15.91:0/219830090 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2354a00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23500c8950 0x7f2350158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355400640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330006f10 0x7f2350158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 -- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23500c8950 msgr2=0x7f2350158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23500c8950 0x7f2350158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 -- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330006f10 msgr2=0x7f2350158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330006f10 0x7f2350158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 -- 172.21.15.91:0/219830090 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500c8150 con 0x7f23500c72f0 2024-02-02T13:37:20.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2355e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f2348001e20 tx=0x7f23480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234803e070 con 0x7f23500c72f0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348003e40 con 0x7f23500c72f0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23500c8430 con 0x7f23500c72f0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234802f650 con 0x7f23500c72f0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.468+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2350158b40 con 0x7f23500c72f0 2024-02-02T13:37:20.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.470+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500c8620 con 0x7f23500c72f0 2024-02-02T13:37:20.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.470+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f234802f7d0 con 0x7f23500c72f0 2024-02-02T13:37:20.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.471+0000 7f234d600640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 0x7f232806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:20.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.471+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23480a9f00 con 0x7f23500c72f0 2024-02-02T13:37:20.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.471+0000 7f2355400640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 0x7f232806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:20.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.472+0000 7f2355400640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 0x7f232806c8f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f23500c7280 tx=0x7f2340008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:20.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.476+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f23480754f0 con 0x7f23500c72f0 2024-02-02T13:37:20.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.711+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2350066cf0 con 0x7f23500c72f0 2024-02-02T13:37:20.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.712+0000 7f234d600640 1 -- 172.21.15.91:0/219830090 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f2348078b10 con 0x7f23500c72f0 2024-02-02T13:37:20.713 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:20.713 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:20.714 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:20.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.714+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 msgr2=0x7f232806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.714+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 0x7f232806c8f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f23500c7280 tx=0x7f2340008040 comp rx=0 tx=0).stop 2024-02-02T13:37:20.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.714+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 msgr2=0x7f2350070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.714+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f2348001e20 tx=0x7f23480014a0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 shutdown_connections 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f232806a430 0x7f232806c8f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23500c8950 0x7f2350158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23500c72f0 0x7f2350070af0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 --2- 172.21.15.91:0/219830090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2330006f10 0x7f2350158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 >> 172.21.15.91:0/219830090 conn(0x7f23500cec80 msgr2=0x7f235006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 shutdown_connections 2024-02-02T13:37:20.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:20.715+0000 7f2357e00640 1 -- 172.21.15.91:0/219830090 wait complete. 2024-02-02T13:37:20.736 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:37:20.953 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":8835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.213531875999999,"mdsmap_epoch":88,"osdmap_epoch":195,"osdmap_epoch_barrier":195,"uptime":15.5432512} 2024-02-02T13:37:20.954 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 8835, "mdsmap_epoch": 88, "osdmap_epoch": 195, "osdmap_epoch_barrier": 195, "rank_uptime": 10.213531876, "state": "up:active", "uptime": 15.5432512, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:37:20.954 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-02-02T13:37:21.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.159+0000 7f8140400640 1 Processor -- start 2024-02-02T13:37:21.161 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.159+0000 7f8140400640 1 -- start start 2024-02-02T13:37:21.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f8140400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f813815f5a0 0x7f81380b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f8140400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f81380b0c40 0x7f81380b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f8140400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813815fa00 con 0x7f8138151f90 2024-02-02T13:37:21.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f8140400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380b10d0 con 0x7f81380b0c40 2024-02-02T13:37:21.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f8140400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8138070f80 con 0x7f813815f5a0 2024-02-02T13:37:21.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f813da00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8138151f90 0x7f813806f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:54890/0 (socket says 172.21.15.91:54890) 2024-02-02T13:37:21.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f813da00640 1 -- 172.21.15.91:0/693448408 learned_addr learned my addr 172.21.15.91:0/693448408 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.161+0000 7f813e400640 1 --2- 172.21.15.91:0/693448408 >> v2:172.21.15.91:3300/0 conn(0x7f81380b0c40 0x7f81380b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f813d000640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f813815f5a0 0x7f81380b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f8137e00640 1 -- 172.21.15.91:0/693448408 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965476557 0 0) 0x7f8128001490 con 0x7f8138151f90 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f8137e00640 1 -- 172.21.15.91:0/693448408 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8118003520 con 0x7f8138151f90 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f813e400640 1 -- 172.21.15.91:0/693448408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f813815f5a0 msgr2=0x7f81380b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f813e400640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f813815f5a0 0x7f81380b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f813e400640 1 -- 172.21.15.91:0/693448408 >> v1:172.21.15.91:6789/0 conn(0x7f8138151f90 legacy=0x7f813806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.162+0000 7f813e400640 1 -- 172.21.15.91:0/693448408 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138152370 con 0x7f81380b0c40 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.163+0000 7f813e400640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f81380b3af0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f81300097a0 tx=0x7f813002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a86762519707d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.163+0000 7f8137e00640 1 -- 172.21.15.91:0/693448408 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130031830 con 0x7f81380b0c40 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.163+0000 7f8137e00640 1 -- 172.21.15.91:0/693448408 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8130031990 con 0x7f81380b0c40 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.163+0000 7f8137e00640 1 -- 172.21.15.91:0/693448408 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130031d20 con 0x7f81380b0c40 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 -- 172.21.15.91:0/693448408 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 msgr2=0x7f81380b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f81380b3af0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f81300097a0 tx=0x7f813002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 -- 172.21.15.91:0/693448408 shutdown_connections 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f81380b3af0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 --2- 172.21.15.91:0/693448408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f813815f5a0 0x7f81380b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 -- 172.21.15.91:0/693448408 >> 172.21.15.91:0/693448408 conn(0x7f81380cec80 msgr2=0x7f81380cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 -- 172.21.15.91:0/693448408 shutdown_connections 2024-02-02T13:37:21.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.164+0000 7f8140400640 1 -- 172.21.15.91:0/693448408 wait complete. 2024-02-02T13:37:21.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.166+0000 7f8140400640 1 Processor -- start 2024-02-02T13:37:21.167 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.166+0000 7f8140400640 1 -- start start 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8138151f90 0x7f8138150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f813815f5a0 0x7f813815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138152c10 con 0x7f81380b0c40 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138152930 con 0x7f813815f5a0 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f8140400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8138152f60 con 0x7f8138151f90 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f813da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f813da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57256/0 (socket says 172.21.15.91:57256) 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.167+0000 7f813da00640 1 -- 172.21.15.91:0/4240091033 learned_addr learned my addr 172.21.15.91:0/4240091033 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 -- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8138151f90 msgr2=0x7f8138150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8138151f90 0x7f8138150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 -- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f813815f5a0 msgr2=0x7f813815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f813815f5a0 0x7f813815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.170 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 -- 172.21.15.91:0/4240091033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009400 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f813da00640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f812800cbb0 tx=0x7f812800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800ad40 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8128012400 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800aea0 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.168+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8138153240 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.169+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8138152550 con 0x7f81380b0c40 2024-02-02T13:37:21.171 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.169+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8138153760 con 0x7f81380b0c40 2024-02-02T13:37:21.172 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.170+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f81280127d0 con 0x7f81380b0c40 2024-02-02T13:37:21.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.172+0000 7f8135600640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 0x7f811006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:21.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.172+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81280887a0 con 0x7f81380b0c40 2024-02-02T13:37:21.173 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.172+0000 7f813d000640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 0x7f811006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:21.174 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.173+0000 7f813d000640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 0x7f811006c8f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f81380c5740 tx=0x7f8120008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:21.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.177+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8138153760 con 0x7f81380b0c40 2024-02-02T13:37:21.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.368+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f813815fa00 con 0x7f81380b0c40 2024-02-02T13:37:21.585 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.583+0000 7f8135600640 1 -- 172.21.15.91:0/4240091033 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v196) v1 ==== 181+0+0 (secure 0 0 0) 0x7f813815fa00 con 0x7f81380b0c40 2024-02-02T13:37:21.585 INFO:teuthology.orchestra.run.smithi091.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 msgr2=0x7f811006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 0x7f811006c8f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f81380c5740 tx=0x7f8120008040 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 msgr2=0x7f813815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f812800cbb0 tx=0x7f812800a430 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 shutdown_connections 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f811006a430 0x7f811006c8f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8138151f90 0x7f8138150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f813815f5a0 0x7f813815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 --2- 172.21.15.91:0/4240091033 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f81380b0c40 0x7f813815b510 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 >> 172.21.15.91:0/4240091033 conn(0x7f81380cec80 msgr2=0x7f8138159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 shutdown_connections 2024-02-02T13:37:21.589 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:21.587+0000 7f8140400640 1 -- 172.21.15.91:0/4240091033 wait complete. 2024-02-02T13:37:21.610 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-02-02T13:37:21.836 INFO:teuthology.orchestra.run.smithi091.stdout:{"osd_mon_report_interval":"5"} 2024-02-02T13:37:21.837 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-02-02T13:37:21.837 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-02-02T13:37:21.837 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:21.837 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:21.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:21.890 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-02-02T13:37:22.144 INFO:teuthology.orchestra.run.smithi091.stderr:16+0 records in 2024-02-02T13:37:22.144 INFO:teuthology.orchestra.run.smithi091.stderr:16+0 records out 2024-02-02T13:37:22.144 INFO:teuthology.orchestra.run.smithi091.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.197984 s, 84.7 MB/s 2024-02-02T13:37:22.145 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:22.145 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:22.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:22.188 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-02-02T13:37:22.433 INFO:teuthology.orchestra.run.smithi091.stderr:17+0 records in 2024-02-02T13:37:22.433 INFO:teuthology.orchestra.run.smithi091.stderr:17+0 records out 2024-02-02T13:37:22.433 INFO:teuthology.orchestra.run.smithi091.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.151136 s, 118 MB/s 2024-02-02T13:37:22.434 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-02-02T13:37:22.435 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:22.641 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.640+0000 7f5589800640 1 Processor -- start 2024-02-02T13:37:22.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.640+0000 7f5589800640 1 -- start start 2024-02-02T13:37:22.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.641+0000 7f5589800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 0x7f55840ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.641+0000 7f5589800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 0x7f55840acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5589800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55840c94a0 con 0x7f558415d3a0 2024-02-02T13:37:22.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5589800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840ad6e0 con 0x7f55840cb580 2024-02-02T13:37:22.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5589800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840c96f0 con 0x7f55840ac960 2024-02-02T13:37:22.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5582a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 0x7f55840ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5583400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 0x7f55840acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5582a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 0x7f55840ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:46600/0 (socket says 172.21.15.91:46600) 2024-02-02T13:37:22.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.642+0000 7f5582a00640 1 -- 172.21.15.91:0/795343579 learned_addr learned my addr 172.21.15.91:0/795343579 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:22.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.643+0000 7f5583400640 1 -- 172.21.15.91:0/795343579 >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 msgr2=0x7f55840ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:22.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.643+0000 7f5583400640 1 --2- 172.21.15.91:0/795343579 >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 0x7f55840ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.643+0000 7f5583400640 1 -- 172.21.15.91:0/795343579 >> v1:172.21.15.113:6789/0 conn(0x7f558415d3a0 legacy=0x7f55840cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.643+0000 7f5583400640 1 -- 172.21.15.91:0/795343579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840c99d0 con 0x7f55840ac960 2024-02-02T13:37:22.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.644+0000 7f5583400640 1 --2- 172.21.15.91:0/795343579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 0x7f55840acdf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f557c009d90 tx=0x7f557c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4b886740a3c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:22.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5582000640 1 -- 172.21.15.91:0/795343579 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c02cc50 con 0x7f55840ac960 2024-02-02T13:37:22.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5582000640 1 -- 172.21.15.91:0/795343579 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c02cdb0 con 0x7f55840ac960 2024-02-02T13:37:22.646 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5582000640 1 -- 172.21.15.91:0/795343579 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c034660 con 0x7f55840ac960 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5589800640 1 -- 172.21.15.91:0/795343579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 msgr2=0x7f55840acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5589800640 1 --2- 172.21.15.91:0/795343579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 0x7f55840acdf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f557c009d90 tx=0x7f557c02bea0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5589800640 1 -- 172.21.15.91:0/795343579 shutdown_connections 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5589800640 1 --2- 172.21.15.91:0/795343579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55840ac960 0x7f55840acdf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.645+0000 7f5589800640 1 --2- 172.21.15.91:0/795343579 >> v2:172.21.15.113:3300/0 conn(0x7f55840cb580 0x7f55840ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.646+0000 7f5589800640 1 -- 172.21.15.91:0/795343579 >> 172.21.15.91:0/795343579 conn(0x7f55840b0700 msgr2=0x7f55840b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:22.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.646+0000 7f5589800640 1 -- 172.21.15.91:0/795343579 shutdown_connections 2024-02-02T13:37:22.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.646+0000 7f5589800640 1 -- 172.21.15.91:0/795343579 wait complete. 2024-02-02T13:37:22.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.647+0000 7f5589800640 1 Processor -- start 2024-02-02T13:37:22.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.647+0000 7f5589800640 1 -- start start 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55840ac960 0x7f558415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558415d3a0 0x7f55840b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840b24f0 con 0x7f55840cb580 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840b18e0 con 0x7f55840ac960 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.648+0000 7f5589800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840b1bc0 con 0x7f558415d3a0 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57262/0 (socket says 172.21.15.91:57262) 2024-02-02T13:37:22.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 -- 172.21.15.91:0/1972669986 learned_addr learned my addr 172.21.15.91:0/1972669986 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:22.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f5582a00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55840ac960 0x7f558415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f5583400640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558415d3a0 0x7f55840b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 -- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558415d3a0 msgr2=0x7f55840b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558415d3a0 0x7f55840b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 -- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55840ac960 msgr2=0x7f558415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55840ac960 0x7f558415e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.649+0000 7f557be00640 1 -- 172.21.15.91:0/1972669986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c009a70 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f557be00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c400 tx=0x7f556c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0052d0 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c005430 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c003830 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55840ad4b0 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.650+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55840b2180 con 0x7f55840cb580 2024-02-02T13:37:22.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.651+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55840b2310 con 0x7f55840cb580 2024-02-02T13:37:22.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.652+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f556c003990 con 0x7f55840cb580 2024-02-02T13:37:22.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.653+0000 7f557aa00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 0x7f555c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:22.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.653+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f556c087ed0 con 0x7f55840cb580 2024-02-02T13:37:22.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.653+0000 7f5582a00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 0x7f555c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:22.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.654+0000 7f5582a00640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 0x7f555c06c8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5574009aa0 tx=0x7f5574008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:22.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.657+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f556c0534b0 con 0x7f55840cb580 2024-02-02T13:37:22.853 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.851+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5584066cf0 con 0x7f55840cb580 2024-02-02T13:37:22.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.855+0000 7f557aa00640 1 -- 172.21.15.91:0/1972669986 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v196) v1 ==== 81+0+33546 (secure 0 0 0) 0x7f556c056ad0 con 0x7f55840cb580 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 196, 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:21.568474+0000", 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:22.857 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:22.858 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:22.859 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:22.860 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:22.861 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.862 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:22.863 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:22.864 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:22.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:22.866 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:22.867 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:22.868 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:22.869 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:22.870 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "196", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:22.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:22.872 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:22.873 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:22.874 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.875 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.878 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:22.879 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.880 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.881 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.882 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.883 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.884 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.885 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:22.886 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.888 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:22.889 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.890 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.891 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.892 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.893 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.894 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.895 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.897 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:22.899 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.901 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.902 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.903 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.904 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.905 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.908 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:22.909 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.910 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:22.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:22.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:22.913 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.914 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.915 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:22.916 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.917 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:22.918 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:22.919 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:22.920 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 msgr2=0x7f555c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 0x7f555c06c8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5574009aa0 tx=0x7f5574008040 comp rx=0 tx=0).stop 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 msgr2=0x7f558415f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c400 tx=0x7f556c00c8c0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 shutdown_connections 2024-02-02T13:37:22.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f555c06a430 0x7f555c06c8f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558415d3a0 0x7f55840b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55840cb580 0x7f558415f5b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 --2- 172.21.15.91:0/1972669986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55840ac960 0x7f558415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 >> 172.21.15.91:0/1972669986 conn(0x7f55840b0700 msgr2=0x7f55840b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 shutdown_connections 2024-02-02T13:37:22.922 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:22.858+0000 7f5589800640 1 -- 172.21.15.91:0/1972669986 wait complete. 2024-02-02T13:37:22.922 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:37:27.923 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.140+0000 7ffa45600640 1 Processor -- start 2024-02-02T13:37:28.142 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.140+0000 7ffa45600640 1 -- start start 2024-02-02T13:37:28.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.141+0000 7ffa45600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffa40159de0 0x7ffa400cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.141+0000 7ffa45600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa4015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.141+0000 7ffa45600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400cbf20 con 0x7ffa4015e170 2024-02-02T13:37:28.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.141+0000 7ffa45600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa40070ca0 con 0x7ffa40159de0 2024-02-02T13:37:28.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.141+0000 7ffa45600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400ca0c0 con 0x7ffa400cbac0 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffa40159de0 0x7ffa400cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffa40159de0 0x7ffa400cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57264/0 (socket says 172.21.15.91:57264) 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 -- 172.21.15.91:0/3271812319 learned_addr learned my addr 172.21.15.91:0/3271812319 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3e000640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa4015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 -- 172.21.15.91:0/3271812319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 msgr2=0x7ffa4015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa4015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 -- 172.21.15.91:0/3271812319 >> v1:172.21.15.113:6789/0 conn(0x7ffa4015e170 legacy=0x7ffa400cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3ea00640 1 -- 172.21.15.91:0/3271812319 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400ca380 con 0x7ffa40159de0 2024-02-02T13:37:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.142+0000 7ffa3e000640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa4015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:28.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.144+0000 7ffa3ea00640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa40159de0 0x7ffa400cb580 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000d150 tx=0x7ffa3002e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5594bd81d9f9abf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:28.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.144+0000 7ffa3d600640 1 -- 172.21.15.91:0/3271812319 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3002ea70 con 0x7ffa40159de0 2024-02-02T13:37:28.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.144+0000 7ffa3d600640 1 -- 172.21.15.91:0/3271812319 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3002ebd0 con 0x7ffa40159de0 2024-02-02T13:37:28.145 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.144+0000 7ffa3d600640 1 -- 172.21.15.91:0/3271812319 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30009600 con 0x7ffa40159de0 2024-02-02T13:37:28.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 -- 172.21.15.91:0/3271812319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa40159de0 msgr2=0x7ffa400cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:28.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa40159de0 0x7ffa400cb580 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000d150 tx=0x7ffa3002e0d0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 -- 172.21.15.91:0/3271812319 shutdown_connections 2024-02-02T13:37:28.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa4015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 --2- 172.21.15.91:0/3271812319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa40159de0 0x7ffa400cb580 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 -- 172.21.15.91:0/3271812319 >> 172.21.15.91:0/3271812319 conn(0x7ffa400b0700 msgr2=0x7ffa400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:28.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 -- 172.21.15.91:0/3271812319 shutdown_connections 2024-02-02T13:37:28.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.145+0000 7ffa45600640 1 -- 172.21.15.91:0/3271812319 wait complete. 2024-02-02T13:37:28.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.146+0000 7ffa45600640 1 Processor -- start 2024-02-02T13:37:28.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.147+0000 7ffa45600640 1 -- start start 2024-02-02T13:37:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.147+0000 7ffa45600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa400c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.147+0000 7ffa45600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa40159de0 0x7ffa401f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa45600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa45600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401eea30 con 0x7ffa4015e170 2024-02-02T13:37:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa45600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4014cac0 con 0x7ffa40159de0 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa45600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401f2430 con 0x7ffa400cbac0 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:57268/0 (socket says 172.21.15.91:57268) 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 -- 172.21.15.91:0/2465344062 learned_addr learned my addr 172.21.15.91:0/2465344062 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 -- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 msgr2=0x7ffa400c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa400c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 -- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa40159de0 msgr2=0x7ffa401f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:28.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa40159de0 0x7ffa401f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.148+0000 7ffa3f400640 1 -- 172.21.15.91:0/2465344062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3000bd40 con 0x7ffa4015e170 2024-02-02T13:37:28.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa3f400640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b410 tx=0x7ffa3800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004020 con 0x7ffa4015e170 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa38004180 con 0x7ffa4015e170 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38011630 con 0x7ffa4015e170 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa401f2710 con 0x7ffa4015e170 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.149+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa401ee650 con 0x7ffa4015e170 2024-02-02T13:37:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.150+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa40066cf0 con 0x7ffa4015e170 2024-02-02T13:37:28.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.151+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffa38011790 con 0x7ffa4015e170 2024-02-02T13:37:28.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.152+0000 7ffa36a00640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 0x7ffa1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:28.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.152+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa38088690 con 0x7ffa4015e170 2024-02-02T13:37:28.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.152+0000 7ffa3ea00640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 0x7ffa1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:28.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.153+0000 7ffa3ea00640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 0x7ffa1806c8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffa30037000 tx=0x7ffa30030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:28.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.157+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffa38053c70 con 0x7ffa4015e170 2024-02-02T13:37:28.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.349+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ffa401eeb70 con 0x7ffa4015e170 2024-02-02T13:37:28.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.353+0000 7ffa36a00640 1 -- 172.21.15.91:0/2465344062 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v196) v1 ==== 81+0+33546 (secure 0 0 0) 0x7ffa401eeb70 con 0x7ffa4015e170 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 196, 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:21.568474+0000", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:28.355 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:28.356 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:28.357 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:28.358 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:28.359 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.360 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.361 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:28.362 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:28.363 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:28.364 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:28.365 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:28.366 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:28.367 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:28.368 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:28.369 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "196", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:28.370 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:28.371 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:28.372 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:28.373 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.374 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.375 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.377 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.378 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.379 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.380 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.381 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.382 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.383 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:28.384 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.385 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.386 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.387 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.388 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.388 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.388 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.390 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.390 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.390 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.390 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.391 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.392 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.393 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.394 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:28.395 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.396 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.397 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.398 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.399 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.401 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.402 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.403 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.404 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.405 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.406 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.407 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.408 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.409 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.410 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.411 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.412 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.413 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.416 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:28.417 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.418 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.419 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:28.420 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.421 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.422 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.423 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.424 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.425 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.426 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:28.427 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:28.428 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:28.430 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.355+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 msgr2=0x7ffa1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.355+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 0x7ffa1806c8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffa30037000 tx=0x7ffa30030000 comp rx=0 tx=0).stop 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 msgr2=0x7ffa401eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b410 tx=0x7ffa3800b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 shutdown_connections 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ffa1806a430 0x7ffa1806c8f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa4015e170 0x7ffa401eee50 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa40159de0 0x7ffa401f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 --2- 172.21.15.91:0/2465344062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa400cbac0 0x7ffa400c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:28.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 >> 172.21.15.91:0/2465344062 conn(0x7ffa400b0700 msgr2=0x7ffa400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:28.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 shutdown_connections 2024-02-02T13:37:28.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:28.356+0000 7ffa45600640 1 -- 172.21.15.91:0/2465344062 wait complete. 2024-02-02T13:37:28.432 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:37:32.841 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:37:32.849 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:37:33.434 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.648+0000 7f29ae400640 1 Processor -- start 2024-02-02T13:37:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.648+0000 7f29ae400640 1 -- start start 2024-02-02T13:37:33.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a8071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f29a80c2170 0x7f29a80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29a80cb770 0x7f29a81650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80cbc00 con 0x7f29a80c2170 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8161930 con 0x7f29a80cb770 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f29ae400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c9c70 con 0x7f29a8161550 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.649+0000 7f299e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f29a80c2170 0x7f29a80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f29a80c2170 0x7f29a80cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54336/0 (socket says 172.21.15.91:54336) 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 -- 172.21.15.91:0/1812811767 learned_addr learned my addr 172.21.15.91:0/1812811767 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 -- 172.21.15.91:0/1812811767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 msgr2=0x7f29a8071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 --2- 172.21.15.91:0/1812811767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 -- 172.21.15.91:0/1812811767 >> v2:172.21.15.113:3300/0 conn(0x7f29a80cb770 msgr2=0x7f29a81650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 --2- 172.21.15.91:0/1812811767 >> v2:172.21.15.113:3300/0 conn(0x7f29a80cb770 0x7f29a81650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.650+0000 7f299e800640 1 -- 172.21.15.91:0/1812811767 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80c9f50 con 0x7f29a80c2170 2024-02-02T13:37:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.651+0000 7f299e800640 1 --2- 172.21.15.91:0/1812811767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a80cb230 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f29a80c2100 tx=0x7f299002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceb84bc4101e7d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.651+0000 7f29a6a00640 1 -- 172.21.15.91:0/1812811767 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990031860 con 0x7f29a80c2170 2024-02-02T13:37:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.651+0000 7f29a6a00640 1 -- 172.21.15.91:0/1812811767 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29900319c0 con 0x7f29a80c2170 2024-02-02T13:37:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.651+0000 7f29a6a00640 1 -- 172.21.15.91:0/1812811767 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990031d50 con 0x7f29a80c2170 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 -- 172.21.15.91:0/1812811767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 msgr2=0x7f29a80cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 --2- 172.21.15.91:0/1812811767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a80cb230 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f29a80c2100 tx=0x7f299002b900 comp rx=0 tx=0).stop 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 -- 172.21.15.91:0/1812811767 shutdown_connections 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 --2- 172.21.15.91:0/1812811767 >> v2:172.21.15.113:3300/0 conn(0x7f29a80cb770 0x7f29a81650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 --2- 172.21.15.91:0/1812811767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a80cb230 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 --2- 172.21.15.91:0/1812811767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a8071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 -- 172.21.15.91:0/1812811767 >> 172.21.15.91:0/1812811767 conn(0x7f29a80b0700 msgr2=0x7f29a80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 -- 172.21.15.91:0/1812811767 shutdown_connections 2024-02-02T13:37:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.652+0000 7f29ae400640 1 -- 172.21.15.91:0/1812811767 wait complete. 2024-02-02T13:37:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.654+0000 7f29ae400640 1 Processor -- start 2024-02-02T13:37:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.654+0000 7f29ae400640 1 -- start start 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 0x7f29a815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c5c40 con 0x7f29a80c2170 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a815d630 con 0x7f29a80cb770 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29ae400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8164ec0 con 0x7f29a8161550 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29a7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.655+0000 7f29a7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54350/0 (socket says 172.21.15.91:54350) 2024-02-02T13:37:33.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 -- 172.21.15.91:0/1561068757 learned_addr learned my addr 172.21.15.91:0/1561068757 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7e00640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f299e800640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 0x7f29a815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 -- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 msgr2=0x7f29a815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a815d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 -- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 msgr2=0x7f29a815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 0x7f29a815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900094d0 con 0x7f29a80c2170 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f299e800640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 0x7f29a815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f29a7400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f299800ba50 tx=0x7f299800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.656+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800cbe0 con 0x7f29a80c2170 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.657+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299800cd60 con 0x7f29a80c2170 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.657+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998014680 con 0x7f29a80c2170 2024-02-02T13:37:33.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.657+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a81651a0 con 0x7f29a80c2170 2024-02-02T13:37:33.660 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.657+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80c6170 con 0x7f29a80c2170 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.658+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a80c6360 con 0x7f29a80c2170 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.659+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2998007370 con 0x7f29a80c2170 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.659+0000 7f299fe00640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 0x7f298006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.659+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29980888e0 con 0x7f29a80c2170 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.660+0000 7f299e800640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 0x7f298006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.660+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=198}) v3 -- 0x7f2980071f60 con 0x7f29a80c2170 2024-02-02T13:37:33.661 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.660+0000 7f299e800640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 0x7f298006c8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2990036000 tx=0x7f299003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:33.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.664+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2998019020 con 0x7f29a80c2170 2024-02-02T13:37:33.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.857+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f29a8066cf0 con 0x7f29a80c2170 2024-02-02T13:37:33.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.859+0000 7f299fe00640 1 -- 172.21.15.91:0/1561068757 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v197) v1 ==== 81+0+33565 (secure 0 0 0) 0x7f2998053ec0 con 0x7f29a80c2170 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 197, 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:29.545809+0000", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:33.862 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:33.863 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:33.864 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:33.865 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:33.866 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:33.867 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:33.868 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:33.869 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:33.870 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:33.871 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:33.872 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:33.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:33.874 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.875 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.876 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:33.877 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "197", 2024-02-02T13:37:33.878 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:33.879 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.880 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:33.881 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:33.882 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.883 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.884 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.885 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:33.886 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.887 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.889 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.890 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.891 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.892 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.893 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.894 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.895 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.896 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.897 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:33.898 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.899 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.900 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.901 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.902 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:33.903 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:33.904 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.905 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.906 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.907 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.908 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.909 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.910 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.911 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:33.912 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.913 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:33.914 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:33.915 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.916 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.917 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.918 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.919 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:33.920 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:33.921 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:33.922 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.923 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:33.924 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:33.925 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.926 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.927 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.928 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.929 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.930 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:33.931 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:33.932 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:33.933 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:33.934 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:33.934 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:33.934 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:33.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.862+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 msgr2=0x7f298006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:33.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.862+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 0x7f298006c8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2990036000 tx=0x7f299003c040 comp rx=0 tx=0).stop 2024-02-02T13:37:33.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.862+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 msgr2=0x7f29a8164870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:33.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.862+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f299800ba50 tx=0x7f299800bf10 comp rx=0 tx=0).stop 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 shutdown_connections 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f298006a430 0x7f298006c8f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29a8161550 0x7f29a815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a80cb770 0x7f29a815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 --2- 172.21.15.91:0/1561068757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29a80c2170 0x7f29a8164870 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 >> 172.21.15.91:0/1561068757 conn(0x7f29a80b0700 msgr2=0x7f29a806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 shutdown_connections 2024-02-02T13:37:33.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:33.863+0000 7f29ae400640 1 -- 172.21.15.91:0/1561068757 wait complete. 2024-02-02T13:37:33.937 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-02-02T13:37:33.937 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:33.937 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:33.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:33.991 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-02-02T13:37:34.402 INFO:teuthology.orchestra.run.smithi091.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-02-02T13:37:34.402 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:37:34.403 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:37:34.403 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.354122 s, 148 MB/s 2024-02-02T13:37:34.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:37:34.404 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:34.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.604+0000 7fc170e00640 1 Processor -- start 2024-02-02T13:37:34.606 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.604+0000 7fc170e00640 1 -- start start 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc170e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 0x7fc16c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc170e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc16c0ce530 0x7fc16c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc170e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c160d90 con 0x7fc16c0707b0 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc170e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c070c40 con 0x7fc16c0ce530 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc170e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c070fe0 con 0x7fc16c1609b0 2024-02-02T13:37:34.608 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc161200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc16c0ce530 0x7fc16c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc161200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc16c0ce530 0x7fc16c070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54354/0 (socket says 172.21.15.91:54354) 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc16a800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc16c0707b0 0x7fc16c0d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56308/0 (socket says 172.21.15.91:56308) 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.606+0000 7fc161200640 1 -- 172.21.15.91:0/1250275717 learned_addr learned my addr 172.21.15.91:0/1250275717 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc161200640 1 -- 172.21.15.91:0/1250275717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 msgr2=0x7fc16c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc169e00640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 0x7fc16c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc161200640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 0x7fc16c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc161200640 1 -- 172.21.15.91:0/1250275717 >> v1:172.21.15.91:6789/0 conn(0x7fc16c0707b0 legacy=0x7fc16c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc161200640 1 -- 172.21.15.91:0/1250275717 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0d2940 con 0x7fc16c0ce530 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc169400640 1 -- 172.21.15.91:0/1250275717 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681084089 0 0) 0x7fc164001490 con 0x7fc16c0707b0 2024-02-02T13:37:34.609 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.607+0000 7fc169e00640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 0x7fc16c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:34.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.608+0000 7fc161200640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c070270 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66d679b17a8024d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:34.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.608+0000 7fc169400640 1 -- 172.21.15.91:0/1250275717 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540317c0 con 0x7fc16c0ce530 2024-02-02T13:37:34.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.608+0000 7fc169400640 1 -- 172.21.15.91:0/1250275717 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154031920 con 0x7fc16c0ce530 2024-02-02T13:37:34.610 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.608+0000 7fc169400640 1 -- 172.21.15.91:0/1250275717 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154031cb0 con 0x7fc16c0ce530 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 -- 172.21.15.91:0/1250275717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 msgr2=0x7fc16c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c070270 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15402b860 comp rx=0 tx=0).stop 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 -- 172.21.15.91:0/1250275717 shutdown_connections 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c070270 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 --2- 172.21.15.91:0/1250275717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c1609b0 0x7fc16c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 -- 172.21.15.91:0/1250275717 >> 172.21.15.91:0/1250275717 conn(0x7fc16c0d3020 msgr2=0x7fc16c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 -- 172.21.15.91:0/1250275717 shutdown_connections 2024-02-02T13:37:34.611 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.609+0000 7fc170e00640 1 -- 172.21.15.91:0/1250275717 wait complete. 2024-02-02T13:37:34.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.610+0000 7fc170e00640 1 Processor -- start 2024-02-02T13:37:34.612 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.611+0000 7fc170e00640 1 -- start start 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.611+0000 7fc170e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c0707b0 0x7fc16c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc170e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc170e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc16c1609b0 0x7fc16c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc170e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1541f0 con 0x7fc16c0ce530 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc170e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c153280 con 0x7fc16c1609b0 2024-02-02T13:37:34.613 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc170e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c153500 con 0x7fc16c0707b0 2024-02-02T13:37:34.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:34.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54360/0 (socket says 172.21.15.91:54360) 2024-02-02T13:37:34.614 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 -- 172.21.15.91:0/2190945358 learned_addr learned my addr 172.21.15.91:0/2190945358 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 -- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c0707b0 msgr2=0x7fc16c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c0707b0 0x7fc16c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 -- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc16c1609b0 msgr2=0x7fc16c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc16c1609b0 0x7fc16c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.612+0000 7fc161200640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1540094d0 con 0x7fc16c0ce530 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc161200640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc15402b8a0 tx=0x7fc1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154032960 con 0x7fc16c0ce530 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154032b00 con 0x7fc16c0ce530 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540453f0 con 0x7fc16c0ce530 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c1537e0 con 0x7fc16c0ce530 2024-02-02T13:37:34.615 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.613+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c153e10 con 0x7fc16c0ce530 2024-02-02T13:37:34.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.614+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c066cf0 con 0x7fc16c0ce530 2024-02-02T13:37:34.616 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.615+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc15404e450 con 0x7fc16c0ce530 2024-02-02T13:37:34.617 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.616+0000 7fc162a00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 0x7fc14406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:34.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.616+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1540aacf0 con 0x7fc16c0ce530 2024-02-02T13:37:34.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.616+0000 7fc169e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 0x7fc14406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:34.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.616+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=198}) v3 -- 0x7fc144071f60 con 0x7fc16c0ce530 2024-02-02T13:37:34.618 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.617+0000 7fc169e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 0x7fc14406c8f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:34.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.621+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc1540af070 con 0x7fc16c0ce530 2024-02-02T13:37:34.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.816+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc16c0b1bb0 con 0x7fc16c0ce530 2024-02-02T13:37:34.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.819+0000 7fc162a00640 1 -- 172.21.15.91:0/2190945358 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v197) v1 ==== 81+0+33565 (secure 0 0 0) 0x7fc1540762e0 con 0x7fc16c0ce530 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 197, 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:29.545809+0000", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:34.822 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:34.823 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:34.824 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:34.825 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:34.826 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.827 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:34.828 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:34.829 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:34.830 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:34.831 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:34.832 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:34.833 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.834 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:34.835 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:34.836 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:34.837 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "197", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:34.838 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:34.839 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:34.840 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:34.841 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.842 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.843 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.844 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.845 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:34.846 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.847 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:34.848 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.849 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.850 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.851 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.852 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.853 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:34.854 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.855 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.856 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.857 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.858 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.859 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.860 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.861 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.862 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.863 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.864 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.865 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.866 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:34.867 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.868 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.869 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.870 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.871 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.872 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.873 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:34.874 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.875 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.876 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.877 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.879 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.880 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.881 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:34.882 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:34.883 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.884 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.885 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.886 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.887 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:34.888 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:34.889 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:34.890 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:34.891 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:34.891 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:34.891 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.821+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 msgr2=0x7fc14406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.821+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 0x7fc14406c8f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c008040 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.821+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 msgr2=0x7fc16c152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.821+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc15402b8a0 tx=0x7fc1540046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 shutdown_connections 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc14406a430 0x7fc14406c8f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc16c1609b0 0x7fc16c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc16c0ce530 0x7fc16c152d40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 --2- 172.21.15.91:0/2190945358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc16c0707b0 0x7fc16c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 >> 172.21.15.91:0/2190945358 conn(0x7fc16c0d3020 msgr2=0x7fc16c0c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 shutdown_connections 2024-02-02T13:37:34.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:34.822+0000 7fc170e00640 1 -- 172.21.15.91:0/2190945358 wait complete. 2024-02-02T13:37:34.894 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:35.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.101+0000 7f1828e00640 1 Processor -- start 2024-02-02T13:37:35.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.101+0000 7f1828e00640 1 -- start start 2024-02-02T13:37:35.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1828e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18240c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1828e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18240b0700 0x7f18240b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1828e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240cc7c0 con 0x7f182415a010 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1828e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240b10d0 con 0x7f18240b0700 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1828e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240ce930 con 0x7f18240ce130 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1821e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f182415a010 0x7f18240ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56324/0 (socket says 172.21.15.91:56324) 2024-02-02T13:37:35.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.102+0000 7f1821e00640 1 -- 172.21.15.91:0/1194041081 learned_addr learned my addr 172.21.15.91:0/1194041081 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1819200640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18240c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772217846 0 0) 0x7f18240cc7c0 con 0x7f182415a010 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1822800640 1 --2- 172.21.15.91:0/1194041081 >> v2:172.21.15.113:3300/0 conn(0x7f18240b0700 0x7f18240b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1804003520 con 0x7f182415a010 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2569071207 0 0) 0x7f1804003520 con 0x7f182415a010 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 msgr2=0x7f18240c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18240c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 >> v2:172.21.15.113:3300/0 conn(0x7f18240b0700 msgr2=0x7f18240b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 --2- 172.21.15.91:0/1194041081 >> v2:172.21.15.113:3300/0 conn(0x7f18240b0700 0x7f18240b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240cd640 con 0x7f182415a010 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1814001670 con 0x7f182415a010 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 >> v1:172.21.15.91:6789/0 conn(0x7f182415a010 legacy=0x7f18240ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f1804003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804004d00 0x7f1804005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240b10d0 con 0x7f1804003970 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240ce930 con 0x7f1804004d00 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.103+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180400db90 con 0x7f18240ce130 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f1804003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1819200640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 -- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 msgr2=0x7f18040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18240ce130 0x7f18040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 -- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804004d00 msgr2=0x7f1804005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:35.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804004d00 0x7f1804005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.104+0000 7f1821e00640 1 -- 172.21.15.91:0/1194041081 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804003520 con 0x7f1804003970 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.105+0000 7f1821e00640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f1804003d90 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f181400c110 tx=0x7f18040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eefef4938ce2922 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.105+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814005ad0 con 0x7f1804003970 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.105+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814005c50 con 0x7f1804003970 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.105+0000 7f1821400640 1 -- 172.21.15.91:0/1194041081 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181402c830 con 0x7f1804003970 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 -- 172.21.15.91:0/1194041081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 msgr2=0x7f1804003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 --2- 172.21.15.91:0/1194041081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f1804003d90 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f181400c110 tx=0x7f18040057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1822800640 1 -- 172.21.15.91:0/1194041081 reap_dead start 2024-02-02T13:37:35.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 -- 172.21.15.91:0/1194041081 shutdown_connections 2024-02-02T13:37:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 -- 172.21.15.91:0/1194041081 >> 172.21.15.91:0/1194041081 conn(0x7f18240cec80 msgr2=0x7f18240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 -- 172.21.15.91:0/1194041081 shutdown_connections 2024-02-02T13:37:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.106+0000 7f1828e00640 1 -- 172.21.15.91:0/1194041081 wait complete. 2024-02-02T13:37:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.108+0000 7f1828e00640 1 Processor -- start 2024-02-02T13:37:35.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.108+0000 7f1828e00640 1 -- start start 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1824157f80 0x7f182406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824070420 0x7f18241f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240b4a10 con 0x7f1804003970 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240708a0 con 0x7f1824070420 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1828e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c8c20 con 0x7f1824157f80 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1821e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1821e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54378/0 (socket says 172.21.15.91:54378) 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1821e00640 1 -- 172.21.15.91:0/455052062 learned_addr learned my addr 172.21.15.91:0/455052062 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.109+0000 7f1819200640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1824157f80 0x7f182406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1822800640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824070420 0x7f18241f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 -- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1824157f80 msgr2=0x7f182406fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1824157f80 0x7f182406fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 -- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824070420 msgr2=0x7f18241f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824070420 0x7f18241f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240b4730 con 0x7f1804003970 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1821e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f181402c4a0 tx=0x7f1814001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814039070 con 0x7f1804003970 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181402dcd0 con 0x7f1804003970 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814007270 con 0x7f1804003970 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240c2530 con 0x7f1804003970 2024-02-02T13:37:35.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.110+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18240c2af0 con 0x7f1804003970 2024-02-02T13:37:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.111+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18240c2cb0 con 0x7f1804003970 2024-02-02T13:37:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.112+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1814032070 con 0x7f1804003970 2024-02-02T13:37:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.113+0000 7f181aa00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 0x7f17fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.113+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f181403c040 con 0x7f1804003970 2024-02-02T13:37:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.113+0000 7f1819200640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 0x7f17fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.113+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=198}) v3 -- 0x7f17fc071f60 con 0x7f1804003970 2024-02-02T13:37:35.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.114+0000 7f1819200640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 0x7f17fc06c8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f18240b0390 tx=0x7f180c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.117+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1814045060 con 0x7f1804003970 2024-02-02T13:37:35.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.343+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18240c2390 con 0x7f1804003970 2024-02-02T13:37:35.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.344+0000 7f181aa00640 1 -- 172.21.15.91:0/455052062 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f18240c2390 con 0x7f1804003970 2024-02-02T13:37:35.345 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:35.346 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:35.346 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:35.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.346+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 msgr2=0x7f17fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.346+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 0x7f17fc06c8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f18240b0390 tx=0x7f180c00b040 comp rx=0 tx=0).stop 2024-02-02T13:37:35.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 msgr2=0x7f18240c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f181402c4a0 tx=0x7f1814001930 comp rx=0 tx=0).stop 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 shutdown_connections 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f17fc06a430 0x7f17fc06c8f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1824157f80 0x7f182406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824070420 0x7f18241f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 --2- 172.21.15.91:0/455052062 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1804003970 0x7f18240c86e0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 >> 172.21.15.91:0/455052062 conn(0x7f18240cec80 msgr2=0x7f182415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 shutdown_connections 2024-02-02T13:37:35.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.347+0000 7f1828e00640 1 -- 172.21.15.91:0/455052062 wait complete. 2024-02-02T13:37:35.367 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:37:35.592 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":8835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.851662172000001,"mdsmap_epoch":88,"osdmap_epoch":197,"osdmap_epoch_barrier":197,"uptime":30.181375732999999} 2024-02-02T13:37:35.592 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 8835, "mdsmap_epoch": 88, "osdmap_epoch": 197, "osdmap_epoch_barrier": 197, "rank_uptime": 24.851662172, "state": "up:active", "uptime": 30.181375733, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:37:35.592 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:37:35.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:35.593 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-02-02T13:37:35.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:35.645 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-02-02T13:37:35.723 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:35.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.928+0000 7f547d000640 1 Processor -- start 2024-02-02T13:37:35.930 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.928+0000 7f547d000640 1 -- start start 2024-02-02T13:37:35.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.929+0000 7f547d000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f547815ad10 0x7f547806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f547d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f547d000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547806d910 con 0x7f547815b0f0 2024-02-02T13:37:35.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f547d000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547806d730 con 0x7f547815ad10 2024-02-02T13:37:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f547d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54780cbf50 con 0x7f54780cb580 2024-02-02T13:37:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f547815ad10 0x7f547806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f547815ad10 0x7f547806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54392/0 (socket says 172.21.15.91:54392) 2024-02-02T13:37:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 -- 172.21.15.91:0/1227853116 learned_addr learned my addr 172.21.15.91:0/1227853116 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475400640 1 --1- 172.21.15.91:0/1227853116 >> v1:172.21.15.91:6789/0 conn(0x7f547815b0f0 0x7f547806d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:56328/0 (socket says 172.21.15.91:56328) 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5476800640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 -- 172.21.15.91:0/1227853116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 msgr2=0x7f54780cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5474a00640 1 -- 172.21.15.91:0/1227853116 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804738015 0 0) 0x7f547806d910 con 0x7f547815b0f0 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.930+0000 7f5475e00640 1 -- 172.21.15.91:0/1227853116 >> v1:172.21.15.91:6789/0 conn(0x7f547815b0f0 legacy=0x7f547806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.931+0000 7f5475e00640 1 -- 172.21.15.91:0/1227853116 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54780cc850 con 0x7f547815ad10 2024-02-02T13:37:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.931+0000 7f5476800640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f5475e00640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f547806b7a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f54680098a0 tx=0x7f546802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b8455b40c2422a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f5474a00640 1 -- 172.21.15.91:0/1227853116 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468031640 con 0x7f547815ad10 2024-02-02T13:37:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f5474a00640 1 -- 172.21.15.91:0/1227853116 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54680317a0 con 0x7f547815ad10 2024-02-02T13:37:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f5474a00640 1 -- 172.21.15.91:0/1227853116 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468031b30 con 0x7f547815ad10 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f547d000640 1 -- 172.21.15.91:0/1227853116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 msgr2=0x7f547806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.932+0000 7f547d000640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f547806b7a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f54680098a0 tx=0x7f546802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 -- 172.21.15.91:0/1227853116 shutdown_connections 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 --2- 172.21.15.91:0/1227853116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f547806b7a0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 -- 172.21.15.91:0/1227853116 >> 172.21.15.91:0/1227853116 conn(0x7f54780b0700 msgr2=0x7f54780b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 -- 172.21.15.91:0/1227853116 shutdown_connections 2024-02-02T13:37:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.933+0000 7f547d000640 1 -- 172.21.15.91:0/1227853116 wait complete. 2024-02-02T13:37:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.934+0000 7f547d000640 1 Processor -- start 2024-02-02T13:37:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.935+0000 7f547d000640 1 -- start start 2024-02-02T13:37:35.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.935+0000 7f547d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.935+0000 7f547d000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.935+0000 7f547d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 0x7f54780c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.935+0000 7f547d000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780c8c40 con 0x7f547815ad10 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f547d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780c8960 con 0x7f547815b0f0 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f547d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54780a85f0 con 0x7f54780cb580 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:54398/0 (socket says 172.21.15.91:54398) 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 -- 172.21.15.91:0/3318568538 learned_addr learned my addr 172.21.15.91:0/3318568538 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 -- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 msgr2=0x7f54780ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 -- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 msgr2=0x7f54780c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:35.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5476800640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 0x7f54780c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 0x7f54780c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5475400640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680094d0 con 0x7f547815ad10 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.936+0000 7f5476800640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 0x7f54780c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f5475400640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f546000c7c0 tx=0x7f546000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000a640 con 0x7f547815ad10 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546000a7c0 con 0x7f547815ad10 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54780a88d0 con 0x7f547815ad10 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000ab70 con 0x7f547815ad10 2024-02-02T13:37:35.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.937+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547814e940 con 0x7f547815ad10 2024-02-02T13:37:35.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.938+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54780a8df0 con 0x7f547815ad10 2024-02-02T13:37:35.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.939+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f546000acd0 con 0x7f547815ad10 2024-02-02T13:37:35.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.940+0000 7f546e000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 0x7f545006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:35.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.940+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54600881b0 con 0x7f547815ad10 2024-02-02T13:37:35.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.940+0000 7f5475e00640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 0x7f545006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:35.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.940+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=198}) v3 -- 0x7f5450071f60 con 0x7f547815ad10 2024-02-02T13:37:35.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.941+0000 7f5475e00640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 0x7f545006c8f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5468036000 tx=0x7f5468030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:35.945 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:35.944+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f54780a8df0 con 0x7f547815ad10 2024-02-02T13:37:36.134 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.133+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f54780a8df0 con 0x7f547815ad10 2024-02-02T13:37:36.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.137+0000 7f546e000640 1 -- 172.21.15.91:0/3318568538 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v197) v1 ==== 81+0+33565 (secure 0 0 0) 0x7f54780a8df0 con 0x7f547815ad10 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 197, 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:29.545809+0000", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:36.139 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:36.140 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:36.141 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:36.142 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:36.143 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:36.144 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:36.145 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:36.146 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:36.147 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:36.148 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:36.149 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:36.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:36.151 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:36.152 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1027, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool,full,full_quota", 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:36.153 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:36.154 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "197", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:36.155 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.156 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:36.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:36.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.159 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.160 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:36.163 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.165 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.168 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.169 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:36.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:36.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:36.173 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.174 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.175 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.179 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.180 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.181 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.183 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:36.184 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.185 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.186 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.187 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.188 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.189 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.190 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.191 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.192 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.193 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.194 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.195 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.196 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.197 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.199 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:36.200 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.202 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.203 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.204 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.205 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:36.206 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:36.207 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:36.208 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 msgr2=0x7f545006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 0x7f545006c8f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5468036000 tx=0x7f5468030000 comp rx=0 tx=0).stop 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 msgr2=0x7f54780c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f546000c7c0 tx=0x7f546000cc80 comp rx=0 tx=0).stop 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 shutdown_connections 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f545006a430 0x7f545006c8f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547815b0f0 0x7f54780c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f547815ad10 0x7f54780c7e10 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:36.210 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.139+0000 7f547d000640 1 --2- 172.21.15.91:0/3318568538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54780cb580 0x7f54780ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:36.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.140+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 >> 172.21.15.91:0/3318568538 conn(0x7f54780b0700 msgr2=0x7f54780c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:36.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.140+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 shutdown_connections 2024-02-02T13:37:36.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:36.140+0000 7f547d000640 1 -- 172.21.15.91:0/3318568538 wait complete. 2024-02-02T13:37:36.211 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-02-02T13:37:41.212 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:41.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.435+0000 7fb16fe00640 1 Processor -- start 2024-02-02T13:37:41.436 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.435+0000 7fb16fe00640 1 -- start start 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.436+0000 7fb16fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb1680cbc90 0x7fb16806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.436+0000 7fb16fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16806d910 con 0x7fb16815b0f0 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.436+0000 7fb16fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680cc850 con 0x7fb16815ad10 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.436+0000 7fb16fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16806d730 con 0x7fb1680cbc90 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16ca00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fb16815b0f0 0x7fb1680cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57020/0 (socket says 172.21.15.91:57020) 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16ca00640 1 -- 172.21.15.91:0/409865077 learned_addr learned my addr 172.21.15.91:0/409865077 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:41.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16de00640 1 --2- 172.21.15.91:0/409865077 >> v2:172.21.15.91:3300/0 conn(0x7fb1680cbc90 0x7fb16806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb167e00640 1 -- 172.21.15.91:0/409865077 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441490921 0 0) 0x7fb16806d910 con 0x7fb16815b0f0 2024-02-02T13:37:41.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb167e00640 1 -- 172.21.15.91:0/409865077 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb148003520 con 0x7fb16815b0f0 2024-02-02T13:37:41.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16de00640 1 -- 172.21.15.91:0/409865077 >> v1:172.21.15.113:6789/0 conn(0x7fb16815ad10 legacy=0x7fb16806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:41.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16de00640 1 -- 172.21.15.91:0/409865077 >> v1:172.21.15.91:6789/0 conn(0x7fb16815b0f0 legacy=0x7fb1680cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.439 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.437+0000 7fb16de00640 1 -- 172.21.15.91:0/409865077 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680ccb30 con 0x7fb1680cbc90 2024-02-02T13:37:41.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.438+0000 7fb16de00640 1 --2- 172.21.15.91:0/409865077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb16806b7a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb160009dc0 tx=0x7fb16002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1f72f896f01c56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb167e00640 1 -- 172.21.15.91:0/409865077 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16002cbb0 con 0x7fb1680cbc90 2024-02-02T13:37:41.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb167e00640 1 -- 172.21.15.91:0/409865077 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16002cd10 con 0x7fb1680cbc90 2024-02-02T13:37:41.440 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb167e00640 1 -- 172.21.15.91:0/409865077 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160034660 con 0x7fb1680cbc90 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb16fe00640 1 -- 172.21.15.91:0/409865077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 msgr2=0x7fb16806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb16fe00640 1 --2- 172.21.15.91:0/409865077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb16806b7a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb160009dc0 tx=0x7fb16002be00 comp rx=0 tx=0).stop 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb16fe00640 1 -- 172.21.15.91:0/409865077 shutdown_connections 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb16fe00640 1 --2- 172.21.15.91:0/409865077 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb16806b7a0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.439+0000 7fb16fe00640 1 -- 172.21.15.91:0/409865077 >> 172.21.15.91:0/409865077 conn(0x7fb1680b0700 msgr2=0x7fb1680b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.440+0000 7fb16fe00640 1 -- 172.21.15.91:0/409865077 shutdown_connections 2024-02-02T13:37:41.441 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.440+0000 7fb16fe00640 1 -- 172.21.15.91:0/409865077 wait complete. 2024-02-02T13:37:41.442 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.441+0000 7fb16fe00640 1 Processor -- start 2024-02-02T13:37:41.443 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.441+0000 7fb16fe00640 1 -- start start 2024-02-02T13:37:41.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16815ad10 0x7fb16815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16815b0f0 0x7fb16815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680c2e40 con 0x7fb1680cbc90 2024-02-02T13:37:41.444 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16815d730 con 0x7fb16815b0f0 2024-02-02T13:37:41.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.442+0000 7fb16fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16815da10 con 0x7fb16815ad10 2024-02-02T13:37:41.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33390/0 (socket says 172.21.15.91:33390) 2024-02-02T13:37:41.447 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 -- 172.21.15.91:0/700669581 learned_addr learned my addr 172.21.15.91:0/700669581 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16ca00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16815ad10 0x7fb16815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 -- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16815ad10 msgr2=0x7fb16815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16815ad10 0x7fb16815cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 -- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16815b0f0 msgr2=0x7fb16815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16815b0f0 0x7fb16815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 -- 172.21.15.91:0/700669581 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009a70 con 0x7fb1680cbc90 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.443+0000 7fb16d400640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c430 tx=0x7fb15800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.444+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158011790 con 0x7fb1680cbc90 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.444+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1580118f0 con 0x7fb1680cbc90 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.444+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158011ce0 con 0x7fb1680cbc90 2024-02-02T13:37:41.449 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.444+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16815dcf0 con 0x7fb1680cbc90 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.444+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16815e2b0 con 0x7fb1680cbc90 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.445+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16815e4d0 con 0x7fb1680cbc90 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.446+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb15801c460 con 0x7fb1680cbc90 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.447+0000 7fb165600640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 0x7fb14006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.447+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb158088910 con 0x7fb1680cbc90 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.447+0000 7fb16ca00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 0x7fb14006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.450 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.448+0000 7fb16ca00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 0x7fb14006c8f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb16814e1f0 tx=0x7fb150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.453 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.452+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb158053e60 con 0x7fb1680cbc90 2024-02-02T13:37:41.642 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.640+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb168066cf0 con 0x7fb1680cbc90 2024-02-02T13:37:41.644 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.643+0000 7fb165600640 1 -- 172.21.15.91:0/700669581 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+33546 (secure 0 0 0) 0x7fb158057480 con 0x7fb1680cbc90 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 198, 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:39.550244+0000", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:41.645 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:41.646 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:41.647 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.648 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:41.649 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:41.650 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:41.651 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.652 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:41.653 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:41.654 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:41.655 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:41.656 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:41.657 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:41.658 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:41.659 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "198", 2024-02-02T13:37:41.660 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:41.661 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:41.662 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.663 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.664 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.665 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.666 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.667 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.668 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.669 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.670 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.671 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.672 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:41.673 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.674 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:41.675 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.676 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.677 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.678 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.679 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.680 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.681 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.682 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:41.683 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.684 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.685 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.686 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.687 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.688 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:41.689 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:41.690 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.691 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.692 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.693 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.694 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.695 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.696 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.697 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.698 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.699 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.701 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:41.702 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:41.703 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:41.704 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.705 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.706 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.707 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.708 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.709 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:41.710 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 msgr2=0x7fb14006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 0x7fb14006c8f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb16814e1f0 tx=0x7fb150009040 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 msgr2=0x7fb1680c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c430 tx=0x7fb15800c8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 shutdown_connections 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb14006a430 0x7fb14006c8f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16815ad10 0x7fb16815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16815b0f0 0x7fb16815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 --2- 172.21.15.91:0/700669581 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb1680cbc90 0x7fb1680c2900 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.645+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 >> 172.21.15.91:0/700669581 conn(0x7fb1680b0700 msgr2=0x7fb16806d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.646+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 shutdown_connections 2024-02-02T13:37:41.712 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.646+0000 7fb16fe00640 1 -- 172.21.15.91:0/700669581 wait complete. 2024-02-02T13:37:41.713 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-02-02T13:37:41.713 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-02-02T13:37:41.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.907+0000 7ff487200640 1 Processor -- start 2024-02-02T13:37:41.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.907+0000 7ff487200640 1 -- start start 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.908+0000 7ff487200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff48815dfb0 0x7ff4880c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff487200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48806faa0 con 0x7ff4880ce4d0 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff487200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880ca420 con 0x7ff4880c9a80 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff487200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48815e800 con 0x7ff48815dfb0 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff485e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff4880ce4d0 0x7ff4880707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57024/0 (socket says 172.21.15.91:57024) 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff485e00640 1 -- 172.21.15.91:0/3161572214 learned_addr learned my addr 172.21.15.91:0/3161572214 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:41.910 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff486800640 1 --2- 172.21.15.91:0/3161572214 >> v2:172.21.15.91:3300/0 conn(0x7ff48815dfb0 0x7ff4880c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff485400640 1 -- 172.21.15.91:0/3161572214 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596429447 0 0) 0x7ff48806faa0 con 0x7ff4880ce4d0 2024-02-02T13:37:41.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff485400640 1 -- 172.21.15.91:0/3161572214 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff468003520 con 0x7ff4880ce4d0 2024-02-02T13:37:41.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff486800640 1 -- 172.21.15.91:0/3161572214 >> v1:172.21.15.113:6789/0 conn(0x7ff4880c9a80 legacy=0x7ff48815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:41.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff486800640 1 -- 172.21.15.91:0/3161572214 >> v1:172.21.15.91:6789/0 conn(0x7ff4880ce4d0 legacy=0x7ff4880707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.911 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.909+0000 7ff486800640 1 -- 172.21.15.91:0/3161572214 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488070320 con 0x7ff48815dfb0 2024-02-02T13:37:41.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.910+0000 7ff486800640 1 --2- 172.21.15.91:0/3161572214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff4880c9ee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff48002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4730c372881d6638 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff485400640 1 -- 172.21.15.91:0/3161572214 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48002cc60 con 0x7ff48815dfb0 2024-02-02T13:37:41.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff485400640 1 -- 172.21.15.91:0/3161572214 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48002cdc0 con 0x7ff48815dfb0 2024-02-02T13:37:41.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff485400640 1 -- 172.21.15.91:0/3161572214 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480034660 con 0x7ff48815dfb0 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 -- 172.21.15.91:0/3161572214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 msgr2=0x7ff4880c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 --2- 172.21.15.91:0/3161572214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff4880c9ee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff48002beb0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 -- 172.21.15.91:0/3161572214 shutdown_connections 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 --2- 172.21.15.91:0/3161572214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff4880c9ee0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 -- 172.21.15.91:0/3161572214 >> 172.21.15.91:0/3161572214 conn(0x7ff4880cb4a0 msgr2=0x7ff4880cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:41.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.911+0000 7ff487200640 1 -- 172.21.15.91:0/3161572214 shutdown_connections 2024-02-02T13:37:41.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.912+0000 7ff487200640 1 -- 172.21.15.91:0/3161572214 wait complete. 2024-02-02T13:37:41.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.913+0000 7ff487200640 1 Processor -- start 2024-02-02T13:37:41.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.913+0000 7ff487200640 1 -- start start 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4880c9a80 0x7ff48815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4880ce4d0 0x7ff48806b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488070ec0 con 0x7ff48815dfb0 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488158430 con 0x7ff4880c9a80 2024-02-02T13:37:41.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff487200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48815a910 con 0x7ff4880ce4d0 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff486800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff486800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33398/0 (socket says 172.21.15.91:33398) 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.914+0000 7ff486800640 1 -- 172.21.15.91:0/2342208881 learned_addr learned my addr 172.21.15.91:0/2342208881 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 -- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4880ce4d0 msgr2=0x7ff48806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4880ce4d0 0x7ff48806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 -- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4880c9a80 msgr2=0x7ff48815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4880c9a80 0x7ff48815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 -- 172.21.15.91:0/2342208881 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009a70 con 0x7ff48815dfb0 2024-02-02T13:37:41.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff486800640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480040650 con 0x7ff48815dfb0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4800407d0 con 0x7ff48815dfb0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.915+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480040b80 con 0x7ff48815dfb0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.916+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4881538d0 con 0x7ff48815dfb0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.916+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488158c00 con 0x7ff48815dfb0 2024-02-02T13:37:41.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.917+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488153a60 con 0x7ff48815dfb0 2024-02-02T13:37:41.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.918+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff480040ce0 con 0x7ff48815dfb0 2024-02-02T13:37:41.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.918+0000 7ff47e000640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 0x7ff46006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:41.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.919+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4800ab790 con 0x7ff48815dfb0 2024-02-02T13:37:41.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.919+0000 7ff485e00640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 0x7ff46006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:41.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.919+0000 7ff485e00640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 0x7ff46006c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff488155c80 tx=0x7ff478007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:41.924 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:41.923+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff480076ce0 con 0x7ff48815dfb0 2024-02-02T13:37:42.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.115+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff48804dff0 con 0x7ff48815dfb0 2024-02-02T13:37:42.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.116+0000 7ff47e000640 1 -- 172.21.15.91:0/2342208881 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+33546 (secure 0 0 0) 0x7ff480009450 con 0x7ff48815dfb0 2024-02-02T13:37:42.118 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:42.118 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:37:42.118 INFO:teuthology.orchestra.run.smithi091.stdout: "epoch": 198, 2024-02-02T13:37:42.118 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "cc006970-f900-4781-8947-81ed22cd959b", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "created": "2024-02-02T13:24:19.251864+0000", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "modified": "2024-02-02T13:37:39.550244+0000", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "last_up_change": "2024-02-02T13:24:25.741545+0000", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "last_in_change": "2024-02-02T13:24:22.715632+0000", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_num": 5799936, 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_set": [ 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "pglog_hardlimit", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "purged_snapdirs", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_deletes", 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "sortbitwise" 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_version": 4, 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "full_ratio": 0.69999998807907104, 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-02-02T13:37:42.119 INFO:teuthology.orchestra.run.smithi091.stdout: "nearfull_ratio": 0.60000002384185791, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_snapshot": "", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_max": 23, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "max_osd": 8, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "require_min_compat_client": "luminous", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "min_compat_client": "jewel", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "require_osd_release": "reef", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_crimson": false, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pools": [ 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 1, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": ".mgr", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:24:26.310552+0000", 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:42.120 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 1, 2024-02-02T13:37:42.121 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "116", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:42.122 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:42.123 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:42.124 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_max": 32, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 1 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr": {} 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 8, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 8, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 0.25, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:42.125 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 22, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_metadata", 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:08.724577+0000", 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:42.126 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 32, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 32, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 32, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 32, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 32, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "194", 2024-02-02T13:37:42.127 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 0, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:42.128 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:42.129 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_bias": 4, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 16, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "recovery_priority": 5 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": "cephfs" 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.130 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 2, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 2, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 2, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 2, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "pool": 23, 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_name": "cephfs_data", 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "create_time": "2024-02-02T13:37:09.746745+0000", 2024-02-02T13:37:42.131 INFO:teuthology.orchestra.run.smithi091.stdout: "flags": 1, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "flags_names": "hashpspool", 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "type": 1, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 2, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "min_size": 1, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_rule": 0, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_count": 0, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_target": 0, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_barrier": 0, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "object_hash": 2, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_autoscale_mode": "off", 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num": 64, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num": 64, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_placement_num_target": 64, 2024-02-02T13:37:42.132 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_target": 64, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_pending": 64, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_pg_merge_meta": { 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "source_pgid": "0.0", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "ready_epoch": 0, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_started": 0, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_epoch_clean": 0, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "source_version": "0'0", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "target_version": "0'0" 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_change": "198", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend": "0", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "last_force_op_resend_preluminous": "0", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "auid": 0, 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_mode": "selfmanaged", 2024-02-02T13:37:42.133 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_seq": 0, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "snap_epoch": 0, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "pool_snaps": [], 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps": "[]", 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_bytes": 33554432, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "quota_max_objects": 0, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "tiers": [], 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "tier_of": -1, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "read_tier": -1, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "write_tier": -1, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_mode": "none", 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_bytes": 0, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_objects": 0, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-02T13:37:42.134 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_flush_age": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "cache_min_evict_age": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profile": "", 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_params": { 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "none" 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_period": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_count": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "use_gmt_hitset": true, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "min_read_recency_for_promote": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "min_write_recency_for_promote": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_grade_decay_rate": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "hit_set_search_last_n": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "grade_table": [], 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "stripe_width": 0, 2024-02-02T13:37:42.135 INFO:teuthology.orchestra.run.smithi091.stdout: "expected_num_objects": 0, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "fast_read": false, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "options": { 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_num_min": 64, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "target_size_ratio": 0.89999997615814209 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "application_metadata": { 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "cephfs": { 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "data": "cephfs" 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "read_balance": { 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "score_acting": 1.3799999952316284, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "score_stable": 1.3799999952316284, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "optimal_score": 1, 2024-02-02T13:37:42.136 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_acting": 1.3799999952316284, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "raw_score_stable": 1.3799999952316284, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity_weighted": 1, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity": 1, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "average_primary_affinity_weighted": 1 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "osds": [ 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.137 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6820", 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.138 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.139 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6822", 2024-02-02T13:37:42.139 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.139 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.139 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.139 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6824", 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6825", 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.140 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6830", 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6831", 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.141 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6827", 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6829", 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32877 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6822/32877", 2024-02-02T13:37:42.142 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6825/32877", 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6831/32877", 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6829/32877", 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.143 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6805", 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.144 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6807", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6808", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6809", 2024-02-02T13:37:42.145 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6814", 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6815", 2024-02-02T13:37:42.146 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6811", 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6812", 2024-02-02T13:37:42.147 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32878 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6807/32878", 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6809/32878", 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6815/32878", 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6812/32878", 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:42.148 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6816", 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6817", 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6818", 2024-02-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6819", 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6826", 2024-02-02T13:37:42.151 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6828", 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6821", 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6823", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32875 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6817/32875", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6819/32875", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6828/32875", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6823/32875", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6800", 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6801", 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6802", 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6803", 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.156 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6810", 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6813", 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.157 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6804", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.91:6806", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 32876 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.91:6801/32876", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.91:6803/32876", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.91:6813/32876", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.91:6806/32876", 2024-02-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.159 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6823", 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6825", 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.160 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6826", 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6827", 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.161 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6830", 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6831", 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.162 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6828", 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6829", 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33080 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6825/33080", 2024-02-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6827/33080", 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6831/33080", 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6829/33080", 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6800", 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.165 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6801", 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6802", 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6803", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6807", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6810", 2024-02-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6804", 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6805", 2024-02-02T13:37:42.168 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33078 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6801/33078", 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6803/33078", 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6810/33078", 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6805/33078", 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.169 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.170 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6816", 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6817", 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.171 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6818", 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6819", 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6822", 2024-02-02T13:37:42.172 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6824", 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.173 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6820", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6821", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33081 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6817/33081", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6819/33081", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6824/33081", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6821/33081", 2024-02-02T13:37:42.174 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "up": 1, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "in": 1, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "weight": 1, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_affinity": 1, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_begin": 0, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "last_clean_end": 0, 2024-02-02T13:37:42.175 INFO:teuthology.orchestra.run.smithi091.stdout: "up_from": 13, 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "up_thru": 193, 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "down_at": 0, 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "lost_at": 0, 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrs": { 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6806", 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6808", 2024-02-02T13:37:42.176 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addrs": { 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6809", 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6811", 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.177 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addrs": { 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6814", 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6815", 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.178 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addrs": { 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "addrvec": [ 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v2", 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6812", 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "172.21.15.113:6813", 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 33079 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.179 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "172.21.15.113:6808/33079", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "172.21.15.113:6811/33079", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_back_addr": "172.21.15.113:6815/33079", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_front_addr": "172.21.15.113:6813/33079", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "state": [ 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "exists", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "up" 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: ] 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_xinfo": [ 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 0, 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.180 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.034379+0000", 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 1, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.947533+0000", 2024-02-02T13:37:42.181 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 2, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.001359+0000", 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 3, 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.182 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.976014+0000", 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 4, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:24.046851+0000", 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.183 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 5, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.935345+0000", 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 6, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.184 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.997482+0000", 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: { 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "osd": 7, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "down_stamp": "0.000000", 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_probability": 0, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "laggy_interval": 0, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "features": 4540138322906710015, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "old_weight": 0, 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "last_purged_snaps_scrub": "2024-02-02T13:24:23.928059+0000", 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: "dead_epoch": 0 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.185 INFO:teuthology.orchestra.run.smithi091.stdout: ], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap": [], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_items": [], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_upmap_primaries": [], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "pg_temp": [], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "primary_temp": [], 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklist": {}, 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "range_blocklist": {}, 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_profiles": { 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "default": { 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "crush-failure-domain": "osd", 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "k": "2", 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "m": "1", 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin": "jerasure", 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: "technique": "reed_sol_van" 2024-02-02T13:37:42.186 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "removed_snaps_queue": [], 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "new_removed_snaps": [], 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "new_purged_snaps": [], 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_node_flags": {}, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "device_class_flags": {}, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode": { 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_enabled": false, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_bucket_count": 0, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "degraded_stretch_mode": 0, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "recovering_stretch_mode": 0, 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: "stretch_mode_bucket": 0 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout: } 2024-02-02T13:37:42.187 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 msgr2=0x7ff46006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 0x7ff46006c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff488155c80 tx=0x7ff478007930 comp rx=0 tx=0).stop 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 msgr2=0x7ff48815bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff480004a40 comp rx=0 tx=0).stop 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 shutdown_connections 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff46006a430 0x7ff46006c8f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4880ce4d0 0x7ff48806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff48815dfb0 0x7ff48815bd10 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 --2- 172.21.15.91:0/2342208881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4880c9a80 0x7ff48815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.119+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 >> 172.21.15.91:0/2342208881 conn(0x7ff4880cb4a0 msgr2=0x7ff4880707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.120+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 shutdown_connections 2024-02-02T13:37:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.120+0000 7ff487200640 1 -- 172.21.15.91:0/2342208881 wait complete. 2024-02-02T13:37:42.190 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:42.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.395+0000 7f77b1000640 1 Processor -- start 2024-02-02T13:37:42.397 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.395+0000 7f77b1000640 1 -- start start 2024-02-02T13:37:42.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.396+0000 7f77b1000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f77ac0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.396+0000 7f77b1000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac070f80 con 0x7f77ac151f90 2024-02-02T13:37:42.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77b1000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac06c600 con 0x7f77ac15f5a0 2024-02-02T13:37:42.398 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77b1000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15fa30 con 0x7f77ac0b0e10 2024-02-02T13:37:42.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77a9e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f77ac151f90 0x7f77ac06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57030/0 (socket says 172.21.15.91:57030) 2024-02-02T13:37:42.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77a9e00640 1 -- 172.21.15.91:0/3770732710 learned_addr learned my addr 172.21.15.91:0/3770732710 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:42.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77aa800640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f77ac0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318047733 0 0) 0x7f77ac070f80 con 0x7f77ac151f90 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.397+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778c003520 con 0x7f77ac151f90 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611383059 0 0) 0x7f778c003520 con 0x7f77ac151f90 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 msgr2=0x7f77ac0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f77ac0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 >> v1:172.21.15.113:6789/0 conn(0x7f77ac15f5a0 legacy=0x7f77ac0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac152370 con 0x7f77ac151f90 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f779c001670 con 0x7f77ac151f90 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 >> v1:172.21.15.91:6789/0 conn(0x7f77ac151f90 legacy=0x7f77ac06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f778c0039a0 0x7f778c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f778c0047c0 0x7f778c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f778c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15fa30 con 0x7f778c0047c0 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c00ba90 con 0x7f778c0039a0 2024-02-02T13:37:42.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c00bc50 con 0x7f77ac0b0e10 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.398+0000 7f77a9400640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f778c0047c0 0x7f778c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9e00640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f778c0039a0 0x7f778c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77aa800640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f778c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9400640 1 -- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 msgr2=0x7f778c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9400640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77ac0b0e10 0x7f778c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9400640 1 -- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f778c0039a0 msgr2=0x7f778c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9400640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f778c0039a0 0x7f778c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.399+0000 7f77a9400640 1 -- 172.21.15.91:0/3770732710 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c003520 con 0x7f778c0047c0 2024-02-02T13:37:42.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77a9400640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f778c0047c0 0x7f778c004c40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7794001720 tx=0x7f778c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75789820aab69d48 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:42.403 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940052d0 con 0x7f778c0047c0 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779402e050 con 0x7f778c0047c0 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77a8a00640 1 -- 172.21.15.91:0/3770732710 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940035f0 con 0x7f778c0047c0 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77b1000640 1 -- 172.21.15.91:0/3770732710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f778c0047c0 msgr2=0x7f778c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77b1000640 1 --2- 172.21.15.91:0/3770732710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f778c0047c0 0x7f778c004c40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7794001720 tx=0x7f778c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77aa800640 1 -- 172.21.15.91:0/3770732710 reap_dead start 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.400+0000 7f77b1000640 1 -- 172.21.15.91:0/3770732710 shutdown_connections 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.401+0000 7f77b1000640 1 -- 172.21.15.91:0/3770732710 >> 172.21.15.91:0/3770732710 conn(0x7f77ac0cec80 msgr2=0x7f77ac0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.401+0000 7f77b1000640 1 -- 172.21.15.91:0/3770732710 shutdown_connections 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.401+0000 7f77b1000640 1 -- 172.21.15.91:0/3770732710 wait complete. 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.402+0000 7f77b1000640 1 Processor -- start 2024-02-02T13:37:42.404 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.402+0000 7f77b1000640 1 -- start start 2024-02-02T13:37:42.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f778c0047c0 0x7f77ac150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77ac0b19f0 0x7f77ac1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0c4840 con 0x7f77ac0c5250 2024-02-02T13:37:42.406 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0b4240 con 0x7f77ac0b19f0 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77b1000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0b4520 con 0x7f778c0047c0 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77aa800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.403+0000 7f77aa800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33416/0 (socket says 172.21.15.91:33416) 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 -- 172.21.15.91:0/2541840947 learned_addr learned my addr 172.21.15.91:0/2541840947 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 -- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f778c0047c0 msgr2=0x7f77ac150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f778c0047c0 0x7f77ac150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 -- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77ac0b19f0 msgr2=0x7f77ac1f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77a9400640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77ac0b19f0 0x7f77ac1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77ac0b19f0 0x7f77ac1f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 -- 172.21.15.91:0/2541840947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0c4a50 con 0x7f77ac0c5250 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77aa800640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0cec30 tx=0x7f77a400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4017400 con 0x7f77ac0c5250 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4011040 con 0x7f77ac0c5250 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40177c0 con 0x7f77ac0c5250 2024-02-02T13:37:42.407 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.404+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ac0c4d30 con 0x7f77ac0c5250 2024-02-02T13:37:42.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.405+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac0b3e70 con 0x7f77ac0c5250 2024-02-02T13:37:42.408 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.406+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac0b46e0 con 0x7f77ac0c5250 2024-02-02T13:37:42.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.406+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77a40203b0 con 0x7f77ac0c5250 2024-02-02T13:37:42.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.407+0000 7f77a2000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 0x7f778406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:42.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.407+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77a4088c10 con 0x7f77ac0c5250 2024-02-02T13:37:42.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.407+0000 7f77a9e00640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 0x7f778406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:42.409 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.408+0000 7f77a9e00640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 0x7f778406c8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f779c0017a0 tx=0x7f779c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:42.413 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.412+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77a4054270 con 0x7f77ac0c5250 2024-02-02T13:37:42.622 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.621+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77ac0c4f20 con 0x7f77ac0c5250 2024-02-02T13:37:42.623 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.622+0000 7f77a2000640 1 -- 172.21.15.91:0/2541840947 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f77a4057890 con 0x7f77ac0c5250 2024-02-02T13:37:42.623 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:42.623 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.624+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 msgr2=0x7f778406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.624+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 0x7f778406c8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f779c0017a0 tx=0x7f779c009040 comp rx=0 tx=0).stop 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.624+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 msgr2=0x7f77ac15a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.624+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0cec30 tx=0x7f77a400f4a0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 shutdown_connections 2024-02-02T13:37:42.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f778406a430 0x7f778406c8f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77ac0c5250 0x7f77ac15a890 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77ac0b19f0 0x7f77ac1f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 --2- 172.21.15.91:0/2541840947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f778c0047c0 0x7f77ac150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 >> 172.21.15.91:0/2541840947 conn(0x7f77ac0cec80 msgr2=0x7f77ac0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 shutdown_connections 2024-02-02T13:37:42.627 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:42.625+0000 7f77b1000640 1 -- 172.21.15.91:0/2541840947 wait complete. 2024-02-02T13:37:42.643 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-02-02T13:37:42.859 INFO:teuthology.orchestra.run.smithi091.stdout:{"cluster_fsid":"cc006970-f900-4781-8947-81ed22cd959b","whoami":0,"id":8835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.119772943000001,"mdsmap_epoch":88,"osdmap_epoch":198,"osdmap_epoch_barrier":197,"uptime":37.449539268999999} 2024-02-02T13:37:42.860 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "cc006970-f900-4781-8947-81ed22cd959b", "fs_name": "cephfs", "id": 8835, "mdsmap_epoch": 88, "osdmap_epoch": 198, "osdmap_epoch_barrier": 197, "rank_uptime": 32.119772943, "state": "up:active", "uptime": 37.449539269, "want_state": "up:active", "whoami": 0 } 2024-02-02T13:37:42.860 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-02-02T13:37:42.860 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:42.860 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:42.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:42.910 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-02-02T13:37:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records in 2024-02-02T13:37:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:50+0 records out 2024-02-02T13:37:43.362 INFO:teuthology.orchestra.run.smithi091.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.400102 s, 131 MB/s 2024-02-02T13:37:43.364 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-02T13:37:43.364 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:37:43.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-02T13:37:43.581 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-02T13:37:43.582 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:43.582 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:43.640 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:43.640 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:43.692 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:43.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.904+0000 7fa1b3400640 1 Processor -- start 2024-02-02T13:37:43.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.904+0000 7fa1b3400640 1 -- start start 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1b3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1ac160a20 0x7fa1ac0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1b3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa1ac0b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1b3400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac0b7730 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1b3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b85e0 con 0x7fa1ac160a20 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1b3400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b9140 con 0x7fa1ac0b5010 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1abe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fa1ac0b4ba0 0x7fa1ac0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57044/0 (socket says 172.21.15.91:57044) 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.906+0000 7fa1abe00640 1 -- 172.21.15.91:0/926217282 learned_addr learned my addr 172.21.15.91:0/926217282 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1b0a00640 1 --2- 172.21.15.91:0/926217282 >> v2:172.21.15.113:3300/0 conn(0x7fa1ac160a20 0x7fa1ac0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1b1400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa1ac0b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283457852 0 0) 0x7fa1ac0b7730 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.909 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18c003520 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400107888 0 0) 0x7fa18c003520 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 msgr2=0x7fa1ac0b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa1ac0b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 >> v2:172.21.15.113:3300/0 conn(0x7fa1ac160a20 msgr2=0x7fa1ac0b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 --2- 172.21.15.91:0/926217282 >> v2:172.21.15.113:3300/0 conn(0x7fa1ac160a20 0x7fa1ac0b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac0b7960 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fa194001670 con 0x7fa1ac0b4ba0 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 >> v1:172.21.15.91:6789/0 conn(0x7fa1ac0b4ba0 legacy=0x7fa1ac0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa18c0039a0 0x7fa18c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa18c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa18c004d00 0x7fa18c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.912 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b85e0 con 0x7fa18c004d00 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b9140 con 0x7fa18c0039a0 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.907+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00b980 con 0x7fa1ac0b5010 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa18c004d00 0x7fa18c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b0a00640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa18c0039a0 0x7fa18c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1abe00640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa18c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 -- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 msgr2=0x7fa18c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1ac0b5010 0x7fa18c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 -- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa18c0039a0 msgr2=0x7fa18c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa18c0039a0 0x7fa18c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.908+0000 7fa1b1400640 1 -- 172.21.15.91:0/926217282 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c003520 con 0x7fa18c004d00 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.909+0000 7fa1b1400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa18c004d00 0x7fa18c005180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400e170 tx=0x7fa18c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49531ef53adfa3a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.909+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a402cbf0 con 0x7fa18c004d00 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a400f040 con 0x7fa18c004d00 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1ab400640 1 -- 172.21.15.91:0/926217282 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4034440 con 0x7fa18c004d00 2024-02-02T13:37:43.913 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 -- 172.21.15.91:0/926217282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa18c004d00 msgr2=0x7fa18c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 --2- 172.21.15.91:0/926217282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa18c004d00 0x7fa18c005180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400e170 tx=0x7fa18c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b1400640 1 -- 172.21.15.91:0/926217282 reap_dead start 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 -- 172.21.15.91:0/926217282 shutdown_connections 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 -- 172.21.15.91:0/926217282 >> 172.21.15.91:0/926217282 conn(0x7fa1ac070480 msgr2=0x7fa1ac070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 -- 172.21.15.91:0/926217282 shutdown_connections 2024-02-02T13:37:43.914 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.910+0000 7fa1b3400640 1 -- 172.21.15.91:0/926217282 wait complete. 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.912+0000 7fa1b3400640 1 Processor -- start 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.912+0000 7fa1b3400640 1 -- start start 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa18c004d00 0x7fa1ac0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ac0d1560 0x7fa1ac0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.915 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0d1f00 con 0x7fa1ac0d3250 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0ca1f0 con 0x7fa1ac0d1560 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.913+0000 7fa1b3400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0c8380 con 0x7fa18c004d00 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33426/0 (socket says 172.21.15.91:33426) 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 -- 172.21.15.91:0/2121220044 learned_addr learned my addr 172.21.15.91:0/2121220044 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1b1400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ac0d1560 0x7fa1ac0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 -- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa18c004d00 msgr2=0x7fa1ac0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa18c004d00 0x7fa1ac0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 -- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ac0d1560 msgr2=0x7fa1ac0d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ac0d1560 0x7fa1ac0d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 -- 172.21.15.91:0/2121220044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac0c8660 con 0x7fa1ac0d3250 2024-02-02T13:37:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1abe00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa19400d8b0 tx=0x7fa19400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.914+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194007560 con 0x7fa1ac0d3250 2024-02-02T13:37:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.915+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1940047e0 con 0x7fa1ac0d3250 2024-02-02T13:37:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.915+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194004b70 con 0x7fa1ac0d3250 2024-02-02T13:37:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.915+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac0c8940 con 0x7fa1ac0d3250 2024-02-02T13:37:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.915+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac0d36b0 con 0x7fa1ac0d3250 2024-02-02T13:37:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.916+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac0b5420 con 0x7fa1ac0d3250 2024-02-02T13:37:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.917+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa194004cd0 con 0x7fa1ac0d3250 2024-02-02T13:37:43.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.917+0000 7fa1a8c00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:43.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.918+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa194088390 con 0x7fa1ac0d3250 2024-02-02T13:37:43.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.918+0000 7fa1b0a00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:43.919 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.918+0000 7fa1b0a00640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 0x7fa18406c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009910 tx=0x7fa19c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:43.923 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:43.922+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa194053970 con 0x7fa1ac0d3250 2024-02-02T13:37:44.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.110+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1ac066cf0 con 0x7fa1ac0d3250 2024-02-02T13:37:44.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.111+0000 7fa1a8c00640 1 -- 172.21.15.91:0/2121220044 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16111 (secure 0 0 0) 0x7fa194056f90 con 0x7fa1ac0d3250 2024-02-02T13:37:44.113 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":198,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:39.550244+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 msgr2=0x7fa18406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 0x7fa18406c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009910 tx=0x7fa19c008040 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 msgr2=0x7fa1ac15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa19400d8b0 tx=0x7fa19400dd70 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 shutdown_connections 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ac0d1560 0x7fa1ac0d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ac0d3250 0x7fa1ac15b6b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 --2- 172.21.15.91:0/2121220044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa18c004d00 0x7fa1ac0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 >> 172.21.15.91:0/2121220044 conn(0x7fa1ac070480 msgr2=0x7fa1ac0b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:44.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 shutdown_connections 2024-02-02T13:37:44.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.114+0000 7fa1b3400640 1 -- 172.21.15.91:0/2121220044 wait complete. 2024-02-02T13:37:44.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:37:08.724577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:37:09.746745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:37:44.135 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:37:44.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.341+0000 7f2098800640 1 Processor -- start 2024-02-02T13:37:44.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.341+0000 7f2098800640 1 -- start start 2024-02-02T13:37:44.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.342+0000 7f2098800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f209015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.342+0000 7f2098800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209006f4f0 0x7f20900b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.342+0000 7f2098800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2090070f80 con 0x7f20900b0c40 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.342+0000 7f2098800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20900b10d0 con 0x7f209006f4f0 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.342+0000 7f2098800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090152370 con 0x7f2090151f90 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209006f4f0 0x7f20900b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f209006f4f0 0x7f20900b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33438/0 (socket says 172.21.15.91:33438) 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 -- 172.21.15.91:0/1820907249 learned_addr learned my addr 172.21.15.91:0/1820907249 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2096800640 1 --1- 172.21.15.91:0/1820907249 >> v1:172.21.15.91:6789/0 conn(0x7f20900b0c40 0x7f20900b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57056/0 (socket says 172.21.15.91:57056) 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095e00640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f209015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 -- 172.21.15.91:0/1820907249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 msgr2=0x7f209015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f209015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 -- 172.21.15.91:0/1820907249 >> v1:172.21.15.91:6789/0 conn(0x7f20900b0c40 legacy=0x7f20900b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2094a00640 1 -- 172.21.15.91:0/1820907249 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15789107 0 0) 0x7f2090070f80 con 0x7f20900b0c40 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095400640 1 -- 172.21.15.91:0/1820907249 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900cb010 con 0x7f209006f4f0 2024-02-02T13:37:44.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.343+0000 7f2095e00640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f209015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:37:44.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.344+0000 7f2095400640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209006f4f0 0x7f20900b0700 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e47d27d217a887f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2094a00640 1 -- 172.21.15.91:0/1820907249 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030840 con 0x7f209006f4f0 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2094a00640 1 -- 172.21.15.91:0/1820907249 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20780309a0 con 0x7f209006f4f0 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2094a00640 1 -- 172.21.15.91:0/1820907249 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030d30 con 0x7f209006f4f0 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 -- 172.21.15.91:0/1820907249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209006f4f0 msgr2=0x7f20900b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209006f4f0 0x7f20900b0700 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207802b980 comp rx=0 tx=0).stop 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 -- 172.21.15.91:0/1820907249 shutdown_connections 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f209006f4f0 0x7f20900b0700 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 --2- 172.21.15.91:0/1820907249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f209015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 -- 172.21.15.91:0/1820907249 >> 172.21.15.91:0/1820907249 conn(0x7f20900cec80 msgr2=0x7f20900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 -- 172.21.15.91:0/1820907249 shutdown_connections 2024-02-02T13:37:44.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.345+0000 7f2098800640 1 -- 172.21.15.91:0/1820907249 wait complete. 2024-02-02T13:37:44.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.347+0000 7f2098800640 1 Processor -- start 2024-02-02T13:37:44.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.347+0000 7f2098800640 1 -- start start 2024-02-02T13:37:44.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209006f4f0 0x7f2090157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f2090152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090156d30 con 0x7f20900b0c40 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090157010 con 0x7f209006f4f0 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2098800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20901572f0 con 0x7f2090151f90 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.348+0000 7f2095400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33446/0 (socket says 172.21.15.91:33446) 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 -- 172.21.15.91:0/3843759474 learned_addr learned my addr 172.21.15.91:0/3843759474 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095e00640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209006f4f0 0x7f2090157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 -- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 msgr2=0x7f2090152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2096800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f2090152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f2090152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 -- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209006f4f0 msgr2=0x7f2090157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209006f4f0 0x7f2090157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.351 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 -- 172.21.15.91:0/3843759474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780094d0 con 0x7f20900b0c40 2024-02-02T13:37:44.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.349+0000 7f2095400640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2078000c00 tx=0x7f20780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.350+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207803e070 con 0x7f20900b0c40 2024-02-02T13:37:44.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.350+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078004030 con 0x7f20900b0c40 2024-02-02T13:37:44.352 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.350+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780043c0 con 0x7f20900b0c40 2024-02-02T13:37:44.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.350+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20901575d0 con 0x7f20900b0c40 2024-02-02T13:37:44.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.350+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20901f05e0 con 0x7f20900b0c40 2024-02-02T13:37:44.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.351+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2090157af0 con 0x7f20900b0c40 2024-02-02T13:37:44.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.352+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2078042070 con 0x7f20900b0c40 2024-02-02T13:37:44.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.353+0000 7f208e000640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 0x7f206806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.353+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20780aa940 con 0x7f20900b0c40 2024-02-02T13:37:44.354 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.353+0000 7f2095e00640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 0x7f206806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.355 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.354+0000 7f2095e00640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 0x7f206806c8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2080006fd0 tx=0x7f2080008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.357+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2090157af0 con 0x7f20900b0c40 2024-02-02T13:37:44.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.545+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2090157af0 con 0x7f20900b0c40 2024-02-02T13:37:44.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.546+0000 7f208e000640 1 -- 172.21.15.91:0/3843759474 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2090157af0 con 0x7f20900b0c40 2024-02-02T13:37:44.547 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:37:44.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 msgr2=0x7f206806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 0x7f206806c8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2080006fd0 tx=0x7f2080008040 comp rx=0 tx=0).stop 2024-02-02T13:37:44.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 msgr2=0x7f2090152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2078000c00 tx=0x7f20780046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.550 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 shutdown_connections 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f206806a430 0x7f206806c8f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090151f90 0x7f2090152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20900b0c40 0x7f2090152570 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 --2- 172.21.15.91:0/3843759474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f209006f4f0 0x7f2090157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.548+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 >> 172.21.15.91:0/3843759474 conn(0x7f20900cec80 msgr2=0x7f20900c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.549+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 shutdown_connections 2024-02-02T13:37:44.551 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.549+0000 7f2098800640 1 -- 172.21.15.91:0/3843759474 wait complete. 2024-02-02T13:37:44.568 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:37:44.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.779+0000 7fa7ac800640 1 Processor -- start 2024-02-02T13:37:44.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.779+0000 7fa7ac800640 1 -- start start 2024-02-02T13:37:44.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.780+0000 7fa7ac800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7a415a010 0x7fa7a40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.780+0000 7fa7ac800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.780+0000 7fa7ac800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a40cc7c0 con 0x7fa7a40cd470 2024-02-02T13:37:44.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.780+0000 7fa7ac800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a40ce930 con 0x7fa7a415a010 2024-02-02T13:37:44.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.780+0000 7fa7ac800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a406c890 con 0x7fa7a40ce130 2024-02-02T13:37:44.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.781+0000 7fa7a9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.781+0000 7fa7a9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7a415a010 0x7fa7a40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.781+0000 7fa7a9400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:37744/0 (socket says 172.21.15.91:37744) 2024-02-02T13:37:44.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.781+0000 7fa7a9400640 1 -- 172.21.15.91:0/1206916335 learned_addr learned my addr 172.21.15.91:0/1206916335 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:44.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.782+0000 7fa7a9e00640 1 -- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 msgr2=0x7fa7a40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:44.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.782+0000 7fa7a9e00640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.782+0000 7fa7a9e00640 1 -- 172.21.15.91:0/1206916335 >> v1:172.21.15.113:6789/0 conn(0x7fa7a40cd470 legacy=0x7fa7a40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.782+0000 7fa7a9e00640 1 -- 172.21.15.91:0/1206916335 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a406cb70 con 0x7fa7a415a010 2024-02-02T13:37:44.784 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.782+0000 7fa7a9400640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:37:44.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.784+0000 7fa7a9e00640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a415a010 0x7fa7a40c9670 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7940097a0 tx=0x7fa79402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da3864fbf00232e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.784+0000 7fa7a8a00640 1 -- 172.21.15.91:0/1206916335 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030710 con 0x7fa7a415a010 2024-02-02T13:37:44.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.784+0000 7fa7a8a00640 1 -- 172.21.15.91:0/1206916335 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79402e050 con 0x7fa7a415a010 2024-02-02T13:37:44.785 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.784+0000 7fa7a8a00640 1 -- 172.21.15.91:0/1206916335 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030a90 con 0x7fa7a415a010 2024-02-02T13:37:44.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 -- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a415a010 msgr2=0x7fa7a40c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a415a010 0x7fa7a40c9670 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7940097a0 tx=0x7fa79402b710 comp rx=0 tx=0).stop 2024-02-02T13:37:44.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 -- 172.21.15.91:0/1206916335 shutdown_connections 2024-02-02T13:37:44.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a40ce130 0x7fa7a40ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 --2- 172.21.15.91:0/1206916335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a415a010 0x7fa7a40c9670 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 -- 172.21.15.91:0/1206916335 >> 172.21.15.91:0/1206916335 conn(0x7fa7a40cec80 msgr2=0x7fa7a40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:44.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 -- 172.21.15.91:0/1206916335 shutdown_connections 2024-02-02T13:37:44.787 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.785+0000 7fa7ac800640 1 -- 172.21.15.91:0/1206916335 wait complete. 2024-02-02T13:37:44.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.787+0000 7fa7ac800640 1 Processor -- start 2024-02-02T13:37:44.788 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.787+0000 7fa7ac800640 1 -- start start 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a415a010 0x7fa7a40c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a40c28d0 0x7fa7a40c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c3a60 con 0x7fa7a40ce130 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c2de0 con 0x7fa7a40c28d0 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7ac800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c30c0 con 0x7fa7a415a010 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33450/0 (socket says 172.21.15.91:33450) 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 -- 172.21.15.91:0/3562488580 learned_addr learned my addr 172.21.15.91:0/3562488580 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 -- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a415a010 msgr2=0x7fa7a40c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a415a010 0x7fa7a40c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7aa800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a40c28d0 0x7fa7a40c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 -- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a40c28d0 msgr2=0x7fa7a40c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a40c28d0 0x7fa7a40c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.788+0000 7fa7a9e00640 1 -- 172.21.15.91:0/3562488580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0094d0 con 0x7fa7a40ce130 2024-02-02T13:37:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7a9e00640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa7940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.792 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79403b8d0 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79403ba70 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa794009400 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79403be20 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.789+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a40c3680 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.790+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4066cf0 con 0x7fa7a40ce130 2024-02-02T13:37:44.793 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.791+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa794039040 con 0x7fa7a40ce130 2024-02-02T13:37:44.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.793+0000 7fa7a2000640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 0x7fa77c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:44.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.793+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7940aa1d0 con 0x7fa7a40ce130 2024-02-02T13:37:44.794 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.793+0000 7fa7a9400640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 0x7fa77c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:44.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.794+0000 7fa7a9400640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 0x7fa77c06c8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa78c006f40 tx=0x7fa78c006e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:44.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.797+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa794075840 con 0x7fa7a40ce130 2024-02-02T13:37:44.972 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.971+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7a40cc9e0 con 0x7fa7a40ce130 2024-02-02T13:37:44.973 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.972+0000 7fa7a2000640 1 -- 172.21.15.91:0/3562488580 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa794078e60 con 0x7fa7a40ce130 2024-02-02T13:37:44.973 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 msgr2=0x7fa77c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 0x7fa77c06c8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa78c006f40 tx=0x7fa78c006e90 comp rx=0 tx=0).stop 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 msgr2=0x7fa7a414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa7940046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 shutdown_connections 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fa77c06a430 0x7fa77c06c8f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7a415a010 0x7fa7a40c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7a40c28d0 0x7fa7a40c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.976 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 --2- 172.21.15.91:0/3562488580 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa7a40ce130 0x7fa7a414bfd0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:44.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.974+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 >> 172.21.15.91:0/3562488580 conn(0x7fa7a40cec80 msgr2=0x7fa7a415ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:44.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.975+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 shutdown_connections 2024-02-02T13:37:44.977 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:44.975+0000 7fa7ac800640 1 -- 172.21.15.91:0/3562488580 wait complete. 2024-02-02T13:37:44.994 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:37:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.205+0000 7f129f800640 1 Processor -- start 2024-02-02T13:37:45.207 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.205+0000 7f129f800640 1 -- start start 2024-02-02T13:37:45.208 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129f800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f1298165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129f800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12980b38a0 con 0x7f12980cd940 2024-02-02T13:37:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129f800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298164fe0 con 0x7f12980cb580 2024-02-02T13:37:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129f800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980cf210 con 0x7f12981654d0 2024-02-02T13:37:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129ce00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f12980cd940 0x7f12980cdd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57070/0 (socket says 172.21.15.91:57070) 2024-02-02T13:37:45.209 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.207+0000 7f129ce00640 1 -- 172.21.15.91:0/1029067982 learned_addr learned my addr 172.21.15.91:0/1029067982 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110639186 0 0) 0x7f12980b38a0 con 0x7f12980cd940 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1278003520 con 0x7f12980cd940 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f129d800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f1298165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022546310 0 0) 0x7f1278003520 con 0x7f12980cd940 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 msgr2=0x7f1298165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f1298165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 >> v1:172.21.15.113:6789/0 conn(0x7f12980cb580 legacy=0x7f12980ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980cba10 con 0x7f12980cd940 2024-02-02T13:37:45.211 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1288001670 con 0x7f12980cd940 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 >> v1:172.21.15.91:6789/0 conn(0x7f12980cd940 legacy=0x7f12980cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12780039a0 0x7f1278003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f12780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f1278005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980cf210 con 0x7f1278004d00 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127800bb40 con 0x7f12780039a0 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127800bd00 con 0x7f12981654d0 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.208+0000 7f129d800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f1278005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129ce00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12780039a0 0x7f1278003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f128fc00640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f12780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129d800640 1 -- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 msgr2=0x7f12780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129d800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12981654d0 0x7f12780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129d800640 1 -- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12780039a0 msgr2=0x7f1278003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129d800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12780039a0 0x7f1278003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.212 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.209+0000 7f129d800640 1 -- 172.21.15.91:0/1029067982 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278003520 con 0x7f1278004d00 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.210+0000 7f129d800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f1278005180 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1290007330 tx=0x7f12780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf720365eef09e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.210+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129002cca0 con 0x7f1278004d00 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.210+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129002ce00 con 0x7f1278004d00 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.210+0000 7f1297e00640 1 -- 172.21.15.91:0/1029067982 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290034660 con 0x7f1278004d00 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 -- 172.21.15.91:0/1029067982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 msgr2=0x7f1278005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 --2- 172.21.15.91:0/1029067982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f1278005180 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1290007330 tx=0x7f12780057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129d800640 1 -- 172.21.15.91:0/1029067982 reap_dead start 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 -- 172.21.15.91:0/1029067982 shutdown_connections 2024-02-02T13:37:45.214 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 -- 172.21.15.91:0/1029067982 >> 172.21.15.91:0/1029067982 conn(0x7f129806d950 msgr2=0x7f129806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:45.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 -- 172.21.15.91:0/1029067982 shutdown_connections 2024-02-02T13:37:45.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.211+0000 7f129f800640 1 -- 172.21.15.91:0/1029067982 wait complete. 2024-02-02T13:37:45.215 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.213+0000 7f129f800640 1 Processor -- start 2024-02-02T13:37:45.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.213+0000 7f129f800640 1 -- start start 2024-02-02T13:37:45.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12980cab60 0x7f1298154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.216 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980c98f0 0x7f12980c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980b5600 con 0x7f1278004d00 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298155490 con 0x7f12980c98f0 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129f800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1298155770 con 0x7f12980cab60 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33478/0 (socket says 172.21.15.91:33478) 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 -- 172.21.15.91:0/3314961918 learned_addr learned my addr 172.21.15.91:0/3314961918 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 -- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12980cab60 msgr2=0x7f1298154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12980cab60 0x7f1298154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 -- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980c98f0 msgr2=0x7f12980c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980c98f0 0x7f12980c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.214+0000 7f129ce00640 1 -- 172.21.15.91:0/3314961918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298155a50 con 0x7f1278004d00 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f129ce00640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f128800d8b0 tx=0x7f128800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288007560 con 0x7f1278004d00 2024-02-02T13:37:45.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880047e0 con 0x7f1278004d00 2024-02-02T13:37:45.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288004b70 con 0x7f1278004d00 2024-02-02T13:37:45.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12980c90f0 con 0x7f1278004d00 2024-02-02T13:37:45.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.215+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12981550d0 con 0x7f1278004d00 2024-02-02T13:37:45.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.216+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12980c9610 con 0x7f1278004d00 2024-02-02T13:37:45.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.217+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1288004cd0 con 0x7f1278004d00 2024-02-02T13:37:45.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.218+0000 7f1295600640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 0x7f127006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.218+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12880882d0 con 0x7f1278004d00 2024-02-02T13:37:45.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.218+0000 7f128fc00640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 0x7f127006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.219+0000 7f128fc00640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 0x7f127006c8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1298162f00 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.223 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.222+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f12980c9610 con 0x7f1278004d00 2024-02-02T13:37:45.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.410+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12980c9610 con 0x7f1278004d00 2024-02-02T13:37:45.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.411+0000 7f1295600640 1 -- 172.21.15.91:0/3314961918 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1288019020 con 0x7f1278004d00 2024-02-02T13:37:45.413 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:37:45.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.413+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 msgr2=0x7f127006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.413+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 0x7f127006c8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1298162f00 tx=0x7f1280009040 comp rx=0 tx=0).stop 2024-02-02T13:37:45.415 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.413+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 msgr2=0x7f12980b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.413+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f128800d8b0 tx=0x7f128800dd70 comp rx=0 tx=0).stop 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 shutdown_connections 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f127006a430 0x7f127006c8f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12980cab60 0x7f1298154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980c98f0 0x7f12980c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 --2- 172.21.15.91:0/3314961918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1278004d00 0x7f12980b50c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 >> 172.21.15.91:0/3314961918 conn(0x7f129806d950 msgr2=0x7f12981635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 shutdown_connections 2024-02-02T13:37:45.416 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.414+0000 7f129f800640 1 -- 172.21.15.91:0/3314961918 wait complete. 2024-02-02T13:37:45.434 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:37:45.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.644+0000 7ff05a400640 1 Processor -- start 2024-02-02T13:37:45.645 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.644+0000 7ff05a400640 1 -- start start 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.645+0000 7ff05a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05415b0f0 0x7ff05406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.645+0000 7ff05a400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff0540cb580 0x7ff0540cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.645+0000 7ff05a400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05406d910 con 0x7ff05415ad10 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.645+0000 7ff05a400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05406d730 con 0x7ff0540cb580 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.645+0000 7ff05a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540cbf50 con 0x7ff05415b0f0 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff05415ad10 0x7ff05406d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57076/0 (socket says 172.21.15.91:57076) 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053400640 1 -- 172.21.15.91:0/610641632 learned_addr learned my addr 172.21.15.91:0/610641632 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:45.647 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053e00640 1 --2- 172.21.15.91:0/610641632 >> v2:172.21.15.91:3300/0 conn(0x7ff0540cb580 0x7ff0540cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff052a00640 1 -- 172.21.15.91:0/610641632 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736271947 0 0) 0x7ff05406d910 con 0x7ff05415ad10 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff04a800640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05415b0f0 0x7ff05406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff052a00640 1 -- 172.21.15.91:0/610641632 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff034003520 con 0x7ff05415ad10 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053e00640 1 -- 172.21.15.91:0/610641632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05415b0f0 msgr2=0x7ff05406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053e00640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05415b0f0 0x7ff05406b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.648 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053e00640 1 -- 172.21.15.91:0/610641632 >> v1:172.21.15.91:6789/0 conn(0x7ff05415ad10 legacy=0x7ff05406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.649 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.646+0000 7ff053e00640 1 -- 172.21.15.91:0/610641632 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540cc850 con 0x7ff0540cb580 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.647+0000 7ff053e00640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0540cb580 0x7ff0540cba10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4589f68133069237 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff052a00640 1 -- 172.21.15.91:0/610641632 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c031830 con 0x7ff0540cb580 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff052a00640 1 -- 172.21.15.91:0/610641632 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c031990 con 0x7ff0540cb580 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff052a00640 1 -- 172.21.15.91:0/610641632 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c031d20 con 0x7ff0540cb580 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 -- 172.21.15.91:0/610641632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0540cb580 msgr2=0x7ff0540cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0540cb580 0x7ff0540cba10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 -- 172.21.15.91:0/610641632 shutdown_connections 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0540cb580 0x7ff0540cba10 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 --2- 172.21.15.91:0/610641632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff05415b0f0 0x7ff05406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 -- 172.21.15.91:0/610641632 >> 172.21.15.91:0/610641632 conn(0x7ff0540b0700 msgr2=0x7ff0540b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 -- 172.21.15.91:0/610641632 shutdown_connections 2024-02-02T13:37:45.650 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.648+0000 7ff05a400640 1 -- 172.21.15.91:0/610641632 wait complete. 2024-02-02T13:37:45.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.650+0000 7ff05a400640 1 Processor -- start 2024-02-02T13:37:45.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.650+0000 7ff05a400640 1 -- start start 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0540cb580 0x7ff054158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05415b0f0 0x7ff0540a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c7600 con 0x7ff05415ad10 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c6020 con 0x7ff05415b0f0 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff05a400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c6300 con 0x7ff0540cb580 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff04a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff04a800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33496/0 (socket says 172.21.15.91:33496) 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.651+0000 7ff04a800640 1 -- 172.21.15.91:0/1201364585 learned_addr learned my addr 172.21.15.91:0/1201364585 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 -- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0540cb580 msgr2=0x7ff054158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0540cb580 0x7ff054158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 -- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05415b0f0 msgr2=0x7ff0540a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05415b0f0 0x7ff0540a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 -- 172.21.15.91:0/1201364585 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c009400 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04a800640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff03c007e40 tx=0x7ff03c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.652+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c014620 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.653+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00ee90 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.653+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0540c65e0 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.653+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0149c0 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.653+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0540c6ba0 con 0x7ff05415ad10 2024-02-02T13:37:45.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.654+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0540c6d30 con 0x7ff05415ad10 2024-02-02T13:37:45.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.655+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff03c014b20 con 0x7ff05415ad10 2024-02-02T13:37:45.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.656+0000 7ff04be00640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 0x7ff02c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:45.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.656+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff03c0891f0 con 0x7ff05415ad10 2024-02-02T13:37:45.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.656+0000 7ff053400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 0x7ff02c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:45.658 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.657+0000 7ff053400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 0x7ff02c06c8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff044001610 tx=0x7ff044007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:45.662 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.661+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff03c054870 con 0x7ff05415ad10 2024-02-02T13:37:45.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.864+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff054066cf0 con 0x7ff05415ad10 2024-02-02T13:37:45.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.864+0000 7ff04be00640 1 -- 172.21.15.91:0/1201364585 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v198) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff03c057e90 con 0x7ff05415ad10 2024-02-02T13:37:45.866 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 msgr2=0x7ff02c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 0x7ff02c06c8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff044001610 tx=0x7ff044007930 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 msgr2=0x7ff0540c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff03c007e40 tx=0x7ff03c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 shutdown_connections 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff02c06a430 0x7ff02c06c8f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff05415b0f0 0x7ff0540a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff05415ad10 0x7ff0540c70c0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 --2- 172.21.15.91:0/1201364585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0540cb580 0x7ff054158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 >> 172.21.15.91:0/1201364585 conn(0x7ff0540b0700 msgr2=0x7ff0540c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 shutdown_connections 2024-02-02T13:37:45.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:45.867+0000 7ff05a400640 1 -- 172.21.15.91:0/1201364585 wait complete. 2024-02-02T13:37:45.884 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T13:37:45.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:45.885 DEBUG:teuthology.orchestra.run.smithi091:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T13:37:46.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:37:46.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:46.015 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T13:37:46.063 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:46.063 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:46.109 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:46.110 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:46.205 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:46.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.411+0000 7ff11f200640 1 Processor -- start 2024-02-02T13:37:46.412 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.411+0000 7ff11f200640 1 -- start start 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.412+0000 7ff11f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff11806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.412+0000 7ff11f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff1180ac8d0 0x7ff1180acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.412+0000 7ff11f200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11815d870 con 0x7ff1180b0700 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.412+0000 7ff11f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180ad270 con 0x7ff1180ac8d0 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.412+0000 7ff11f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11806bce0 con 0x7ff11815d490 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff11d200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff1180b0700 0x7ff1180c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57084/0 (socket says 172.21.15.91:57084) 2024-02-02T13:37:46.414 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff11d200640 1 -- 172.21.15.91:0/810698507 learned_addr learned my addr 172.21.15.91:0/810698507 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108651660 0 0) 0x7ff11815d870 con 0x7ff1180b0700 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f8003520 con 0x7ff1180b0700 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117e00640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff11806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff10f200640 1 --2- 172.21.15.91:0/810698507 >> v2:172.21.15.113:3300/0 conn(0x7ff1180ac8d0 0x7ff1180acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400342294 0 0) 0x7ff0f8003520 con 0x7ff1180b0700 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 msgr2=0x7ff11806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff11806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 >> v2:172.21.15.113:3300/0 conn(0x7ff1180ac8d0 msgr2=0x7ff1180acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 --2- 172.21.15.91:0/810698507 >> v2:172.21.15.113:3300/0 conn(0x7ff1180ac8d0 0x7ff1180acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118161220 con 0x7ff1180b0700 2024-02-02T13:37:46.418 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7ff1100019f0 con 0x7ff1180b0700 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.413+0000 7ff10f200640 1 --2- 172.21.15.91:0/810698507 >> v2:172.21.15.113:3300/0 conn(0x7ff1180ac8d0 0x7ff1180acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 >> v1:172.21.15.91:6789/0 conn(0x7ff1180b0700 legacy=0x7ff1180c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff0f8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8003ef0 0x7ff0f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0f8004d00 0x7ff0f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180ad270 con 0x7ff0f8004d00 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11806bce0 con 0x7ff0f8003ef0 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f800b9d0 con 0x7ff11815d490 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0f8004d00 0x7ff0f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff117e00640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff0f8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff10f200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8003ef0 0x7ff0f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.419 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 -- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 msgr2=0x7ff0f8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11815d490 0x7ff0f8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 -- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8003ef0 msgr2=0x7ff0f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8003ef0 0x7ff0f80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.414+0000 7ff11d200640 1 -- 172.21.15.91:0/810698507 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8003520 con 0x7ff0f8004d00 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.415+0000 7ff11d200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0f8004d00 0x7ff0f8005180 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff11000a860 tx=0x7ff0f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9282626c1210190b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110034630 con 0x7ff0f8004d00 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11002f070 con 0x7ff0f8004d00 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff117400640 1 -- 172.21.15.91:0/810698507 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110034a30 con 0x7ff0f8004d00 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 -- 172.21.15.91:0/810698507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0f8004d00 msgr2=0x7ff0f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 --2- 172.21.15.91:0/810698507 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0f8004d00 0x7ff0f8005180 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff11000a860 tx=0x7ff0f80057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11d200640 1 -- 172.21.15.91:0/810698507 reap_dead start 2024-02-02T13:37:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 -- 172.21.15.91:0/810698507 shutdown_connections 2024-02-02T13:37:46.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 -- 172.21.15.91:0/810698507 >> 172.21.15.91:0/810698507 conn(0x7ff1180cec80 msgr2=0x7ff1180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:46.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 -- 172.21.15.91:0/810698507 shutdown_connections 2024-02-02T13:37:46.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.416+0000 7ff11f200640 1 -- 172.21.15.91:0/810698507 wait complete. 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.418+0000 7ff11f200640 1 Processor -- start 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.418+0000 7ff11f200640 1 -- start start 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8004d00 0x7ff1180b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11806fce0 0x7ff1181f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180b2550 con 0x7ff11806f860 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118143590 con 0x7ff0f8004d00 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff11f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff118142980 con 0x7ff11806fce0 2024-02-02T13:37:46.423 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff10f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.419+0000 7ff10f200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33510/0 (socket says 172.21.15.91:33510) 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 -- 172.21.15.91:0/1684313315 learned_addr learned my addr 172.21.15.91:0/1684313315 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 -- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11806fce0 msgr2=0x7ff1181f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11806fce0 0x7ff1181f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 -- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8004d00 msgr2=0x7ff1180b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8004d00 0x7ff1180b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 -- 172.21.15.91:0/1684313315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118142c60 con 0x7ff11806f860 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff10f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff10000f570 tx=0x7ff10000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100014890 con 0x7ff11806f860 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10000c030 con 0x7ff11806f860 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.420+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100014c30 con 0x7ff11806f860 2024-02-02T13:37:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.421+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff118142f40 con 0x7ff11806f860 2024-02-02T13:37:46.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.421+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181f2830 con 0x7ff11806f860 2024-02-02T13:37:46.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.422+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11806b950 con 0x7ff11806f860 2024-02-02T13:37:46.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.423+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff1000133e0 con 0x7ff11806f860 2024-02-02T13:37:46.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.423+0000 7ff114c00640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 0x7ff0f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.423+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff100088cc0 con 0x7ff11806f860 2024-02-02T13:37:46.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.424+0000 7ff117e00640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 0x7ff0f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.424+0000 7ff117e00640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 0x7ff0f006c8f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff108009910 tx=0x7ff108008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.429 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.428+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff11806b950 con 0x7ff11806f860 2024-02-02T13:37:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.623+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff11806b950 con 0x7ff11806f860 2024-02-02T13:37:46.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.626+0000 7ff114c00640 1 -- 172.21.15.91:0/1684313315 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16111 (secure 0 0 0) 0x7ff11806b950 con 0x7ff11806f860 2024-02-02T13:37:46.627 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:46.627 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":198,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:39.550244+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 msgr2=0x7ff0f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 0x7ff0f006c8f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff108009910 tx=0x7ff108008040 comp rx=0 tx=0).stop 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 msgr2=0x7ff1181f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff10000f570 tx=0x7ff10000fa30 comp rx=0 tx=0).stop 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 shutdown_connections 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff0f006a430 0x7ff0f006c8f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff11806fce0 0x7ff1181f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff11806f860 0x7ff1181f1db0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 --2- 172.21.15.91:0/1684313315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0f8004d00 0x7ff1180b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 >> 172.21.15.91:0/1684313315 conn(0x7ff1180cec80 msgr2=0x7ff1180afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:46.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 shutdown_connections 2024-02-02T13:37:46.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.628+0000 7ff11f200640 1 -- 172.21.15.91:0/1684313315 wait complete. 2024-02-02T13:37:46.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:37:08.724577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:37:09.746745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:37:46.648 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:37:46.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.852+0000 7fc86fe00640 1 Processor -- start 2024-02-02T13:37:46.852 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.852+0000 7fc86fe00640 1 -- start start 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.853+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc8700706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.853+0000 7fc86fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8700c9450 0x7fc8700c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.853+0000 7fc86fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8700ce3b0 con 0x7fc87015d3a0 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.853+0000 7fc86fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8700adbd0 con 0x7fc8700c9450 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.853+0000 7fc86fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700c59d0 con 0x7fc8700add80 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86ea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fc87015d3a0 0x7fc87006b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57090/0 (socket says 172.21.15.91:57090) 2024-02-02T13:37:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86ea00640 1 -- 172.21.15.91:0/2039354451 learned_addr learned my addr 172.21.15.91:0/2039354451 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300551990 0 0) 0x7fc8700ce3b0 con 0x7fc87015d3a0 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc850003520 con 0x7fc87015d3a0 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86f400640 1 --2- 172.21.15.91:0/2039354451 >> v2:172.21.15.113:3300/0 conn(0x7fc8700c9450 0x7fc8700c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc865e00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc8700706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743472196 0 0) 0x7fc850003520 con 0x7fc87015d3a0 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 msgr2=0x7fc8700706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc8700706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 >> v2:172.21.15.113:3300/0 conn(0x7fc8700c9450 msgr2=0x7fc8700c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 --2- 172.21.15.91:0/2039354451 >> v2:172.21.15.113:3300/0 conn(0x7fc8700c9450 0x7fc8700c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.856 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8700ce8e0 con 0x7fc87015d3a0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.854+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fc860001670 con 0x7fc87015d3a0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 >> v1:172.21.15.91:6789/0 conn(0x7fc87015d3a0 legacy=0x7fc87006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc850005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700adbd0 con 0x7fc8500047c0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700c59d0 con 0x7fc850003970 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000bb90 con 0x7fc8700add80 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86f400640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc850005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86ea00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 -- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 msgr2=0x7fc850005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8700add80 0x7fc850005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 -- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc850003970 msgr2=0x7fc850003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.857 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc865e00640 1 -- 172.21.15.91:0/2039354451 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850003520 con 0x7fc8500047c0 2024-02-02T13:37:46.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.855+0000 7fc86ea00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:46.858 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.856+0000 7fc865e00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc85800ba00 tx=0x7fc8500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6640f2cd4b06c763 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858032620 con 0x7fc8500047c0 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858032780 con 0x7fc8500047c0 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86e000640 1 -- 172.21.15.91:0/2039354451 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858032b10 con 0x7fc8500047c0 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 -- 172.21.15.91:0/2039354451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8500047c0 msgr2=0x7fc850004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 --2- 172.21.15.91:0/2039354451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc85800ba00 tx=0x7fc8500057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86f400640 1 -- 172.21.15.91:0/2039354451 reap_dead start 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 -- 172.21.15.91:0/2039354451 shutdown_connections 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 -- 172.21.15.91:0/2039354451 >> 172.21.15.91:0/2039354451 conn(0x7fc8700cea90 msgr2=0x7fc8700ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 -- 172.21.15.91:0/2039354451 shutdown_connections 2024-02-02T13:37:46.859 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.857+0000 7fc86fe00640 1 -- 172.21.15.91:0/2039354451 wait complete. 2024-02-02T13:37:46.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.859+0000 7fc86fe00640 1 Processor -- start 2024-02-02T13:37:46.860 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.859+0000 7fc86fe00640 1 -- start start 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8500047c0 0x7fc8700acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc870149b70 0x7fc8701f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700b02b0 con 0x7fc8700acff0 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700af630 con 0x7fc870149b70 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc86fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700af910 con 0x7fc8500047c0 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc865e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc865e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33520/0 (socket says 172.21.15.91:33520) 2024-02-02T13:37:46.861 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.860+0000 7fc865e00640 1 -- 172.21.15.91:0/3711910129 learned_addr learned my addr 172.21.15.91:0/3711910129 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 -- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8500047c0 msgr2=0x7fc8700acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc86ea00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8500047c0 0x7fc8700acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8500047c0 0x7fc8700acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc86f400640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc870149b70 0x7fc8701f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 -- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc870149b70 msgr2=0x7fc8701f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:46.862 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc870149b70 0x7fc8701f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 -- 172.21.15.91:0/3711910129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8700afbf0 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.861+0000 7fc865e00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc85800bb20 tx=0x7fc858004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.862+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85803b420 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.862+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85803b580 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.862+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8700afed0 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.862+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85803b930 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.862+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc870143950 con 0x7fc8700acff0 2024-02-02T13:37:46.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.863+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87004dff0 con 0x7fc8700acff0 2024-02-02T13:37:46.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.864+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc85803ba90 con 0x7fc8700acff0 2024-02-02T13:37:46.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.865+0000 7fc867400640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 0x7fc84806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:46.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.865+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8580aa640 con 0x7fc8700acff0 2024-02-02T13:37:46.865 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.865+0000 7fc86ea00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 0x7fc84806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:46.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.865+0000 7fc86ea00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 0x7fc84806c8f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc860000ff0 tx=0x7fc860009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:46.869 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:46.869+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc858075c20 con 0x7fc8700acff0 2024-02-02T13:37:47.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.057+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8700b4a10 con 0x7fc8700acff0 2024-02-02T13:37:47.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.058+0000 7fc867400640 1 -- 172.21.15.91:0/3711910129 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc858006c30 con 0x7fc8700acff0 2024-02-02T13:37:47.058 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:37:47.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 msgr2=0x7fc84806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 0x7fc84806c8f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc860000ff0 tx=0x7fc860009040 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 msgr2=0x7fc870149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc85800bb20 tx=0x7fc858004730 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 shutdown_connections 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fc84806a430 0x7fc84806c8f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc870149b70 0x7fc8701f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8700acff0 0x7fc870149630 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 --2- 172.21.15.91:0/3711910129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8500047c0 0x7fc8700acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 >> 172.21.15.91:0/3711910129 conn(0x7fc8700cea90 msgr2=0x7fc8700b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 shutdown_connections 2024-02-02T13:37:47.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.060+0000 7fc86fe00640 1 -- 172.21.15.91:0/3711910129 wait complete. 2024-02-02T13:37:47.077 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:37:47.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.288+0000 7f6c61000640 1 Processor -- start 2024-02-02T13:37:47.289 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.288+0000 7f6c61000640 1 -- start start 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c61000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c5c151f90 0x7f6c5c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c61000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 0x7f6c5c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c61000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c070f80 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c61000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c152370 con 0x7f6c5c151f90 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c61000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0cb010 con 0x7f6c5c0b0e10 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c59400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6c5c06f4f0 0x7f6c5c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57098/0 (socket says 172.21.15.91:57098) 2024-02-02T13:37:47.291 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.290+0000 7f6c59400640 1 -- 172.21.15.91:0/1441843337 learned_addr learned my addr 172.21.15.91:0/1441843337 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c5a800640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 0x7f6c5c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580081101 0 0) 0x7f6c5c070f80 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c59e00640 1 --2- 172.21.15.91:0/1441843337 >> v2:172.21.15.113:3300/0 conn(0x7f6c5c151f90 0x7f6c5c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003520 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417236670 0 0) 0x7f6c3c003520 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 msgr2=0x7f6c5c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 0x7f6c5c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 >> v2:172.21.15.113:3300/0 conn(0x7f6c5c151f90 msgr2=0x7f6c5c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 --2- 172.21.15.91:0/1441843337 >> v2:172.21.15.113:3300/0 conn(0x7f6c5c151f90 0x7f6c5c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c0cb2f0 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f6c44001670 con 0x7f6c5c06f4f0 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 >> v1:172.21.15.91:6789/0 conn(0x7f6c5c06f4f0 legacy=0x7f6c5c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.293 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c3c003970 0x7f6c3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 0x7f6c3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c152370 con 0x7f6c3c0047c0 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0cb010 con 0x7f6c3c003970 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.291+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c00b980 con 0x7f6c5c0b0e10 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 -- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 msgr2=0x7f6c3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0b0e10 0x7f6c3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 -- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c3c003970 msgr2=0x7f6c3c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c3c003970 0x7f6c3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.294 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.292+0000 7f6c59400640 1 -- 172.21.15.91:0/1441843337 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c003520 con 0x7f6c3c0047c0 2024-02-02T13:37:47.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.293+0000 7f6c59400640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c3c004c40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400c240 tx=0x7f6c3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c5124c11567d99 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.295 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.293+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4402cb00 con 0x7f6c3c0047c0 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.293+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4402cc60 con 0x7f6c3c0047c0 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.293+0000 7f6c58a00640 1 -- 172.21.15.91:0/1441843337 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440058d0 con 0x7f6c3c0047c0 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 -- 172.21.15.91:0/1441843337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 msgr2=0x7f6c3c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 --2- 172.21.15.91:0/1441843337 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c3c004c40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400c240 tx=0x7f6c3c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c5a800640 1 -- 172.21.15.91:0/1441843337 reap_dead start 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 -- 172.21.15.91:0/1441843337 shutdown_connections 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 -- 172.21.15.91:0/1441843337 >> 172.21.15.91:0/1441843337 conn(0x7f6c5c0cec80 msgr2=0x7f6c5c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 -- 172.21.15.91:0/1441843337 shutdown_connections 2024-02-02T13:37:47.296 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.294+0000 7f6c61000640 1 -- 172.21.15.91:0/1441843337 wait complete. 2024-02-02T13:37:47.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.296+0000 7f6c61000640 1 Processor -- start 2024-02-02T13:37:47.297 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.296+0000 7f6c61000640 1 -- start start 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0c52d0 0x7f6c5c0c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 0x7f6c5c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0c49f0 con 0x7f6c3c0047c0 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c15caf0 con 0x7f6c5c0c3cb0 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c61000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c1510a0 con 0x7f6c5c0c52d0 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.298 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33544/0 (socket says 172.21.15.91:33544) 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c59e00640 1 -- 172.21.15.91:0/3597067198 learned_addr learned my addr 172.21.15.91:0/3597067198 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.297+0000 7f6c59400640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0c52d0 0x7f6c5c0c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c5a800640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 0x7f6c5c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 -- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0c52d0 msgr2=0x7f6c5c0c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0c52d0 0x7f6c5c0c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 -- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 msgr2=0x7f6c5c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 0x7f6c5c0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 -- 172.21.15.91:0/3597067198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c0c4150 con 0x7f6c3c0047c0 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c5a800640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 0x7f6c5c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:47.299 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c59e00640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e7a0 tx=0x7f6c4c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00eeb0 con 0x7f6c3c0047c0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00c940 con 0x7f6c3c0047c0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00cd10 con 0x7f6c3c0047c0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.298+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c0c4430 con 0x7f6c3c0047c0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.299+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c0b2600 con 0x7f6c3c0047c0 2024-02-02T13:37:47.300 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.299+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c15c950 con 0x7f6c3c0047c0 2024-02-02T13:37:47.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.300+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c4c013470 con 0x7f6c3c0047c0 2024-02-02T13:37:47.301 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.301+0000 7f6c52000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 0x7f6c3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.301+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4c088a50 con 0x7f6c3c0047c0 2024-02-02T13:37:47.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.301+0000 7f6c59400640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 0x7f6c3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.302+0000 7f6c59400640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 0x7f6c3406c8f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6c44034000 tx=0x7f6c44007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.306+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c5c15c950 con 0x7f6c3c0047c0 2024-02-02T13:37:47.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.493+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c5c15cc60 con 0x7f6c3c0047c0 2024-02-02T13:37:47.494 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.494+0000 7f6c52000640 1 -- 172.21.15.91:0/3597067198 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c4c017020 con 0x7f6c3c0047c0 2024-02-02T13:37:47.494 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:37:47.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.496+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 msgr2=0x7f6c3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.496+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 0x7f6c3406c8f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6c44034000 tx=0x7f6c44007000 comp rx=0 tx=0).stop 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.496+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 msgr2=0x7f6c5c150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.496+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e7a0 tx=0x7f6c4c00ec60 comp rx=0 tx=0).stop 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 shutdown_connections 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f6c3406a430 0x7f6c3406c8f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c5c0c52d0 0x7f6c5c0c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c5c0c3cb0 0x7f6c5c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 --2- 172.21.15.91:0/3597067198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c3c0047c0 0x7f6c5c150b60 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 >> 172.21.15.91:0/3597067198 conn(0x7f6c5c0cec80 msgr2=0x7f6c5c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:47.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 shutdown_connections 2024-02-02T13:37:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.497+0000 7f6c61000640 1 -- 172.21.15.91:0/3597067198 wait complete. 2024-02-02T13:37:47.516 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:37:47.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.718+0000 7f3432000640 1 Processor -- start 2024-02-02T13:37:47.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.719+0000 7f3432000640 1 -- start start 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3432000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f342c151f90 0x7f342c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3432000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f342c0b0e10 0x7f342c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3432000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c070f80 con 0x7f342c06f4f0 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3432000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c152370 con 0x7f342c0b0e10 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3432000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0cb010 con 0x7f342c151f90 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342be00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f342c0b0e10 0x7f342c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3423e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f342c06f4f0 0x7f342c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57112/0 (socket says 172.21.15.91:57112) 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f3423e00640 1 -- 172.21.15.91:0/4127619416 learned_addr learned my addr 172.21.15.91:0/4127619416 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342b400640 1 --2- 172.21.15.91:0/4127619416 >> v2:172.21.15.113:3300/0 conn(0x7f342c151f90 0x7f342c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342be00640 1 -- 172.21.15.91:0/4127619416 >> v2:172.21.15.113:3300/0 conn(0x7f342c151f90 msgr2=0x7f342c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342aa00640 1 -- 172.21.15.91:0/4127619416 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338001553 0 0) 0x7f342c070f80 con 0x7f342c06f4f0 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342be00640 1 --2- 172.21.15.91:0/4127619416 >> v2:172.21.15.113:3300/0 conn(0x7f342c151f90 0x7f342c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342be00640 1 -- 172.21.15.91:0/4127619416 >> v1:172.21.15.91:6789/0 conn(0x7f342c06f4f0 legacy=0x7f342c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.722 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.720+0000 7f342be00640 1 -- 172.21.15.91:0/4127619416 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0cb2f0 con 0x7f342c0b0e10 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f342be00640 1 --2- 172.21.15.91:0/4127619416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c0b0e10 0x7f342c0b3af0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f342c0cc770 tx=0x7f342402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a7047eb995a82b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f342aa00640 1 -- 172.21.15.91:0/4127619416 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342402cc80 con 0x7f342c0b0e10 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f342aa00640 1 -- 172.21.15.91:0/4127619416 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342402cde0 con 0x7f342c0b0e10 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f342aa00640 1 -- 172.21.15.91:0/4127619416 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424034660 con 0x7f342c0b0e10 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 -- 172.21.15.91:0/4127619416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c0b0e10 msgr2=0x7f342c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 --2- 172.21.15.91:0/4127619416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c0b0e10 0x7f342c0b3af0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f342c0cc770 tx=0x7f342402bed0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 -- 172.21.15.91:0/4127619416 shutdown_connections 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 --2- 172.21.15.91:0/4127619416 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c0b0e10 0x7f342c0b3af0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 --2- 172.21.15.91:0/4127619416 >> v2:172.21.15.113:3300/0 conn(0x7f342c151f90 0x7f342c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.722+0000 7f3432000640 1 -- 172.21.15.91:0/4127619416 >> 172.21.15.91:0/4127619416 conn(0x7f342c0cec80 msgr2=0x7f342c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:47.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.723+0000 7f3432000640 1 -- 172.21.15.91:0/4127619416 shutdown_connections 2024-02-02T13:37:47.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.723+0000 7f3432000640 1 -- 172.21.15.91:0/4127619416 wait complete. 2024-02-02T13:37:47.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.724+0000 7f3432000640 1 Processor -- start 2024-02-02T13:37:47.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.724+0000 7f3432000640 1 -- start start 2024-02-02T13:37:47.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c0b0e10 0x7f342c0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f342c151f90 0x7f342c1f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1f2db0 con 0x7f342c06f4f0 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0c8240 con 0x7f342c0b0e10 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f3432000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15dc40 con 0x7f342c151f90 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f342b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f342b400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33566/0 (socket says 172.21.15.91:33566) 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.725+0000 7f342b400640 1 -- 172.21.15.91:0/472623193 learned_addr learned my addr 172.21.15.91:0/472623193 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 -- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f342c151f90 msgr2=0x7f342c1f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f342c151f90 0x7f342c1f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 -- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c0b0e10 msgr2=0x7f342c0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c0b0e10 0x7f342c0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 -- 172.21.15.91:0/472623193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009a70 con 0x7f342c06f4f0 2024-02-02T13:37:47.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f342b400640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d830 tx=0x7f341c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00b6c0 con 0x7f342c06f4f0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00b820 con 0x7f342c06f4f0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00bbb0 con 0x7f342c06f4f0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.726+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c0c6110 con 0x7f342c06f4f0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.727+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c0c66d0 con 0x7f342c06f4f0 2024-02-02T13:37:47.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.728+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c066cf0 con 0x7f342c06f4f0 2024-02-02T13:37:47.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.728+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f341c01e460 con 0x7f342c06f4f0 2024-02-02T13:37:47.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.729+0000 7f3423400640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 0x7f340406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:47.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.729+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f341c089020 con 0x7f342c06f4f0 2024-02-02T13:37:47.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.729+0000 7f3423e00640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 0x7f340406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:47.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.730+0000 7f3423e00640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 0x7f340406c8f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3414007fd0 tx=0x7f3414007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:47.734 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.734+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f341c0546a0 con 0x7f342c06f4f0 2024-02-02T13:37:47.920 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.920+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f342c0c24a0 con 0x7f342c06f4f0 2024-02-02T13:37:47.921 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.921+0000 7f3423400640 1 -- 172.21.15.91:0/472623193 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f341c057cc0 con 0x7f342c06f4f0 2024-02-02T13:37:47.921 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 msgr2=0x7f340406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 0x7f340406c8f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3414007fd0 tx=0x7f3414007960 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 msgr2=0x7f342c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d830 tx=0x7f341c00dcf0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 shutdown_connections 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f340406a430 0x7f340406c8f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f342c151f90 0x7f342c1f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c0b0e10 0x7f342c0c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 --2- 172.21.15.91:0/472623193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f342c06f4f0 0x7f342c15d700 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 >> 172.21.15.91:0/472623193 conn(0x7f342c0cec80 msgr2=0x7f342c0b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 shutdown_connections 2024-02-02T13:37:47.926 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:47.924+0000 7f3432000640 1 -- 172.21.15.91:0/472623193 wait complete. 2024-02-02T13:37:47.941 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T13:37:48.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.147+0000 7ff6f0600640 1 Processor -- start 2024-02-02T13:37:48.148 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.148+0000 7ff6f0600640 1 -- start start 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6f0600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e8070270 0x7ff6e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6f0600640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e8160d90 con 0x7ff6e80b6bd0 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6f0600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e8070bf0 con 0x7ff6e81609b0 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6f0600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e8070fe0 con 0x7ff6e8070270 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6ed200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e8070270 0x7ff6e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6ed200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff6e8070270 0x7ff6e80706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33574/0 (socket says 172.21.15.91:33574) 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6ed200640 1 -- 172.21.15.91:0/2131942443 learned_addr learned my addr 172.21.15.91:0/2131942443 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:48.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.149+0000 7ff6ee600640 1 --1- 172.21.15.91:0/2131942443 >> v1:172.21.15.91:6789/0 conn(0x7ff6e80b6bd0 0x7ff6e80b7020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57118/0 (socket says 172.21.15.91:57118) 2024-02-02T13:37:48.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.150+0000 7ff6ed200640 1 -- 172.21.15.91:0/2131942443 >> v1:172.21.15.113:6789/0 conn(0x7ff6e81609b0 legacy=0x7ff6e80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:48.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.150+0000 7ff6e7e00640 1 -- 172.21.15.91:0/2131942443 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490583901 0 0) 0x7ff6e0001110 con 0x7ff6e80b6bd0 2024-02-02T13:37:48.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.150+0000 7ff6ed200640 1 -- 172.21.15.91:0/2131942443 >> v1:172.21.15.91:6789/0 conn(0x7ff6e80b6bd0 legacy=0x7ff6e80b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.150+0000 7ff6ed200640 1 -- 172.21.15.91:0/2131942443 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80d2940 con 0x7ff6e8070270 2024-02-02T13:37:48.151 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.151+0000 7ff6ed200640 1 --2- 172.21.15.91:0/2131942443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e8070270 0x7ff6e80706b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009870 tx=0x7ff6d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8c611a804866f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.151+0000 7ff6e7e00640 1 -- 172.21.15.91:0/2131942443 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0031870 con 0x7ff6e8070270 2024-02-02T13:37:48.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.151+0000 7ff6e7e00640 1 -- 172.21.15.91:0/2131942443 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d00319d0 con 0x7ff6e8070270 2024-02-02T13:37:48.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.151+0000 7ff6e7e00640 1 -- 172.21.15.91:0/2131942443 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0031d60 con 0x7ff6e8070270 2024-02-02T13:37:48.152 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 -- 172.21.15.91:0/2131942443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e8070270 msgr2=0x7ff6e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 --2- 172.21.15.91:0/2131942443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e8070270 0x7ff6e80706b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009870 tx=0x7ff6d002b910 comp rx=0 tx=0).stop 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 -- 172.21.15.91:0/2131942443 shutdown_connections 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 --2- 172.21.15.91:0/2131942443 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e8070270 0x7ff6e80706b0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 -- 172.21.15.91:0/2131942443 >> 172.21.15.91:0/2131942443 conn(0x7ff6e80d3020 msgr2=0x7ff6e80d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 -- 172.21.15.91:0/2131942443 shutdown_connections 2024-02-02T13:37:48.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.152+0000 7ff6f0600640 1 -- 172.21.15.91:0/2131942443 wait complete. 2024-02-02T13:37:48.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.154+0000 7ff6f0600640 1 Processor -- start 2024-02-02T13:37:48.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.154+0000 7ff6f0600640 1 -- start start 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e8070270 0x7ff6e8153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e80b6bd0 0x7ff6e815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8158f70 con 0x7ff6e81609b0 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8153970 con 0x7ff6e8070270 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6f0600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e81585d0 con 0x7ff6e80b6bd0 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6ee600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6ee600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33590/0 (socket says 172.21.15.91:33590) 2024-02-02T13:37:48.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.155+0000 7ff6ee600640 1 -- 172.21.15.91:0/3905030309 learned_addr learned my addr 172.21.15.91:0/3905030309 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ed200640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e80b6bd0 0x7ff6e815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6edc00640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e8070270 0x7ff6e8153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 -- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e80b6bd0 msgr2=0x7ff6e815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e80b6bd0 0x7ff6e815a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 -- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e8070270 msgr2=0x7ff6e8153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e8070270 0x7ff6e8153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 -- 172.21.15.91:0/3905030309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00094d0 con 0x7ff6e81609b0 2024-02-02T13:37:48.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6ee600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c780 tx=0x7ff6e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00072d0 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0007430 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0001d10 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e81588b0 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.156+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e815a820 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.157+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e81590b0 con 0x7ff6e81609b0 2024-02-02T13:37:48.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.158+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff6e00047e0 con 0x7ff6e81609b0 2024-02-02T13:37:48.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.159+0000 7ff6e5600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 0x7ff6c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.160+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6e00879e0 con 0x7ff6e81609b0 2024-02-02T13:37:48.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.160+0000 7ff6edc00640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 0x7ff6c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.160 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.160+0000 7ff6edc00640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 0x7ff6c006c8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8007fd0 tx=0x7ff6d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.164+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff6e81590b0 con 0x7ff6e81609b0 2024-02-02T13:37:48.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.364+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff6e8160d90 con 0x7ff6e81609b0 2024-02-02T13:37:48.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.365+0000 7ff6e5600640 1 -- 172.21.15.91:0/3905030309 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v198) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff6e8160d90 con 0x7ff6e81609b0 2024-02-02T13:37:48.365 INFO:teuthology.orchestra.run.smithi091.stderr:listed 0 entries 2024-02-02T13:37:48.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 msgr2=0x7ff6c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 0x7ff6c006c8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8007fd0 tx=0x7ff6d8009040 comp rx=0 tx=0).stop 2024-02-02T13:37:48.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 msgr2=0x7ff6e8153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c780 tx=0x7ff6e000cc40 comp rx=0 tx=0).stop 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 shutdown_connections 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff6c006a430 0x7ff6c006c8f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e80b6bd0 0x7ff6e815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff6e81609b0 0x7ff6e8153fa0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 --2- 172.21.15.91:0/3905030309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e8070270 0x7ff6e8153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 >> 172.21.15.91:0/3905030309 conn(0x7ff6e80d3020 msgr2=0x7ff6e80cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:48.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 shutdown_connections 2024-02-02T13:37:48.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.367+0000 7ff6f0600640 1 -- 172.21.15.91:0/3905030309 wait complete. 2024-02-02T13:37:48.387 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-02T13:37:48.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:48.387 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-02T13:37:48.497 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:37:48.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:48.497 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-02T13:37:48.550 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:48.550 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:48.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:37:48.605 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:48.605 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:48.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:37:48.660 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:48.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.867+0000 7f0073c00640 1 Processor -- start 2024-02-02T13:37:48.868 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.867+0000 7f0073c00640 1 -- start start 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.868+0000 7f0073c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 0x7f006c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.868+0000 7f0073c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f006c06f4f0 0x7f006c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.868+0000 7f0073c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c070f80 con 0x7f006c0b0c40 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.868+0000 7f0073c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0b10d0 con 0x7f006c06f4f0 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.868+0000 7f0073c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c152370 con 0x7f006c151f90 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f0071c00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f006c0b0c40 0x7f006c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57122/0 (socket says 172.21.15.91:57122) 2024-02-02T13:37:48.870 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f0071c00640 1 -- 172.21.15.91:0/1803044195 learned_addr learned my addr 172.21.15.91:0/1803044195 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f0063e00640 1 --2- 172.21.15.91:0/1803044195 >> v2:172.21.15.113:3300/0 conn(0x7f006c06f4f0 0x7f006c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f0071200640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 0x7f006c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.872 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824324767 0 0) 0x7f006c070f80 con 0x7f006c0b0c40 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.869+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004c003520 con 0x7f006c0b0c40 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756844441 0 0) 0x7f004c003520 con 0x7f006c0b0c40 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 msgr2=0x7f006c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 0x7f006c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 >> v2:172.21.15.113:3300/0 conn(0x7f006c06f4f0 msgr2=0x7f006c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 --2- 172.21.15.91:0/1803044195 >> v2:172.21.15.113:3300/0 conn(0x7f006c06f4f0 0x7f006c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0cb010 con 0x7f006c0b0c40 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f00640019f0 con 0x7f006c0b0c40 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 >> v1:172.21.15.91:6789/0 conn(0x7f006c0b0c40 legacy=0x7f006c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c003970 0x7f004c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c004360 0x7f004c004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 0x7f004c004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0b10d0 con 0x7f004c003970 2024-02-02T13:37:48.873 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c152370 con 0x7f004c004360 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c00b980 con 0x7f006c151f90 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c003970 0x7f004c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 -- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 msgr2=0x7f004c004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c151f90 0x7f004c004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0063e00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c004360 0x7f004c004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 -- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c004360 msgr2=0x7f004c004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c004360 0x7f004c004840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.874 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.870+0000 7f0071200640 1 -- 172.21.15.91:0/1803044195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0cb010 con 0x7f004c003970 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f0071200640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c003970 0x7f004c003d90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f005c009ac0 tx=0x7f004c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebda896101a3dca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c02c710 con 0x7f004c003970 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c02c870 con 0x7f004c003970 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f006be00640 1 -- 172.21.15.91:0/1803044195 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c02cc00 con 0x7f004c003970 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f0073c00640 1 -- 172.21.15.91:0/1803044195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c003970 msgr2=0x7f004c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f0073c00640 1 --2- 172.21.15.91:0/1803044195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c003970 0x7f004c003d90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f005c009ac0 tx=0x7f004c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.875 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.872+0000 7f0071c00640 1 -- 172.21.15.91:0/1803044195 reap_dead start 2024-02-02T13:37:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.873+0000 7f0073c00640 1 -- 172.21.15.91:0/1803044195 shutdown_connections 2024-02-02T13:37:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.873+0000 7f0073c00640 1 -- 172.21.15.91:0/1803044195 >> 172.21.15.91:0/1803044195 conn(0x7f006c0cec80 msgr2=0x7f006c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.873+0000 7f0073c00640 1 -- 172.21.15.91:0/1803044195 shutdown_connections 2024-02-02T13:37:48.876 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.873+0000 7f0073c00640 1 -- 172.21.15.91:0/1803044195 wait complete. 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.874+0000 7f0073c00640 1 Processor -- start 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.874+0000 7f0073c00640 1 -- start start 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c003970 0x7f006c1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c1599b0 0x7f006c0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c159e60 con 0x7f006c150680 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cd600 con 0x7f004c003970 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.875+0000 7f0073c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0c7430 con 0x7f006c1599b0 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.877 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33608/0 (socket says 172.21.15.91:33608) 2024-02-02T13:37:48.878 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 -- 172.21.15.91:0/775729905 learned_addr learned my addr 172.21.15.91:0/775729905 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:48.879 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0071200640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c003970 0x7f006c1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 -- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c1599b0 msgr2=0x7f006c0c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c1599b0 0x7f006c0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 -- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c003970 msgr2=0x7f006c1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c003970 0x7f006c1f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 -- 172.21.15.91:0/775729905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0c7150 con 0x7f006c150680 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.876+0000 7f0063e00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f00540108c0 tx=0x7f0054010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.877+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054009630 con 0x7f006c150680 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.877+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400eb60 con 0x7f006c150680 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.877+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0ccfb0 con 0x7f006c150680 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.877+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005401c400 con 0x7f006c150680 2024-02-02T13:37:48.880 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.877+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c150b00 con 0x7f006c150680 2024-02-02T13:37:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.878+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c066cf0 con 0x7f006c150680 2024-02-02T13:37:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.879+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f005401c560 con 0x7f006c150680 2024-02-02T13:37:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.880+0000 7f0069600640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 0x7f004406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:48.881 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.880+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0054088e00 con 0x7f006c150680 2024-02-02T13:37:48.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.881+0000 7f0071200640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 0x7f004406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:48.882 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.881+0000 7f0071200640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 0x7f004406c920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f005c036000 tx=0x7f005c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:48.885 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:48.884+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f00540544a0 con 0x7f006c150680 2024-02-02T13:37:49.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.114+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f006c0cd740 con 0x7f006c150680 2024-02-02T13:37:49.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.115+0000 7f0069600640 1 -- 172.21.15.91:0/775729905 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f0054057ac0 con 0x7f006c150680 2024-02-02T13:37:49.116 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:49.116 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:49.116 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 msgr2=0x7f004406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 0x7f004406c920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f005c036000 tx=0x7f005c030000 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 msgr2=0x7f006c159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f00540108c0 tx=0x7f0054010d80 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 shutdown_connections 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f004406a460 0x7f004406c920 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c1599b0 0x7f006c0c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f006c150680 0x7f006c159470 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 --2- 172.21.15.91:0/775729905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004c003970 0x7f006c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 >> 172.21.15.91:0/775729905 conn(0x7f006c0cec80 msgr2=0x7f006c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 shutdown_connections 2024-02-02T13:37:49.119 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.117+0000 7f0073c00640 1 -- 172.21.15.91:0/775729905 wait complete. 2024-02-02T13:37:49.138 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:49.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.330+0000 7f8807000640 1 Processor -- start 2024-02-02T13:37:49.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.330+0000 7f8807000640 1 -- start start 2024-02-02T13:37:49.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.331+0000 7f8807000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f880015a010 0x7f88000c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.331+0000 7f8807000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000ce130 0x7f88000ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.331+0000 7f8807000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88000cd470 0x7f88000b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.331+0000 7f8807000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000cc7c0 con 0x7f880015a010 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f8807000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000ce930 con 0x7f88000cd470 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f8807000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880006c890 con 0x7f88000ce130 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f880015a010 0x7f88000c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f880015a010 0x7f88000c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33612/0 (socket says 172.21.15.91:33612) 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 -- 172.21.15.91:0/1315673312 learned_addr learned my addr 172.21.15.91:0/1315673312 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f8805000640 1 --2- 172.21.15.91:0/1315673312 >> v2:172.21.15.113:3300/0 conn(0x7f88000cd470 0x7f88000b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ff400640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000ce130 0x7f88000ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 -- 172.21.15.91:0/1315673312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000ce130 msgr2=0x7f88000ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000ce130 0x7f88000ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 -- 172.21.15.91:0/1315673312 >> v2:172.21.15.113:3300/0 conn(0x7f88000cd470 msgr2=0x7f88000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 --2- 172.21.15.91:0/1315673312 >> v2:172.21.15.113:3300/0 conn(0x7f88000cd470 0x7f88000b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.332+0000 7f87ffe00640 1 -- 172.21.15.91:0/1315673312 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880006cb70 con 0x7f880015a010 2024-02-02T13:37:49.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.333+0000 7f87ffe00640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f880015a010 0x7f88000c9670 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f88000ce5c0 tx=0x7f87f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c9b160d615b958 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.334+0000 7f87fea00640 1 -- 172.21.15.91:0/1315673312 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0031870 con 0x7f880015a010 2024-02-02T13:37:49.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.334+0000 7f87fea00640 1 -- 172.21.15.91:0/1315673312 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f00319d0 con 0x7f880015a010 2024-02-02T13:37:49.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.334+0000 7f87fea00640 1 -- 172.21.15.91:0/1315673312 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0031d60 con 0x7f880015a010 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.334+0000 7f8807000640 1 -- 172.21.15.91:0/1315673312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f880015a010 msgr2=0x7f88000c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.334+0000 7f8807000640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f880015a010 0x7f88000c9670 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f88000ce5c0 tx=0x7f87f002b910 comp rx=0 tx=0).stop 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 -- 172.21.15.91:0/1315673312 shutdown_connections 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 --2- 172.21.15.91:0/1315673312 >> v2:172.21.15.113:3300/0 conn(0x7f88000cd470 0x7f88000b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000ce130 0x7f88000ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 --2- 172.21.15.91:0/1315673312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f880015a010 0x7f88000c9670 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 -- 172.21.15.91:0/1315673312 >> 172.21.15.91:0/1315673312 conn(0x7f88000cec80 msgr2=0x7f88000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 -- 172.21.15.91:0/1315673312 shutdown_connections 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.335+0000 7f8807000640 1 -- 172.21.15.91:0/1315673312 wait complete. 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.337+0000 7f8807000640 1 Processor -- start 2024-02-02T13:37:49.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.337+0000 7f8807000640 1 -- start start 2024-02-02T13:37:49.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f880015a010 0x7f88000c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000c6200 0x7f88000c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8800144760 con 0x7f88000ce130 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001f2a70 con 0x7f880015a010 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f8807000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000c6780 con 0x7f88000c6200 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33626/0 (socket says 172.21.15.91:33626) 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 -- 172.21.15.91:0/560736456 learned_addr learned my addr 172.21.15.91:0/560736456 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ff400640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f880015a010 0x7f88000c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 -- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000c6200 msgr2=0x7f88000c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000c6200 0x7f88000c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 -- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f880015a010 msgr2=0x7f88000c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f880015a010 0x7f88000c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.338+0000 7f87ffe00640 1 -- 172.21.15.91:0/560736456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00094d0 con 0x7f88000ce130 2024-02-02T13:37:49.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f87ffe00640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f003ac60 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f003adc0 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00445f0 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88001440a0 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.339+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8800157850 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.340+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800066cf0 con 0x7f88000ce130 2024-02-02T13:37:49.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.341+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f87f0044750 con 0x7f88000ce130 2024-02-02T13:37:49.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.342+0000 7f87f7e00640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 0x7f87d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.342+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87f002d040 con 0x7f88000ce130 2024-02-02T13:37:49.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.342+0000 7f87ff400640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 0x7f87d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.343+0000 7f87ff400640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 0x7f87d806c8f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.347+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f87f0076800 con 0x7f88000ce130 2024-02-02T13:37:49.541 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.539+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88000b4c00 con 0x7f88000ce130 2024-02-02T13:37:49.543 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.541+0000 7f87f7e00640 1 -- 172.21.15.91:0/560736456 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16111 (secure 0 0 0) 0x7f87f0035020 con 0x7f88000ce130 2024-02-02T13:37:49.543 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:49.543 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":198,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:39.550244+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 msgr2=0x7f87d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 0x7f87d806c8f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 msgr2=0x7f88000c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f00046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 shutdown_connections 2024-02-02T13:37:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f87d806a430 0x7f87d806c8f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88000c6200 0x7f88000c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f880015a010 0x7f88000c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.544+0000 7f8807000640 1 --2- 172.21.15.91:0/560736456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f88000ce130 0x7f88000c4e80 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.545+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 >> 172.21.15.91:0/560736456 conn(0x7f88000cec80 msgr2=0x7f88000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.545+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 shutdown_connections 2024-02-02T13:37:49.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.545+0000 7f8807000640 1 -- 172.21.15.91:0/560736456 wait complete. 2024-02-02T13:37:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T13:24:26.310552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T13:37:08.724577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T13:37:09.746745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T13:37:49.567 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T13:37:49.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.767+0000 7fcfd1800640 1 Processor -- start 2024-02-02T13:37:49.768 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.767+0000 7fcfd1800640 1 -- start start 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.768+0000 7fcfd1800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.768+0000 7fcfd1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0ccd70 0x7fcfcc0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.768+0000 7fcfd1800640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0ad2e0 con 0x7fcfcc06c820 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.768+0000 7fcfd1800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0ad5c0 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.768+0000 7fcfd1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc06b4f0 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33634/0 (socket says 172.21.15.91:33634) 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcb400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcfcc06c820 0x7fcfcc0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:57136/0 (socket says 172.21.15.91:57136) 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 -- 172.21.15.91:0/4195919417 learned_addr learned my addr 172.21.15.91:0/4195919417 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfd0e00640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0ccd70 0x7fcfcc0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 -- 172.21.15.91:0/4195919417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0ccd70 msgr2=0x7fcfcc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:49.771 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0ccd70 0x7fcfcc0c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 -- 172.21.15.91:0/4195919417 >> v1:172.21.15.91:6789/0 conn(0x7fcfcc06c820 legacy=0x7fcfcc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcbe00640 1 -- 172.21.15.91:0/4195919417 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc06b7d0 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.769+0000 7fcfcaa00640 1 -- 172.21.15.91:0/4195919417 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558245497 0 0) 0x7fcfcc0ad2e0 con 0x7fcfcc06c820 2024-02-02T13:37:49.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.771+0000 7fcfcbe00640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009870 tx=0x7fcfbc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82d6461f77ff470 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.771+0000 7fcfcaa00640 1 -- 172.21.15.91:0/4195919417 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc030840 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.771+0000 7fcfcaa00640 1 -- 172.21.15.91:0/4195919417 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc0309a0 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.771+0000 7fcfcaa00640 1 -- 172.21.15.91:0/4195919417 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc030d30 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 -- 172.21.15.91:0/4195919417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc15d2c0 msgr2=0x7fcfcc06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009870 tx=0x7fcfbc02b980 comp rx=0 tx=0).stop 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 -- 172.21.15.91:0/4195919417 shutdown_connections 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0ccd70 0x7fcfcc0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 --2- 172.21.15.91:0/4195919417 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc15d2c0 0x7fcfcc06bfd0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 -- 172.21.15.91:0/4195919417 >> 172.21.15.91:0/4195919417 conn(0x7fcfcc0cea90 msgr2=0x7fcfcc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 -- 172.21.15.91:0/4195919417 shutdown_connections 2024-02-02T13:37:49.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.772+0000 7fcfd1800640 1 -- 172.21.15.91:0/4195919417 wait complete. 2024-02-02T13:37:49.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.774+0000 7fcfd1800640 1 Processor -- start 2024-02-02T13:37:49.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.774+0000 7fcfd1800640 1 -- start start 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.775+0000 7fcfd1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc06c820 0x7fcfcc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.775+0000 7fcfd1800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.775+0000 7fcfd1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15d2c0 0x7fcfcc15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfd1800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15fe10 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfd1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15b6c0 con 0x7fcfcc06c820 2024-02-02T13:37:49.777 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfd1800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15d9b0 con 0x7fcfcc15d2c0 2024-02-02T13:37:49.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:33640/0 (socket says 172.21.15.91:33640) 2024-02-02T13:37:49.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 -- 172.21.15.91:0/2070731918 learned_addr learned my addr 172.21.15.91:0/2070731918 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfd0e00640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15d2c0 0x7fcfcc15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 -- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15d2c0 msgr2=0x7fcfcc15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15d2c0 0x7fcfcc15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 -- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc06c820 msgr2=0x7fcfcc15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc06c820 0x7fcfcc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.776+0000 7fcfcb400640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0094d0 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfcb400640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c430 tx=0x7fcfb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40074e0 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb4007660 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc15dc90 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4004a20 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.777+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc15e250 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.778+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc15e470 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.779+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcfb4004b80 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.780+0000 7fcfc3e00640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 0x7fcfa406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:49.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.780+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfb40887f0 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.780+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7fcfa4071f60 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.780+0000 7fcfcbe00640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 0x7fcfa406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:49.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.781+0000 7fcfcbe00640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 0x7fcfa406c8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc036000 tx=0x7fcfbc03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:49.786 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.785+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcfb4053d40 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.979 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.977+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfcc04dff0 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.980 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.978+0000 7fcfc3e00640 1 -- 172.21.15.91:0/2070731918 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfb4057360 con 0x7fcfcc0ccd70 2024-02-02T13:37:49.980 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 1 2024-02-02T13:37:49.983 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 msgr2=0x7fcfa406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 0x7fcfa406c8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc036000 tx=0x7fcfbc03b040 comp rx=0 tx=0).stop 2024-02-02T13:37:49.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 msgr2=0x7fcfcc15f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:49.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c430 tx=0x7fcfb400c8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 shutdown_connections 2024-02-02T13:37:49.984 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcfa406a430 0x7fcfa406c8f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc15d2c0 0x7fcfcc15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcfcc0ccd70 0x7fcfcc15f350 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 --2- 172.21.15.91:0/2070731918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc06c820 0x7fcfcc15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.981+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 >> 172.21.15.91:0/2070731918 conn(0x7fcfcc0cea90 msgr2=0x7fcfcc0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.982+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 shutdown_connections 2024-02-02T13:37:49.985 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:49.982+0000 7fcfd1800640 1 -- 172.21.15.91:0/2070731918 wait complete. 2024-02-02T13:37:50.001 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T13:37:50.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.211+0000 7fd178400640 1 Processor -- start 2024-02-02T13:37:50.213 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.211+0000 7fd178400640 1 -- start start 2024-02-02T13:37:50.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd170151f90 0x7fd17015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.217 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 0x7fd1700b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1700b0c40 0x7fd1700b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700b10d0 con 0x7fd1700b0c40 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd170070f80 con 0x7fd17006f4f0 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd178400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd170152370 con 0x7fd170151f90 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.212+0000 7fd176400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1700b0c40 0x7fd1700b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.218 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd1700b0c40 0x7fd1700b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51338/0 (socket says 172.21.15.91:51338) 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 -- 172.21.15.91:0/589128631 learned_addr learned my addr 172.21.15.91:0/589128631 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd175000640 1 --2- 172.21.15.91:0/589128631 >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 0x7fd1700b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd175a00640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd170151f90 0x7fd17015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 -- 172.21.15.91:0/589128631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd170151f90 msgr2=0x7fd17015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd170151f90 0x7fd17015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 -- 172.21.15.91:0/589128631 >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 msgr2=0x7fd1700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:50.219 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 --2- 172.21.15.91:0/589128631 >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 0x7fd1700b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd176400640 1 -- 172.21.15.91:0/589128631 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700cb010 con 0x7fd1700b0c40 2024-02-02T13:37:50.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.213+0000 7fd175000640 1 --2- 172.21.15.91:0/589128631 >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 0x7fd1700b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:50.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.214+0000 7fd176400640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1700b0c40 0x7fd1700b3af0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd1700ceac0 tx=0x7fd16802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ce54f7ef1fde11 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd16fe00640 1 -- 172.21.15.91:0/589128631 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031870 con 0x7fd1700b0c40 2024-02-02T13:37:50.220 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd16fe00640 1 -- 172.21.15.91:0/589128631 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1680319d0 con 0x7fd1700b0c40 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd16fe00640 1 -- 172.21.15.91:0/589128631 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031d60 con 0x7fd1700b0c40 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 -- 172.21.15.91:0/589128631 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1700b0c40 msgr2=0x7fd1700b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1700b0c40 0x7fd1700b3af0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd1700ceac0 tx=0x7fd16802b910 comp rx=0 tx=0).stop 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 -- 172.21.15.91:0/589128631 shutdown_connections 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1700b0c40 0x7fd1700b3af0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.221 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 --2- 172.21.15.91:0/589128631 >> v2:172.21.15.113:3300/0 conn(0x7fd17006f4f0 0x7fd1700b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 --2- 172.21.15.91:0/589128631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd170151f90 0x7fd17015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 -- 172.21.15.91:0/589128631 >> 172.21.15.91:0/589128631 conn(0x7fd1700cec80 msgr2=0x7fd1700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:50.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.215+0000 7fd178400640 1 -- 172.21.15.91:0/589128631 shutdown_connections 2024-02-02T13:37:50.222 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.216+0000 7fd178400640 1 -- 172.21.15.91:0/589128631 wait complete. 2024-02-02T13:37:50.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.217+0000 7fd178400640 1 Processor -- start 2024-02-02T13:37:50.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.217+0000 7fd178400640 1 -- start start 2024-02-02T13:37:50.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd17006f4f0 0x7fd17015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.227 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1700b0c40 0x7fd17015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c7780 con 0x7fd170151f90 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170151520 con 0x7fd1700b0c40 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd178400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c7430 con 0x7fd17006f4f0 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.218+0000 7fd176400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.228 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51346/0 (socket says 172.21.15.91:51346) 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 -- 172.21.15.91:0/993424504 learned_addr learned my addr 172.21.15.91:0/993424504 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 -- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd17006f4f0 msgr2=0x7fd17015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd17006f4f0 0x7fd17015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 -- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1700b0c40 msgr2=0x7fd17015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1700b0c40 0x7fd17015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.229 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680094d0 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd176400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd1680090b0 tx=0x7fd1680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16803ad40 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1680443f0 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.219+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1680447a0 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.220+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17015bc10 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.220+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17015c1d0 con 0x7fd170151f90 2024-02-02T13:37:50.230 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.221+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170066cf0 con 0x7fd170151f90 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.222+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd16804d450 con 0x7fd170151f90 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.222+0000 7fd16d600640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 0x7fd14806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.222+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd16802d040 con 0x7fd170151f90 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.223+0000 7fd175a00640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 0x7fd14806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.223+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7fd148071f60 con 0x7fd170151f90 2024-02-02T13:37:50.231 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.223+0000 7fd175a00640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 0x7fd14806c8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd160007130 tx=0x7fd160007310 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.232 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.227+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd168076620 con 0x7fd170151f90 2024-02-02T13:37:50.420 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.418+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1700711d0 con 0x7fd170151f90 2024-02-02T13:37:50.421 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.419+0000 7fd16d600640 1 -- 172.21.15.91:0/993424504 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd168079c40 con 0x7fd170151f90 2024-02-02T13:37:50.422 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 32 2024-02-02T13:37:50.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.422+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 msgr2=0x7fd14806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.422+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 0x7fd14806c8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd160007130 tx=0x7fd160007310 comp rx=0 tx=0).stop 2024-02-02T13:37:50.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.422+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 msgr2=0x7fd1700c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.422+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd1680090b0 tx=0x7fd1680046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 shutdown_connections 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fd14806a430 0x7fd14806c8f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd170151f90 0x7fd1700c7ba0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1700b0c40 0x7fd17015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 --2- 172.21.15.91:0/993424504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd17006f4f0 0x7fd17015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 >> 172.21.15.91:0/993424504 conn(0x7fd1700cec80 msgr2=0x7fd1700c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:50.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 shutdown_connections 2024-02-02T13:37:50.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.423+0000 7fd178400640 1 -- 172.21.15.91:0/993424504 wait complete. 2024-02-02T13:37:50.444 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T13:37:50.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.650+0000 7ff4e3200640 1 Processor -- start 2024-02-02T13:37:50.651 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.650+0000 7ff4e3200640 1 -- start start 2024-02-02T13:37:50.652 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.651+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.651+0000 7ff4e3200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 0x7ff4dc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.651+0000 7ff4e3200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0cc7c0 con 0x7ff4dc0cd470 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.651+0000 7ff4e3200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0ce930 con 0x7ff4dc0ce130 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.651+0000 7ff4e3200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc06c890 con 0x7ff4dc15a010 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.652+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.652+0000 7ff4db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 0x7ff4dc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.652+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:38438/0 (socket says 172.21.15.91:38438) 2024-02-02T13:37:50.653 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.652+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1513246857 learned_addr learned my addr 172.21.15.91:0/1513246857 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:50.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.653+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1513246857 >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 msgr2=0x7ff4dc0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:50.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.653+0000 7ff4dbe00640 1 --2- 172.21.15.91:0/1513246857 >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 0x7ff4dc0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.653+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1513246857 >> v1:172.21.15.113:6789/0 conn(0x7ff4dc0cd470 legacy=0x7ff4dc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.653+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1513246857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc06cb70 con 0x7ff4dc15a010 2024-02-02T13:37:50.654 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.653+0000 7ff4db400640 1 --2- 172.21.15.91:0/1513246857 >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 0x7ff4dc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:37:50.655 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.654+0000 7ff4dbe00640 1 --2- 172.21.15.91:0/1513246857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bb10 tx=0x7ff4cc02e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d4bc8895fc17de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.654+0000 7ff4daa00640 1 -- 172.21.15.91:0/1513246857 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc036440 con 0x7ff4dc15a010 2024-02-02T13:37:50.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.654+0000 7ff4daa00640 1 -- 172.21.15.91:0/1513246857 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0365a0 con 0x7ff4dc15a010 2024-02-02T13:37:50.656 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.654+0000 7ff4daa00640 1 -- 172.21.15.91:0/1513246857 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc036930 con 0x7ff4dc15a010 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 -- 172.21.15.91:0/1513246857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 msgr2=0x7ff4dc0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1513246857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bb10 tx=0x7ff4cc02e160 comp rx=0 tx=0).stop 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 -- 172.21.15.91:0/1513246857 shutdown_connections 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1513246857 >> v2:172.21.15.113:3300/0 conn(0x7ff4dc0ce130 0x7ff4dc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1513246857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc0c9670 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 -- 172.21.15.91:0/1513246857 >> 172.21.15.91:0/1513246857 conn(0x7ff4dc0cec80 msgr2=0x7ff4dc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 -- 172.21.15.91:0/1513246857 shutdown_connections 2024-02-02T13:37:50.657 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.655+0000 7ff4e3200640 1 -- 172.21.15.91:0/1513246857 wait complete. 2024-02-02T13:37:50.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.657+0000 7ff4e3200640 1 Processor -- start 2024-02-02T13:37:50.659 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.658+0000 7ff4e3200640 1 -- start start 2024-02-02T13:37:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 0x7ff4dc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc15cd00 con 0x7ff4dc0ce130 2024-02-02T13:37:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0b1a10 con 0x7ff4dc15c310 2024-02-02T13:37:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e3200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c5210 con 0x7ff4dc15a010 2024-02-02T13:37:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51348/0 (socket says 172.21.15.91:51348) 2024-02-02T13:37:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1310022022 learned_addr learned my addr 172.21.15.91:0/1310022022 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:50.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4db400640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.659+0000 7ff4e1200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 0x7ff4dc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 msgr2=0x7ff4dc14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.668 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 msgr2=0x7ff4dc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 0x7ff4dc15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009400 con 0x7ff4dc0ce130 2024-02-02T13:37:50.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4e1200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 0x7ff4dc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:50.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4dbe00640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc02e1a0 tx=0x7ff4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc036440 con 0x7ff4dc0ce130 2024-02-02T13:37:50.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc034ba0 con 0x7ff4dc0ce130 2024-02-02T13:37:50.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc00b770 con 0x7ff4dc0ce130 2024-02-02T13:37:50.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc040400 con 0x7ff4dc0ce130 2024-02-02T13:37:50.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.660+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc0c3f10 con 0x7ff4dc0ce130 2024-02-02T13:37:50.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.661+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc066cf0 con 0x7ff4dc0ce130 2024-02-02T13:37:50.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.662+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4cc040560 con 0x7ff4dc0ce130 2024-02-02T13:37:50.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.663+0000 7ff4d3e00640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 0x7ff4b406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:50.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.664+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4cc0adb90 con 0x7ff4dc0ce130 2024-02-02T13:37:50.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.664+0000 7ff4db400640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 0x7ff4b406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:50.671 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.664+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7ff4b4071f90 con 0x7ff4dc0ce130 2024-02-02T13:37:50.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.664+0000 7ff4db400640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 0x7ff4b406c920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009c60 tx=0x7ff4c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:50.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.668+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4cc03b060 con 0x7ff4dc0ce130 2024-02-02T13:37:50.863 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.861+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4dc0b4c00 con 0x7ff4dc0ce130 2024-02-02T13:37:50.864 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.862+0000 7ff4d3e00640 1 -- 172.21.15.91:0/1310022022 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4cc04e020 con 0x7ff4dc0ce130 2024-02-02T13:37:50.864 INFO:teuthology.orchestra.run.smithi091.stdout:pg_num: 64 2024-02-02T13:37:50.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 msgr2=0x7ff4b406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 0x7ff4b406c920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009c60 tx=0x7ff4c4008040 comp rx=0 tx=0).stop 2024-02-02T13:37:50.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 msgr2=0x7ff4dc0c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:50.866 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc02e1a0 tx=0x7ff4cc0046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 shutdown_connections 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff4b406a460 0x7ff4b406c920 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc15a010 0x7ff4dc14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc15c310 0x7ff4dc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 --2- 172.21.15.91:0/1310022022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff4dc0ce130 0x7ff4dc0c4cd0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 >> 172.21.15.91:0/1310022022 conn(0x7ff4dc0cec80 msgr2=0x7ff4dc0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.865+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 shutdown_connections 2024-02-02T13:37:50.867 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:50.866+0000 7ff4e3200640 1 -- 172.21.15.91:0/1310022022 wait complete. 2024-02-02T13:37:50.886 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:51.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.085+0000 7f40f6400640 1 Processor -- start 2024-02-02T13:37:51.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.085+0000 7f40f6400640 1 -- start start 2024-02-02T13:37:51.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40f6400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40f015a010 0x7f40f00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40f6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40f00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40f6400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f00cc7c0 con 0x7f40f00ce130 2024-02-02T13:37:51.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40f6400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f00b10d0 con 0x7f40f015a010 2024-02-02T13:37:51.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40f6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00ce930 con 0x7f40f00b0700 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40e6800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f40f00ce130 0x7f40f00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50656/0 (socket says 172.21.15.91:50656) 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.086+0000 7f40e6800640 1 -- 172.21.15.91:0/1590811474 learned_addr learned my addr 172.21.15.91:0/1590811474 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40ef400640 1 --2- 172.21.15.91:0/1590811474 >> v2:172.21.15.113:3300/0 conn(0x7f40f015a010 0x7f40f00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422597093 0 0) 0x7f40f00cc7c0 con 0x7f40f00ce130 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d0003520 con 0x7f40f00ce130 2024-02-02T13:37:51.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40efe00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40f00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765969117 0 0) 0x7f40d0003520 con 0x7f40f00ce130 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 msgr2=0x7f40f00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40f00b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 >> v2:172.21.15.113:3300/0 conn(0x7f40f015a010 msgr2=0x7f40f00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 --2- 172.21.15.91:0/1590811474 >> v2:172.21.15.113:3300/0 conn(0x7f40f015a010 0x7f40f00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.103 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00cd640 con 0x7f40f00ce130 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f40d8001670 con 0x7f40f00ce130 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 >> v1:172.21.15.91:6789/0 conn(0x7f40f00ce130 legacy=0x7f40f00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d0003970 0x7f40d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.104 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00b10d0 con 0x7f40d0004d00 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00ce930 con 0x7f40d0003970 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.087+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d000b980 con 0x7f40f00b0700 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40e6800640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40ef400640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d0003970 0x7f40d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 -- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 msgr2=0x7f40d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.105 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00b0700 0x7f40d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 -- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d0003970 msgr2=0x7f40d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d0003970 0x7f40d0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.088+0000 7f40efe00640 1 -- 172.21.15.91:0/1590811474 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d0003520 con 0x7f40d0004d00 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.089+0000 7f40efe00640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40d0005180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007390 tx=0x7f40d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2be34605fde753 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.089+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e802ccb0 con 0x7f40d0004d00 2024-02-02T13:37:51.106 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.089+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e802ce10 con 0x7f40d0004d00 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.089+0000 7f40eea00640 1 -- 172.21.15.91:0/1590811474 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8034660 con 0x7f40d0004d00 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 -- 172.21.15.91:0/1590811474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 msgr2=0x7f40d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 --2- 172.21.15.91:0/1590811474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40d0005180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007390 tx=0x7f40d00057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40efe00640 1 -- 172.21.15.91:0/1590811474 reap_dead start 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 -- 172.21.15.91:0/1590811474 shutdown_connections 2024-02-02T13:37:51.107 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 -- 172.21.15.91:0/1590811474 >> 172.21.15.91:0/1590811474 conn(0x7f40f00cec80 msgr2=0x7f40f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 -- 172.21.15.91:0/1590811474 shutdown_connections 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.090+0000 7f40f6400640 1 -- 172.21.15.91:0/1590811474 wait complete. 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.092+0000 7f40f6400640 1 Processor -- start 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.092+0000 7f40f6400640 1 -- start start 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.108 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 0x7f40f0155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 0x7f40f0070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c66a0 con 0x7f40d0004d00 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c6980 con 0x7f40f00c4a20 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40f6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c6c60 con 0x7f40f00c5cc0 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40ef400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40ef400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51362/0 (socket says 172.21.15.91:51362) 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.093+0000 7f40ef400640 1 -- 172.21.15.91:0/1256920287 learned_addr learned my addr 172.21.15.91:0/1256920287 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40e6800640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 0x7f40f0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40efe00640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 0x7f40f0070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 -- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 msgr2=0x7f40f0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 0x7f40f0070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 -- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 msgr2=0x7f40f0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 0x7f40f0155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00c6f40 con 0x7f40d0004d00 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40e6800640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 0x7f40f0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40efe00640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 0x7f40f0070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40ef400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f40e00108e0 tx=0x7f40e0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000eb60 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e000ecc0 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e001c400 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f00c7220 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.094+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0155660 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.095+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f0066cf0 con 0x7f40d0004d00 2024-02-02T13:37:51.112 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.096+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f40e001c560 con 0x7f40d0004d00 2024-02-02T13:37:51.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.097+0000 7f40e7e00640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 0x7f40c806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.097+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40e0089080 con 0x7f40d0004d00 2024-02-02T13:37:51.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.098+0000 7f40e6800640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 0x7f40c806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.098+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7f40c8071f90 con 0x7f40d0004d00 2024-02-02T13:37:51.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.098+0000 7f40e6800640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 0x7f40c806c920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f40d80017a0 tx=0x7f40d8000ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.102+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f40e00bbca0 con 0x7f40d0004d00 2024-02-02T13:37:51.327 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.325+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40f00b4c00 con 0x7f40d0004d00 2024-02-02T13:37:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.326+0000 7f40e7e00640 1 -- 172.21.15.91:0/1256920287 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f40e0021020 con 0x7f40d0004d00 2024-02-02T13:37:51.328 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:51.329 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:51.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 msgr2=0x7f40c806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 0x7f40c806c920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f40d80017a0 tx=0x7f40d8000ed0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 msgr2=0x7f40f00c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f40e00108e0 tx=0x7f40e0010da0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 shutdown_connections 2024-02-02T13:37:51.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f40c806a460 0x7f40c806c920 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f00c5cc0 0x7f40f0070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.328+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f00c4a20 0x7f40f0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.329+0000 7f40f6400640 1 --2- 172.21.15.91:0/1256920287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40d0004d00 0x7f40f00c7740 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.329+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 >> 172.21.15.91:0/1256920287 conn(0x7f40f00cec80 msgr2=0x7f40f015aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:51.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.329+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 shutdown_connections 2024-02-02T13:37:51.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.329+0000 7f40f6400640 1 -- 172.21.15.91:0/1256920287 wait complete. 2024-02-02T13:37:51.348 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:51.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.536+0000 7f8335e00640 1 Processor -- start 2024-02-02T13:37:51.538 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.536+0000 7f8335e00640 1 -- start start 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f8335e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f833015d590 0x7f83300b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f8335e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83300b1ba0 0x7f83300b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f8335e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83300b1980 con 0x7f83300b1360 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f8335e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83300d07b0 con 0x7f833015d590 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f8335e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83300cff10 con 0x7f83300b1ba0 2024-02-02T13:37:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f832ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f833015d590 0x7f83300b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f832ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f833015d590 0x7f83300b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51378/0 (socket says 172.21.15.91:51378) 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.538+0000 7f832ea00640 1 -- 172.21.15.91:0/2125434385 learned_addr learned my addr 172.21.15.91:0/2125434385 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832e000640 1 --2- 172.21.15.91:0/2125434385 >> v2:172.21.15.113:3300/0 conn(0x7f83300b1ba0 0x7f83300b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832ea00640 1 -- 172.21.15.91:0/2125434385 >> v2:172.21.15.113:3300/0 conn(0x7f83300b1ba0 msgr2=0x7f83300b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832ea00640 1 --2- 172.21.15.91:0/2125434385 >> v2:172.21.15.113:3300/0 conn(0x7f83300b1ba0 0x7f83300b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832ea00640 1 -- 172.21.15.91:0/2125434385 >> v1:172.21.15.91:6789/0 conn(0x7f83300b1360 legacy=0x7f8330069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832ea00640 1 -- 172.21.15.91:0/2125434385 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300d01f0 con 0x7f833015d590 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.539+0000 7f832d600640 1 -- 172.21.15.91:0/2125434385 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414158647 0 0) 0x7f83300b1980 con 0x7f83300b1360 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f832ea00640 1 --2- 172.21.15.91:0/2125434385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f83300b08e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d925472b5835444d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f832d600640 1 -- 172.21.15.91:0/2125434385 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200317e0 con 0x7f833015d590 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f832d600640 1 -- 172.21.15.91:0/2125434385 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8320031940 con 0x7f833015d590 2024-02-02T13:37:51.546 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f832d600640 1 -- 172.21.15.91:0/2125434385 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320031cd0 con 0x7f833015d590 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f8335e00640 1 -- 172.21.15.91:0/2125434385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 msgr2=0x7f83300b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.540+0000 7f8335e00640 1 --2- 172.21.15.91:0/2125434385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f83300b08e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002b880 comp rx=0 tx=0).stop 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 -- 172.21.15.91:0/2125434385 shutdown_connections 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 --2- 172.21.15.91:0/2125434385 >> v2:172.21.15.113:3300/0 conn(0x7f83300b1ba0 0x7f83300b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 --2- 172.21.15.91:0/2125434385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f83300b08e0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 -- 172.21.15.91:0/2125434385 >> 172.21.15.91:0/2125434385 conn(0x7f83300d0fa0 msgr2=0x7f83300d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 -- 172.21.15.91:0/2125434385 shutdown_connections 2024-02-02T13:37:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.541+0000 7f8335e00640 1 -- 172.21.15.91:0/2125434385 wait complete. 2024-02-02T13:37:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.542+0000 7f8335e00640 1 Processor -- start 2024-02-02T13:37:51.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.542+0000 7f8335e00640 1 -- start start 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 0x7f83300b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83300b1ba0 0x7f83300b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300b5b10 con 0x7f833015d590 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300b5df0 con 0x7f83300b1360 2024-02-02T13:37:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.543+0000 7f8335e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83300b60d0 con 0x7f83300b1ba0 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51392/0 (socket says 172.21.15.91:51392) 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 -- 172.21.15.91:0/1607811057 learned_addr learned my addr 172.21.15.91:0/1607811057 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 -- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83300b1ba0 msgr2=0x7f83300b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83300b1ba0 0x7f83300b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832ea00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 0x7f83300b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 -- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 msgr2=0x7f83300b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 0x7f83300b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200094d0 con 0x7f833015d590 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832ea00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 0x7f83300b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f832f400640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8328001560 tx=0x7f832800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328003da0 con 0x7f833015d590 2024-02-02T13:37:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328005070 con 0x7f833015d590 2024-02-02T13:37:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.544+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328005420 con 0x7f833015d590 2024-02-02T13:37:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.545+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83300b63b0 con 0x7f833015d590 2024-02-02T13:37:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.545+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83301557c0 con 0x7f833015d590 2024-02-02T13:37:51.556 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.546+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8330066cf0 con 0x7f833015d590 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.547+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8328005580 con 0x7f833015d590 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.547+0000 7f8327400640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 0x7f830806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.547+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83280881b0 con 0x7f833015d590 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.548+0000 7f832ea00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 0x7f830806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.548+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7f8308071f60 con 0x7f833015d590 2024-02-02T13:37:51.557 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.548+0000 7f832ea00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 0x7f830806c8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8320036000 tx=0x7f832003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.558 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.552+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8328017070 con 0x7f833015d590 2024-02-02T13:37:51.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.730+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f833006a800 con 0x7f833015d590 2024-02-02T13:37:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.732+0000 7f8327400640 1 -- 172.21.15.91:0/1607811057 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16130 (secure 0 0 0) 0x7f83280537a0 con 0x7f833015d590 2024-02-02T13:37:51.734 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:51.734 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":199,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:49.553575+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 msgr2=0x7f830806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 0x7f830806c8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8320036000 tx=0x7f832003d040 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 msgr2=0x7f833015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8328001560 tx=0x7f832800cc70 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 shutdown_connections 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f830806a430 0x7f830806c8f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83300b1ba0 0x7f83300b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f833015d590 0x7f833015b890 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 --2- 172.21.15.91:0/1607811057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83300b1360 0x7f83300b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 >> 172.21.15.91:0/1607811057 conn(0x7f83300d0fa0 msgr2=0x7f833006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 shutdown_connections 2024-02-02T13:37:51.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.735+0000 7f8335e00640 1 -- 172.21.15.91:0/1607811057 wait complete. 2024-02-02T13:37:51.753 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:37:51.754 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:51.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.951+0000 7fe370000640 1 Processor -- start 2024-02-02T13:37:51.953 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.951+0000 7fe370000640 1 -- start start 2024-02-02T13:37:51.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe370000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 0x7fe3680cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe370000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680d12c0 con 0x7fe3680d2760 2024-02-02T13:37:51.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe370000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe368161ed0 con 0x7fe3680ce070 2024-02-02T13:37:51.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe370000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680cded0 con 0x7fe36815d650 2024-02-02T13:37:51.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe36e000640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe3680d2760 0x7fe3681617c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50674/0 (socket says 172.21.15.91:50674) 2024-02-02T13:37:51.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.953+0000 7fe36e000640 1 -- 172.21.15.91:0/1378434156 learned_addr learned my addr 172.21.15.91:0/1378434156 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:51.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe36d600640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 0x7fe3680cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.956 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470286679 0 0) 0x7fe3680d12c0 con 0x7fe3680d2760 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348003520 con 0x7fe3680d2760 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727459257 0 0) 0x7fe348003520 con 0x7fe3680d2760 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 msgr2=0x7fe3680cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 0x7fe3680cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 >> v1:172.21.15.113:6789/0 conn(0x7fe3680ce070 legacy=0x7fe3681610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368071410 con 0x7fe3680d2760 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe3600019f0 con 0x7fe3680d2760 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 >> v1:172.21.15.91:6789/0 conn(0x7fe3680d2760 legacy=0x7fe3681617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3480039a0 0x7fe348003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 0x7fe3480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348004d00 0x7fe348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680cded0 con 0x7fe3480039a0 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34800ba30 con 0x7fe348004d00 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34800bbf0 con 0x7fe36815d650 2024-02-02T13:37:51.957 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.954+0000 7fe36d600640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3480039a0 0x7fe348003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.955+0000 7fe36d600640 1 -- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 msgr2=0x7fe3480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.955+0000 7fe36d600640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe36815d650 0x7fe3480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.955+0000 7fe36d600640 1 -- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348004d00 msgr2=0x7fe348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:51.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.955+0000 7fe36d600640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348004d00 0x7fe348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.955+0000 7fe36d600640 1 -- 172.21.15.91:0/1378434156 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348003520 con 0x7fe3480039a0 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.956+0000 7fe36d600640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3480039a0 0x7fe348003dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe358009a90 tx=0x7fe3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad9e46f2339dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.956+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35802c6b0 con 0x7fe3480039a0 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.956+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35802c810 con 0x7fe3480039a0 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.956+0000 7fe367e00640 1 -- 172.21.15.91:0/1378434156 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35802cba0 con 0x7fe3480039a0 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 -- 172.21.15.91:0/1378434156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3480039a0 msgr2=0x7fe348003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 --2- 172.21.15.91:0/1378434156 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3480039a0 0x7fe348003dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe358009a90 tx=0x7fe3480057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe36e000640 1 -- 172.21.15.91:0/1378434156 reap_dead start 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 -- 172.21.15.91:0/1378434156 shutdown_connections 2024-02-02T13:37:51.959 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 -- 172.21.15.91:0/1378434156 >> 172.21.15.91:0/1378434156 conn(0x7fe3680d30a0 msgr2=0x7fe3680d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:51.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 -- 172.21.15.91:0/1378434156 shutdown_connections 2024-02-02T13:37:51.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.957+0000 7fe370000640 1 -- 172.21.15.91:0/1378434156 wait complete. 2024-02-02T13:37:51.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.958+0000 7fe370000640 1 Processor -- start 2024-02-02T13:37:51.960 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 -- start start 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3480039a0 0x7fe3680b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3680b2420 0x7fe3680b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680b3570 con 0x7fe3680b1fa0 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680b4a20 con 0x7fe3480039a0 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.959+0000 7fe370000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681f2dd0 con 0x7fe3680b2420 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51402/0 (socket says 172.21.15.91:51402) 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 -- 172.21.15.91:0/504527376 learned_addr learned my addr 172.21.15.91:0/504527376 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:51.962 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 -- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3680b2420 msgr2=0x7fe3680b89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3680b2420 0x7fe3680b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 -- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3480039a0 msgr2=0x7fe3680b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3480039a0 0x7fe3680b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680b29f0 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.960+0000 7fe36cc00640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c560 tx=0x7fe35000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.961+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500052d0 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.961+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe350005470 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.961+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350012870 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.961+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3680b2cd0 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.961+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3680b3290 con 0x7fe3680b1fa0 2024-02-02T13:37:51.963 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.962+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368071a30 con 0x7fe3680b1fa0 2024-02-02T13:37:51.964 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.963+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe3500129d0 con 0x7fe3680b1fa0 2024-02-02T13:37:51.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.964+0000 7fe365600640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 0x7fe34006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:51.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.964+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe350088520 con 0x7fe3680b1fa0 2024-02-02T13:37:51.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.964+0000 7fe36d600640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 0x7fe34006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:51.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.964+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7fe340071f60 con 0x7fe3680b1fa0 2024-02-02T13:37:51.965 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.964+0000 7fe36d600640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 0x7fe34006c8f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe358036000 tx=0x7fe35802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:51.969 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:51.968+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe350053a70 con 0x7fe3680b1fa0 2024-02-02T13:37:52.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.189+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe368066cf0 con 0x7fe3680b1fa0 2024-02-02T13:37:52.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.190+0000 7fe365600640 1 -- 172.21.15.91:0/504527376 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fe350057090 con 0x7fe3680b1fa0 2024-02-02T13:37:52.191 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:52.192 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:52.192 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:52.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.192+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 msgr2=0x7fe34006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.192+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 0x7fe34006c8f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe358036000 tx=0x7fe35802e000 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.192+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 msgr2=0x7fe3680b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.192+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c560 tx=0x7fe35000ca20 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 shutdown_connections 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe34006a430 0x7fe34006c8f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3680b2420 0x7fe3680b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe3680b1fa0 0x7fe3680b8470 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 --2- 172.21.15.91:0/504527376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3480039a0 0x7fe3680b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 >> 172.21.15.91:0/504527376 conn(0x7fe3680d30a0 msgr2=0x7fe3680c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 shutdown_connections 2024-02-02T13:37:52.195 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.193+0000 7fe370000640 1 -- 172.21.15.91:0/504527376 wait complete. 2024-02-02T13:37:52.215 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:37:52.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.421+0000 7f9c2a200640 1 Processor -- start 2024-02-02T13:37:52.422 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.421+0000 7f9c2a200640 1 -- start start 2024-02-02T13:37:52.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.422+0000 7f9c2a200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c240b2470 0x7f9c240d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.422+0000 7f9c2a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240d3680 0x7f9c240d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.422+0000 7f9c2a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c24161090 con 0x7f9c2415d330 2024-02-02T13:37:52.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.422+0000 7f9c2a200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c24161cd0 con 0x7f9c240b2470 2024-02-02T13:37:52.424 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.422+0000 7f9c2a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240d3b10 con 0x7f9c240d3680 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c240b2470 0x7f9c240d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c240b2470 0x7f9c240d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51410/0 (socket says 172.21.15.91:51410) 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 -- 172.21.15.91:0/3825020928 learned_addr learned my addr 172.21.15.91:0/3825020928 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c23e00640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240d3680 0x7f9c240d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 -- 172.21.15.91:0/3825020928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240d3680 msgr2=0x7f9c240d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240d3680 0x7f9c240d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 -- 172.21.15.91:0/3825020928 >> v1:172.21.15.113:6789/0 conn(0x7f9c2415d330 legacy=0x7f9c240b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.425 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.423+0000 7f9c1a800640 1 -- 172.21.15.91:0/3825020928 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c24161330 con 0x7f9c240b2470 2024-02-02T13:37:52.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.424+0000 7f9c1a800640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b2470 0x7f9c240d3140 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009870 tx=0x7f9c0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98bacb29bc356da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c22a00640 1 -- 172.21.15.91:0/3825020928 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0317c0 con 0x7f9c240b2470 2024-02-02T13:37:52.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c22a00640 1 -- 172.21.15.91:0/3825020928 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c031920 con 0x7f9c240b2470 2024-02-02T13:37:52.426 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c22a00640 1 -- 172.21.15.91:0/3825020928 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c031cb0 con 0x7f9c240b2470 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 -- 172.21.15.91:0/3825020928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b2470 msgr2=0x7f9c240d3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b2470 0x7f9c240d3140 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009870 tx=0x7f9c0c02b860 comp rx=0 tx=0).stop 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 -- 172.21.15.91:0/3825020928 shutdown_connections 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240d3680 0x7f9c240d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 --2- 172.21.15.91:0/3825020928 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c240b2470 0x7f9c240d3140 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 -- 172.21.15.91:0/3825020928 >> 172.21.15.91:0/3825020928 conn(0x7f9c24070660 msgr2=0x7f9c24070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.425+0000 7f9c2a200640 1 -- 172.21.15.91:0/3825020928 shutdown_connections 2024-02-02T13:37:52.427 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.426+0000 7f9c2a200640 1 -- 172.21.15.91:0/3825020928 wait complete. 2024-02-02T13:37:52.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.427+0000 7f9c2a200640 1 Processor -- start 2024-02-02T13:37:52.428 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.427+0000 7f9c2a200640 1 -- start start 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240b2470 0x7f9c24148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 0x7f9c2415e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24149030 con 0x7f9c2415d330 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240cc080 con 0x7f9c240d3680 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c2a200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240cb470 con 0x7f9c240b2470 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c23e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c23e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51416/0 (socket says 172.21.15.91:51416) 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.428+0000 7f9c23e00640 1 -- 172.21.15.91:0/416140759 learned_addr learned my addr 172.21.15.91:0/416140759 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23400640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240b2470 0x7f9c24148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.431 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 -- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240b2470 msgr2=0x7f9c24148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240b2470 0x7f9c24148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c1a800640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 0x7f9c2415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 -- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 msgr2=0x7f9c2415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 0x7f9c2415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0094d0 con 0x7f9c2415d330 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c1a800640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 0x7f9c2415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c23e00640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c007e40 tx=0x7f9c1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c014620 con 0x7f9c2415d330 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00ee90 con 0x7f9c2415d330 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0149e0 con 0x7f9c2415d330 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.429+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c240cb750 con 0x7f9c2415d330 2024-02-02T13:37:52.432 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.430+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c240cbd10 con 0x7f9c2415d330 2024-02-02T13:37:52.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.431+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c240cc210 con 0x7f9c2415d330 2024-02-02T13:37:52.433 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.431+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c1c014b40 con 0x7f9c2415d330 2024-02-02T13:37:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.433+0000 7f9c1be00640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.433+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c1c0892c0 con 0x7f9c2415d330 2024-02-02T13:37:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.433+0000 7f9c23400640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.434 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.433+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7f9bfc071f60 con 0x7f9c2415d330 2024-02-02T13:37:52.435 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.433+0000 7f9c23400640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9c140015f0 tx=0x7f9c14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.438 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.437+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c1c0548b0 con 0x7f9c2415d330 2024-02-02T13:37:52.665 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.664+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c241618c0 con 0x7f9c2415d330 2024-02-02T13:37:52.666 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.665+0000 7f9c1be00640 1 -- 172.21.15.91:0/416140759 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f9c1c057ed0 con 0x7f9c2415d330 2024-02-02T13:37:52.667 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:52.667 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":88,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T13:37:10.719224+0000","modified":"2024-02-02T13:37:13.738863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8835":{"gid":8835,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6835/115193290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":115193290},{"type":"v1","addr":"172.21.15.91:6835","nonce":115193290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-02-02T13:37:52.667 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 88 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 msgr2=0x7f9bfc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9c140015f0 tx=0x7f9c14009040 comp rx=0 tx=0).stop 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 msgr2=0x7f9c240cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c007e40 tx=0x7f9c1c00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 shutdown_connections 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f9bfc06a430 0x7f9bfc06c8f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.667+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c2415d330 0x7f9c240cc510 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.668+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240d3680 0x7f9c2415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.668+0000 7f9c2a200640 1 --2- 172.21.15.91:0/416140759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c240b2470 0x7f9c24148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.668+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 >> 172.21.15.91:0/416140759 conn(0x7f9c24070660 msgr2=0x7f9c240d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:52.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.668+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 shutdown_connections 2024-02-02T13:37:52.670 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.668+0000 7f9c2a200640 1 -- 172.21.15.91:0/416140759 wait complete. 2024-02-02T13:37:52.687 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:37:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.885+0000 7f399c200640 1 Processor -- start 2024-02-02T13:37:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.886+0000 7f399c200640 1 -- start start 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f399c200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f399415d650 0x7f39940cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f399c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39940ce070 0x7f39940ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f399c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940d12c0 con 0x7f39941610b0 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f399c200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940cea30 con 0x7f399415d650 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f399c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940cded0 con 0x7f39940ce070 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f3999800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f399415d650 0x7f39940cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f3999800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f399415d650 0x7f39940cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51420/0 (socket says 172.21.15.91:51420) 2024-02-02T13:37:52.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.887+0000 7f3999800640 1 -- 172.21.15.91:0/729841136 learned_addr learned my addr 172.21.15.91:0/729841136 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:52.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.888+0000 7f3999800640 1 -- 172.21.15.91:0/729841136 >> v2:172.21.15.113:3300/0 conn(0x7f39940ce070 msgr2=0x7f39940ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:52.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.888+0000 7f3999800640 1 --2- 172.21.15.91:0/729841136 >> v2:172.21.15.113:3300/0 conn(0x7f39940ce070 0x7f39940ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.888+0000 7f3999800640 1 -- 172.21.15.91:0/729841136 >> v1:172.21.15.113:6789/0 conn(0x7f39941610b0 legacy=0x7f3994161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:52.890 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.888+0000 7f3999800640 1 -- 172.21.15.91:0/729841136 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994071410 con 0x7f399415d650 2024-02-02T13:37:52.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.889+0000 7f3999800640 1 --2- 172.21.15.91:0/729841136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f39940cd990 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3994161010 tx=0x7f398402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aea514a0fc0a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.889+0000 7f3993e00640 1 -- 172.21.15.91:0/729841136 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398402ca80 con 0x7f399415d650 2024-02-02T13:37:52.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.889+0000 7f3993e00640 1 -- 172.21.15.91:0/729841136 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398402cbe0 con 0x7f399415d650 2024-02-02T13:37:52.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.889+0000 7f3993e00640 1 -- 172.21.15.91:0/729841136 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39840343f0 con 0x7f399415d650 2024-02-02T13:37:52.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 -- 172.21.15.91:0/729841136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 msgr2=0x7f39940cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 --2- 172.21.15.91:0/729841136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f39940cd990 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3994161010 tx=0x7f398402bee0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 -- 172.21.15.91:0/729841136 shutdown_connections 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 --2- 172.21.15.91:0/729841136 >> v2:172.21.15.113:3300/0 conn(0x7f39940ce070 0x7f39940ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 --2- 172.21.15.91:0/729841136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f39940cd990 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 -- 172.21.15.91:0/729841136 >> 172.21.15.91:0/729841136 conn(0x7f39940d30a0 msgr2=0x7f39940d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 -- 172.21.15.91:0/729841136 shutdown_connections 2024-02-02T13:37:52.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.890+0000 7f399c200640 1 -- 172.21.15.91:0/729841136 wait complete. 2024-02-02T13:37:52.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.892+0000 7f399c200640 1 Processor -- start 2024-02-02T13:37:52.893 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.892+0000 7f399c200640 1 -- start start 2024-02-02T13:37:52.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39940ce070 0x7f3994158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 0x7f399415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b1f70 con 0x7f399415d650 2024-02-02T13:37:52.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399415e6b0 con 0x7f39941610b0 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399415e990 con 0x7f39940ce070 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51434/0 (socket says 172.21.15.91:51434) 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 -- 172.21.15.91:0/1245273931 learned_addr learned my addr 172.21.15.91:0/1245273931 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3999800640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39940ce070 0x7f3994158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399a200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 0x7f399415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 -- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39940ce070 msgr2=0x7f3994158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39940ce070 0x7f3994158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 -- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 msgr2=0x7f399415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 0x7f399415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f3998e00640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009b30 con 0x7f399415d650 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.893+0000 7f399a200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 0x7f399415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f3998e00640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b410 tx=0x7f397c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c011830 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0119b0 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c011d60 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940b21c0 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.894+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3994158a40 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.895+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3994071a30 con 0x7f399415d650 2024-02-02T13:37:52.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.896+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f397c01a400 con 0x7f399415d650 2024-02-02T13:37:52.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.897+0000 7f3991600640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 0x7f396c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:52.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.898+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f397c088a60 con 0x7f399415d650 2024-02-02T13:37:52.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.898+0000 7f3999800640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 0x7f396c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:52.899 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.898+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7f396c071f60 con 0x7f399415d650 2024-02-02T13:37:52.900 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.899+0000 7f3999800640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 0x7f396c06c8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3984035000 tx=0x7f3984030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:52.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:52.902+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f397c015020 con 0x7f399415d650 2024-02-02T13:37:53.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.093+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39940b26e0 con 0x7f399415d650 2024-02-02T13:37:53.098 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.097+0000 7f3991600640 1 -- 172.21.15.91:0/1245273931 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16130 (secure 0 0 0) 0x7f39940b26e0 con 0x7f399415d650 2024-02-02T13:37:53.098 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:53.098 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":199,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:49.553575+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-02-02T13:37:08.724577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-02-02T13:37:09.746745+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:37:53.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.099+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 msgr2=0x7f396c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.099+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 0x7f396c06c8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3984035000 tx=0x7f3984030000 comp rx=0 tx=0).stop 2024-02-02T13:37:53.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 msgr2=0x7f399415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.101 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b410 tx=0x7f397c00b8d0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 shutdown_connections 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f396c06a430 0x7f396c06c8f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39941610b0 0x7f399415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f399415d650 0x7f399415a1f0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 --2- 172.21.15.91:0/1245273931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39940ce070 0x7f3994158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 >> 172.21.15.91:0/1245273931 conn(0x7f39940d30a0 msgr2=0x7f399415b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 shutdown_connections 2024-02-02T13:37:53.102 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.100+0000 7f399c200640 1 -- 172.21.15.91:0/1245273931 wait complete. 2024-02-02T13:37:53.120 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T13:37:53.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.327+0000 7f4e5d800640 1 Processor -- start 2024-02-02T13:37:53.329 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.328+0000 7f4e5d800640 1 -- start start 2024-02-02T13:37:53.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4e58160a50 0x7f4e580b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e58069f80 0x7f4e5806a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e5806a900 0x7f4e580cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.330 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580d1d10 con 0x7f4e58160a50 2024-02-02T13:37:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580cfbf0 con 0x7f4e58069f80 2024-02-02T13:37:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e5d800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b2670 con 0x7f4e5806a900 2024-02-02T13:37:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e56a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4e58160a50 0x7f4e580b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e56a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4e58160a50 0x7f4e580b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51444/0 (socket says 172.21.15.91:51444) 2024-02-02T13:37:53.331 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e56a00640 1 -- 172.21.15.91:0/2642987797 learned_addr learned my addr 172.21.15.91:0/2642987797 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e56000640 1 --2- 172.21.15.91:0/2642987797 >> v2:172.21.15.113:3300/0 conn(0x7f4e58069f80 0x7f4e5806a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.329+0000 7f4e57400640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e5806a900 0x7f4e580cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.330+0000 7f4e56a00640 1 -- 172.21.15.91:0/2642987797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e5806a900 msgr2=0x7f4e580cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.330+0000 7f4e56a00640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e5806a900 0x7f4e580cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.330+0000 7f4e56a00640 1 -- 172.21.15.91:0/2642987797 >> v2:172.21.15.113:3300/0 conn(0x7f4e58069f80 msgr2=0x7f4e5806a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.330+0000 7f4e56a00640 1 --2- 172.21.15.91:0/2642987797 >> v2:172.21.15.113:3300/0 conn(0x7f4e58069f80 0x7f4e5806a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.332 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.330+0000 7f4e56a00640 1 -- 172.21.15.91:0/2642987797 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5806ad90 con 0x7f4e58160a50 2024-02-02T13:37:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.331+0000 7f4e56a00640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58160a50 0x7f4e580b2130 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b3a60 tx=0x7f4e4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e8adb65428a861 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.331+0000 7f4e55600640 1 -- 172.21.15.91:0/2642987797 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004230 con 0x7f4e58160a50 2024-02-02T13:37:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.331+0000 7f4e55600640 1 -- 172.21.15.91:0/2642987797 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40004390 con 0x7f4e58160a50 2024-02-02T13:37:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.331+0000 7f4e55600640 1 -- 172.21.15.91:0/2642987797 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40003810 con 0x7f4e58160a50 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 -- 172.21.15.91:0/2642987797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58160a50 msgr2=0x7f4e580b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58160a50 0x7f4e580b2130 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b3a60 tx=0x7f4e4002b8f0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 -- 172.21.15.91:0/2642987797 shutdown_connections 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e5806a900 0x7f4e580cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 --2- 172.21.15.91:0/2642987797 >> v2:172.21.15.113:3300/0 conn(0x7f4e58069f80 0x7f4e5806a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 --2- 172.21.15.91:0/2642987797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58160a50 0x7f4e580b2130 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 -- 172.21.15.91:0/2642987797 >> 172.21.15.91:0/2642987797 conn(0x7f4e580d0ee0 msgr2=0x7f4e580d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 -- 172.21.15.91:0/2642987797 shutdown_connections 2024-02-02T13:37:53.334 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.332+0000 7f4e5d800640 1 -- 172.21.15.91:0/2642987797 wait complete. 2024-02-02T13:37:53.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.334+0000 7f4e5d800640 1 Processor -- start 2024-02-02T13:37:53.335 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.334+0000 7f4e5d800640 1 -- start start 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5806a900 0x7f4e5815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e58160a50 0x7f4e5815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b4750 con 0x7f4e58069f80 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815f830 con 0x7f4e5806a900 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e5d800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815fb10 con 0x7f4e58160a50 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51454/0 (socket says 172.21.15.91:51454) 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e56a00640 1 -- 172.21.15.91:0/3479973245 learned_addr learned my addr 172.21.15.91:0/3479973245 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e57400640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e58160a50 0x7f4e5815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.337 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.335+0000 7f4e56000640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5806a900 0x7f4e5815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 -- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e58160a50 msgr2=0x7f4e5815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e58160a50 0x7f4e5815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 -- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5806a900 msgr2=0x7f4e5815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5806a900 0x7f4e5815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009460 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e56a00640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40003e50 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40004500 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5815fdf0 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.336+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40006280 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.337+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5815f4b0 con 0x7f4e58069f80 2024-02-02T13:37:53.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.338+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5815cb00 con 0x7f4e58069f80 2024-02-02T13:37:53.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.338+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4e4002f050 con 0x7f4e58069f80 2024-02-02T13:37:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.339+0000 7f4e4ea00640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 0x7f4e3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.339+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e400aac20 con 0x7f4e58069f80 2024-02-02T13:37:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.340+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=200}) v3 -- 0x7f4e30071f60 con 0x7f4e58069f80 2024-02-02T13:37:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.340+0000 7f4e56000640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 0x7f4e3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.340+0000 7f4e56000640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 0x7f4e3006c8f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4e581f5d80 tx=0x7f4e480039f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.345 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.344+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4e4003f060 con 0x7f4e58069f80 2024-02-02T13:37:53.576 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.575+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4e58066cf0 con 0x7f4e58069f80 2024-02-02T13:37:53.578 INFO:tasks.ceph.mon.a.smithi091.stderr:2024-02-02T13:37:53.577+0000 7faab1400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T13:37:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.580+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(200..200 src has 1..200) v4 ==== 374+0+0 (secure 0 0 0) 0x7f4e400b1080 con 0x7f4e58069f80 2024-02-02T13:37:53.581 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.580+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f4e30072810 con 0x7f4e58069f80 2024-02-02T13:37:53.598 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.597+0000 7f4e4ea00640 1 -- 172.21.15.91:0/3479973245 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4e40049090 con 0x7f4e58069f80 2024-02-02T13:37:53.599 INFO:teuthology.orchestra.run.smithi091.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 msgr2=0x7f4e3006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 0x7f4e3006c8f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4e581f5d80 tx=0x7f4e480039f0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 msgr2=0x7f4e5815ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 shutdown_connections 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f4e3006a430 0x7f4e3006c8f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e58160a50 0x7f4e5815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5806a900 0x7f4e5815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 --2- 172.21.15.91:0/3479973245 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e58069f80 0x7f4e5815ef70 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 >> 172.21.15.91:0/3479973245 conn(0x7f4e580d0ee0 msgr2=0x7f4e581e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 shutdown_connections 2024-02-02T13:37:53.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.600+0000 7f4e5d800640 1 -- 172.21.15.91:0/3479973245 wait complete. 2024-02-02T13:37:53.616 INFO:tasks.ceph.mds.a.smithi091.stderr: -9088> 2024-02-02T13:37:16.220+0000 7f3321600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T13:37:53.619 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T13:37:53.683 INFO:tasks.ceph.mds.a.smithi091.stdout:starting mds.a at 2024-02-02T13:37:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.814+0000 7fe188e00640 1 Processor -- start 2024-02-02T13:37:53.815 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.814+0000 7fe188e00640 1 -- start start 2024-02-02T13:37:53.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.815+0000 7fe188e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe18415b2f0 0x7fe1840b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.815+0000 7fe188e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe1840b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.815+0000 7fe188e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18415f1b0 con 0x7fe184071650 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.815+0000 7fe188e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18415fdc0 con 0x7fe18415b2f0 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.815+0000 7fe188e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415b6d0 con 0x7fe1840711f0 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe18415b2f0 0x7fe1840b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe18415b2f0 0x7fe1840b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51470/0 (socket says 172.21.15.91:51470) 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 -- 172.21.15.91:0/258889680 learned_addr learned my addr 172.21.15.91:0/258889680 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe179200640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe1840b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 -- 172.21.15.91:0/258889680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 msgr2=0x7fe1840b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe1840b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 -- 172.21.15.91:0/258889680 >> v1:172.21.15.113:6789/0 conn(0x7fe184071650 legacy=0x7fe1840c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:53.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.816+0000 7fe181e00640 1 -- 172.21.15.91:0/258889680 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18415f490 con 0x7fe18415b2f0 2024-02-02T13:37:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.817+0000 7fe181e00640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe18415b2f0 0x7fe1840b1a00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fe174009e80 tx=0x7fe17402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24576e733ecb6ed1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.817+0000 7fe181400640 1 -- 172.21.15.91:0/258889680 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17402c9c0 con 0x7fe18415b2f0 2024-02-02T13:37:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe181400640 1 -- 172.21.15.91:0/258889680 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17402cb20 con 0x7fe18415b2f0 2024-02-02T13:37:53.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe181400640 1 -- 172.21.15.91:0/258889680 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740343f0 con 0x7fe18415b2f0 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 -- 172.21.15.91:0/258889680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe18415b2f0 msgr2=0x7fe1840b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe18415b2f0 0x7fe1840b1a00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fe174009e80 tx=0x7fe17402be20 comp rx=0 tx=0).stop 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 -- 172.21.15.91:0/258889680 shutdown_connections 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe1840b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 --2- 172.21.15.91:0/258889680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe18415b2f0 0x7fe1840b1a00 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 -- 172.21.15.91:0/258889680 >> 172.21.15.91:0/258889680 conn(0x7fe1840ceb90 msgr2=0x7fe1840cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:53.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 -- 172.21.15.91:0/258889680 shutdown_connections 2024-02-02T13:37:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.818+0000 7fe188e00640 1 -- 172.21.15.91:0/258889680 wait complete. 2024-02-02T13:37:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.820+0000 7fe188e00640 1 Processor -- start 2024-02-02T13:37:53.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.820+0000 7fe188e00640 1 -- start start 2024-02-02T13:37:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe184153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe18415b2f0 0x7fe1841f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18414f8e0 con 0x7fe184071650 2024-02-02T13:37:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415bee0 con 0x7fe18415b2f0 2024-02-02T13:37:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe188e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415c1c0 con 0x7fe1840711f0 2024-02-02T13:37:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.821+0000 7fe179200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51478/0 (socket says 172.21.15.91:51478) 2024-02-02T13:37:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 -- 172.21.15.91:0/3774654573 learned_addr learned my addr 172.21.15.91:0/3774654573 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe181e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe184153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 -- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 msgr2=0x7fe184153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe184153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 -- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe18415b2f0 msgr2=0x7fe1841f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe18415b2f0 0x7fe1841f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009b30 con 0x7fe184071650 2024-02-02T13:37:53.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe181e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe184153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe179200640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d5f0 tx=0x7fe16c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.822+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0125f0 con 0x7fe184071650 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.823+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c012750 con 0x7fe184071650 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.823+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18415c4a0 con 0x7fe184071650 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.823+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c012b00 con 0x7fe184071650 2024-02-02T13:37:53.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.823+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1840c5280 con 0x7fe184071650 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.824+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18415fa30 con 0x7fe184071650 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.825+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe16c012c60 con 0x7fe184071650 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.825+0000 7fe17aa00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 0x7fe15c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.826+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe16c088dc0 con 0x7fe184071650 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.826+0000 7fe181e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 0x7fe15c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.826+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fe15c072070 con 0x7fe184071650 2024-02-02T13:37:53.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.827+0000 7fe181e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 0x7fe15c06c8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe174035000 tx=0x7fe174030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:53.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:53.830+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe18415fa30 con 0x7fe184071650 2024-02-02T13:37:54.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.056+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe18415c9c0 con 0x7fe184071650 2024-02-02T13:37:54.669 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.668+0000 7fe17aa00640 1 -- 172.21.15.91:0/3774654573 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe16c054350 con 0x7fe184071650 2024-02-02T13:37:54.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 msgr2=0x7fe15c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 0x7fe15c06c8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe174035000 tx=0x7fe174030000 comp rx=0 tx=0).stop 2024-02-02T13:37:54.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 msgr2=0x7fe18415b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.672 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d5f0 tx=0x7fe16c00dab0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 shutdown_connections 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe15c06a430 0x7fe15c06c8f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe18415b2f0 0x7fe1841f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe184071650 0x7fe18415b850 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 --2- 172.21.15.91:0/3774654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1840711f0 0x7fe184153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 >> 172.21.15.91:0/3774654573 conn(0x7fe1840ceb90 msgr2=0x7fe1840ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.671+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 shutdown_connections 2024-02-02T13:37:54.673 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.672+0000 7fe188e00640 1 -- 172.21.15.91:0/3774654573 wait complete. 2024-02-02T13:37:54.695 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T13:37:54.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.889+0000 7fe4e2400640 1 Processor -- start 2024-02-02T13:37:54.891 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.889+0000 7fe4e2400640 1 -- start start 2024-02-02T13:37:54.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4e2400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4dc151f90 0x7fe4dc15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc06f4f0 0x7fe4dc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4e2400640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc070f80 con 0x7fe4dc0b0c40 2024-02-02T13:37:54.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4e2400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc0b10d0 con 0x7fe4dc151f90 2024-02-02T13:37:54.892 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4e2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc152370 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4dbe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe4dc0b0c40 0x7fe4dc0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50694/0 (socket says 172.21.15.91:50694) 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4dbe00640 1 -- 172.21.15.91:0/2306399753 learned_addr learned my addr 172.21.15.91:0/2306399753 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.891+0000 7fe4db400640 1 --2- 172.21.15.91:0/2306399753 >> v2:172.21.15.91:3300/0 conn(0x7fe4dc151f90 0x7fe4dc15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4d2800640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc06f4f0 0x7fe4dc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4daa00640 1 -- 172.21.15.91:0/2306399753 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252849768 0 0) 0x7fe4dc070f80 con 0x7fe4dc0b0c40 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4daa00640 1 -- 172.21.15.91:0/2306399753 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4bc003520 con 0x7fe4dc0b0c40 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4db400640 1 -- 172.21.15.91:0/2306399753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc06f4f0 msgr2=0x7fe4dc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4db400640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc06f4f0 0x7fe4dc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4db400640 1 -- 172.21.15.91:0/2306399753 >> v1:172.21.15.91:6789/0 conn(0x7fe4dc0b0c40 legacy=0x7fe4dc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.892+0000 7fe4db400640 1 -- 172.21.15.91:0/2306399753 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0cb010 con 0x7fe4dc151f90 2024-02-02T13:37:54.895 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.894+0000 7fe4db400640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc151f90 0x7fe4dc15f5a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0097a0 tx=0x7fe4cc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646996ca8eb0e547 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:54.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.894+0000 7fe4daa00640 1 -- 172.21.15.91:0/2306399753 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc02d040 con 0x7fe4dc151f90 2024-02-02T13:37:54.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.894+0000 7fe4daa00640 1 -- 172.21.15.91:0/2306399753 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc031810 con 0x7fe4dc151f90 2024-02-02T13:37:54.896 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.894+0000 7fe4daa00640 1 -- 172.21.15.91:0/2306399753 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc031ba0 con 0x7fe4dc151f90 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 -- 172.21.15.91:0/2306399753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc151f90 msgr2=0x7fe4dc15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc151f90 0x7fe4dc15f5a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0097a0 tx=0x7fe4cc02b8b0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 -- 172.21.15.91:0/2306399753 shutdown_connections 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc06f4f0 0x7fe4dc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 --2- 172.21.15.91:0/2306399753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc151f90 0x7fe4dc15f5a0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 -- 172.21.15.91:0/2306399753 >> 172.21.15.91:0/2306399753 conn(0x7fe4dc0cec80 msgr2=0x7fe4dc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 -- 172.21.15.91:0/2306399753 shutdown_connections 2024-02-02T13:37:54.897 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.895+0000 7fe4e2400640 1 -- 172.21.15.91:0/2306399753 wait complete. 2024-02-02T13:37:54.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.897+0000 7fe4e2400640 1 Processor -- start 2024-02-02T13:37:54.898 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.897+0000 7fe4e2400640 1 -- start start 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0b0c40 0x7fe4dc159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc151f90 0x7fe4dc151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc159fe0 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15bf20 con 0x7fe4dc0b0c40 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4e2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15c150 con 0x7fe4dc151f90 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4db400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4db400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51504/0 (socket says 172.21.15.91:51504) 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.898+0000 7fe4db400640 1 -- 172.21.15.91:0/4075141311 learned_addr learned my addr 172.21.15.91:0/4075141311 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 -- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc151f90 msgr2=0x7fe4dc151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.901 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc151f90 0x7fe4dc151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 -- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0b0c40 msgr2=0x7fe4dc159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0b0c40 0x7fe4dc159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009400 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4db400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098c0 tx=0x7fe4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc03ad60 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc044400 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0445b0 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc15c430 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.902 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.899+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc15c9f0 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.900+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc066cf0 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.901+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe4cc04d440 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.902+0000 7fe4d3e00640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 0x7fe4b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:54.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.902+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe4cc0aac70 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.902+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fe4b4072070 con 0x7fe4dc06f4f0 2024-02-02T13:37:54.904 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.902+0000 7fe4d2800640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 0x7fe4b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:54.905 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.903+0000 7fe4d2800640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 0x7fe4b406c8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006a90 tx=0x7fe4c4006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:54.908 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:54.907+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe4cc035060 con 0x7fe4dc06f4f0 2024-02-02T13:37:55.100 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.099+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe4dc06c7b0 con 0x7fe4dc06f4f0 2024-02-02T13:37:55.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.685+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(201..201 src has 1..201) v4 ==== 296+0+0 (secure 0 0 0) 0x7fe4cc07a5b0 con 0x7fe4dc06f4f0 2024-02-02T13:37:55.687 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.686+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=202}) v3 -- 0x7fe4b40704a0 con 0x7fe4dc06f4f0 2024-02-02T13:37:55.697 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.695+0000 7fe4d3e00640 1 -- 172.21.15.91:0/4075141311 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v201) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe4cc079c80 con 0x7fe4dc06f4f0 2024-02-02T13:37:55.698 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_metadata' removed 2024-02-02T13:37:55.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.701+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 msgr2=0x7fe4b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:55.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.701+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 0x7fe4b406c8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006a90 tx=0x7fe4c4006a20 comp rx=0 tx=0).stop 2024-02-02T13:37:55.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.701+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 msgr2=0x7fe4dc0c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:55.703 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.701+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098c0 tx=0x7fe4cc0046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 shutdown_connections 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe4b406a430 0x7fe4b406c8f0 secure :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006a90 tx=0x7fe4c4006a20 comp rx=0 tx=0).stop 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4dc151f90 0x7fe4dc151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0b0c40 0x7fe4dc159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 --2- 172.21.15.91:0/4075141311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4dc06f4f0 0x7fe4dc0c8540 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 >> 172.21.15.91:0/4075141311 conn(0x7fe4dc0cec80 msgr2=0x7fe4dc159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 shutdown_connections 2024-02-02T13:37:55.704 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.702+0000 7fe4e2400640 1 -- 172.21.15.91:0/4075141311 wait complete. 2024-02-02T13:37:55.725 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T13:37:55.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.929+0000 7f89a3e00640 1 Processor -- start 2024-02-02T13:37:55.931 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.930+0000 7f89a3e00640 1 -- start start 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f89a40704e0 0x7f89a40b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40b2300 0x7f89a40d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89a40708c0 0x7f89a40d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40b6070 con 0x7f89a40708c0 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a4071650 con 0x7f89a40704e0 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40d0cb0 con 0x7f89a40b2300 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89a40708c0 0x7f89a40d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89a40708c0 0x7f89a40d16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51514/0 (socket says 172.21.15.91:51514) 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a3400640 1 -- 172.21.15.91:0/1686725626 learned_addr learned my addr 172.21.15.91:0/1686725626 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a2000640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40b2300 0x7f89a40d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.931+0000 7f89a2a00640 1 --2- 172.21.15.91:0/1686725626 >> v2:172.21.15.113:3300/0 conn(0x7f89a40704e0 0x7f89a40b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.932+0000 7f89a3400640 1 -- 172.21.15.91:0/1686725626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40b2300 msgr2=0x7f89a40d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:55.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.932+0000 7f89a3400640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40b2300 0x7f89a40d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.932+0000 7f89a3400640 1 -- 172.21.15.91:0/1686725626 >> v2:172.21.15.113:3300/0 conn(0x7f89a40704e0 msgr2=0x7f89a40b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.932+0000 7f89a3400640 1 --2- 172.21.15.91:0/1686725626 >> v2:172.21.15.113:3300/0 conn(0x7f89a40704e0 0x7f89a40b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.932+0000 7f89a3400640 1 -- 172.21.15.91:0/1686725626 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40cdfc0 con 0x7f89a40708c0 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.933+0000 7f89a3400640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40d16d0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f89a408b640 tx=0x7f899c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3fcd47ee7df2fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.933+0000 7f89a1600640 1 -- 172.21.15.91:0/1686725626 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c031750 con 0x7f89a40708c0 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.933+0000 7f89a1600640 1 -- 172.21.15.91:0/1686725626 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c0318b0 con 0x7f89a40708c0 2024-02-02T13:37:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.933+0000 7f89a1600640 1 -- 172.21.15.91:0/1686725626 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c031c40 con 0x7f89a40708c0 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 -- 172.21.15.91:0/1686725626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 msgr2=0x7f89a40d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40d16d0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f89a408b640 tx=0x7f899c02b7f0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 -- 172.21.15.91:0/1686725626 shutdown_connections 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40d16d0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1686725626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40b2300 0x7f89a40d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1686725626 >> v2:172.21.15.113:3300/0 conn(0x7f89a40704e0 0x7f89a40b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 -- 172.21.15.91:0/1686725626 >> 172.21.15.91:0/1686725626 conn(0x7f89a40cfbf0 msgr2=0x7f89a40d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 -- 172.21.15.91:0/1686725626 shutdown_connections 2024-02-02T13:37:55.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.934+0000 7f89a3e00640 1 -- 172.21.15.91:0/1686725626 wait complete. 2024-02-02T13:37:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.936+0000 7f89a3e00640 1 Processor -- start 2024-02-02T13:37:55.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.936+0000 7f89a3e00640 1 -- start start 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40704e0 0x7f89a40c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89a40b2300 0x7f89a40b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40cc820 con 0x7f89a40708c0 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a41572e0 con 0x7f89a40b2300 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40cca70 con 0x7f89a40704e0 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51522/0 (socket says 172.21.15.91:51522) 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 -- 172.21.15.91:0/1667930301 learned_addr learned my addr 172.21.15.91:0/1667930301 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:55.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2a00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40704e0 0x7f89a40c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a3400640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89a40b2300 0x7f89a40b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 -- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40704e0 msgr2=0x7f89a40c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40704e0 0x7f89a40c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 -- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89a40b2300 msgr2=0x7f89a40b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89a40b2300 0x7f89a40b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.937+0000 7f89a2000640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c0094d0 con 0x7f89a40708c0 2024-02-02T13:37:55.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f89a2000640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8994007e30 tx=0x7f899400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994014620 con 0x7f89a40708c0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400ee90 con 0x7f89a40708c0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940149c0 con 0x7f89a40708c0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a40ccd50 con 0x7f89a40708c0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.938+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a40ca6f0 con 0x7f89a40708c0 2024-02-02T13:37:55.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.939+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a404dff0 con 0x7f89a40708c0 2024-02-02T13:37:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.940+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8994014b20 con 0x7f89a40708c0 2024-02-02T13:37:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.941+0000 7f899aa00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 0x7f897c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:55.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.941+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8994089150 con 0x7f89a40708c0 2024-02-02T13:37:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.941+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=202}) v3 -- 0x7f897c071200 con 0x7f89a40708c0 2024-02-02T13:37:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.941+0000 7f89a2a00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 0x7f897c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:55.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.942+0000 7f89a2a00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 0x7f897c06c8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f898c006fd0 tx=0x7f898c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:55.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:55.946+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8994054890 con 0x7f89a40708c0 2024-02-02T13:37:56.139 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.138+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89a40d07e0 con 0x7f89a40708c0 2024-02-02T13:37:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.697+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 7 ==== osd_map(202..202 src has 1..202) v4 ==== 296+0+0 (secure 0 0 0) 0x7f8994011260 con 0x7f89a40708c0 2024-02-02T13:37:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.697+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f897c06ff90 con 0x7f89a40708c0 2024-02-02T13:37:56.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.710+0000 7f899aa00640 1 -- 172.21.15.91:0/1667930301 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v202) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8994057eb0 con 0x7f89a40708c0 2024-02-02T13:37:56.716 INFO:teuthology.orchestra.run.smithi091.stderr:pool 'cephfs_data' removed 2024-02-02T13:37:56.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 msgr2=0x7f897c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 0x7f897c06c8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f898c006fd0 tx=0x7f898c008040 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 msgr2=0x7f89a40ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8994007e30 tx=0x7f899400e4b0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 shutdown_connections 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f897c06a430 0x7f897c06c8f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89a40b2300 0x7f89a40b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89a40708c0 0x7f89a40ca1b0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 --2- 172.21.15.91:0/1667930301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89a40704e0 0x7f89a40c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.718+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 >> 172.21.15.91:0/1667930301 conn(0x7f89a40cfbf0 msgr2=0x7f89a4156310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:56.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.719+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 shutdown_connections 2024-02-02T13:37:56.721 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.719+0000 7f89a3e00640 1 -- 172.21.15.91:0/1667930301 wait complete. 2024-02-02T13:37:56.744 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-02T13:37:56.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.930+0000 7f5463000640 1 Processor -- start 2024-02-02T13:37:56.932 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.931+0000 7f5463000640 1 -- start start 2024-02-02T13:37:56.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5463000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f545c15b2f0 0x7f545c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.933 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5463000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f545c0b0da0 0x7f545c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5463000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c15f1b0 con 0x7f545c0711f0 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5463000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c15fdc0 con 0x7f545c0b0da0 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5463000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c15b6d0 con 0x7f545c15b2f0 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5461000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f545c0b0da0 0x7f545c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5453200640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f545c0711f0 0x7f545c0b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50702/0 (socket says 172.21.15.91:50702) 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f5453200640 1 -- 172.21.15.91:0/168621356 learned_addr learned my addr 172.21.15.91:0/168621356 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:56.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.932+0000 7f545be00640 1 --2- 172.21.15.91:0/168621356 >> v2:172.21.15.113:3300/0 conn(0x7f545c15b2f0 0x7f545c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:56.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.933+0000 7f5461000640 1 -- 172.21.15.91:0/168621356 >> v2:172.21.15.113:3300/0 conn(0x7f545c15b2f0 msgr2=0x7f545c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:56.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.933+0000 7f5461000640 1 --2- 172.21.15.91:0/168621356 >> v2:172.21.15.113:3300/0 conn(0x7f545c15b2f0 0x7f545c0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.933+0000 7f5461000640 1 -- 172.21.15.91:0/168621356 >> v1:172.21.15.91:6789/0 conn(0x7f545c0711f0 legacy=0x7f545c0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.933+0000 7f5461000640 1 -- 172.21.15.91:0/168621356 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c15f490 con 0x7f545c0b0da0 2024-02-02T13:37:56.935 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.933+0000 7f545b400640 1 -- 172.21.15.91:0/168621356 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150821405 0 0) 0x7f545c15f1b0 con 0x7f545c0711f0 2024-02-02T13:37:56.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.934+0000 7f5461000640 1 --2- 172.21.15.91:0/168621356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c0b0da0 0x7f545c0c9480 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f545c0b4890 tx=0x7f545402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f70e7a27d2bd06f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:56.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.934+0000 7f545b400640 1 -- 172.21.15.91:0/168621356 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545402e070 con 0x7f545c0b0da0 2024-02-02T13:37:56.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.934+0000 7f545b400640 1 -- 172.21.15.91:0/168621356 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54540315a0 con 0x7f545c0b0da0 2024-02-02T13:37:56.936 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.934+0000 7f545b400640 1 -- 172.21.15.91:0/168621356 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454031930 con 0x7f545c0b0da0 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 -- 172.21.15.91:0/168621356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c0b0da0 msgr2=0x7f545c0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 --2- 172.21.15.91:0/168621356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c0b0da0 0x7f545c0c9480 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f545c0b4890 tx=0x7f545402b840 comp rx=0 tx=0).stop 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 -- 172.21.15.91:0/168621356 shutdown_connections 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 --2- 172.21.15.91:0/168621356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c0b0da0 0x7f545c0c9480 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 --2- 172.21.15.91:0/168621356 >> v2:172.21.15.113:3300/0 conn(0x7f545c15b2f0 0x7f545c0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 -- 172.21.15.91:0/168621356 >> 172.21.15.91:0/168621356 conn(0x7f545c0ceb90 msgr2=0x7f545c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 -- 172.21.15.91:0/168621356 shutdown_connections 2024-02-02T13:37:56.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.935+0000 7f5463000640 1 -- 172.21.15.91:0/168621356 wait complete. 2024-02-02T13:37:56.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.936+0000 7f5463000640 1 Processor -- start 2024-02-02T13:37:56.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 -- start start 2024-02-02T13:37:56.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 0x7f545c1f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f545c0b0da0 0x7f545c0c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c1f0a50 con 0x7f545c15b2f0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0c80b0 con 0x7f545c0711f0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.937+0000 7f5463000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0c8390 con 0x7f545c0b0da0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51544/0 (socket says 172.21.15.91:51544) 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 -- 172.21.15.91:0/1454150369 learned_addr learned my addr 172.21.15.91:0/1454150369 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f545be00640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 0x7f545c1f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 -- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f545c0b0da0 msgr2=0x7f545c0c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f545c0b0da0 0x7f545c0c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 -- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 msgr2=0x7f545c1f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 0x7f545c1f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 -- 172.21.15.91:0/1454150369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454009400 con 0x7f545c15b2f0 2024-02-02T13:37:56.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f545be00640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 0x7f545c1f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.938+0000 7f5461000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5454034040 tx=0x7f54540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.939+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454032840 con 0x7f545c15b2f0 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.939+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54540329a0 con 0x7f545c15b2f0 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.939+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454032d50 con 0x7f545c15b2f0 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.939+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0c8670 con 0x7f545c15b2f0 2024-02-02T13:37:56.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.939+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c0c8950 con 0x7f545c15b2f0 2024-02-02T13:37:56.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.940+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c1f0e80 con 0x7f545c15b2f0 2024-02-02T13:37:56.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.941+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f545404d3f0 con 0x7f545c15b2f0 2024-02-02T13:37:56.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.942+0000 7f5458c00640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:56.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.942+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54540aa1f0 con 0x7f545c15b2f0 2024-02-02T13:37:56.943 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.942+0000 7f545be00640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:56.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.943+0000 7f545be00640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 0x7f543406c8f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f545c15eb90 tx=0x7f544c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:56.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:56.946+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f545c1f0e80 con 0x7f545c15b2f0 2024-02-02T13:37:57.135 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.133+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f545c1f0e80 con 0x7f545c15b2f0 2024-02-02T13:37:57.143 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.142+0000 7f5458c00640 1 -- 172.21.15.91:0/1454150369 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f545c1f0e80 con 0x7f545c15b2f0 2024-02-02T13:37:57.146 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 msgr2=0x7f543406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 0x7f543406c8f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f545c15eb90 tx=0x7f544c00b040 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 msgr2=0x7f545c15cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5454034040 tx=0x7f54540046c0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 shutdown_connections 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f545c0b0da0 0x7f545c0c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f545c15b2f0 0x7f545c15cc80 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 --2- 172.21.15.91:0/1454150369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f545c0711f0 0x7f545c1f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 >> 172.21.15.91:0/1454150369 conn(0x7f545c0ceb90 msgr2=0x7f545c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 shutdown_connections 2024-02-02T13:37:57.147 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.145+0000 7f5463000640 1 -- 172.21.15.91:0/1454150369 wait complete. 2024-02-02T13:37:57.165 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-02-02T13:37:57.357 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.356+0000 7fb808200640 1 Processor -- start 2024-02-02T13:37:57.358 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.356+0000 7fb808200640 1 -- start start 2024-02-02T13:37:57.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.357+0000 7fb808200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 0x7fb8000c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.357+0000 7fb808200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8000b0700 0x7fb8000b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.357+0000 7fb808200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8000cc7c0 con 0x7fb80015a010 2024-02-02T13:37:57.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.357+0000 7fb808200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8000b10d0 con 0x7fb8000b0700 2024-02-02T13:37:57.359 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.357+0000 7fb808200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000ce930 con 0x7fb8000ce130 2024-02-02T13:37:57.360 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.358+0000 7fb806200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8000b0700 0x7fb8000b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.358+0000 7fb806200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8000b0700 0x7fb8000b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.91:43438/0 (socket says 172.21.15.91:43438) 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.358+0000 7fb806200640 1 -- 172.21.15.91:0/2304941167 learned_addr learned my addr 172.21.15.91:0/2304941167 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.358+0000 7fb804e00640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 0x7fb8000c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb7ffe00640 1 -- 172.21.15.91:0/2304941167 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064660541 0 0) 0x7fb8000cc7c0 con 0x7fb80015a010 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb7ffe00640 1 -- 172.21.15.91:0/2304941167 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e0003520 con 0x7fb80015a010 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb806200640 1 -- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 msgr2=0x7fb8000c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb806200640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 0x7fb8000c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb806200640 1 -- 172.21.15.91:0/2304941167 >> v1:172.21.15.113:6789/0 conn(0x7fb80015a010 legacy=0x7fb8000ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb806200640 1 -- 172.21.15.91:0/2304941167 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000cd640 con 0x7fb8000b0700 2024-02-02T13:37:57.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.359+0000 7fb804e00640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 0x7fb8000c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:37:57.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.360+0000 7fb806200640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000b0700 0x7fb8000b0b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009e50 tx=0x7fb7f802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be601471323ecdc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:57.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.361+0000 7fb7ffe00640 1 -- 172.21.15.91:0/2304941167 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f802cb00 con 0x7fb8000b0700 2024-02-02T13:37:57.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.361+0000 7fb7ffe00640 1 -- 172.21.15.91:0/2304941167 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f802cc60 con 0x7fb8000b0700 2024-02-02T13:37:57.362 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.361+0000 7fb7ffe00640 1 -- 172.21.15.91:0/2304941167 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80343f0 con 0x7fb8000b0700 2024-02-02T13:37:57.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 -- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000b0700 msgr2=0x7fb8000b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000b0700 0x7fb8000b0b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009e50 tx=0x7fb7f802bf60 comp rx=0 tx=0).stop 2024-02-02T13:37:57.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 -- 172.21.15.91:0/2304941167 shutdown_connections 2024-02-02T13:37:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000b0700 0x7fb8000b0b90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 --2- 172.21.15.91:0/2304941167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000ce130 0x7fb8000c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 -- 172.21.15.91:0/2304941167 >> 172.21.15.91:0/2304941167 conn(0x7fb8000cec80 msgr2=0x7fb8000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 -- 172.21.15.91:0/2304941167 shutdown_connections 2024-02-02T13:37:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.362+0000 7fb808200640 1 -- 172.21.15.91:0/2304941167 wait complete. 2024-02-02T13:37:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.363+0000 7fb808200640 1 Processor -- start 2024-02-02T13:37:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.364+0000 7fb808200640 1 -- start start 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.364+0000 7fb808200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 0x7fb8001575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.364+0000 7fb808200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb808200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80015a010 0x7fb80015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb808200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800157ae0 con 0x7fb8000ce130 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb808200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000c3660 con 0x7fb80015a010 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb808200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000c3940 con 0x7fb8000b0700 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51552/0 (socket says 172.21.15.91:51552) 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 -- 172.21.15.91:0/1187894281 learned_addr learned my addr 172.21.15.91:0/1187894281 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb806200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80015a010 0x7fb80015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb805800640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 0x7fb8001575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 -- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 msgr2=0x7fb8001575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 0x7fb8001575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 -- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80015a010 msgr2=0x7fb80015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80015a010 0x7fb80015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb804e00640 1 -- 172.21.15.91:0/1187894281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8009400 con 0x7fb8000ce130 2024-02-02T13:37:57.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.365+0000 7fb805800640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 0x7fb8001575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:57.368 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb804e00640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800ebe0 tx=0x7fb7e800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8009760 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800cc30 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800eee0 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f8009b30 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.366+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8000c3f00 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.367+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb800066cf0 con 0x7fb8000ce130 2024-02-02T13:37:57.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.368+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb7e8013470 con 0x7fb8000ce130 2024-02-02T13:37:57.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.369+0000 7fb7fd600640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 0x7fb7d806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:57.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.369+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb7e8088930 con 0x7fb8000ce130 2024-02-02T13:37:57.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.369+0000 7fb805800640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 0x7fb7d806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:57.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.370+0000 7fb805800640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 0x7fb7d806c920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0001610 tx=0x7fb7f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:57.374 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.373+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb7e8054210 con 0x7fb8000ce130 2024-02-02T13:37:57.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.631+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7fb800073e10 con 0x7fb8000ce130 2024-02-02T13:37:57.706 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.704+0000 7fb7fd600640 1 -- 172.21.15.91:0/1187894281 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v633) v1 ==== 135+0+0 (secure 0 0 0) 0x7fb7e8057830 con 0x7fb8000ce130 2024-02-02T13:37:57.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.707+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 msgr2=0x7fb7d806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.709 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.707+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 0x7fb7d806c920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0001610 tx=0x7fb7f0009040 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.707+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 msgr2=0x7fb8000c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.707+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800ebe0 tx=0x7fb7e800c520 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 shutdown_connections 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fb7d806a460 0x7fb7d806c920 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80015a010 0x7fb80015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8000ce130 0x7fb8000c5cc0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 --2- 172.21.15.91:0/1187894281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8000b0700 0x7fb8001575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 >> 172.21.15.91:0/1187894281 conn(0x7fb8000cec80 msgr2=0x7fb8000ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 shutdown_connections 2024-02-02T13:37:57.710 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:57.708+0000 7fb808200640 1 -- 172.21.15.91:0/1187894281 wait complete. 2024-02-02T13:37:57.728 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-02-02T13:37:57.728 INFO:tasks.cephfs_test_runner: 2024-02-02T13:37:57.728 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-02-02T13:37:57.729 INFO:tasks.cephfs_test_runner:Ran 10 tests in 758.297s 2024-02-02T13:37:57.729 INFO:tasks.cephfs_test_runner: 2024-02-02T13:37:57.729 INFO:tasks.cephfs_test_runner:OK 2024-02-02T13:37:57.729 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-02-02T13:37:57.740 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-02T13:37:57.748 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-02T13:37:57.748 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:57.748 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:57.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:37:57.797 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2024-02-02T13:37:57.797 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:57.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053842,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-02-02T13:37:57.842 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:57.843 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:57.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:37:57.901 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-02T13:37:57.901 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T13:37:57.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053849,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-02T13:37:57.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:57.956 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:37:58.014 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T13:37:58.014 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-02T13:37:58.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.015 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T13:37:58.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.110 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-02T13:37:58.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.198 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2024-02-02T13:37:58.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.337 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2024-02-02T13:37:58.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.382 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2024-02-02T13:37:58.515 INFO:teuthology.orchestra.run.smithi091.stderr:Cannot find device "ceph-brx" 2024-02-02T13:37:58.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:37:58.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.519 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:37:58.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-02T13:37:58.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-02-02T13:37:58.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-02T13:37:58.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-02T13:37:58.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-02-02T13:37:58.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.894 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-02-02T13:37:58.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T13:37:58.937 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-02-02T13:37:59.066 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-02-02T13:37:59.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:37:59.070 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-02-02T13:37:59.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-02T13:37:59.080 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:37:59.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.301+0000 7f06c1a00640 1 Processor -- start 2024-02-02T13:37:59.302 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.301+0000 7f06c1a00640 1 -- start start 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06c1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc15a010 0x7f06bc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06c1a00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f06bc0ce130 0x7f06bc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06c1a00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0cc7c0 con 0x7f06bc0cd470 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06c1a00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0ce930 con 0x7f06bc0ce130 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06c1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc06c890 con 0x7f06bc15a010 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06bb400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f06bc0cd470 0x7f06bc0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50704/0 (socket says 172.21.15.91:50704) 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06bb400640 1 -- 172.21.15.91:0/2067212310 learned_addr learned my addr 172.21.15.91:0/2067212310 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.302+0000 7f06b1e00640 1 --2- 172.21.15.91:0/2067212310 >> v2:172.21.15.91:3300/0 conn(0x7f06bc0ce130 0x7f06bc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.304 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06baa00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc15a010 0x7f06bc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06b1e00640 1 -- 172.21.15.91:0/2067212310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc15a010 msgr2=0x7f06bc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06b1e00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc15a010 0x7f06bc0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.305 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06b1e00640 1 -- 172.21.15.91:0/2067212310 >> v1:172.21.15.91:6789/0 conn(0x7f06bc0cd470 legacy=0x7f06bc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06b1e00640 1 -- 172.21.15.91:0/2067212310 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc06cb70 con 0x7f06bc0ce130 2024-02-02T13:37:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.303+0000 7f06ba000640 1 -- 172.21.15.91:0/2067212310 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512956217 0 0) 0x7f06bc0cc7c0 con 0x7f06bc0cd470 2024-02-02T13:37:59.306 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.304+0000 7f06b1e00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0ce130 0x7f06bc0ccf30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009800 tx=0x7f06a402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d4388e164cbbcd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.304+0000 7f06ba000640 1 -- 172.21.15.91:0/2067212310 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a402f8a0 con 0x7f06bc0ce130 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06ba000640 1 -- 172.21.15.91:0/2067212310 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a402fa00 con 0x7f06bc0ce130 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06ba000640 1 -- 172.21.15.91:0/2067212310 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a402fd90 con 0x7f06bc0ce130 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 -- 172.21.15.91:0/2067212310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0ce130 msgr2=0x7f06bc0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0ce130 0x7f06bc0ccf30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009800 tx=0x7f06a402dae0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 -- 172.21.15.91:0/2067212310 shutdown_connections 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0ce130 0x7f06bc0ccf30 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 --2- 172.21.15.91:0/2067212310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc15a010 0x7f06bc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 -- 172.21.15.91:0/2067212310 >> 172.21.15.91:0/2067212310 conn(0x7f06bc0cec80 msgr2=0x7f06bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 -- 172.21.15.91:0/2067212310 shutdown_connections 2024-02-02T13:37:59.307 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.305+0000 7f06c1a00640 1 -- 172.21.15.91:0/2067212310 wait complete. 2024-02-02T13:37:59.308 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.307+0000 7f06c1a00640 1 Processor -- start 2024-02-02T13:37:59.309 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.307+0000 7f06c1a00640 1 -- start start 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 0x7f06bc0c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 0x7f06bc0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc157dd0 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0c5170 con 0x7f06bc15a010 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.308+0000 7f06c1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0b3a20 con 0x7f06bc0ce130 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51562/0 (socket says 172.21.15.91:51562) 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 -- 172.21.15.91:0/3790442435 learned_addr learned my addr 172.21.15.91:0/3790442435 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06b1e00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 0x7f06bc0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06baa00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 0x7f06bc0c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 -- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 msgr2=0x7f06bc0c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.311 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 0x7f06bc0c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 -- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 msgr2=0x7f06bc0c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 0x7f06bc0c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06bb400640 1 -- 172.21.15.91:0/3790442435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009400 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06b1e00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 0x7f06bc0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.309+0000 7f06baa00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 0x7f06bc0c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:37:59.312 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06bb400640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c780 tx=0x7f06b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400ce30 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b400a640 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc0b3d00 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400aa70 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.310+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc0b42c0 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.313 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.311+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc0b44b0 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.312+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f06b400abd0 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.313+0000 7f06b3400640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 0x7f069406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.313+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f06b4087d00 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.314 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.313+0000 7f06baa00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 0x7f069406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.315 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.314+0000 7f06baa00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 0x7f069406c920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.318 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.317+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f06b40535e0 con 0x7f06bc0c2ae0 2024-02-02T13:37:59.508 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.507+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06bc0cca10 con 0x7f069406a460 2024-02-02T13:37:59.512 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.511+0000 7f06b3400640 1 -- 172.21.15.91:0/3790442435 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47580 (secure 0 0 0) 0x7f06ac00f860 con 0x7f069406a460 2024-02-02T13:37:59.513 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:59.513 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:37:59.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.513+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 msgr2=0x7f069406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.513+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 0x7f069406c920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).stop 2024-02-02T13:37:59.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 msgr2=0x7f06bc0c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.515 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c780 tx=0x7f06b400cc40 comp rx=0 tx=0).stop 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 shutdown_connections 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f069406a460 0x7f069406c920 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f06bc0c2ae0 0x7f06bc0c2f90 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06bc15a010 0x7f06bc0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 --2- 172.21.15.91:0/3790442435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06bc0ce130 0x7f06bc0c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 >> 172.21.15.91:0/3790442435 conn(0x7f06bc0cec80 msgr2=0x7f06bc0b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 shutdown_connections 2024-02-02T13:37:59.516 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.514+0000 7f06c1a00640 1 -- 172.21.15.91:0/3790442435 wait complete. 2024-02-02T13:37:59.535 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2024-02-02T13:37:58.510397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1449980,"statfs":{"total":1600000000,"available":1484780205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490837,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-123,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002460"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":343,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-02-02T13:37:53.586168+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:53.586168+0000","last_peered":"2024-02-02T13:37:53.586168+0000","last_clean":"2024-02-02T13:37:53.586168+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:53.586168+0000","last_undegraded":"2024-02-02T13:37:53.586168+0000","last_fullsized":"2024-02-02T13:37:53.586168+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9931,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185381,"statfs":{"total":200000000,"available":189830170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.64600000000000002,"15min":0.61499999999999999},"max":{"1min":1.4570000000000001,"5min":2.266,"15min":2.5310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.81699999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.524,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1240000000000001,"15min":1.1040000000000001},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.66300000000000003},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.893},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.56299999999999994},"max":{"1min":1.385,"5min":1.6399999999999999,"15min":2.5840000000000001},"last":1.216}]},{"osd":2,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2210000000000001,"15min":1.234},"min":{"1min":0.71799999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"max":{"1min":1.623,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.113},{"interface":"front","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2030000000000001},"min":{"1min":0.91900000000000004,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.5700000000000001,"5min":1.6639999999999999,"15min":3.5819999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2050000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.66200000000000003},"max":{"1min":1.5740000000000001,"5min":1.635,"15min":1.819},"last":1.004},{"interface":"front","average":{"1min":1.323,"5min":1.333,"15min":1.3280000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.48999999999999999},"max":{"1min":1.6910000000000001,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.258,"15min":1.242},"min":{"1min":0.79500000000000004,"5min":0.72899999999999998,"15min":0.57599999999999996},"max":{"1min":1.7110000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":1.046},{"interface":"front","average":{"1min":1.278,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.753,"15min":0.61199999999999999},"max":{"1min":1.655,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.349,"15min":1.361},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.54000000000000004},"max":{"1min":1.758,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.367},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.90700000000000003,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":3.7949999999999999},"last":1.173}]},{"osd":6,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.506,"15min":1.5109999999999999},"min":{"1min":1.194,"5min":0.95899999999999996,"15min":0.64200000000000002},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":3.6840000000000002},"last":1.345},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4139999999999999,"15min":1.4159999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.63700000000000001},"max":{"1min":1.8140000000000001,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":1.0800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175773,"statfs":{"total":200000000,"available":179991573,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.121,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.64000000000000001,"15min":0.498},"max":{"1min":1.5469999999999999,"5min":1.603,"15min":2.3780000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.3660000000000001,"5min":1.776,"15min":1.776},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.0660000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.54000000000000004},"max":{"1min":1.478,"5min":1.4790000000000001,"15min":1.556},"last":0.624},{"interface":"front","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.55900000000000005},"max":{"1min":1.51,"5min":1.677,"15min":2.9300000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2390000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.84799999999999998,"15min":0.63700000000000001},"max":{"1min":1.556,"5min":1.75,"15min":2.6309999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.212,"15min":1.1799999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.63},"max":{"1min":1.4890000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.085},{"interface":"front","average":{"1min":1.242,"5min":1.2689999999999999,"15min":1.1970000000000001},"min":{"1min":0.96899999999999997,"5min":0.90600000000000003,"15min":0.61299999999999999},"max":{"1min":1.6399999999999999,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.94499999999999995,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.5469999999999999,"5min":1.8420000000000001,"15min":17.686},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4350000000000001,"15min":1.4710000000000001},"min":{"1min":1.022,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.78,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.2849999999999999},"min":{"1min":1.0269999999999999,"5min":0.82399999999999995,"15min":0.45900000000000002},"max":{"1min":1.595,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4390000000000001,"15min":1.4079999999999999},"min":{"1min":1.1040000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8400000000000001,"5min":1.95,"15min":3.331},"last":1.04}]},{"osd":7,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.464,"15min":1.4610000000000001},"min":{"1min":1.0640000000000001,"5min":0.80300000000000005,"15min":0.61199999999999999},"max":{"1min":1.871,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.254},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4590000000000001,"15min":1.4430000000000001},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.65300000000000002},"max":{"1min":1.9970000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.093,"15min":1.046},"min":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.8,"15min":1.8},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.49199999999999999},"max":{"1min":1.4930000000000001,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":1.0660000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1839999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.841,"15min":4.5899999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.143,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4500000000000002},"last":1.123}]},{"osd":2,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.95699999999999996,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6419999999999999,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.575},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2430000000000001,"15min":1.2010000000000001},"min":{"1min":0.94899999999999995,"5min":0.84099999999999997,"15min":0.65300000000000002},"max":{"1min":1.498,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.526}]},{"osd":3,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.282,"15min":1.2190000000000001},"min":{"1min":0.93100000000000005,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.7769999999999999},"last":1.165},{"interface":"front","average":{"1min":1.375,"5min":1.2909999999999999,"15min":1.248},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":5.4610000000000003},"last":1.371}]},{"osd":4,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1950000000000001,"15min":1.2030000000000001},"min":{"1min":0.77700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.8460000000000001},"last":1.214},{"interface":"front","average":{"1min":1.157,"5min":1.3,"15min":1.29},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.55100000000000005},"max":{"1min":1.8440000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.478}]},{"osd":6,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.329},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.64700000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.244},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":0.98099999999999998,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8109999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.413,"15min":1.4119999999999999},"min":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.7190000000000001,"5min":1.893,"15min":1.9670000000000001},"last":1.29},{"interface":"front","average":{"1min":1.486,"5min":1.468,"15min":1.4379999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.61599999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.6539999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188007,"statfs":{"total":200000000,"available":192519381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0609999999999999,"15min":1.0429999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.395,"5min":1.4570000000000001,"15min":4.5949999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.077},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"max":{"1min":1.474,"5min":1.6579999999999999,"15min":1.839},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.095,"15min":1.056},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.3540000000000001,"5min":1.54,"15min":2.4809999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.123,"15min":1.0960000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.5029999999999999,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.1379999999999999},"min":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.68500000000000005},"max":{"1min":1.405,"5min":1.556,"15min":3.1200000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1919999999999999,"15min":1.1559999999999999},"min":{"1min":0.86799999999999999,"5min":0.71199999999999997,"15min":0.58799999999999997},"max":{"1min":1.4399999999999999,"5min":1.736,"15min":2.6000000000000001},"last":1.044}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.278,"15min":1.2450000000000001},"min":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.68999999999999995},"max":{"1min":1.5309999999999999,"5min":1.637,"15min":2.9079999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1850000000000001,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6120000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.29,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.4529999999999998},"last":1.0760000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3999999999999999,"15min":1.371},"min":{"1min":0.99199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.617,"5min":2.403,"15min":2.7989999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.423,"15min":1.429},"min":{"1min":1.0189999999999999,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.113}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.45,"15min":1.4410000000000001},"min":{"1min":1.008,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.825,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.151},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.466,"15min":1.4690000000000001},"min":{"1min":1.133,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.984,"5min":2.0409999999999999,"15min":2.698},"last":1.2490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186012,"statfs":{"total":200000000,"available":190476314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79600000000000004,"15min":0.755},"min":{"1min":0.63600000000000001,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.4199999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.71199999999999997,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.323,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":1.135}]},{"osd":1,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.71999999999999997,"5min":0.58899999999999997,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5609999999999999,"15min":2.403},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1419999999999999,"15min":1.0700000000000001},"min":{"1min":0.90700000000000003,"5min":0.60299999999999998,"15min":0.47999999999999998},"max":{"1min":1.4099999999999999,"5min":1.5589999999999999,"15min":2.2709999999999999},"last":1.008}]},{"osd":2,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1020000000000001,"15min":1.026},"min":{"1min":0.83199999999999996,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":1.659,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.91700000000000004},"min":{"1min":0.72999999999999998,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.45,"15min":3.4510000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.1910000000000001},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.53000000000000003},"max":{"1min":1.4830000000000001,"5min":1.5940000000000001,"15min":3.585},"last":1.099},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.248},"min":{"1min":1.0189999999999999,"5min":0.72499999999999998,"15min":0.61299999999999999},"max":{"1min":1.5800000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.353}]},{"osd":5,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2190000000000001,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.73799999999999999,"15min":0.57099999999999995},"max":{"1min":1.546,"5min":1.609,"15min":3.738},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.26,"15min":1.26},"min":{"1min":0.97499999999999998,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.6719999999999999,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3999999999999999,"15min":1.3320000000000001},"min":{"1min":1.159,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.5589999999999999,"5min":1.823,"15min":1.8979999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.95399999999999996,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.6200000000000001,"5min":1.667,"15min":3.6059999999999999},"last":0.95599999999999996}]},{"osd":7,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3129999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.577,"5min":1.851,"15min":3.391},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.397,"15min":1.3560000000000001},"min":{"1min":1.1180000000000001,"5min":0.83699999999999997,"15min":0.625},"max":{"1min":1.7669999999999999,"5min":1.8360000000000001,"15min":2.0409999999999999},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.27900000000000003},"max":{"1min":1.238,"5min":1.345,"15min":7.1840000000000002},"last":0.745},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0169999999999999,"15min":0.90600000000000003},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.411,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2270000000000001,"5min":1.387,"15min":2.1680000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.071,"15min":0.98199999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.35899999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.6499999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98999999999999999,"15min":0.90800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":2.1339999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.109,"15min":1.0509999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.432},"max":{"1min":1.488,"5min":1.653,"15min":2.6589999999999998},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2969999999999999,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.56799999999999995},"max":{"1min":1.5369999999999999,"5min":1.877,"15min":2.8759999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2330000000000001},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.503},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":2.1360000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.246,"15min":1.1910000000000001},"min":{"1min":1.1100000000000001,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":1.504,"5min":2.028,"15min":2.8239999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.337,"15min":1.3069999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.627},"max":{"1min":1.696,"5min":2.21,"15min":3.0070000000000001},"last":1.6830000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.371,"15min":1.2929999999999999},"min":{"1min":1.01,"5min":0.98999999999999999,"15min":0.53800000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.8620000000000001},"last":1.448},{"interface":"front","average":{"1min":1.357,"5min":1.329,"15min":1.2789999999999999},"min":{"1min":1.054,"5min":0.93999999999999995,"15min":0.70499999999999996},"max":{"1min":1.79,"5min":1.7949999999999999,"15min":2.9750000000000001},"last":1.653}]},{"osd":7,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4219999999999999,"15min":1.341},"min":{"1min":1.242,"5min":1.1220000000000001,"15min":0.65100000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.7749999999999999},"last":1.319},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3680000000000001,"15min":1.3129999999999999},"min":{"1min":1.028,"5min":0.96699999999999997,"15min":0.52500000000000002},"max":{"1min":1.7629999999999999,"5min":1.9330000000000001,"15min":3.0830000000000002},"last":1.4650000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13389,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181922,"statfs":{"total":200000000,"available":186288998,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84399999999999997,"15min":0.87},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":1.46,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.96299999999999997,"15min":1.046},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.308},"max":{"1min":1.387,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.53000000000000003,"5min":0.45200000000000001,"15min":0.34200000000000003},"max":{"1min":1.321,"5min":1.615,"15min":1.639},"last":1.167},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":3.0259999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.704,"15min":3.1539999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.778,"15min":4.8760000000000003},"last":1.3120000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.163,"15min":1.1859999999999999},"min":{"1min":0.83099999999999996,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":2.9449999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.95299999999999996,"5min":0.82699999999999996,"15min":0.68899999999999995},"max":{"1min":1.7430000000000001,"5min":1.748,"15min":3.2970000000000002},"last":1.087}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3400000000000001,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.753},"max":{"1min":1.8420000000000001,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.022,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":3.0249999999999999},"last":1.407}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.069,"5min":0.998,"15min":0.73599999999999999},"max":{"1min":1.879,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.369},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3420000000000001,"15min":1.3660000000000001},"min":{"1min":1.036,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.9910000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.77100000000000002},"max":{"1min":1.804,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4710000000000001,"15min":1.4650000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.78000000000000003},"max":{"1min":2.0169999999999999,"5min":2.0699999999999998,"15min":4.3040000000000003},"last":1.4399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9876,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185436,"statfs":{"total":200000000,"available":189886965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86699999999999999,"15min":0.90500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27300000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":4.8070000000000004},"last":0.436},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89800000000000002,"15min":1.0580000000000001},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":10.816000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.0009999999999999},"min":{"1min":0.53300000000000003,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.325,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.97399999999999998,"15min":1.0960000000000001},"min":{"1min":0.49199999999999999,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.6830000000000001,"15min":10.781000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95599999999999996,"15min":1.0069999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.379},"max":{"1min":1.3440000000000001,"5min":1.607,"15min":5.8410000000000002},"last":0.496},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0549999999999999,"15min":1.1140000000000001},"min":{"1min":0.54400000000000004,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.653,"5min":1.653,"15min":4.9530000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.93899999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.607,"5min":2.129,"15min":3.669},"last":1.105},{"interface":"front","average":{"1min":1.175,"5min":1.26,"15min":1.292},"min":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73199999999999998},"max":{"1min":1.8580000000000001,"5min":1.893,"15min":4.8780000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.26,"15min":1.415},"min":{"1min":0.77800000000000002,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":2.278,"5min":2.278,"15min":10.651},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.2909999999999999},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.64400000000000002},"max":{"1min":1.526,"5min":1.954,"15min":3.7869999999999999},"last":1.0680000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.341,"15min":1.359},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.66500000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.9210000000000003},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.296,"15min":1.444},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8,"5min":1.95,"15min":10.708},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.347,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":10.74},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.98299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":3.863},"last":1.016}]}]}],"pool_statfs":[]}} 2024-02-02T13:37:59.538 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:37:59.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.742+0000 7f2f8dc00640 1 Processor -- start 2024-02-02T13:37:59.743 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.742+0000 7f2f8dc00640 1 -- start start 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.743+0000 7f2f8dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f880cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.743+0000 7f2f8dc00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8806b5f0 con 0x7f2f880c9240 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f8dc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8806c450 con 0x7f2f881597a0 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f8dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806cff0 con 0x7f2f880cd690 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f87400640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f2f880c9240 0x7f2f8806f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:50720/0 (socket says 172.21.15.91:50720) 2024-02-02T13:37:59.745 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f87400640 1 -- 172.21.15.91:0/1036579098 learned_addr learned my addr 172.21.15.91:0/1036579098 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:37:59.746 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f7de00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f880cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296770382 0 0) 0x7f2f8806b5f0 con 0x7f2f880c9240 2024-02-02T13:37:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.744+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003520 con 0x7f2f880c9240 2024-02-02T13:37:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2445129930 0 0) 0x7f2f68003520 con 0x7f2f880c9240 2024-02-02T13:37:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 msgr2=0x7f2f880cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:59.747 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f880cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 >> v1:172.21.15.113:6789/0 conn(0x7f2f881597a0 legacy=0x7f2f8806c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88159b80 con 0x7f2f880c9240 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f2f80001670 con 0x7f2f880c9240 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 >> v1:172.21.15.91:6789/0 conn(0x7f2f880c9240 legacy=0x7f2f8806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f680039a0 0x7f2f68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806cff0 con 0x7f2f680047c0 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800cb70 con 0x7f2f680039a0 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800cd30 con 0x7f2f880cd690 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f87400640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f86a00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f680039a0 0x7f2f68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 -- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 msgr2=0x7f2f68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:59.748 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f68005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 -- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f680039a0 msgr2=0x7f2f68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:37:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f680039a0 0x7f2f68003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f7de00640 1 -- 172.21.15.91:0/1036579098 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68003520 con 0x7f2f680047c0 2024-02-02T13:37:59.749 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.745+0000 7f2f87400640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f880cd690 0x7f2f68005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.746+0000 7f2f7de00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f68004c40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009a90 tx=0x7f2f680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27c1cf37ab4b357 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7002c660 con 0x7f2f680047c0 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7002c7c0 con 0x7f2f680047c0 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f86000640 1 -- 172.21.15.91:0/1036579098 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7002cb50 con 0x7f2f680047c0 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f8dc00640 1 -- 172.21.15.91:0/1036579098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 msgr2=0x7f2f68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/1036579098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f68004c40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009a90 tx=0x7f2f680057f0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f87400640 1 -- 172.21.15.91:0/1036579098 reap_dead start 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f8dc00640 1 -- 172.21.15.91:0/1036579098 shutdown_connections 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f8dc00640 1 -- 172.21.15.91:0/1036579098 >> 172.21.15.91:0/1036579098 conn(0x7f2f880cec80 msgr2=0x7f2f880cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.747+0000 7f2f8dc00640 1 -- 172.21.15.91:0/1036579098 shutdown_connections 2024-02-02T13:37:59.750 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.748+0000 7f2f8dc00640 1 -- 172.21.15.91:0/1036579098 wait complete. 2024-02-02T13:37:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.749+0000 7f2f8dc00640 1 Processor -- start 2024-02-02T13:37:59.751 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.749+0000 7f2f8dc00640 1 -- start start 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f88154500 0x7f2f881f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f881f3060 0x7f2f880c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880ce030 con 0x7f2f680047c0 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881f3b10 con 0x7f2f881f3060 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.750+0000 7f2f8dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88155820 con 0x7f2f88154500 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:51574/0 (socket says 172.21.15.91:51574) 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 -- 172.21.15.91:0/76642987 learned_addr learned my addr 172.21.15.91:0/76642987 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 -- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f88154500 msgr2=0x7f2f881f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.753 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f7de00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f88154500 0x7f2f881f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f88154500 0x7f2f881f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 -- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f881f3060 msgr2=0x7f2f880c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f881f3060 0x7f2f880c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 -- 172.21.15.91:0/76642987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f881f3550 con 0x7f2f680047c0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.751+0000 7f2f86a00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d910 tx=0x7f2f7800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.752+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78003da0 con 0x7f2f680047c0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.752+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7800b770 con 0x7f2f680047c0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.752+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800bae0 con 0x7f2f680047c0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.752+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f881f3830 con 0x7f2f680047c0 2024-02-02T13:37:59.754 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.752+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f880c60d0 con 0x7f2f680047c0 2024-02-02T13:37:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.753+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f880c6260 con 0x7f2f680047c0 2024-02-02T13:37:59.755 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.754+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f7800bc40 con 0x7f2f680047c0 2024-02-02T13:37:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.755+0000 7f2f7f400640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 0x7f2f6006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:37:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.755+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f78088830 con 0x7f2f680047c0 2024-02-02T13:37:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.755+0000 7f2f7de00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 0x7f2f6006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:37:59.757 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.756+0000 7f2f7de00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 0x7f2f6006c920 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2f70036000 tx=0x7f2f70030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:37:59.761 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.760+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f780541b0 con 0x7f2f680047c0 2024-02-02T13:37:59.951 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.950+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f8806bb20 con 0x7f2f6006a460 2024-02-02T13:37:59.954 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.953+0000 7f2f7f400640 1 -- 172.21.15.91:0/76642987 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47580 (secure 0 0 0) 0x7f2f70004140 con 0x7f2f6006a460 2024-02-02T13:37:59.955 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:37:59.956 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 msgr2=0x7f2f6006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 0x7f2f6006c920 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2f70036000 tx=0x7f2f70030000 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 msgr2=0x7f2f881552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d910 tx=0x7f2f7800ddd0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 shutdown_connections 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f2f6006a460 0x7f2f6006c920 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f88154500 0x7f2f881f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f881f3060 0x7f2f880c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 --2- 172.21.15.91:0/76642987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f680047c0 0x7f2f881552e0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 >> 172.21.15.91:0/76642987 conn(0x7f2f880cec80 msgr2=0x7f2f88158820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 shutdown_connections 2024-02-02T13:37:59.958 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:37:59.956+0000 7f2f8dc00640 1 -- 172.21.15.91:0/76642987 wait complete. 2024-02-02T13:37:59.977 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2024-02-02T13:37:58.510397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1449980,"statfs":{"total":1600000000,"available":1484780205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490837,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-123,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002460"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":343,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-02-02T13:37:53.586168+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:53.586168+0000","last_peered":"2024-02-02T13:37:53.586168+0000","last_clean":"2024-02-02T13:37:53.586168+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:53.586168+0000","last_undegraded":"2024-02-02T13:37:53.586168+0000","last_fullsized":"2024-02-02T13:37:53.586168+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9931,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185381,"statfs":{"total":200000000,"available":189830170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.64600000000000002,"15min":0.61499999999999999},"max":{"1min":1.4570000000000001,"5min":2.266,"15min":2.5310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.81699999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.524,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1240000000000001,"15min":1.1040000000000001},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.66300000000000003},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.893},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.56299999999999994},"max":{"1min":1.385,"5min":1.6399999999999999,"15min":2.5840000000000001},"last":1.216}]},{"osd":2,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2210000000000001,"15min":1.234},"min":{"1min":0.71799999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"max":{"1min":1.623,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.113},{"interface":"front","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2030000000000001},"min":{"1min":0.91900000000000004,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.5700000000000001,"5min":1.6639999999999999,"15min":3.5819999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2050000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.66200000000000003},"max":{"1min":1.5740000000000001,"5min":1.635,"15min":1.819},"last":1.004},{"interface":"front","average":{"1min":1.323,"5min":1.333,"15min":1.3280000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.48999999999999999},"max":{"1min":1.6910000000000001,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.258,"15min":1.242},"min":{"1min":0.79500000000000004,"5min":0.72899999999999998,"15min":0.57599999999999996},"max":{"1min":1.7110000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":1.046},{"interface":"front","average":{"1min":1.278,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.753,"15min":0.61199999999999999},"max":{"1min":1.655,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.349,"15min":1.361},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.54000000000000004},"max":{"1min":1.758,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.367},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.90700000000000003,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":3.7949999999999999},"last":1.173}]},{"osd":6,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.506,"15min":1.5109999999999999},"min":{"1min":1.194,"5min":0.95899999999999996,"15min":0.64200000000000002},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":3.6840000000000002},"last":1.345},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4139999999999999,"15min":1.4159999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.63700000000000001},"max":{"1min":1.8140000000000001,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":1.0800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175773,"statfs":{"total":200000000,"available":179991573,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.121,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.64000000000000001,"15min":0.498},"max":{"1min":1.5469999999999999,"5min":1.603,"15min":2.3780000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.3660000000000001,"5min":1.776,"15min":1.776},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.0660000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.54000000000000004},"max":{"1min":1.478,"5min":1.4790000000000001,"15min":1.556},"last":0.624},{"interface":"front","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.55900000000000005},"max":{"1min":1.51,"5min":1.677,"15min":2.9300000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2390000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.84799999999999998,"15min":0.63700000000000001},"max":{"1min":1.556,"5min":1.75,"15min":2.6309999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.212,"15min":1.1799999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.63},"max":{"1min":1.4890000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.085},{"interface":"front","average":{"1min":1.242,"5min":1.2689999999999999,"15min":1.1970000000000001},"min":{"1min":0.96899999999999997,"5min":0.90600000000000003,"15min":0.61299999999999999},"max":{"1min":1.6399999999999999,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.94499999999999995,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.5469999999999999,"5min":1.8420000000000001,"15min":17.686},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4350000000000001,"15min":1.4710000000000001},"min":{"1min":1.022,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.78,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.2849999999999999},"min":{"1min":1.0269999999999999,"5min":0.82399999999999995,"15min":0.45900000000000002},"max":{"1min":1.595,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4390000000000001,"15min":1.4079999999999999},"min":{"1min":1.1040000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8400000000000001,"5min":1.95,"15min":3.331},"last":1.04}]},{"osd":7,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.464,"15min":1.4610000000000001},"min":{"1min":1.0640000000000001,"5min":0.80300000000000005,"15min":0.61199999999999999},"max":{"1min":1.871,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.254},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4590000000000001,"15min":1.4430000000000001},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.65300000000000002},"max":{"1min":1.9970000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.093,"15min":1.046},"min":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.8,"15min":1.8},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.49199999999999999},"max":{"1min":1.4930000000000001,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":1.0660000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1839999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.841,"15min":4.5899999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.143,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4500000000000002},"last":1.123}]},{"osd":2,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.95699999999999996,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6419999999999999,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.575},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2430000000000001,"15min":1.2010000000000001},"min":{"1min":0.94899999999999995,"5min":0.84099999999999997,"15min":0.65300000000000002},"max":{"1min":1.498,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.526}]},{"osd":3,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.282,"15min":1.2190000000000001},"min":{"1min":0.93100000000000005,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.7769999999999999},"last":1.165},{"interface":"front","average":{"1min":1.375,"5min":1.2909999999999999,"15min":1.248},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":5.4610000000000003},"last":1.371}]},{"osd":4,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1950000000000001,"15min":1.2030000000000001},"min":{"1min":0.77700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.8460000000000001},"last":1.214},{"interface":"front","average":{"1min":1.157,"5min":1.3,"15min":1.29},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.55100000000000005},"max":{"1min":1.8440000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.478}]},{"osd":6,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.329},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.64700000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.244},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":0.98099999999999998,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8109999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.413,"15min":1.4119999999999999},"min":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.7190000000000001,"5min":1.893,"15min":1.9670000000000001},"last":1.29},{"interface":"front","average":{"1min":1.486,"5min":1.468,"15min":1.4379999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.61599999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.6539999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188007,"statfs":{"total":200000000,"available":192519381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0609999999999999,"15min":1.0429999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.395,"5min":1.4570000000000001,"15min":4.5949999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.077},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"max":{"1min":1.474,"5min":1.6579999999999999,"15min":1.839},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.095,"15min":1.056},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.3540000000000001,"5min":1.54,"15min":2.4809999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.123,"15min":1.0960000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.5029999999999999,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.1379999999999999},"min":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.68500000000000005},"max":{"1min":1.405,"5min":1.556,"15min":3.1200000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1919999999999999,"15min":1.1559999999999999},"min":{"1min":0.86799999999999999,"5min":0.71199999999999997,"15min":0.58799999999999997},"max":{"1min":1.4399999999999999,"5min":1.736,"15min":2.6000000000000001},"last":1.044}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.278,"15min":1.2450000000000001},"min":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.68999999999999995},"max":{"1min":1.5309999999999999,"5min":1.637,"15min":2.9079999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1850000000000001,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6120000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.29,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.4529999999999998},"last":1.0760000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3999999999999999,"15min":1.371},"min":{"1min":0.99199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.617,"5min":2.403,"15min":2.7989999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.423,"15min":1.429},"min":{"1min":1.0189999999999999,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.113}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.45,"15min":1.4410000000000001},"min":{"1min":1.008,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.825,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.151},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.466,"15min":1.4690000000000001},"min":{"1min":1.133,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.984,"5min":2.0409999999999999,"15min":2.698},"last":1.2490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186012,"statfs":{"total":200000000,"available":190476314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79600000000000004,"15min":0.755},"min":{"1min":0.63600000000000001,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.4199999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.71199999999999997,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.323,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":1.135}]},{"osd":1,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.71999999999999997,"5min":0.58899999999999997,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5609999999999999,"15min":2.403},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1419999999999999,"15min":1.0700000000000001},"min":{"1min":0.90700000000000003,"5min":0.60299999999999998,"15min":0.47999999999999998},"max":{"1min":1.4099999999999999,"5min":1.5589999999999999,"15min":2.2709999999999999},"last":1.008}]},{"osd":2,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1020000000000001,"15min":1.026},"min":{"1min":0.83199999999999996,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":1.659,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.91700000000000004},"min":{"1min":0.72999999999999998,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.45,"15min":3.4510000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.1910000000000001},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.53000000000000003},"max":{"1min":1.4830000000000001,"5min":1.5940000000000001,"15min":3.585},"last":1.099},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.248},"min":{"1min":1.0189999999999999,"5min":0.72499999999999998,"15min":0.61299999999999999},"max":{"1min":1.5800000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.353}]},{"osd":5,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2190000000000001,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.73799999999999999,"15min":0.57099999999999995},"max":{"1min":1.546,"5min":1.609,"15min":3.738},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.26,"15min":1.26},"min":{"1min":0.97499999999999998,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.6719999999999999,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3999999999999999,"15min":1.3320000000000001},"min":{"1min":1.159,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.5589999999999999,"5min":1.823,"15min":1.8979999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.95399999999999996,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.6200000000000001,"5min":1.667,"15min":3.6059999999999999},"last":0.95599999999999996}]},{"osd":7,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3129999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.577,"5min":1.851,"15min":3.391},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.397,"15min":1.3560000000000001},"min":{"1min":1.1180000000000001,"5min":0.83699999999999997,"15min":0.625},"max":{"1min":1.7669999999999999,"5min":1.8360000000000001,"15min":2.0409999999999999},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.27900000000000003},"max":{"1min":1.238,"5min":1.345,"15min":7.1840000000000002},"last":0.745},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0169999999999999,"15min":0.90600000000000003},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.411,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2270000000000001,"5min":1.387,"15min":2.1680000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.071,"15min":0.98199999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.35899999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.6499999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98999999999999999,"15min":0.90800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":2.1339999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.109,"15min":1.0509999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.432},"max":{"1min":1.488,"5min":1.653,"15min":2.6589999999999998},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2969999999999999,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.56799999999999995},"max":{"1min":1.5369999999999999,"5min":1.877,"15min":2.8759999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2330000000000001},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.503},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":2.1360000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.246,"15min":1.1910000000000001},"min":{"1min":1.1100000000000001,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":1.504,"5min":2.028,"15min":2.8239999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.337,"15min":1.3069999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.627},"max":{"1min":1.696,"5min":2.21,"15min":3.0070000000000001},"last":1.6830000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.371,"15min":1.2929999999999999},"min":{"1min":1.01,"5min":0.98999999999999999,"15min":0.53800000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.8620000000000001},"last":1.448},{"interface":"front","average":{"1min":1.357,"5min":1.329,"15min":1.2789999999999999},"min":{"1min":1.054,"5min":0.93999999999999995,"15min":0.70499999999999996},"max":{"1min":1.79,"5min":1.7949999999999999,"15min":2.9750000000000001},"last":1.653}]},{"osd":7,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4219999999999999,"15min":1.341},"min":{"1min":1.242,"5min":1.1220000000000001,"15min":0.65100000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.7749999999999999},"last":1.319},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3680000000000001,"15min":1.3129999999999999},"min":{"1min":1.028,"5min":0.96699999999999997,"15min":0.52500000000000002},"max":{"1min":1.7629999999999999,"5min":1.9330000000000001,"15min":3.0830000000000002},"last":1.4650000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13389,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181922,"statfs":{"total":200000000,"available":186288998,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84399999999999997,"15min":0.87},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":1.46,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.96299999999999997,"15min":1.046},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.308},"max":{"1min":1.387,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.53000000000000003,"5min":0.45200000000000001,"15min":0.34200000000000003},"max":{"1min":1.321,"5min":1.615,"15min":1.639},"last":1.167},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":3.0259999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.704,"15min":3.1539999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.778,"15min":4.8760000000000003},"last":1.3120000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.163,"15min":1.1859999999999999},"min":{"1min":0.83099999999999996,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":2.9449999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.95299999999999996,"5min":0.82699999999999996,"15min":0.68899999999999995},"max":{"1min":1.7430000000000001,"5min":1.748,"15min":3.2970000000000002},"last":1.087}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3400000000000001,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.753},"max":{"1min":1.8420000000000001,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.022,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":3.0249999999999999},"last":1.407}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.069,"5min":0.998,"15min":0.73599999999999999},"max":{"1min":1.879,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.369},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3420000000000001,"15min":1.3660000000000001},"min":{"1min":1.036,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.9910000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.77100000000000002},"max":{"1min":1.804,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4710000000000001,"15min":1.4650000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.78000000000000003},"max":{"1min":2.0169999999999999,"5min":2.0699999999999998,"15min":4.3040000000000003},"last":1.4399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9876,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185436,"statfs":{"total":200000000,"available":189886965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86699999999999999,"15min":0.90500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27300000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":4.8070000000000004},"last":0.436},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89800000000000002,"15min":1.0580000000000001},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":10.816000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.0009999999999999},"min":{"1min":0.53300000000000003,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.325,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.97399999999999998,"15min":1.0960000000000001},"min":{"1min":0.49199999999999999,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.6830000000000001,"15min":10.781000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95599999999999996,"15min":1.0069999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.379},"max":{"1min":1.3440000000000001,"5min":1.607,"15min":5.8410000000000002},"last":0.496},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0549999999999999,"15min":1.1140000000000001},"min":{"1min":0.54400000000000004,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.653,"5min":1.653,"15min":4.9530000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.93899999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.607,"5min":2.129,"15min":3.669},"last":1.105},{"interface":"front","average":{"1min":1.175,"5min":1.26,"15min":1.292},"min":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73199999999999998},"max":{"1min":1.8580000000000001,"5min":1.893,"15min":4.8780000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.26,"15min":1.415},"min":{"1min":0.77800000000000002,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":2.278,"5min":2.278,"15min":10.651},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.2909999999999999},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.64400000000000002},"max":{"1min":1.526,"5min":1.954,"15min":3.7869999999999999},"last":1.0680000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.341,"15min":1.359},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.66500000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.9210000000000003},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.296,"15min":1.444},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8,"5min":1.95,"15min":10.708},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.347,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":10.74},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.98299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":3.863},"last":1.016}]}]}],"pool_statfs":[]}} 2024-02-02T13:37:59.978 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-02T13:37:59.978 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:38:00.178 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.177+0000 7f484ca00640 1 Processor -- start 2024-02-02T13:38:00.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.177+0000 7f484ca00640 1 -- start start 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.178+0000 7f484ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 0x7f484406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.178+0000 7f484ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48440ac8d0 0x7f48440acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.178+0000 7f484ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484415d870 con 0x7f48440b0700 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f484ca00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48440ad2a0 con 0x7f48440ac8d0 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f484ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484406bce0 con 0x7f484415d490 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48440ac8d0 0x7f48440acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f48440ac8d0 0x7f48440acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46476/0 (socket says 172.21.15.91:46476) 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 -- 172.21.15.91:0/3888381365 learned_addr learned my addr 172.21.15.91:0/3888381365 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f484a000640 1 --2- 172.21.15.91:0/3888381365 >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 0x7f484406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 -- 172.21.15.91:0/3888381365 >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 msgr2=0x7f484406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 --2- 172.21.15.91:0/3888381365 >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 0x7f484406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 -- 172.21.15.91:0/3888381365 >> v1:172.21.15.113:6789/0 conn(0x7f48440b0700 legacy=0x7f48440c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.179+0000 7f4849600640 1 -- 172.21.15.91:0/3888381365 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4844161220 con 0x7f48440ac8d0 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.180+0000 7f484a000640 1 --2- 172.21.15.91:0/3888381365 >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 0x7f484406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.181+0000 7f4849600640 1 --2- 172.21.15.91:0/3888381365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440ac8d0 0x7f48440acd60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f48441613b0 tx=0x7f482c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45fb0fc1b3de9a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.181+0000 7f4848c00640 1 -- 172.21.15.91:0/3888381365 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c031860 con 0x7f48440ac8d0 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.181+0000 7f4848c00640 1 -- 172.21.15.91:0/3888381365 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0319c0 con 0x7f48440ac8d0 2024-02-02T13:38:00.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.181+0000 7f4848c00640 1 -- 172.21.15.91:0/3888381365 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c031d50 con 0x7f48440ac8d0 2024-02-02T13:38:00.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 -- 172.21.15.91:0/3888381365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440ac8d0 msgr2=0x7f48440acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.183 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 --2- 172.21.15.91:0/3888381365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440ac8d0 0x7f48440acd60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f48441613b0 tx=0x7f482c02b900 comp rx=0 tx=0).stop 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 -- 172.21.15.91:0/3888381365 shutdown_connections 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 --2- 172.21.15.91:0/3888381365 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440ac8d0 0x7f48440acd60 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 --2- 172.21.15.91:0/3888381365 >> v2:172.21.15.113:3300/0 conn(0x7f484415d490 0x7f484406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 -- 172.21.15.91:0/3888381365 >> 172.21.15.91:0/3888381365 conn(0x7f48440cec80 msgr2=0x7f48440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 -- 172.21.15.91:0/3888381365 shutdown_connections 2024-02-02T13:38:00.184 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.182+0000 7f484ca00640 1 -- 172.21.15.91:0/3888381365 wait complete. 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.183+0000 7f484ca00640 1 Processor -- start 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.184+0000 7f484ca00640 1 -- start start 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.184+0000 7f484ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 0x7f48440b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484ca00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 0x7f484414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484ca00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441f6e60 con 0x7f48440b0700 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441f61e0 con 0x7f48440ac8d0 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48441f64c0 con 0x7f484415d490 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46492/0 (socket says 172.21.15.91:46492) 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 -- 172.21.15.91:0/1254895319 learned_addr learned my addr 172.21.15.91:0/1254895319 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:00.188 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484a000640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 0x7f48440b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484aa00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 0x7f484414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 -- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 msgr2=0x7f484414da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 0x7f484414da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 -- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 msgr2=0x7f48440b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 0x7f48440b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f4849600640 1 -- 172.21.15.91:0/1254895319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0094d0 con 0x7f48440b0700 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484aa00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 0x7f484414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.185+0000 7f484a000640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 0x7f48440b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f4849600640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c031830 con 0x7f48440b0700 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0443f0 con 0x7f48440b0700 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0447a0 con 0x7f48440b0700 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48441f67a0 con 0x7f48440b0700 2024-02-02T13:38:00.189 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.186+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48441f5e70 con 0x7f48440b0700 2024-02-02T13:38:00.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.187+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4844066cf0 con 0x7f48440b0700 2024-02-02T13:38:00.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.188+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f482c04d450 con 0x7f48440b0700 2024-02-02T13:38:00.190 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.189+0000 7f4842000640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 0x7f481c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.189+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f482c0aade0 con 0x7f48440b0700 2024-02-02T13:38:00.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.189+0000 7f484a000640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 0x7f481c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.191 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.190+0000 7f484a000640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 0x7f481c06c8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4844157ea0 tx=0x7f4834008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.194 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.193+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f482c076630 con 0x7f48440b0700 2024-02-02T13:38:00.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.381+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f484406b690 con 0x7f481c06a430 2024-02-02T13:38:00.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.387+0000 7f4842000640 1 -- 172.21.15.91:0/1254895319 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47580 (secure 0 0 0) 0x7f483400c040 con 0x7f481c06a430 2024-02-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 msgr2=0x7f481c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 0x7f481c06c8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4844157ea0 tx=0x7f4834008040 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 msgr2=0x7f48441601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 shutdown_connections 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f481c06a430 0x7f481c06c8f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f484415d490 0x7f484414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48440b0700 0x7f48441601b0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 --2- 172.21.15.91:0/1254895319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48440ac8d0 0x7f48440b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 >> 172.21.15.91:0/1254895319 conn(0x7f48440cec80 msgr2=0x7f48441613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 shutdown_connections 2024-02-02T13:38:00.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.390+0000 7f484ca00640 1 -- 172.21.15.91:0/1254895319 wait complete. 2024-02-02T13:38:00.410 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2024-02-02T13:37:58.510397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1449980,"statfs":{"total":1600000000,"available":1484780205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490837,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-123,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002460"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":343,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-02-02T13:37:53.586168+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:53.586168+0000","last_peered":"2024-02-02T13:37:53.586168+0000","last_clean":"2024-02-02T13:37:53.586168+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:53.586168+0000","last_undegraded":"2024-02-02T13:37:53.586168+0000","last_fullsized":"2024-02-02T13:37:53.586168+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9931,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185381,"statfs":{"total":200000000,"available":189830170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.64600000000000002,"15min":0.61499999999999999},"max":{"1min":1.4570000000000001,"5min":2.266,"15min":2.5310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.81699999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.524,"5min":1.7889999999999999,"15min":4.1609999999999996},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1240000000000001,"15min":1.1040000000000001},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.66300000000000003},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.893},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.56299999999999994},"max":{"1min":1.385,"5min":1.6399999999999999,"15min":2.5840000000000001},"last":1.216}]},{"osd":2,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2210000000000001,"15min":1.234},"min":{"1min":0.71799999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"max":{"1min":1.623,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.113},{"interface":"front","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2030000000000001},"min":{"1min":0.91900000000000004,"5min":0.71699999999999997,"15min":0.623},"max":{"1min":1.5700000000000001,"5min":1.6639999999999999,"15min":3.5819999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2050000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.66200000000000003},"max":{"1min":1.5740000000000001,"5min":1.635,"15min":1.819},"last":1.004},{"interface":"front","average":{"1min":1.323,"5min":1.333,"15min":1.3280000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.48999999999999999},"max":{"1min":1.6910000000000001,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.258,"15min":1.242},"min":{"1min":0.79500000000000004,"5min":0.72899999999999998,"15min":0.57599999999999996},"max":{"1min":1.7110000000000001,"5min":1.8169999999999999,"15min":1.8260000000000001},"last":1.046},{"interface":"front","average":{"1min":1.278,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.753,"15min":0.61199999999999999},"max":{"1min":1.655,"5min":1.8049999999999999,"15min":2.0939999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.349,"15min":1.361},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.54000000000000004},"max":{"1min":1.758,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.367},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.90700000000000003,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":3.7949999999999999},"last":1.173}]},{"osd":6,"last update":"Fri Feb 2 13:37:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.506,"15min":1.5109999999999999},"min":{"1min":1.194,"5min":0.95899999999999996,"15min":0.64200000000000002},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":3.6840000000000002},"last":1.345},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4139999999999999,"15min":1.4159999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.63700000000000001},"max":{"1min":1.8140000000000001,"5min":1.8999999999999999,"15min":2.0179999999999998},"last":1.0800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175773,"statfs":{"total":200000000,"available":179991573,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.121,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.64000000000000001,"15min":0.498},"max":{"1min":1.5469999999999999,"5min":1.603,"15min":2.3780000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.3660000000000001,"5min":1.776,"15min":1.776},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.0660000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.54000000000000004},"max":{"1min":1.478,"5min":1.4790000000000001,"15min":1.556},"last":0.624},{"interface":"front","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.55900000000000005},"max":{"1min":1.51,"5min":1.677,"15min":2.9300000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2390000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.84799999999999998,"15min":0.63700000000000001},"max":{"1min":1.556,"5min":1.75,"15min":2.6309999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.212,"15min":1.1799999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.63},"max":{"1min":1.4890000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.085},{"interface":"front","average":{"1min":1.242,"5min":1.2689999999999999,"15min":1.1970000000000001},"min":{"1min":0.96899999999999997,"5min":0.90600000000000003,"15min":0.61299999999999999},"max":{"1min":1.6399999999999999,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.94499999999999995,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.5469999999999999,"5min":1.8420000000000001,"15min":17.686},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4350000000000001,"15min":1.4710000000000001},"min":{"1min":1.022,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.78,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.2849999999999999},"min":{"1min":1.0269999999999999,"5min":0.82399999999999995,"15min":0.45900000000000002},"max":{"1min":1.595,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4390000000000001,"15min":1.4079999999999999},"min":{"1min":1.1040000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8400000000000001,"5min":1.95,"15min":3.331},"last":1.04}]},{"osd":7,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.464,"15min":1.4610000000000001},"min":{"1min":1.0640000000000001,"5min":0.80300000000000005,"15min":0.61199999999999999},"max":{"1min":1.871,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.254},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4590000000000001,"15min":1.4430000000000001},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.65300000000000002},"max":{"1min":1.9970000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.093,"15min":1.046},"min":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.8,"15min":1.8},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.49199999999999999},"max":{"1min":1.4930000000000001,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":1.0660000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1839999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.841,"15min":4.5899999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.143,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4500000000000002},"last":1.123}]},{"osd":2,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.95699999999999996,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6419999999999999,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.575},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2430000000000001,"15min":1.2010000000000001},"min":{"1min":0.94899999999999995,"5min":0.84099999999999997,"15min":0.65300000000000002},"max":{"1min":1.498,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.526}]},{"osd":3,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.282,"15min":1.2190000000000001},"min":{"1min":0.93100000000000005,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.7769999999999999},"last":1.165},{"interface":"front","average":{"1min":1.375,"5min":1.2909999999999999,"15min":1.248},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":5.4610000000000003},"last":1.371}]},{"osd":4,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1950000000000001,"15min":1.2030000000000001},"min":{"1min":0.77700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.8460000000000001},"last":1.214},{"interface":"front","average":{"1min":1.157,"5min":1.3,"15min":1.29},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.55100000000000005},"max":{"1min":1.8440000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.478}]},{"osd":6,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.329},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.64700000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.244},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":0.98099999999999998,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8109999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.413,"15min":1.4119999999999999},"min":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.7190000000000001,"5min":1.893,"15min":1.9670000000000001},"last":1.29},{"interface":"front","average":{"1min":1.486,"5min":1.468,"15min":1.4379999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.61599999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.6539999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188007,"statfs":{"total":200000000,"available":192519381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0609999999999999,"15min":1.0429999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.395,"5min":1.4570000000000001,"15min":4.5949999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.077},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"max":{"1min":1.474,"5min":1.6579999999999999,"15min":1.839},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.095,"15min":1.056},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.3540000000000001,"5min":1.54,"15min":2.4809999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.123,"15min":1.0960000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.5029999999999999,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.1379999999999999},"min":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.68500000000000005},"max":{"1min":1.405,"5min":1.556,"15min":3.1200000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1919999999999999,"15min":1.1559999999999999},"min":{"1min":0.86799999999999999,"5min":0.71199999999999997,"15min":0.58799999999999997},"max":{"1min":1.4399999999999999,"5min":1.736,"15min":2.6000000000000001},"last":1.044}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.278,"15min":1.2450000000000001},"min":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.68999999999999995},"max":{"1min":1.5309999999999999,"5min":1.637,"15min":2.9079999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.175}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1850000000000001,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6120000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.216},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.29,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.4529999999999998},"last":1.0760000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3999999999999999,"15min":1.371},"min":{"1min":0.99199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.617,"5min":2.403,"15min":2.7989999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.423,"15min":1.429},"min":{"1min":1.0189999999999999,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.113}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.45,"15min":1.4410000000000001},"min":{"1min":1.008,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.825,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.151},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.466,"15min":1.4690000000000001},"min":{"1min":1.133,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.984,"5min":2.0409999999999999,"15min":2.698},"last":1.2490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186012,"statfs":{"total":200000000,"available":190476314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79600000000000004,"15min":0.755},"min":{"1min":0.63600000000000001,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.4199999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.71199999999999997,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.323,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":1.135}]},{"osd":1,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.71999999999999997,"5min":0.58899999999999997,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5609999999999999,"15min":2.403},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1419999999999999,"15min":1.0700000000000001},"min":{"1min":0.90700000000000003,"5min":0.60299999999999998,"15min":0.47999999999999998},"max":{"1min":1.4099999999999999,"5min":1.5589999999999999,"15min":2.2709999999999999},"last":1.008}]},{"osd":2,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1020000000000001,"15min":1.026},"min":{"1min":0.83199999999999996,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":1.659,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94099999999999995,"15min":0.91700000000000004},"min":{"1min":0.72999999999999998,"5min":0.42899999999999999,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.45,"15min":3.4510000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.1910000000000001},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.53000000000000003},"max":{"1min":1.4830000000000001,"5min":1.5940000000000001,"15min":3.585},"last":1.099},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.248},"min":{"1min":1.0189999999999999,"5min":0.72499999999999998,"15min":0.61299999999999999},"max":{"1min":1.5800000000000001,"5min":1.8149999999999999,"15min":1.877},"last":1.353}]},{"osd":5,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2190000000000001,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.73799999999999999,"15min":0.57099999999999995},"max":{"1min":1.546,"5min":1.609,"15min":3.738},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.26,"15min":1.26},"min":{"1min":0.97499999999999998,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.6719999999999999,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3999999999999999,"15min":1.3320000000000001},"min":{"1min":1.159,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.5589999999999999,"5min":1.823,"15min":1.8979999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.95399999999999996,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.6200000000000001,"5min":1.667,"15min":3.6059999999999999},"last":0.95599999999999996}]},{"osd":7,"last update":"Fri Feb 2 13:36:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3129999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.577,"5min":1.851,"15min":3.391},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.397,"15min":1.3560000000000001},"min":{"1min":1.1180000000000001,"5min":0.83699999999999997,"15min":0.625},"max":{"1min":1.7669999999999999,"5min":1.8360000000000001,"15min":2.0409999999999999},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173724,"statfs":{"total":200000000,"available":177893402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.27900000000000003},"max":{"1min":1.238,"5min":1.345,"15min":7.1840000000000002},"last":0.745},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0169999999999999,"15min":0.90600000000000003},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.411,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2270000000000001,"5min":1.387,"15min":2.1680000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.071,"15min":0.98199999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.35899999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.6499999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98999999999999999,"15min":0.90800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":2.1339999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.109,"15min":1.0509999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.432},"max":{"1min":1.488,"5min":1.653,"15min":2.6589999999999998},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2969999999999999,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.56799999999999995},"max":{"1min":1.5369999999999999,"5min":1.877,"15min":2.8759999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2330000000000001},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.503},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":2.1360000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.246,"15min":1.1910000000000001},"min":{"1min":1.1100000000000001,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":1.504,"5min":2.028,"15min":2.8239999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.337,"15min":1.3069999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.627},"max":{"1min":1.696,"5min":2.21,"15min":3.0070000000000001},"last":1.6830000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.371,"15min":1.2929999999999999},"min":{"1min":1.01,"5min":0.98999999999999999,"15min":0.53800000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.8620000000000001},"last":1.448},{"interface":"front","average":{"1min":1.357,"5min":1.329,"15min":1.2789999999999999},"min":{"1min":1.054,"5min":0.93999999999999995,"15min":0.70499999999999996},"max":{"1min":1.79,"5min":1.7949999999999999,"15min":2.9750000000000001},"last":1.653}]},{"osd":7,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4219999999999999,"15min":1.341},"min":{"1min":1.242,"5min":1.1220000000000001,"15min":0.65100000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.7749999999999999},"last":1.319},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3680000000000001,"15min":1.3129999999999999},"min":{"1min":1.028,"5min":0.96699999999999997,"15min":0.52500000000000002},"max":{"1min":1.7629999999999999,"5min":1.9330000000000001,"15min":3.0830000000000002},"last":1.4650000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13389,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181922,"statfs":{"total":200000000,"available":186288998,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84399999999999997,"15min":0.87},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":1.46,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.96299999999999997,"15min":1.046},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.308},"max":{"1min":1.387,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.53000000000000003,"5min":0.45200000000000001,"15min":0.34200000000000003},"max":{"1min":1.321,"5min":1.615,"15min":1.639},"last":1.167},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":3.0259999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.704,"15min":3.1539999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.778,"15min":4.8760000000000003},"last":1.3120000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.163,"15min":1.1859999999999999},"min":{"1min":0.83099999999999996,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":2.9449999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.95299999999999996,"5min":0.82699999999999996,"15min":0.68899999999999995},"max":{"1min":1.7430000000000001,"5min":1.748,"15min":3.2970000000000002},"last":1.087}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3400000000000001,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.753},"max":{"1min":1.8420000000000001,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.022,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":3.0249999999999999},"last":1.407}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.069,"5min":0.998,"15min":0.73599999999999999},"max":{"1min":1.879,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.369},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3420000000000001,"15min":1.3660000000000001},"min":{"1min":1.036,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.9910000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.77100000000000002},"max":{"1min":1.804,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4710000000000001,"15min":1.4650000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.78000000000000003},"max":{"1min":2.0169999999999999,"5min":2.0699999999999998,"15min":4.3040000000000003},"last":1.4399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9876,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185436,"statfs":{"total":200000000,"available":189886965,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86699999999999999,"15min":0.90500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27300000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":4.8070000000000004},"last":0.436},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89800000000000002,"15min":1.0580000000000001},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":10.816000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.0009999999999999},"min":{"1min":0.53300000000000003,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.325,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.97399999999999998,"15min":1.0960000000000001},"min":{"1min":0.49199999999999999,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.6830000000000001,"15min":10.781000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95599999999999996,"15min":1.0069999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.379},"max":{"1min":1.3440000000000001,"5min":1.607,"15min":5.8410000000000002},"last":0.496},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0549999999999999,"15min":1.1140000000000001},"min":{"1min":0.54400000000000004,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.653,"5min":1.653,"15min":4.9530000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.93899999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.607,"5min":2.129,"15min":3.669},"last":1.105},{"interface":"front","average":{"1min":1.175,"5min":1.26,"15min":1.292},"min":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73199999999999998},"max":{"1min":1.8580000000000001,"5min":1.893,"15min":4.8780000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.26,"15min":1.415},"min":{"1min":0.77800000000000002,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":2.278,"5min":2.278,"15min":10.651},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.2909999999999999},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.64400000000000002},"max":{"1min":1.526,"5min":1.954,"15min":3.7869999999999999},"last":1.0680000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.341,"15min":1.359},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.66500000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.9210000000000003},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.296,"15min":1.444},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8,"5min":1.95,"15min":10.708},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.347,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":10.74},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.98299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":3.863},"last":1.016}]}]}],"pool_statfs":[]}} 2024-02-02T13:38:00.411 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T13:38:00.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.625+0000 7f73dea00640 1 Processor -- start 2024-02-02T13:38:00.626 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.625+0000 7f73dea00640 1 -- start start 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.626+0000 7f73dea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73d815d3b0 0x7f73d80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.626+0000 7f73dea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.626+0000 7f73dea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80cb4b0 con 0x7f73d80cff10 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.626+0000 7f73dea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80d1c10 con 0x7f73d815d3b0 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.626+0000 7f73dea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80cb790 con 0x7f73d80cdde0 2024-02-02T13:38:00.628 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.627+0000 7f73de000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.627+0000 7f73de000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:58296/0 (socket says 172.21.15.91:58296) 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.627+0000 7f73de000640 1 -- 172.21.15.91:0/2410110097 learned_addr learned my addr 172.21.15.91:0/2410110097 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.627+0000 7f73dd600640 1 --2- 172.21.15.91:0/2410110097 >> v2:172.21.15.113:3300/0 conn(0x7f73d815d3b0 0x7f73d80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.628+0000 7f73de000640 1 -- 172.21.15.91:0/2410110097 >> v2:172.21.15.113:3300/0 conn(0x7f73d815d3b0 msgr2=0x7f73d80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.628+0000 7f73de000640 1 --2- 172.21.15.91:0/2410110097 >> v2:172.21.15.113:3300/0 conn(0x7f73d815d3b0 0x7f73d80cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.628+0000 7f73de000640 1 -- 172.21.15.91:0/2410110097 >> v1:172.21.15.113:6789/0 conn(0x7f73d80cff10 legacy=0x7f73d80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.628+0000 7f73d7e00640 1 -- 172.21.15.91:0/2410110097 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902971766 0 0) 0x7f73d80cb4b0 con 0x7f73d80cff10 2024-02-02T13:38:00.629 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.628+0000 7f73de000640 1 -- 172.21.15.91:0/2410110097 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80cba70 con 0x7f73d80cdde0 2024-02-02T13:38:00.630 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.629+0000 7f73de000640 1 --2- 172.21.15.91:0/2410110097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009870 tx=0x7f73d002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37938c7f118c5604 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73d7e00640 1 -- 172.21.15.91:0/2410110097 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0030840 con 0x7f73d80cdde0 2024-02-02T13:38:00.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73d7e00640 1 -- 172.21.15.91:0/2410110097 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d00309a0 con 0x7f73d80cdde0 2024-02-02T13:38:00.631 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73d7e00640 1 -- 172.21.15.91:0/2410110097 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0030d30 con 0x7f73d80cdde0 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 -- 172.21.15.91:0/2410110097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 msgr2=0x7f73d80b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 --2- 172.21.15.91:0/2410110097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009870 tx=0x7f73d002b980 comp rx=0 tx=0).stop 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 -- 172.21.15.91:0/2410110097 shutdown_connections 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 --2- 172.21.15.91:0/2410110097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80cdde0 0x7f73d80b5eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 --2- 172.21.15.91:0/2410110097 >> v2:172.21.15.113:3300/0 conn(0x7f73d815d3b0 0x7f73d80cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 -- 172.21.15.91:0/2410110097 >> 172.21.15.91:0/2410110097 conn(0x7f73d80d0cc0 msgr2=0x7f73d80d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.630+0000 7f73dea00640 1 -- 172.21.15.91:0/2410110097 shutdown_connections 2024-02-02T13:38:00.632 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.631+0000 7f73dea00640 1 -- 172.21.15.91:0/2410110097 wait complete. 2024-02-02T13:38:00.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.632+0000 7f73dea00640 1 Processor -- start 2024-02-02T13:38:00.633 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.632+0000 7f73dea00640 1 -- start start 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 0x7f73d8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80c9320 0x7f73d80c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815d810 con 0x7f73d80cff10 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815daf0 con 0x7f73d80cdde0 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.633+0000 7f73dea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c97c0 con 0x7f73d80c9320 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46502/0 (socket says 172.21.15.91:46502) 2024-02-02T13:38:00.636 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 -- 172.21.15.91:0/3941826185 learned_addr learned my addr 172.21.15.91:0/3941826185 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dd600640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 0x7f73d8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73de000640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80c9320 0x7f73d80c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 -- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80c9320 msgr2=0x7f73d80c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80c9320 0x7f73d80c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 -- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 msgr2=0x7f73d8159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 0x7f73d8159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dcc00640 1 -- 172.21.15.91:0/3941826185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d00094d0 con 0x7f73d80cff10 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.634+0000 7f73dd600640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 0x7f73d8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:38:00.637 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73dcc00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c410 tx=0x7f73c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c00035f0 con 0x7f73d80cff10 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c0003770 con 0x7f73d80cff10 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0003b20 con 0x7f73d80cff10 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d80c9aa0 con 0x7f73d80cff10 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.635+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d815ed70 con 0x7f73d80cff10 2024-02-02T13:38:00.638 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.636+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d815dc60 con 0x7f73d80cff10 2024-02-02T13:38:00.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.637+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f73c0003c80 con 0x7f73d80cff10 2024-02-02T13:38:00.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.638+0000 7f73d5600640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 0x7f73b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:00.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.638+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f73c0087bb0 con 0x7f73d80cff10 2024-02-02T13:38:00.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.638+0000 7f73dd600640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 0x7f73b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:00.640 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.639+0000 7f73dd600640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 0x7f73b006c8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f73c80074c0 tx=0x7f73c800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:00.643 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.642+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f73c0053530 con 0x7f73d80cff10 2024-02-02T13:38:00.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.831+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73d80cbd50 con 0x7f73d80cff10 2024-02-02T13:38:00.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.832+0000 7f73d5600640 1 -- 172.21.15.91:0/3941826185 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+12527 (secure 0 0 0) 0x7f73c0017090 con 0x7f73d80cff10 2024-02-02T13:38:00.834 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:38:00.834 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":202,"fsid":"cc006970-f900-4781-8947-81ed22cd959b","created":"2024-02-02T13:24:19.251864+0000","modified":"2024-02-02T13:37:56.686689+0000","last_up_change":"2024-02-02T13:24:25.741545+0000","last_in_change":"2024-02-02T13:24:22.715632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T13:24:26.310552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":32877},{"type":"v1","addr":"172.21.15.91:6822","nonce":32877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":32877},{"type":"v1","addr":"172.21.15.91:6825","nonce":32877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":32877},{"type":"v1","addr":"172.21.15.91:6831","nonce":32877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6827","nonce":32877},{"type":"v1","addr":"172.21.15.91:6829","nonce":32877}]},"public_addr":"172.21.15.91:6822/32877","cluster_addr":"172.21.15.91:6825/32877","heartbeat_back_addr":"172.21.15.91:6831/32877","heartbeat_front_addr":"172.21.15.91:6829/32877","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6805","nonce":32878},{"type":"v1","addr":"172.21.15.91:6807","nonce":32878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":32878},{"type":"v1","addr":"172.21.15.91:6809","nonce":32878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":32878},{"type":"v1","addr":"172.21.15.91:6815","nonce":32878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6811","nonce":32878},{"type":"v1","addr":"172.21.15.91:6812","nonce":32878}]},"public_addr":"172.21.15.91:6807/32878","cluster_addr":"172.21.15.91:6809/32878","heartbeat_back_addr":"172.21.15.91:6815/32878","heartbeat_front_addr":"172.21.15.91:6812/32878","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":32875},{"type":"v1","addr":"172.21.15.91:6817","nonce":32875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":32875},{"type":"v1","addr":"172.21.15.91:6819","nonce":32875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6826","nonce":32875},{"type":"v1","addr":"172.21.15.91:6828","nonce":32875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":32875},{"type":"v1","addr":"172.21.15.91:6823","nonce":32875}]},"public_addr":"172.21.15.91:6817/32875","cluster_addr":"172.21.15.91:6819/32875","heartbeat_back_addr":"172.21.15.91:6828/32875","heartbeat_front_addr":"172.21.15.91:6823/32875","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":32876},{"type":"v1","addr":"172.21.15.91:6801","nonce":32876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":32876},{"type":"v1","addr":"172.21.15.91:6803","nonce":32876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":32876},{"type":"v1","addr":"172.21.15.91:6813","nonce":32876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":32876},{"type":"v1","addr":"172.21.15.91:6806","nonce":32876}]},"public_addr":"172.21.15.91:6801/32876","cluster_addr":"172.21.15.91:6803/32876","heartbeat_back_addr":"172.21.15.91:6813/32876","heartbeat_front_addr":"172.21.15.91:6806/32876","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6823","nonce":33080},{"type":"v1","addr":"172.21.15.113:6825","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":33080},{"type":"v1","addr":"172.21.15.113:6827","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":33080},{"type":"v1","addr":"172.21.15.113:6831","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":33080},{"type":"v1","addr":"172.21.15.113:6829","nonce":33080}]},"public_addr":"172.21.15.113:6825/33080","cluster_addr":"172.21.15.113:6827/33080","heartbeat_back_addr":"172.21.15.113:6831/33080","heartbeat_front_addr":"172.21.15.113:6829/33080","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":33078},{"type":"v1","addr":"172.21.15.113:6801","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":33078},{"type":"v1","addr":"172.21.15.113:6803","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6807","nonce":33078},{"type":"v1","addr":"172.21.15.113:6810","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":33078},{"type":"v1","addr":"172.21.15.113:6805","nonce":33078}]},"public_addr":"172.21.15.113:6801/33078","cluster_addr":"172.21.15.113:6803/33078","heartbeat_back_addr":"172.21.15.113:6810/33078","heartbeat_front_addr":"172.21.15.113:6805/33078","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":33081},{"type":"v1","addr":"172.21.15.113:6817","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":33081},{"type":"v1","addr":"172.21.15.113:6819","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":33081},{"type":"v1","addr":"172.21.15.113:6824","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":33081},{"type":"v1","addr":"172.21.15.113:6821","nonce":33081}]},"public_addr":"172.21.15.113:6817/33081","cluster_addr":"172.21.15.113:6819/33081","heartbeat_back_addr":"172.21.15.113:6824/33081","heartbeat_front_addr":"172.21.15.113:6821/33081","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":33079},{"type":"v1","addr":"172.21.15.113:6808","nonce":33079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6809","nonce":33079},{"type":"v1","addr":"172.21.15.113:6811","nonce":33079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":33079},{"type":"v1","addr":"172.21.15.113:6815","nonce":33079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":33079},{"type":"v1","addr":"172.21.15.113:6813","nonce":33079}]},"public_addr":"172.21.15.113:6808/33079","cluster_addr":"172.21.15.113:6811/33079","heartbeat_back_addr":"172.21.15.113:6815/33079","heartbeat_front_addr":"172.21.15.113:6813/33079","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.034379+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.947533+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.001359+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.976014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:24.046851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.935345+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.997482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T13:24:23.928059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.91:6835/115193290":"2024-02-03T13:37:53.577122+0000","172.21.15.91:6834/115193290":"2024-02-03T13:37:53.577122+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 msgr2=0x7f73b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 0x7f73b006c8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f73c80074c0 tx=0x7f73c800b040 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 msgr2=0x7f73d80c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c410 tx=0x7f73c000c8d0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 shutdown_connections 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f73b006a430 0x7f73b006c8f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80c9320 0x7f73d80c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73d80cff10 0x7f73d80c8de0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 --2- 172.21.15.91:0/3941826185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d80cdde0 0x7f73d8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 >> 172.21.15.91:0/3941826185 conn(0x7f73d80d0cc0 msgr2=0x7f73d806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 shutdown_connections 2024-02-02T13:38:00.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:00.835+0000 7f73dea00640 1 -- 172.21.15.91:0/3941826185 wait complete. 2024-02-02T13:38:01.857 INFO:tasks.ceph:Scrubbing osd.0 2024-02-02T13:38:01.858 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:02.081 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.080+0000 7f1fd8c00640 1 Processor -- start 2024-02-02T13:38:02.082 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.080+0000 7f1fd8c00640 1 -- start start 2024-02-02T13:38:02.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.081+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fd4160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fd40adfe0 0x7f1fd415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd8c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd40c9260 con 0x7f1fd415e610 2024-02-02T13:38:02.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd415dc40 con 0x7f1fd40adfe0 2024-02-02T13:38:02.083 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd8c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4161460 con 0x7f1fd415d430 2024-02-02T13:38:02.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd2800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1fd415e610 0x7f1fd40ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44268/0 (socket says 172.21.15.91:44268) 2024-02-02T13:38:02.084 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd2800640 1 -- 172.21.15.91:0/3654107953 learned_addr learned my addr 172.21.15.91:0/3654107953 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:02.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083053923 0 0) 0x7f1fd40c9260 con 0x7f1fd415e610 2024-02-02T13:38:02.085 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fd4160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb4003520 con 0x7f1fd415e610 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.082+0000 7f1fd1400640 1 --2- 172.21.15.91:0/3654107953 >> v2:172.21.15.113:3300/0 conn(0x7f1fd40adfe0 0x7f1fd415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196219489 0 0) 0x7f1fb4003520 con 0x7f1fd415e610 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 msgr2=0x7f1fd4160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fd4160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 >> v2:172.21.15.113:3300/0 conn(0x7f1fd40adfe0 msgr2=0x7f1fd415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 --2- 172.21.15.91:0/3654107953 >> v2:172.21.15.113:3300/0 conn(0x7f1fd40adfe0 0x7f1fd415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd4161700 con 0x7f1fd415e610 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f1fcc0019f0 con 0x7f1fd415e610 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 >> v1:172.21.15.91:6789/0 conn(0x7f1fd415e610 legacy=0x7f1fd40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fb4003970 0x7f1fb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.086 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fb4004d00 0x7f1fb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd415dc40 con 0x7f1fb4003970 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4161460 con 0x7f1fb4004d00 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400ba40 con 0x7f1fd415d430 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fb4003970 0x7f1fb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.083+0000 7f1fd1400640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.084+0000 7f1fd1e00640 1 -- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 msgr2=0x7f1fb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.084+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fd415d430 0x7f1fb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.084+0000 7f1fd1e00640 1 -- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fb4004d00 msgr2=0x7f1fb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.084+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fb4004d00 0x7f1fb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.087 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.084+0000 7f1fd1e00640 1 -- 172.21.15.91:0/3654107953 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4003520 con 0x7f1fb4003970 2024-02-02T13:38:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fb4003970 0x7f1fb4003d90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009ac0 tx=0x7f1fb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0788ccf92e99446 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc402c6b0 con 0x7f1fb4003970 2024-02-02T13:38:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc402c810 con 0x7f1fb4003970 2024-02-02T13:38:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd0a00640 1 -- 172.21.15.91:0/3654107953 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc402cba0 con 0x7f1fb4003970 2024-02-02T13:38:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd8c00640 1 -- 172.21.15.91:0/3654107953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fb4003970 msgr2=0x7f1fb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd8c00640 1 --2- 172.21.15.91:0/3654107953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fb4003970 0x7f1fb4003d90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009ac0 tx=0x7f1fb40057f0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.085+0000 7f1fd2800640 1 -- 172.21.15.91:0/3654107953 reap_dead start 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.086+0000 7f1fd8c00640 1 -- 172.21.15.91:0/3654107953 shutdown_connections 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.086+0000 7f1fd8c00640 1 -- 172.21.15.91:0/3654107953 >> 172.21.15.91:0/3654107953 conn(0x7f1fd40ceb90 msgr2=0x7f1fd40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.086+0000 7f1fd8c00640 1 -- 172.21.15.91:0/3654107953 shutdown_connections 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.086+0000 7f1fd8c00640 1 -- 172.21.15.91:0/3654107953 wait complete. 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.087+0000 7f1fd8c00640 1 Processor -- start 2024-02-02T13:38:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 -- start start 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fb4003970 0x7f1fd41f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 0x7f1fd40b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd41f7170 0x7f1fd40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c6e50 con 0x7f1fd40c91f0 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c7130 con 0x7f1fd41f7170 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.088+0000 7f1fd8c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c7410 con 0x7f1fb4003970 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 0x7f1fd40b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 0x7f1fd40b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46516/0 (socket says 172.21.15.91:46516) 2024-02-02T13:38:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 -- 172.21.15.91:0/277118304 learned_addr learned my addr 172.21.15.91:0/277118304 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fb4003970 0x7f1fd41f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 -- 172.21.15.91:0/277118304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fb4003970 msgr2=0x7f1fd41f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fb4003970 0x7f1fd41f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 -- 172.21.15.91:0/277118304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd41f7170 msgr2=0x7f1fd40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd41f7170 0x7f1fd40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40c76f0 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.089+0000 7f1fd1400640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 0x7f1fd40b1dc0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00f600 tx=0x7f1fbc00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.090+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc014870 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.090+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0149d0 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.090+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc014d40 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.090+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd40b2930 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.090+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd40700d0 con 0x7f1fd40c91f0 2024-02-02T13:38:02.093 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.091+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fd40c6fc0 con 0x7f1fd40c91f0 2024-02-02T13:38:02.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.092+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1fbc00d490 con 0x7f1fd40c91f0 2024-02-02T13:38:02.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fca000640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1fac06a460 0x7f1fac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1fbc088fe0 con 0x7f1fd40c91f0 2024-02-02T13:38:02.094 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1fac06a460 0x7f1fac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fca000640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f1fac0702c0 0x7f1fac0726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fac072d90 con 0x7f1fac0702c0 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fbc08b600 con 0x7f1fd40c91f0 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.093+0000 7f1fd2800640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f1fac0702c0 0x7f1fac0726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.094+0000 7f1fd1e00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1fac06a460 0x7f1fac06c920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4036000 tx=0x7f1fc4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.095 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.094+0000 7f1fd2800640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f1fac0702c0 0x7f1fac0726e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.096 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.095+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== osd.0 v2:172.21.15.91:6820/32877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f1fac072d90 con 0x7f1fac0702c0 2024-02-02T13:38:02.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.107+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 --> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1fd40c7580 con 0x7f1fac0702c0 2024-02-02T13:38:02.109 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fca000640 1 -- 172.21.15.91:0/277118304 <== osd.0 v2:172.21.15.91:6820/32877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f1fd40c7580 con 0x7f1fac0702c0 2024-02-02T13:38:02.109 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:02.109 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:02.109 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:02.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f1fac0702c0 msgr2=0x7f1fac0726e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6820/32877,v1:172.21.15.91:6822/32877] conn(0x7f1fac0702c0 0x7f1fac0726e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1fac06a460 msgr2=0x7f1fac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1fac06a460 0x7f1fac06c920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4036000 tx=0x7f1fc4030000 comp rx=0 tx=0).stop 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 msgr2=0x7f1fd40b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 --2- 172.21.15.91:0/277118304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fd40c91f0 0x7f1fd40b1dc0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00f600 tx=0x7f1fbc00fac0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd2800640 1 -- 172.21.15.91:0/277118304 reap_dead start 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 shutdown_connections 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 >> 172.21.15.91:0/277118304 conn(0x7f1fd40ceb90 msgr2=0x7f1fd4157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 shutdown_connections 2024-02-02T13:38:02.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.108+0000 7f1fd8c00640 1 -- 172.21.15.91:0/277118304 wait complete. 2024-02-02T13:38:02.128 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-02-02T13:38:02.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.336+0000 7ff53b600640 1 Processor -- start 2024-02-02T13:38:02.338 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.336+0000 7ff53b600640 1 -- start start 2024-02-02T13:38:02.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff53b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff53415b0f0 0x7ff53406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff53b600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5340cb580 0x7ff5340cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.339 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff53b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53406d910 con 0x7ff53415ad10 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff53b600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53406d730 con 0x7ff5340cb580 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff53b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5340cbf50 con 0x7ff53415b0f0 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5340cb580 0x7ff5340cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff5340cb580 0x7ff5340cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46520/0 (socket says 172.21.15.91:46520) 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 -- 172.21.15.91:0/2589638224 learned_addr learned my addr 172.21.15.91:0/2589638224 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 -- 172.21.15.91:0/2589638224 >> v2:172.21.15.113:3300/0 conn(0x7ff53415b0f0 msgr2=0x7ff53406b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 --2- 172.21.15.91:0/2589638224 >> v2:172.21.15.113:3300/0 conn(0x7ff53415b0f0 0x7ff53406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 -- 172.21.15.91:0/2589638224 >> v1:172.21.15.113:6789/0 conn(0x7ff53415ad10 legacy=0x7ff53406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.340 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.338+0000 7ff539600640 1 -- 172.21.15.91:0/2589638224 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340cc850 con 0x7ff5340cb580 2024-02-02T13:38:02.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.339+0000 7ff539600640 1 --2- 172.21.15.91:0/2589638224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5340cb580 0x7ff5340cba10 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009e50 tx=0x7ff52c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd4afc6756fd051 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.340+0000 7ff533e00640 1 -- 172.21.15.91:0/2589638224 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c02c9f0 con 0x7ff5340cb580 2024-02-02T13:38:02.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.340+0000 7ff533e00640 1 -- 172.21.15.91:0/2589638224 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c02cb50 con 0x7ff5340cb580 2024-02-02T13:38:02.341 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.340+0000 7ff533e00640 1 -- 172.21.15.91:0/2589638224 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0343f0 con 0x7ff5340cb580 2024-02-02T13:38:02.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.340+0000 7ff53b600640 1 -- 172.21.15.91:0/2589638224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5340cb580 msgr2=0x7ff5340cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.340+0000 7ff53b600640 1 --2- 172.21.15.91:0/2589638224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5340cb580 0x7ff5340cba10 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009e50 tx=0x7ff52c02be50 comp rx=0 tx=0).stop 2024-02-02T13:38:02.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 -- 172.21.15.91:0/2589638224 shutdown_connections 2024-02-02T13:38:02.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 --2- 172.21.15.91:0/2589638224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5340cb580 0x7ff5340cba10 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.342 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 --2- 172.21.15.91:0/2589638224 >> v2:172.21.15.113:3300/0 conn(0x7ff53415b0f0 0x7ff53406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 -- 172.21.15.91:0/2589638224 >> 172.21.15.91:0/2589638224 conn(0x7ff5340b0700 msgr2=0x7ff5340b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 -- 172.21.15.91:0/2589638224 shutdown_connections 2024-02-02T13:38:02.343 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.341+0000 7ff53b600640 1 -- 172.21.15.91:0/2589638224 wait complete. 2024-02-02T13:38:02.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.342+0000 7ff53b600640 1 Processor -- start 2024-02-02T13:38:02.344 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.342+0000 7ff53b600640 1 -- start start 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5340cb580 0x7ff5340c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 0x7ff534157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340cab40 con 0x7ff53415b0f0 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341576d0 con 0x7ff5340cb580 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.343+0000 7ff53b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340ca6b0 con 0x7ff53415ad10 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46524/0 (socket says 172.21.15.91:46524) 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 -- 172.21.15.91:0/2796958060 learned_addr learned my addr 172.21.15.91:0/2796958060 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff52bc00640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 0x7ff534157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 -- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 msgr2=0x7ff534157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 0x7ff534157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff538c00640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5340cb580 0x7ff5340c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.346 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 -- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5340cb580 msgr2=0x7ff5340c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5340cb580 0x7ff5340c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff539600640 1 -- 172.21.15.91:0/2796958060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c009b30 con 0x7ff53415b0f0 2024-02-02T13:38:02.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.344+0000 7ff52bc00640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 0x7ff534157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:02.347 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff539600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff52c036040 tx=0x7ff52c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c03e450 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c03e5d0 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c03e980 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340c9aa0 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.345+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5340ca060 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.346+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5340ca840 con 0x7ff53415b0f0 2024-02-02T13:38:02.348 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.347+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff52c03eae0 con 0x7ff53415b0f0 2024-02-02T13:38:02.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.348+0000 7ff531600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 0x7ff50c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.348+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff52c0ab4c0 con 0x7ff53415b0f0 2024-02-02T13:38:02.349 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.348+0000 7ff538c00640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 0x7ff50c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.350 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.349+0000 7ff538c00640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 0x7ff50c06c8f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff5240015d0 tx=0x7ff524009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.353 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.352+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff52c076db0 con 0x7ff53415b0f0 2024-02-02T13:38:02.544 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.543+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5340ccb30 con 0x7ff50c06a430 2024-02-02T13:38:02.545 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.544+0000 7ff531600640 1 -- 172.21.15.91:0/2796958060 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff5240050d0 con 0x7ff50c06a430 2024-02-02T13:38:02.545 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 0 to deep-scrub 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.546+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 msgr2=0x7ff50c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.546+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 0x7ff50c06c8f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff5240015d0 tx=0x7ff524009040 comp rx=0 tx=0).stop 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.546+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 msgr2=0x7ff5340c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.546+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff52c036040 tx=0x7ff52c004a40 comp rx=0 tx=0).stop 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 shutdown_connections 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff50c06a430 0x7ff50c06c8f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff53415ad10 0x7ff534157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.548 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff53415b0f0 0x7ff5340c91c0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 --2- 172.21.15.91:0/2796958060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5340cb580 0x7ff5340c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 >> 172.21.15.91:0/2796958060 conn(0x7ff5340b0700 msgr2=0x7ff53406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 shutdown_connections 2024-02-02T13:38:02.549 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.547+0000 7ff53b600640 1 -- 172.21.15.91:0/2796958060 wait complete. 2024-02-02T13:38:02.566 INFO:tasks.ceph:Scrubbing osd.1 2024-02-02T13:38:02.566 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:02.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.768+0000 7fde0ea00640 1 Processor -- start 2024-02-02T13:38:02.770 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.769+0000 7fde0ea00640 1 -- start start 2024-02-02T13:38:02.772 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 0x7fde08165380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde080cde70 0x7fde080ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ea00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0806b630 con 0x7fde080c9150 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde080ce860 con 0x7fde080cde70 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde08161c30 con 0x7fde080b0ee0 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fddffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde080cde70 0x7fde080ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fddffe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde080cde70 0x7fde080ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46540/0 (socket says 172.21.15.91:46540) 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fde0ca00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fde080c9150 0x7fde08162020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44278/0 (socket says 172.21.15.91:44278) 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.770+0000 7fddffe00640 1 -- 172.21.15.91:0/2155931162 learned_addr learned my addr 172.21.15.91:0/2155931162 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fde07e00640 1 --2- 172.21.15.91:0/2155931162 >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 0x7fde08165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fddffe00640 1 -- 172.21.15.91:0/2155931162 >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 msgr2=0x7fde08165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fddffe00640 1 --2- 172.21.15.91:0/2155931162 >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 0x7fde08165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fddffe00640 1 -- 172.21.15.91:0/2155931162 >> v1:172.21.15.91:6789/0 conn(0x7fde080c9150 legacy=0x7fde08162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.773 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fddffe00640 1 -- 172.21.15.91:0/2155931162 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0806b880 con 0x7fde080cde70 2024-02-02T13:38:02.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fde07e00640 1 --2- 172.21.15.91:0/2155931162 >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 0x7fde08165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:02.774 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.771+0000 7fde07400640 1 -- 172.21.15.91:0/2155931162 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203580450 0 0) 0x7fde0806b630 con 0x7fde080c9150 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.772+0000 7fddffe00640 1 --2- 172.21.15.91:0/2155931162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080cde70 0x7fde080ce320 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fde080cb570 tx=0x7fddf002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e343fe5f80b16c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.772+0000 7fde07400640 1 -- 172.21.15.91:0/2155931162 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0030870 con 0x7fde080cde70 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.772+0000 7fde07400640 1 -- 172.21.15.91:0/2155931162 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf00309d0 con 0x7fde080cde70 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.772+0000 7fde07400640 1 -- 172.21.15.91:0/2155931162 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0030d60 con 0x7fde080cde70 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 -- 172.21.15.91:0/2155931162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080cde70 msgr2=0x7fde080ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 --2- 172.21.15.91:0/2155931162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080cde70 0x7fde080ce320 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fde080cb570 tx=0x7fddf002b9b0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 -- 172.21.15.91:0/2155931162 shutdown_connections 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 --2- 172.21.15.91:0/2155931162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080cde70 0x7fde080ce320 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 --2- 172.21.15.91:0/2155931162 >> v2:172.21.15.113:3300/0 conn(0x7fde080b0ee0 0x7fde08165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 -- 172.21.15.91:0/2155931162 >> 172.21.15.91:0/2155931162 conn(0x7fde080ceb90 msgr2=0x7fde080cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.775 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 -- 172.21.15.91:0/2155931162 shutdown_connections 2024-02-02T13:38:02.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.773+0000 7fde0ea00640 1 -- 172.21.15.91:0/2155931162 wait complete. 2024-02-02T13:38:02.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.775+0000 7fde0ea00640 1 Processor -- start 2024-02-02T13:38:02.776 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.775+0000 7fde0ea00640 1 -- start start 2024-02-02T13:38:02.778 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 0x7fde081638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080c9150 0x7fde0815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080cde70 0x7fde080704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08161eb0 con 0x7fde080b0ee0 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08162190 con 0x7fde080cde70 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde0ea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde08162470 con 0x7fde080c9150 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 0x7fde081638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 0x7fde081638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46548/0 (socket says 172.21.15.91:46548) 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 -- 172.21.15.91:0/4255590688 learned_addr learned my addr 172.21.15.91:0/4255590688 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fddffe00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080c9150 0x7fde0815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 -- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080c9150 msgr2=0x7fde0815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080c9150 0x7fde0815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 -- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080cde70 msgr2=0x7fde080704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080cde70 0x7fde080704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.779 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fde07e00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf00094d0 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.776+0000 7fddffe00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080c9150 0x7fde0815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde07e00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 0x7fde081638c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c7f0 tx=0x7fddf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8007820 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8007980 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8007cf0 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde08162750 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.777+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde08162d10 con 0x7fde080b0ee0 2024-02-02T13:38:02.780 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.778+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde0806be40 con 0x7fde080b0ee0 2024-02-02T13:38:02.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.779+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fddf80113f0 con 0x7fde080b0ee0 2024-02-02T13:38:02.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fde04c00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdde006a430 0x7fdde006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fddf80889e0 con 0x7fde080b0ee0 2024-02-02T13:38:02.781 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fde04c00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7fdde0070290 0x7fdde00726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdde0072d60 con 0x7fdde0070290 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf808d2a0 con 0x7fde080b0ee0 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.780+0000 7fddffe00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdde006a430 0x7fdde006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.781+0000 7fde0ca00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7fdde0070290 0x7fdde00726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.781+0000 7fddffe00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdde006a430 0x7fdde006c8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddf0036000 tx=0x7fddf003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.782 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.781+0000 7fde0ca00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7fdde0070290 0x7fdde00726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:02.783 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.782+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== osd.1 v2:172.21.15.91:6805/32878 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fdde0072d60 con 0x7fdde0070290 2024-02-02T13:38:02.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.794+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 --> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fde0806bb60 con 0x7fdde0070290 2024-02-02T13:38:02.796 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde04c00640 1 -- 172.21.15.91:0/4255590688 <== osd.1 v2:172.21.15.91:6805/32878 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fde00001e10 con 0x7fdde0070290 2024-02-02T13:38:02.796 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:02.797 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:02.797 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:02.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7fdde0070290 msgr2=0x7fdde00726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6805/32878,v1:172.21.15.91:6807/32878] conn(0x7fdde0070290 0x7fdde00726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdde006a430 msgr2=0x7fdde006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fdde006a430 0x7fdde006c8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddf0036000 tx=0x7fddf003b040 comp rx=0 tx=0).stop 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 msgr2=0x7fde081638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ea00640 1 --2- 172.21.15.91:0/4255590688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde080b0ee0 0x7fde081638c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c7f0 tx=0x7fddf800ccb0 comp rx=0 tx=0).stop 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.795+0000 7fde0ca00640 1 -- 172.21.15.91:0/4255590688 reap_dead start 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.796+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 shutdown_connections 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.796+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 >> 172.21.15.91:0/4255590688 conn(0x7fde080ceb90 msgr2=0x7fde08161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.796+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 shutdown_connections 2024-02-02T13:38:02.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:02.796+0000 7fde0ea00640 1 -- 172.21.15.91:0/4255590688 wait complete. 2024-02-02T13:38:02.814 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-02-02T13:38:03.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.021+0000 7f3830c00640 1 Processor -- start 2024-02-02T13:38:03.023 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.022+0000 7f3830c00640 1 -- start start 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.023+0000 7f3830c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f382c161550 0x7f382c071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.023+0000 7f3830c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c161930 con 0x7f382c0cb940 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f3830c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c0c9c70 con 0x7f382c0c2170 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f3830c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c0c9f50 con 0x7f382c161550 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f382a800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f382c0cb940 0x7f382c0cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44292/0 (socket says 172.21.15.91:44292) 2024-02-02T13:38:03.025 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f382a800640 1 -- 172.21.15.91:0/3281628867 learned_addr learned my addr 172.21.15.91:0/3281628867 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:03.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f3829e00640 1 --2- 172.21.15.91:0/3281628867 >> v2:172.21.15.113:3300/0 conn(0x7f382c161550 0x7f382c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602048734 0 0) 0x7f382c161930 con 0x7f382c0cb940 2024-02-02T13:38:03.027 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.024+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f380c003520 con 0x7f382c0cb940 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3869146253 0 0) 0x7f380c003520 con 0x7f382c0cb940 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 >> v2:172.21.15.113:3300/0 conn(0x7f382c161550 msgr2=0x7f382c071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 --2- 172.21.15.91:0/3281628867 >> v2:172.21.15.113:3300/0 conn(0x7f382c161550 0x7f382c071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 >> v1:172.21.15.113:6789/0 conn(0x7f382c0c2170 legacy=0x7f382c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c06f740 con 0x7f382c0cb940 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7f38240019f0 con 0x7f382c0cb940 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 >> v1:172.21.15.91:6789/0 conn(0x7f382c0cb940 legacy=0x7f382c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f380c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f380c0051d0 0x7f380c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c9f50 con 0x7f380c0047c0 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c00c120 con 0x7f380c0051d0 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c00c2e0 con 0x7f380c0039c0 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3829400640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f380c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.028 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.025+0000 7f3829e00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f382a800640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f380c0051d0 0x7f380c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f3829400640 1 -- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f380c0039c0 msgr2=0x7f380c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f3829400640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f3829400640 1 -- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f380c0051d0 msgr2=0x7f380c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f3829400640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f380c0051d0 0x7f380c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f3829400640 1 -- 172.21.15.91:0/3281628867 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c003520 con 0x7f380c0047c0 2024-02-02T13:38:03.029 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.026+0000 7f382a800640 1 -- 172.21.15.91:0/3281628867 reap_dead start 2024-02-02T13:38:03.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.027+0000 7f3829400640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f380c004c40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3814001750 tx=0x7f380c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654149782d7954cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.027+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140052d0 con 0x7f380c0047c0 2024-02-02T13:38:03.030 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.027+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381402e050 con 0x7f380c0047c0 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.027+0000 7f3828a00640 1 -- 172.21.15.91:0/3281628867 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814003830 con 0x7f380c0047c0 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 -- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 msgr2=0x7f380c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f380c004c40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3814001750 tx=0x7f380c0057f0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 -- 172.21.15.91:0/3281628867 shutdown_connections 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 --2- 172.21.15.91:0/3281628867 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f380c004c40 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 -- 172.21.15.91:0/3281628867 >> 172.21.15.91:0/3281628867 conn(0x7f382c0b0700 msgr2=0x7f382c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 -- 172.21.15.91:0/3281628867 shutdown_connections 2024-02-02T13:38:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.028+0000 7f3830c00640 1 -- 172.21.15.91:0/3281628867 wait complete. 2024-02-02T13:38:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.030+0000 7f3830c00640 1 Processor -- start 2024-02-02T13:38:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.030+0000 7f3830c00640 1 -- start start 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c14d9a0 0x7f382c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c159dd0 0x7f382c14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c6570 con 0x7f380c0047c0 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c40c0 con 0x7f382c159dd0 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3830c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c5090 con 0x7f382c14d9a0 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3829e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3829e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46566/0 (socket says 172.21.15.91:46566) 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.031+0000 7f3829e00640 1 -- 172.21.15.91:0/85849976 learned_addr learned my addr 172.21.15.91:0/85849976 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 -- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c14d9a0 msgr2=0x7f382c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c14d9a0 0x7f382c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 -- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c159dd0 msgr2=0x7f382c14cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.033 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c159dd0 0x7f382c14cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 -- 172.21.15.91:0/85849976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c0c4c00 con 0x7f380c0047c0 2024-02-02T13:38:03.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3829e00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f381c00f630 tx=0x7f381c00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c014870 con 0x7f380c0047c0 2024-02-02T13:38:03.034 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c0149d0 con 0x7f380c0047c0 2024-02-02T13:38:03.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c0c42d0 con 0x7f380c0047c0 2024-02-02T13:38:03.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c014d40 con 0x7f380c0047c0 2024-02-02T13:38:03.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.032+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c0c4890 con 0x7f380c0047c0 2024-02-02T13:38:03.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.034+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c066cf0 con 0x7f380c0047c0 2024-02-02T13:38:03.035 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.034+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f381c015410 con 0x7f380c0047c0 2024-02-02T13:38:03.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.035+0000 7f3822000640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.035+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f381c089110 con 0x7f380c0047c0 2024-02-02T13:38:03.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.035+0000 7f3829400640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.037 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.036+0000 7f3829400640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3814033000 tx=0x7f381402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.041 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.040+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f381c054960 con 0x7f380c0047c0 2024-02-02T13:38:03.055 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:03.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:03.238 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.236+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f382c06fa20 con 0x7f380406a430 2024-02-02T13:38:03.239 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.237+0000 7f3822000640 1 -- 172.21.15.91:0/85849976 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f38140052d0 con 0x7f380406a430 2024-02-02T13:38:03.239 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 1 to deep-scrub 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 msgr2=0x7f380406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3814033000 tx=0x7f381402f000 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 msgr2=0x7f382c0c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f381c00f630 tx=0x7f381c00faf0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 shutdown_connections 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f380406a430 0x7f380406c8f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f382c14d9a0 0x7f382c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f382c159dd0 0x7f382c14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 --2- 172.21.15.91:0/85849976 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380c0047c0 0x7f382c0c5c40 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 >> 172.21.15.91:0/85849976 conn(0x7f382c0b0700 msgr2=0x7f382c06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 shutdown_connections 2024-02-02T13:38:03.242 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.240+0000 7f3830c00640 1 -- 172.21.15.91:0/85849976 wait complete. 2024-02-02T13:38:03.260 INFO:tasks.ceph:Scrubbing osd.2 2024-02-02T13:38:03.260 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:03.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.466+0000 7f3054000640 1 Processor -- start 2024-02-02T13:38:03.468 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.466+0000 7f3054000640 1 -- start start 2024-02-02T13:38:03.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3054000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 0x7f304c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3054000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f304c0cb240 0x7f304c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3054000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c0ca1d0 con 0x7f304c159e40 2024-02-02T13:38:03.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3054000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c15db00 con 0x7f304c0cb240 2024-02-02T13:38:03.469 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3054000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15dda0 con 0x7f304c15a220 2024-02-02T13:38:03.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f304c0cb240 0x7f304c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f304c0cb240 0x7f304c0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46582/0 (socket says 172.21.15.91:46582) 2024-02-02T13:38:03.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 -- 172.21.15.91:0/2787270896 learned_addr learned my addr 172.21.15.91:0/2787270896 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:03.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3050c00640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 0x7f304c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.470 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 -- 172.21.15.91:0/2787270896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 msgr2=0x7f304c0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 0x7f304c0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 -- 172.21.15.91:0/2787270896 >> v1:172.21.15.113:6789/0 conn(0x7f304c159e40 legacy=0x7f304c0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.468+0000 7f3052000640 1 -- 172.21.15.91:0/2787270896 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c15e080 con 0x7f304c0cb240 2024-02-02T13:38:03.471 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.469+0000 7f3050c00640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 0x7f304c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:03.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.470+0000 7f3052000640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c0cc7a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e50 tx=0x7f304402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e61bc44ad1417 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.470+0000 7f304be00640 1 -- 172.21.15.91:0/2787270896 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304402ca70 con 0x7f304c0cb240 2024-02-02T13:38:03.472 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.470+0000 7f304be00640 1 -- 172.21.15.91:0/2787270896 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304402cbd0 con 0x7f304c0cb240 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.470+0000 7f304be00640 1 -- 172.21.15.91:0/2787270896 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440343f0 con 0x7f304c0cb240 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 -- 172.21.15.91:0/2787270896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 msgr2=0x7f304c0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c0cc7a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e50 tx=0x7f304402bed0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 -- 172.21.15.91:0/2787270896 shutdown_connections 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c0cc7a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 --2- 172.21.15.91:0/2787270896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c15a220 0x7f304c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 -- 172.21.15.91:0/2787270896 >> 172.21.15.91:0/2787270896 conn(0x7f304c069e30 msgr2=0x7f304c06a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 -- 172.21.15.91:0/2787270896 shutdown_connections 2024-02-02T13:38:03.473 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.471+0000 7f3054000640 1 -- 172.21.15.91:0/2787270896 wait complete. 2024-02-02T13:38:03.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.473+0000 7f3054000640 1 Processor -- start 2024-02-02T13:38:03.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.473+0000 7f3054000640 1 -- start start 2024-02-02T13:38:03.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c159e40 0x7f304c1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f304c15a220 0x7f304c1f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15c840 con 0x7f304c0cb240 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15bc30 con 0x7f304c15a220 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3054000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15bf10 con 0x7f304c159e40 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.474+0000 7f3051600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46584/0 (socket says 172.21.15.91:46584) 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 -- 172.21.15.91:0/4044328086 learned_addr learned my addr 172.21.15.91:0/4044328086 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3050c00640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c159e40 0x7f304c1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 -- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c159e40 msgr2=0x7f304c1f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3052000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f304c15a220 0x7f304c1f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f304c159e40 0x7f304c1f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 -- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f304c15a220 msgr2=0x7f304c1f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f304c15a220 0x7f304c1f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.477 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044009b30 con 0x7f304c0cb240 2024-02-02T13:38:03.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.475+0000 7f3051600640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c15b650 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c430 tx=0x7f303c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.476+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c011790 con 0x7f304c0cb240 2024-02-02T13:38:03.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.476+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c0118f0 con 0x7f304c0cb240 2024-02-02T13:38:03.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.476+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c011ce0 con 0x7f304c0cb240 2024-02-02T13:38:03.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.476+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c15c1f0 con 0x7f304c0cb240 2024-02-02T13:38:03.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.476+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c0c5c60 con 0x7f304c0cb240 2024-02-02T13:38:03.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.477+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f304c15e640 con 0x7f304c0cb240 2024-02-02T13:38:03.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.478+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f303c01c460 con 0x7f304c0cb240 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.479+0000 7f3049600640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f302406a430 0x7f302406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.479+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f303c088630 con 0x7f304c0cb240 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.479+0000 7f3050c00640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f302406a430 0x7f302406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3049600640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f3024070290 0x7f30240726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3024072d60 con 0x7f3024070290 2024-02-02T13:38:03.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f303c08ac10 con 0x7f304c0cb240 2024-02-02T13:38:03.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3052000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f3024070290 0x7f30240726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3050c00640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f302406a430 0x7f302406c8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f304c0c5a50 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.480+0000 7f3052000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f3024070290 0x7f30240726b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.481+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== osd.2 v2:172.21.15.91:6816/32875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f3024072d60 con 0x7f3024070290 2024-02-02T13:38:03.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.494+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 --> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f304c15e360 con 0x7f3024070290 2024-02-02T13:38:03.495 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.494+0000 7f3049600640 1 -- 172.21.15.91:0/4044328086 <== osd.2 v2:172.21.15.91:6816/32875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f304402c970 con 0x7f3024070290 2024-02-02T13:38:03.496 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:03.496 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:03.496 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f3024070290 msgr2=0x7f30240726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6816/32875,v1:172.21.15.91:6817/32875] conn(0x7f3024070290 0x7f30240726b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f302406a430 msgr2=0x7f302406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f302406a430 0x7f302406c8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f304c0c5a50 tx=0x7f3034008040 comp rx=0 tx=0).stop 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 msgr2=0x7f304c15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 --2- 172.21.15.91:0/4044328086 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f304c0cb240 0x7f304c15b650 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c430 tx=0x7f303c00c8f0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3052000640 1 -- 172.21.15.91:0/4044328086 reap_dead start 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 shutdown_connections 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 >> 172.21.15.91:0/4044328086 conn(0x7f304c069e30 msgr2=0x7f304c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 shutdown_connections 2024-02-02T13:38:03.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.495+0000 7f3054000640 1 -- 172.21.15.91:0/4044328086 wait complete. 2024-02-02T13:38:03.520 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-02-02T13:38:03.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.722+0000 7efe1da00640 1 Processor -- start 2024-02-02T13:38:03.723 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.722+0000 7efe1da00640 1 -- start start 2024-02-02T13:38:03.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.723+0000 7efe1da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe181609b0 0x7efe180cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.723+0000 7efe1da00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe18160d90 con 0x7efe180ce530 2024-02-02T13:38:03.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.723+0000 7efe1da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe18070fe0 con 0x7efe18070980 2024-02-02T13:38:03.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.723+0000 7efe1da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180d2940 con 0x7efe181609b0 2024-02-02T13:38:03.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.723+0000 7efe0fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7efe180ce530 0x7efe18070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44302/0 (socket says 172.21.15.91:44302) 2024-02-02T13:38:03.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe0fe00640 1 -- 172.21.15.91:0/1548502250 learned_addr learned my addr 172.21.15.91:0/1548502250 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:03.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16a00640 1 --2- 172.21.15.91:0/1548502250 >> v2:172.21.15.113:3300/0 conn(0x7efe181609b0 0x7efe180cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167756916 0 0) 0x7efe00001110 con 0x7efe180ce530 2024-02-02T13:38:03.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf8003520 con 0x7efe180ce530 2024-02-02T13:38:03.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344449351 0 0) 0x7efdf8003520 con 0x7efe180ce530 2024-02-02T13:38:03.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 >> v2:172.21.15.113:3300/0 conn(0x7efe181609b0 msgr2=0x7efe180cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.729 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 --2- 172.21.15.91:0/1548502250 >> v2:172.21.15.113:3300/0 conn(0x7efe181609b0 0x7efe180cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 >> v1:172.21.15.113:6789/0 conn(0x7efe18070980 legacy=0x7efe180b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180d2c20 con 0x7efe180ce530 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7efe00001800 con 0x7efe180ce530 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 >> v1:172.21.15.91:6789/0 conn(0x7efe180ce530 legacy=0x7efe18070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.724+0000 7efe16000640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80039c0 0x7efdf8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16000640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 0x7efdf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16000640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80052a0 0x7efdf800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180d2940 con 0x7efdf80047c0 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf800c1a0 con 0x7efdf80052a0 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf800c360 con 0x7efdf80039c0 2024-02-02T13:38:03.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 0x7efdf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe17400640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80052a0 0x7efdf800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe16a00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80039c0 0x7efdf8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 -- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80039c0 msgr2=0x7efdf8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80039c0 0x7efdf8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 -- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80052a0 msgr2=0x7efdf800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80052a0 0x7efdf800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe0fe00640 1 -- 172.21.15.91:0/1548502250 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8003520 con 0x7efdf80047c0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.725+0000 7efe17400640 1 -- 172.21.15.91:0/1548502250 reap_dead start 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.726+0000 7efe0fe00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 0x7efdf8004c40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efe180ce910 tx=0x7efdf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6047a56dfe5e8527 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00005ad0 con 0x7efdf80047c0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe00005c50 con 0x7efdf80047c0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe16000640 1 -- 172.21.15.91:0/1548502250 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0002c7c0 con 0x7efdf80047c0 2024-02-02T13:38:03.731 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 -- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 msgr2=0x7efdf8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 0x7efdf8004c40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efe180ce910 tx=0x7efdf80057f0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 -- 172.21.15.91:0/1548502250 shutdown_connections 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 --2- 172.21.15.91:0/1548502250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdf80047c0 0x7efdf8004c40 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 -- 172.21.15.91:0/1548502250 >> 172.21.15.91:0/1548502250 conn(0x7efe180d3020 msgr2=0x7efe180d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 -- 172.21.15.91:0/1548502250 shutdown_connections 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.727+0000 7efe1da00640 1 -- 172.21.15.91:0/1548502250 wait complete. 2024-02-02T13:38:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.730+0000 7efe1da00640 1 Processor -- start 2024-02-02T13:38:03.733 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.731+0000 7efe1da00640 1 -- start start 2024-02-02T13:38:03.735 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80047c0 0x7efe1815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe180aca20 0x7efe1815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe181f69c0 con 0x7efe180acea0 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1815cd90 con 0x7efe180aca20 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe1da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe181f6de0 con 0x7efdf80047c0 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.732+0000 7efe17400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46610/0 (socket says 172.21.15.91:46610) 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 -- 172.21.15.91:0/2045498328 learned_addr learned my addr 172.21.15.91:0/2045498328 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:03.736 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe0fe00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe180aca20 0x7efe1815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe16a00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80047c0 0x7efe1815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 -- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80047c0 msgr2=0x7efe1815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80047c0 0x7efe1815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 -- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe180aca20 msgr2=0x7efe1815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe180aca20 0x7efe1815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 -- 172.21.15.91:0/2045498328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180ad450 con 0x7efe180acea0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe17400640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7efe10007fd0 tx=0x7efe1000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.733+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10006b50 con 0x7efe180acea0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.734+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10006cd0 con 0x7efe180acea0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.734+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10003bb0 con 0x7efe180acea0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.734+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe181f6020 con 0x7efe180acea0 2024-02-02T13:38:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.734+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe181f65e0 con 0x7efe180acea0 2024-02-02T13:38:03.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.735+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18066cf0 con 0x7efe180acea0 2024-02-02T13:38:03.738 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.737+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efe10003d10 con 0x7efe180acea0 2024-02-02T13:38:03.739 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.738+0000 7efe0ea00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 0x7efdf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:03.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.738+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe10087f30 con 0x7efe180acea0 2024-02-02T13:38:03.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.738+0000 7efe16a00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 0x7efdf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:03.740 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.739+0000 7efe16a00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 0x7efdf006c8f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efe180683b0 tx=0x7efe08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.743+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efe10053810 con 0x7efe180acea0 2024-02-02T13:38:03.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.933+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe180b66b0 con 0x7efdf006a430 2024-02-02T13:38:03.934 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.933+0000 7efe0ea00640 1 -- 172.21.15.91:0/2045498328 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe08004020 con 0x7efdf006a430 2024-02-02T13:38:03.935 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 2 to deep-scrub 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.935+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 msgr2=0x7efdf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.935+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 0x7efdf006c8f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efe180683b0 tx=0x7efe08008040 comp rx=0 tx=0).stop 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 msgr2=0x7efe1815faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7efe10007fd0 tx=0x7efe1000ec10 comp rx=0 tx=0).stop 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 shutdown_connections 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7efdf006a430 0x7efdf006c8f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efe180acea0 0x7efe1815faf0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe180aca20 0x7efe1815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 --2- 172.21.15.91:0/2045498328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdf80047c0 0x7efe1815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:03.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 >> 172.21.15.91:0/2045498328 conn(0x7efe180d3020 msgr2=0x7efe180b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:03.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 shutdown_connections 2024-02-02T13:38:03.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:03.936+0000 7efe1da00640 1 -- 172.21.15.91:0/2045498328 wait complete. 2024-02-02T13:38:03.956 INFO:tasks.ceph:Scrubbing osd.3 2024-02-02T13:38:03.956 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:04.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.152+0000 7f8969000640 1 Processor -- start 2024-02-02T13:38:04.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.152+0000 7f8969000640 1 -- start start 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.153+0000 7f8969000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89640b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.153+0000 7f8969000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89640b0bd0 0x7f89640ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.153+0000 7f8969000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89640cca70 con 0x7f896415a1e0 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.153+0000 7f8969000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89640b1060 con 0x7f89640b0bd0 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.153+0000 7f8969000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89640cc770 con 0x7f89640c9f90 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89640b0bd0 0x7f89640ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f89640b0bd0 0x7f89640ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46626/0 (socket says 172.21.15.91:46626) 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 -- 172.21.15.91:0/880128616 learned_addr learned my addr 172.21.15.91:0/880128616 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.155 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f895be00640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89640b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 -- 172.21.15.91:0/880128616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 msgr2=0x7f89640b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:04.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89640b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 -- 172.21.15.91:0/880128616 >> v1:172.21.15.113:6789/0 conn(0x7f896415a1e0 legacy=0x7f8964070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:04.156 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.154+0000 7f8962800640 1 -- 172.21.15.91:0/880128616 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640ca570 con 0x7f89640b0bd0 2024-02-02T13:38:04.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.155+0000 7f8962800640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89640b0bd0 0x7f89640ccf00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f895c009e50 tx=0x7f895c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bfbd694f5d8f50 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.155+0000 7f8961400640 1 -- 172.21.15.91:0/880128616 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c02ca70 con 0x7f89640b0bd0 2024-02-02T13:38:04.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.155+0000 7f8961400640 1 -- 172.21.15.91:0/880128616 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c02cbd0 con 0x7f89640b0bd0 2024-02-02T13:38:04.157 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.155+0000 7f8961400640 1 -- 172.21.15.91:0/880128616 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c0343f0 con 0x7f89640b0bd0 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 -- 172.21.15.91:0/880128616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89640b0bd0 msgr2=0x7f89640ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89640b0bd0 0x7f89640ccf00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f895c009e50 tx=0x7f895c02bed0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 -- 172.21.15.91:0/880128616 shutdown_connections 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f89640b0bd0 0x7f89640ccf00 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 --2- 172.21.15.91:0/880128616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89640b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 -- 172.21.15.91:0/880128616 >> 172.21.15.91:0/880128616 conn(0x7f89640ceb90 msgr2=0x7f89640cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 -- 172.21.15.91:0/880128616 shutdown_connections 2024-02-02T13:38:04.158 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.156+0000 7f8969000640 1 -- 172.21.15.91:0/880128616 wait complete. 2024-02-02T13:38:04.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.158+0000 7f8969000640 1 Processor -- start 2024-02-02T13:38:04.159 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.158+0000 7f8969000640 1 -- start start 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89640b0bd0 0x7f8964158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89641f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 0x7f89641e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641e2ef0 con 0x7f896415a1e0 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641591c0 con 0x7f89640b0bd0 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8969000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89641e3140 con 0x7f89640c9f90 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8962800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 0x7f89641e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8962800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 0x7f89641e57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46638/0 (socket says 172.21.15.91:46638) 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8962800640 1 -- 172.21.15.91:0/4197338464 learned_addr learned my addr 172.21.15.91:0/4197338464 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f895be00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89641f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.159+0000 7f8961e00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89640b0bd0 0x7f8964158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 -- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 msgr2=0x7f89641f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.162 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89641f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 -- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89640b0bd0 msgr2=0x7f8964158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89640b0bd0 0x7f8964158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c009b30 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f895be00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89640c9f90 0x7f89641f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8962800640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 0x7f89641e57a0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f895c036040 tx=0x7f895c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c009510 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c03e450 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89641e3420 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.160+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c03e7e0 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.161+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89641e39e0 con 0x7f896415a1e0 2024-02-02T13:38:04.163 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.161+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89640ca850 con 0x7f896415a1e0 2024-02-02T13:38:04.164 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.163+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f895c03e940 con 0x7f896415a1e0 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.163+0000 7f895a000640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f893c06a430 0x7f893c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f895c0ab460 con 0x7f896415a1e0 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f895a000640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7f893c070290 0x7f893c0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f8961e00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f893c06a430 0x7f893c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f893c072d60 con 0x7f893c070290 2024-02-02T13:38:04.165 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f895c0adb20 con 0x7f896415a1e0 2024-02-02T13:38:04.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f895be00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7f893c070290 0x7f893c0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.164+0000 7f8961e00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f893c06a430 0x7f893c06c8f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8954006fd0 tx=0x7f8954008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.165+0000 7f895be00640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7f893c070290 0x7f893c0726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.166 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.165+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== osd.3 v2:172.21.15.91:6800/32876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f893c072d60 con 0x7f893c070290 2024-02-02T13:38:04.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.178+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 --> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8964066cf0 con 0x7f893c070290 2024-02-02T13:38:04.179 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.178+0000 7f895a000640 1 -- 172.21.15.91:0/4197338464 <== osd.3 v2:172.21.15.91:6800/32876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f8964066cf0 con 0x7f893c070290 2024-02-02T13:38:04.180 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:04.180 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:04.180 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.178+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7f893c070290 msgr2=0x7f893c0726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.178+0000 7f8969000640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6800/32876,v1:172.21.15.91:6801/32876] conn(0x7f893c070290 0x7f893c0726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f893c06a430 msgr2=0x7f893c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f893c06a430 0x7f893c06c8f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8954006fd0 tx=0x7f8954008040 comp rx=0 tx=0).stop 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 msgr2=0x7f89641e57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 --2- 172.21.15.91:0/4197338464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f896415a1e0 0x7f89641e57a0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f895c036040 tx=0x7f895c004a40 comp rx=0 tx=0).stop 2024-02-02T13:38:04.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8962800640 1 -- 172.21.15.91:0/4197338464 reap_dead start 2024-02-02T13:38:04.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 shutdown_connections 2024-02-02T13:38:04.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 >> 172.21.15.91:0/4197338464 conn(0x7f89640ceb90 msgr2=0x7f89640c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 shutdown_connections 2024-02-02T13:38:04.182 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.179+0000 7f8969000640 1 -- 172.21.15.91:0/4197338464 wait complete. 2024-02-02T13:38:04.197 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-02-02T13:38:04.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.388+0000 7f3e84a00640 1 Processor -- start 2024-02-02T13:38:04.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.388+0000 7f3e84a00640 1 -- start start 2024-02-02T13:38:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.389+0000 7f3e84a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.389+0000 7f3e84a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 0x7f3e7c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e84a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c160d90 con 0x7f3e7c0707b0 2024-02-02T13:38:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e84a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c070c40 con 0x7f3e7c0ce530 2024-02-02T13:38:04.391 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e84a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c070fe0 con 0x7f3e7c1609b0 2024-02-02T13:38:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e82000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e81600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 0x7f3e7c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.392 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e82000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.91:58428/0 (socket says 172.21.15.91:58428) 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.390+0000 7f3e82000640 1 -- 172.21.15.91:0/3713825849 learned_addr learned my addr 172.21.15.91:0/3713825849 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.391+0000 7f3e82000640 1 -- 172.21.15.91:0/3713825849 >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 msgr2=0x7f3e7c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.391+0000 7f3e82000640 1 --2- 172.21.15.91:0/3713825849 >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 0x7f3e7c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.391+0000 7f3e82000640 1 -- 172.21.15.91:0/3713825849 >> v1:172.21.15.113:6789/0 conn(0x7f3e7c0707b0 legacy=0x7f3e7c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.391+0000 7f3e82000640 1 -- 172.21.15.91:0/3713825849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0d2940 con 0x7f3e7c1609b0 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.391+0000 7f3e81600640 1 --2- 172.21.15.91:0/3713825849 >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 0x7f3e7c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T13:38:04.393 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.392+0000 7f3e82000640 1 --2- 172.21.15.91:0/3713825849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009870 tx=0x7f3e6c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2361ced7471e76bd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e80c00640 1 -- 172.21.15.91:0/3713825849 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0317c0 con 0x7f3e7c1609b0 2024-02-02T13:38:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e80c00640 1 -- 172.21.15.91:0/3713825849 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c031920 con 0x7f3e7c1609b0 2024-02-02T13:38:04.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e80c00640 1 -- 172.21.15.91:0/3713825849 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c031cb0 con 0x7f3e7c1609b0 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e84a00640 1 -- 172.21.15.91:0/3713825849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 msgr2=0x7f3e7c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3713825849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009870 tx=0x7f3e6c02b860 comp rx=0 tx=0).stop 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.393+0000 7f3e84a00640 1 -- 172.21.15.91:0/3713825849 shutdown_connections 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.394+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3713825849 >> v2:172.21.15.113:3300/0 conn(0x7f3e7c0ce530 0x7f3e7c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.394+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3713825849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c1609b0 0x7f3e7c0cdff0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.394+0000 7f3e84a00640 1 -- 172.21.15.91:0/3713825849 >> 172.21.15.91:0/3713825849 conn(0x7f3e7c0d3020 msgr2=0x7f3e7c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.394+0000 7f3e84a00640 1 -- 172.21.15.91:0/3713825849 shutdown_connections 2024-02-02T13:38:04.395 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.394+0000 7f3e84a00640 1 -- 172.21.15.91:0/3713825849 wait complete. 2024-02-02T13:38:04.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.395+0000 7f3e84a00640 1 Processor -- start 2024-02-02T13:38:04.396 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.395+0000 7f3e84a00640 1 -- start start 2024-02-02T13:38:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 0x7f3e7c155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 0x7f3e7c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c156530 con 0x7f3e7c0707b0 2024-02-02T13:38:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1f6d20 con 0x7f3e7c1609b0 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e84a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c154440 con 0x7f3e7c0ce530 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.396+0000 7f3e82000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46650/0 (socket says 172.21.15.91:46650) 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 -- 172.21.15.91:0/3361809720 learned_addr learned my addr 172.21.15.91:0/3361809720 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 -- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 msgr2=0x7f3e7c155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e81600640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 0x7f3e7c155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 0x7f3e7c155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 -- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 msgr2=0x7f3e7c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 0x7f3e7c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 0x7f3e7c153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 -- 172.21.15.91:0/3361809720 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64009400 con 0x7f3e7c0707b0 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e81600640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 0x7f3e7c155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:04.400 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 0x7f3e7c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.397+0000 7f3e82000640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c02b8a0 tx=0x7f3e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.398+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c032860 con 0x7f3e7c0707b0 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.398+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c032a00 con 0x7f3e7c0707b0 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.398+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c032db0 con 0x7f3e7c0707b0 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.398+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c0094d0 con 0x7f3e7c0707b0 2024-02-02T13:38:04.401 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.398+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c1f6760 con 0x7f3e7c0707b0 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.399+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c066cf0 con 0x7f3e7c0707b0 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.400+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e6c04e450 con 0x7f3e7c0707b0 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.400+0000 7f3e7a000640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 0x7f3e5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.401+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e6c0aa800 con 0x7f3e7c0707b0 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.401+0000 7f3e81600640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 0x7f3e5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.402 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.401+0000 7f3e81600640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 0x7f3e5406c8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c155ad0 tx=0x7f3e64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.410 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.409+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e6c076200 con 0x7f3e7c0707b0 2024-02-02T13:38:04.599 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.598+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e7c0b63d0 con 0x7f3e5406a430 2024-02-02T13:38:04.600 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.598+0000 7f3e7a000640 1 -- 172.21.15.91:0/3361809720 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3e64010df0 con 0x7f3e5406a430 2024-02-02T13:38:04.600 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 3 to deep-scrub 2024-02-02T13:38:04.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 msgr2=0x7f3e5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.602 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 0x7f3e5406c8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c155ad0 tx=0x7f3e64008040 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 msgr2=0x7f3e7c1f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c02b8a0 tx=0x7f3e6c0046c0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 shutdown_connections 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f3e5406a430 0x7f3e5406c8f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e7c0ce530 0x7f3e7c155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e7c1609b0 0x7f3e7c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 --2- 172.21.15.91:0/3361809720 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e7c0707b0 0x7f3e7c1f5e10 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 >> 172.21.15.91:0/3361809720 conn(0x7f3e7c0d3020 msgr2=0x7f3e7c15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 shutdown_connections 2024-02-02T13:38:04.603 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.601+0000 7f3e84a00640 1 -- 172.21.15.91:0/3361809720 wait complete. 2024-02-02T13:38:04.619 INFO:tasks.ceph:Scrubbing osd.4 2024-02-02T13:38:04.619 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:04.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.811+0000 7ff895800640 1 Processor -- start 2024-02-02T13:38:04.812 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.811+0000 7ff895800640 1 -- start start 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff89015b2f0 0x7ff8900b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8900711f0 0x7ff8900b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89015fdc0 con 0x7ff8900711f0 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89015b6d0 con 0x7ff89015b2f0 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.812+0000 7ff895800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89015f1b0 con 0x7ff890071650 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8900711f0 0x7ff8900b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8900711f0 0x7ff8900b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46660/0 (socket says 172.21.15.91:46660) 2024-02-02T13:38:04.814 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 -- 172.21.15.91:0/746704431 learned_addr learned my addr 172.21.15.91:0/746704431 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88ea00640 1 --2- 172.21.15.91:0/746704431 >> v2:172.21.15.113:3300/0 conn(0x7ff89015b2f0 0x7ff8900b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88f400640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 -- 172.21.15.91:0/746704431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 msgr2=0x7ff8900b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 -- 172.21.15.91:0/746704431 >> v2:172.21.15.113:3300/0 conn(0x7ff89015b2f0 msgr2=0x7ff8900b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 --2- 172.21.15.91:0/746704431 >> v2:172.21.15.113:3300/0 conn(0x7ff89015b2f0 0x7ff8900b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88e000640 1 -- 172.21.15.91:0/746704431 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89015f490 con 0x7ff8900711f0 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.813+0000 7ff88f400640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.814+0000 7ff88e000640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8900711f0 0x7ff8900b0690 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff8900b4890 tx=0x7ff87802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a82f6050dc20611 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.816 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.814+0000 7ff88d600640 1 -- 172.21.15.91:0/746704431 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780317c0 con 0x7ff8900711f0 2024-02-02T13:38:04.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff88d600640 1 -- 172.21.15.91:0/746704431 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878031920 con 0x7ff8900711f0 2024-02-02T13:38:04.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff88d600640 1 -- 172.21.15.91:0/746704431 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878031cb0 con 0x7ff8900711f0 2024-02-02T13:38:04.817 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 -- 172.21.15.91:0/746704431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8900711f0 msgr2=0x7ff8900b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8900711f0 0x7ff8900b0690 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff8900b4890 tx=0x7ff87802b860 comp rx=0 tx=0).stop 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 -- 172.21.15.91:0/746704431 shutdown_connections 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 --2- 172.21.15.91:0/746704431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8900711f0 0x7ff8900b0690 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 --2- 172.21.15.91:0/746704431 >> v2:172.21.15.113:3300/0 conn(0x7ff89015b2f0 0x7ff8900b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 -- 172.21.15.91:0/746704431 >> 172.21.15.91:0/746704431 conn(0x7ff8900ceb90 msgr2=0x7ff8900cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 -- 172.21.15.91:0/746704431 shutdown_connections 2024-02-02T13:38:04.818 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.815+0000 7ff895800640 1 -- 172.21.15.91:0/746704431 wait complete. 2024-02-02T13:38:04.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.817+0000 7ff895800640 1 Processor -- start 2024-02-02T13:38:04.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.817+0000 7ff895800640 1 -- start start 2024-02-02T13:38:04.819 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900711f0 0x7ff8901f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 0x7ff8901f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900ac8d0 con 0x7ff89015b2f0 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901f0a60 con 0x7ff8900711f0 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff895800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901f3b80 con 0x7ff890071650 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff88f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 0x7ff8901f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff88f400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 0x7ff8901f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46666/0 (socket says 172.21.15.91:46666) 2024-02-02T13:38:04.820 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.818+0000 7ff88f400640 1 -- 172.21.15.91:0/4087516061 learned_addr learned my addr 172.21.15.91:0/4087516061 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88ea00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900711f0 0x7ff8901f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88e000640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 -- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 msgr2=0x7ff8900ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff890071650 0x7ff8900ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 -- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900711f0 msgr2=0x7ff8901f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900711f0 0x7ff8901f0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780094d0 con 0x7ff89015b2f0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88ea00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900711f0 0x7ff8901f0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff88f400640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 0x7ff8901f2ff0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff88800cd50 tx=0x7ff88800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88800f680 con 0x7ff89015b2f0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800f800 con 0x7ff89015b2f0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.819+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8901f3e60 con 0x7ff89015b2f0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.820+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88800fbb0 con 0x7ff89015b2f0 2024-02-02T13:38:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.820+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8900acdc0 con 0x7ff89015b2f0 2024-02-02T13:38:04.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.820+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff89015fa50 con 0x7ff89015b2f0 2024-02-02T13:38:04.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.822+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff88800b040 con 0x7ff89015b2f0 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.822+0000 7ff886a00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86806a430 0x7ff86806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff888089100 con 0x7ff89015b2f0 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff88ea00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86806a430 0x7ff86806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff886a00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7ff868070290 0x7ff8680726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff868072d60 con 0x7ff868070290 2024-02-02T13:38:04.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff88808b6e0 con 0x7ff89015b2f0 2024-02-02T13:38:04.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff88e000640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7ff868070290 0x7ff8680726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:04.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.823+0000 7ff88ea00640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86806a430 0x7ff86806c8f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff880006fd0 tx=0x7ff880008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.824+0000 7ff88e000640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7ff868070290 0x7ff8680726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:04.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.826+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== osd.4 v2:172.21.15.113:6823/33080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7ff868072d60 con 0x7ff868070290 2024-02-02T13:38:04.839 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.838+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 --> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff89015f770 con 0x7ff868070290 2024-02-02T13:38:04.840 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff886a00640 1 -- 172.21.15.91:0/4087516061 <== osd.4 v2:172.21.15.113:6823/33080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7ff878031b10 con 0x7ff868070290 2024-02-02T13:38:04.840 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:04.840 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:04.840 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7ff868070290 msgr2=0x7ff8680726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.113:6823/33080,v1:172.21.15.113:6825/33080] conn(0x7ff868070290 0x7ff8680726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86806a430 msgr2=0x7ff86806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7ff86806a430 0x7ff86806c8f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff880006fd0 tx=0x7ff880008040 comp rx=0 tx=0).stop 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 msgr2=0x7ff8901f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 --2- 172.21.15.91:0/4087516061 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff89015b2f0 0x7ff8901f2ff0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff88800cd50 tx=0x7ff88800a650 comp rx=0 tx=0).stop 2024-02-02T13:38:04.841 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff88f400640 1 -- 172.21.15.91:0/4087516061 reap_dead start 2024-02-02T13:38:04.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 shutdown_connections 2024-02-02T13:38:04.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 >> 172.21.15.91:0/4087516061 conn(0x7ff8900ceb90 msgr2=0x7ff8900cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:04.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.839+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 shutdown_connections 2024-02-02T13:38:04.842 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:04.840+0000 7ff895800640 1 -- 172.21.15.91:0/4087516061 wait complete. 2024-02-02T13:38:04.857 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-02-02T13:38:05.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.058+0000 7fcb08c00640 1 Processor -- start 2024-02-02T13:38:05.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.059+0000 7fcb08c00640 1 -- start start 2024-02-02T13:38:05.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb08c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb0406ae70 0x7fcb040cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb08c00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb040cde90 0x7fcb040cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb08c00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb040d0480 con 0x7fcb0415d480 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb08c00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040cfec0 con 0x7fcb040cde90 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb08c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040d01a0 con 0x7fcb0406ae70 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb01e00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fcb0415d480 0x7fcb040cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44318/0 (socket says 172.21.15.91:44318) 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb01e00640 1 -- 172.21.15.91:0/2001210545 learned_addr learned my addr 172.21.15.91:0/2001210545 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:05.062 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.060+0000 7fcb02800640 1 --2- 172.21.15.91:0/2001210545 >> v2:172.21.15.91:3300/0 conn(0x7fcb040cde90 0x7fcb040cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb01400640 1 --2- 172.21.15.91:0/2001210545 >> v2:172.21.15.113:3300/0 conn(0x7fcb0406ae70 0x7fcb040cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb00a00640 1 -- 172.21.15.91:0/2001210545 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056847833 0 0) 0x7fcb040d0480 con 0x7fcb0415d480 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb00a00640 1 -- 172.21.15.91:0/2001210545 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae4003520 con 0x7fcb0415d480 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb02800640 1 -- 172.21.15.91:0/2001210545 >> v2:172.21.15.113:3300/0 conn(0x7fcb0406ae70 msgr2=0x7fcb040cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb02800640 1 --2- 172.21.15.91:0/2001210545 >> v2:172.21.15.113:3300/0 conn(0x7fcb0406ae70 0x7fcb040cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb02800640 1 -- 172.21.15.91:0/2001210545 >> v1:172.21.15.91:6789/0 conn(0x7fcb0415d480 legacy=0x7fcb040cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.061+0000 7fcb02800640 1 -- 172.21.15.91:0/2001210545 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040cb580 con 0x7fcb040cde90 2024-02-02T13:38:05.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.062+0000 7fcb02800640 1 --2- 172.21.15.91:0/2001210545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb040cc2c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcb04070e80 tx=0x7fcafc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46b33c3c27dad09 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb00a00640 1 -- 172.21.15.91:0/2001210545 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc031830 con 0x7fcb040cde90 2024-02-02T13:38:05.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb00a00640 1 -- 172.21.15.91:0/2001210545 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc031990 con 0x7fcb040cde90 2024-02-02T13:38:05.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb00a00640 1 -- 172.21.15.91:0/2001210545 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc031d20 con 0x7fcb040cde90 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 -- 172.21.15.91:0/2001210545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 msgr2=0x7fcb040cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2001210545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb040cc2c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcb04070e80 tx=0x7fcafc02b8d0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 -- 172.21.15.91:0/2001210545 shutdown_connections 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2001210545 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb040cc2c0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2001210545 >> v2:172.21.15.113:3300/0 conn(0x7fcb0406ae70 0x7fcb040cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.063+0000 7fcb08c00640 1 -- 172.21.15.91:0/2001210545 >> 172.21.15.91:0/2001210545 conn(0x7fcb0409a160 msgr2=0x7fcb040d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.064+0000 7fcb08c00640 1 -- 172.21.15.91:0/2001210545 shutdown_connections 2024-02-02T13:38:05.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.064+0000 7fcb08c00640 1 -- 172.21.15.91:0/2001210545 wait complete. 2024-02-02T13:38:05.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.065+0000 7fcb08c00640 1 Processor -- start 2024-02-02T13:38:05.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.065+0000 7fcb08c00640 1 -- start start 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb0406ae70 0x7fcb0414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb0415d480 0x7fcb041e6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041e7900 con 0x7fcb040cde90 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040c9280 con 0x7fcb0406ae70 2024-02-02T13:38:05.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb08c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041e6c80 con 0x7fcb0415d480 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb01400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb01400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46676/0 (socket says 172.21.15.91:46676) 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.066+0000 7fcb01400640 1 -- 172.21.15.91:0/2316640560 learned_addr learned my addr 172.21.15.91:0/2316640560 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 -- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb0415d480 msgr2=0x7fcb041e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb0415d480 0x7fcb041e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 -- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb0406ae70 msgr2=0x7fcb0414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb0406ae70 0x7fcb0414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 -- 172.21.15.91:0/2316640560 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009400 con 0x7fcb040cde90 2024-02-02T13:38:05.069 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb01400640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcaec007ef0 tx=0x7fcaec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec014620 con 0x7fcb040cde90 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec014780 con 0x7fcb040cde90 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec014b30 con 0x7fcb040cde90 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.067+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb041e6f60 con 0x7fcb040cde90 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.068+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb041e7520 con 0x7fcb040cde90 2024-02-02T13:38:05.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.069+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04066cf0 con 0x7fcb040cde90 2024-02-02T13:38:05.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.070+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcaec014c90 con 0x7fcb040cde90 2024-02-02T13:38:05.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.070+0000 7fcafa000640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 0x7fcadc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.070+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcaec088fe0 con 0x7fcb040cde90 2024-02-02T13:38:05.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.071+0000 7fcb01e00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 0x7fcadc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.071+0000 7fcb01e00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 0x7fcadc06c8f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4001570 tx=0x7fcaf4007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.076 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.075+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcaec0548e0 con 0x7fcb040cde90 2024-02-02T13:38:05.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.262+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb040ce360 con 0x7fcadc06a430 2024-02-02T13:38:05.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.263+0000 7fcafa000640 1 -- 172.21.15.91:0/2316640560 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcaf40058b0 con 0x7fcadc06a430 2024-02-02T13:38:05.264 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 4 to deep-scrub 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 msgr2=0x7fcadc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 0x7fcadc06c8f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4001570 tx=0x7fcaf4007930 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 msgr2=0x7fcb041e6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcaec007ef0 tx=0x7fcaec00e4b0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 shutdown_connections 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcadc06a430 0x7fcadc06c8f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb0415d480 0x7fcb041e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb040cde90 0x7fcb041e6110 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 --2- 172.21.15.91:0/2316640560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb0406ae70 0x7fcb0414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 >> 172.21.15.91:0/2316640560 conn(0x7fcb0409a160 msgr2=0x7fcb0415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.265+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 shutdown_connections 2024-02-02T13:38:05.267 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.266+0000 7fcb08c00640 1 -- 172.21.15.91:0/2316640560 wait complete. 2024-02-02T13:38:05.282 INFO:tasks.ceph:Scrubbing osd.5 2024-02-02T13:38:05.283 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:05.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.472+0000 7fde50000640 1 Processor -- start 2024-02-02T13:38:05.474 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.472+0000 7fde50000640 1 -- start start 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde50000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde4815a1e0 0x7fde480c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde50000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde480b0da0 0x7fde480ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde50000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480cca70 con 0x7fde48070ea0 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde50000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480cc770 con 0x7fde4815a1e0 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde50000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde480ca570 con 0x7fde480b0da0 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde4815a1e0 0x7fde480c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fde4815a1e0 0x7fde480c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46684/0 (socket says 172.21.15.91:46684) 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 -- 172.21.15.91:0/2371608239 learned_addr learned my addr 172.21.15.91:0/2371608239 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 -- 172.21.15.91:0/2371608239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde480b0da0 msgr2=0x7fde480ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 --2- 172.21.15.91:0/2371608239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde480b0da0 0x7fde480ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 -- 172.21.15.91:0/2371608239 >> v1:172.21.15.113:6789/0 conn(0x7fde48070ea0 legacy=0x7fde480b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.476 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.474+0000 7fde4d600640 1 -- 172.21.15.91:0/2371608239 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480ca850 con 0x7fde4815a1e0 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.475+0000 7fde4d600640 1 --2- 172.21.15.91:0/2371608239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde4815a1e0 0x7fde480c9f90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fde38009e80 tx=0x7fde3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99846dc324752e07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde47e00640 1 -- 172.21.15.91:0/2371608239 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3802ca70 con 0x7fde4815a1e0 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde47e00640 1 -- 172.21.15.91:0/2371608239 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3802cbd0 con 0x7fde4815a1e0 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde47e00640 1 -- 172.21.15.91:0/2371608239 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380343f0 con 0x7fde4815a1e0 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 -- 172.21.15.91:0/2371608239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde4815a1e0 msgr2=0x7fde480c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 --2- 172.21.15.91:0/2371608239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde4815a1e0 0x7fde480c9f90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fde38009e80 tx=0x7fde3802bed0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 -- 172.21.15.91:0/2371608239 shutdown_connections 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 --2- 172.21.15.91:0/2371608239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde480b0da0 0x7fde480ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 --2- 172.21.15.91:0/2371608239 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde4815a1e0 0x7fde480c9f90 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 -- 172.21.15.91:0/2371608239 >> 172.21.15.91:0/2371608239 conn(0x7fde480ceb90 msgr2=0x7fde480cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 -- 172.21.15.91:0/2371608239 shutdown_connections 2024-02-02T13:38:05.478 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.476+0000 7fde50000640 1 -- 172.21.15.91:0/2371608239 wait complete. 2024-02-02T13:38:05.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.478+0000 7fde50000640 1 Processor -- start 2024-02-02T13:38:05.479 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.478+0000 7fde50000640 1 -- start start 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 0x7fde4814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480b0da0 0x7fde480c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4815a1e0 0x7fde480af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4814df50 con 0x7fde48070ea0 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c6780 con 0x7fde480b0da0 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde50000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c6a20 con 0x7fde4815a1e0 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde4d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 0x7fde4814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.481 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde4d600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 0x7fde4814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46688/0 (socket says 172.21.15.91:46688) 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.479+0000 7fde4d600640 1 -- 172.21.15.91:0/83278824 learned_addr learned my addr 172.21.15.91:0/83278824 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 -- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4815a1e0 msgr2=0x7fde480af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4815a1e0 0x7fde480af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 -- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480b0da0 msgr2=0x7fde480c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480b0da0 0x7fde480c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde38009b30 con 0x7fde48070ea0 2024-02-02T13:38:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde4d600640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 0x7fde4814d700 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fde38009e80 tx=0x7fde38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3803e450 con 0x7fde48070ea0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3803e5b0 con 0x7fde48070ea0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3803e960 con 0x7fde48070ea0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde480c6d00 con 0x7fde48070ea0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.480+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4814dc40 con 0x7fde48070ea0 2024-02-02T13:38:05.483 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.481+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde48071330 con 0x7fde48070ea0 2024-02-02T13:38:05.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.482+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde3803eac0 con 0x7fde48070ea0 2024-02-02T13:38:05.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde45600640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde2006a430 0x7fde2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.484 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fde380ab600 con 0x7fde48070ea0 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde45600640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7fde20070290 0x7fde200726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde20072d60 con 0x7fde20070290 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde4cc00640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde2006a430 0x7fde2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.483+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde380adbe0 con 0x7fde48070ea0 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.484+0000 7fde4cc00640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde2006a430 0x7fde2006c8f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fde480b0220 tx=0x7fde30007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.485 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.484+0000 7fde4e000640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7fde20070290 0x7fde200726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.486 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.485+0000 7fde4e000640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7fde20070290 0x7fde200726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.488 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.487+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== osd.5 v2:172.21.15.113:6800/33078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fde20072d60 con 0x7fde20070290 2024-02-02T13:38:05.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.498+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 --> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fde480c71c0 con 0x7fde20070290 2024-02-02T13:38:05.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.499+0000 7fde45600640 1 -- 172.21.15.91:0/83278824 <== osd.5 v2:172.21.15.113:6800/33078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fde480c71c0 con 0x7fde20070290 2024-02-02T13:38:05.501 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:05.501 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:05.501 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7fde20070290 msgr2=0x7fde200726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.113:6800/33078,v1:172.21.15.113:6801/33078] conn(0x7fde20070290 0x7fde200726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde2006a430 msgr2=0x7fde2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fde2006a430 0x7fde2006c8f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fde480b0220 tx=0x7fde30007990 comp rx=0 tx=0).stop 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 msgr2=0x7fde4814d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 --2- 172.21.15.91:0/83278824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde48070ea0 0x7fde4814d700 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fde38009e80 tx=0x7fde38004a40 comp rx=0 tx=0).stop 2024-02-02T13:38:05.502 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde4e000640 1 -- 172.21.15.91:0/83278824 reap_dead start 2024-02-02T13:38:05.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 shutdown_connections 2024-02-02T13:38:05.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 >> 172.21.15.91:0/83278824 conn(0x7fde480ceb90 msgr2=0x7fde480ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 shutdown_connections 2024-02-02T13:38:05.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.500+0000 7fde50000640 1 -- 172.21.15.91:0/83278824 wait complete. 2024-02-02T13:38:05.518 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-02-02T13:38:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.711+0000 7fe815200640 1 Processor -- start 2024-02-02T13:38:05.713 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.712+0000 7fe815200640 1 -- start start 2024-02-02T13:38:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe815200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe810070270 0x7fe8100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.714 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe815200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe810160d90 con 0x7fe8100b6bd0 2024-02-02T13:38:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe815200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe810070bf0 con 0x7fe8101609b0 2024-02-02T13:38:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe815200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe810070fe0 con 0x7fe810070270 2024-02-02T13:38:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe80e800640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe8100b6bd0 0x7fe8100b7020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44320/0 (socket says 172.21.15.91:44320) 2024-02-02T13:38:05.715 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.713+0000 7fe80e800640 1 -- 172.21.15.91:0/2490878107 learned_addr learned my addr 172.21.15.91:0/2490878107 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:05.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80d400640 1 --2- 172.21.15.91:0/2490878107 >> v2:172.21.15.113:3300/0 conn(0x7fe810070270 0x7fe8100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378569650 0 0) 0x7fe808001110 con 0x7fe8100b6bd0 2024-02-02T13:38:05.716 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f0003520 con 0x7fe8100b6bd0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902495985 0 0) 0x7fe7f0003520 con 0x7fe8100b6bd0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 >> v2:172.21.15.113:3300/0 conn(0x7fe810070270 msgr2=0x7fe8100706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 --2- 172.21.15.91:0/2490878107 >> v2:172.21.15.113:3300/0 conn(0x7fe810070270 0x7fe8100706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 >> v1:172.21.15.113:6789/0 conn(0x7fe8101609b0 legacy=0x7fe8100cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100d2940 con 0x7fe8100b6bd0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2658807289 0 0) 0x7fe808001800 con 0x7fe8100b6bd0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 >> v1:172.21.15.91:6789/0 conn(0x7fe8100b6bd0 legacy=0x7fe8100b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0003970 0x7fe7f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f00047c0 0x7fe7f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 0x7fe7f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810070fe0 con 0x7fe7f0005380 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f000c210 con 0x7fe7f0003970 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f000c3d0 con 0x7fe7f00047c0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.714+0000 7fe80e800640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 0x7fe7f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.717 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80de00640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0003970 0x7fe7f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80d400640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f00047c0 0x7fe7f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80e800640 1 -- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f00047c0 msgr2=0x7fe7f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80e800640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f00047c0 0x7fe7f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80e800640 1 -- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0003970 msgr2=0x7fe7f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80e800640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0003970 0x7fe7f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.718 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.715+0000 7fe80e800640 1 -- 172.21.15.91:0/2490878107 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0003520 con 0x7fe7f0005380 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.716+0000 7fe80e800640 1 -- 172.21.15.91:0/2490878107 reap_dead start 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.716+0000 7fe80e800640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 0x7fe7f000b980 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c240 tx=0x7fe7f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01b26f24cca1b6e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.716+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808005ad0 con 0x7fe7f0005380 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.716+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe808005c50 con 0x7fe7f0005380 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.716+0000 7fe80ca00640 1 -- 172.21.15.91:0/2490878107 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80802c8b0 con 0x7fe7f0005380 2024-02-02T13:38:05.719 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 -- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 msgr2=0x7fe7f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 0x7fe7f000b980 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c240 tx=0x7fe7f00057f0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 -- 172.21.15.91:0/2490878107 shutdown_connections 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 --2- 172.21.15.91:0/2490878107 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7f0005380 0x7fe7f000b980 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 -- 172.21.15.91:0/2490878107 >> 172.21.15.91:0/2490878107 conn(0x7fe8100d3020 msgr2=0x7fe8100d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 -- 172.21.15.91:0/2490878107 shutdown_connections 2024-02-02T13:38:05.720 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.717+0000 7fe815200640 1 -- 172.21.15.91:0/2490878107 wait complete. 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.719+0000 7fe815200640 1 Processor -- start 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.719+0000 7fe815200640 1 -- start start 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 0x7fe8100aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 0x7fe810150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100cc860 con 0x7fe8100cb4e0 2024-02-02T13:38:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810150ec0 con 0x7fe7f0005380 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe815200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100cbc50 con 0x7fe810150500 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46700/0 (socket says 172.21.15.91:46700) 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 -- 172.21.15.91:0/2573849219 learned_addr learned my addr 172.21.15.91:0/2573849219 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80de00640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 0x7fe8100aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80e800640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 0x7fe810150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 -- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 msgr2=0x7fe810150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 0x7fe810150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 -- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 msgr2=0x7fe8100aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 0x7fe8100aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80d400640 1 -- 172.21.15.91:0/2573849219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100cbf30 con 0x7fe8100cb4e0 2024-02-02T13:38:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80de00640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 0x7fe8100aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.720+0000 7fe80e800640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 0x7fe810150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe80d400640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe800009ce0 tx=0x7fe800007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800014610 con 0x7fe8100cb4e0 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000f050 con 0x7fe8100cb4e0 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000149b0 con 0x7fe8100cb4e0 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8100cc210 con 0x7fe8100cb4e0 2024-02-02T13:38:05.726 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.721+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8100cb960 con 0x7fe8100cb4e0 2024-02-02T13:38:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.722+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8100cc9d0 con 0x7fe8100cb4e0 2024-02-02T13:38:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.723+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe800014b10 con 0x7fe8100cb4e0 2024-02-02T13:38:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.724+0000 7fe806000640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 0x7fe7e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.724+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe800088ed0 con 0x7fe8100cb4e0 2024-02-02T13:38:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.724+0000 7fe80de00640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 0x7fe7e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.725+0000 7fe80de00640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 0x7fe7e806c920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005580 tx=0x7fe7f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:05.730 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.728+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe800054720 con 0x7fe8100cb4e0 2024-02-02T13:38:05.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.902+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8100b63d0 con 0x7fe7e806a460 2024-02-02T13:38:05.903 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.902+0000 7fe806000640 1 -- 172.21.15.91:0/2573849219 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe7f8001430 con 0x7fe7e806a460 2024-02-02T13:38:05.903 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 5 to deep-scrub 2024-02-02T13:38:05.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.904+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 msgr2=0x7fe7e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.904+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 0x7fe7e806c920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005580 tx=0x7fe7f8009040 comp rx=0 tx=0).stop 2024-02-02T13:38:05.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.904+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 msgr2=0x7fe81015e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:05.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.904+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe800009ce0 tx=0x7fe800007700 comp rx=0 tx=0).stop 2024-02-02T13:38:05.906 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 shutdown_connections 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe7e806a460 0x7fe7e806c920 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe810150500 0x7fe810150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe8100cb4e0 0x7fe81015e710 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 --2- 172.21.15.91:0/2573849219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f0005380 0x7fe8100aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 >> 172.21.15.91:0/2573849219 conn(0x7fe8100d3020 msgr2=0x7fe81015f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 shutdown_connections 2024-02-02T13:38:05.907 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:05.905+0000 7fe815200640 1 -- 172.21.15.91:0/2573849219 wait complete. 2024-02-02T13:38:05.923 INFO:tasks.ceph:Scrubbing osd.6 2024-02-02T13:38:05.923 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:06.110 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.109+0000 7fad1ae00640 1 Processor -- start 2024-02-02T13:38:06.111 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.109+0000 7fad1ae00640 1 -- start start 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.110+0000 7fad1ae00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad1415d430 0x7fad14160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.110+0000 7fad1ae00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fad140ac8d0 0x7fad140acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad1ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad140c9260 con 0x7fad140adfe0 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad1ae00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad140ad280 con 0x7fad140ac8d0 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad1ae00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415dc40 con 0x7fad1415d430 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fad140ac8d0 0x7fad140acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fad140ac8d0 0x7fad140acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46712/0 (socket says 172.21.15.91:46712) 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 -- 172.21.15.91:0/1082271749 learned_addr learned my addr 172.21.15.91:0/1082271749 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad13e00640 1 --2- 172.21.15.91:0/1082271749 >> v2:172.21.15.113:3300/0 conn(0x7fad1415d430 0x7fad14160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.113 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 -- 172.21.15.91:0/1082271749 >> v2:172.21.15.113:3300/0 conn(0x7fad1415d430 msgr2=0x7fad14160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:06.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 --2- 172.21.15.91:0/1082271749 >> v2:172.21.15.113:3300/0 conn(0x7fad1415d430 0x7fad14160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 -- 172.21.15.91:0/1082271749 >> v1:172.21.15.113:6789/0 conn(0x7fad140adfe0 legacy=0x7fad1415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:06.114 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.111+0000 7fad18e00640 1 -- 172.21.15.91:0/1082271749 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14161460 con 0x7fad140ac8d0 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.112+0000 7fad18e00640 1 --2- 172.21.15.91:0/1082271749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad140ac8d0 0x7fad140acd40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009e50 tx=0x7fad0c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb29dc2ce5c86f5d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad13400640 1 -- 172.21.15.91:0/1082271749 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c02cb20 con 0x7fad140ac8d0 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad13400640 1 -- 172.21.15.91:0/1082271749 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c02cc80 con 0x7fad140ac8d0 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad13400640 1 -- 172.21.15.91:0/1082271749 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0343f0 con 0x7fad140ac8d0 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 -- 172.21.15.91:0/1082271749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad140ac8d0 msgr2=0x7fad140acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 --2- 172.21.15.91:0/1082271749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad140ac8d0 0x7fad140acd40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009e50 tx=0x7fad0c02bf80 comp rx=0 tx=0).stop 2024-02-02T13:38:06.115 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 -- 172.21.15.91:0/1082271749 shutdown_connections 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 --2- 172.21.15.91:0/1082271749 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad140ac8d0 0x7fad140acd40 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 --2- 172.21.15.91:0/1082271749 >> v2:172.21.15.113:3300/0 conn(0x7fad1415d430 0x7fad14160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 -- 172.21.15.91:0/1082271749 >> 172.21.15.91:0/1082271749 conn(0x7fad140ceb90 msgr2=0x7fad140cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 -- 172.21.15.91:0/1082271749 shutdown_connections 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.113+0000 7fad1ae00640 1 -- 172.21.15.91:0/1082271749 wait complete. 2024-02-02T13:38:06.116 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.115+0000 7fad1ae00640 1 Processor -- start 2024-02-02T13:38:06.117 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.115+0000 7fad1ae00640 1 -- start start 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ac8d0 0x7fad14155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad140adfe0 0x7fad140b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 0x7fad1415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14156100 con 0x7fad1415d430 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14159880 con 0x7fad140ac8d0 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad1ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad14159b60 con 0x7fad140adfe0 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.116+0000 7fad18e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 0x7fad1415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 0x7fad1415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46716/0 (socket says 172.21.15.91:46716) 2024-02-02T13:38:06.118 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 -- 172.21.15.91:0/2206008208 learned_addr learned my addr 172.21.15.91:0/2206008208 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad0b200640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad140adfe0 0x7fad140b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad13e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ac8d0 0x7fad14155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 -- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad140adfe0 msgr2=0x7fad140b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad140adfe0 0x7fad140b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 -- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ac8d0 msgr2=0x7fad14155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ac8d0 0x7fad14155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c009b30 con 0x7fad1415d430 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad18e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 0x7fad1415ac00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fad0c036040 tx=0x7fad0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.117+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009510 con 0x7fad1415d430 2024-02-02T13:38:06.120 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.118+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c007e50 con 0x7fad1415d430 2024-02-02T13:38:06.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.118+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad14159e40 con 0x7fad1415d430 2024-02-02T13:38:06.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.118+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c03e690 con 0x7fad1415d430 2024-02-02T13:38:06.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.118+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1415a400 con 0x7fad1415d430 2024-02-02T13:38:06.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.119+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad141619e0 con 0x7fad1415d430 2024-02-02T13:38:06.121 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.120+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fad0c03e7f0 con 0x7fad1415d430 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.120+0000 7fad10c00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7facec06a430 0x7facec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.120+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad0c0ab2e0 con 0x7fad1415d430 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad13e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7facec06a430 0x7facec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad10c00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7facec070290 0x7facec0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facec072d60 con 0x7facec070290 2024-02-02T13:38:06.122 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0c0ad9a0 con 0x7fad1415d430 2024-02-02T13:38:06.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad13e00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7facec06a430 0x7facec06c8f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b7c0 tx=0x7fad04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.121+0000 7fad0b200640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7facec070290 0x7facec0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.123 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.122+0000 7fad0b200640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7facec070290 0x7facec0726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.125 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.124+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== osd.6 v2:172.21.15.113:6816/33081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7facec072d60 con 0x7facec070290 2024-02-02T13:38:06.137 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.136+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 --> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fad14161700 con 0x7facec070290 2024-02-02T13:38:06.138 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.137+0000 7fad10c00640 1 -- 172.21.15.91:0/2206008208 <== osd.6 v2:172.21.15.113:6816/33081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7facfc004fd0 con 0x7facec070290 2024-02-02T13:38:06.139 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:06.139 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:06.139 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.137+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7facec070290 msgr2=0x7facec0726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.137+0000 7fad1ae00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.113:6816/33081,v1:172.21.15.113:6817/33081] conn(0x7facec070290 0x7facec0726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7facec06a430 msgr2=0x7facec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7facec06a430 0x7facec06c8f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b7c0 tx=0x7fad04008040 comp rx=0 tx=0).stop 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 msgr2=0x7fad1415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 --2- 172.21.15.91:0/2206008208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad1415d430 0x7fad1415ac00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fad0c036040 tx=0x7fad0c004a40 comp rx=0 tx=0).stop 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad18e00640 1 -- 172.21.15.91:0/2206008208 reap_dead start 2024-02-02T13:38:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 shutdown_connections 2024-02-02T13:38:06.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 >> 172.21.15.91:0/2206008208 conn(0x7fad140ceb90 msgr2=0x7fad140cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 shutdown_connections 2024-02-02T13:38:06.141 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.138+0000 7fad1ae00640 1 -- 172.21.15.91:0/2206008208 wait complete. 2024-02-02T13:38:06.156 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-02-02T13:38:06.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.359+0000 7f1dd5e00640 1 Processor -- start 2024-02-02T13:38:06.361 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.359+0000 7f1dd5e00640 1 -- start start 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dd5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1dd015d3a0 0x7f1dd00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dd5e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1dd00ac960 0x7f1dd00acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dd5e00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd00c94a0 con 0x7f1dd00cb580 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dd5e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd00ad6e0 con 0x7f1dd00ac960 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dd5e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd00c96f0 con 0x7f1dd015d3a0 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dcea00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f1dd00cb580 0x7f1dd00cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44326/0 (socket says 172.21.15.91:44326) 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dcea00640 1 -- 172.21.15.91:0/947787810 learned_addr learned my addr 172.21.15.91:0/947787810 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:06.363 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dcf400640 1 --2- 172.21.15.91:0/947787810 >> v2:172.21.15.91:3300/0 conn(0x7f1dd00ac960 0x7f1dd00acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dc5e00640 1 --2- 172.21.15.91:0/947787810 >> v2:172.21.15.113:3300/0 conn(0x7f1dd015d3a0 0x7f1dd00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384478846 0 0) 0x7f1dd00c94a0 con 0x7f1dd00cb580 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.361+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db0003520 con 0x7f1dd00cb580 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.362+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 <== mon.0 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363625115 0 0) 0x7f1db0003520 con 0x7f1dd00cb580 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.362+0000 7f1dcf400640 1 -- 172.21.15.91:0/947787810 >> v2:172.21.15.113:3300/0 conn(0x7f1dd015d3a0 msgr2=0x7f1dd00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.362+0000 7f1dcf400640 1 --2- 172.21.15.91:0/947787810 >> v2:172.21.15.113:3300/0 conn(0x7f1dd015d3a0 0x7f1dd00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.362+0000 7f1dcf400640 1 -- 172.21.15.91:0/947787810 >> v1:172.21.15.91:6789/0 conn(0x7f1dd00cb580 legacy=0x7f1dd00cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.364 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.362+0000 7f1dcf400640 1 -- 172.21.15.91:0/947787810 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00c99d0 con 0x7f1dd00ac960 2024-02-02T13:38:06.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.363+0000 7f1dcf400640 1 --2- 172.21.15.91:0/947787810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd00acdf0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00ca7a0 tx=0x7f1dc802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4fdab74a02777f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.363+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00019f0 con 0x7f1dd00ac960 2024-02-02T13:38:06.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.363+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc8033440 con 0x7f1dd00ac960 2024-02-02T13:38:06.365 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.363+0000 7f1dce000640 1 -- 172.21.15.91:0/947787810 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80337d0 con 0x7f1dd00ac960 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 -- 172.21.15.91:0/947787810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 msgr2=0x7f1dd00acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/947787810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd00acdf0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00ca7a0 tx=0x7f1dc802b7a0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 -- 172.21.15.91:0/947787810 shutdown_connections 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/947787810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd00acdf0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/947787810 >> v2:172.21.15.113:3300/0 conn(0x7f1dd015d3a0 0x7f1dd00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 -- 172.21.15.91:0/947787810 >> 172.21.15.91:0/947787810 conn(0x7f1dd00b0700 msgr2=0x7f1dd00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 -- 172.21.15.91:0/947787810 shutdown_connections 2024-02-02T13:38:06.366 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.364+0000 7f1dd5e00640 1 -- 172.21.15.91:0/947787810 wait complete. 2024-02-02T13:38:06.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.366+0000 7f1dd5e00640 1 Processor -- start 2024-02-02T13:38:06.367 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.366+0000 7f1dd5e00640 1 -- start start 2024-02-02T13:38:06.369 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dd00cb580 0x7f1dd00c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dd015d3a0 0x7f1dd00c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd015f280 con 0x7f1dd00ac960 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd015f520 con 0x7f1dd015d3a0 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dd5e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd015f800 con 0x7f1dd00cb580 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46724/0 (socket says 172.21.15.91:46724) 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 -- 172.21.15.91:0/3800076093 learned_addr learned my addr 172.21.15.91:0/3800076093 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dc5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dd00cb580 0x7f1dd00c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 -- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dd00cb580 msgr2=0x7f1dd00c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcf400640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dd015d3a0 0x7f1dd00c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dd00cb580 0x7f1dd00c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.367+0000 7f1dcea00640 1 -- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dd015d3a0 msgr2=0x7f1dd00c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.370 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dcea00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dd015d3a0 0x7f1dd00c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dcea00640 1 -- 172.21.15.91:0/3800076093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc8009400 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dcea00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0001ff0 tx=0x7f1dc000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0012400 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0012560 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd00ad4b0 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0012930 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.368+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd015fdc0 con 0x7f1dd00ac960 2024-02-02T13:38:06.371 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.369+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd0066cf0 con 0x7f1dd00ac960 2024-02-02T13:38:06.372 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.370+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1dc0012a90 con 0x7f1dd00ac960 2024-02-02T13:38:06.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.371+0000 7f1dc7e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 0x7f1da806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.371+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1dc0088e60 con 0x7f1dd00ac960 2024-02-02T13:38:06.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.372+0000 7f1dc5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 0x7f1da806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.373 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.372+0000 7f1dc5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 0x7f1da806c8f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1dd015c290 tx=0x7f1db8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.376+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1dc0054740 con 0x7f1dd00ac960 2024-02-02T13:38:06.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.563+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd00ad330 con 0x7f1da806a430 2024-02-02T13:38:06.565 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.564+0000 7f1dc7e00640 1 -- 172.21.15.91:0/3800076093 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1db80095c0 con 0x7f1da806a430 2024-02-02T13:38:06.565 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 6 to deep-scrub 2024-02-02T13:38:06.567 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 msgr2=0x7f1da806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 0x7f1da806c8f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1dd015c290 tx=0x7f1db8008040 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 msgr2=0x7f1dd0154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0001ff0 tx=0x7f1dc000a430 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 shutdown_connections 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f1da806a430 0x7f1da806c8f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dd00cb580 0x7f1dd00c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dd015d3a0 0x7f1dd00c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 --2- 172.21.15.91:0/3800076093 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1dd00ac960 0x7f1dd0154b90 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 >> 172.21.15.91:0/3800076093 conn(0x7f1dd00b0700 msgr2=0x7f1dd015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 shutdown_connections 2024-02-02T13:38:06.568 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.566+0000 7f1dd5e00640 1 -- 172.21.15.91:0/3800076093 wait complete. 2024-02-02T13:38:06.586 INFO:tasks.ceph:Scrubbing osd.7 2024-02-02T13:38:06.586 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-02-02T13:38:06.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.793+0000 7f6836000640 1 Processor -- start 2024-02-02T13:38:06.795 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.794+0000 7f6836000640 1 -- start start 2024-02-02T13:38:06.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f6836000640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f683015a1e0 0x7f68300c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f6836000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6830070ea0 0x7f68300b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f6836000640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300b1020 con 0x7f68300b0bd0 2024-02-02T13:38:06.797 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f6836000640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300cca70 con 0x7f683015a1e0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f6836000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300cc770 con 0x7f6830070ea0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f682fe00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f68300b0bd0 0x7f683015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:44342/0 (socket says 172.21.15.91:44342) 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f682fe00640 1 -- 172.21.15.91:0/1764813495 learned_addr learned my addr 172.21.15.91:0/1764813495 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.795+0000 7f682f400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f683015a1e0 0x7f68300c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682ea00640 1 --2- 172.21.15.91:0/1764813495 >> v2:172.21.15.113:3300/0 conn(0x7f6830070ea0 0x7f68300b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682e000640 1 -- 172.21.15.91:0/1764813495 <== mon.0 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618236715 0 0) 0x7f6828001490 con 0x7f68300b0bd0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682e000640 1 -- 172.21.15.91:0/1764813495 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6810003520 con 0x7f68300b0bd0 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682f400640 1 -- 172.21.15.91:0/1764813495 >> v2:172.21.15.113:3300/0 conn(0x7f6830070ea0 msgr2=0x7f68300b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682f400640 1 --2- 172.21.15.91:0/1764813495 >> v2:172.21.15.113:3300/0 conn(0x7f6830070ea0 0x7f68300b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682f400640 1 -- 172.21.15.91:0/1764813495 >> v1:172.21.15.91:6789/0 conn(0x7f68300b0bd0 legacy=0x7f683015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.798 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.796+0000 7f682f400640 1 -- 172.21.15.91:0/1764813495 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300ca570 con 0x7f683015a1e0 2024-02-02T13:38:06.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.797+0000 7f682f400640 1 --2- 172.21.15.91:0/1764813495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c9f90 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f68200097a0 tx=0x7f682002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb682dfafbf851 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.797+0000 7f682e000640 1 -- 172.21.15.91:0/1764813495 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820031830 con 0x7f683015a1e0 2024-02-02T13:38:06.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f682e000640 1 -- 172.21.15.91:0/1764813495 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6820031990 con 0x7f683015a1e0 2024-02-02T13:38:06.799 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f682e000640 1 -- 172.21.15.91:0/1764813495 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820031d00 con 0x7f683015a1e0 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 -- 172.21.15.91:0/1764813495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 msgr2=0x7f68300c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 --2- 172.21.15.91:0/1764813495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c9f90 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f68200097a0 tx=0x7f682002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 -- 172.21.15.91:0/1764813495 shutdown_connections 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 --2- 172.21.15.91:0/1764813495 >> v2:172.21.15.113:3300/0 conn(0x7f6830070ea0 0x7f68300b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 --2- 172.21.15.91:0/1764813495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c9f90 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 -- 172.21.15.91:0/1764813495 >> 172.21.15.91:0/1764813495 conn(0x7f68300ceb90 msgr2=0x7f68300cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 -- 172.21.15.91:0/1764813495 shutdown_connections 2024-02-02T13:38:06.800 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.798+0000 7f6836000640 1 -- 172.21.15.91:0/1764813495 wait complete. 2024-02-02T13:38:06.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.800+0000 7f6836000640 1 Processor -- start 2024-02-02T13:38:06.801 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.800+0000 7f6836000640 1 -- start start 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6830070ea0 0x7f6830154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68300b0bd0 0x7f68300c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830154aa0 con 0x7f683015a1e0 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301546c0 con 0x7f6830070ea0 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f6836000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b3c90 con 0x7f68300b0bd0 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f682fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.803 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f682fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46744/0 (socket says 172.21.15.91:46744) 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.801+0000 7f682fe00640 1 -- 172.21.15.91:0/2656011421 learned_addr learned my addr 172.21.15.91:0/2656011421 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 -- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68300b0bd0 msgr2=0x7f68300c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68300b0bd0 0x7f68300c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 -- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6830070ea0 msgr2=0x7f6830154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6830070ea0 0x7f6830154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820009400 con 0x7f683015a1e0 2024-02-02T13:38:06.804 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f682fe00640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c6110 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f682800cb40 tx=0x7f682800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800aca0 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800ae00 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828012620 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68300b1470 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.802+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68300b1a30 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.803+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68300ca850 con 0x7f683015a1e0 2024-02-02T13:38:06.805 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.804+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6828007070 con 0x7f683015a1e0 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f6827400640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f680806a430 0x7f680806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6828088550 con 0x7f683015a1e0 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f682f400640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f680806a430 0x7f680806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f6827400640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7f6808070290 0x7f68080726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6808072d60 con 0x7f6808070290 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.805+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f682808ab30 con 0x7f683015a1e0 2024-02-02T13:38:06.807 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.806+0000 7f682f400640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f680806a430 0x7f680806c8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6820036000 tx=0x7f6820030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.806+0000 7f682ea00640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7f6808070290 0x7f68080726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:06.808 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.807+0000 7f682ea00640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7f6808070290 0x7f68080726b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:06.810 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.809+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== osd.7 v2:172.21.15.113:6806/33079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f6808072d60 con 0x7f6808070290 2024-02-02T13:38:06.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.821+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 --> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f68300b1020 con 0x7f6808070290 2024-02-02T13:38:06.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6827400640 1 -- 172.21.15.91:0/2656011421 <== osd.7 v2:172.21.15.113:6806/33079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f68300b1020 con 0x7f6808070290 2024-02-02T13:38:06.824 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2024-02-02T13:38:06.824 INFO:teuthology.orchestra.run.smithi091.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-02-02T13:38:06.824 INFO:teuthology.orchestra.run.smithi091.stdout:} 2024-02-02T13:38:06.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7f6808070290 msgr2=0x7f68080726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.113:6806/33079,v1:172.21.15.113:6808/33079] conn(0x7f6808070290 0x7f68080726b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f680806a430 msgr2=0x7f680806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f680806a430 0x7f680806c8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6820036000 tx=0x7f6820030000 comp rx=0 tx=0).stop 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 msgr2=0x7f68300c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.823+0000 7f6836000640 1 --2- 172.21.15.91:0/2656011421 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f683015a1e0 0x7f68300c6110 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f682800cb40 tx=0x7f682800a060 comp rx=0 tx=0).stop 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.824+0000 7f682fe00640 1 -- 172.21.15.91:0/2656011421 reap_dead start 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.824+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 shutdown_connections 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.824+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 >> 172.21.15.91:0/2656011421 conn(0x7f68300ceb90 msgr2=0x7f68300adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.824+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 shutdown_connections 2024-02-02T13:38:06.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:06.824+0000 7f6836000640 1 -- 172.21.15.91:0/2656011421 wait complete. 2024-02-02T13:38:06.842 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-02-02T13:38:07.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.053+0000 7f8d44400640 1 Processor -- start 2024-02-02T13:38:07.055 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.053+0000 7f8d44400640 1 -- start start 2024-02-02T13:38:07.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d44400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c15b0f0 0x7f8d3c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.056 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d44400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d3c0cb580 0x7f8d3c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d44400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c06d910 con 0x7f8d3c15ad10 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d44400640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c06d730 con 0x7f8d3c0cb580 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d44400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cbf50 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d42400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d3c0cb580 0x7f8d3c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d42400640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8d3c0cb580 0x7f8d3c0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46746/0 (socket says 172.21.15.91:46746) 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d42400640 1 -- 172.21.15.91:0/3056421162 learned_addr learned my addr 172.21.15.91:0/3056421162 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:07.057 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.055+0000 7f8d41000640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c15b0f0 0x7f8d3c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.056+0000 7f8d42400640 1 -- 172.21.15.91:0/3056421162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c15b0f0 msgr2=0x7f8d3c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:07.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.056+0000 7f8d42400640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c15b0f0 0x7f8d3c06b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.056+0000 7f8d42400640 1 -- 172.21.15.91:0/3056421162 >> v1:172.21.15.113:6789/0 conn(0x7f8d3c15ad10 legacy=0x7f8d3c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.056+0000 7f8d42400640 1 -- 172.21.15.91:0/3056421162 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0cc850 con 0x7f8d3c0cb580 2024-02-02T13:38:07.058 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.057+0000 7f8d42400640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c0cb580 0x7f8d3c0cba10 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d3402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e08ead3bec0caee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.057+0000 7f8d3be00640 1 -- 172.21.15.91:0/3056421162 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34030840 con 0x7f8d3c0cb580 2024-02-02T13:38:07.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.057+0000 7f8d3be00640 1 -- 172.21.15.91:0/3056421162 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d340309a0 con 0x7f8d3c0cb580 2024-02-02T13:38:07.059 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.057+0000 7f8d3be00640 1 -- 172.21.15.91:0/3056421162 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34030d30 con 0x7f8d3c0cb580 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 -- 172.21.15.91:0/3056421162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c0cb580 msgr2=0x7f8d3c0cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c0cb580 0x7f8d3c0cba10 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d3402b980 comp rx=0 tx=0).stop 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 -- 172.21.15.91:0/3056421162 shutdown_connections 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c0cb580 0x7f8d3c0cba10 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 --2- 172.21.15.91:0/3056421162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c15b0f0 0x7f8d3c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.060 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 -- 172.21.15.91:0/3056421162 >> 172.21.15.91:0/3056421162 conn(0x7f8d3c0b0700 msgr2=0x7f8d3c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:07.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 -- 172.21.15.91:0/3056421162 shutdown_connections 2024-02-02T13:38:07.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.058+0000 7f8d44400640 1 -- 172.21.15.91:0/3056421162 wait complete. 2024-02-02T13:38:07.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.060+0000 7f8d44400640 1 Processor -- start 2024-02-02T13:38:07.061 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.060+0000 7f8d44400640 1 -- start start 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c0cb580 0x7f8d3c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3c15ad10 0x7f8d3c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0c7fa0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0a8b50 con 0x7f8d3c15ad10 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.061+0000 7f8d44400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0a8e30 con 0x7f8d3c0cb580 2024-02-02T13:38:07.063 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46748/0 (socket says 172.21.15.91:46748) 2024-02-02T13:38:07.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 -- 172.21.15.91:0/3457259438 learned_addr learned my addr 172.21.15.91:0/3457259438 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:07.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d41a00640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c0cb580 0x7f8d3c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.064 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d41000640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3c15ad10 0x7f8d3c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 -- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c0cb580 msgr2=0x7f8d3c0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c0cb580 0x7f8d3c0a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 -- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3c15ad10 msgr2=0x7f8d3c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3c15ad10 0x7f8d3c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.062+0000 7f8d42400640 1 -- 172.21.15.91:0/3457259438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340094d0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.065 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d42400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34039c90 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34039e10 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c0a9110 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34043640 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.063+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0a96d0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.064+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c066cf0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.065+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8d340437a0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.066+0000 7f8d39600640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 0x7f8d1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.066+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d3403e070 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.066+0000 7f8d41a00640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 0x7f8d1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.068 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.067+0000 7f8d41a00640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 0x7f8d1406c8f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f60 tx=0x7f8d2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.072 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.071+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8d34076ea0 con 0x7f8d3c15b0f0 2024-02-02T13:38:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.258+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d3c0ccb30 con 0x7f8d1406a430 2024-02-02T13:38:07.260 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.259+0000 7f8d39600640 1 -- 172.21.15.91:0/3457259438 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8d2c0014a0 con 0x7f8d1406a430 2024-02-02T13:38:07.260 INFO:teuthology.orchestra.run.smithi091.stderr:instructed osd(s) 7 to deep-scrub 2024-02-02T13:38:07.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 msgr2=0x7f8d1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 0x7f8d1406c8f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f60 tx=0x7f8d2c009040 comp rx=0 tx=0).stop 2024-02-02T13:38:07.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 msgr2=0x7f8d3c0afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d34004730 comp rx=0 tx=0).stop 2024-02-02T13:38:07.263 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 shutdown_connections 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f8d1406a430 0x7f8d1406c8f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8d3c15b0f0 0x7f8d3c0afe90 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3c15ad10 0x7f8d3c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 --2- 172.21.15.91:0/3457259438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d3c0cb580 0x7f8d3c0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.261+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 >> 172.21.15.91:0/3457259438 conn(0x7f8d3c0b0700 msgr2=0x7f8d3c0c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.262+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 shutdown_connections 2024-02-02T13:38:07.264 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.262+0000 7f8d44400640 1 -- 172.21.15.91:0/3457259438 wait complete. 2024-02-02T13:38:07.281 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:38:07.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.485+0000 7f7fa8600640 1 Processor -- start 2024-02-02T13:38:07.487 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.486+0000 7f7fa8600640 1 -- start start 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa8600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7fa00b0c40 0x7f7fa00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa8600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa015fa00 con 0x7f7fa0151f90 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa8600640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00b10d0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0070f80 con 0x7f7fa015f5a0 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa6600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7fa00b0c40 0x7f7fa00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.489 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa6600640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7fa00b0c40 0x7f7fa00b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46758/0 (socket says 172.21.15.91:46758) 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.487+0000 7f7fa6600640 1 -- 172.21.15.91:0/1629508050 learned_addr learned my addr 172.21.15.91:0/1629508050 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.488+0000 7f7fa5200640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.488+0000 7f7fa6600640 1 -- 172.21.15.91:0/1629508050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 msgr2=0x7f7fa00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.488+0000 7f7fa6600640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa00b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.488+0000 7f7fa6600640 1 -- 172.21.15.91:0/1629508050 >> v1:172.21.15.113:6789/0 conn(0x7f7fa0151f90 legacy=0x7f7fa006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:07.490 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.488+0000 7f7fa6600640 1 -- 172.21.15.91:0/1629508050 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0152370 con 0x7f7fa00b0c40 2024-02-02T13:38:07.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.489+0000 7f7fa6600640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa00b3af0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcba5696282f7de8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.489+0000 7f7f9fe00640 1 -- 172.21.15.91:0/1629508050 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98031640 con 0x7f7fa00b0c40 2024-02-02T13:38:07.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.489+0000 7f7f9fe00640 1 -- 172.21.15.91:0/1629508050 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f980317a0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.491 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.489+0000 7f7f9fe00640 1 -- 172.21.15.91:0/1629508050 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98031b30 con 0x7f7fa00b0c40 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 -- 172.21.15.91:0/1629508050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 msgr2=0x7f7fa00b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa00b3af0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f9802b8f0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 -- 172.21.15.91:0/1629508050 shutdown_connections 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa00b3af0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 --2- 172.21.15.91:0/1629508050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa00b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 -- 172.21.15.91:0/1629508050 >> 172.21.15.91:0/1629508050 conn(0x7f7fa00cec80 msgr2=0x7f7fa00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 -- 172.21.15.91:0/1629508050 shutdown_connections 2024-02-02T13:38:07.492 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.490+0000 7f7fa8600640 1 -- 172.21.15.91:0/1629508050 wait complete. 2024-02-02T13:38:07.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.492+0000 7f7fa8600640 1 Processor -- start 2024-02-02T13:38:07.493 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.492+0000 7f7fa8600640 1 -- start start 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fa0151f90 0x7f7fa014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c6d10 con 0x7f7fa00b0c40 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c7820 con 0x7f7fa0151f90 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa8600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa014b690 con 0x7f7fa015f5a0 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa5c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa5c00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:46764/0 (socket says 172.21.15.91:46764) 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.493+0000 7f7fa5c00640 1 -- 172.21.15.91:0/776581002 learned_addr learned my addr 172.21.15.91:0/776581002 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa6600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.496 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 -- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 msgr2=0x7f7fa015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 -- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fa0151f90 msgr2=0x7f7fa014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fa0151f90 0x7f7fa014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 -- 172.21.15.91:0/776581002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980094d0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa5c00640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000ba50 tx=0x7f7f9000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000cbe0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9000cd80 con 0x7f7fa00b0c40 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa00c6ef0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.497 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90014680 con 0x7f7fa00b0c40 2024-02-02T13:38:07.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.494+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa00c74b0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.495+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0066cf0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.498 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.496+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7f900147e0 con 0x7f7fa00b0c40 2024-02-02T13:38:07.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.498+0000 7f7f9d600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 0x7f7f7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:07.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.498+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f90088420 con 0x7f7fa00b0c40 2024-02-02T13:38:07.499 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.498+0000 7f7fa5200640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 0x7f7f7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:07.500 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.499+0000 7f7fa5200640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 0x7f7f7806c8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b3fa0 tx=0x7f7f88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:07.503 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.502+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7f90053e50 con 0x7f7fa00b0c40 2024-02-02T13:38:07.691 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.689+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fa00cb2f0 con 0x7f7f7806a430 2024-02-02T13:38:07.694 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.693+0000 7f7f9d600640 1 -- 172.21.15.91:0/776581002 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47589 (secure 0 0 0) 0x7f7f88009e40 con 0x7f7f7806a430 2024-02-02T13:38:07.695 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:38:07.696 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 msgr2=0x7f7f7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 0x7f7f7806c8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b3fa0 tx=0x7f7f88008040 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 msgr2=0x7f7fa015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000ba50 tx=0x7f7f9000bf10 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 shutdown_connections 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7f7f7806a430 0x7f7f7806c8f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fa015f5a0 0x7f7fa015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fa0151f90 0x7f7fa014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 --2- 172.21.15.91:0/776581002 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7fa00b0c40 0x7f7fa015dec0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 >> 172.21.15.91:0/776581002 conn(0x7f7fa00cec80 msgr2=0x7f7fa00c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 shutdown_connections 2024-02-02T13:38:07.698 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:07.696+0000 7f7fa8600640 1 -- 172.21.15.91:0/776581002 wait complete. 2024-02-02T13:38:07.715 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":604,"stamp":"2024-02-02T13:38:06.512609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10060,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552439,"statfs":{"total":1600000000,"available":1589698196,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003028"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":344,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-02-02T13:37:55.699022+0000","last_change":"2024-02-02T13:24:27.821895+0000","last_active":"2024-02-02T13:37:55.699022+0000","last_peered":"2024-02-02T13:37:55.699022+0000","last_clean":"2024-02-02T13:37:55.699022+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:37:55.699022+0000","last_undegraded":"2024-02-02T13:37:55.699022+0000","last_fullsized":"2024-02-02T13:37:55.699022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T13:24:26.766692+0000","last_clean_scrub_stamp":"2024-02-02T13:24:26.766692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T14:13:51.095214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193622,"statfs":{"total":200000000,"available":198268989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.1080000000000001,"15min":1.087},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6459999999999999,"5min":2.266,"15min":2.5310000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":1.117},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":4.1609999999999996},"last":1.093}]},{"osd":1,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.105,"15min":1.1000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.159},{"interface":"front","average":{"1min":1.181,"5min":1.179,"15min":1.161},"min":{"1min":0.628,"5min":0.628,"15min":0.56299999999999994},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":1.1950000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.2130000000000001,"15min":1.228},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.621,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.375},{"interface":"front","average":{"1min":1.208,"5min":1.252,"15min":1.204},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.5819999999999999},"last":1.272}]},{"osd":3,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.216,"15min":1.1839999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.47},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3160000000000001,"15min":1.321},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.48999999999999999},"max":{"1min":1.883,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":1.5900000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.2929999999999999,"15min":1.26},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":4.242,"5min":4.242,"15min":4.242},"last":1.429},{"interface":"front","average":{"1min":1.25,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.61199999999999999},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.9550000000000001,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.56200000000000006},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5189999999999999,"15min":1.5149999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.63700000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.3240000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.121,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.64000000000000001,"15min":0.498},"max":{"1min":1.5469999999999999,"5min":1.603,"15min":2.3780000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.3660000000000001,"5min":1.776,"15min":1.776},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.0660000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.54000000000000004},"max":{"1min":1.478,"5min":1.4790000000000001,"15min":1.556},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.55900000000000005},"max":{"1min":1.51,"5min":1.677,"15min":2.9300000000000002},"last":1.1759999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2390000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.84799999999999998,"15min":0.63700000000000001},"max":{"1min":1.556,"5min":1.75,"15min":2.6309999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.212,"15min":1.1799999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.63},"max":{"1min":1.4890000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.91400000000000003}]},{"osd":3,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.022},{"interface":"front","average":{"1min":1.242,"5min":1.2689999999999999,"15min":1.1970000000000001},"min":{"1min":0.96899999999999997,"5min":0.90600000000000003,"15min":0.61299999999999999},"max":{"1min":1.6399999999999999,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.94499999999999995,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.5469999999999999,"5min":1.8420000000000001,"15min":17.686},"last":1.456},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4350000000000001,"15min":1.4710000000000001},"min":{"1min":1.022,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.78,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.3029999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.2849999999999999},"min":{"1min":1.0269999999999999,"5min":0.82399999999999995,"15min":0.45900000000000002},"max":{"1min":1.595,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4390000000000001,"15min":1.4079999999999999},"min":{"1min":1.1040000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8400000000000001,"5min":1.95,"15min":3.331},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.464,"15min":1.4610000000000001},"min":{"1min":1.0640000000000001,"5min":0.80300000000000005,"15min":0.61199999999999999},"max":{"1min":1.871,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4590000000000001,"15min":1.4430000000000001},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.65300000000000002},"max":{"1min":1.9970000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.369}]}]},{"osd":5,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.093,"15min":1.046},"min":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.8,"15min":1.8},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.49199999999999999},"max":{"1min":1.4930000000000001,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":1.1419999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1839999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.841,"15min":4.5899999999999999},"last":1.02},{"interface":"front","average":{"1min":1.143,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4500000000000002},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.95699999999999996,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6419999999999999,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2430000000000001,"15min":1.2010000000000001},"min":{"1min":0.94899999999999995,"5min":0.84099999999999997,"15min":0.65300000000000002},"max":{"1min":1.498,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.282,"15min":1.2190000000000001},"min":{"1min":0.93100000000000005,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.7769999999999999},"last":1.327},{"interface":"front","average":{"1min":1.375,"5min":1.2909999999999999,"15min":1.248},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":5.4610000000000003},"last":1.306}]},{"osd":4,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1950000000000001,"15min":1.2030000000000001},"min":{"1min":0.77700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.8460000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.157,"5min":1.3,"15min":1.29},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.55100000000000005},"max":{"1min":1.8440000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.2410000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.329},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.64700000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.387},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":0.98099999999999998,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8109999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.413,"15min":1.4119999999999999},"min":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.7190000000000001,"5min":1.893,"15min":1.9670000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.468,"15min":1.4379999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.61599999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.452}]}]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0609999999999999,"15min":1.0429999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.395,"5min":1.4570000000000001,"15min":4.5949999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.077},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"max":{"1min":1.474,"5min":1.6579999999999999,"15min":1.839},"last":0.84299999999999997}]},{"osd":1,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.095,"15min":1.056},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.3540000000000001,"5min":1.54,"15min":2.4809999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.123,"15min":1.0960000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.5029999999999999,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":0.88500000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.1379999999999999},"min":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.68500000000000005},"max":{"1min":1.405,"5min":1.556,"15min":3.1200000000000001},"last":1.04},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1919999999999999,"15min":1.1559999999999999},"min":{"1min":0.86799999999999999,"5min":0.71199999999999997,"15min":0.58799999999999997},"max":{"1min":1.4399999999999999,"5min":1.736,"15min":2.6000000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.278,"15min":1.2450000000000001},"min":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.68999999999999995},"max":{"1min":1.5309999999999999,"5min":1.637,"15min":2.9079999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":0.94999999999999996}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1850000000000001,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6120000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.29,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.4529999999999998},"last":1.6639999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3999999999999999,"15min":1.371},"min":{"1min":0.99199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.617,"5min":2.403,"15min":2.7989999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.423,"15min":1.429},"min":{"1min":1.0189999999999999,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.45,"15min":1.4410000000000001},"min":{"1min":1.008,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.825,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.488},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.466,"15min":1.4690000000000001},"min":{"1min":1.133,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.984,"5min":2.0409999999999999,"15min":2.698},"last":1.3009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.76000000000000001},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.2190000000000001,"5min":1.379,"15min":1.4199999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.93000000000000005,"15min":0.90500000000000003},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.286,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.94599999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.39700000000000002},"max":{"1min":1.925,"5min":1.925,"15min":2.403},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.081},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.47999999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.2709999999999999},"last":1.022}]},{"osd":2,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1359999999999999,"15min":1.04},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.95599999999999996,"15min":0.91700000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.41499999999999998},"max":{"1min":1.3380000000000001,"5min":1.45,"15min":3.4510000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.204,"15min":1.1899999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.53000000000000003},"max":{"1min":1.573,"5min":1.5940000000000001,"15min":3.585},"last":1.145},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3129999999999999,"15min":1.258},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.61299999999999999},"max":{"1min":1.764,"5min":1.8149999999999999,"15min":1.877},"last":1.325}]},{"osd":5,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.234,"15min":1.204},"min":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.57099999999999995},"max":{"1min":1.4279999999999999,"5min":1.609,"15min":3.738},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2749999999999999,"15min":1.258},"min":{"1min":0.89200000000000002,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":1.288}]},{"osd":6,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3460000000000001},"min":{"1min":1.0409999999999999,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":1.377},{"interface":"front","average":{"1min":1.153,"5min":1.2709999999999999,"15min":1.2689999999999999},"min":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.5269999999999999,"5min":1.667,"15min":3.6059999999999999},"last":1.0449999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3380000000000001,"15min":1.2869999999999999},"min":{"1min":1.032,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.845,"5min":1.851,"15min":3.391},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4239999999999999,"15min":1.365},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.625},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.0409999999999999},"last":1.528}]}]},{"osd":2,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.27900000000000003},"max":{"1min":1.238,"5min":1.345,"15min":7.1840000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0169999999999999,"15min":0.90600000000000003},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.411,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":1.1679999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2270000000000001,"5min":1.387,"15min":2.1680000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.071,"15min":0.98199999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.35899999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.6499999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98999999999999999,"15min":0.90800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":2.1339999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.109,"15min":1.0509999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.432},"max":{"1min":1.488,"5min":1.653,"15min":2.6589999999999998},"last":1.1100000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2969999999999999,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.56799999999999995},"max":{"1min":1.5369999999999999,"5min":1.877,"15min":2.8759999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2330000000000001},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.503},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":2.1360000000000001},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.246,"15min":1.1910000000000001},"min":{"1min":1.1100000000000001,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":1.504,"5min":2.028,"15min":2.8239999999999998},"last":1.288},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.337,"15min":1.3069999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.627},"max":{"1min":1.696,"5min":2.21,"15min":3.0070000000000001},"last":1.252}]},{"osd":6,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.371,"15min":1.2929999999999999},"min":{"1min":1.01,"5min":0.98999999999999999,"15min":0.53800000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.8620000000000001},"last":1.528},{"interface":"front","average":{"1min":1.357,"5min":1.329,"15min":1.2789999999999999},"min":{"1min":1.054,"5min":0.93999999999999995,"15min":0.70499999999999996},"max":{"1min":1.79,"5min":1.7949999999999999,"15min":2.9750000000000001},"last":1.204}]},{"osd":7,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4219999999999999,"15min":1.341},"min":{"1min":1.242,"5min":1.1220000000000001,"15min":0.65100000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.7749999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3680000000000001,"15min":1.3129999999999999},"min":{"1min":1.028,"5min":0.96699999999999997,"15min":0.52500000000000002},"max":{"1min":1.7629999999999999,"5min":1.9330000000000001,"15min":3.0830000000000002},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84399999999999997,"15min":0.87},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":1.46,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.96299999999999997,"15min":1.046},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.308},"max":{"1min":1.387,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.53000000000000003,"5min":0.45200000000000001,"15min":0.34200000000000003},"max":{"1min":1.321,"5min":1.615,"15min":1.639},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":3.0259999999999998},"last":0.87}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.704,"15min":3.1539999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.778,"15min":4.8760000000000003},"last":0.95699999999999996}]},{"osd":4,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.163,"15min":1.1859999999999999},"min":{"1min":0.83099999999999996,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":2.9449999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.95299999999999996,"5min":0.82699999999999996,"15min":0.68899999999999995},"max":{"1min":1.7430000000000001,"5min":1.748,"15min":3.2970000000000002},"last":1.4890000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3400000000000001,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.753},"max":{"1min":1.8420000000000001,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.022,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":3.0249999999999999},"last":1.2829999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.069,"5min":0.998,"15min":0.73599999999999999},"max":{"1min":1.879,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3420000000000001,"15min":1.3660000000000001},"min":{"1min":1.036,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.9910000000000001},"last":1.325}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.77100000000000002},"max":{"1min":1.804,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4710000000000001,"15min":1.4650000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.78000000000000003},"max":{"1min":2.0169999999999999,"5min":2.0699999999999998,"15min":4.3040000000000003},"last":1.3380000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193623,"statfs":{"total":200000000,"available":198270008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86699999999999999,"15min":0.90500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27300000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":4.8070000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89800000000000002,"15min":1.0580000000000001},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":10.816000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.0009999999999999},"min":{"1min":0.53300000000000003,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.325,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.97399999999999998,"15min":1.0960000000000001},"min":{"1min":0.49199999999999999,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.6830000000000001,"15min":10.781000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95599999999999996,"15min":1.0069999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.379},"max":{"1min":1.3440000000000001,"5min":1.607,"15min":5.8410000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0549999999999999,"15min":1.1140000000000001},"min":{"1min":0.54400000000000004,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.653,"5min":1.653,"15min":4.9530000000000003},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.93899999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.607,"5min":2.129,"15min":3.669},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.175,"5min":1.26,"15min":1.292},"min":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73199999999999998},"max":{"1min":1.8580000000000001,"5min":1.893,"15min":4.8780000000000001},"last":1.2629999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.26,"15min":1.415},"min":{"1min":0.77800000000000002,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":2.278,"5min":2.278,"15min":10.651},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.2909999999999999},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.64400000000000002},"max":{"1min":1.526,"5min":1.954,"15min":3.7869999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.341,"15min":1.359},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.66500000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.9210000000000003},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.296,"15min":1.444},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8,"5min":1.95,"15min":10.708},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.347,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":10.74},"last":1.04},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.98299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":3.863},"last":1.0229999999999999}]}]}],"pool_statfs":[]}} 2024-02-02T13:38:07.719 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-02-02T13:24:26.766692+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=2, tm_hour=13, tm_min=24, tm_sec=26, tm_wday=4, tm_yday=33, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=2, tm_hour=13, tm_min=38, tm_sec=0, tm_wday=4, tm_yday=33, tm_isdst=0) 2024-02-02T13:38:07.719 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-02-02T13:38:27.721 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T13:38:27.937 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.937+0000 7fe206e00640 1 Processor -- start 2024-02-02T13:38:27.938 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.937+0000 7fe206e00640 1 -- start start 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.938+0000 7fe206e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe200151f90 0x7fe20015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.938+0000 7fe206e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe2000b0e10 0x7fe2000b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.938+0000 7fe206e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe200070f80 con 0x7fe20006f4f0 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe206e00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe200152370 con 0x7fe2000b0e10 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe206e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000cb010 con 0x7fe200151f90 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe2000b0e10 0x7fe2000b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:27.939 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe2000b0e10 0x7fe2000b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60892/0 (socket says 172.21.15.91:60892) 2024-02-02T13:38:27.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 -- 172.21.15.91:0/478569560 learned_addr learned my addr 172.21.15.91:0/478569560 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:27.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 -- 172.21.15.91:0/478569560 >> v2:172.21.15.113:3300/0 conn(0x7fe200151f90 msgr2=0x7fe20015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:27.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 --2- 172.21.15.91:0/478569560 >> v2:172.21.15.113:3300/0 conn(0x7fe200151f90 0x7fe20015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 -- 172.21.15.91:0/478569560 >> v1:172.21.15.113:6789/0 conn(0x7fe20006f4f0 legacy=0x7fe2000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:27.940 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.939+0000 7fe204e00640 1 -- 172.21.15.91:0/478569560 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000cb2f0 con 0x7fe2000b0e10 2024-02-02T13:38:27.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.940+0000 7fe204e00640 1 --2- 172.21.15.91:0/478569560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2000b0e10 0x7fe2000b3af0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fe2000cc770 tx=0x7fe1f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e9cf687ab8c5d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:27.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.941+0000 7fe1fea00640 1 -- 172.21.15.91:0/478569560 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f802ca80 con 0x7fe2000b0e10 2024-02-02T13:38:27.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.941+0000 7fe1fea00640 1 -- 172.21.15.91:0/478569560 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f802cbe0 con 0x7fe2000b0e10 2024-02-02T13:38:27.941 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.941+0000 7fe1fea00640 1 -- 172.21.15.91:0/478569560 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80343f0 con 0x7fe2000b0e10 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.941+0000 7fe206e00640 1 -- 172.21.15.91:0/478569560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2000b0e10 msgr2=0x7fe2000b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.941+0000 7fe206e00640 1 --2- 172.21.15.91:0/478569560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2000b0e10 0x7fe2000b3af0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fe2000cc770 tx=0x7fe1f802bee0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 -- 172.21.15.91:0/478569560 shutdown_connections 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 --2- 172.21.15.91:0/478569560 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2000b0e10 0x7fe2000b3af0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 --2- 172.21.15.91:0/478569560 >> v2:172.21.15.113:3300/0 conn(0x7fe200151f90 0x7fe20015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 -- 172.21.15.91:0/478569560 >> 172.21.15.91:0/478569560 conn(0x7fe2000cec80 msgr2=0x7fe2000cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 -- 172.21.15.91:0/478569560 shutdown_connections 2024-02-02T13:38:27.942 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.942+0000 7fe206e00640 1 -- 172.21.15.91:0/478569560 wait complete. 2024-02-02T13:38:27.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.943+0000 7fe206e00640 1 Processor -- start 2024-02-02T13:38:27.944 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.944+0000 7fe206e00640 1 -- start start 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 0x7fe2000cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe200151f90 0x7fe2000cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c7c50 con 0x7fe20006f4f0 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c7e60 con 0x7fe200151f90 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe206e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c8140 con 0x7fe2000b0e10 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60900/0 (socket says 172.21.15.91:60900) 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 -- 172.21.15.91:0/1714784324 learned_addr learned my addr 172.21.15.91:0/1714784324 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ff400640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 0x7fe2000cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:27.946 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe204e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe200151f90 0x7fe2000cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 -- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 msgr2=0x7fe2000cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 0x7fe2000cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 -- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe200151f90 msgr2=0x7fe2000cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe200151f90 0x7fe2000cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ffe00640 1 -- 172.21.15.91:0/1714784324 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8009b30 con 0x7fe20006f4f0 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.945+0000 7fe1ff400640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 0x7fe2000cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:27.947 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe1ffe00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b4a0 tx=0x7fe1f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010610 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0010770 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010b20 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000c8420 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.946+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000c5ef0 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.947+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200066cf0 con 0x7fe20006f4f0 2024-02-02T13:38:27.948 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.948+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe1f0010c80 con 0x7fe20006f4f0 2024-02-02T13:38:27.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.949+0000 7fe1f7e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 0x7fe1d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:27.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.950+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe1f0088220 con 0x7fe20006f4f0 2024-02-02T13:38:27.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.950+0000 7fe1ff400640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 0x7fe1d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:27.950 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.951+0000 7fe1ff400640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 0x7fe1d806c8f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe2001f6150 tx=0x7fe1e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:27.955 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:27.954+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe1f0053b00 con 0x7fe20006f4f0 2024-02-02T13:38:28.144 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.143+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 --> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2000714e0 con 0x7fe1d806a430 2024-02-02T13:38:28.150 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.149+0000 7fe1f7e00640 1 -- 172.21.15.91:0/1714784324 <== mgr.4110 v2:172.21.15.91:6832/32622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47383 (secure 0 0 0) 0x7fe1e800ad60 con 0x7fe1d806a430 2024-02-02T13:38:28.151 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:38:28.151 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2024-02-02T13:38:28.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.151+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 msgr2=0x7fe1d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.151+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 0x7fe1d806c8f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe2001f6150 tx=0x7fe1e8008040 comp rx=0 tx=0).stop 2024-02-02T13:38:28.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 msgr2=0x7fe200150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b4a0 tx=0x7fe1f000b960 comp rx=0 tx=0).stop 2024-02-02T13:38:28.153 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 shutdown_connections 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fe1d806a430 0x7fe1d806c8f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2000b0e10 0x7fe2000cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe200151f90 0x7fe2000cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 --2- 172.21.15.91:0/1714784324 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe20006f4f0 0x7fe200150bd0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 >> 172.21.15.91:0/1714784324 conn(0x7fe2000cec80 msgr2=0x7fe2000b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 shutdown_connections 2024-02-02T13:38:28.154 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.152+0000 7fe206e00640 1 -- 172.21.15.91:0/1714784324 wait complete. 2024-02-02T13:38:28.173 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2024-02-02T13:38:26.518103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10060,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552439,"statfs":{"total":1600000000,"available":1589698196,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003376"},"pg_stats":[{"pgid":"1.0","version":"202'119","reported_seq":358,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-02-02T13:38:07.649938+0000","last_change":"2024-02-02T13:38:07.650092+0000","last_active":"2024-02-02T13:38:07.649938+0000","last_peered":"2024-02-02T13:38:07.649938+0000","last_clean":"2024-02-02T13:38:07.649938+0000","last_became_active":"2024-02-02T13:24:27.821377+0000","last_became_peered":"2024-02-02T13:24:27.821377+0000","last_unstale":"2024-02-02T13:38:07.649938+0000","last_undegraded":"2024-02-02T13:38:07.649938+0000","last_fullsized":"2024-02-02T13:38:07.649938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'119","last_scrub_stamp":"2024-02-02T13:38:07.649887+0000","last_deep_scrub":"202'119","last_deep_scrub_stamp":"2024-02-02T13:38:07.649887+0000","last_clean_scrub_stamp":"2024-02-02T13:38:07.649887+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:56:11.268640+0000","scrub_duration":0.0056894440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1690,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193622,"statfs":{"total":200000000,"available":198268989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.1080000000000001,"15min":1.087},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6459999999999999,"5min":2.266,"15min":2.5310000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":1.117},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":4.1609999999999996},"last":0.87}]},{"osd":1,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.105,"15min":1.1000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.179,"15min":1.161},"min":{"1min":0.628,"5min":0.628,"15min":0.56299999999999994},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":1.1339999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.2130000000000001,"15min":1.228},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.621,"5min":1.8580000000000001,"15min":2.6669999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.252,"15min":1.204},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.5819999999999999},"last":0.96899999999999997}]},{"osd":3,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.216,"15min":1.1839999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.177},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3160000000000001,"15min":1.321},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.48999999999999999},"max":{"1min":1.883,"5min":1.9650000000000001,"15min":2.4620000000000002},"last":1.492}]},{"osd":4,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.2929999999999999,"15min":1.26},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":4.242,"5min":4.242,"15min":4.242},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.61199999999999999},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.264}]},{"osd":5,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.9550000000000001,"5min":2.0259999999999998,"15min":2.2200000000000002},"last":1.421},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.56200000000000006},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":1.506}]},{"osd":6,"last update":"Fri Feb 2 13:38:01 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5189999999999999,"15min":1.5149999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"last":1.657},{"interface":"front","average":{"1min":1.452,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.63700000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.4379999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.121,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.64000000000000001,"15min":0.498},"max":{"1min":1.5469999999999999,"5min":1.603,"15min":2.3780000000000001},"last":1.113},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.52400000000000002},"max":{"1min":1.3660000000000001,"5min":1.776,"15min":1.776},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.0660000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.54000000000000004},"max":{"1min":1.478,"5min":1.4790000000000001,"15min":1.556},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.55900000000000005},"max":{"1min":1.51,"5min":1.677,"15min":2.9300000000000002},"last":1.0289999999999999}]},{"osd":2,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2390000000000001,"15min":1.169},"min":{"1min":0.85999999999999999,"5min":0.84799999999999998,"15min":0.63700000000000001},"max":{"1min":1.556,"5min":1.75,"15min":2.6309999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.131,"5min":1.212,"15min":1.1799999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.63},"max":{"1min":1.4890000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.236}]},{"osd":3,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.80500000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.7749999999999999,"15min":1.8819999999999999},"last":1.397},{"interface":"front","average":{"1min":1.242,"5min":1.2689999999999999,"15min":1.1970000000000001},"min":{"1min":0.96899999999999997,"5min":0.90600000000000003,"15min":0.61299999999999999},"max":{"1min":1.6399999999999999,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.94499999999999995,"5min":0.67900000000000005,"15min":0.55100000000000005},"max":{"1min":1.5469999999999999,"5min":1.8420000000000001,"15min":17.686},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4350000000000001,"15min":1.4710000000000001},"min":{"1min":1.022,"5min":0.72099999999999997,"15min":0.505},"max":{"1min":1.78,"5min":2.0529999999999999,"15min":17.388999999999999},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.2849999999999999},"min":{"1min":1.0269999999999999,"5min":0.82399999999999995,"15min":0.45900000000000002},"max":{"1min":1.595,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4390000000000001,"15min":1.4079999999999999},"min":{"1min":1.1040000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8400000000000001,"5min":1.95,"15min":3.331},"last":1.2050000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.464,"15min":1.4610000000000001},"min":{"1min":1.0640000000000001,"5min":0.80300000000000005,"15min":0.61199999999999999},"max":{"1min":1.871,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4590000000000001,"15min":1.4430000000000001},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.65300000000000002},"max":{"1min":1.9970000000000001,"5min":2.1160000000000001,"15min":2.1419999999999999},"last":1.153}]}]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.093,"15min":1.046},"min":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.8,"15min":1.8},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.056,"5min":1.0860000000000001,"15min":1.0620000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.49199999999999999},"max":{"1min":1.4930000000000001,"5min":1.5629999999999999,"15min":1.7150000000000001},"last":1.2450000000000001}]},{"osd":1,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1839999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.841,"15min":4.5899999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4500000000000002},"last":1.1850000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.274},"min":{"1min":0.95699999999999996,"5min":0.84599999999999997,"15min":0.72499999999999998},"max":{"1min":1.6419999999999999,"5min":1.8120000000000001,"15min":3.0339999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2430000000000001,"15min":1.2010000000000001},"min":{"1min":0.94899999999999995,"5min":0.84099999999999997,"15min":0.65300000000000002},"max":{"1min":1.498,"5min":1.7210000000000001,"15min":3.0720000000000001},"last":1.22}]},{"osd":3,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.282,"15min":1.2190000000000001},"min":{"1min":0.93100000000000005,"5min":0.77000000000000002,"15min":0.70399999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.7769999999999999},"last":1.327},{"interface":"front","average":{"1min":1.375,"5min":1.2909999999999999,"15min":1.248},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.61599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":5.4610000000000003},"last":1.474}]},{"osd":4,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1950000000000001,"15min":1.2030000000000001},"min":{"1min":0.77700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.8460000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.3,"15min":1.29},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.55100000000000005},"max":{"1min":1.8440000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.375}]},{"osd":6,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.329},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.64700000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.083},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":0.98099999999999998,"5min":0.70199999999999996,"15min":0.59099999999999997},"max":{"1min":1.8109999999999999,"5min":1.845,"15min":1.8640000000000001},"last":1.409}]},{"osd":7,"last update":"Fri Feb 2 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.413,"15min":1.4119999999999999},"min":{"1min":0.83099999999999996,"5min":0.73899999999999999,"15min":0.66500000000000004},"max":{"1min":1.7190000000000001,"5min":1.893,"15min":1.9670000000000001},"last":1.444},{"interface":"front","average":{"1min":1.486,"5min":1.468,"15min":1.4379999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.61599999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.73}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0609999999999999,"15min":1.0429999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.395,"5min":1.4570000000000001,"15min":4.5949999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.077},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"max":{"1min":1.474,"5min":1.6579999999999999,"15min":1.839},"last":0.92500000000000004}]},{"osd":1,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.095,"15min":1.056},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.39000000000000001},"max":{"1min":1.3540000000000001,"5min":1.54,"15min":2.4809999999999999},"last":1.161},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.123,"15min":1.0960000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.5029999999999999,"5min":1.9490000000000001,"15min":4.5060000000000002},"last":1.1970000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.1379999999999999},"min":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.68500000000000005},"max":{"1min":1.405,"5min":1.556,"15min":3.1200000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1919999999999999,"15min":1.1559999999999999},"min":{"1min":0.86799999999999999,"5min":0.71199999999999997,"15min":0.58799999999999997},"max":{"1min":1.4399999999999999,"5min":1.736,"15min":2.6000000000000001},"last":1.004}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.278,"15min":1.2450000000000001},"min":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.68999999999999995},"max":{"1min":1.5309999999999999,"5min":1.637,"15min":2.9079999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5980000000000001,"5min":1.853,"15min":3.9540000000000002},"last":1.363}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1850000000000001,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6120000000000001,"5min":2.0830000000000002,"15min":4.3940000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.29,"15min":1.3120000000000001},"min":{"1min":0.91100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.4529999999999998},"last":1.089}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3999999999999999,"15min":1.371},"min":{"1min":0.99199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.617,"5min":2.403,"15min":2.7989999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.423,"15min":1.429},"min":{"1min":1.0189999999999999,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":2.6549999999999998,"15min":4.2809999999999997},"last":1.403}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.45,"15min":1.4410000000000001},"min":{"1min":1.008,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.825,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.429},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.466,"15min":1.4690000000000001},"min":{"1min":1.133,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.984,"5min":2.0409999999999999,"15min":2.698},"last":1.4610000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.76000000000000001},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.35099999999999998},"max":{"1min":1.2190000000000001,"5min":1.379,"15min":1.4199999999999999},"last":1.01},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.93000000000000005,"15min":0.90500000000000003},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.36099999999999999},"max":{"1min":1.286,"5min":1.5169999999999999,"15min":2.6040000000000001},"last":0.754}]},{"osd":1,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.94599999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.39700000000000002},"max":{"1min":1.925,"5min":1.925,"15min":2.403},"last":1.109},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.081},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.47999999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.2709999999999999},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1359999999999999,"15min":1.04},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.39800000000000002},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.95599999999999996,"15min":0.91700000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.41499999999999998},"max":{"1min":1.3380000000000001,"5min":1.45,"15min":3.4510000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.204,"15min":1.1899999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.53000000000000003},"max":{"1min":1.573,"5min":1.5940000000000001,"15min":3.585},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3129999999999999,"15min":1.258},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.61299999999999999},"max":{"1min":1.764,"5min":1.8149999999999999,"15min":1.877},"last":1.181}]},{"osd":5,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.234,"15min":1.204},"min":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.57099999999999995},"max":{"1min":1.4279999999999999,"5min":1.609,"15min":3.738},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2749999999999999,"15min":1.258},"min":{"1min":0.89200000000000002,"5min":0.83399999999999996,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.7549999999999999,"15min":3.4950000000000001},"last":1.3799999999999999}]},{"osd":6,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3460000000000001},"min":{"1min":1.0409999999999999,"5min":0.84699999999999998,"15min":0.58299999999999996},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.2709999999999999,"15min":1.2689999999999999},"min":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.63300000000000001},"max":{"1min":1.5269999999999999,"5min":1.667,"15min":3.6059999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Feb 2 13:37:58 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3380000000000001,"15min":1.2869999999999999},"min":{"1min":1.032,"5min":0.80600000000000005,"15min":0.65900000000000003},"max":{"1min":1.845,"5min":1.851,"15min":3.391},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4239999999999999,"15min":1.365},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.625},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.0409999999999999},"last":1.264}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1113,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194198,"statfs":{"total":200000000,"available":198859357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.27900000000000003},"max":{"1min":1.238,"5min":1.345,"15min":7.1840000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0169999999999999,"15min":0.90600000000000003},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.30299999999999999},"max":{"1min":1.411,"5min":1.5720000000000001,"15min":2.5190000000000001},"last":1.179}]},{"osd":1,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.93300000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2270000000000001,"5min":1.387,"15min":2.1680000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.071,"15min":0.98199999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.35899999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.6499999999999999},"last":1.042}]},{"osd":3,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98999999999999999,"15min":0.90800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":2.1339999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.109,"15min":1.0509999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.432},"max":{"1min":1.488,"5min":1.653,"15min":2.6589999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2969999999999999,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.56799999999999995},"max":{"1min":1.5369999999999999,"5min":1.877,"15min":2.8759999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2330000000000001},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.503},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":2.1360000000000001},"last":1.615}]},{"osd":5,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.246,"15min":1.1910000000000001},"min":{"1min":1.1100000000000001,"5min":0.80300000000000005,"15min":0.56100000000000005},"max":{"1min":1.504,"5min":2.028,"15min":2.8239999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.337,"15min":1.3069999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.627},"max":{"1min":1.696,"5min":2.21,"15min":3.0070000000000001},"last":1.2270000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.371,"15min":1.2929999999999999},"min":{"1min":1.01,"5min":0.98999999999999999,"15min":0.53800000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.8620000000000001},"last":1.395},{"interface":"front","average":{"1min":1.357,"5min":1.329,"15min":1.2789999999999999},"min":{"1min":1.054,"5min":0.93999999999999995,"15min":0.70499999999999996},"max":{"1min":1.79,"5min":1.7949999999999999,"15min":2.9750000000000001},"last":1.3580000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:43 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4219999999999999,"15min":1.341},"min":{"1min":1.242,"5min":1.1220000000000001,"15min":0.65100000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.7749999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3680000000000001,"15min":1.3129999999999999},"min":{"1min":1.028,"5min":0.96699999999999997,"15min":0.52500000000000002},"max":{"1min":1.7629999999999999,"5min":1.9330000000000001,"15min":3.0830000000000002},"last":1.2589999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1112,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194199,"statfs":{"total":200000000,"available":198860376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84399999999999997,"15min":0.87},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":1.46,"5min":1.6759999999999999,"15min":3.5680000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.96299999999999997,"15min":1.046},"min":{"1min":0.54300000000000004,"5min":0.43099999999999999,"15min":0.308},"max":{"1min":1.387,"5min":1.6699999999999999,"15min":5.8099999999999996},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.53000000000000003,"5min":0.45200000000000001,"15min":0.34200000000000003},"max":{"1min":1.321,"5min":1.615,"15min":1.639},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":3.0259999999999998},"last":1.0780000000000001}]},{"osd":3,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.704,"15min":3.1539999999999999},"last":1.3},{"interface":"front","average":{"1min":1.077,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.778,"15min":4.8760000000000003},"last":1.387}]},{"osd":4,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.163,"15min":1.1859999999999999},"min":{"1min":0.83099999999999996,"5min":0.65400000000000003,"15min":0.625},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":2.9449999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.95299999999999996,"5min":0.82699999999999996,"15min":0.68899999999999995},"max":{"1min":1.7430000000000001,"5min":1.748,"15min":3.2970000000000002},"last":1.4650000000000001}]},{"osd":5,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3400000000000001,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.753},"max":{"1min":1.8420000000000001,"5min":1.8779999999999999,"15min":4.1849999999999996},"last":1.105},{"interface":"front","average":{"1min":1.304,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.022,"5min":0.78800000000000003,"15min":0.747},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":3.0249999999999999},"last":1.343}]},{"osd":6,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.069,"5min":0.998,"15min":0.73599999999999999},"max":{"1min":1.879,"5min":1.9870000000000001,"15min":4.3819999999999997},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3420000000000001,"15min":1.3660000000000001},"min":{"1min":1.036,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.9910000000000001},"last":1.431}]},{"osd":7,"last update":"Fri Feb 2 13:37:48 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.77100000000000002},"max":{"1min":1.804,"5min":2.1600000000000001,"15min":3.4689999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4710000000000001,"15min":1.4650000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.78000000000000003},"max":{"1min":2.0169999999999999,"5min":2.0699999999999998,"15min":4.3040000000000003},"last":1.4950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1689,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193623,"statfs":{"total":200000000,"available":198270008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86699999999999999,"15min":0.90500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.27300000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":4.8070000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89800000000000002,"15min":1.0580000000000001},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.29199999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":10.816000000000001},"last":1.3680000000000001}]},{"osd":2,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.0009999999999999},"min":{"1min":0.53300000000000003,"5min":0.373,"15min":0.32700000000000001},"max":{"1min":1.325,"5min":1.5009999999999999,"15min":4.8490000000000002},"last":1.272},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.97399999999999998,"15min":1.0960000000000001},"min":{"1min":0.49199999999999999,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.6830000000000001,"15min":10.781000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95599999999999996,"15min":1.0069999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.379},"max":{"1min":1.3440000000000001,"5min":1.607,"15min":5.8410000000000002},"last":1.044},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0549999999999999,"15min":1.1140000000000001},"min":{"1min":0.54400000000000004,"5min":0.46600000000000003,"15min":0.36699999999999999},"max":{"1min":1.653,"5min":1.653,"15min":4.9530000000000003},"last":1.153}]},{"osd":4,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.93899999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.607,"5min":2.129,"15min":3.669},"last":1.212},{"interface":"front","average":{"1min":1.175,"5min":1.26,"15min":1.292},"min":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73199999999999998},"max":{"1min":1.8580000000000001,"5min":1.893,"15min":4.8780000000000001},"last":1.4179999999999999}]},{"osd":5,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.26,"15min":1.415},"min":{"1min":0.77800000000000002,"5min":0.752,"15min":0.72599999999999998},"max":{"1min":2.278,"5min":2.278,"15min":10.651},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.2909999999999999},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.64400000000000002},"max":{"1min":1.526,"5min":1.954,"15min":3.7869999999999999},"last":1.4410000000000001}]},{"osd":6,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.341,"15min":1.359},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.66500000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.9210000000000003},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.296,"15min":1.444},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8,"5min":1.95,"15min":10.708},"last":1.2350000000000001}]},{"osd":7,"last update":"Fri Feb 2 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.347,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":10.74},"last":1.573},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.98299999999999998,"5min":0.78500000000000003,"15min":0.66800000000000004},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":3.863},"last":1.306}]}]}],"pool_statfs":[]}} 2024-02-02T13:38:28.174 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-02-02T13:38:28.377 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.376+0000 7fcaeea00640 1 Processor -- start 2024-02-02T13:38:28.378 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.376+0000 7fcaeea00640 1 -- start start 2024-02-02T13:38:28.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.377+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 0x7fcae80b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.377+0000 7fcaeea00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcae80b0bd0 0x7fcae80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80cca70 con 0x7fcae815a1e0 2024-02-02T13:38:28.379 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeea00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae80b1060 con 0x7fcae80b0bd0 2024-02-02T13:38:28.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80cc770 con 0x7fcae80c9f90 2024-02-02T13:38:28.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcae80b0bd0 0x7fcae80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcae80b0bd0 0x7fcae80ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60904/0 (socket says 172.21.15.91:60904) 2024-02-02T13:38:28.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 -- 172.21.15.91:0/3257023184 learned_addr learned my addr 172.21.15.91:0/3257023184 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:28.380 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcae7400640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 0x7fcae80b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 -- 172.21.15.91:0/3257023184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 msgr2=0x7fcae80b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 0x7fcae80b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.378+0000 7fcaeca00640 1 -- 172.21.15.91:0/3257023184 >> v1:172.21.15.113:6789/0 conn(0x7fcae815a1e0 legacy=0x7fcae8070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.379+0000 7fcaeca00640 1 -- 172.21.15.91:0/3257023184 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80ca570 con 0x7fcae80b0bd0 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.379+0000 7fcae7400640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 0x7fcae80b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcaeca00640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80b0bd0 0x7fcae80ccf00 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009870 tx=0x7fcae002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949cdb193977c5db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcae6a00640 1 -- 172.21.15.91:0/3257023184 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0030840 con 0x7fcae80b0bd0 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcae6a00640 1 -- 172.21.15.91:0/3257023184 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae00309a0 con 0x7fcae80b0bd0 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcae6a00640 1 -- 172.21.15.91:0/3257023184 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0030d30 con 0x7fcae80b0bd0 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcaeea00640 1 -- 172.21.15.91:0/3257023184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80b0bd0 msgr2=0x7fcae80ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.380+0000 7fcaeea00640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80b0bd0 0x7fcae80ccf00 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009870 tx=0x7fcae002b980 comp rx=0 tx=0).stop 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 -- 172.21.15.91:0/3257023184 shutdown_connections 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80b0bd0 0x7fcae80ccf00 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 --2- 172.21.15.91:0/3257023184 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80c9f90 0x7fcae80b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 -- 172.21.15.91:0/3257023184 >> 172.21.15.91:0/3257023184 conn(0x7fcae80ceb90 msgr2=0x7fcae80cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:28.382 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 -- 172.21.15.91:0/3257023184 shutdown_connections 2024-02-02T13:38:28.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.381+0000 7fcaeea00640 1 -- 172.21.15.91:0/3257023184 wait complete. 2024-02-02T13:38:28.383 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.382+0000 7fcaeea00640 1 Processor -- start 2024-02-02T13:38:28.384 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.382+0000 7fcaeea00640 1 -- start start 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80b0bd0 0x7fcae80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 0x7fcae81e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8158840 con 0x7fcae80c9f90 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8156350 con 0x7fcae815a1e0 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.383+0000 7fcaeea00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8158250 con 0x7fcae80b0bd0 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60920/0 (socket says 172.21.15.91:60920) 2024-02-02T13:38:28.385 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 -- 172.21.15.91:0/2262966162 learned_addr learned my addr 172.21.15.91:0/2262966162 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcaeca00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 0x7fcae81e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 -- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80b0bd0 msgr2=0x7fcae80c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7e00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80b0bd0 0x7fcae80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80b0bd0 0x7fcae80c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 -- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 msgr2=0x7fcae81e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 0x7fcae81e6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcae7400640 1 -- 172.21.15.91:0/2262966162 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00094d0 con 0x7fcae80c9f90 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.384+0000 7fcaeca00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 0x7fcae81e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcae7400640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fcad800cf60 tx=0x7fcad800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800d840 con 0x7fcae80c9f90 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800d9a0 con 0x7fcae80c9f90 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800dd30 con 0x7fcae80c9f90 2024-02-02T13:38:28.387 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae8155d90 con 0x7fcae80c9f90 2024-02-02T13:38:28.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.385+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae8158460 con 0x7fcae80c9f90 2024-02-02T13:38:28.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.386+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae8158980 con 0x7fcae80c9f90 2024-02-02T13:38:28.388 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.387+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcad8015440 con 0x7fcae80c9f90 2024-02-02T13:38:28.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.388+0000 7fcadfe00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 0x7fcac006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.388+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcad8088ff0 con 0x7fcae80c9f90 2024-02-02T13:38:28.389 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.388+0000 7fcae7e00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 0x7fcac006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.390 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.389+0000 7fcae7e00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 0x7fcac006c920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.394 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.393+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcad80bbca0 con 0x7fcae80c9f90 2024-02-02T13:38:28.586 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.585+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fcae80b3ab0 con 0x7fcae80c9f90 2024-02-02T13:38:28.592 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.591+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcad800de90 con 0x7fcae80c9f90 2024-02-02T13:38:28.593 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.591+0000 7fcadfe00640 1 -- 172.21.15.91:0/2262966162 <== mon.0 v2:172.21.15.91:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcad801a020 con 0x7fcae80c9f90 2024-02-02T13:38:28.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.594+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 msgr2=0x7fcac006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.594+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 0x7fcac006c920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).stop 2024-02-02T13:38:28.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.594+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 msgr2=0x7fcae81e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.594+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fcad800cf60 tx=0x7fcad800cf90 comp rx=0 tx=0).stop 2024-02-02T13:38:28.596 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 shutdown_connections 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fcac006a460 0x7fcac006c920 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcae815a1e0 0x7fcae81e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcae80c9f90 0x7fcae81e6a30 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 --2- 172.21.15.91:0/2262966162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcae80b0bd0 0x7fcae80c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 >> 172.21.15.91:0/2262966162 conn(0x7fcae80ceb90 msgr2=0x7fcae815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 shutdown_connections 2024-02-02T13:38:28.597 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.595+0000 7fcaeea00640 1 -- 172.21.15.91:0/2262966162 wait complete. 2024-02-02T13:38:28.616 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T13:38:28.617 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T13:38:28.821 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.820+0000 7fac9fe00640 1 Processor -- start 2024-02-02T13:38:28.822 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.820+0000 7fac9fe00640 1 -- start start 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9fe00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac9815f5a0 0x7fac980b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9fe00640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980b1070 con 0x7fac980b0c40 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9806c600 con 0x7fac98151f90 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9fe00640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9815fa30 con 0x7fac9815f5a0 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9ca00640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac9815f5a0 0x7fac980b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9de00640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fac980b0c40 0x7fac980b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:55854/0 (socket says 172.21.15.91:55854) 2024-02-02T13:38:28.823 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9de00640 1 -- 172.21.15.91:0/30024536 learned_addr learned my addr 172.21.15.91:0/30024536 (peer_addr_for_me v1:172.21.15.91:0/0) 2024-02-02T13:38:28.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9ca00640 1 -- 172.21.15.91:0/30024536 >> v1:172.21.15.113:6789/0 conn(0x7fac98151f90 legacy=0x7fac9806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T13:38:28.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9ca00640 1 -- 172.21.15.91:0/30024536 >> v1:172.21.15.91:6789/0 conn(0x7fac980b0c40 legacy=0x7fac980b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.822+0000 7fac9ca00640 1 -- 172.21.15.91:0/30024536 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98070f80 con 0x7fac9815f5a0 2024-02-02T13:38:28.824 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.823+0000 7fac9ca00640 1 --2- 172.21.15.91:0/30024536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980b0700 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fac800097a0 tx=0x7fac8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23effb4c30f73021 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac97e00640 1 -- 172.21.15.91:0/30024536 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80031830 con 0x7fac9815f5a0 2024-02-02T13:38:28.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac97e00640 1 -- 172.21.15.91:0/30024536 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fac80031990 con 0x7fac9815f5a0 2024-02-02T13:38:28.825 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac97e00640 1 -- 172.21.15.91:0/30024536 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80031d40 con 0x7fac9815f5a0 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac9fe00640 1 -- 172.21.15.91:0/30024536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 msgr2=0x7fac980b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac9fe00640 1 --2- 172.21.15.91:0/30024536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980b0700 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fac800097a0 tx=0x7fac8002b8d0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac9fe00640 1 -- 172.21.15.91:0/30024536 shutdown_connections 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac9fe00640 1 --2- 172.21.15.91:0/30024536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980b0700 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.824+0000 7fac9fe00640 1 -- 172.21.15.91:0/30024536 >> 172.21.15.91:0/30024536 conn(0x7fac980cec80 msgr2=0x7fac980cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.825+0000 7fac9fe00640 1 -- 172.21.15.91:0/30024536 shutdown_connections 2024-02-02T13:38:28.826 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.825+0000 7fac9fe00640 1 -- 172.21.15.91:0/30024536 wait complete. 2024-02-02T13:38:28.827 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.826+0000 7fac9fe00640 1 Processor -- start 2024-02-02T13:38:28.828 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.826+0000 7fac9fe00640 1 -- start start 2024-02-02T13:38:28.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac980b0c40 0x7fac98150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac98151f90 0x7fac980ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.829 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ae040 con 0x7fac9815f5a0 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ad3c0 con 0x7fac98151f90 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.827+0000 7fac9fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ad6a0 con 0x7fac980b0c40 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:60948/0 (socket says 172.21.15.91:60948) 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 -- 172.21.15.91:0/3622430684 learned_addr learned my addr 172.21.15.91:0/3622430684 (peer_addr_for_me v2:172.21.15.91:0/0) 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 -- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac980b0c40 msgr2=0x7fac98150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac980b0c40 0x7fac98150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 -- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac98151f90 msgr2=0x7fac980ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac98151f90 0x7fac980ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 -- 172.21.15.91:0/3622430684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80009400 con 0x7fac9815f5a0 2024-02-02T13:38:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac9de00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fac9000caa0 tx=0x7fac9000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000ab40 con 0x7fac9815f5a0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.828+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fac9000aca0 con 0x7fac9815f5a0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.829+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90012620 con 0x7fac9815f5a0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.829+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980ad980 con 0x7fac9815f5a0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.829+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980cda50 con 0x7fac9815f5a0 2024-02-02T13:38:28.831 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.830+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac980ae360 con 0x7fac9815f5a0 2024-02-02T13:38:28.832 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.831+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fac90007070 con 0x7fac9815f5a0 2024-02-02T13:38:28.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.831+0000 7fac95600640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 0x7fac7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T13:38:28.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.832+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac900885d0 con 0x7fac9815f5a0 2024-02-02T13:38:28.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.832+0000 7fac9d400640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 0x7fac7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T13:38:28.833 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.832+0000 7fac9d400640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 0x7fac7006c8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac88003cc0 tx=0x7fac880079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T13:38:28.837 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:28.836+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fac980ae360 con 0x7fac9815f5a0 2024-02-02T13:38:29.066 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.065+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac980ae360 con 0x7fac9815f5a0 2024-02-02T13:38:29.067 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.066+0000 7fac95600640 1 -- 172.21.15.91:0/3622430684 <== mon.0 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fac980ae360 con 0x7fac9815f5a0 2024-02-02T13:38:29.067 INFO:teuthology.orchestra.run.smithi091.stdout: 2024-02-02T13:38:29.068 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":90,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/934213102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":934213102},{"type":"v1","addr":"172.21.15.91:6837","nonce":934213102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/3269459751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3269459751},{"type":"v1","addr":"172.21.15.113:6834","nonce":3269459751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1038697829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":1038697829},{"type":"v1","addr":"172.21.15.113:6835","nonce":1038697829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/1969835024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1969835024},{"type":"v1","addr":"172.21.15.91:6835","nonce":1969835024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[]} 2024-02-02T13:38:29.068 INFO:teuthology.orchestra.run.smithi091.stderr:dumped fsmap epoch 90 2024-02-02T13:38:29.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.068+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 msgr2=0x7fac7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:29.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.068+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 0x7fac7006c8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac88003cc0 tx=0x7fac880079b0 comp rx=0 tx=0).stop 2024-02-02T13:38:29.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.068+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 msgr2=0x7fac980c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T13:38:29.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.068+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fac9000caa0 tx=0x7fac9000cf60 comp rx=0 tx=0).stop 2024-02-02T13:38:29.070 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.068+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 shutdown_connections 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:6832/32622,v1:172.21.15.91:6833/32622] conn(0x7fac7006a430 0x7fac7006c8f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9815f5a0 0x7fac980c7810 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac98151f90 0x7fac980ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 --2- 172.21.15.91:0/3622430684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac980b0c40 0x7fac98150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 >> 172.21.15.91:0/3622430684 conn(0x7fac980cec80 msgr2=0x7fac9806fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 shutdown_connections 2024-02-02T13:38:29.071 INFO:teuthology.orchestra.run.smithi091.stderr:2024-02-02T13:38:29.069+0000 7fac9fe00640 1 -- 172.21.15.91:0/3622430684 wait complete. 2024-02-02T13:38:29.088 DEBUG:tasks.cephfs.filesystem:already dead... 2024-02-02T13:38:29.088 INFO:teuthology.misc:Shutting down mds daemons... 2024-02-02T13:38:29.088 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-02T13:38:29.088 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:29.104 INFO:tasks.ceph.mds.a:Stopped 2024-02-02T13:38:29.104 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-02T13:38:29.104 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:29.170 INFO:tasks.ceph.mds.c:Stopped 2024-02-02T13:38:29.170 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-02T13:38:29.170 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:29.203 INFO:tasks.ceph.mds.b:Stopped 2024-02-02T13:38:29.204 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-02T13:38:29.204 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:29.274 INFO:tasks.ceph.mds.d:Stopped 2024-02-02T13:38:29.274 INFO:teuthology.misc:Shutting down osd daemons... 2024-02-02T13:38:29.274 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-02-02T13:38:29.275 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:33.194 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:33.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:34.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-02-02T13:38:35.376 INFO:tasks.ceph.osd.0:Stopped 2024-02-02T13:38:35.376 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-02-02T13:38:35.376 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:35.467 INFO:tasks.ceph.osd.1:Stopped 2024-02-02T13:38:35.468 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-02-02T13:38:35.468 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:35.563 INFO:tasks.ceph.osd.2:Stopped 2024-02-02T13:38:35.563 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-02-02T13:38:35.563 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:35.643 INFO:tasks.ceph.osd.3:Stopped 2024-02-02T13:38:35.643 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-02-02T13:38:35.643 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:35.734 INFO:tasks.ceph.osd.4:Stopped 2024-02-02T13:38:35.734 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-02-02T13:38:35.734 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:35.811 INFO:tasks.ceph.osd.5:Stopped 2024-02-02T13:38:35.811 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-02-02T13:38:35.811 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:39.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-02-02T13:38:39.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-02-02T13:38:41.912 INFO:tasks.ceph.osd.6:Stopped 2024-02-02T13:38:41.913 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-02-02T13:38:41.913 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:41.983 INFO:tasks.ceph.osd.7:Stopped 2024-02-02T13:38:41.983 INFO:teuthology.misc:Shutting down mgr daemons... 2024-02-02T13:38:41.984 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-02-02T13:38:41.984 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:42.065 INFO:tasks.ceph.mgr.y:Stopped 2024-02-02T13:38:42.065 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-02-02T13:38:42.066 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:42.158 INFO:tasks.ceph.mgr.x:Stopped 2024-02-02T13:38:42.158 INFO:teuthology.misc:Shutting down mon daemons... 2024-02-02T13:38:42.158 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-02-02T13:38:42.158 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:42.203 INFO:tasks.ceph.mon.a:Stopped 2024-02-02T13:38:42.203 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-02-02T13:38:42.203 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:42.243 INFO:tasks.ceph.mon.b:Stopped 2024-02-02T13:38:42.243 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-02-02T13:38:42.243 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T13:38:42.319 INFO:tasks.ceph.mon.c:Stopped 2024-02-02T13:38:42.319 INFO:tasks.ceph:Checking cluster log for badness... 2024-02-02T13:38:42.319 DEBUG:teuthology.orchestra.run.smithi091:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-02-02T13:38:42.425 INFO:teuthology.orchestra.run.smithi091.stdout:2024-02-02T13:24:24.763038+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 125433.498170 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-02-02T13:38:42.426 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-02-02T13:38:42.426 DEBUG:teuthology.orchestra.run.smithi091:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-02-02T13:38:42.521 DEBUG:teuthology.orchestra.run.smithi091:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-02-02T13:38:42.607 DEBUG:teuthology.orchestra.run.smithi091:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-02-02T13:38:42.695 INFO:teuthology.orchestra.run.smithi091.stdout:2024-02-02T13:24:24.763038+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 125433.498170 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-02-02T13:38:42.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:38:42.696 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-02-02T13:38:42.932 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:38:42.932 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-02-02T13:38:43.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:38:43.063 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-02-02T13:38:43.225 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:38:43.226 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-02-02T13:38:43.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:38:43.366 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-02-02T13:38:43.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:38:43.665 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-02-02T13:38:43.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:38:43.826 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-02-02T13:38:43.961 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:38:43.961 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-02-02T13:38:44.115 INFO:tasks.ceph:Archiving mon data... 2024-02-02T13:38:44.116 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/data/mon.a.tgz 2024-02-02T13:38:44.116 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2024-02-02T13:38:44.175 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.G4RmFmekxh 2024-02-02T13:38:44.175 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.G4RmFmekxh 2024-02-02T13:38:44.396 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.G4RmFmekxh 2024-02-02T13:38:44.544 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.G4RmFmekxh is 474KB 2024-02-02T13:38:44.655 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.G4RmFmekxh 2024-02-02T13:38:44.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-02-02T13:38:44.710 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/data/mon.b.tgz 2024-02-02T13:38:44.710 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-02-02T13:38:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.twtgJWVEvH 2024-02-02T13:38:44.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.twtgJWVEvH 2024-02-02T13:38:44.978 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.twtgJWVEvH 2024-02-02T13:38:45.130 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.twtgJWVEvH is 500KB 2024-02-02T13:38:45.241 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.twtgJWVEvH 2024-02-02T13:38:45.296 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/data/mon.c.tgz 2024-02-02T13:38:45.296 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-02-02T13:38:45.349 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.KhiJOuzdty 2024-02-02T13:38:45.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.KhiJOuzdty 2024-02-02T13:38:45.557 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.KhiJOuzdty 2024-02-02T13:38:45.704 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.KhiJOuzdty is 500KB 2024-02-02T13:38:45.812 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.KhiJOuzdty 2024-02-02T13:38:45.868 INFO:tasks.ceph:Cleaning ceph cluster... 2024-02-02T13:38:45.869 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-02T13:38:45.876 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-02T13:38:45.972 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-02-02T13:38:45.972 DEBUG:teuthology.orchestra.run.smithi091:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-02-02T13:38:45.978 DEBUG:teuthology.orchestra.run.smithi113:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-02-02T13:38:46.066 INFO:teuthology.orchestra.run.smithi091.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-02-02T13:38:46.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-02-02T13:38:46.093 INFO:tasks.ceph:Archiving crash dumps... 2024-02-02T13:38:46.094 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi091/crash 2024-02-02T13:38:46.095 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-02T13:38:46.173 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi113/crash 2024-02-02T13:38:46.174 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-02T13:38:46.267 INFO:tasks.ceph:Shutting down logrotate 2024-02-02T13:38:46.267 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:46.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:46.397 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:46.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-02T13:38:46.595 INFO:tasks.ceph:Compressing logs... 2024-02-02T13:38:46.595 DEBUG:teuthology.orchestra.run.smithi091:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T13:38:46.603 DEBUG:teuthology.orchestra.run.smithi113:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T13:38:52.309 INFO:tasks.ceph:Archiving logs... 2024-02-02T13:38:52.309 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi091/log 2024-02-02T13:38:52.310 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-02T13:38:53.584 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi113/log 2024-02-02T13:38:53.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-02T13:38:54.180 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-02T13:38:54.191 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-02T13:38:54.192 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-02T13:38:54.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-02T13:38:54.352 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-02T13:38:54.362 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-02T13:38:54.362 DEBUG:teuthology.orchestra.run.smithi091:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T13:38:54.370 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T13:38:54.417 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2024-02-02T13:38:54.424 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-02-02T13:38:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T13:38:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2024-02-02T13:38:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +1728us[+1737us] +/- 320ms 2024-02-02T13:38:54.428 INFO:teuthology.orchestra.run.smithi091.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -996us[ -988us] +/- 57ms 2024-02-02T13:38:54.428 INFO:teuthology.orchestra.run.smithi091.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +1259us[+1268us] +/- 69ms 2024-02-02T13:38:54.428 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:^- hv01.front.sepia.ceph.com 2 6 377 35 +1786us[+1802us] +/- 320ms 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -952us[ -935us] +/- 57ms 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +1283us[+1299us] +/- 69ms 2024-02-02T13:38:54.433 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T13:38:54.435 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-02T13:38:54.447 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-02T13:38:54.447 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-02T13:38:54.462 DEBUG:teuthology.orchestra.run.smithi091:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-02T13:38:54.707 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-02T13:38:54.931 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T13:38:55.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:38:55.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T13:38:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:38:55.138 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-02T13:38:55.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-02T13:38:55.168 INFO:teuthology.task.internal:Duration was 1364.231687 seconds 2024-02-02T13:38:55.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-02T13:38:55.182 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-02T13:38:55.183 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-02T13:38:55.190 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-02T13:38:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T13:38:55.354 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T13:38:55.668 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-02T13:38:55.668 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi091.front.sepia.ceph.com 2024-02-02T13:38:55.668 DEBUG:teuthology.orchestra.run.smithi091:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-02T13:38:55.730 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-02-02T13:38:55.731 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-02T13:38:55.806 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-02T13:38:55.806 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T13:38:55.813 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T13:38:55.907 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-02T13:38:55.907 DEBUG:teuthology.orchestra.run.smithi091:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-02T13:38:55.913 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-02T13:38:56.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-02T13:38:56.131 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-02T13:38:56.131 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-02T13:38:56.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-02T13:38:56.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-02T13:38:56.329 DEBUG:teuthology.orchestra.run.smithi091:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-02T13:38:56.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-02T13:38:56.437 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = core 2024-02-02T13:38:56.443 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-02-02T13:38:56.497 DEBUG:teuthology.orchestra.run.smithi091:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-02T13:38:56.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:38:56.551 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-02T13:38:56.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T13:38:56.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-02T13:38:56.618 INFO:teuthology.task.internal:Transferring archived files... 2024-02-02T13:38:56.618 DEBUG:teuthology.misc:Transferring archived files from smithi091:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi091 2024-02-02T13:38:56.619 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-02T13:38:56.725 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405/remote/smithi113 2024-02-02T13:38:56.725 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-02T13:38:56.832 INFO:teuthology.task.internal:Removing archive directory... 2024-02-02T13:38:56.833 DEBUG:teuthology.orchestra.run.smithi091:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-02T13:38:56.841 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-02T13:38:56.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-02T13:38:56.906 INFO:teuthology.task.internal:Not uploading archives. 2024-02-02T13:38:56.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-02T13:38:56.926 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-02T13:38:56.927 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-02T13:38:56.935 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-02T13:38:56.983 INFO:teuthology.orchestra.run.smithi091.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 2 13:38 /home/ubuntu/cephtest 2024-02-02T13:38:56.991 INFO:teuthology.orchestra.run.smithi113.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 2 13:38 /home/ubuntu/cephtest 2024-02-02T13:38:56.994 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-02T13:38:57.006 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-02T13:38:57.039 INFO:teuthology.nuke:Checking targets against current locks 2024-02-02T13:38:57.064 DEBUG:teuthology.nuke:shortname: smithi091 2024-02-02T13:38:57.065 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T13:38:57.088 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 13:06:27.517825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCdICmb/r8PUIMuJy7WjtsllZdEjiLuEeZ/8qghtbmxYAKoEFdGpcR5nZbsIBxVY/ApKVDN249OmUhF5FXx8LA='} 2024-02-02T13:38:57.092 DEBUG:teuthology.nuke:shortname: smithi113 2024-02-02T13:38:57.092 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T13:38:57.116 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 13:06:27.516014', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxTOeL51Zfohp5F+kujWCvAxf0T/oaoI5vrfYOa0yAXSgNzJzhyZ+0KzGVKxl60ZjkC0P6sfN6ruIBB0pVukGw='} 2024-02-02T13:38:57.145 INFO:teuthology.orchestra.console.smithi091:Power off 2024-02-02T13:38:57.145 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-02T13:38:57.171 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off 2024-02-02T13:38:57.171 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:38:57.187 INFO:teuthology.orchestra.console.smithi113:Power off 2024-02-02T13:38:57.188 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-02T13:38:57.198 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2024-02-02T13:38:57.210 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-02-02T13:38:57.210 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:38:57.232 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-02T13:39:01.199 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:01.233 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:01.310 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2024-02-02T13:39:01.342 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-02T13:39:05.312 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:05.343 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:05.426 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2024-02-02T13:39:05.453 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-02T13:39:09.430 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:09.454 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T13:39:09.542 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off 2024-02-02T13:39:09.564 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-02-02T13:39:09.643 INFO:teuthology.orchestra.console.smithi091:Power off completed 2024-02-02T13:39:09.665 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-02-02T13:39:09.771 INFO:teuthology.run:Summary data: client.0-kernel-sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda client.1-kernel-sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1364.231686592102 failure_reason: '"2024-02-02T13:24:24.763038+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 125433.498170 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-02-02T13:39:09.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-02T13:39:09.851 INFO:teuthology.run:FAIL